Opened 5 years ago

Last modified 5 weeks ago

#3252 new bug

Replace deprecated pkg_resources

Reported by: Cas Owned by:
Priority: minor Milestone: 2.x
Component: Core Version: develop
Keywords: Cc:


Going forward in Python 3 the pkg_resources api is rather slow and has been marked deprecated.

To find and load resources using pkgutil or new importlib.resources is the solution. There are however a few issues that don't make them direct drop in replacements.

Firstly to get the resource filename pkgutil can only get_data, file contents, so to get the file path, a custom method will need to be implemented like this:

def get_filename(package, resource):
    """Rewrite of pkgutil.get_data() that return the file path.

    loader = get_loader(package)
    if loader is None or not hasattr(loader, 'get_data'):
        return None
    mod = sys.modules.get(package) or loader.load_module(package)
    if mod is None or not hasattr(mod, '__file__'):
        return None

    # Modify the resource name to be compatible with the loader.get_data
    # signature - an os.path format "filename" starting with the dirname of
    # the package's __file__
    parts = resource.split('/')
    parts.insert(0, os.path.dirname(mod.__file__))
    resource_name = os.path.normpath(os.path.join(*parts))

    return resource_name

And for importlib.resources there is an equivalent on the migration guide.

The only issue is that neither work perfectly with zipped eggs, like pkg_resources, which uses a python egg cache. So will need to workaround this issue by trying not to pass around resource filenames in the plugins.

I have checked the code of the plugins and most GTK methods will accept the resource data but the WebUI requires a filepath for scripts so somehow need to pass either the data or package details for the web server to load the data without needing the absolute path.

A few notes about importlib.resources, it is only built-in to Python 3.7 so harder to package and it requires all data directories to be packages which requires adding files to these locations.

Change History (1)

comment:1 Changed 5 weeks ago by Lord-Kamina

I would like to try tackling this. I've already begun the work, but I believe I'm getting burned by the current implementation of deluge.common.resource_filename

Could you please give me a primer of what methods, if any, need to absolutely get an absolute path, as well as where in the WebUI is the issue?

Note: See TracTickets for help on using tickets.