Opened 5 years ago

#3252 new bug

Replace deprecated pkg_resources

Reported by: Cas Owned by:
Priority: minor Milestone: 2.x
Component: Core Version: develop
Keywords: Cc:

Description

Going forward in Python 3 the pkg_resources api is rather slow and has been marked deprecated.

To find and load resources using pkgutil or new importlib.resources is the solution. There are however a few issues that don't make them direct drop in replacements.

Firstly to get the resource filename pkgutil can only get_data, file contents, so to get the file path, a custom method will need to be implemented like this:

def get_filename(package, resource):
    """Rewrite of pkgutil.get_data() that return the file path.

    From: https://github.com/DLR-RY/pando-core/blob/master/pando/pkg.py
    """
    loader = get_loader(package)
    if loader is None or not hasattr(loader, 'get_data'):
        return None
    mod = sys.modules.get(package) or loader.load_module(package)
    if mod is None or not hasattr(mod, '__file__'):
        return None

    # Modify the resource name to be compatible with the loader.get_data
    # signature - an os.path format "filename" starting with the dirname of
    # the package's __file__
    parts = resource.split('/')
    parts.insert(0, os.path.dirname(mod.__file__))
    resource_name = os.path.normpath(os.path.join(*parts))

    return resource_name

And for importlib.resources there is an equivalent on the migration guide.

The only issue is that neither work perfectly with zipped eggs, like pkg_resources, which uses a python egg cache. So will need to workaround this issue by trying not to pass around resource filenames in the plugins.

I have checked the code of the plugins and most GTK methods will accept the resource data but the WebUI requires a filepath for scripts so somehow need to pass either the data or package details for the web server to load the data without needing the absolute path.

A few notes about importlib.resources, it is only built-in to Python 3.7 so harder to package and it requires all data directories to be packages which requires adding init.py files to these locations.

Change History (0)

Note: See TracTickets for help on using tickets.