Opened 5 years ago
Last modified 4 months ago
#3252 new bug
Replace deprecated pkg_resources
Reported by: | Calum | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | 2.x |
Component: | Core | Version: | develop |
Keywords: | Cc: |
Description
Going forward in Python 3 the pkg_resources api is rather slow and has been marked deprecated.
To find and load resources using pkgutil or new importlib.resources is the solution. There are however a few issues that don't make them direct drop in replacements.
Firstly to get the resource filename pkgutil can only get_data, file contents, so to get the file path, a custom method will need to be implemented like this:
def get_filename(package, resource): """Rewrite of pkgutil.get_data() that return the file path. From: https://github.com/DLR-RY/pando-core/blob/master/pando/pkg.py """ loader = get_loader(package) if loader is None or not hasattr(loader, 'get_data'): return None mod = sys.modules.get(package) or loader.load_module(package) if mod is None or not hasattr(mod, '__file__'): return None # Modify the resource name to be compatible with the loader.get_data # signature - an os.path format "filename" starting with the dirname of # the package's __file__ parts = resource.split('/') parts.insert(0, os.path.dirname(mod.__file__)) resource_name = os.path.normpath(os.path.join(*parts)) return resource_name
And for importlib.resources there is an equivalent on the migration guide.
The only issue is that neither work perfectly with zipped eggs, like pkg_resources, which uses a python egg cache. So will need to workaround this issue by trying not to pass around resource filenames in the plugins.
I have checked the code of the plugins and most GTK methods will accept the resource data but the WebUI requires a filepath for scripts so somehow need to pass either the data or package details for the web server to load the data without needing the absolute path.
A few notes about importlib.resources, it is only built-in to Python 3.7 so harder to package and it requires all data directories to be packages which requires adding init.py files to these locations.
I would like to try tackling this. I've already begun the work, but I believe I'm getting burned by the current implementation of
deluge.common.resource_filename
Could you please give me a primer of what methods, if any, need to absolutely get an absolute path, as well as where in the WebUI is the issue?