#1076 closed bug (Fixed)
[webui] 1.2.x - impossible to reverse proxy with apache
Reported by: | orbisvicis | Owned by: | Damien Churchill |
---|---|---|---|
Priority: | critical | Milestone: | 1.3 |
Component: | Web UI | Version: | 1.2.0_dev |
Keywords: | webui, regression | Cc: |
Description
The 1.1.x webui included a command-switch to dynamically change the webroot - that is, prefix all urls and redirects with a base url (did it work for all headers/cookies, or only redirects?). See http://forum.deluge-torrent.info/viewtopic.php?f=7&t=10165&st=0&sk=t&sd=a#p53605
1.2.x does not offer this, and because 1.2.x uses ajax/xmlhttprequest it is not possible to reverse proxy deluge-web behind apache, for several reasons:
1) Javascript does not preserve the REFERER Header. Apache mod_rewrite rules such as the following will no longer work. RewriteCond %{HTTP_REFERER} protocol://some.website.uri:port/deluge/.* RewriteCond %{REQUEST_URI} ^/deluge/.* RewriteRule /(.*) /deluge/$1 [PT]
(notice that without tracking the Referer, Rewrite rules most likely would conflict with other webapps)
2) Apache substitute filters are somewhat broken. The following will not work (to rewrite the urls in the xml[ajax],css,javascript,and html)
FilterProvider text_inflate inflate resp=Content-Type $text FilterChain +text_inflate
FilterProvider html_sed sed resp=Content-Type $html FilterChain +html_sed OutputSed s#url(["']\(["']\{1,\}\)["'])#url(/deluge\1)#
FilterProvider js_sed sed resp=Content-Type $javascript FilterChain +js_sed OutputSed s#'\(\(/[a-zA-Z]\{1,\}\)\{1,\}/[a-zA-Z]\{1,\}\.[a-zA-Z]\{2,3\}\)'#'/deluge\1'#
FilterProvider xml_sed sed resp=Content-Type $xml FilterChain +xml_sed OutputSed s#\(\(/[a-zA-Z]\{1,\}\)\{1,\}/[a-zA-Z]\{1,\}\.[a-zA-Z]\{2,3\}\)#/deluge\1#
FilterProvider css_sed sed resp=Content-Type $css FilterChain +css_sed OutputSed s#url(\([)]\{1,\}\))#url(/deluge\1)#
FilterProvider text_deflate deflate resp=Content-Type $text FilterChain +text_deflate
(notice in this case the limitation with mod_proxy_html: it does not parse inline css. That's why it is also necessary to rewrite the html.)
This is why I'm asking for a 1.2.x re-implementation of this very critical 1.1.x feature.
Change History (24)
comment:1 by , 15 years ago
comment:2 by , 15 years ago
Milestone: | 1.2.0 → 1.3.0 |
---|
comment:3 by , 15 years ago
What is needed to accomplish this?
web.Web.__init__
::add option to specifiy prefixweb.Web.__init__
::pass prefix- ? -> what is web.WebUI for?
server.TopLevel.__init__
::modify putChild() calls- unhardcode all url links in templates
- maybe move static content (js,etc?) to template
- ? -> where/how are template variables such as ${stylesheet} defined?
btw 1.3.0 seems far in the future..
comment:4 by , 15 years ago
It's going to require quite a few changes, barely any of the html is template generated, it is all created via javascript, so will need to ensure that all paths in that are honouring the prefix. And if you look at the roadmap you will see that 1.3 is only 2 months away. 1.2 was the big change and after this we are having smaller release cycles.
comment:5 by , 15 years ago
You'll be pleased to hear that this is now mostly complete for 1.3. There are probably still some minor issues to iron out but it is working fine for the most part.
comment:6 by , 15 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
From my limited testing this is now possible, so marking as fixed.
comment:7 by , 15 years ago
Sweet, thanks. I've just checked out the trunk from svn, but I can't figure out how to use it. I don't see any webroot options in web/web.py.
Once I get it working, I'll take it for a test-drive, maybe find new ways to improve the web-apache interface.
comment:8 by , 15 years ago
We've switched over to using git now, take a look at GitRepo, and then there are tarballs available over on http://git.deluge-torrent.org.
There's no config param, it's just an entry in the web.conf. I was thinking of perhaps adding it as a switch to deluge-web, would this be useful?
comment:9 by , 15 years ago
I had kinda hoped to do something clever to auto-detect if the UI was being requested via a proxy, but there was no easy way I could see to get the original url, do you know if that's possible as it would be rather nice if I could do that?
comment:10 by , 15 years ago
Yeah, the switch would be helpful, so I can start the deluge web interface as usual, from /etc/{init.d,rc.d}/ scripts with the correct webroot: "deluge-web --prefix="/deluge" ... etc.
As for the clever auto-detect thing, I think the concrete command-line switch should always take precedence, but would this do the trick? ProxyPass /deluge/ http://127.0.0.1:8112/?webroot="/deluge" After which the deluge-web appends this string to all links. However, I'm not sure if the query string is preserved in headers (http-redirects etc)
comment:11 by , 15 years ago
Nah unfortunately it's not. I was hoping you could do something like:
SetEnv X-Original-Path "...."
but unfortunately that requires a patch on Apache so isn't really ideal.
Query strings also aren't preserved, I gave that a try. Or not at least with the basic ProxyPass.
I'll let you know when I've added the command line switch.
comment:12 by , 15 years ago
no, but mod_headers can manually set headers, for example:
RequestHeader set X-Deluge-Webroot "...."
In a <Location> block the header should be set on all requests, even redirects.
Assuming the URLs are initiated on startup would changing them affect the global state of the application?
- Say an initial rogue request header set "X-Deluge-Webroot" to "/impossible/to/guess/path"
- Then a request header without "X-Deluge-Webroot" was received
Would clients still be able to access content at http://127.0.0.1:8112/original_path/ (or would the webroot default to the last "X-Deluge-Webroot" received?)
comment:13 by , 15 years ago
No no it would be on a per-request basis, I've changed all the resources to be relative paths so the base only needs to be set on the html page.
I've added the -b/--base switch to deluge-web now so you are able to set it there as well as by the config file.
I'll see if mod_headers will work!
comment:14 by , 15 years ago
Yeah it's possible to do via mod_headers, latest git you can now set the base via X-Deluge-Base and it dynamically switches it on that.
comment:15 by , 15 years ago
Hehe that header parameter is incredibly cool.
Two suggestions:
- Validate the base URI so that both pairs of options work:
- -b "/deluge"
RequestHeader set X-Deluge-Base "/deluge"
- -b "/deluge/"
RequestHeader set X-Deluge-Base "/deluge/"
- Add a switch to deluge-web toggling the header parameter availability, perhaps "--enable-header-base" or similar.
comment:16 by , 15 years ago
I don't know how near complete 1.3 is, but this is logged when I click on the "Add" button:
[error] proxy: Error reading from remote server returned by /deluge/json, referer: https://my.website.net:myport/deluge/ [error] client denied by server configuration: proxy:http://127.0.0.1:8112/json, referer: https://my.website.net:myport/deluge/ [error] (70007)The timeout specified has expired: proxy: error reading status line from remote server 127.0.0.1, referer: https://my.website.net:myport/deluge/
with accompanying:
"POST /deluge/json HTTP/1.1" 502 492 "https://my.website.net:myport/deluge/" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2) Gecko/20100207 Namoroka/3.6"
comment:17 by , 15 years ago
Yeah good idea, should probably validate it. I'm thinking check for preceding / and add it if that's missing, and check for a tailing slash and add it if that's missing.
As for that error, I know that one ajax request will quite often timeout by design, I swapped the events checking from polling to a more "push"-ey style system where it sends the request and it stays connected until the server receives an event or the request times out.
So is there anyway you can check which request it is timing out??
comment:18 by , 15 years ago
Good call.
I'd like to check, but I can no longer get a connection between deluged and deluge-web. I'm accessing the web-interface via ssh port-tunnelling - no apache proxying. Not sure what to do but I tried deleting ~/.config/deluge/
from deluged
... [INFO ] 14:57:47 rpcserver:219 Deluge client disconnected: Connection was closed cleanly. [INFO ] 14:57:49 rpcserver:199 Deluge Client connection made from: 127.0.0.1:38721 [INFO ] 14:57:49 rpcserver:219 Deluge client disconnected: Connection was closed cleanly. [INFO ] 14:57:51 rpcserver:199 Deluge Client connection made from: 127.0.0.1:38723 ...
from deluge-web
... [INFO ] 14:57:51 client:224 Connection lost to daemon at 127.0.0.1:58846 reason: Connection was closed cleanly. [INFO ] 14:57:53 client:217 Connecting to daemon at 127.0.0.1:58846.. [INFO ] 14:57:53 client:121 Connected to daemon at 127.0.0.1:58846.. [INFO ] 14:57:53 client:224 Connection lost to daemon at 127.0.0.1:58846 reason: Connection was closed cleanly. ...
When/if I get this working i'll see if the request that times out is logged by deluge-web. If not I can sniff it out with wireshark.
comment:20 by , 15 years ago
Double nevermind. Seems whenever I start the components as "deluged -d -L info" -> "deluge-web -L info" they don't communicate. I'm not sure why.
However, when started normally, I found that the request that times out (nonfunctional "Add" button) has nothing to do with the apache reverse proxy - it also times out @ 127.0.0.1:8112.
This is what I could see from wireshark:
I'm pretty sure this is the request that times out. It's frequency in the logs matches the frequency of me clicking the "Add" button. Doesn't seem to have anything to do with "adding" torrents though.
{"method":"web.get_host_status","params":["0f76211d03b29f3990b9f6a54eb0d655a1c6e231"],"id":87}
This seems to be the update loop:
{"method":"web.update_ui","params":[["queue","name","total_size","state","progress","num_seeds","total_seeds","num_peers","total_peers","download_payload_rate","upload_payload_rate","eta","ratio","distributed_copies","is_auto_managed","time_added","tracker_host"],{}],"id":84}
These methods only occured at the beginning of the conversation, probably auth stuff
{"method":"web.connect","params":["0f76211d03b29f3990b9f6a54eb0d655a1c6e231"],"id":20}
{"method":"auth.check_session","params":[],"id":1}
Another thing, don't know if this is interesting, but this is an error generated when restarting the web server while a client is connected:
[ERROR ] 15:32:40 json_api:212 Error calling method `web.get_host_status` [ERROR ] 15:32:40 json_api:213 'NoneType' object is not iterable Traceback (most recent call last): File "/opt/python-virtual-deluge/lib/python2.5/site-packages/deluge-1.2.900_dev-py2.5.egg/deluge/ui/web/json_api.py", line 204, in _handle_request result = self._exec_local(method, params, request) File "/opt/python-virtual-deluge/lib/python2.5/site-packages/deluge-1.2.900_dev-py2.5.egg/deluge/ui/web/json_api.py", line 171, in _exec_local return meth(*params) File "/opt/python-virtual-deluge/lib/python2.5/site-packages/deluge-1.2.900_dev-py2.5.egg/deluge/ui/web/json_api.py", line 693, in get_host_status (host_id, host, port, user, password) = self.get_host(host_id) TypeError: 'NoneType' object is not iterable
comment:21 by , 15 years ago
I can get you the TwistedServer responses but I think they're all compressed and stuff.
comment:22 by , 15 years ago
Found some misplaced resources
File does not exist: /usr/share/apache2/icons/add.png, referer: https://aequorin.homeunix.net:62389/deluge/ File does not exist: /usr/share/apache2/icons/error.png, referer: https://my.website.net:myport/deluge/ File does not exist: /usr/share/apache2/icons/remove.png, referer: https://my.website.net:myport/deluge/
comment:23 by , 15 years ago
Fixed all the remaining static resources now!
I'll check out the request timeout issue also.
comment:24 by , 15 years ago
Upgraded to the current dev trunk - 1.3.900 - and everything is working. Thanks!
The 1.1.x webui included a command-switch to dynamically change the webroot - that is, prefix all urls and redirects with a base url (did it work for all headers/cookies, or only redirects?). See http://forum.deluge-torrent.info/viewtopic.php?f=7&t=10165&st=0&sk=t&sd=a#p53605
1.2.x does not offer this, and because 1.2.x uses ajax/xmlhttprequest it is not possible to reverse proxy deluge-web behind apache, for several reasons:
This is why I'm asking for a 1.2.x re-implementation of this very critical 1.1.x feature.