Opened 12 years ago
Closed 11 years ago
#2173 closed bug (Invalid)
Core freezes under heavy load
Reported by: | underscor | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | Future |
Component: | Core | Version: | master |
Keywords: | Cc: |
Description
Hi guys,
Not quite sure what would be a good way to provide supporting information to help diagnose this, but please let me know what I can run/give you.
Deluge pretty often will enter a state of "not-answering" under heavy load. This usually happens when downloading at ~>15MB/s. It will suddenly stop writing to disk, and stop responding to api probes. Any ideas on what I can do to help figure out why this is occuring?
(We've (archive.org) been using it to fetch large amounts of data via bittorrent, and it's been working awesomely, aside from a few issues handling high-volume downloads as per above. Thanks for such a great project. =) )
Attachments (1)
Change History (21)
Changed 12 years ago by underscor
comment:1 Changed 12 years ago by andar
comment:2 Changed 12 years ago by underscor
# deluged --version deluged: 1.3.900-dev libtorrent: 0.15.10.0
comment:3 Changed 12 years ago by andar
You could try using the latest version of libtorrent (0.16.4) and see if that helps your problem. There were a lot of improvements done to 0.16.x.
comment:4 Changed 12 years ago by underscor
Rebuilding. Will let you know what happens :)
comment:5 Changed 12 years ago by underscor
Changed: VERSION=15
to VERSION=16
in get_libtorrent.sh, and rebuilt/reinstalled. However, now I get:
19:47:12.500 [ERROR ][deluge.main :1181] dynamic module does not define init function (initlibtorrent) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/deluge-1.3.900_dev-py2.7-linux-x86_64.egg/deluge/main.py", line 261, in start_daemon Daemon(options, args) File "/usr/local/lib/python2.7/dist-packages/deluge-1.3.900_dev-py2.7-linux-x86_64.egg/deluge/core/daemon.py", line 130, in __init__ from deluge.core.core import Core File "/usr/local/lib/python2.7/dist-packages/deluge-1.3.900_dev-py2.7-linux-x86_64.egg/deluge/core/core.py", line 37, in <module> from deluge._libtorrent import lt File "/usr/local/lib/python2.7/dist-packages/deluge-1.3.900_dev-py2.7-linux-x86_64.egg/deluge/_libtorrent.py", line 59, in <module> import libtorrent as lt ImportError: dynamic module does not define init function (initlibtorrent)
Any ideas?
comment:6 Changed 12 years ago by underscor
Oddly, I can import libtorrent fine in a python shell using the same interpreter that deluge is being launched by.
>>> import libtorrent as lt >>>
comment:7 Changed 12 years ago by andar
Does your OS not have a libtorrent package that you could simply use?
comment:8 Changed 12 years ago by underscor
Yes, it does, but not of 0.16.4, which you suggested may improve the problem.
comment:9 Changed 12 years ago by damoxc
Are you running a 64bit copy?
comment:10 Changed 12 years ago by damoxc
Actually I see you are from that stack trace, I'm just building an updated package you can (hopefully) try out.
comment:11 Changed 12 years ago by underscor
Awesome, thanks! =)
comment:12 Changed 12 years ago by damoxc
You can grab all of them from http://damoxc.net/libtorrent-packages.tar.gz. They seem to be working for me so hopefully will for you too!
comment:13 Changed 12 years ago by underscor
You're amazing <3
0 8:56PM:abuie@bt-download02:/1/libtorrent 54211 Ï python Python 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import libtorrent >>> print libtorrent.version 0.16.4.0 >>>
0 8:58PM:abuie@bt-download02:/1/build/deluge 54224 Ï deluged --version deluged: 1.3.900-dev libtorrent: 0.16.4.0
Stresstesting this build now, will let you know soon if the "freezing" issue I've been seeing continues to occur.
comment:14 Changed 12 years ago by underscor
Hmm, this is a new one. I can open a separate bug report if you'd like.
deluge-console 'add -p "/1/downloads/9c8534b29cce486a571aa76f8225c35ee2e29d6d_wcd_ketsui-extra--dodon_manabu-namiki-_flac_lossless_1644875" "http://bt-download04/torrents.php?action=download&id=1644875&authkey=c28aba95c183165ef8ed2e8ae6a3bbbd&torrent_pass=4np8z7qkzb6dbv8pzdbqoqznvm7avir1"' /1/"http:/bt-download04/torrents.php?action=download&id=1644875&authkey=c28aba95c183165ef8ed2e8ae6a3bbbd&torrent_pass=4np8z7qkzb6dbv8pzdbqoqznvm7avir1" doesn't exist!
comment:15 Changed 12 years ago by damoxc
If you wouldn't mind! Looks like the URL is being mangled at the beginning from a first glance. How's the core freezing? Or are you unable to test due to this issue?
comment:16 Changed 12 years ago by underscor
Ticket #2174 created. I am currently unable to test due to this issue, however.
comment:17 Changed 12 years ago by damoxc
Thanks! Out of interest is there any particular reason you're running git master?
comment:18 Changed 12 years ago by underscor
Mostly for deluge-console status, but it also seems to perform better than 1.3.5 did under the same load. (1.3.5 would freeze/crash/misbehave when dumping in large numbers of torrents much more often than master does)
comment:19 Changed 12 years ago by underscor
Wow, that totally fixed it. Went from pushing <15MBps and crashy to a week up pulling 40-50MBps. :D
However, it is dropping announces now. This seems to be a hardcoded variable in libtorrent:
libtorrent/src/session.cpp:266 active_tracker_limit = 2000 libtorrent/src/session.cpp:1170 active_tracker_limit(1600) // don't announce to trackers more than once every 1.125 seconds
Looks like I'll need to build my own version to bump that up for this project. Since I'm using dpkg, (and currently using your builds), I suppose I'll need to learn debian packaging. You don't happen to have a preferred tutorial for that, do you? :)
Thanks a lot for all your help! <3
comment:20 Changed 11 years ago by Cas
- Resolution set to invalid
- Status changed from new to closed
What version of libtorrent?