Opened 10 years ago

Closed 10 years ago

#2173 closed bug (Invalid)

Core freezes under heavy load

Reported by: underscor Owned by:
Priority: major Milestone: Future
Component: Core Version: master
Keywords: Cc:

Description

Hi guys,

Not quite sure what would be a good way to provide supporting information to help diagnose this, but please let me know what I can run/give you.

Deluge pretty often will enter a state of "not-answering" under heavy load. This usually happens when downloading at ~>15MB/s. It will suddenly stop writing to disk, and stop responding to api probes. Any ideas on what I can do to help figure out why this is occuring?

(We've (archive.org) been using it to fetch large amounts of data via bittorrent, and it's been working awesomely, aside from a few issues handling high-volume downloads as per above. Thanks for such a great project. =) )

Attachments (1)

core.conf (2.4 KB) - added by underscor 10 years ago.

Download all attachments as: .zip

Change History (21)

Changed 10 years ago by underscor

comment:1 Changed 10 years ago by andar

What version of libtorrent?

comment:2 Changed 10 years ago by underscor

# deluged --version
deluged: 1.3.900-dev
libtorrent: 0.15.10.0

comment:3 Changed 10 years ago by andar

You could try using the latest version of libtorrent (0.16.4) and see if that helps your problem. There were a lot of improvements done to 0.16.x.

comment:4 Changed 10 years ago by underscor

Rebuilding. Will let you know what happens :)

comment:5 Changed 10 years ago by underscor

Changed: VERSION=15

to VERSION=16

in get_libtorrent.sh, and rebuilt/reinstalled. However, now I get:

19:47:12.500 [ERROR   ][deluge.main            :1181] dynamic module does not define init function (initlibtorrent)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/deluge-1.3.900_dev-py2.7-linux-x86_64.egg/deluge/main.py", line 261, in start_daemon
    Daemon(options, args)
  File "/usr/local/lib/python2.7/dist-packages/deluge-1.3.900_dev-py2.7-linux-x86_64.egg/deluge/core/daemon.py", line 130, in __init__
    from deluge.core.core import Core
  File "/usr/local/lib/python2.7/dist-packages/deluge-1.3.900_dev-py2.7-linux-x86_64.egg/deluge/core/core.py", line 37, in <module>
    from deluge._libtorrent import lt
  File "/usr/local/lib/python2.7/dist-packages/deluge-1.3.900_dev-py2.7-linux-x86_64.egg/deluge/_libtorrent.py", line 59, in <module>
    import libtorrent as lt
ImportError: dynamic module does not define init function (initlibtorrent)

Any ideas?

comment:6 Changed 10 years ago by underscor

Oddly, I can import libtorrent fine in a python shell using the same interpreter that deluge is being launched by.

>>> import libtorrent as lt
>>>

comment:7 Changed 10 years ago by andar

Does your OS not have a libtorrent package that you could simply use?

comment:8 Changed 10 years ago by underscor

Yes, it does, but not of 0.16.4, which you suggested may improve the problem.

http://packages.ubuntu.com/precise/python-libtorrent

comment:9 Changed 10 years ago by damoxc

Are you running a 64bit copy?

comment:10 Changed 10 years ago by damoxc

Actually I see you are from that stack trace, I'm just building an updated package you can (hopefully) try out.

comment:11 Changed 10 years ago by underscor

Awesome, thanks! =)

comment:12 Changed 10 years ago by damoxc

You can grab all of them from http://damoxc.net/libtorrent-packages.tar.gz. They seem to be working for me so hopefully will for you too!

comment:13 Changed 10 years ago by underscor

You're amazing <3

0  8:56PM:abuie@bt-download02:/1/libtorrent 54211 Ï python
Python 2.7.3 (default, Aug  1 2012, 05:14:39)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import libtorrent
>>> print libtorrent.version
0.16.4.0
>>>
0  8:58PM:abuie@bt-download02:/1/build/deluge 54224 Ï deluged --version
deluged: 1.3.900-dev
libtorrent: 0.16.4.0

Stresstesting this build now, will let you know soon if the "freezing" issue I've been seeing continues to occur.

comment:14 Changed 10 years ago by underscor

Hmm, this is a new one. I can open a separate bug report if you'd like.

deluge-console 'add -p "/1/downloads/9c8534b29cce486a571aa76f8225c35ee2e29d6d_wcd_ketsui-extra--dodon_manabu-namiki-_flac_lossless_1644875" "http://bt-download04/torrents.php?action=download&id=1644875&authkey=c28aba95c183165ef8ed2e8ae6a3bbbd&torrent_pass=4np8z7qkzb6dbv8pzdbqoqznvm7avir1"'
/1/"http:/bt-download04/torrents.php?action=download&id=1644875&authkey=c28aba95c183165ef8ed2e8ae6a3bbbd&torrent_pass=4np8z7qkzb6dbv8pzdbqoqznvm7avir1" doesn't exist!

comment:15 Changed 10 years ago by damoxc

If you wouldn't mind! Looks like the URL is being mangled at the beginning from a first glance. How's the core freezing? Or are you unable to test due to this issue?

comment:16 Changed 10 years ago by underscor

Ticket #2174 created. I am currently unable to test due to this issue, however.

comment:17 Changed 10 years ago by damoxc

Thanks! Out of interest is there any particular reason you're running git master?

comment:18 Changed 10 years ago by underscor

Mostly for deluge-console status, but it also seems to perform better than 1.3.5 did under the same load. (1.3.5 would freeze/crash/misbehave when dumping in large numbers of torrents much more often than master does)

comment:19 Changed 10 years ago by underscor

Wow, that totally fixed it. Went from pushing <15MBps and crashy to a week up pulling 40-50MBps. :D

However, it is dropping announces now. This seems to be a hardcoded variable in libtorrent:

libtorrent/src/session.cpp:266 active_tracker_limit = 2000

libtorrent/src/session.cpp:1170 active_tracker_limit(1600) // don't announce to trackers more than once every 1.125 seconds

Looks like I'll need to build my own version to bump that up for this project. Since I'm using dpkg, (and currently using your builds), I suppose I'll need to learn debian packaging. You don't happen to have a preferred tutorial for that, do you? :)

Thanks a lot for all your help! <3

comment:20 Changed 10 years ago by Cas

  • Resolution set to invalid
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.