Speed optimizations to the daemon

[bro]

I've been profiling the daemon and I've created a patch that decreases the CPU usage of the daemon while clients are connected.

torrent.get_status

This method is called a lot of times! To speed this up I've restructured and split the code so that most of the calls that previously were to get_status now calls create_status_dict which simply creates a dict based on the cached info instead of asking libtorrent for a new status.

The internal dictionary used to create the dictionary with the status values is now created only once each for torrent instead of every time get_status is called.

The inner methods of get_status are moved out to avoid creating them for each call.

There are also changes to speed up frequent tests like has_metadata, and as few calls to libtorrent as possible.

core.get_torrents_status / libtorrent.post_torrent_updates

libtorrent.post_torrent_updates tells libtorrent to post an alert with the torrents that have changed since the last call to post_torrent_updates.

This is called from core.get_torrents_status when a client requests an updated status dict. If the statuses for the torrents were updated less than 2 seconds ago (the interval the GTKUI asks for updates), it will return a status dict created from the cached data instead of calling post_torrent_updates. This should make the daemon perform better with multiple clients connected.

Startup (loading torrents)

To speed this up I've added a variable wait_on_handler in alertmanager which is set just when the torrents are loaded. alertmanager will then call the handlers without waiting. This speeds up adding the torrents considerably (~40%).

Log statements (log.isEnabledFor)

Many of the log statements require "heavy" computations, so when summing together, the total time for the log statements is many seconds (roughly 5-10s on my desktop) when loading many torrents (I've tested with 2000).

Therefore I've added log.isEnabledFor on the most important log statements. When profiling startup of 2000 torrents, the log statements I've added log.isEnabledFor for used from 150ms to 600ms each (in total).

Redundant and unecessary RPC messages

I've removed some redundant RPC requests, and combined some to have less packets transmitted on startup.

• statusbar.py and systemtray.py:
  • Combined the requests for some config values into one RPC.
  • Removed calls to send_status_request on startup. These only result in two requests being sent almost at the same time.

• sessionproxy.py: Removed the call to self.get_torrents_status on startup. A few milliseconds after this request the torrentview requests the same data, resulting in the torrent list being transmitted twice when the GTK Client connects.

[bro]

​https://github.com/bendikro/deluge/commit/5cd86aa5bcbd246f4ed6b4bed026cd72045304a3

This patch is not "commitable" as it contains some comments here and there. I'd appreciate some feedback on the changes and maybe if there are other things to do, or if some things could be done differently.

[bro]

Changes based on feedback in comments: ​https://github.com/bendikro/deluge/commit/0f67dc168b03b2bed661e8a75b67c033070072a0

[gazpachoking]

Merged in ​8c106ce8c

[Cas]

Milestone renamed