RuntimeError when emiting event

[Bro]

I've seen this traceback a couple of times, last time months ago, so this does not happen often.

I presume this issue applies to master as well.

The call in YaRSS2 that causes this is component.get("EventManager").emit(...)

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/threading.py", line 524, in __bootstrap
    self.__bootstrap_inner()
  File "/usr/local/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/site-packages/twisted/python/threadpool.py", line 167, in _worker
    result = context.call(ctx, function, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "build/bdist.linux-x86_64/egg/yarss2/rssfeed_scheduler.py", line 135, in rssfeed_update_handler

  File "/home/bro/programmer/deluge/deluge/deluge/core/eventmanager.py", line 51, in emit
    component.get("RPCServer").emit_event(event)
  File "/home/bro/programmer/deluge/deluge/deluge/core/rpcserver.py", line 450, in emit_event
    for session_id, interest in self.factory.interested_events.iteritems():
exceptions.RuntimeError: dictionary changed size during iteration

[andar]

Maybe a copy of interested_events dict should be used to iterate through as interested_events could be updated periodically and especially when a client connects.

[Chase]

Simplest answer would just be to switch to items() from iteritems(), which would be just want andar said, iterating through a copy.

[andar]

Will that work for Python 3? Wasn't the behaviour of items() changed to that of iteritems()?

[Chase]

Yeah, items() returns a dictionary "view" in python 3, which reflects changes in the dictionary. Not sure if changing the dictionary during iteration causes an exception in that case, I'll check it out.

[Chase]

Yeah, turns out views act pretty much the same as iteritems() in python 2, same exception will occur if changed during iteration. I guess we could just listify it for iteration.

EDIT: Or just use items() for now, it'll have to be changed one way or the other when we want python 3 support.

[andar]

I'd suggest any new code we write should be python 3 aware as it will make porting in the future a bit easier ;)

[Chase]

Sounds good andar. In that case I'd suggest just using items() on 1.3-stable, (I don't think we want to update that to py3,) and adding six as a dependency on master, where we can do list(six.iteritems(self.factory.interested_events)) which will do the same thing on both python 2 and 3.

EDIT: Without six we can do just list(self.factory.interested_events.items()) which will work on both, but makes an extra list on python 2. I think having six is worth it if we want to start writing code that works on both python 2 and 3.

[Calum]

So we apply gazpachoking's list(self.factory.interested_events.items()) suggestion to 1.3 and consider using six in git master?

[Chase]

I'd skip the list() in 1.3, as dict.items() already returns a list on python < 3. I don't imagine we will ever want to port 1.3 to python 3.

[andar]

Are you sure we need six? Can't we just change it to iterate over the .keys()? I suspect the only thing that changes is session_id's get added and removed as clients connect/disconnect.

That being said, if we are serious about porting to python3, maybe the six module isn't a bad dependency.

[Chase]

We certainly don't need six, and if it was only this one spot I'd say we shouldn't use it. But it has a lot of useful functions I suspect we will want to use other places if we want the code to be python 2 and 3 compatible. Using .keys() will have the same problem we have now on python 3, the dict can't change while you are iterating over that. The six iter*() functions just give a way to consistently get an iterator on both python 2 and 3, which we would then turn into a list before iterating over.

[andar]

I guess what I meant to say was copy .keys(), ie, list(self.factory.interested_events.keys()).

[Chase]

Yeah, that will work, it'll create an extra list in python 2 though, as .keys() will already create a list.

[Calum]

Fixed in 1.3-stable: ​b0d78da13b5

When we get around to moving to py3 it's items() is easy to spot and be fixed along with others.