Alternative To A While Loop In Twisted Which Doesn't Block The Reactor Thread

I'm making a chat application in twisted. Suppose my server is designed in such a way that whenever it detects a client online, it sends the client all the pending-messages (those

Solution 1:

I think that https://glyph.twistedmatrix.com/2011/11/blocking-vs-running.html addresses this point.

The answer here depends on what exactly self.retrieveFromQueue(self) does. You implied it's something like:

if self.list_of_messages:
    return self.list_of_messages.pop(0)
return b"empty"

If this is the case, then the answer is one thing. On the other hand, if the implementation is something more like:

returnself.remote_mq_client.retrieve_queue_item(self.queue_identifier)

then the answer might be something else entirely. However, note that it's the implementation of retrieveFromQueue upon which the answer appears to hinge.

That there is a while loop isn't quite as important. The while loop reflects the fact that (to use Glyph's words), this code is getting work done.

You may decide that the amount of work this loop represents is too great to all get done at one time. If there are hundreds of millions of queued messages then copying them one by one into the connection's send buffer will probably use both a noticable amount of time and memory. In this case, you may wish to consider the producer/consumer pattern and its support in Twisted. This won't make the code any less (or more) "blocking" but it will make it run for shorter periods of time at a time.

So the questions to answer here are really:

• whether or not retrieveFromQueue blocks
• if it does not block, whether or not there will be so many queued messages that processing them all will cause connectionMade to run for so long that other clients notice a disruption in service