Python Redis Subscribe Can Not Get All Datas?
I am using python to get datas from redis and then parser it to kafka. It works well in most situations. But when I use python to simulate build the data to redis, or there was a f
Solution 1:
I assume you are using redis-py.
The documentation refers to listen
as older version of the lib... Maybe you should use another method for message reading. For example with a callback
p = r.pubsub()
defmy_handler(message):
print'MY HANDLER: ', message['data']
if item['type'] == 'message':
num += 1
a.parser(item['data'])
print num
p.subscribe('bdwaf', my_handler)
# read the subscribe confirmation message
p.get_message()
Edit:
It is possible that your redis server is running out of memory when you publish 20000 messages at once. Try increasing redis memory in redis.conf
file
maxmemory 500mb # or greater if needed
It is a memory problem, check out this question for more information on how to handle it.
Post a Comment for "Python Redis Subscribe Can Not Get All Datas?"