Python Asyncio Task Got Bad Yield

I am confused about how to play around with the asyncio module in Python 3.4. I have a searching API for a search engine, and want to each search request to be run either parallel,

Solution 1:

The problem is that you can't just call existing synchronous code as if it was an asyncio.coroutine and get asynchronous behavior. When you call yield from searching(...), you're only going to get asynchronous behavior if searching itself is actually an asyncio.coroutine, or at least returns an asyncio.Future. Right now, searching is just a regular synchronous function, so calling yield from searching(...) is just going to throw an error, because it doesn't return a Future or coroutine.

To get the behavior you want, you'll need to have an asynchronous version of searching in addition to a synchronous version (or just drop the synchronous version altogether if you don't need it). You have a few options to support both:

1. Rewrite searching as an asyncio.coroutine that it uses asyncio-compatible calls to do its I/O, rather than blocking I/O. This will make it work in an asyncio context, but it means you won't be able to call it directly in a synchronous context anymore. Instead, you'd need to also provide an alternative synchronous searching method that starts an asyncio event loop and calls return loop.run_until_complete(self.searching(...)). See this question for more details on that.

2. Keep your synchronous implementation of searching, and provide an alternative asynchronous API that uses BaseEventLoop.run_in_executor to run your the searching method in a background thread:

   classsearch(object):
     ...
     self.s = some_search_engine()
     ...
     defsearching(self, *args, **kwargs):
       ret = {}
       ...
       return ret
   
      @asyncio.coroutinedefsearching_async(self, *args, **kwargs):
         loop = kwargs.get('loop', asyncio.get_event_loop())
         try:
             del kwargs['loop']  # assuming searching doesn't take loop as an argexcept KeyError:
             pass
         r = yieldfrom loop.run_in_executor(None, self.searching, *args)  # Passing None tells asyncio to use the default ThreadPoolExecutorreturn r
   

   Testing script:

   s = search()
   loop = asyncio.get_event_loop()
   loop.run_until_complete(s.searching_async(arg1, arg2, ...))
   loop.close()
   

   This way, you can keep your synchronous code as is, and at least provide methods that can be used in asyncio code without blocking the event loop. It's not as clean a solution as it would be if you actually used asynchronous I/O in your code, but its better than nothing.

3. Provide two completely separate versions of searching, one that uses blocking I/O, and one that's asyncio-compatible. This gives ideal implementations for both contexts, but requires twice the work.