Using Asyncio For Non-async Functions In Python?

Suppose there is a library that makes various database queries: import time def queryFoo(): time.sleep(4) return 'foo' def queryBar(): time.sleep(4) return 'bar'

Solution 1:

If some function is blocking and not async by nature, only proper way to run it inside asyncio event loop is to run it inside thread using run_in_executor:

# Our example blocking functionsimport time


defqueryFoo():
    time.sleep(3)
    return'foo'defqueryBar():
    time.sleep(3)
    return'bar'# Run them using asyncioimport asyncio
from concurrent.futures import ThreadPoolExecutor


_executor = ThreadPoolExecutor(10)


asyncdefin_thread(func):
    loop = asyncio.get_event_loop()
    returnawait loop.run_in_executor(_executor, func)


asyncdefmain():
    results = await asyncio.gather(
        in_thread(queryFoo), 
        in_thread(queryBar),
    )

    print(results)


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    try:
        loop.run_until_complete(main())
    finally:
        loop.run_until_complete(loop.shutdown_asyncgens())
        loop.close()

It does job.

If you however want to avoid using threads only way to do it - is to rewrite queryFoo/queryBar to be async by nature.

Solution 2:

I presume you are after concurrency and hopefully do not insist on using asyncio module itself in which case this little example could be helpful:

import asyncio
import timefrom concurrent.futures import ThreadPoolExecutor

def queryFoo():
    time.sleep(2)
    return "FOO"

def queryBar():
    time.sleep(4)
    return "BAR"

with ThreadPoolExecutor(max_workers=2) as executor:
    foo = executor.submit(queryFoo)
    bar = executor.submit(queryBar)
    results = [foo.result(), bar.result()]

print(results)

It runs both queryFoo() and queryBar() in parallel and collects their results in a list in order in which they've been mentioned in an assignment to results.