Studying Parallel Programming Python

import multiprocessing from multiprocessing import Pool from source.RUN import* def func(r,grid,pos,h): return r,grid,pos,h p = multiprocessing.Pool() # Creates a pool wi

Solution 1:

I'm still having a somewhat difficult time understanding your question. But I think this is what you're looking for:

You want to be able to call a function that creates a pool given r, grid, pos, h Iterate over pos feed it to the Pool, then return the results. You also want to be able to access that function from different modules. If that's what you're asking, you can do it like this:

async_module.py:

from multiprocessing import Pool

# Not sure where the LISTE function gets defined, but it needs to be in here.

def do_LISTE(*args):
    # args is a tuple containing (r, grid, pos[i, :], h)
    # we use tuple expansion (*args( to send each parameter to LISTE
    return LISTE(*args)

def async_process(r,grid,pos,h):
    return r,grid,pos,h
    p = multiprocessing.Pool()  # Creates a pool with as many workers as you have CPU cores
    results = p.map(do_LISTE, [(r,grid,pos[i,:], h) for i in pos[-1]<2])
    p.close()
    p.join()
    return results

Then in some other module:

from async_module import async_process

def do_async_processing():
    r = "something"
    grid = get_grid()
    pos = get_pos()
    h = 345
    results = async_process(r, grid, pos, h)

if __name__ == "__main__":
    do_async_processing()  # Make sure the entry point is protected by `if __name__ == "__main__":`.