Python: Transfer A Class Method To Another Computer

I have created an class that is used for analysising a specific type of data that I produce. I use this class on a local computer but occasionally there is too much data to work lo

Solution 1:

I'm the dill author. dill is able to pass a class method to another computer, as seen below.

>$ python
Python 3.5.6 (default, Sep 20 2018, 12:15:10) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>classFoo(object):...defbar(self, x):...return self.y + x...def__init__(self, y):...    self.y = y...>>>import dill>>>>>>f = Foo(5)>>>>>>withopen('foo.pkl', 'wb') as pkl:...  dill.dump(f.bar, pkl)...>>>

Then in a new session (or on another computer)...

>$ python
Python 3.5.6 (default, Sep 202018, 12:15:10) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type"help", "copyright", "credits"or"license"for more information.
>>> import dill
>>> withopen('foo.pkl', 'rb') as pkl:
...   b = dill.load(pkl)
... >>> b(4)
9

Without more specific code from you, it's hard to say why you aren't seeing this behavior... but dill does provide the ability to pass a class definition (or just a class method) to another computer.

This behavior is what enables code like pathos to pass the class method to another computer within a ParallelPool or a ProcessPool -- the latter is across processes, while the former can be across distributed resources.

dude>$ python
Python 3.5.6 (default, Sep 20 2018, 12:15:10) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>>>>classFoo(object):...defbar(self, x):...return self.y + x...def__init__(self, y):...    self.y = y...>>>import pathos>>>p = pathos.pools.ParallelPool()>>>p.map(Foo(4).bar, [1,2,3])
[5, 6, 7]
>>>p.close(); p.join()>>>