How To Access Variables From A Class Decorator From Within The Method It's Applied On?
Solution 1:
This answer is based on everything @AlexHall and @juanpa.arrivillaga wrote here: Class decorator for methods from other class. I want to thank them for their help.
Let foo()
be a method from class Foobar
, and let foo()
be decorated with a MyDecoratorClass()
-instance. So the question is:
Can the code running in
foo()
access variables from theMyDecoratorClass()
-instance?
For this to work properly, we need to think first about how many MyDecoratorClass()
instances get created over the course of the program. After lots of research and help from @AlexHall and @juanpa.arrivillaga, I concluded that there are basically three options. Let's first glance over them rapidly and then investigate them profoundly one-by-one.
Overview
OPTION 1 One
MyDecoratorClass()
-instance spawns at the very beginning of your program for the (unbound)foo()
method, and it's the only instance used to invokefoo()
. Each time you invokefoo()
, thisMyDecoratorClass()
-instance inserts the correspondingFoobar()
instance in the method through a trick.
This approach allows for communication between the code running in foo()
and the MyDecoratorClass()
-instance. However, if you've got several Foobar()
-instances f1
and f2
in your program, then f1.foo()
can have an impact on the way f2.foo()
behaves - because they share the same MyDecoratorClass()
-instance!
OPTION 2 Again one
MyDecoratorClass()
-instance spawns at the very beginning of the program for the (unbound)foo()
method. However, each time you access it, it returns a NEWMyDecoratorClass()
-instance on the fly. This instance is short-lived. It dies immediately after completing the method.
This approach doesn't allow for any communication between the code running in foo()
and the MyDecoratorClass()
-instance. Imagine you're inside the foo()
code and you try to access a variable from the MyDecoratorClass()
-instance:
@MyDecoratorClassdeffoo(self):
# I want to access the 'decorator_var' right here:
value = self.foo.decorator_var
print(f"foo decorator_var = {value}")
The moment you even try to reach decorator_var
, you essentially get a new MyDecoratorClass()
-instance returned from the __get__()
method!
OPTION 3 Just like before, one
MyDecoratorClass()
-instance spawns at the very beginning of the program for the (unbound)foo()
method. Each time you access it (which implies calling its__get__()
method), it checks who is trying to access. If it's an unknownFoobar()
-object, the__get__()
method returns a NEWMyDecoratorClass()
-instance with a boundfoo()
-method. If it's a knownFoobar()
-object, the__get__()
method retrieves theMyDecoratorClass()
-instance it has spawn before for that veryFoobar()
-object, and returns it.
This option ensures a one-to-one relationship: each Foobar()
-object gets exactly one MyDecoratorClass()
-instance to wrap its foo()
method. And each MyDecoratorClass()
-instance belongs to exactly one Foobar()
-object. Very neat!
(*) The MyDecoratorClass()
-instance spawn at the very beginning of the program for the unbound foo()
method is the only exception here. But this instance gets only used for its __get__()
method, which serves as a MyDecoratorClass()
-instance-factory: spawning, returning and storing exactly one MyDecoratorClass()
-instance per Foobar()
instance upon which foo()
has been invoked.
Let's go through each of the options. Before doing so, I'd like to stress that the only implementation difference between the three options is in the __get__()
method!
1. FIRST OPTION: Stick to one instance
Let MyDecoratorClass
be a decorator for method foo
defined in class Foobar
:
import functools, types
classMyDecoratorClass:
def__init__(self, method) -> None:
functools.update_wrapper(self, method)
self.method = method
def__get__(self, obj, objtype) -> object:
returnlambda *args, **kwargs: self.__call__(obj, *args, **kwargs)
def__call__(self, *args, **kwargs) -> object:
return self.method(*args, **kwargs)
classFoobar:
def__init__(self):
pass @MyDecoratorClassdeffoo(self):
print(f"foo!")
Even if you never instantiate Foobar()
, the Python interpreter will still create ONE instance of MyDecoratorClass
in the very beginning of your program. This one instance is created for the UNBOUND method foo()
. OPTION 1 basically implies to stick to this MyDecoratorClass()
-instance for the rest of the program. To achieve this, we need to make sure that the __get__()
method doesn't re-instantiate MyDecoratorClass()
. Instead, it should make the existing MyDecoratorClass()
APPEAR to hold a bound method:
┌────────────────────────────────────────────────────────────────────────┐
│ def__get__(self, obj, objtype=None): │
│ returnlambda *args, **kwargs: self.__call__(obj, *args, **kwargs) │
└────────────────────────────────────────────────────────────────────────┘
As you can see, self.method
NEVER gets bound to a Foobar()
-instance. Instead, it just appears that way. Let's do a test to prove this. Instantiate Foobar()
and invoke the foo()
method:
>>> f = Foobar()
>>> f.foo()
The method invocation essentially exists of two parts:
PART 1
f.foo
invokes the__get__()
method. This gets invoked on the ONE AND ONLYMyDecoratorClass()
instance, which holds an unbound method inself.method
. It then returns a lambda-reference to its__call__()
method, but with theFoobar()
instance added to the *args tuple.
PART 2 The parenthesis
'()'
afterf.foo
are applied on WHATEVER__get__()
returned. In this case, we know that__get__()
returned the__call__()
method from the ONE AND ONLYMyDecoratorClass()
instance (actually a bit modified with lambda), so naturally that method gets invoked.Inside the
__call__()
method, we invoke the stored method (the original foo) like so:self.method(*args, **kwargs)
While
self.method
is an unbound version offoo()
, theFoobar()
instance is right there in the first element of *args!
In short: Each time you invoke the foo()
method on a Foobar()
-instance, you deal with the ONE AND ONLY MyDecoratorClass()
-instance which holds an unbound foo()
method-reference and makes it appear to be bound to the very Foobar()
-instance you invoked foo()
on!
Some extra tests
You can verify that self.method
is always unbound in the __call__()
method with:
hasattr(self.method, '__self__')
self.method.__self__ is not None
which always prints False
!
You can also put a print-statement in the __init__()
method to verify that MyDecoratorClass()
gets instantiated only once, even if you invoke foo()
on multiple Foobar()
objects.
Notes As @AlexHall pointed out, this:
returnlambda *args, **kwargs: self.__call__(obj, *args, **kwargs)
is essentially the same as:
returnlambda *args, **kwargs: self(obj, *args, **kwargs)
That's because applying parenthesis '()'
on an object is essentially the same as invoking its __call__()
method. You can also replace the return statement with:
return functools.partial(self, obj)
or even:
return types.MethodType(self, obj)
2. SECOND OPTION: Create a new instance per invocation
In this second option, we instantiate a new MyDecoratorClass()
-instance upon each and every foo()
invocation:
┌─────────────────────────────────────────────────────────────┐
│ def__get__(self, obj, objtype=None): │
│ returntype(self)(self.method.__get__(obj, objtype)) │
└─────────────────────────────────────────────────────────────┘
This MyDecoratorClass()
-instance is very short-lived. I've checked with a print-statement in the __del__()
method that it gets garbage collected right after foo() ends!
So this is what happens if you invoke foo()
on several Foobar()
instances:
>>> f1 = Foobar()
>>> f2 = Foobar()
>>> f1.foo()
>>> f2.foo()
As always, a MyDecoratorClass()
-instance for the unbound foo()
method gets spawn before any Foobar()
-object gets born. It remains alive until the end of the program. Let's call this one the immortal MyDecoratorClass()
-instance.
The moment you invoke foo()
, you create a new short-lived MyDecoratorClass()
-instance. Remember, the foo()
invocation essentially happens in two steps:
STEP 1
f1.foo
invokes the__get__()
method on the immortalMyDecoratorClass()
- instance (there is no other at this point!). Unlike OPTION 1, we now spawn a NEWMyDecoratorClass()
and pass it a boundfoo()
method as argument. This newMyDecoratorClass()
-instance gets returned.
STEP 2 The parenthesis
'()'
afterf1.foo
are applied on WHATEVER__get__()
returned. We know it's a NEWMyDecoratorClass()
-instance, so the parenthesis'()'
invoke its__call__()
method. Inside the__call__()
method, we still got this:self.method(*args, **kwargs)
This time however, there is NO
Foobar()
-object hidden in the args tuple, but the stored method is bound now - so there is no need for that!
f1.foo()
completes and the short-lived MyDecoratorClass()
-instance gets garbage collected (you can test this with a print-statement in the __del__()
method).
It's time for f2.foo()
now. As the short-lived MyDecoratorClass()
-instance died, it invokes the __get__()
method on the immortal one (what else?). In the process, a NEW instance gets created and the cycle repeats.
In short: Each foo()
invocation starts with calling the __get__()
method on the immortal MyDecoratorClass()
-instance. This object always returns a NEW but short-lived MyDecoratorClass()
-instance with a bound foo()
-method. It dies after completing the job.
3. THIRD OPTION: One `MyDecoratorClass()`-instance per `Foobar()`-instance
The third and last option combines the best of both worlds. It creates one MyDecoratorClass()
-instance per Foobar()
-instance.
Keep an __obj_dict__
dictionary as a class-variable and implement the __get__()
method like so:
┌───────────────────────────────────────────────────────────────┐
│ def__get__(self, obj, objtype): │
│ if obj in MyDecoratorClass.__obj_dict__: │
│ # Return existing MyDecoratorClass() instance for │
│ # the given object, and make sure it holds a bound │
│ # method. │
│ m = MyDecoratorClass.__obj_dict__[obj] │
│ assert m.method.__self__ is obj │
│ return m │
│ # Create a new MyDecoratorClass() instance WITH a bound │
│ # method, and store it in the dictionary. │
│ m = type(self)(self.method.__get__(obj, objtype)) │
│ MyDecoratorClass.__obj_dict__[obj] = m │
│ return m │
└───────────────────────────────────────────────────────────────┘
So whenever foo()
gets invoked, the __get__()
method checks if a MyDecoratorClass()
-instance was already spawn (with bound method) for the given Foobar()
-object. If yes, that MyDecoratorClass()
-instance gets returned. Otherwise, a new one gets spawn and stored in the class dictionary MyDecoratorClass.__obj_dict__
.
(*) Note: This MyDecoratorClass.__obj_dict__
is a class-level dictionary you have to create yourself in the class definition.
(*) Note: Also here, the __get__()
method always gets invoked on the immortal MyDecoratorClass()
-instance that is spawn at the very beginning of the program - before any Foobar()
-objects were born. However, what's important is what the __get__()
method returns.
WARNING
Keeping an __obj_dict__
to store all the Foobar()
-instances has a downside. None of them will ever die. Depending on the situation, this can be a huge memory leak. So think about a proper solution before applying OPTION 3.
I also believe this approach doesn't allow recursion. To be tested.
4. Data exchange between the code in `foo()` and the `MyDecoratorClass()`-instance
Let's go back to the initial question:
Let
foo()
be a method from classFoobar
, and letfoo()
be decorated with aMyDecoratorClass()
-instance. Can the code running infoo()
access variables from theMyDecoratorClass()
-instance?
If you implement the first or the third option, you can access any MyDecoratorClass()
-instance variable from within the foo()
code:
@MyDecoratorClassdeffoo(self):
value = self.foo.decorator_var
print(f"foo decorator_var = {value}")
With self.foo
actually accessing the MyDecoratorClass()
-instance. After all, MyDecoratorClass()
is a wrapper for self.foo
!
Now if you implement option 1, you need to keep in mind that decorator_var
is shared amongst all Foobar()
-objects. For option 3, each Foobar()
-object has its own MyDecoratorClass()
for the foo()
method.
5. One step further: apply `@MyDecoratorClass` on several methods
Option 3 worked fine - until I applied @MyDecoratorClass
on two methods:
classFoobar:
def__init__(self):
pass @MyDecoratorClassdeffoo(self):
print(f"foo!")
@MyDecoratorClassdefbar(self):
print("bar!")
Now try this:
>>> f = Foobar()
>>> f.foo()
>>> f.bar()
foo!
foo!
Once a MyDecoratorClass()
-instance exists for the Foobar()
object, you'll always access this existing one to invoke the method. In our case, this MyDecoratorClass()
-instance got bound to the foo()
method, so bar()
never executes!
The solution is to revise the way we store the MyDecoratorClass()
-instance in __obj_dict__
. Don't just spawn and store one MyDecoratorClass()
-instance per Foobar()
-object, but one instance per (Foobar()
, method
) combination! That requires an extra parameter for our decorator, eg:
@MyDecoratorClass("foo")deffoo(self):
print(f"foo!")
@MyDecoratorClass("bar")defbar(self):
print("bar!")
A decorator with a parameter essentially means double-wrapping the underlying method/function! So let's design a wrapper for that:
defmy_wrapper(name="unknown"):
def_my_wrapper_(method):
return MyDecoratorClass(method, name)
return _my_wrapper_
and now use this wrapper:
classFoobar:
def__init__(self):
pass @my_wrapper("foo")deffoo(self):
print(f"foo!")
@my_wrapper("bar")defbar(self):
print("bar!")
Finally, we need to refactor the MyDecoratorClass
:
import functools, types
classMyDecoratorClass:
__obj_dict__ = {}
def__init__(self, method, name="unknown") -> None:
functools.update_wrapper(self, method)
self.method = method
self.method_name = name
returndef__get__(self, obj, objtype) -> object:
if obj in MyDecoratorClass.__obj_dict__.keys():
# Return existing MyDecoratorClass() instance for# the given object-method_name combination, and make# sure it holds a bound method.if self.method_name in MyDecoratorClass.__obj_dict__[obj].keys():
m = MyDecoratorClass.__obj_dict__[obj][self.method_name]
return m
else:
# Create a new MyDecoratorClass() instance WITH a bound# method, and store it in the dictionary.
m = type(self)(self.method.__get__(obj, objtype), self.method_name)
MyDecoratorClass.__obj_dict__[obj][self.method_name] = m
return m
# Create a new MyDecoratorClass() instance WITH a bound# method, and store it in the dictionary.
m = type(self)(self.method.__get__(obj, objtype), self.method_name)
MyDecoratorClass.__obj_dict__[obj] = {}
MyDecoratorClass.__obj_dict__[obj][self.method_name] = m
return m
def__call__(self, *args, **kwargs) -> object:
return self.method(*args, **kwargs)
def__del__(self):
print(f"{id(self)} garbage collected!")
Let's revise: at the beginning of the program, before any Foobar()
-object is born, the Python interpreter already spawns two MyDecoratorClass()
-instances: one for the unbound foo()
and another for the unbound bar()
method. These are our immortal MyDecoratorClass()
-instances whose __get__()
methods serve as MyDecoratorClass()
factories.
Nothing new here. This happened also before we did these changes. However, now we store the method_name
at the moment the factories are built! This way, the factory method __get__()
can make use of that information to spawn and store not just one MyDecoratorClass()
-instances per Foobar()
object, but one for the (Foobar()
, "foo"
) and (Foobar()
, "bar"
) combination!
This is the complete self-contained program:
import functools, types
classMyDecoratorClass:
__obj_dict__ = {}
def__init__(self, method, name="unknown") -> None:
functools.update_wrapper(self, method)
self.method = method
self.method_name = name
returndef__get__(self, obj, objtype) -> object:
if obj in MyDecoratorClass.__obj_dict__.keys():
# Return existing MyDecoratorClass() instance for# the given object-method_name combination, and make# sure it holds a bound method.if self.method_name in MyDecoratorClass.__obj_dict__[obj].keys():
m = MyDecoratorClass.__obj_dict__[obj][self.method_name]
return m
else:
# Create a new MyDecoratorClass() instance WITH a bound# method, and store it in the dictionary.
m = type(self)(self.method.__get__(obj, objtype), self.method_name)
MyDecoratorClass.__obj_dict__[obj][self.method_name] = m
return m
# Create a new MyDecoratorClass() instance WITH a bound# method, and store it in the dictionary.
m = type(self)(self.method.__get__(obj, objtype), self.method_name)
MyDecoratorClass.__obj_dict__[obj] = {}
MyDecoratorClass.__obj_dict__[obj][self.method_name] = m
return m
def__call__(self, *args, **kwargs) -> object:
return self.method(*args, **kwargs)
def__del__(self):
print(f"{id(self)} garbage collected!")
defmy_wrapper(name="unknown"):
def_my_wrapper_(method):
return MyDecoratorClass(method, name)
return _my_wrapper_
classFoobar:
def__init__(self):
pass @my_wrapper("foo")deffoo(self):
print(f"foo!")
@my_wrapper("bar")defbar(self):
print("bar!")
Post a Comment for "How To Access Variables From A Class Decorator From Within The Method It's Applied On?"