How To Access Variables From A Class Decorator From Within The Method It's Applied On?

NOTE I've compiled an answer based on everything written by @AlexHall and @juanpa.arrivillaga. See below. I'm writing a Class Decorator to be applied on methods. This practice is

Solution 1:

This answer is based on everything @AlexHall and @juanpa.arrivillaga wrote here: Class decorator for methods from other class. I want to thank them for their help.

---

Let foo() be a method from class Foobar, and let foo() be decorated with a MyDecoratorClass()-instance. So the question is:

Can the code running in foo() access variables from the MyDecoratorClass()-instance?

For this to work properly, we need to think first about how many MyDecoratorClass() instances get created over the course of the program. After lots of research and help from @AlexHall and @juanpa.arrivillaga, I concluded that there are basically three options. Let's first glance over them rapidly and then investigate them profoundly one-by-one.

Overview

OPTION 1 One MyDecoratorClass()-instance spawns at the very beginning of your program for the (unbound) foo() method, and it's the only instance used to invoke foo(). Each time you invoke foo(), this MyDecoratorClass()-instance inserts the corresponding Foobar() instance in the method through a trick.

This approach allows for communication between the code running in foo() and the MyDecoratorClass()-instance. However, if you've got several Foobar()-instances f1 and f2 in your program, then f1.foo() can have an impact on the way f2.foo() behaves - because they share the same MyDecoratorClass()-instance!

OPTION 2 Again one MyDecoratorClass()-instance spawns at the very beginning of the program for the (unbound) foo() method. However, each time you access it, it returns a NEW MyDecoratorClass()-instance on the fly. This instance is short-lived. It dies immediately after completing the method.

This approach doesn't allow for any communication between the code running in foo() and the MyDecoratorClass()-instance. Imagine you're inside the foo() code and you try to access a variable from the MyDecoratorClass()-instance:

@MyDecoratorClassdeffoo(self):
    # I want to access the 'decorator_var' right here:
    value = self.foo.decorator_var
    print(f"foo decorator_var = {value}")

The moment you even try to reach decorator_var, you essentially get a new MyDecoratorClass()-instance returned from the __get__() method!

OPTION 3 Just like before, one MyDecoratorClass()-instance spawns at the very beginning of the program for the (unbound) foo() method. Each time you access it (which implies calling its __get__() method), it checks who is trying to access. If it's an unknown Foobar()-object, the __get__() method returns a NEW MyDecoratorClass()-instance with a bound foo()-method. If it's a known Foobar()-object, the __get__() method retrieves the MyDecoratorClass()-instance it has spawn before for that very Foobar()-object, and returns it.

This option ensures a one-to-one relationship: each Foobar()-object gets exactly one MyDecoratorClass()-instance to wrap its foo() method. And each MyDecoratorClass()-instance belongs to exactly one Foobar()-object. Very neat!

_{(*) The MyDecoratorClass()-instance spawn at the very beginning of the program for the unbound foo() method is the only exception here. But this instance gets only used for its __get__() method, which serves as a MyDecoratorClass()-instance-factory: spawning, returning and storing exactly one MyDecoratorClass()-instance per Foobar() instance upon which foo() has been invoked.}

Let's go through each of the options. Before doing so, I'd like to stress that the only implementation difference between the three options is in the __get__() method!

---

1. FIRST OPTION: Stick to one instance

Let MyDecoratorClass be a decorator for method foo defined in class Foobar:

import functools, types

classMyDecoratorClass:
    def__init__(self, method) -> None:
        functools.update_wrapper(self, method)
        self.method = method

    def__get__(self, obj, objtype) -> object:
    returnlambda *args, **kwargs: self.__call__(obj, *args, **kwargs)

    def__call__(self, *args, **kwargs) -> object:
        return self.method(*args, **kwargs)

classFoobar:
    def__init__(self):
        pass    @MyDecoratorClassdeffoo(self):
        print(f"foo!")

Even if you never instantiate Foobar(), the Python interpreter will still create ONE instance of MyDecoratorClass in the very beginning of your program. This one instance is created for the UNBOUND method foo(). OPTION 1 basically implies to stick to this MyDecoratorClass()-instance for the rest of the program. To achieve this, we need to make sure that the __get__() method doesn't re-instantiate MyDecoratorClass(). Instead, it should make the existing MyDecoratorClass() APPEAR to hold a bound method:

    ┌────────────────────────────────────────────────────────────────────────┐
    │ def__get__(self, obj, objtype=None):                                  │
    │     returnlambda *args, **kwargs: self.__call__(obj, *args, **kwargs) │
    └────────────────────────────────────────────────────────────────────────┘

As you can see, self.method NEVER gets bound to a Foobar()-instance. Instead, it just appears that way. Let's do a test to prove this. Instantiate Foobar() and invoke the foo() method:

>>> f = Foobar()
>>> f.foo()

The method invocation essentially exists of two parts:

PART 1 f.foo invokes the __get__() method. This gets invoked on the ONE AND ONLY MyDecoratorClass() instance, which holds an unbound method in self.method. It then returns a lambda-reference to its __call__() method, but with the Foobar() instance added to the *args tuple.

PART 2 The parenthesis '()' after f.foo are applied on WHATEVER __get__() returned. In this case, we know that __get__() returned the __call__() method from the ONE AND ONLY MyDecoratorClass() instance (actually a bit modified with lambda), so naturally that method gets invoked.

Inside the __call__() method, we invoke the stored method (the original foo) like so:

self.method(*args, **kwargs)

While self.method is an unbound version of foo(), the Foobar() instance is right there in the first element of *args!

In short: Each time you invoke the foo() method on a Foobar()-instance, you deal with the ONE AND ONLY MyDecoratorClass()-instance which holds an unbound foo() method-reference and makes it appear to be bound to the very Foobar()-instance you invoked foo() on!

Some extra tests You can verify that self.method is always unbound in the __call__() method with:

• hasattr(self.method, '__self__')
• self.method.__self__ is not None

which always prints False!

You can also put a print-statement in the __init__() method to verify that MyDecoratorClass() gets instantiated only once, even if you invoke foo() on multiple Foobar() objects.

Notes As @AlexHall pointed out, this:

returnlambda *args, **kwargs: self.__call__(obj, *args, **kwargs)

is essentially the same as:

returnlambda *args, **kwargs: self(obj, *args, **kwargs)

That's because applying parenthesis '()' on an object is essentially the same as invoking its __call__() method. You can also replace the return statement with:

return functools.partial(self, obj)

or even:

return types.MethodType(self, obj)

---

2. SECOND OPTION: Create a new instance per invocation

In this second option, we instantiate a new MyDecoratorClass()-instance upon each and every foo() invocation:

    ┌─────────────────────────────────────────────────────────────┐
    │ def__get__(self, obj, objtype=None):                       │
    │     returntype(self)(self.method.__get__(obj, objtype))    │
    └─────────────────────────────────────────────────────────────┘

This MyDecoratorClass()-instance is very short-lived. I've checked with a print-statement in the __del__() method that it gets garbage collected right after foo() ends!

So this is what happens if you invoke foo() on several Foobar() instances:

>>> f1 = Foobar()
>>> f2 = Foobar()
>>> f1.foo()
>>> f2.foo()

As always, a MyDecoratorClass()-instance for the unbound foo() method gets spawn before any Foobar()-object gets born. It remains alive until the end of the program. Let's call this one the immortal MyDecoratorClass()-instance.

The moment you invoke foo(), you create a new short-lived MyDecoratorClass()-instance. Remember, the foo() invocation essentially happens in two steps:

STEP 1 f1.foo invokes the __get__() method on the immortal MyDecoratorClass()- instance (there is no other at this point!). Unlike OPTION 1, we now spawn a NEW MyDecoratorClass() and pass it a bound foo() method as argument. This new MyDecoratorClass()-instance gets returned.

STEP 2 The parenthesis '()' after f1.foo are applied on WHATEVER __get__() returned. We know it's a NEW MyDecoratorClass()-instance, so the parenthesis '()' invoke its __call__() method. Inside the __call__() method, we still got this:

self.method(*args, **kwargs)

This time however, there is NO Foobar()-object hidden in the args tuple, but the stored method is bound now - so there is no need for that!

f1.foo() completes and the short-lived MyDecoratorClass()-instance gets garbage collected (you can test this with a print-statement in the __del__() method).

It's time for f2.foo() now. As the short-lived MyDecoratorClass()-instance died, it invokes the __get__() method on the immortal one (what else?). In the process, a NEW instance gets created and the cycle repeats.

In short: Each foo() invocation starts with calling the __get__() method on the immortal MyDecoratorClass()-instance. This object always returns a NEW but short-lived MyDecoratorClass()-instance with a bound foo()-method. It dies after completing the job.

---

3. THIRD OPTION: One `MyDecoratorClass()`-instance per `Foobar()`-instance

The third and last option combines the best of both worlds. It creates one MyDecoratorClass()-instance per Foobar()-instance.

Keep an __obj_dict__ dictionary as a class-variable and implement the __get__() method like so:

    ┌───────────────────────────────────────────────────────────────┐
    │ def__get__(self, obj, objtype):                              │
    │     if obj in MyDecoratorClass.__obj_dict__:                  │
    │         # Return existing MyDecoratorClass() instance for     │
    │         # the given object, and make sure it holds a bound    │
    │         # method.                                             │
    │         m = MyDecoratorClass.__obj_dict__[obj]                │
    │         assert m.method.__self__ is obj                       │
    │         return m                                              │
    │     # Create a new MyDecoratorClass() instance WITH a bound   │
    │     # method, and store it in the dictionary.                 │
    │     m = type(self)(self.method.__get__(obj, objtype))         │
    │     MyDecoratorClass.__obj_dict__[obj] = m                    │
    │     return m                                                  │
    └───────────────────────────────────────────────────────────────┘

So whenever foo() gets invoked, the __get__() method checks if a MyDecoratorClass()-instance was already spawn (with bound method) for the given Foobar()-object. If yes, that MyDecoratorClass()-instance gets returned. Otherwise, a new one gets spawn and stored in the class dictionary MyDecoratorClass.__obj_dict__.

_{(*) Note: This MyDecoratorClass.__obj_dict__ is a class-level dictionary you have to create yourself in the class definition.}

_{(*) Note: Also here, the __get__() method always gets invoked on the immortal MyDecoratorClass()-instance that is spawn at the very beginning of the program - before any Foobar()-objects were born. However, what's important is what the __get__() method returns.}

WARNING Keeping an __obj_dict__ to store all the Foobar()-instances has a downside. None of them will ever die. Depending on the situation, this can be a huge memory leak. So think about a proper solution before applying OPTION 3.

I also believe this approach doesn't allow recursion. To be tested.

---

4. Data exchange between the code in `foo()` and the `MyDecoratorClass()`-instance

Let's go back to the initial question:

Let foo() be a method from class Foobar, and let foo() be decorated with a MyDecoratorClass()-instance. Can the code running in foo() access variables from the MyDecoratorClass()-instance?

If you implement the first or the third option, you can access any MyDecoratorClass()-instance variable from within the foo() code:

@MyDecoratorClassdeffoo(self):
    value = self.foo.decorator_var
    print(f"foo decorator_var = {value}")

With self.foo actually accessing the MyDecoratorClass()-instance. After all, MyDecoratorClass() is a wrapper for self.foo!

Now if you implement option 1, you need to keep in mind that decorator_var is shared amongst all Foobar()-objects. For option 3, each Foobar()-object has its own MyDecoratorClass() for the foo() method.

---

5. One step further: apply `@MyDecoratorClass` on several methods

Option 3 worked fine - until I applied @MyDecoratorClass on two methods:

classFoobar:
    def__init__(self):
        pass    @MyDecoratorClassdeffoo(self):
        print(f"foo!")

    @MyDecoratorClassdefbar(self):
        print("bar!")

Now try this:

>>> f = Foobar()
>>> f.foo()
>>> f.bar()
foo!
foo!

Once a MyDecoratorClass()-instance exists for the Foobar() object, you'll always access this existing one to invoke the method. In our case, this MyDecoratorClass()-instance got bound to the foo() method, so bar() never executes!

The solution is to revise the way we store the MyDecoratorClass()-instance in __obj_dict__. Don't just spawn and store one MyDecoratorClass()-instance per Foobar()-object, but one instance per (Foobar(), method) combination! That requires an extra parameter for our decorator, eg:

@MyDecoratorClass("foo")deffoo(self):
    print(f"foo!")

@MyDecoratorClass("bar")defbar(self):
    print("bar!")

A decorator with a parameter essentially means double-wrapping the underlying method/function! So let's design a wrapper for that:

defmy_wrapper(name="unknown"):
    def_my_wrapper_(method):
        return MyDecoratorClass(method, name)
    return _my_wrapper_

and now use this wrapper:

classFoobar:
    def__init__(self):
        pass    @my_wrapper("foo")deffoo(self):
        print(f"foo!")

    @my_wrapper("bar")defbar(self):
        print("bar!")

Finally, we need to refactor the MyDecoratorClass:

import functools, types

classMyDecoratorClass:
    __obj_dict__ = {}
    def__init__(self, method, name="unknown") -> None:
        functools.update_wrapper(self, method)
        self.method = method
        self.method_name = name
        returndef__get__(self, obj, objtype) -> object:
        if obj in MyDecoratorClass.__obj_dict__.keys():
            # Return existing MyDecoratorClass() instance for# the given object-method_name combination, and make# sure it holds a bound method.if self.method_name in MyDecoratorClass.__obj_dict__[obj].keys():
                m = MyDecoratorClass.__obj_dict__[obj][self.method_name]
                return m
            else:
                # Create a new MyDecoratorClass() instance WITH a bound# method, and store it in the dictionary.
                m = type(self)(self.method.__get__(obj, objtype), self.method_name)
                MyDecoratorClass.__obj_dict__[obj][self.method_name] = m
                return m

        # Create a new MyDecoratorClass() instance WITH a bound# method, and store it in the dictionary.
        m = type(self)(self.method.__get__(obj, objtype), self.method_name)
        MyDecoratorClass.__obj_dict__[obj] = {}
        MyDecoratorClass.__obj_dict__[obj][self.method_name] = m
        return m

    def__call__(self, *args, **kwargs) -> object:
        return self.method(*args, **kwargs)


    def__del__(self):
        print(f"{id(self)} garbage collected!")

Let's revise: at the beginning of the program, before any Foobar()-object is born, the Python interpreter already spawns two MyDecoratorClass()-instances: one for the unbound foo() and another for the unbound bar() method. These are our immortal MyDecoratorClass()-instances whose __get__() methods serve as MyDecoratorClass() factories.

Nothing new here. This happened also before we did these changes. However, now we store the method_name at the moment the factories are built! This way, the factory method __get__() can make use of that information to spawn and store not just one MyDecoratorClass()-instances per Foobar() object, but one for the (Foobar(), "foo") and (Foobar(), "bar") combination!

This is the complete self-contained program:

import functools, types

classMyDecoratorClass:
    __obj_dict__ = {}
    def__init__(self, method, name="unknown") -> None:
        functools.update_wrapper(self, method)
        self.method = method
        self.method_name = name
        returndef__get__(self, obj, objtype) -> object:
        if obj in MyDecoratorClass.__obj_dict__.keys():
            # Return existing MyDecoratorClass() instance for# the given object-method_name combination, and make# sure it holds a bound method.if self.method_name in MyDecoratorClass.__obj_dict__[obj].keys():
                m = MyDecoratorClass.__obj_dict__[obj][self.method_name]
                return m
            else:
                # Create a new MyDecoratorClass() instance WITH a bound# method, and store it in the dictionary.
                m = type(self)(self.method.__get__(obj, objtype), self.method_name)
                MyDecoratorClass.__obj_dict__[obj][self.method_name] = m
                return m

        # Create a new MyDecoratorClass() instance WITH a bound# method, and store it in the dictionary.
        m = type(self)(self.method.__get__(obj, objtype), self.method_name)
        MyDecoratorClass.__obj_dict__[obj] = {}
        MyDecoratorClass.__obj_dict__[obj][self.method_name] = m
        return m

    def__call__(self, *args, **kwargs) -> object:
        return self.method(*args, **kwargs)


    def__del__(self):
        print(f"{id(self)} garbage collected!")


defmy_wrapper(name="unknown"):
    def_my_wrapper_(method):
        return MyDecoratorClass(method, name)
    return _my_wrapper_

classFoobar:
    def__init__(self):
        pass    @my_wrapper("foo")deffoo(self):
        print(f"foo!")

    @my_wrapper("bar")defbar(self):
        print("bar!")