Python

Understanding Decorators (Part 1)

Senior Instructor at DevelopIntelligence (2019-present)Admin ·

In order to understand decorators, you need to understand four concepts:

functions are objects, and therefore they can be returned from other functions, and passed to functions as arguments;functions can be nested–that is, you can define a function inside another function;the LEGB rule for resolving identifiers in Python (explained below)

Wait…that’s only three. Well, we programmers are off by one more often than we’d care to admit. But seriously, let’s start with these, and add the fourth later, when we get to Part 2.

Assuming the above concepts make sense to you, we can get started. But before we do, let’s motivate why we would create a decorator in the first place. A common use case for decorators is to remove repeated boilerplate code. Suppose we have a module of functions, all of which operate on non-negative numbers, but fail when a negative number is passed.

We could put a check for the sign of the number in each and every function, but as programmers, we live by the DRY principle–don’t repeat yourself. It’s unsightly to look at repeated code, and worse, repeated code can be the source of bugs–if we change only some of the instances of the repeated code, we’ll surely have a mess on our hands.

Let’s consider a function which calculates the square root of its argument:

def my_sqrt(val):    """Compute square root using Newton's method"""      if val < 0.0:        raise ValueError('math domain error')      prev, curr = 0.0, 1.0     while abs(curr - prev) > 1.0e-8:        prev = curr        curr = prev - (prev * prev - val) / (2 * prev)     return curr

And imagine we have other functions, all of which do the same check at lines 3–4 above.

We’d like to partial out the test into a decorator. We’ll call the decorator ensure_positive.

def ensure_positive(somefunc):    """Decorator for insuring function argument is positive"""    def innerfunc(arg):        if arg < 0.0:            raise ValueError(f'Argument to {somefunc.__name__} must be positive!')        return somefunc(arg)     return innerfunc

First off, we can see a decorator is just a function. In this case, the function accepts a single argument, somefunc, which as we’ll see, is a function. (Decorators are, by definition, function wrappers, i.e., functions which take other functions as their argument.)

At line 3 we see a nested (or inner) function–this is the one that does the work. This function accepts a single argument, arg, and it raises an exception if it’s less than zero. That’s not very interesting. The really interesting part is what it does if arg is not less than zero–it invokes the function somefunc and passes arg to it.

Where did somefunc come from? It’s defined in the enclosing scope–it’s the parameter to the ensure_positive function. When Python encounters an identifier (or name) such as x, it first checks the local function to try to resolve it. If there is a local x, that’s the one Python uses. If there is no x in the local scope, the enclosing function (or functions) is checked next. If no x there, Python looks for a global x, and if it doesn’t find a global x, Python looks for a builtin function by that name. If it’s still not found, Python throws a NameError. This process is known as the LEGB Rule–Local, Enclosing, Global, and Built-in.

So our inner function invokes the function passed to the outer function and returns whatever that function (somefunc) returns (line 6).

Finally, at line 8–and if you followed this so far, here’s where you may likely get confused–ensure_positive returns the inner function, or more correctly, returns a reference to the inner function (remember, functions can accepts functions as parameters and return functions as well).

Let’s put aside decorators for a moment and try our square root function:

>>> my_sqrt(-58)ValueError: math domain error

OK, our function works as expected. Now let’s remove the if statement and try it…well, there’s nothing to see. Newton’s method fails to converge on negative numbers, and the function runs forever.

Now let’s do what’s called a manual decorator assignment. What this does is it wraps the my_sqrt function with the decorator we wrote:

>>> improved_sqrt = ensure_positive(my_sqrt)

We’ve now created a new (and improved) function called improved_sqrt. We can see that it’s different than my_sqrt:

>>> id(my_sqrt)4403525312>>> id(improved_sqrt)4403524416

That is to say, the two functions are distinct, they live in different places in the computer’s memory. Let’s run them:

>>> my_sqrt(53)7.280109889280518>>> improved_sqrt(53)7.280109889280518

Now let’s run the new one with a negative argument:

>>> improved_sqrt(-53)Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/Users/dws/PycharmProjects/classer/classer.py", line 4, in innerfunc    raise ValueError(f'Argument to {somefunc.__name__} must be positive!')ValueError: Argument to my_sqrt must be positive!

It works! One slight problem we see is that the name of the function in the exception is wrong–my_sqrt instead of improved_sqrt. We’ll ignore that for now.

Now let’s back up a minute and consider that manual decorator assignment. It created a new function–so we have the original function, which does not do the error checking we want and the new function which does. There’s really no reason to have two functions. Let’s redo the manual decorator assignment so that we end up with only one function. Imagine that I exited and restarted Python, so the memory has been wiped clean.

>>> my_sqrt = ensure_positive(my_sqrt)

You might have done a double take when reading the above. It’s confusing at first. Think of it as “my_sqrt is a new wrapped version of the original my_sqrt” (because that’s in fact what it is).

Let’s try it.

>>> my_sqrt(-53)Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/Users/dws/PycharmProjects/classer/classer.py", line 4, in innerfunc    raise ValueError(f'Argument to {somefunc.__name__} must be positive!')ValueError: Argument to my_sqrt must be positive!

It works! (And the function name is correct in the exception too…)

This idea of a function wrapper is so common, Python gave it its own operator–the @ symbol:

@ensure_positivedef my_sqrt(val):    """Compute square root using Newton's method"""    prev, curr = 0.0, 1.0            while abs(curr - prev) > 1.0e-8:        prev = curr        curr = prev - (prev * prev - val) / (2 * prev)     return curr

Line 1 is exactly the same as:

>>> my_sqrt = ensure_positive(my_sqrt)

and it is executed as the Python interpreter reads the code.

We can use this same syntax with any functions. Let’s consider a factorial function:

>>> def fact(n):...     if n < 2:...         return n...     return n * fact(n - 1)...>>> fact(5)120>>> fact(-2)-2

It works for positive numbers, but fails for negative numbers. Let’s decorate the fact()function to fix that:

>>> @ensure_positive... def fact(n):...     if n < 2:...         return n...     return n * fact(n - 1)...>>> fact(5)120>>> fact(-2)Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/Users/dws/PycharmProjects/classer/classer.py", line 4, in innerfunc    raise ValueError(f'Argument to {somefunc.__name__} must be positive!')ValueError: Argument to fact must be positive!

How about Fibonacci?

>>> def fib(n):...     if n < 2:...         return n...     return fib(n - 1) + fib(n - 2)...>>> fib(0)0>>> fib(23)28657>>> fib(-3)-3

As with fact(), fib() is wrong for negative numbers. Let’s fix it:

>>> @ensure_positive... def fib(n):...     if n < 2:...         return n...     return fib(n - 1) + fib(n - 2)...>>> fib(0)0>>> fib(30)832040>>> fib(-2)Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/Users/dws/PycharmProjects/classer/classer.py", line 4, in innerfunc    raise ValueError(f'Argument to {somefunc.__name__} must be positive!')ValueError: Argument to fib must be positive!

I hope this post has cleared up the myste Want to add a caption to this image? Click the Settings icon. ry of decorators! In Part 2 we’ll build upon what we’ve learned here, and also discuss some built-in decorators we can make use of.

3k views ·