Python Decorators
Decorators
Decorators are a powerful Python feature that allow you to modify or enhance functions without changing their source code, commonly used for logging, timing, authentication, and more. To understand decorators, you must first understand two other key concepts in Python: functions are first-class objects and closures.
Functions are first-class objects
In python, functions are considered to be first-class objects but, what does that mean? It means that we can use functions as values. Because they are values we can assign them to variables.
Example:
# We can define a simple function
def greet():
print("Hello, World!")
# and then pass that function to be stored in the
# variable my_func
my_func = greet
# We can now call the function that is stored in
# my_func appending parentheses
my_func()
Output:
Hello, World!
You can see here that when we assigned the greet function to the variable my_func we did not use parentheses at the end. This is because we did not want python to call greet but simply store it in the my_func variable.
We can also pass functions to other functions as parameters.
Example:
def greet():
print("Hello, World!")
def execute_function(func):
print("Calling function")
func()
print("Called function")
execute_function(greet)
Output:
Calling function
Hello, World!
Called function
We can even return functions as a result from another function. Let’s see this in action.
Example:
def my_outer_function():
def my_return_function():
print("Hello, there!")
return my_return_function
my_func = my_outer_function()
my_func()
Output
Hello, there!
Closures
Closures are functions that capture or “close over” variables defined in the closures outer scope. Let’s take a look.
Example
def greet(greeting):
def greet_person(person):
print(f"{greeting}, {person}!")
return greet_person
my_greeting = greet("Hello")
my_greeting("Bob")
my_greeting("Janice")
Output
Hello, Bob!
Hello, Janice!
Here you can see that the greet_person function defined in the greet function “closes over” the greeting parameter. The resulting greet_person function is then assigned to the my_greeting variable and we can call it with different names.
One of the uses of closures is to define a partial function declarations. Just like before we can “close over” a value.
Example
def add_num(first_num):
def add(second_num):
return first_num + second_num
return add
add_ten_to = add_num(10)
add_five_to = add_num(5)
print(add_ten_to(12))
print(add_ten_to(6))
print(add_five_to(7))
print(add_five_to(9))
Output
22
16
12
14
Decorators
Time to take a look at decorators but, what exactly is a decorator? In its simplest form a decorator is a function that takes a function adds some functionality to it and returns a new function.
Decorating functions without Arguments
This is the simplest form of a decorator because we do not need to be concerned with function arguments. It’s important to note that because we are not dealing with function arguments in this simplest form the wrapped function can never accept function arguments.
Example:
def simple_decorator(func):
def wrapper():
print("Before func")
func()
print("After func")
return wrapper
def say_hello():
print("Hello!")
new_func = simple_decorator(say_hello)
new_func()
Output
Before func
Hello!
After func
We also do not really need to store these new functions to a variable. We can call them directly using Call Chaining.
Example:
def simple_decorator(func):
def wrapper():
print("Before func")
func()
print("After func")
return wrapper
def say_hello():
print("Hello!")
simple_decorator(say_hello)()
Output
Before func
Hello!
After func
The two previous examples are functionally the same.
We can streamline the process a bit using the Decorator Syntax.
Example
def simple_decorator(func):
def wrapper():
print("Before func")
func()
print("After func")
return wrapper
# This is functionally equivalent
@simple_decorator
def say_hello():
print("Hello")
say_hello()
# to this
def say_hello():
print("Hello")
simple_decorator(say_hello)()
# and to this
my_func = simple_decorator(say_hello)
my_func()
Decorating Functions with Arguments
This is all fine and dandy if we have functions with no arguments but, what if our functions that we need to decorator need to have arguments? All we need to do is capture those arguments in the wrapper function. Let’s take a look at that.
Example
from time import perf_counter_ns
def my_decorator(func): # 1
def wrapper(*args, **kwargs): # 2
print(f"Calling {func.__name__}:")
print(f"args: {args}")
print(f"kwargs: {kwargs}")
result = func(*args, **kwargs) # 3
print(f"returned {result}")
return result
return wrapper
@my_decorator # 4
def count(start, end):
start_time = perf_counter_ns()
for i in range(start, end+1):
print(i)
stop_time = perf_counter_ns()
return stop_time - start_time
count(1, 10)
Output
Calling count
args: (1, 10)
kwargs: {}
1
2
3
4
5
6
7
8
9
10
returned: 39333
So, what happened here?
- We defined our decorator that takes a function as a parameter
- In the decorator we defined a wrapper function that captures the arguments provided
- In the wrapper function we passed our parameters to the provided function
- We decorated our count function definition with our decorator
This is all functionally equivalent to:
Example
from time import perf_counter_ns
def my_decorator(func):
def wrapper(*args, **kwargs):
print(f"Calling {func.__name__}:")
print(f"args: {args}")
print(f"kwargs: {kwargs}")
result = func(*args, **kwargs)
print(f"returned {result}")
return result
return wrapper
def count(start, end):
start_time = perf_counter_ns()
for i in range(start, end+1):
print(i)
stop_time = perf_counter_ns()
return stop_time - start_time
my_decorator(count)(1, 10)
# or
my_new_func = my_decorator(count)
my_new_func(1, 10)
Output
Calling count:
args: (1, 10)
kwargs: {}
1
2
3
4
5
6
7
8
9
10
returned 38000
38000
A practical example would be moving our performance time to its own decorator so we can use it on multiple functions.
Example
from time import perf_counter_ns
def timer(func):
def wrapper(*args, **kwargs):
start_time = perf_counter_ns()
result = func(*args, **kwargs)
stop_time = perf_counter_ns()
total_time = stop_time - start_time
print(f"{func.__name__} took {total_time}ns to run")
return result
return wrapper
@timer
def count(start, end):
for i in range(start, end+1):
print(i)
count(1, 10)
Output
1
2
3
4
5
6
7
8
9
10
count took 91167ns to run
We can see now that our count function looks a lot cleaner and our timer code is a lot more portable. We can now use it to measure the run time of all kinds of functions by just adding it as a decorator to the function definition.
A real world example would be checking if a user is authenticated before executing a function
Example
class AuthenticationError(RuntimeError):
pass
def require_authenticated(func):
def wrapper(*args, **kwargs):
user = kwargs.get("user")
if user and user.get("is_authenticated"):
return func(*args, **kwargs)
raise AuthenticationError("User not authenticated")
return wrapper
@require_authenticated
def say_hello_to_user(user):
name = user.get("name")
print(f"Hello, {name}!")
authenticated_user = {"name": "Bob", "is_authenticated": True}
unauthenticated_user = {"is_authenticated": False}
say_hello_to_user(user=authenticated_user)
print("=========================================")
say_hello_to_user(user=unauthenticated_user)
Output
Hello, Bob!
=========================================
Traceback (most recent call last):
File "<python-input-39>", line 3, in <module>
say_hello_to_user(user=unauthenticated_user)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<python-input-34>", line 9, in wrapper
raise AuthenticationError("User not autheticated")
AuthenticationError: User not autheticated
Decorators That Accept Parameters
If we want to pass a parameter to our decorator like this…
Example
@allow_role("admin")
my_restricted_func():
# Does things here
we need to add another layer to our decorator.
Example
from dataclasses import dataclass
@dataclass
class User()
username: str
roles: list[str]
def check_args_for_user(*args, **kwargs):
if len(args) > 0 and type(args[0]) is User:
return args[0]
if type(kwargs.get("user")) == User:
return kwargs.get("user")
return None
def allow_role(role): # 1
def decorator(func): # 2
def wrapper(*args, **kwargs): # 3
user = check_args_for_user(*args, **kwargs)
if user and role in user.roles:
return func(*args, **kwargs)
raise RuntimeError("User not Authorized")
return wrapper
return decorator
@allow_role("admin") # 4
def process_admin_request(user):
print(f"{user.username} is logged in")
user = User(username="john", roles=["admin"])
process_admin_request(user) # 5
Output
john is logged in
In this example we have added another layer to our decorator. The outer layer of the decorator no longer returns the wrapper for our function but instead returns a decorator that returns a wrapper for our function.
You can see here:
- We define a function that returns a decorator
- This is our decorator that is going to return the wrapped function
- Here is our wrapped function that will be called at run time
- Here python calls our function thanks to the parentheses and gets the decorator for the function
- finally we call our wrapped function passing in our user object
Decorator Timing
It important to understand when our code is running when it comes to decorators.
At its simplest you can think of everything outside of the wrapper function runs
when the file is imported and ran immediately and everything that is in the
wrapper function runs when the function is called.
Example
def run_twice(func):
print("This will run when the file is imported")
def wrapper(*args, **kwargs):
print("This will run before the wrapped function is called")
result = func(*args, **kwargs)
result = func(*args, **kwargs)
print("This will run after the wrapped function is called")
print("This will also run when the file is imported")
return wrapper
@run_twice
def say_hello():
print("Hello")
print("============================================")
say_hello()
Output
This will run when the file is imported
This will also run when the file is imported
============================================
This will run before the wrapped function is called
Hello
Hello
This will run after the wrapped function is called
Here our two print statements that were defined outside the wrapper are executed when the function say_hello is defined and the print statement in the wrapper function are not executed until we call the say_hello function.
You can think of it working like this
def run_twice(func):
print("This will run when the file is imported")
def wrapper(*args, **kwargs):
print("This will run before the wrapped function is called")
result = func(*args, **kwargs)
result = func(*args, **kwargs)
print("This will run after the wrapped function is called")
print("This will also run when the file is imported")
return wrapper
my_decorator = run_twice
def say_hello():
print("Hello")
say_hello = my_decorator(say_hello)
print("============================================")
say_hello()
The out put will be the same as our previous example. What the decorator is doing behind the scenes with that @ symbol is that it runs our decorator function passing in our function and since our function returns a function called wrapper that gets rebound to the say_hello name. When we later call our say_hello function we are not actually calling say_hello but the wrapper returned from our run_twice decorator.
The same is true for decorators that take parameters with one small difference.
Example
def run_x_times(num):
print("This will run when the file is imported")
def decorator(func):
print("So will this")
def wrapper(*args, **kwargs):
print("This will run when the function is called")
results = []
for _ in range(1, num + 1):
results.append(func(*args, **kwargs))
return results
return wrapper
return decorator
@run_x_times(5)
def say_hello():
print("Hello")
print("========================================================")
say_hello()
Output
This will run when the file is imported
So will this
========================================================
This will run when the function is called
Hello
Hello
Hello
Hello
Hello
Just like in our first decorator example we can take a look at what python is doing under the hood by manually following the process ourselves.
Example
def run_x_times(num):
print("This will run when the file is imported")
def decorator(func):
print("So will this")
def wrapper(*args, **kwargs):
print("This will run when the function is called")
results = []
for _ in range(1, num + 1):
results.append(func(*args, **kwargs))
return results
return wrapper
return decorator
my_decorator = run_x_times(5)
def say_hello():
print("Hello")
say_hello = my_decorator(say_hello)
print("========================================================")
say_hello()
The output is exactly the same as our previous example using the @ syntax.
This time instead of assigning the decorator straight to the my_decorator
variable when ran the run_x_time function passing in a parameter of 5 and that
returned our decorator. Then after we defined our say_hello function we passed
that function to our decorator stored in the my_decorator variable and over
wrote what was in say_hello with our new function wrapper.
There is one last piece to the timing puzzle. What happens when we use multiple decorators on the same function? The decorators wrap those functions from the bottom to top and executes those wrappers from the top down. Lets take a look at it.
Example
def decorator_one(func):
print("I'm decorator one")
def wrapper(*args, **kwargs):
print("Running Wrapper One")
return func(*args, **kwargs)
return wrapper
def decorator_two(func):
print("I'm decorator two")
def wrapper(*args, **kwargs):
print("Running Wrapper Two")
return func(*args, **kwargs)
return wrapper
@decorator_one
@decorator_two
def say_hello():
print("Hello")
print("========================================================")
say_hello()
Output
I'm decorator two
I'm decorator one
========================================================
Running Wrapper One
Running Wrapper Two
Hello
From the output you can see in what order the decorators executed in. They wrapped the function from the bottom up, executing decorator_two first, and the wrapped functions from the top down, executing the wrapper function from decorator_two first. If we were not using the decorator’s @ syntax what would that look like?
def decorator_one(func):
print("I'm decorator one")
def wrapper_one(*args, **kwargs):
print("Running Wrapper One")
return func(*args, **kwargs)
return wrapper_one
def decorator_two(func):
print("I'm decorator two")
def wrapper_two(*args, **kwargs):
print("Running Wrapper Two")
return func(*args, **kwargs)
return wrapper_two
top_decorator = decorator_one
bottom_decorator = decorator_two
def say_hello():
print("Hello")
say_hello = bottom_decorator(say_hello)
say_hello = top_decorator(say_hello)
# say_hello is now
# wrapper_one -> wrapper_two -> say_hello
print("========================================================")
say_hello()
This has the exact same output as the previous example. This works exactly the same for decorators that take parameters.
Example
def wrap_function(layer):
def decorator(func):
print(f"I am the {layer} decorator")
def wrapper(*args, **kwargs):
print(f"Running the {layer} wrapper")
return func(*args, **kwargs)
return wrapper
return decorator
@wrap_function("top")
@wrap_function("middle")
@wrap_function("bottom")
def say_hello():
print("Hello")
# say_hello is now
# wrapper_top -> wrapper_middle -> wrapper_bottom -> say_hello
print("========================================================")
say_hello()
Output
I am the bottom decorator
I am the middle decorator
I am the top decorator
========================================================
Running the top wrapper
Running the middle wrapper
Running the bottom wrapper
Hello
Which like before we can deconstruct to see exactly how it is running without the decorator’s @ syntax
Example
def wrap_function(layer):
def decorator(func):
print(f"I am the {layer} decorator")
def wrapper(*args, **kwargs):
print(f"Running the {layer} wrapper")
return func(*args, **kwargs)
return wrapper
return decorator
top_wrapper = wrap_function("top")
middle_wrapper = wrap_function("middle")
bottom_wrapper = wrap_function("bottom")
def say_hello():
print("Hello")
say_hello = bottom_wrapper(say_hello)
say_hello = middle_wrapper(say_hello)
say_hello = top_wrapper(say_hello)
# say_hello is now
# wrapper_one -> wrapper_two -> say_hello
print("========================================================")
say_hello()
This example gives the exact same output as the previous example.
Now that timing is out of the way, there is one last “gotcha” when it comes to decorators, and that is metadata.
Metadata
When we wrap a function using a decorator we essentially overwrite all of that functions metadata. If you have anything that relies on metadata like documentation generators for example, it will not have the metadata it needs to generate accurate documentation. However, the python standard library comes to the rescue. In the functools package there is a decorator to be used in decorators called wraps that preserves our metadata for us. Let’s take a look at an example not using wraps first.
def no_wraps(func):
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
@no_wraps
def say_hello():
print("Hello")
print(say_hello.__name__)
Output
wrapper
As we can see that even though we expected the name of the say_hello function to be say_hello it had been changed to wrapper. That is because the function that is stored in the say_hello symbol is no longer the say_hello function but the wrapper function. This would cause our documentation generator to get confused because all the metadata information for say_hello has been overwritten by the wrapper. Let’s see how we can fix this using functools.
import functools
def with_wraps(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
@with_wraps
def say_hello():
print("Hello")
print(say_hello.__name__)
Output
say_hello
Awesome! Our function now returns the correct metadata. The functools’ wraps decorator takes the metadata from our original function and assigns it to the wrapper. Now our tools will not get confused.