Decorators and Generators

Decorators

Decorators are particularly useful for code reusability and separation of concern, i.e. keeping the core functions separated from auxiliary functionalities. We can imagine decorators are “plug-ins” to attach to functions.

Technically, decorators are what’s defined as higher order functions (HOFs), meaning functions taking other functions as input and/or returning a function. Examples of HOFs are map() - which takes an input function to apply over each of the elements of a sequence, filter() - whose input function verifies a pass/fail type of condition over each element of a sequence, apply(), sorting functions, callbacks and more.

Decorators are in fact a specific type of higher-order functions (HOFs), in that they take a function as input and return a function, allowing one to extend or alter the behavior of the input function.

def my_decorator(function):
    def wrapper():
        # Do something before
        print("Before")
        result = function()
        # Do something after
        print("After")
        return result
    return wrapper

Although decorator functions can be called explicitly to wrap an existing function:

def greet():
    print("Hi")

greet_wrapped = my_decorator(greet)

greet_wrapped()

Before
Hi
After

They shine when used when used to decorate the function via @ when it is defined, which is equivalent to the explicit wrapping above:

@my_decorator
def greet():
    print("Hi")

greet()

Before
Hi
After

Decorators with Parameters to the Decorated Function

Naturally, the function that’s decorated can also accept parameters, which will be passed down by the decorating function.
The decorator syntax (@) provides a concise way to apply decorators to functions.

def run_with_description(func):
    def wrapper(*args, **kwargs):
        print("Running pre function call tasks..")
        print(
            f'Calling "{func.__name__}" with {len(args)} arguments: {args} \n'
            f'and {len(kwargs)} keyword arguments: {kwargs}'
        )
        res = func(*args, **kwargs)
        print("Running post function call tasks..")
        print(f'"{func.__name__}" is returning "{res}" of type {type(res)}')
        return res

    return wrapper

Let’s decorate a simple function to see how it works:

@run_with_description
def add(a, b):
    return a + b

add(3, 5)

Running pre function call tasks..
Calling "add" with 2 arguments: (3, 5) 
and 0 keyword arguments: {}
Running post function call tasks..
"add" is returning "8" of type <class 'int'>

8

add("Hello", " Miraiers!")

Running pre function call tasks..
Calling "add" with 2 arguments: ('Hello', ' Miraiers!') 
and 0 keyword arguments: {}
Running post function call tasks..
"add" is returning "Hello Miraiers!" of type <class 'str'>

'Hello Miraiers!'

Let’s decorate a slightly more complex function which also accepts optional parameters:

@run_with_description
def greet(greeting, name, punctuation="", start_convo=False):
    return (
        f"{greeting}, {name}{punctuation}{' How are you doing?' if start_convo else ''}"
    )

greet("Good Morning", "Sir")

Running pre function call tasks..
Calling "greet" with 2 arguments: ('Good Morning', 'Sir') 
and 0 keyword arguments: {}
Running post function call tasks..
"greet" is returning "Good Morning, Sir" of type <class 'str'>

'Good Morning, Sir'

greet("Hey", "Buddy", start_convo=True, punctuation="!")

Running pre function call tasks..
Calling "greet" with 2 arguments: ('Hey', 'Buddy') 
and 2 keyword arguments: {'start_convo': True, 'punctuation': '!'}
Running post function call tasks..
"greet" is returning "Hey, Buddy! How are you doing?" of type <class 'str'>

'Hey, Buddy! How are you doing?'

Decorators with Parameters to the Decorator and the Decorated Function

In some cases, the decorated function and the decorator function have parameters of their own:

import random as rand

def repeat(times):
    def inner(func):
        def wrapper(*args, **kwargs):
            for _ in range(times):
                res = func(*args, **kwargs)
                print(f"\"{func.__name__}\" function - run nr. {_} result: {res}")
            return res
        return wrapper
    return inner

@repeat(3)
def add(a, b):
    return a + b

@repeat(3)
def add_random(range):
    a, b = rand.randrange(range), rand.randrange(range) # randrange returns a randomly selected element from 0 to `range` excluded
    print(f"a: {a}, b: {b}")
    return a + b

add(2, 5) # equivalent to repeat(3)(add)(2,5), where `add` is the original f. prior to being decorated

"add" function - run nr. 0 result: 7
"add" function - run nr. 1 result: 7
"add" function - run nr. 2 result: 7

7

add_random(10)

a: 9, b: 8
"add_random" function - run nr. 0 result: 17
a: 0, b: 3
"add_random" function - run nr. 1 result: 3
a: 0, b: 7
"add_random" function - run nr. 2 result: 7

7

Generators

Generators are special type of iterators that define how to yield one element at a time, holding a state that can be used to yield the value for the next element based on the previous ones. A generator can be defined as follows:

def my_generator():
    yield "Python"
    yield "Training"
    yield "Materials"

The yield keyword returns a value and pauses the generator function’s execution until the following value is requested. We can request each value either via standard for iteration or via next():

for value in my_generator():
    print(value)

Python
Training
Materials

generator = my_generator()
print(next(generator)) 
print(next(generator))
print(next(generator))

Python
Training
Materials

After each invocation, the generator’s state freezes up. That is to say, local variable bindings, the instruction pointer, and the evaluation stack are made available to the following call to next(). This means one can hold a state between generating values. Calling next() beyond their last yield will trigger a StopIteration exception.

When the next element is requested from a generator (either via next() or via explicit for iteration), the code in the body of the generator-function is executed until a yield or return statement is encountered.

A core aspect of generators is the the separation between the logic for building the element of the n-th iteration (based on the (n-1)th element) and the way the generated values are utilized. Holding a state and using intermediate states to generate sequences iteratively allows to separate the logic for generating a sequence from the logic that makes use of the values (even in contexts when it is not known a priori when to stop).

To exemplify that, we show how we can define the Fibonacci sequence using a generator, and use it to sum the first 20 elements. The generator fibonacci() defines the logic, by updating the last two generated elements a, b as intermediate state. The call to the generator function is then given to sum() as input.

def fibonacci(n):
    a, b = 0, 1
    for _ in range(n):
        yield b
        a, b = b, a + b

print(f"{sum(fibonacci(20)) = }")

sum(fibonacci(20)) = 17710

Generators can be easily converted to lists, but also used in comprehensions or generator expressions:

print(f"{list(fibonacci(20))}")

[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765]

print([_ for _ in fibonacci(20) if _% 2 == 0])

[2, 8, 34, 144, 610, 2584]

print(sum(_ for _ in fibonacci(20) if _% 2 == 0))

3382