numbers = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
squares_list = [x**2 for x in numbers]
squares_list[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]Pythonic Constructs
Python features are best leveraged when the code is written in a more “Pythonic” style. Pythonic code is concise, readable, and idiomatic, making it easier to understand and maintain.
Comprehensions
Comprehensions are concise, readable, yet powerful constructs for creating and/or transforming Python structures in an iterative way. A comprehension consists of brackets containing an expression followed by a for clause, then zero or more for or if clauses.
List Comprehensions
A list comprehension is a concise way to create a list. As an example, we can create a list with the squares of the numbers from 0 to 9 as follows:
Set Comprehensions
Analogously to list comprehensions, we can make a set of the squares from 0 to 9 by using a set comprehension:
numbers = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
squares_set = {x**2 for x in numbers}
squares_set{0, 1, 4, 9, 16, 25, 36, 49, 64, 81}Dictionary comprehensions
Similarly to the previous types of comprehensions, dictionary comprehensions are a more concise way to create dictionaries and iterate over them. Here we create a dictionary where the keys are numbers from 0 to 9 and the values are their squares:
numbers = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
squares_dict = {x: x**2 for x in numbers}
squares_dict{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}We can use the dict.items() method to iterate over keys and values in a very idiomatic way - as opposed to extracting each value while iterating over the keys - and use comprehensions to transform either of the two:
letters = {"a": "aa", "b": "bb", "c": "cc"}
upper_keys = {key.upper(): value for key, value in letters.items()}
print(f"{upper_keys=}")upper_keys={'A': 'aa', 'B': 'bb', 'C': 'cc'}upper_values = {key: value.upper() for key, value in letters.items()}
print(f"{upper_values=}")upper_values={'a': 'AA', 'b': 'BB', 'c': 'CC'}Note how the above is much more idiomatic than e.g. :
upper_values_by_key = {key: letters[key].upper() for key in letters.keys()}Comprehensions with multiple for clauses
Comprehensions can be characterized by multiple for clauses to perform nested iterations. In this subsection, we’ll look at lists, but comprehension like the following can be applied to other structures as well, like sets and dictionaries.
We apply multiple for clauses to iterate over lists within a single comprehension. In doing so, it’s crucial to understand the looping order and scope, which follows that of the nested for-loop clauses, namely from the outermost for clause to the innermost one.
Let’s first look at a generic example with a list of lists:
nested_list = [[1, 2, 3], [4, 5], [6, 7, 8, 9]]
[one_element for inner_list in nested_list for one_element in inner_list][1, 2, 3, 4, 5, 6, 7, 8, 9]This is equivalent to the following non-Pythonic implementation:
nested_list = [[1, 2, 3], [4, 5], [6, 7, 8, 9]]
flattened_list = []
for inner_list in nested_list:
for one_element in inner_list:
flattened_list.append(one_element)Note that you can also use “nested comprehensions” to e.g. create nested lists:
nested_list = [[x**y for x in range(3)] for y in range(3)]
nested_list[[1, 1, 1], [0, 1, 2], [0, 1, 4]]Let’s now look at an example with a 3x3 matrix-like structure, also represented as a list of lists (although in real case scenarios, matrices as mathematical objects are better implemented with dedicated libraries):
matrix = [
[1, 2, 3],
[4, 5, 6],
[7, 8, 9],
]Similarly here below, we use multiple for to created a list from a list of lists. When dealing with matrices, this corresponds to the so-called flattening operation. Here, we consider each nested list as a row of the matrix:
flattened = [num for row in matrix for num in row]
flattened[1, 2, 3, 4, 5, 6, 7, 8, 9]What if we want to allow only certains elements to be included in the list based on a certain condition? We can keep the Pythonic syntax and add an if clause. We show the latter by creating a list from our 3x3 matrix, but only including the squares of the even numbers:
matrix_even_squares = [number**2 for row in matrix for number in row if number % 2 == 0]
matrix_even_squares[4, 16, 36, 64]Let’s look at another example with 3 for clauses. We start by creating a 3D matrix-like structure. Without going into much detail about 3d-matrices, we can imagine it as a cube, where each “slice” is a 2-dimensional matrix:
matrix_3d = [
# first slice
[
[1, 2, 3],
[4, 5, 6],
[7, 8, 9],
],
# second slice
[
[10, 11, 12],
[13, 14, 15],
[16, 17, 18],
],
]This means that a complete walkthrough of the matrix using nested forloops consists of an outer for iterating over slices, an nested for iterating over rows, and the innermost for iterating over the elements of each row. Similarly to what shown before, the more Pythonic way using comprehensions looks like the following:
flattened_3d = [num for slice in matrix_3d for row in slice for num in row]
print(flattened_3d)[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]Another useful way to leverage multiple for clauses is to create a Cartesian product of two lists, i.e. all the possible pairs of elements from the two lists:
numbers = [1, 2, 3]
letters = ['a', 'b', 'c']
cartesian_product = [(number, letter) for number in numbers for letter in letters]
print(cartesian_product)[(1, 'a'), (1, 'b'), (1, 'c'), (2, 'a'), (2, 'b'), (2, 'c'), (3, 'a'), (3, 'b'), (3, 'c')]Generator Expressions
Quoting the official docs:
A generator expression is an expression that returns an iterator.
A generator expression takes the form f(x) for x in <range>, e.g. x*x for x in range(10), where the first element can also include tuples, as in (k, func(k)) for k in keylist. They are especially handy with functions working with iterators, like sum(), min(), and max(). A key aspect of generator expressions is that they evaluate elements one at a time, which brings benefits in terms of memory footprint and allows for “lazy” evaluation of certain operations.
Let’s consider a simple example where we want to sum the squares of the first 10 nonnegative integers. Knowing about list comprehensions, a simple approach is the following:
sum_of_squares = sum([x * x for x in range(10)])
sum_of_squares285With this code, the full list of all squares is evaluated and built in memory first with the comprehension [x*x for x in range(10)], and then passed as input to sum(). The same objective can be attained by feeding sum() directly with the generator expression x*x for x in range(10):
sum_of_squares = sum(x*x for x in range(10))
sum_of_squares285Since computing the sum can simply evaluate and add one element at a time, no list is created and memory is conserved in the process. In larger scenarios, the iterative nature of generator expressions can have a great impact on the memory footprint.
Generator Expressions with multiple for clauses
Generator expressions can be used with multiple for clauses as well, in a way similar to list comprehensions:
nested_list = [[1, 2, 3], [4, 5], [6, 7, 8, 9]]
sum_nested = sum(x for inner_list in nested_list for x in inner_list)
sum_nested45Conditions Evaluation with any and all
any() and all() are used to check conditions over iterables. Like sum, they can combine with generator expressions. Furthermore, any() and all() employ lazy evaluation. any() stops evaluating elements of an iterable when it evaluates to True for the first time. all() short-circuits when it evaluates to the first False.
numbers = [22, 35, 24, 83, 18]
if any(n % 7 == 0 for n in numbers):
print("There's a multiple of 7") There's a multiple of 7any() only iterates till the 2nd element (35)
numbers = [21, 35, 24, 83, 18]
if all(n % 7 == 0 for n in numbers):
print("All numbers are multiples of 7") all() stops iterating after the 3d element (24)
next with Generator Expressions
next(iterator, default) retrieves the next item from an iterator. Combining next() with generator expressions makes for a very concise syntax. It can also be used to e.g. get the first element satisfying a condition, or a fallback value if no element exists.
next() will evaluate each value, one at a time from an iterator, in this case the one returned by the generator expression.
The following example prints the square of the first multiple of 5:
numbers = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
print(next(x**2 for x in numbers if x % 5 == 0))0If the iterator has no elements, a StopIteration exception is raised:
next(x**2 for x in range(1, 5) if x % 5 == 0)--------------------------------------------------------------------------- StopIteration Traceback (most recent call last) Cell In[20], line 1 ----> 1 next(x**2 for x in range(1, 5) if x % 5 == 0) StopIteration:
However, a default value, like None, can be passed to next to be returned in case of no elements, thus avoiding the exception:
print(next((x**2 for x in range(1, 5) if x % 5 == 0), None))NoneWhen using next() with a generator expression and a default value, the generator expression must be parenthesized to be a valid Python syntax.
Tuples and Unpacking
Tuples are immutable sequences used to store collections of items.
Unpacking is a concise syntax to assign multiple members of a tuple at once, useful when e.g. the various elements have a special meaning. The amount of assigned variables on the left-hand side must match the number of tuple elements.
coords = (1.0, 2.0, 3.0)
x, y, z = coords
print(f"{x=}, {y=}, {z=}")x=1.0, y=2.0, z=3.0Note how assigning x, y, z this way is more idiomatic than using coords[0], coords[1], coords[2]. If the number of unpacked elements does not match, and exception is raised:
coords = (1.0, 2.0, 3.0, 4.0)
x, y, z = coords--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[23], line 2 1 coords = (1.0, 2.0, 3.0, 4.0) ----> 2 x, y, z = coords ValueError: too many values to unpack (expected 3)
Comprehensions for list of tuples
Combining list comprehension and tuple unpacking can make for more expressive syntax when looping over tuple members:
tuples = [(2, 3), (3, 4), (4, 5), (5, 6)]
[base**exp for base, exp in tuples][8, 81, 1024, 15625]Note how this is much more idiomatic and expressive than:
[x[0]**x[1] for x in tuples][8, 81, 1024, 15625]The Unpacking Operator
The unpacking operator * allows us to unpack iterables into individual elements:
# Basic unpacking of a list
numbers = [1, 2, 3]
print(*numbers) # equivalent to print(1, 2, 3)
# Unpacking of a tuple
colors = ('red', 'green', 'blue')
print(*colors) # equivalent to print('red', 'green', 'blue')
# Unpacking of a dictionary
person = {'name': 'John', 'age': 30, 'city': 'New York'}
print(*person) # equivalent to print('name', 'age', 'city')1 2 3
red green blue
name age city* can also be used to conveniently capture multiple elements as a list during the tuple/list unpacking with left and right side variables having different lengths. In the example below, the * operator allows one variable (reptiles in this case) to capture multiple remaining values. In this scenario, the variable with * will always receive a list, even if it is unpacking a tuple.
animals = ('dog', 'chicken', 'snake', 'lizard')
(mammal, bird, *reptiles) = animals
print(f"{mammal=}, {bird=}, {reptiles=}")mammal='dog', bird='chicken', reptiles=['snake', 'lizard']zip() and Unzipping
zip() can be conveniently used to iterate on matching elements of multiple sequences.
names = ['Dave', 'Richard', 'Martin']
ages = [25, 30, 35]
for name, age in zip(names, ages):
print(f"{name} is {age} years old")Dave is 25 years old
Richard is 30 years old
Martin is 35 years oldThis way, we spare explicit index lookups in the equivalent non-Pythonic implementation:
names = ['Dave', 'Richard', 'Martin']
ages = [25, 30, 35]
for i in range(len(names)):
print(f"{names[i]} is {ages[i]} years old")zip() can be used with the unpacking operator * to distribute each nth element of tuples from a list into separate tuples.
coordinates = [(1.0, 10.0), (2.0, 20.0), (3.0, 30.0)]
x, y = zip(*coordinates)
print(f"x: {x}\ny: {y}")x: (1.0, 2.0, 3.0)
y: (10.0, 20.0, 30.0)Once more, the code is much more idiomatic and expressive than
x = tuple([coord[0] for coord in coordinates])
y = tuple([coord[1] for coord in coordinates])zip() in conjunction with the * operator can be used to unzip a list:
a = [1, 2, 3]
b = ['a', 'b', 'c']
a2, b2 = zip(*zip(a, b))
print(f"{a2=}, {b2=}")a2=(1, 2, 3), b2=('a', 'b', 'c')Iterating with enumerate
Using enumerate() allows to iterate over elements and their corresponding index.
for index, value in enumerate(['a', 'b', 'c']):
print(f"Index: {index}, Value: {value}")Index: 0, Value: a
Index: 1, Value: b
Index: 2, Value: cBy default, the index first value is 0, but this can be changed using the start argument.
for index, value in enumerate(['a', 'b', 'c'], start=1):
print(f"Counter: {index}, Value: {value}")Counter: 1, Value: a
Counter: 2, Value: b
Counter: 3, Value: cSimilarly to zip(), the use of enumerate() saves us from explicit index lookups:
values = ['a', 'b', 'c']
for i in range(len(values)):
print(f"Index: {i}, Value: {values[i]}")Index: 0, Value: a
Index: 1, Value: b
Index: 2, Value: cList Comprehensions with enumerate
The following examples combine enumerate and list comprehension to raise a list of values to powers incrementally larger. Let’s start with showing the more Pythonic way of iteration over (index, value). It combines list comprehension and enumerate:
values = [4, 9, 3, 6]
powers = [val**i for i, val in enumerate(values)]
powers[1, 9, 9, 216]Here below we show the less Pythonic way of iteration over (index, value) using index lookups:
powers = [values[i]**i for i in range(len(values))]
powers[1, 9, 9, 216]Finally, we show the non-Pythonic equivalent with neither enumerate nor list comprehension:
powers = []
for i in range(len(values)):
powers.append(values[i]**i)
powers[1, 9, 9, 216]