Chapter 6 Define and Use Functions

In programming or software development, program codes must be well structured to be manageable. Some program codes can be reused to make programming and software development more efficient. Functions and modules serve these two goals. Chapter 6 shows you how to define and use functions in Python and how to make and use modules in programming.

Learning Objectives

After completing this chapter, you should be able to

• explain what functions are in Python.
• define new functions correctly.
• use functions, including both built-in and programmer-defined functions.
• use return statements properly to return various values from a function.
• use positional arguments in defining and using functions.
• use variable-length lists of arguments in defining and using functions.
• use keyed arguments in defining and using functions.
• use positional arguments, keyed arguments, and variable-length lists of arguments.
• explain what recursive functions are and how they work.
• define and use recursive functions.
• explain what anonymous/lambda functions are.
• define and use anonymous/lambda functions.
• use special functions such as mapping, filtering, and reducing.
• explain what generators are and what advantages they have.
• define a function as a generator.
• explain what closures and decorators are and how they are used.
• define and use closures and decorators.
• describe the properties of functions and use them properly.

6.1 Defining and Using Functions in Python

You have already seen and used some built-in functions in the previous chapters. These built-in functions are built into Python Virtual Machine (PVM) in its standard distribution so that you can use them without importing any Python modules. For example, the built-in function sum can be used directly to calculate the sum of a sequence of numbers, as shown in the following code sample:

>>> sum([12, 23, 25, 65, 52])

177

As you will see, many Python modules, available via either Python distribution or a third party, have also defined functions ready for you to use in your programs.

For functions defined in a standard or third-party module, you must import the module before you can call the function by using the dot notation or operator, such as m.f(…), where m is a name referring to the module, and f is the name of the function to be used. Within the pair of parentheses are data as arguments of the function call, to be passed to the function for processing. The following is an example calling the pow() function from the math module:

>>> import math

>>> math.pow(35,12)

3.3792205080566405e+18

Note that when calling a function, a pair of parentheses must be attached to the name of the function, even if there is no argument to pass. Otherwise, the name of the function will be evaluated as a first-class object, and the type of the object and the name will be returned, as shown in the following example:

>>> print(sum)

<built-in function sum>

>>> print(id)

<built-in function id>

Although many modules have been developed by others and many functions have been made available, programmers do need to define their own functions for their specific purposes.

To define a function in Python, the def compound statement is used. The general syntax is as follows:

def <function name>(parameters):

Where function name is a legitimate identifier in the local scope, parameters are legitimate variable names that can be used to pass values (arguments) to the function, and a code block (function body, in this compound statement) is the real program code that does the computing or information processing. What makes a code block in a function definition different from code blocks in other compound statements such as for, while, and if is that it will always return a value with the return statement. Even if you do not have anything to return from a function definition and do not have a return statement, special value None will still be automatically returned from the function.

The following is a real example showing how a function is defined in Python:

In [ ]:

def factorial(n):

"""This function calculates and returns n!, the factorial of an integer n > 0."""

if (not isinstance (n, int)) or (n < 1):

return None

r = 1

for i in range(n):

r *= (i + 1)

return r

As documented in the docstring, the function is to calculate and return the factorial of an integer if the number is greater than 0; otherwise, it returns None.

The function you defined can be used in the same way as built-in functions. The following example shows how to call the factorial function defined above:

In [ ]:

N = 16

fn = factorial(N)

print(f'factorial of {N} is {fn}.')

Out [ ]:

factorial of 16 is 20922789888000.

Sometimes you need to return more than one value from a function. To do that, you can either put the values in a compound data type such as a list, tuple, set, or dictionary, or just put the value all behind return. In the latter case, the values will be automatically packed in a tuple by the return statement.

The following example calculates and returns both the quotient and remainder of two integers at the same time.

In [ ]:

def idivmod(n, m):

if (not isinstance(n, int)) or (not isinstance(n, int)) or (m == 0):

return None

return n // m, n % m

n, m = 23, 5

print(f'The value returned from idivmod({n}, {m}) is {idivmod(n, m)}.')

Out [ ]:

The value returned from idivmod(23, 5) is (4, 3).

In the remainder of this section, we show how to program to solve slightly more complicated problems with Python.

A perfect number is an integer that equals the sum of all its factors excluding the number itself. For example, 6 is a perfect number because 6 = 1 + 2 + 3. So is 28. It sounds simple, but the next perfect number is very far from 28: 496. A program for finding perfect numbers is shown in Table 6-1.

Table 6-1: Case study: How to find perfect numbers
The problem	In this case study, we are going to write a program to ask for a big integer from the user, then find all the perfect numbers smaller than the big integer.
The analysis and design	Step 1. Take an input from user, and convert it into an integer Step 2. Loop from 2 to the big integer a. Test each integer to see if it is a perfect number b. If yes, print the number and all its factors Step 3. Finish Steps to check if a number is a perfect number: Step 4. Find all its factors, including 1 but excluding the number itself, and put them into a list Step 5. Sum up the factors with sum(factors) Step 6. If the number == sum(factors), then return True.
The code	`"""` `This program is used to find all the perfect numbers that are less than N given by a user.` `"""` `def perfect(n):` `factor = [1]` `# create a list with 1 as the first factor` `for j in range(2, (n // 2) + 1):` `# only need loop to n // 2 + 1` `if n % j == 0:` `# if j is a factor` `factor.append(j)` `# add j to the list` `if n == sum(factor):` `# if the sum of the factors = n` `return [True, factor]` `# return True as well as factors` `else:` `return [False, []]` `upper_bound = int(input("Tell me the upper bound:"))` `for i in range(2, upper_bound):` `test = perfect(i)` `if test[0]:` `print(f"{i} = {test[1]}")`
The result	Tell me the upper bound:32198765 6 = [1, 2, 3] 28 = [1, 2, 4, 7, 14] 496 = [1, 2, 4, 8, 16, 31, 62, 124, 248] 8128 = [1, 2, 4, 8, 16, 32, 64, 127, 254, 508, 1016, 2032, 4064]

6.2 Parameters and Arguments in Functions

When you define a function, you can use variables within the parentheses right next to the function name to specify what values can be taken when the function is called. These variables within the parentheses are called parameters, and the values to be passed to the parameters in a function call are called arguments.

In Python, a function call may take positional arguments, keyword arguments, variable-length lists of nonkeyword arguments, variable-length lists of keyword arguments, and default arguments. These are determined in the definition of the function.

In a function definition, a simple variable name can be used for a parameter expecting a positional argument or keyword argument, to give a parameter, such as x, a default V using assignment x = V. When a parameter is given a default value, the default value will be used when no value is given to the parameter in a function call.

To indicate that parameter y will hold a variable-length list of nonkeyword arguments, use *y; to indicate that parameter z will take variable-length of keyword arguments, use **z. Keyword arguments will be explained shortly.

When calling a function, positional arguments are the arguments passed to their respective parameters in accordance with their positions. That is, the first parameter in a function call is passed to the first argument in the definition of the function, the second parameter is passed to the second argument, and so on. This is illustrated in the example below:

In [ ]:

def func_demo1(a, b, c):

print(f'a = {a}, b = {b}, c = {c}')

func_demo1(1, 2, 3)

Out [ ]:

a = 1, b = 2, c = 3

This shows that the first argument was passed to the first parameter a, the second argument was passed to the second parameter b, and the third argument was passed to the third parameter c.

When calling a function, its parameter name, such as x, can be used as a keyword to explicitly indicate that a specific value, such as v, will be passed to x. This is done using x = v syntax, called a keyword argument in a function call. The following example shows how keyword arguments are used.

In [ ]:

def func_demo1(a, b, c):

print(f'a = {a}, b = {b}, c = {c}')

func_demo1(b =1, a = 2, c = 3)

Out [ ]:

a = 2, b = 1, c = 3

In this case, the order of the arguments does not matter because the code explicitly indicates which argument is given to which parameter. Please note that in a function call when keyword argument is used, no more positional arguments, except variable-length nonkeyword arguments, may follow. An error will occur otherwise, as shown below:

In [ ]:

def func_demo1(a, b, c):

print(f'a = {a}, b = {b}, c = {c}')

func_demo1(b=1, 2, 3)

Out [ ]:

File "<ipython-input-51-6539f4d878e5>", line4

func_demo1(b = 1, 2, 3)

SyntaxError: positional argument follows keyword argument

Also, in a function definition, parameters expected to be used as keywords must be placed behind those expecting positional arguments. An error will occur otherwise, as shown in the following example:

In [ ]:

def func_demo1(a, b, c):

print(f'a = {a}, b = {b}, c = {c}')

func_demo1(3, 1, a = 2)

Out [ ]:

----------------------------------

TypeError Traceback (most recent call last)

<ipython-input-185-f540d2127799> in <module>

2 print(f'a = {a}, b = {b}, c = {c}')

3 ---->

4 func_demo1(3, 1, a = 2)

TypeError : func_demo1() got multiple values for argument 'a'

When a parameter in a function definition has a default value, the argument for the parameter can be omitted if the default value is to be used. The function defined in the following example will calculate the square of number x by default, but it can also calculate x power of y by passing a particular value to y:

In [ ]:

def powerof(x, y = 2):

return f'{x} ** {y} = {x ** y}'

print(powerof(12))

print(powerof(23, 5))

print(powerof(13, y = 6))

print(powerof(y = 21, x = 3)) # with keyword arguments, the order doesn't matter

Out [ ]:

12 ** 2 = 144 23 ** 5 = 6436343 13 ** 6 = 4826809 3 ** 21 = 10460353203

The following example demonstrates how to define a function that can take variable-length nonkeyword arguments. The function is to calculate the product of a series of numbers:

In [ ]:

def product(*n):

s = 1

for i in n:

s *= i

return f'Product of all numbers in {n} is {s}'

print(product(1,2,5,32,67))

print(product(11,32,25,3,7))

print(product(19,12,15,322,6))

Out [ ]:

Product of all numbers in (1, 2, 5, 32, 67) is 21440

Product of all numbers in (11, 32, 25, 3, 7) is 184800

Product of all numbers in (19, 12, 15, 322, 6) is 6607440

As can be seen, using variable-length nonkeyword positional arguments has made the function more powerful.

Sometimes in a function call, you may also want to use variable-length keyword arguments. The following example shows how this can be done:

In [ ]:

def reporting(**kwargs):

for k, v in kwargs.items():

print(f'{k}:{v}')

reporting(First_name ='John', Last_name ='Doe', Sex ='Male')

reporting()

Out [ ]:

First_name:John

Last_name:Doe

Sex:Male

When calling a function with variable-length keyword arguments, the keywords look like parameter names to which the values are passed. However, in the case of variable-length keyword arguments, these keywords cannot be used as variables inside the function definition because they are not in the parameter list when the function is defined.

While variable-length nonkeyword arguments are passed to a function as a tuple, variable-length keyword arguments are passed to the function as a dictionary. As such, what can be achieved by using variable-length keyword arguments can also be achieved by passing a dictionary instead. For example, the above example can be easily rewritten as follows:

In [ ]:

def dict_reporting(ps): # kw is a dictionary

for k, v in ps.items():

print(f'{k}:{v}')

pdict = {'First_name':'John', 'Last_name':'Doe', 'Sex':'Male'}

dict_reporting(pdict)

Out [ ]:

First_name:John

Last_name:Doe

Sex:Male

Similarly, functions taking variable-length nonkeyword arguments can be easily rewritten to take a tuple. So the example of list_product function can be rewritten as follows:

In [ ]:

def product_tuple(nt):

s = 1

for i in nt:

s *= i

return f'Product of all numbers in {nt} is {s}'

print(product_tuple((1,2,5,32,67)))

print(product_tuple((11,32,25,3,7)))

print(product_tuple((19,12,15,322,6)))

Out [ ]:

Product of all numbers in (1, 2, 5, 32, 67) is 21440

Product of all numbers in (11, 32, 25, 3, 7) is 184800

Product of all numbers in (19, 12, 15, 322, 6) is 6607440

The difference between using variable-length arguments and passing a tuple or dictionary is that, because the length of variable-length arguments can be 0, you do not have to pass any argument at all if you don’t have one. With a tuple or dictionary, on the other hand, you would have to have a legitimate argument for the corresponding parameter in the function definition unless you have set a default value to it. Also, as can be seen from the two examples above, in some applications, using variable-length arguments is more natural and elegant.

6.3 Recursive Functions

A function is recursive if it calls itself either directly or indirectly. Recursion is a powerful concept in computing and computational theory. In computational theory, it has been proven that any problem computable by modern computers can be represented as a recursive function.

In programming, recursive functions do not make your programs run fast. However, they do provide a powerful means of algorithm design to solve a problem and make neater program code.

Take the factorial function n! as an example. You know that n! is defined as 1 * 2 * 3 * … * n. If you use fac(n) to refer to the factorial function, you cannot simply define the function in Python as

def fac(n):

return 1 * 2 * 3 * … * n # this does not work in Python

because the dots (…) do not make sense to computers in this context. With what you have learned so far, the function can be defined as

def fac(n):

if n == 0:

return 1

product = 1

for i in range(n): # using loop

product *= (i + 1)

return product

The function above has seven lines of code.

Since you know that 0! = 1, and n! can be computed as n * (n − 1)! if n > 0, you can program a recursive function in Python to solve the factorial problem, as shown in the case study in Table 6-2.

Table 6-2: Case study: How to use a recursive function for calculating factorial n
The problem	Define a recursive function to calculate factorial n as n!
The analysis and design	We know that 0! = 1, and n! can be computed as n * (n − 1)! when n > 0. So if we use fac(n) to denote n!, this will be translated as fac(n) = 1 if n = 0 and as fac(n)= n * fac(n − 1) if n > 0. Accordingly, we can define a recursive factorial function in Python as follows:
The code	`def fac(n):` `if n == 0:` `return 1` `else:` `return n * fac(n - 1)` `n = 9` `print(f"{n}! = {fac(n)}")`
The result	9! = 362880

As you can see, the function has become shorter and neater, although it often takes more memory and more time to run.

The next case study, in Table 6-3, shows how a recursive function can be used to find the greatest common divisor of two integers.

Table 6-3: Case study: How to use a recursive function
The problem	This simple problem aims to find the greatest common divisor (GCD) for two given integers. A common divisor of two integers is an integer that can divide both integers, and the GCD is the biggest one among the common divisors. For example, 1 and 2 are common divisors of 4 and 6, and 2 is the GCD of 4 and 6.
The analysis and design	At first glance, a straightforward approach is to find all the divisors for each integer,
The code	`"""` `Ask for two integers and find the GCD of the two using the improved Euclidean algorithm.` `"""` `def my_gcd(a, b):` `global depth` `if b == 0:` `# condition to finish` `return b` `else:` `b, a = sorted((abs(a - b), b))` `# sort and reassign` `depth += 1` `# recursion depth increased by 1` `print(f"recursion` `#{depth} for{(a, b)}")` `return my_gcd(a, b)` `i = int(input("Tell me the first integer:"))` `j = int(input("Tell me the second integer:"))` `if i < j:` `i, j = j, i` `depth = 0` `print(f"The greatest common divisor of {i} and {j} is {my_gcd(i, j)} after {depth} recursions.")`
The result	Tell me the first integer:3238 Tell me the second integer:326 recursion #1 for (2912, 326) recursion #2 for (2586, 326) recursion #3 for (2260, 326) recursion #4 for (1934, 326) recursion #5 for (1608, 326) recursion #6 for (1282, 326) recursion #7 for (956, 326) recursion #8 for (630, 326) recursion #9 for (326, 304) recursion #10 for (304, 22) recursion #11 for (282, 22) recursion #12 for (260, 22) recursion #13 for (238, 22) recursion #14 for (216, 22) recursion #15 for (194, 22) recursion #16 for (172, 22) recursion #17 for (150, 22) recursion #18 for (128, 22) recursion #19 for (106, 22) recursion #20 for (84, 22) recursion #21 for (62, 22) recursion #22 for (40, 22) recursion #23 for (22, 18) recursion #24 for (18, 4) recursion #25 for (14, 4) recursion #26 for (10, 4) recursion #27 for (6, 4) recursion #28 for (4, 2) recursion #29 for (2, 2) recursion #30 for (2, 0) The greatest common divisor of 3238 and 326 is 2 after 30 recursions.

As noted, the program took 30 recursions to find the GCD of 3238 and 326. Can we make it more efficient? The answer is yes, and it is fun to design and code a better and faster program to solve a problem, as shown in the case study of this same problem in Table 6-4.

Table 6-4: Case study: How to use a recursive function—revised
The problem	This simple problem aims to find the greatest common divisor (GCD) for two given integers. A common divisor of two integers is an integer that can divide both integers, and the GCD is the biggest one among the common divisors. For example, 1 and 2 are common divisors of 4 and 6, and 2 is the GCD of 4 and 6.
The analysis and design	The original Euclidean algorithm is great because it only needs subtraction to find out the greatest common divisor, but sometimes it will involve too many steps, especially if we do the calculation manually. For example, to find the greatest common divisor of 4 and 40000, one needs to complete 10000 subtractions to find out that 4 is the GCD. An obvious and straightforward improvement to the algorithm is to use modular operation in place of subtraction.
The code	`"""` `Ask for two integers and find the GCD of the two using the improved Euclidean algorithm.` `"""` `def my_gcd(a, b):` `global depth` `if b == 0:` `# condition to exit from recursion` `return a` `else:` `b, a = sorted(((a % b), b))` `# sort and reassign` `depth += 1` `# recursion depth increased by 1` `print(f"recursion` `# {depth} for {(a, b)}")` `return my_gcd(a, b)` `i = int(input("Tell me the first integer:"))` `j = int(input("Tell me the second integer:"))` `if i < j:` `i, j = j, i` `depth = 0` `print(f"The greatest common divisor of {i} and {j} is {my_gcd(i, j)} after {depth} recursions.")`
The result	Tell me the first integer:3238 Tell me the second integer:326 recursion #1 for (326, 304) recursion #2 for (304, 22) recursion #3 for (22, 18) recursion #4 for (18, 4) recursion #5 for (4, 2) recursion #6 for (2, 0) The greatest common divisor of 3238 and 326 is 2 after 6 recursions.

As you can see, for the same numbers, the improved algorithm took only six recursions, whereas the original took 30. Another benefit of using the improved algorithm is that because Python Virtual Machine (PVM) has a limit on the maximum depth of recursions due to the limitations of computer memory, the improved algorithm will be able to handle much bigger numbers than the original algorithm. You may run the two programs on 24230336504090 and 356879542 to see the difference between the two algorithms.

6.4 Anonymous Functions: lambda Expressions

In previous sections, you saw functions with names, which make it possible to call a function by its name. Sometimes, especially when the operations of the function are simple and used only once, it is more convenient to simply use a small code block as a function without defining a function with a name. This is where anonymous functions or lambda expressions come to play.

The word lambda originates from lambda calculus, which has played an important role in the development of modern computational theory and functional programming. You are encouraged to search for lambda calculus on the internet for more details.

In Python, an anonymous function can be defined using the following syntax:

lambda <formal argument list> : <expression whose value is to be returned>

In the above syntax, a formal argument list is a list of variables separated by commas but without surrounding parentheses, and everything behind the colon takes the role of the code block in the regular function definition. But it must be a single expression whose value is to be returned by the lambda function without a keyword return.

The following is an example of a lambda function in Python that is used to construct an odd number from a given integer:

>>> lambda n: 2 * n + 1

>>>

Because an anonymous function is meant to have no name, the common use of such a function is to have it called directly when it is defined, as shown below:

>>> (lambda n: 2 * n + 1)(4)

Note that a pair of parentheses encloses the entire lambda expression to signify the end of the lambda expression.

A lambda expression can also have two or more formal arguments, as shown below:

>>> (lambda x, y: x + y)(3, 5)

In the example above, we first defined a lambda function within a pair of parentheses, then applied it to a list of two actual arguments within a pair of parentheses.

An anonymous function can even be defined to take a variable-length list of arguments, as shown in the following example:

>>> (lambda& * x: sum(x) * 3)(1, 2, 3, 4, 5)

Although an anonymous function is meant to have no name, that does not stop you from giving it a name, as shown in the next example:

>>> double = lambda x: 2 * x

We can then call the function with the name, as shown below:

>>> double(23)

Our next anonymous function takes two arguments and checks if one is a multiple of the other:

>>> is_multiple = lambda m, n: m % n == 0

>>> is_multiple(32, 4)

True

>>> is_multiple(32, 5)

False

The next section will show how lambda expressions can be used to program more effectively.

6.5 Special Functions: Mapping, Filtering, and Reducing

As mentioned, Python treats everything as objects, including functions, which can be accessed in the same way as ordinary objects. The following is an example:

>>> f_objects = [abs, len, open] # the list has three functions as its members

>>> f_objects[0](-2) # f_objects[0] refers to the first item in the list, which is built-in function abs

>>> f_objects[1](f_objects) # f_objects[1] refers to the second item in the list, which is built-in function len

This has provided programmers with great possibilities. For example, Python has three special built-in functions that can take other functions as arguments and apply them to a list. These special functions include mapping, filtering, and reducing.

Mapping

It is easier to explain what mapping does with a code example:

>>> integers = [-12, 32, -67, -78, -90, 88] # this list has negative numbers

>>> list(map(abs, integers)) # this maps the function abs to each integer in the list

[12, 32, 67, 78, 90, 88]

Note that abs() has been applied to every member of the list.

Basically, the map function can apply any function to as many lists as required for its arguments. The following is an example:

>>> def sum(a, b): # this function requires two arguments

… return a + b

…

>>> list(map(sum, [1, 2, 3], [5,8,9])) # this maps the sum function to two lists for the two arguments

[6, 10, 12]

Given what you have already learned about anonymous functions, you can generate a list of odd numbers neatly, as follows:

>>> list(map(lambda n: 2 * n + 1, range(10)))

[1, 3, 5, 7, 9, 11, 13, 15, 17, 19]

Similarly, we can generate all multiples of a number k, such as 3 in the example below:

>>> list(map(lambda n: 3 * n, range(10)))

[0, 3, 6, 9, 12, 15, 18, 21, 24, 27]

Filtering

The filtering function takes the same form as the mapping function but is used to extract items from the iterable that satisfy the Boolean filtering condition at hand. The following sample code keeps only even numbers in the generated list:

>>> def even(n):

… return n % 2 == 0 # if it can be divided by 2, then return true; return false otherwise

…

>>> list(filter(even, range(10))) # filter applies to each even number and keeps only some

[0, 2, 4, 6, 8]

The code above can be neatly rewritten by using a lambda expression, as follows:

>>> list(filter(lambda n: n % 2 == 0, range(10)))

[0, 2, 4, 6, 8]

The filter function may play an important role in selecting items from a given list based on certain criteria, as shown in the following example:

>>> st = "The filter function may play an important role in selecting words from a given text. In the following example, only words that contain the letter o are selected"

>>> list(filter(lambda s: 'o' in s, st.split()))

['function', 'important', 'role', 'words', 'from', 'following', 'only', 'words', 'contains', 'o']

Note that only the words that include the letter o are selected.

Reducing

The reduce function is not a built-in function but defined in the functools module. It takes the same form as the mapping and filtering functions but applies the function to items in the iterable progressively until the list is exhausted and reduced to a single value or object, as shown in the following example in JupyterLab:

In [ ]:

def pow(m, n):

return m ** n

from functools import reduce

reduce(pow, range(2, 6)) # equal to pow(pow(pow(2, 3), 4), 5)

Out [ ]:

1152921504606846976

We can recode the above operation with the lambda expression (anonymous function) we learned in the previous section, as shown below:

In [ ]:	`reduce(lambda n, m: n ** m, range(2, 6))`
Out [ ]:	1152921504606846976

As seen from the examples above, a lambda expression becomes handy when you use a function only once and the function is simple enough to be written with an expression.

6.6 Generators: Turning a Function into a Generator of Iterables

As you have seen, sequences (including lists, tuples, and strings) are an important way of organizing data. In addition to having a relatively static list accessible through a variable, Python also provides a means to make a generator that can generate members of a sequence dynamically.

Assume we want to find out a sequence of perfect numbers within a given range. Instead of finding all perfect numbers within the range and returning them in a list, we can define a generator of perfect numbers using the yield statement in place of the return statement, as shown in the following example:

In [ ]:

def isPerfect(n):

factor = [1]

for j in range(2, (n // 2) + 1):

if n % j == 0:

factor.append(j)

if n == sum(factor):

return [True, factor]

else:

return [False, []]

def perfectGenerator(m = 100):

for i in range(m):

testResult = isPerfect(i)

if testResult[0]:

yield [i, testResult[1]]

myPerfects = perfectGenerator(10000)

print(myPerfects)

Out [ ]:

As we can see, instead of returning a list of all the perfect numbers and factors within the given range, the function actually returned a generator.

To get the next perfect number in the generator, we use the built-in function next, as shown below:

In [ ]:

print(next(myPerfects))

Out [ ]:

[1, [1]]

[6, [1, 2, 3]]

[28, [1, 2, 4, 7, 14]]

[496, [1, 2, 4, 8, 16, 31, 62, 124, 248]]

[8128, [1, 2, 4, 8, 16, 32, 64, 127, 254, 508, 1016, 2032, 4064]]

Note that in the output above, we found five perfect numbers between 1 and 10000. Output on each line is a list, and the first member is a perfect number, whereas the second member contains a list of its factors (whose sum is equal to the first number).

Recall that we have used the built-in function range() with for loops. We can use user-defined generators with for loops as well, as shown below:

In [ ]:

myPerfects = perfectGenerator(10000)

for p in myPerfects:

print(p)

Out [ ]:

[1, [1]]

[6, [1, 2, 3]]

[28, [1, 2, 4, 7, 14]]

[496, [1, 2, 4, 8, 16, 31, 62, 124, 248]]

[8128, [1, 2, 4, 8, 16, 32, 64, 127, 254, 508, 1016, 2032, 4064]]

You may wonder why we need generators instead of returning a list of objects. The reason is for performance in terms of both speed and memory usage. The next two examples show the difference in speed between a generator and a normal function returning a list of perfect numbers.

In [ ]:		`import time` `# time module for time and timing related functions` `from memory_profiler import profile` `# a module for profiling memory usage` `def isPerfect(n):` `factor = [1]` `for j in range(2, (n // 2) + 1):` `if n % j == 0:` `factor.append(j)` `if n == sum(factor):` `return [True, factor]` `else:` `return [False, []]` `def getAllPerfects(n = 10):` `perfects=[]` `for i in range(1, n + 1):` `testResult = isPerfect(i)` `if testResult[0]:` `perfects.append((i, testResult[1]))` `return perfects` `t0 = time.process_time()` `perfectNumbers = getAllPerfects(10000)` `t1 = time.process_time()` `print(f"{t1 - t0} seconds are used to get a list of perfect numbers")`
Out [ ]:		1.1875 seconds are used to get a list of perfect numbers
In [ ]:	`import time` `# time module for time and timing related functions` `from memory_profiler import profile` `# a module for profiling memory usage` `def isPerfect(n):` `factor = [1]` `for j in range(2, (n // 2) + 1):` `if n % j == 0:` `factor.append(j)` `if n == sum(factor):` `return [True, factor]` `else:` `return [False, []]` `def perfectGenerator(m = 100):` `for i in range(m):` `testResult = isPerfect(i)` `if testResult[0]:` `yield [i, testResult[1]]` `t0 = time.process_time()` `myPerfects = perfectGenerator(10000)` `t1 = time.process_time()` `print(f"{t1 - t0} seconds are used to get a generator of perfect numbers")`
Out [ ]:	0.0 seconds are used to get a generator of perfect numbers

The first code sample is to find a list of perfect numbers within a given range using a normal function to return a list of perfect numbers, whereas the second code sample uses a generator instead. As you can see, to get a generator of perfect numbers took no time (0.0 seconds), whereas using a function to return a list of perfect numbers took 1.1875 seconds.

The gain from returning a generator instead of a complete list from a function is even more obvious in terms of memory usage, because a list would usually take a bigger chunk of computer memory, which increases drastically along with the growth of the list, whereas for a list generator, the memory usage is determined by the coding of the function and will not change. In our examples above, the generator that generates perfect numbers from 1 to 100000 has almost the same number of lines of code as the function that produces a list of perfect numbers in the same range, but the function consumes much more memory than the generator because of the use of list. To get a sense of how much memory would be consumed by a bigger list, see the following example:

In [ ]:

import sys

print(f"The size of a list containing numbers from 0 to 100 is {sys.getsizeof(list(range(100)))}Bytes")

print(f"The size of list containing numbers from 0 to 1000 is {sys.getsizeof(list(range(1000)))}Bytes")

print(f"The size of list containing numbers from 0 to 10000 is {sys.getsizeof(list(range(10000)))}Bytes")

Out [ ]:

The size of a list containing numbers from 0 to 100 is 1008Bytes

The size of a list containing numbers from 0 to 1000 is 9112Bytes

The size of a list containing numbers from 0 to 10000 is 90112Bytes

As you can see, a simple list containing integer numbers from 0 to 10000 takes up almost 90KB of memory.

6.7 Closures: Turning a Function into a Closure

In Python and some other programming languages such as JavaScript, closures are the result of a nested function—that is, one function that is defined inside another—as shown in the following example:

In [ ]:

def outer(greeting):

print('Good morning!')

def inner(msg):

print(msg)

inner(greeting)

outer('Good day!')

Out [ ]:

Good morning!

Good day!

In the example above, the outer function can also return the inner function with the parameter list as a first-order object, as shown below:

In [ ]:

def outer():

greeting = 'Good morning,'

def greet(who):

print(greeting, who)

return greet

cl = outer() # cl will hold function greet

name = input('What is your name?')

cl(name) # local greeting in outer still attached

Out [ ]:

Good morning, Joe

The output was produced when you input Joe for the name.

What has been returned from call of outer() is a function object. However, because of the way the function is defined and returned, the value of the variable greeting, defined locally within the outer function, has been attached to the inner function object. This is called closure—the binding of certain data to a function without actually executing the function.

6.8 Decorators: Using Function as a Decorator in Python

Python has many powerful means and program constructs that you can use to solve problems and get the jobs done. Decorators are one of them.

In Python, a decorator can be a function, but it is used to modify the functionality of other functions. Suppose we want to keep a log of the time a function is called, the parameters passed, and the time the call took to run. A decorator function can be defined as follows:

In [ ]:

import time # import the time module

# define a function as a decorator

def calllog(function_logged): # calllog function will be used as a high order function and decorator

def wrapper_function(*args, **kwargs):

t0 = time.asctime()

t1 = time.time() # time started in seconds as a float number

func_handle = function_logged(*args, **kwargs)

t2 = time.time() # time ended in seconds as a float number

call_args = ''

if args:

call_args += str(args)

if kwargs:

call_args += str(kwargs)

with open("calllog.txt", "w") as logs:

log_str = f"Call to {function_logged.__name__}{call_args}\n"

log_str += f"was made at {t0}, taking {t2 - t1} \n"

logs.write(log_str)

print(f"Logging string written to file is:\n {log_str}")

return func_handle

return wrapper_function

@calllog # calllog is used as decorator

def real_function(message = "Operation",m=2, n=3):

print(message,'\n', f'{m}**{n}=\n', m**n)

real_function(message="Operation m to the power of n", m=123456789, n=199)

Out [ ]:

Operation m to the power of n

123456789**199=

162734808309284601324179505945965695588773528919575975271865178655668529042309237619989258306106107470796567847777669497957864200046671201974041895254724605598956528520334232016359876319939446187437522130137245495140379083438313330693339877194631448240511529289603414404095019761182651422306321569455963302539494765513157383217158990638416187333222649224813255256278312840473976263295612792832084345917448681553425787599413514884427065211671134509510806599010879043404958116158993472332668322594948098345782426549383176674184144954140872264177848916671946547383801435396524232967278397336246191565592631874430342301391129941032341482155018485364846220355524845852067119382456173306815310415554778390870245864089998438555977549083028217284953868425189944782520828778542782626615331157474809673798175226065273267575248913342682006093549957522358406575109001573542167699914508147555771409008130451113932601682234782006419441548442926373709258668550718362777098506123158504564363626666999333401394668554955717542096588806234055802753715295178315329322302199784060472701645370398274690922242108750777206411131334391598657540208216549131267004257438048558678889138741539585727504580552194202608283674513707600806785959282522564179900246170273659285795738065271914865782904333906031177020338769884454165407411399431838155103129975186500016431885323447126919263386332087431824129594956585137199246787097451146655754268905549434601930247921212078151140658777822152688458823924816621267882515972976339888586466699424919861815375638098673752689005536584716235442340479656682376241467062062297627346560365999071034663278109

Logging string written to file is:

Call to real_function{'message': 'Operation m to the power of n', 'm': 123456789, 'n': 199}

was made at Mon Mar 13 12:07:27 2023, taking 0.0

As shown above, the log written to the calllog.txt file is as follows:

Call to real_function{'message': 'Operation m to the power of n', 'm': 123456789, 'n': 199}

was made at Mon Mar 13 12:07:27 2023, taking 0.0

Our next example is to use the calllog function as a decorator, as defined above, to record the time it takes to find all perfect numbers in a given range, the example we have worked on in Chapter 4.

This time, since we have learned how to define and use functions, we will define functions for factorization and perfect number testing, respectively, so that we can log the time on calls to the function. The code is as follows:

In [ ]:

def factors(n): # function for finding factors of a given number

# the function will return a list of factors for n

factor_list = [1] # make a list with 1 as a single element

for f in range(2,n): # start from 2, with n as excluded from factors

if n%f == 0: # f is a factor of n

if not f in factor_list:

factor_list.append(f)

# now we have a list of factors for n

return factor_list

# function to find all perfect numbers in a given range

def perfect_numbers(a, b):

if a>b: # then we need to swap a and b

c = a; a = b; b = c

perfect_list = [] # make an empty list ready to hold all perfect numbers

for n in range(a, b+1): # b is included

factor_list = factors(n)

if n == sum(factor_list):

perfect_list.append([n, factor_list]) # keep factors too for checking

return perfect_list

# now the main

@calllog # calllog is used as a decorator

def do_perfect():

success = False

# use a while loop to keep asking for inputs until success is True

while not success:

num1 = input("Enter the first number: ")

num2 = input("Enter the second number: ")

# try to convert the inputs to floats and divide them

try:

a, b = int(num1), int(num2)

# set success to True if no error has occurred by now

success = True

perfect_list = perfect_numbers(a, b)

# now we have found all the perfect numbers in the range

print(f"Perfect numbers found between {a} and {b}:")

for n in perfect_list:

print(n, end=" ")

# handle the possible errors and exceptions

except ValueError:

print("Invalid input. Please enter numbers only.")

except ZeroDivisionError:

print("Cannot divide by zero. Please enter a nonzero number.")

except Exception as e:

print(f"An unexpected error occurred: {e}")

# end of do_perfect

do_perfect()

Out [ ]:

Perfect numbers found between 3 and 10000:

[6, [1, 2, 3]]

[28, [1, 2, 4, 7, 14]]

[496, [1, 2, 4, 8, 16, 31, 62, 124, 248]]

[8128, [1, 2, 4, 8, 16, 32, 64, 127, 254, 508, 1016, 2032, 4064]] Logging string written to file is:

Call to do_perfect

was made at Mon Mar 13 13:18:15 2023, taking 5.2627036571502686

As shown above, the log written to the calllog.txt is

Call to do_perfect

was made at Mon Mar 13 13:18:15 2023, taking 5.2627036571502686

As you can see, although a decorated function is called in the same way as other functions, a lot of other things can be done through a decorator during the call.

It is also worth noting that perfect numbers are very rare. Between 3 and 10000, there are only four perfect numbers, and it took a little over 5 seconds to find them. Tested on the same machine, it took more than 100 second to look for all the perfect numbers between 3 and 100000, and there are no perfect numbers between 10000 and 100000.

6.9 Properties of Functions

Functions are important building blocks of programs in all programming languages. In Python, a function will have the following properties, and most of them can be written by the programmer of the function.

__doc__

This holds the function’s docstring, written by the programmer, or None if no docstring is written by the coder. The docstring will show up when the help() function is called on a function, though help() will also show some standard information even if no docstring is available.

__name__

This contains the name of the function.

__qualname__

This contains the qualified name of the function and is new to Python 3.3 and later versions. By qualified name, we mean the name that includes the path leading to the location where the function is defined. For example, a function F can be a method defined in a class C, which is defined in a module M, in which case the qualified name will be M.C.F.

__module__

This stores the name of the module in which the function was defined. It contains None if module info is unavailable.

__defaults__

This stores a tuple containing default argument values for those arguments that have defaults or stores None if no arguments have a default value.

__code__

This contains the actual code object representing the compiled function body.

__globals__

This contains a reference to the dictionary that holds the function’s global variables, the global namespace of the module in which the function was defined. It is a read-only function automatically generated by Python Virtual Machine (PVM).

__dict__

This stores a dictionary describing names of attributes and their values in the namespace of the function. In Python, a namespace is a mapping from names to objects and is implemented as a Python dictionary.

__closure__

This stores a tuple of cells that contain bindings for the function’s free variables. Each cell has an attribute called cell_contents from which a cell’s value can be retrieved. It is automatically generated by PVM.

__annotations__

This stores a dictionary containing annotations of parameters. The keys of the dictionary are the parameter names, and return as key for the return annotation, if provided, and the values of the dictionary are the expected data type of the parameters as well as the expected data type of the object to be returned by the function.

__kwdefaults__

This stores a dictionary containing default values for keyword-only parameters.

Chapter Summary

• Functions are important building blocks of programs in almost all programming languages.
• Different languages use different keywords to signify the definition of a function. Python uses def to start the definition of a function (and method within a class definition).
• In Python, the definition of one function can contain the definition of other function or functions.
• A function can have positional arguments, variable-length lists of arguments, and keyword arguments.
• When calling a function, the parameters for positional arguments must come first, followed by variable-length lists of arguments. Keyword arguments come last.
• A recursive function is a function that calls itself within its definition.
• Anonymous functions are functions without a name. They begin with the keyword lambda. Anonymous functions are useful when the function is only used once.
• Mapping, filtering, and reducing are special functions that can used to apply a function, including an anonymous function, to a list of parameters.
• Mapping applies a function to each item of a list.
• Filtering applies some criteria specified in a Boolean function to a list to filter out the items that do not meet the criteria.
• Reducing sequentially applies a function to the members of a list and reduces the list to a single member.
• Functions are often used to process and return the results of information processing using the return statement.
• In a function definition, the return statement can be replaced with the yield statement to turn the function into a generator of a sequence.
Note that a function cannot be turned into a generator by simply replacing the return statement with a yield statement.

Exercises

1. Python has a built-in input function for taking input from users. However, it treats everything from the user as a string. For this exercise, define a function named getInteger, which takes one optional argument as a prompt and gets and returns an integer from the user.
2. Define a function that has one argument, n, that will take a natural number and return the product of all the odd numbers between 1 and n.
3. Define a function that has one argument, n, that will take an integer and return the product of all the odd numbers between 1 and n, or between n and −1 if n is a negative integer, or that will return None if n is not an integer.
4. The Fibonacci sequence (F_n) is well-known in mathematics, and is defined as follows:

F₀ = 1, F₁ = 1, F_n = F_n−1 + F_n−2

Define a recursive function that takes one argument, n, and calculates and returns the nth item (F_n) of the Fibonacci sequence.
5. Define a recursive function that takes one argument, n, and calculates and returns the entire list of all items from F₀ to F_n, of the Fibonacci sequence.
6. Define a function that takes a variable-length list of numbers and returns the product of all these numbers.

Projects

1. Study the federal personal income tax rates for the current tax year and define a function that takes one argument as net taxable income and calculates and returns the total federal income tax due.
2. Study both the federal personal income tax rates and the provincial tax rates for your province for the current tax year. Define a function that takes one argument as net taxable income and calculate and return both total federal income tax due and total provincial tax due.
3. Modify the function defined for 6.6 by adding a keyword argument with the default value f or F to tell the function to calculate and return the federal tax due. If the argument passed to the keyword parameter is p or P, then provincial tax due should be calculated and returned. Test the modified function with different values passed to the keyword argument to make sure it works as expected.
4. The Tower of Hanoi is a mental game. It consists of three rods and N disks of different sizes that can move onto any rod one by one, but at no time is a bigger disk allowed to be on top of a smaller disk. The three rods are labelled A, B, and C. The game begins with all disks stack on rod A, and the goal is to move all the disks onto rod C. Write a recursive function that takes one argument as the number of disks initially on rod A and print out the moves to be taken in order to move all the disks from A to C. Each move can be represented as i: Rs Rd, where i is 0, 1, 2…, Rs and Rd is A, B or C, but Rs and Rd cannot be the same. It means that at step i, take one disk from source rod Rs and slide it onto destination rod Rd. For example, if there are three disks on rod A, the moves will be as follows:
1: A C, 2 : A B, 3 : C B, 4 : A C, 5 : B A, 6 : B C, 7 : A C

Chapter 7. Object-Oriented Programming with Python