Motivating Example for Automation with Python Functions

Suppose you want test out the Collatz Conjecture over the integers in the range [1,10]. In short, the Collatz conjecture states that if we start at any positive integer n, and iteratively take n = n/2 if the number is even or n = 3*n + 1 if the number is odd, then eventually n will become 1. We can generate the first iteration by creating a list of integers using the range function, and apply the Collatz Conjecture to each element using a loop like we saw in our previous tutorial:

i_1 = [x for x in range(1, 11)]
i_2 = []
for i in i_1:
	if i == 1:
		i_2.append(i)  # Skip if we've reached 1
	elif i % 2 == 0:  # % is the modulo operator
		i_2.append(int(i/2))  # int converts float type to integer type
	else:
		i_2.append(int(3*i + 1)) 
print(i_2)
> [1, 1, 10, 2, 16, 3, 22, 4, 28, 5]

It looks like two numbers have reached one, so we’ll have to repeat:

i_3 = []
for i in i_2:
	if i == 1:
		i_3.append(i) # Skip if we've reached 1
	elif i % 2 == 0:  # % is the modulo operator
		i_3.append(int(i/2))  # int converts float type to integer type
	else:
		i_3.append(int(3*i + 1)) 
print(i_3)
> [1, 1, 5, 1, 8, 10, 11, 2, 14, 16]

Now we have three numbers that have reached 1. It looks like we’ll have to repeat this process, so we’ll copy and paste the loop several times:

i_4 = []
for i in i_3:
	if i == 1:
		i_4.append(i) # Skip if we've reached 1
	elif i % 2 == 0:  # % is the modulo operator
		i_4.append(int(i/2))  # int converts float type to integer type
	else:
		i_4.append(int(3*i + 1)) 
i_5 = []
for i in i_4:
	if i == 1:
		i_5.append(i) # Skip if we've reached 1
	elif i % 2 == 0:  # % is the modulo operator
		i_5.append(int(i/2))  # int converts float type to integer type
	else:
		i_5.append(int(3*i + 1)) 
i_6 = []
for i in i_6:
	if i == 1:
		i_6.append(i) # Skip if we've reached 1
	elif i % 2 == 0:  # % is the modulo operator
		i_6.append(int(i/2))  # int converts float type to integer type
	else:
		i_6.append(int(3*i + 1)) 
i_7 = []
for i in i_6:
	if i == 1:
		i_7.append(i) # Skip if we've reached 1
	elif i % 2 == 0:  # % is the modulo operator
		i_7.append(int(i/2))  # int converts float type to integer type
	else:
		i_7.append(int(3*i + 1)) 
print(i_7)  # Final answer?
> []

Oh dear. The Collatz Conjecture makes no prediction that numbers will disappear after 7 iterations. We have either disproved the conjecture and are worthy of a Nobel Prize, or there is an error in our code. After some tedious searching, we can see that in the sixth iteration loop, we accidentally changed the for loop to iterate over the empty set i_6 instead of the previous iteration set i_5.

Our problem is that we have fallen into an error trap. Error traps are situations in which the likelihood of making a mistake is high. The error trap in this situation was the repeated copying of code, each of which had several minor edits. The repetitive nature of this task can lead to mental fatigue and a higher probability of making a typo. How can we prevent falling into this trap and speed up the coding process? The answer is in functions. With a function, we can define the Collatz Conjecture formula only once, and loop over all of our iterations using only a few lines of code!

Get Our Python Developer Kit for Free

I put together a Python Developer Kit with over 100 pre-built Python scripts covering data structures, Pandas, NumPy, Seaborn, machine learning, file processing, web scraping and a whole lot more - and I want you to have it for free. Enter your email address below and I'll send a copy your way.

Yes, I'll take a free Python Developer Kit

Introduction to Automation with Python Functions

Python functions are a set of objects that ease performance of repetitive tasks. This tutorial will introduce Python’s function format, uses, and how to build your own functions. The topics covered in the tutorial will be the following:

Function Structure

A Python function has the following format: [output] = [function]([input]) The [function]([input]) side is the function is the “call”, and after evaluation within the body of the function will “return” an output. The above format assumes that the output is assigned to a variable, but this doesn’t have to be the case. Functions can take various Python objects as input and produce various Python objects as output. Some functions take no input, and some functions produce no output object. We have used functions regularly in the last few tutorials. For example, in Data Types we used several type conversion functions like float:

x = float(10)

Here the float function took the 10 integer object, returned a float object 10.0, and assigned it to x. We can use the type function again to check the type of the output.

type(x)
> float

Which confirmed that the float function did indeed produce a float object. But what did the type function produce?

y = type(x)
type(y)
> type

It produced a type object. Now let’s look at the print function:

x = float(10)
print(x)
> 10.0
y = print(x)
> 10.0
print(y)
> None

Notice that when we assigned the print function output to the variable y it still printed 10.0 to the terminal, and the function output was in fact a None object. When a function performs an action not associated with an output object, it is termed a “side effect” of the function. Side effects can include printing a line to the terminal, modifying a file, or plotting a graph. We will focus more on side effect generating functions such as plotting in future tutorials. Python has a multitude of built-in functions that we won’t cover in this tutorial, but select functions will be addressed in detail throughout these lessons.

User-Defined Functions

We’ve looked at the basic structure of functions as well as a few of Python’s built-in functions, but how do we create our own functions? A user-defined function has the following format:

def [name]([input]=[default]):
	[actions]
	return([output])

Like loops and if statements, the actions associated with functions can be extended over multiple lines using indentation (tab or 4 spaces). The return statement will tell Python to exit the function and output whatever object is included in [output]. Make note that once the return statement is executed, no further lines in the function will be executed (i.e. like a break statement). For example, lets make a function that doubles its numeric input:

def double(n):
	x = 2*n
	return(x)
double(1)
> 2
double(8)
> 16

In the above function, n is called a “dummy variable” representing the input object. Multiple objects in can be represented as a tuple. For example, the following function will divide x by y:

def divide(x, y):
	o = x/y
	return(o)
divide(6, 3)
> 3.0
divide(3, 6)
> 0.5

Notice that the order of the inputs determined the order of the dummy variables associated with them. We can override this ordering by using dummy variable names directly within the function call:

divide(x=6, y=3)
> 3.0
divide(y=3, x=6)
> 3.0

Calling a function without the required input will raise a TypeError.

divide(x=1)
> TypeError: divide() missing 1 required positional argument: 'y'

Likewise, including too many variables will also raise a TypeError.

divide(x=1, y=1, z=1)
> TypeError: divide() got an unexpected keyword argument 'z'

If we want a function to use a default value for a variable not passed to it, we can define it within the function definition by using an =.

def divide(x, y=2):
	o = x/y
	return(o)
divide(x=1)  # Use default y=2 value
> 0.5
divide(x=1, y=4)  # Input values override defaults
> 0.25

One reason function input variables are called “dummy variables” instead of “variables” is because they can only be found within the namespace of the function. Think of a function’s namespace as a sandbox in which the variable can be referenced. For example:

y = 15
def example(): # We can define a function without arguments
	y = 0
	print(y)
	return()  # We can also return without an output
print(y)
example()
print(y)
> 15
> 0
> 15

Lets break down the above example. In the first line we define y as 15. We then define the example function in which y is assigned 0. Printing y after the function definition will return 15 as we defined prior to the definition. Calling the example function will define y as zero, and the internal print function will print y as 0. Using the print function again after the example function call will again print 15 as we first defined. This functionality is based on how Python looks up variable names within namespaces. When we set the value of y at the start, we define it for the main program’s namespace, and printing y from the main program will return the variables defined within the main program. When we set the value of y within the function, we assign it to the function’s namespace, which was only reached by the variables within the function.

Note that functions themselves do not follow this rule. Functions created within the main program can be called within a function.

Namespaces can be confusing to Python beginners, therefore we won’t go into detail on them in this tutorial. The main takeaway here is that with some exceptions, assume that only variables passed into the input of a function can be used within the function, and assume that only variables returned by the function can be used in the main program.

Function Objects

In some of the previous sections, we’ve referred to functions as function objects. This is because the functions are actually objects defined within a namespace in Python, much like variables or data types. Like other objects, we can pass functions to variables:

def double(n):
	x = 2*n
	return(x)
y = double
print(y)
> <function double at 0x0000029149A6BC80>
type(y)
> function

Likewise we can pass objects to data structures:

l = [double, double]
print(l)
> [<function double at 0x0000029149A6BC80>, <function double at 0x0000029149A6BC80>]
l[1](2)
> 4

Functions objects can also be passed to other functions. We can see the usefulness of this property by using the map function. The map function will apply a given function to a given list, and return the sequence of output as a generator object:

l = [1, 2, 3, 4, 5]
[i for i in map(double, l)]
> [2, 4, 6, 8, 10]

Get Our Python Developer Kit for Free

I put together a Python Developer Kit with over 100 pre-built Python scripts covering data structures, Pandas, NumPy, Seaborn, machine learning, file processing, web scraping and a whole lot more - and I want you to have it for free. Enter your email address below and I'll send a copy your way.

Yes, I'll take a free Python Developer Kit

Generator Functions

You may recall in our previous discussion on Python control flows a description of the range function as a generator. Generators are useful for creating sequences of output without having to store the output or recalculate already computed values. We can create our own generator functions. A generator function is distinguished from a regular function through the use of a yield expression instead of a return statement with the following format:

def [name]([input]=[default]):
	[actions]
	yield([output])

The first time the generator function is called as an iterator, the code executes to the yield expression and returns a value. On each subsequent execution the code returns to the previous yield expression, and continues until the next yield expression is encountered. For example, we can create our own version of the range function:

def myRange(n):
	i = 0
	while i < n:
		yield(i)
		i += 1
y = myRange(11)
print([x for x in y])
> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

For a given generator function object, the next function can be used to generate successive values of the generator:

y = myRange(2)
next(y)
> 0
next(y)
> 1
next(y)
> StopIteration

The next function was useful for calculating successive values manually, however when the function can no longer reach the next yield expression, a StopIteration exception is raised. It is important to remember that functions are objects, otherwise generator functions can produce confusing results.

next(myRange(11))
> 0
next(myRange(11))
> 0

Despite using the next function twice, we received the same results. This is because we called the generator function rather than a stored object of the generator function. Essentially, calling next on the original generator created a “fresh” version of the generator, and could not “remember” previous executions. This is why it’s important to store generators before use, and call next on the variable in which its stored.

x = myRange(11)
next(x)
> 0
y = myRange(11)
next(y)
> 0
next(x) # Each variable "remembers" where the generator was
> 1

Caution: When building a generator function that produces numbers over an infinite set, include a maximum iteration counter to prevent infinite loops. For example, the following generator will count by one on each iteration:

def counter():
	i = 0
	while True:  # Do not do this
		yield(i)
		i += 1
y = counter()
next(y)
> 0
next(y)
> 1

The counter function seems to be working well, however if we used the generator in a list comprehension we’ll see a major flaw.

[x for x in y]  # This operation will crash the program

The list comprehension will crash the program because the generator will never stop producing additional numbers to add to the list. This forms an infinite loop within the while loop in the generating function. Always include a maximum number of iterations for a loop like so:

def safeCounter(max=1000):
	i = 0
	while i < max:  # Allow the loop to eventually exit
		yield(i)
		i += 1

Functions vs. Methods

In the previous sections and tutorials, we’ve used the terms “method” and “function”, but what the the differences between the two? Put simply:

  • Functions: are standalone objects
  • Methods: are functions attached to, and defined by, objects.

For example, range is a function because it isn’t associated with or dependent upon any other object, and [string].append() is a method because it is defined by the [string] object.

Get Our Python Developer Kit for Free

I put together a Python Developer Kit with over 100 pre-built Python scripts covering data structures, Pandas, NumPy, Seaborn, machine learning, file processing, web scraping and a whole lot more - and I want you to have it for free. Enter your email address below and I'll send a copy your way.

Yes, I'll take a free Python Developer Kit

Anonymous Functions: `lambda` functions

Anonymous functions are just as the title implies: functions without names. These are function objects that cannot be called outside of their initial definition. Anonymous functions are useful for short, one-line functions where a full function definition would be unnecessary. Python defines anonymous functions using the lambda expression, which are commonly referred to as lambda functions.

Lambda functions have the format lambda [input]: [output]. Notice how the function is compact, and the output is evaluated and returned within the same line. For example, a lambda function can be used to quickly check if numbers in a list are even:

n = [1, 2, 3, 4, 5]
[i for i in map(lambda x: x % 2 == 0, n)] 
> [False, True, False, True, False]

Functions Example

Let’s return to our motivating example from the start of this tutorial. How can we use functions to automate an evaluation of the Collatz Conjecture? Let’s first see a simple use of formulating the conjecture as a function:

def Collatz(i):
	if i == 1:
		return(1)  # Skip if we've reached 1
	elif i % 2 == 0:  # % is the modulo operator
		return(int(i/2))  # int converts float type to integer type
	else:
		return(int(3*i + 1)) 
i_1 = Collatz(10)
print(i_1)
> 5
i_2 = Collatz(i_1)
print(i_2)
> 16

This certainly makes copying and pasting each iteration easier, but how can we perform all iterations without copying and pasting anything? We can use a generator:

def CollatzIterate(i, n):
	k = 1
	while k < n:
		if i == 1:
			i = 1
		elif i % 2 == 0:  
			i = int(i/2)
		else:
			i = int(3*i + 1)
		yield(i)
		k += 1
all = [x for x in CollatzIterate(20, 10)]
print(all)
> [10, 5, 16, 8, 4, 2, 1, 1, 1]

Automation with Python Functions Exercises

Create a generator function that produces a consecutive Fibonacci Number with each next call.     Solution

Use a lambda function to print out all the values of the following dictionary:

ing = {"Flour":"Self-Rising",
	   "Sugar":"White Granulated",
	   "Fat":"Unsalted Butter",
	   "Milk":"Whole",
	   "Fruit":"Peaches"}

    Solution

What will the following code produce? Why?

v = [0.2, 0.5, 8.0, 0.5]
def norm_squared():
    sum = 0
    for i in v:
        sum += v[i]**2
    return(sum)
norm_squared()

    Solution

Given a number n, write a Python function that determines the total number of Collatz Conjecture iterations are required on n to reach a value of 1.     Solution

Use the map function,range function, and a lambda function to calculate the squares of the numbers in the range [1,10] on a single line.     Solution

Solutions

The following generator function will produce the Fibonacci Numbers with each iteration. Note that a maximum counter was added to prevent infinite loops.

def Fibonacci(max=10^5):
	a = 0
	b = 1
	i = 1
	while i < max:
		yield(a + b)
		(a, b) = (b, a + b)

ing = {"Flour":"Self-Rising",
	   "Sugar":"White Granulated",
	   "Fat":"Unsalted Butter",
	   "Milk":"Whole",
	   "Fruit":"Peaches"}
x = lambda item: ing[item]
[x(item) for item in ing] 
> ['Self-Rising', 'White Granulated', 'Unsalted Butter', 'Whole', 'Peaches']

It will return an error because v was not defined within the namespace of the function, but only in the namespace of the main program. Remember the rule of thumb: generally functions will only see variables passed into them, and the main program will generally only see variables passed from functions. Exceptions to this rule include global variables, which will be discussed in later tutorials.

4.

def CollatzIterate(i, n):  # Same as above
	k = 1
	while k < n:
		if i == 1:
			i = 1
		elif i % 2 == 0:  
			i = int(i/2)
		else:
			i = int(3*i + 1)
		yield(i)
		k += 1

def CountCollatz(n, max=200):
	count = 1
	v = CollatzIterate(n, max)  
	i = next(v)
	while i > 1:
		i = next(v)
		count += 1
	return(count)

5.

[y for y in map(lambda x: x**2, [x for x in range(1,11)])]

Did you find this free tutorial helpful? Share this article with your friends, classmates, and coworkers on Facebook and Twitter! When you spread the word on social media, you’re helping us grow so we can continue to provide free tutorials like this one for years to come.