Understanding Filter, Map, And Reduce In Python

Even though lot of people use Python in an object oriented style, it has several functions that enable functional programming. For those of you who don’t know, functional programming is a programming paradigm based on lambda calculus that treats computation as an evaluation of mathematical functions. Some of prominent functional programming languages include Scala, Haskell, Clojure, and so on. You should go through this nice article on functional programming that explains it in layman’s terms. Coming back to the topic at hand, Python provides features like lambda, filter, map, and reduce that can basically cover most of what you would need to operate on data. Let’s go ahead and play with them to understand their awesomeness, shall we?  

Lambda function

The lambda function is enables us to create anonymous functions. These are functions without a name. This reminds me of the famous Clint Eastwood character in the Dollars Trilogy i.e. Man With No Name. Lambda functions are basically functions that we can throw away because their scope is very localized. Why would we use such a function? Well, lambda is very frequently used with filter, map, and reduce. So it’s important to understand how it works!

Let’s go into the Python shell to understand this. Go to your terminal and type “python”:

>>> f = lambda a, b, c: a * b + c
>>> f(3,4,5)
17

It’s pretty simple, right? You define a function with some input arguments and the output definition. We use the keyword “lambda” to indicate that it’s a lambda function. In this case, ‘a’, ‘b’, and ‘c’ to the left of the colon are input arguments. The definition on the right operates on these input arguments to generate the output. Note that we don’t need the “return” statement. The expression to the right is evaluated and returned by default. Bear in mind that lambda functions can only be used when that function can be written as an expression. They don’t support multi-statement functions or functions that don’t return a value.

Filter

The “filter” function operates on a list and returns a subset of that list after applying the filtering rule.

output_list = filter(f, input_list)

The filtering rule is provided as an input argument to the function, often in the form of a lambda function. In our case, ‘f’ is that filtering rule. Pretty straightforward, right? Let’s consider a simple case where we want to get the list of all numbers that are greater than 5 from a given input list.

>>> my_list = [1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> output_list = filter(lambda x: x > 5, my_list)
>>> output_list
[6, 7, 8, 9]

As we can see, the first argument is a function defined by “lambda x: x > 5”. The second argument is the input list on which “filter” will operate to produce the output.

Map

The “map” function transforms a given list into a new list by transforming each element using a rule.

output_list = map(f, input_list)

Let’s consider an example where we want to map each number of a list to its squared value.

>>> my_list = [1, 5, 7, 8, 11]
>>> output_list = map(lambda x: x * x, my_list)
>>> output_list
[1, 25, 49, 64, 121]

Another good thing about map() is that it can take multiple lists as input arguments. This is very useful when we do vector mathematics. The map() function will take multiple lists, operate on them, and then return an output list. One thing to note is that all the lists should be of the same size because map() applies the function to corresponding elements in those lists. Let’s say we want to element-wise add two lists:

>>> new_list = [-4, 3, 2, -6, 5]
>>> map(lambda x, y: x + y, my_list, new_list)
[-3, 8, 9, 2, 16]

Reduce

The “reduce” function will transform a given list into a single value by applying a given function continuously to all the elements. It basically keeps operating on pairs of elements until there are no more elements left. Let’s say we want to find the product of all the elements in a given list:

>>> reduce(lambda a, b: a * b, my_list)

It’s as simple as that! There’s no need to write some loopy function to compute the product.

Combining all four of them

Let’s flex our analytical muscles and consider an example where we can see all four operators in action. Let’s say you want to find the sum of squares of all the even numbers between 1 and 151. Instead of writing a big function, we can do it in a single line:

>>> my_sum = reduce(lambda a, b: a + b, map(lambda x: x * x, filter(lambda y: not y % 2, range(151))))
>>> my_sum
573800

As we can see, the sum is 573800. Too complex? Okay let’s break it down. There are three lambda functions defined here:

f1 = lambda a, b: a + b
f2 = lambda x: x * x
f3 = lambda y: not y % 2

If we substitute these names, then we will get:

>>> reduce(f1, map(f2, filter(f3, range(151))))

Here, filter(f3, range(151)) outputs a list of all the even numbers between 1 and 151. Let’s call this “even_list”:

>>> reduce(f1, map(f2, even_list))

Now, map(f2, even_list) will compute the squares of all the elements in “even_list”. Let’s call this “squared_even_list”:

>>> reduce(f1, squared_even_list)

Finally, this function will compute the sum of all the elements in squared_even_list. This computing paradigm has been hugely influential in developing modern architectures and algorithms to operate on enormous amounts of data.

———————————————————————————————————