Thinking in Functions, Part II: Higher-order Functions

In the previous article, we have learned about the Input/Output pattern and how to use it when writing functions Today I'd like to continue the series by talking about one of the most powerful concepts of functional programming—higher-order functions.

Higher-order function

A higher-order function is a function that accepts another function as an argument or returns a function. Or both.

Here's an example of a higher-order function:

1// Function `fn` accepts `anotherFn` as an argument,
2function fn(anotherFn) {
3  // ...and calls it with `x` to get the value of `y`.
4  const y = anotherFn(x);
5}

While this example is rather abstract and may look unfamiliar, there are multiple higher-order functions that you're already using on a daily basis. In JavaScript, a lot of standard data types methods are higher-order functions. For example:

• Array.prototype.map
• String.prototype.replace
• Array.prototype.reduce

All those functions accept another function as an argument and that makes them higher-order functions. Let's analyze what that means by taking a closer look at Array.prototype.map:

1const numbers = [1, 2, 3];
2
3// Go through each number in the `numbers` array
4// and multiply it by 2.
5numbers.map((number) => number * 2); // [2, 4, 6]

You know that the .map() method goes (iterates) through each array member and applies some transformation to it. Notice how you never see the "goes through each array member" logic while using this method, you only describe the transformation to apply. That's because .map() is responsible for iteration, and when it comes to the point to map each value, it executes the function accepted as an argument.

This is the key principle of higher-order functions: logic encapsulation.

By doing so, the .map() function establishes a usage contract with us. As with any contract, there are some terms to make it work:

• A higher-order function controls when to call a passed function;
• A higher-order function controls what arguments are to the accepted function.

Both those requirements are related to the fact that higher-order functions accept a function definition, in other words: instruction for the action. The given function definition is accessed by the higher-order function as an argument, making it in charge of when and how to call a given function.

To better understand this concept, let's build our own map function.

1function map(arr, mapFn) {
2  let result = [];
3
4  for (let i = 0; i < arr.length; i++) {
5    // Get the current array member by index.
6    const member = arr[i];
7
8    // Call the `mapFn` function we accept as an argument,
9    // and provide the current array member to it.
10    const mappedValue = mapFn(member);
11
12    // Push the result of the `mapFn` function
13    // into the end array of transformed members.
14    result.push(mappedValue);
15  }
16
17  return result;
18}

Our custom map function can be used like so:

1map([1, 2, 3], (number) => number * 2);
2// Identical to:
3// [1, 2, 3].map((number) => number * 2)

You can see that the iteration details like the for cycle and the internal results array are not exposed to the mapFn function, and our custom map function controls precisely when to call the given mapFn argument and what data to provide it:

const mappedValue = mapFn(member);

The point of our map function is that it can do much more than the multiplication of numbers. In fact, as it accepts a function that controls what to do with each array member, I dare say our map function can do anything!

1map(["buy", "gold"], (word) => word.toUpperCase());
2// ["BUY", "GOLD"]

But why this function is so powerful? Because it encapsulates the how (iteration) and accepts the what (transformation). It keeps the logic that's a part of its contract hidden, but provides a way to customize a certain behavior via arguments to remain versatile.

Returning a function

A higher-order function may also return a function. In that case, it acts opposite: instead of being in charge of when and how to call a given function, it generates a function and makes you in charge of when and how that function is called.

Let's use the same map function we've written earlier, but now rewrite it so it returns a function:

1// Instead of accepting the array straight away,
2function map(mapFn) {
3  // ...we return a function that accepts that array.
4  return (arr) => {
5    let result = [];
6
7    // ...leaving the iteration logic as-is.
8    for (let i = 0; i < arr.length; i++) {
9      const member = arr[i];
10      const mappedValue = mapFn(member);
11      result.push(mappedValue);
12    }
13
14    return result;
15  };
16}

Since the map now accepts only one argument, we need to change the way we call it:

map((number) => number * 2)([1, 2, 3]);

The fn(x)(y) call signature is not common in JavaScript. Moreover, it's rather confusing.

History digression: Not a long time ago such, call signature was used in React to describe higher-order components, so it may ring some distant hook-less bells.

export default connect(options)(MyComponent);

Don't worry, we don't have to abide by this unusual call signature. Instead, let's break that map function call into two separate functions.

1// Calling `map` now returns a new function.
2const multiplyByTwo = map((number) => number * 2);
3
4// That returned function already knows it should
5// multiply each array item by 2.
6// Now we only call it with the actual array.
7multiplyByTwo([1, 2, 3]); // [2, 4, 6]
8
9// We can reuse the `multiplyByTwo` function
10// without having to repeat what it does,
11// only changing the data it gets.
12multiplyByTwo([4, 5, 6]); // [8, 10, 12]

Our map function doesn't do any iteration on its own anymore, yet it generates another function (multiplyByTwo) that remembers what transformations to do, only waiting for us to provide it with data.

Applications

A higher-order function is a great instrument to use when designing functions. The point of higher-order functions is to encapsulate logic. That encapsulation may solve one or more purposes:

• Abstract implementation details in favor of declarative code.
• Encapsulate logic for versatile reuse.

Abstract logic

The most basic example is when you need to repeat a certain action N amount of times. Functional abstractions can come in handy compared to a lengthy for loop. Of course, it would still be using the loop internally, abstracting the iteration as it matters not when you use the function.

1function repeat(fn, times) {
2  for (let i = 0; i < times; i++) {
3    fn();
4  }
5}
6
7// Where are not concerned with "how" (iteration),
8// but focus on the "what" (declaration), making
9// this function declarative.
10repeat(() => console.log("Hello"), 3);

By moving the imperative code to the implementation details of a higher-order function, we often gain improved code readability, making our logic easier to reason about. Compare these two examples: one with imperative code and another with declarative higher-order function:

1// Imperative
2const letters = ["a", "b", "c"];
3const nextLetters = [];
4
5for (let i = 0; i < letters.length; i++) {
6  nextLetters.push(letters[i].toUpperCase());
7}

1// Declarative
2const letters = ["a", "b", "c"];
3const nextLetters = map(letters, (letter) => letter.toUpperCase());

While both examples map Array letters to upper case, you need much less cognitive effort to understand that intention behind the second example.

It's also about reusing and composing—creating new logic by combining existing functions. Take a look at how the abstractions below give you an idea of what's happening without you peaking into how they are written:

1map(ids, toUserDetail);
2map(users, toPosts);
3reduce(posts, toTotalLikes);

Encapsulate logic

Let's say we have a sort function in our application:

1function sort(comparator, array) {
2  array.sort(comparator);
3}

We are using this function to sort multiple things by rating: products, books, users.

1sort((left, right) => left.rating - right.rating, products);
2sort((left, right) => left.rating - right.rating, books);
3sort((left, right) => left.rating - right.rating, users);

You can see how the comparator function is the same for each call, regardless of what data we're working with. We sort by rating a lot in our app, so let's abstract that comparator into its own function and reuse it:

1function byRating(left, right) {
2  return left.rating - right.rating;
3}
4
5sort(byRating, products);
6sort(byRating, books);
7sort(byRating, users);

That's better! Our sorting calls, however, still operate on the criteria-agnostic sort function. It's a minor thing, but we also have to import two functions (sort and byRating) anywhere we need to sort by rating.

Let's take the comparator out of the equation and lock it in a sortByRating function that sorts a given array by rating straight away:

1function sortByRating(array) {
2  sort(byRating, array);
3}
4
5sortByRating(products);

Now the rating comparator is built-in into the sortByRating function and we can reuse it anywhere we sort by rating. It's a single function, it's short, it's great. Case closed.

Our application grows in size and requirements, and we find ourselves sorting not only by rating, but also by reviews and downloads. If we follow the same abstraction strategy further, we'll stumble upon a problem:

1function sortByRating(array) {
2  sort(byRating, array);
3}
4
5function sortByReviews(array) {
6  sort(byReviews, array);
7}
8
9function sortByDownloads(array) {
10  sort(byDownloads, array);
11}

Because we've moved out the comparator from the sortBy* arguments, whenever we need to encapsulate a different comparison logic, we inevitably create a new function. By doing so, we're introducing a different kind of problem: neither of the sortBy* functions above share the intention of sorting an array, instead they repeat the implementation (sort) all over the place.

We can approach this abstraction task with higher-order functions, which would allow us to create exactly one concise and deterministic function to satisfy our requirements.

1function sort(comparator) {
2  return (array) => {
3    array.sort(comparator);
4  };
5}

The sort function accepts a comparator and returns an applicator function that does the sorting. Notice how the nature of comparator and array are variative, coming from arguments, yet the function's intention (array.sort) does not repeat despite that dynamic nature.

Now we can create multiple sorting functions encapsulating different criteria like so:

1const sortByRating = sort(byRating);
2const sortByReviews = sort(byReviews);
3const sortByDownloads = sort(byDownloads);
4
5sortByRating(products);
6sortByReviews(books);
7sortByDownloads(songs);

This is a great example of logic encapsulation and reuse. It's beautiful also.

Mention: Currying

Higher-order functions are also fundamental to partial application and currying—two techniques that are irreplaceable in functional programming. Don't worry if they sound alien to you, we are going to talk about them in the future chapters of this series.

Putting it into practice

Just as with any other function, applying the Input/Output pattern is a great place to start when writing a higher-order function. With that, there are a few additional questions to ask yourself:

1. What action is being delegated to the argument function?
2. When should the argument function be called?
3. What data is provided to the argument function?
4. Does the returned data of the argument function affects the parent function?

It's crucial to establish a clear separation between the responsibilities of a higher-order function and the argument function it accepts.

Exercise: Try to write your own filter() function: it accepts an array and a function that returns a Boolean. It returns a new array, with the members for which the argument function returned true:

filter([1, 3, 5], (number) => number > 2); // [3, 5]

Use the map function we've created earlier in this article as a reference.

Real-life example

While working on one of my projects, I've decided to create a custom function that would allow me to handle an XMLHttpRequest instance as a Promise. The intention was to make such requests declaration shorter and support the async/await syntax. I've started by creating a helper function:

1function createXHR(options) {
2  const req = new XMLHttpRequest();
3  req.open(options.method, options.url);
4
5  return new Promise((resolve, reject) => {
6    req.addEventListener("load", resolve);
7    req.addEventListener("abort", reject);
8    req.addEventListener("error", reject);
9    req.send();
10  });
11}

I would then use that createXHR function in my tests like this:

1test("handles an HTTPS GET request", async () => {
2  const res = await createXHR({
3    method: "GET",
4    url: "https://test.server",
5  });
6});

Thing is, I also needed to configure the request differently for various testing scenarios: set headers, send request body, or attach event listeners. To support that, I went to my createXHR function and extended its logic:

1function createXHR(options) {
2  const req = new XMLHttpRequest()
3  req.responseType = options.responseType || 'text'
4
5  if (options?.headers) {
6    Object.entries(options.headers).forEach([header, value] => {
7      req.setRequestHeader(header, value)
8    })
9  }
10
11  req.addEventListener('error', options.onError)
12
13  return new Promise((resolve, reject) => {
14    // ...
15    req.send(options.body)
16  })
17}

As the test scenarios grew in diversity, my createXHR function grew in complexity. It resulted in an overly complex function that was hard to read and even harder to use. Why did that happen?

My mistake was to assume that the createXHR function should configure a request on its own. Describing a request configuration as the options object wasn't a sound choice either, since the object is a finite data structure and cannot represent all the variety of how a request can be declared.

Instead, my helper function should have allowed for each individual call to configure a request instance it needs. And it could do that by becoming a higher-order function and accepting an action that configures a request instance as an argument.

1// Accept a `middleware` function,
2function createXHR(middleware) {
3  const req = new XMLHttpRequest();
4
5  // ...that configures the given `XMLHttpRequest` instance,
6  middleware(req);
7
8  // ...and still promisifies its execution.
9  return new Promise((resolve, reject) => {
10    req.addEventListener("loadend", resolve);
11    req.addEventListener("abort", reject);
12    req.addEventListener("error", reject);
13  });
14}

The reason XMLHttpRequest instance is declared within the function and not accepted as an argument is because you cannot change certain options once a request has been sent.

Notice how cleaner that function becomes as it delegates the request configuration to a middleware function. With that, each test can provide its own way to set up a request and still receive a Promise in return.

1test("submits a new blog post", async () => {
2  const req = await createXHR((req) => {
3    req.open("POST", "/posts");
4    req.setRequestHeader("Content-Type", "application/json");
5    req.send(JSON.stringify({ title: "Thinking in functions", part: 2 }));
6  });
7});
8
9test("handles error gracefully", async () => {
10  const req = await createXHR((req) => {
11    req.open("GET", "/posts/thinking-in-functions");
12    req.addEventListener("error", handleError);
13    req.send();
14  });
15});

Afterword

High-order functions may be a hard concept to grasp at first, but give it some time, apply it in practice, and the understanding will come. It's a crucial part of functional programming and a great step towards thinking in functions. I hope this article has contributed to your knowledge, and you feel an extra tool in your arsenal now.

Looking forward to seeing you in the next part of the "Thinking in Functions" series!