Writing a Custom Webpack Loader

Introduction

This article uses the latest webpack version, which at the moment of writing is webpack v5. webpack is a backbone that powers most of the modern frontend tooling: Create React App, NextJS, Gatsby, —the list can go on and on. Over the years webpack had grown a reputation of being hard to configure, which is rather unfortunate, because no matter how abstracted webpack is in your setup, you can always benefit from knowing how to customize it.

Today I'd like to speak about one specific way of customizing webpack: writing a custom loader. I'm surprised to find this topic rather undocumented, so most of the information you read in this article is a result of reverse-engineering multiple existing loaders to let you and me learn.

Basics

A loader in webpack is a function that transforms a source code of imported modules.

Think of a loader as a function of a module's content that creates or augments that module's export:

const styles = cssLoader(require('./styles.css'))

Loaders are the heart and soul of how webpack compiles TypeScript to JavaScript, turns SASS into CSS, and JSX into React.createElement calls. In fact, webpack doesn't really do all that, loaders do! Meanwhile, webpack is responsible for establishing a transformation chain for your source code and making sure that the loaders are executed at the right stage of the build process.

Here's how you apply loaders to files in a webpack configuration:

1// webpack.config.js
2module.exports = {
3  module: {
4    rules: [
5      {
6        // Capture all "*.js" imports,
7        test: /\.js$/,
8        // ...and transform them via "babel-loader".
9        use: ['babel-loader'],
10      },
11      {
12        // Capture all the "*.css" imports,
13        test: /\.css$/
14        // ...and transform them via "css-loader".
15        use: ['css-loader']
16      }
17    ],
18  },
19}

In the configuration above, all *.js files that you import are passed through the babel-loader, while all *.css files are transformed via the css-loader. You can provide a list of loaders for the same subset of modules, in which case the loaders are applied from right to left:

1{
2  test: /\.ext$/
3  use: ['third-loader', 'second-loader', 'first-loader']
4}

This makes more sense once you look at a loader as a function that passes its result (transformed code) to the next loader in the chain:

third(second(first(source)))

Limitations

Loaders are designed to transform code. Unlike plugins, loaders cannot affect the build process, but rather transform individual imported modules during the build.

Generally, anything that lies outside of processing and generating code can be achieved through plugins. At the same time, one thing that a plugin shouldn't do is transform code, making the boundary between a loader and a plugin rather clear.

For example, here are some of the use-cases to write a custom loader:

• Support imports of a custom file format (i.e. *.graphql or *.prisma);
• Append meta data to the transformed files (i.e. inject frontmatter to *.mdx files).
• Transform imported modules (i.e. auto-prefix imported *.css files).

Writing a custom loader

In this article we're going to write an MP3 loader that would turn any *.mp3 import into a React component for playing that audio file.

1import AudioPlayer from './audio.mp3'
2
3function MyComponent {
4  return (
5    <AudioPlayer />
6  )
7}

Declaration

A loader is a function that accepts a source code and returns a transformed source code. Let's start from that: create a mp3-loader.js file and declare a new function in there:

1// src/mp3-loader.js
2module.exports = function (source) {
3  return source
4}

At the current state, your loader is going to return the imported source (MP3 file binary) as-is.

We will be using the <audio> HTML element to play an imported audio file. For that, we need to know the path of the imported MP3 file, emit it in the webpack's build directory, and provide that file path as the src attrbiute of the <audio> element.

We can get an absolute path of the imported module by accessing the this.resourcePath property in a loader's context. For example, given the following import:

import AudioPlayer from './audio.mp3'

The this.resourcePath will contain the absolute path to the ./audio.mp3 file. Knowing this, let's emit an mp3 file with the same name:

1// src/mp3-loader.js
2const path = require('path')
3
4module.exports = function (source) {
5  // webpack exposes an absolute path to the imported module
6  // under the "this.resourcePath" property. Get the file name
7  // of the imported module. For example:
8  // "/User/admin/audio.mp3" (this.resourcePath) -> "audio.mp3".
9  const filename = path.basename(this.resourcePath)
10
11  // Next, create an asset info object.
12  // webpack uses this object when outputting the build's stats,
13  // so you could see info about the emitted asset.
14  const assetInfo = { sourceFilename: filename }
15
16  // Finally, emit the imported audio file's "source"
17  // in the webpack's build directory using a built-in
18  // "emitFile" method.
19  this.emitFile(filename, source, null, assetInfo)
20
21  // For now, return the mp3 binary as-is.
22  return source
23}

Note that you need to remain in the webpack's context in order to access this.resourcePath and this.emitFile. Ensure your loader is not an arrow function, because that's going to re-assign the loader's context and you'll lose access to the properties and methods that webpack exposes to your loader. Now the imported audio file will be emitted alongside our JavaScript modules. Let's proceed to the next step—returning a React component in our loader.

1// src/mp3-loader.js
2const path = require('path')
3
4module.exports = function (source) {
5  const filename = path.basename(this.resourcePath)
6  const assetInfo = { sourceFilename: filename }
7  this.emitFile(filename, source, null, assetInfo)
8
9  return `
10import React from 'react'
11export default function Player(props) {
12  return <audio controls src="${filename}" />
13}
14  `
15}
16
17// Mark the loader as raw so that the emitted audio binary
18// does not get processed in any way.
19module.exports.raw = true

Keep the imports that your loader needs in the loader's scope, while the imports that your transformed code needs in its scope, inlined. Everything that a loader imports and does is an implementation detail of the build, and won't be accessible in the compiled bundle. Both input and output of a loader's function must be a string. That's why we declare a new Player React component in a string, including all the necessary imports inside it. Now, whenever our application imports an MP3 file, instead of importing its binary, it'll import the Player component that the loader generated.

Congratulations! You've just written a custom webpack loader that turns MP3 files into interactive audio players. Now let's configure your webpack configuration to apply that loader to all the *.mp3 files.

Using a custom loader

There are two ways to use a loader: tell webpack to resolve it from a local file, or publish it and install as a regular dependency. Unless your loader's purpose is generic, or you plan to reuse it across multiple projects, I strongly recommend using the loader from a local file.

From a local file

To use a local webpack loader you need to alias it in the resolveLoader property in your webpack configuration:

1// webpack.config.js
2const path = require('path')
3module.exports = {
4  module: {
5    rules: [
6      {
7        test: /\.mp3$/,
8        // Reference the loader by the same name
9        // that you aliased in "resolveLoader.alias" below.
10        use: ['babel-loader', 'mp3-loader'],
11      },
12    ],
13  },
14  resolveLoader: {
15    alias: {
16      'mp3-loader': path.resolve(__dirname, 'src/mp3-loader.js'),
17    },
18  },
19}

Since we've returne JSX from our loader, we need to tell webpack to transform it into regular JavaScript. That's why we've included babel-loader as the next loader to apply after our mp3-loader is done (remember loaders are applied from right to left).

From a package

If you decide to distribute your loader as a node module, you can use it just as any other Node.js dependency.

It's a common convention to distribute loaders in the [name]-loader format. Consider this when publishing your loader. Let's say you've published your loader under a mp3-loader package on NPM. This is how you would use it in your project:

npm install mp3-loader

1// webpack.config.js
2module.exports = {
3  module: {
4    rules: [
5      {
6        test: /\.mp3$/,
7        use: ['babel-loader', 'mp3-loader'],
8      },
9    ],
10  },
11}

Notice that you don't have to import your loader, webpack will resolve it from node_modules automatically.

Trying your loader

To see the mp3-loader in action, run the webpack CLI command, given you've configured your webpack.config.js to use the custom loader:

1$ npx webpack
2asset audio.mp3 2.38 MiB [compared for emit] [from: src/audio.mp3] (auxiliary name: main)
3asset main.js 858 KiB [compared for emit] (name: main)
4webpack 5.37.0 compiled successfully in 1347 ms

You can inspect the end result of the mp3-loader in this example repository:

Testing your loader

Since loaders depend on the compilation context, I recommend testing them as a part of webpack's compilation, making such tests integration tests. Your tests' expectations will depend on what your loader does, so make sure to model them accordingly.

In the case of our mp3-loader we expect two things to happen:

• Imported audio file must be emitted in the build assets;
• Compiled code must create an <audio> React component.

Let's reflect those expectations in a test:

1// test/mp3-loader.test.js
2const path = require('path')
3const webpack = require('webpack')
4const { createFsFromVolume, Volume } = require('memfs')
5
6// A custom wrapper to promisify webpack compilation.
7function compileAsync(compiler) {
8  return new Promise((resolve, reject) => {
9    compiler.run((error, stats) => {
10      if (error || stats.hasErrors()) {
11        const resolvedError = error || stats.toJson('errors-only')[0]
12        reject(resolvedError.message)
13      }
14
15      resolve(stats)
16    })
17  })
18}
19
20it('converts "*.mp3" import into an audio player', async () => {
21  // Configure a webpack compiler.
22  const compiler = webpack({
23    mode: 'development',
24    entry: path.resolve(__dirname, '../src/index.js'),
25    output: {
26      filename: 'index.js',
27    },
28    module: {
29      rules: [
30        {
31          test: /\.mp3$/,
32          use: ['babel-loader', require.resolve('../src/mp3-loader.js')],
33        },
34        {
35          test: /\.js$/,
36          use: ['babel-loader'],
37        },
38      ],
39    },
40  })
41
42  // Create an in-memory file system so that the build assets
43  // are not emitted to disk during test runs.
44  const memoryFs = createFsFromVolume(new Volume())
45  compiler.outputFileSystem = memoryFs
46
47  // Compile the bundle.
48  await compileAsync(compiler)
49
50  // Expect the imported audio file to be emitted alongside the build.
51  expect(compiler.outputFileSystem.existsSync('dist/audio.mp3')).toEqual(true)
52
53  // Expect the compiled code to create an "audio" element in React.
54  const compiledCode = compiler.outputFileSystem.readFileSync(
55    'dist/index.js',
56    'utf8'
57  )
58  expect(compiledCode).toContain('.createElement(\\"audio\\"')
59})

Recipes

Loader options

Loaders can accept options to change their behavior.

You can pass options to a loader in the webpack configuration:

1// webpack.config.js
2module.exports = {
3  module: {
4    rules: [
5      {
6        test: /\.mp3$/,
7        use: [
8          {
9            loader: 'mp3-loader',
10            options: {
11              maxSizeBytes: 1000000,
12            },
13          },
14        ],
15      },
16    ],
17  },
18}

Above, we've created a custom maxSizeBytes option for the loader. The options object can later be accessed in the loader by calling this.getOptions():

1// src/mp3-loader.js
2module.exports = function (source) {
3  const options = this.getOptions()
4  console.log(options.maxSizeBytes)
5  // ...parametrize your loader's behavior.
6}

Validating options

It's a good tone to validate your loader's options to prevent improper usage and narrow down issues.

webpack comes with a schema-utils dependency installed, which you can use to validate your loader's options:

1// src/mp3-loader.js
2const { validate } = require('schema-utils')
3
4// Describe your loader's options in a JSON Schema.
5const schema = {
6  properties: {
7    maxSizeBytes: {
8      type: 'number',
9    },
10  },
11}
12
13module.exports = function (source) {
14  const options = this.getOptions()
15
16  // Validate the options early in your loader.
17  validate(schema, options)
18
19  // ...the rest of your loader.
20}

The schema object is defined using the JSON Schema format.

Logger

Here's an example of how we can take the custom maxSizeBytes loader option into account, and produce a warning if the imported audio file exceeds the maximum allowed size:

1// src/mp3-loader.js
2const fs = require('fs')
3
4module.exports = function (source) {
5  const options = this.getOptions()
6  const logger = this.getLogger()
7  const assetStats = fs.statSync(this.resourcePath)
8
9  if (assetStats.size > options.maxSizeBytes) {
10    logger.warn('Imported MP3 file is too large!')
11  }
12}

Learn more about the webpack's logger interface.

Context properties

See all the available properties by inspecting this in your loader.

Context methods

See all the available methods by inspecting this in your loader.

References

The topic of webpack customization is rather undocumented and difficult to scout for. When learning about writing webpack loaders, I've referred to multiple existing loaders that I've used in the past to see how they work. Below you can find the list of such loaders to reference yourself:

• Featured webpack loaders
• file-loader
• css-loader