Node.js - Streaming 101: Transform Streams Explained

Mar 7, 2025

Welcome back to Node.js Streaming 101! 🚀

If you haven’t read Part 1 (Intro to Streams) or Part 2 (Duplex Streams), I highly recommend checking them out first. Today, we’re talking about one of the most powerful and versatile types of streams in Node.js: Transform Streams.

🔄 What Are Transform Streams?

A Transform Stream is a special kind of Duplex Stream that modifies the data as it passes through. Think of it like a real-time filter:

  • Input: A stream of text
  • Processing: Convert it to uppercase
  • Output: The modified text

Unlike Duplex Streams (which send and receive separate data), Transform Streams take in data, modify it, and send it forward—all in one flow.

🔥 Where Are Transform Streams Used?

  • Compression & Decompression 📦 → (e.g., Gzip, Brotli)
  • Encryption & Decryption 🔐 → (e.g., AES, Hashing)
  • Data Processing 🛠 → (e.g., converting text formats, JSON transformation)
  • Real-time Modifications 🖊 → (e.g., replacing words in a text stream)

🏗 Creating a Simple Transform Stream

Let’s start by building a basic Transform Stream that converts text to uppercase.

import { Transform } from "stream";

class UppercaseTransform extends Transform {
  _transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
}

const transformStream = new UppercaseTransform();

process.stdin.pipe(transformStream).pipe(process.stdout);

🔍 What’s Happening?

  • We create a custom Transform Stream (UppercaseTransform).
  • _transform(chunk, encoding, callback):
    • Converts text to uppercase.
    • Pushes the transformed data forward.
  • We pipe process.stdin → transformStream → process.stdout.
  • Anything typed into the terminal gets converted to uppercase in real-time.

🧪 Try It Out!

  1. Run the script (node transform-stream.mjs).
  2. Type something and hit Enter.
  3. The text will be converted to uppercase!
hello world
HELLO WORLD

Boom! 🚀 You've just created your first Transform Stream.

🔗 Using Transform Streams with pipeline()

As we saw in Part 1, pipeline() makes handling streams much cleaner and manages error handling automatically.

Let’s refactor our uppercase stream using pipeline():

import { Transform, pipeline } from "stream/promises";

class UppercaseTransform extends Transform {
  _transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
}

async function processStream() {
  const transformStream = new UppercaseTransform();

  try {
    await pipeline(process.stdin, transformStream, process.stdout);
  } catch (err) {
    console.error("Pipeline failed:", err);
  }
}

processStream();

✅ Why use pipeline()?

  • Handles errors automatically.
  • Prevents memory leaks.
  • Supports async/await, making it modern and readable.

⚡ Real-World Example: Compressing Files with Gzip

A practical use of Transform Streams is file compression using Gzip.

import { createReadStream, createWriteStream } from "fs";
import { pipeline } from "stream/promises";
import { createGzip } from "zlib";

async function compressFile() {
  try {
    await pipeline(
      createReadStream("largefile.txt"),
      createGzip(), // Transform stream compresses the data
      createWriteStream("largefile.txt.gz")
    );
    console.log("File compressed successfully!");
  } catch (err) {
    console.error("Compression failed:", err);
  }
}

compressFile();

🔍 What’s Happening?

  • Read file → Compress with Gzip → Write to new file.
  • Uses createGzip(), which is a built-in Transform Stream.
  • Automatically handles backpressure & errors.

🎯 Final Thoughts

Transform Streams are incredibly powerful for modifying data on the fly. Whether you're compressing files, encrypting data, or processing real-time input, they give you full control over the data flow.

Missed the earlier parts? Check out:

Let me know if you have any questions, and happy coding! 🚀

diogoaoliveira.com