Node.js Streams

If you work with files or HTTP in Node.js, you will eventually run into streams. The concept can feel intimidating at first, but the foundational principle is straightforward: a stream lets data move through your application piece by piece, instead of loading the entire payload into memory at once.

Consider an HTTP server receiving a 5 GB file upload. If you tried to load all 5 GB into memory before writing it to disk, Node would quickly exhaust its available heap and crash. With streams, the server handles small pieces sequentially as they arrive over the network:

Client -> HTTP Request -> small chunks -> Disk File

The data is kept in constant motion rather than pooled in RAM.

---

Chunks, Buffers, and Stream Types

A chunk is simply a fragment of the overall payload. Instead of receiving 5 GB in one shot, Node receives a sequence of smaller pieces:

chunk 1 -> 64 KB
chunk 2 -> 64 KB
chunk 3 -> 64 KB
...

The exact size of each chunk depends on network packets, operating system buffers, and internal stream settings, so your code should never depend on a fixed chunk size. For binary data, Node represents these chunks as Buffer instances.

Every stream fits into one of two fundamental roles:

• Readable streams produce data.
• Writable streams consume data.

In an HTTP upload, the incoming request (req) is readable because the client is sending data inward. A file stream created with fs.createWriteStream is writable because your application sends data into the file on disk. Connecting them means reading chunks from the source and passing them directly to the destination:

HTTP Request (Readable) -> chunks -> File Stream (Writable)

---

Handling Chunks: From Manual Events to pipe()

In Node's http module, the req object passed to the request handler is an instance of IncomingMessage, which implements the Readable stream interface:

import { createServer } from "node:http";

const server = createServer((req, res) => {
  req.on("data", (chunk) => {
    console.log("Received:", chunk.length, "bytes");
  });

  req.on("end", () => {
    console.log("Finished");
    res.end("Done");
  });

  req.on("error", (err) => {
    console.error("Request error:", err.message);
  });
});

server.listen(3000);

Readable streams emit events as data arrives:

• "data" fires each time a new chunk is available.
• "end" fires once when there are no more chunks left to read.
• "error" fires if the connection drops or reading fails.

To save an upload to disk, you can manually forward those chunks to a writable file stream:

import { createServer } from "node:http";
import { createWriteStream } from "node:fs";

const server = createServer((req, res) => {
  const file = createWriteStream("output.txt");

  req.on("data", (chunk) => {
    file.write(chunk);
  });

  req.on("end", () => {
    file.end();
    res.end("Done");
  });
});

server.listen(3000);

Notice what this code actually does. It never asks for the complete 5 GB file. It asks for the next chunk, writes that chunk to disk, and repeats until the stream ends.

Writing this boilerplate manually gets tedious, and that is where .pipe() comes in. The manual event listeners above can be replaced with a single call:

req.pipe(file);

The .pipe() method attaches the readable source to the writable destination, reading incoming chunks and writing them out automatically until the source finishes.

---

Symmetry, Backpressure, and Flow Control

Because streams share a unified interface, this same pattern works symmetrically in both directions.

During an upload, the request produces data and the file consumes it:

// Client -> Disk
req.pipe(file);

During a download, the roles reverse. The file on disk becomes the readable source, and the HTTP response (res, which is writable) becomes the destination:

// Disk -> Client
file.pipe(res);

The direction changes, but the mechanism remains identical: Readable.pipe(Writable).

There is an important detail hiding underneath .pipe(): backpressure.

Imagine a fast client sending data over a high-speed connection while your server is writing to a slow disk. If the network produces data faster than the disk can write it, those unwritten chunks must wait in memory:

Network (FAST) -> Request -> [ RAM Buffer fills up ] -> Disk (SLOW)

Without flow control, data accumulates in RAM and will eventually crash the process. Backpressure is the mechanism that prevents this. When a writable stream's internal buffer fills up, its .write() method returns false, signaling the readable stream to pause. Once the destination flushes its buffer, it signals the source to resume.

Calling .pipe() handles this flow control automatically, which is why piping is preferable to manually wiring data and write() calls.

---

The Mental Model

When working with Node.js streams, you do not need to memorize every method upfront. The primary model to keep in mind is:

              chunks
                 v
Readable -----------------> Writable
 (Source)      .pipe()    (Destination)

• Upload: Client -> HTTP Request (Readable) -> File (Writable)
• Download: File (Readable) -> HTTP Response (Writable) -> Client
• pipe(): Connects the source to the destination and manages backpressure.

Every advanced stream feature in Node—such as stream/promises, the pipeline() helper, transform streams for compression or encryption, and custom buffering—builds directly on top of this chunk-by-chunk flow.