### What does this PR do? Fixes #23314 where `zlib.zstdCompress()` created data that caused an out-of-memory error when decompressed with `Bun.zstdDecompressSync()`. #### 1. `zlib.zstdCompress()` now sets `pledgedSrcSize` The async convenience method now automatically sets the `pledgedSrcSize` option to the input buffer size. This ensures the compressed frame includes the content size in the header, making sync and async compression produce identical output. **Node.js compatibility**: `pledgedSrcSize` is a documented Node.js option: - [`vendor/node/doc/api/zlib.md:754-758`](https://github.com/oven-sh/bun/blob/main/vendor/node/doc/api/zlib.md#L754-L758) - [`vendor/node/lib/zlib.js:893`](https://github.com/oven-sh/bun/blob/main/vendor/node/lib/zlib.js#L893) - [`vendor/node/src/node_zlib.cc:890-904`](https://github.com/oven-sh/bun/blob/main/vendor/node/src/node_zlib.cc#L890-L904) #### 2. Added `bun.zstd.decompressAlloc()` - centralized safe decompression Created a new function in `src/deps/zstd.zig` that handles decompression in one place with automatic safety features: - **Handles unknown content sizes**: Automatically switches to streaming decompression when the zstd frame doesn't include content size (e.g., from streams without `pledgedSrcSize`) - **16MB safety limit**: For security, if the reported decompressed size exceeds 16MB, streaming decompression is used instead of blindly trusting the header - **Fast path for small files**: Still uses efficient pre-allocation for files < 16MB with known sizes This centralized fix automatically protects: - `Bun.zstdDecompressSync()` / `Bun.zstdDecompress()` - `StandaloneModuleGraph` source map decompression - Any other code using `bun.zstd` decompression ### How did you verify your code works? **Before:** ```typescript const input = "hello world"; // Async compression const compressed = await new Promise((resolve, reject) => { zlib.zstdCompress(input, (err, result) => { if (err) reject(err); else resolve(result); }); }); // This would fail with "Out of memory" const decompressed = Bun.zstdDecompressSync(compressed); ``` **Error**: `RangeError: Out of memory` (tried to allocate UINT64_MAX bytes) **After:** ```typescript const input = "hello world"; // Async compression (now includes content size) const compressed = await new Promise((resolve, reject) => { zlib.zstdCompress(input, (err, result) => { if (err) reject(err); else resolve(result); }); }); // ✅ Works! Falls back to streaming decompression if needed const decompressed = Bun.zstdDecompressSync(compressed); console.log(decompressed.toString()); // "hello world" ``` **Tests:** - ✅ All existing tests pass - ✅ New regression tests for async/sync compression compatibility (`test/regression/issue/23314/zstd-async-compress.test.ts`) - ✅ Test for large (>16MB) decompression using streaming (`test/regression/issue/23314/zstd-large-decompression.test.ts`) - ✅ Test for various input sizes and types (`test/regression/issue/23314/zstd-large-input.test.ts`) **Security:** The 16MB safety limit protects against malicious zstd frames that claim huge decompressed sizes in the header, preventing potential OOM attacks. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Co-authored-by: Claude Bot <claude-bot@bun.sh> Co-authored-by: Claude <noreply@anthropic.com>
Tests
Finding tests
Tests are located in the test/ directory and are organized using the following structure:
test/js/- tests for JavaScript APIs.cli/- tests for commands, configs, and stdout.bundler/- tests for the transpiler/bundler.regression/- tests that reproduce a specific issue.harness.ts- utility functions that can be imported from any test.
The tests in test/js/ directory are further categorized by the type of API.
test/js/bun/- tests forBun-specific APIs.node/- tests for Node.js APIs.web/- tests for Web APIs, likefetch().first_party/- tests for npm packages that are built-in, likeundici.third_party/- tests for npm packages that are not built-in, but are popular, likeesbuild.
Running tests
To run a test, use Bun's built-in test command: bun test.
bun test # Run all tests
bun test js/bun # Only run tests in a directory
bun test sqlite.test.ts # Only run a specific test
If you encounter lots of errors, try running bun install, then trying again.
Writing tests
Tests are written in TypeScript (preferred) or JavaScript using Jest's describe(), test(), and expect() APIs.
import { describe, test, expect } from "bun:test";
import { gcTick } from "harness";
describe("TextEncoder", () => {
test("can encode a string", async () => {
const encoder = new TextEncoder();
const actual = encoder.encode("bun");
await gcTick();
expect(actual).toBe(new Uint8Array([0x62, 0x75, 0x6E]));
});
});
If you are fixing a bug that was reported from a GitHub issue, remember to add a test in the test/regression/ directory.
// test/regression/issue/02005.test.ts
import { it, expect } from "bun:test";
it("regex literal should work with non-latin1", () => {
const text = "这是一段要替换的文字";
expect(text.replace(new RegExp("要替换"), "")).toBe("这是一段的文字");
expect(text.replace(/要替换/, "")).toBe("这是一段的文字");
});
In the future, a bot will automatically close or re-open issues when a regression is detected or resolved.
Zig tests
These tests live in various .zig files throughout Bun's codebase, leveraging Zig's builtin test keyword.
Currently, they're not run automatically nor is there a simple way to run all of them. We will make this better soon.
TypeScript
Test files should be written in TypeScript. The types in packages/bun-types should be updated to support all new APIs. Changes to the .d.ts files in packages/bun-types will be immediately reflected in test files; no build step is necessary.
Writing a test will often require using invalid syntax, e.g. when checking for errors when an invalid input is passed to a function. TypeScript provides a number of escape hatches here.
// @ts-expect-error- This should be your first choice. It tells TypeScript that the next line should fail typechecking.// @ts-ignore- Ignore the next line entirely.// @ts-nocheck- Put this at the top of the file to disable typechecking on the entire file. Useful for autogenerated test files, or when ignoring/disabling type checks an a per-line basis is too onerous.