Files
bun.sh/test
robobun d273f7fdde fix: zstd decompression with async-compressed data (#23314) (#23317)
### What does this PR do?

Fixes #23314 where `zlib.zstdCompress()` created data that caused an
out-of-memory error when decompressed with `Bun.zstdDecompressSync()`.

#### 1. `zlib.zstdCompress()` now sets `pledgedSrcSize`

The async convenience method now automatically sets the `pledgedSrcSize`
option to the input buffer size. This ensures the compressed frame
includes the content size in the header, making sync and async
compression produce identical output.

**Node.js compatibility**: `pledgedSrcSize` is a documented Node.js
option:
-
[`vendor/node/doc/api/zlib.md:754-758`](https://github.com/oven-sh/bun/blob/main/vendor/node/doc/api/zlib.md#L754-L758)
-
[`vendor/node/lib/zlib.js:893`](https://github.com/oven-sh/bun/blob/main/vendor/node/lib/zlib.js#L893)
-
[`vendor/node/src/node_zlib.cc:890-904`](https://github.com/oven-sh/bun/blob/main/vendor/node/src/node_zlib.cc#L890-L904)

#### 2. Added `bun.zstd.decompressAlloc()` - centralized safe
decompression

Created a new function in `src/deps/zstd.zig` that handles decompression
in one place with automatic safety features:

- **Handles unknown content sizes**: Automatically switches to streaming
decompression when the zstd frame doesn't include content size (e.g.,
from streams without `pledgedSrcSize`)
- **16MB safety limit**: For security, if the reported decompressed size
exceeds 16MB, streaming decompression is used instead of blindly
trusting the header
- **Fast path for small files**: Still uses efficient pre-allocation for
files < 16MB with known sizes

This centralized fix automatically protects:
- `Bun.zstdDecompressSync()` / `Bun.zstdDecompress()`
- `StandaloneModuleGraph` source map decompression
- Any other code using `bun.zstd` decompression

### How did you verify your code works?

**Before:**
```typescript
const input = "hello world";

// Async compression
const compressed = await new Promise((resolve, reject) => {
  zlib.zstdCompress(input, (err, result) => {
    if (err) reject(err);
    else resolve(result);
  });
});

// This would fail with "Out of memory"
const decompressed = Bun.zstdDecompressSync(compressed);
```
**Error**: `RangeError: Out of memory` (tried to allocate UINT64_MAX
bytes)

**After:**
```typescript
const input = "hello world";

// Async compression (now includes content size)
const compressed = await new Promise((resolve, reject) => {
  zlib.zstdCompress(input, (err, result) => {
    if (err) reject(err);
    else resolve(result);
  });
});

//  Works! Falls back to streaming decompression if needed
const decompressed = Bun.zstdDecompressSync(compressed);
console.log(decompressed.toString()); // "hello world"
```

**Tests:**
-  All existing tests pass
-  New regression tests for async/sync compression compatibility
(`test/regression/issue/23314/zstd-async-compress.test.ts`)
-  Test for large (>16MB) decompression using streaming
(`test/regression/issue/23314/zstd-large-decompression.test.ts`)
-  Test for various input sizes and types
(`test/regression/issue/23314/zstd-large-input.test.ts`)

**Security:**
The 16MB safety limit protects against malicious zstd frames that claim
huge decompressed sizes in the header, preventing potential OOM attacks.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-06 19:56:40 -07:00
..
2025-09-26 03:06:18 -07:00
2025-09-30 05:26:32 -07:00
2025-09-03 03:39:31 -07:00
2025-09-30 05:26:32 -07:00
2025-09-30 05:26:32 -07:00

Tests

Finding tests

Tests are located in the test/ directory and are organized using the following structure:

  • test/
    • js/ - tests for JavaScript APIs.
    • cli/ - tests for commands, configs, and stdout.
    • bundler/ - tests for the transpiler/bundler.
    • regression/ - tests that reproduce a specific issue.
    • harness.ts - utility functions that can be imported from any test.

The tests in test/js/ directory are further categorized by the type of API.

  • test/js/
    • bun/ - tests for Bun-specific APIs.
    • node/ - tests for Node.js APIs.
    • web/ - tests for Web APIs, like fetch().
    • first_party/ - tests for npm packages that are built-in, like undici.
    • third_party/ - tests for npm packages that are not built-in, but are popular, like esbuild.

Running tests

To run a test, use Bun's built-in test command: bun test.

bun test # Run all tests
bun test js/bun # Only run tests in a directory
bun test sqlite.test.ts # Only run a specific test

If you encounter lots of errors, try running bun install, then trying again.

Writing tests

Tests are written in TypeScript (preferred) or JavaScript using Jest's describe(), test(), and expect() APIs.

import { describe, test, expect } from "bun:test";
import { gcTick } from "harness";

describe("TextEncoder", () => {
  test("can encode a string", async () => {
    const encoder = new TextEncoder();
    const actual = encoder.encode("bun");
    await gcTick();
    expect(actual).toBe(new Uint8Array([0x62, 0x75, 0x6E]));
  });
});

If you are fixing a bug that was reported from a GitHub issue, remember to add a test in the test/regression/ directory.

// test/regression/issue/02005.test.ts

import { it, expect } from "bun:test";

it("regex literal should work with non-latin1", () => {
  const text = "这是一段要替换的文字";
  expect(text.replace(new RegExp("要替换"), "")).toBe("这是一段的文字");
  expect(text.replace(/要替换/, "")).toBe("这是一段的文字");
});

In the future, a bot will automatically close or re-open issues when a regression is detected or resolved.

Zig tests

These tests live in various .zig files throughout Bun's codebase, leveraging Zig's builtin test keyword.

Currently, they're not run automatically nor is there a simple way to run all of them. We will make this better soon.

TypeScript

Test files should be written in TypeScript. The types in packages/bun-types should be updated to support all new APIs. Changes to the .d.ts files in packages/bun-types will be immediately reflected in test files; no build step is necessary.

Writing a test will often require using invalid syntax, e.g. when checking for errors when an invalid input is passed to a function. TypeScript provides a number of escape hatches here.

  • // @ts-expect-error - This should be your first choice. It tells TypeScript that the next line should fail typechecking.
  • // @ts-ignore - Ignore the next line entirely.
  • // @ts-nocheck - Put this at the top of the file to disable typechecking on the entire file. Useful for autogenerated test files, or when ignoring/disabling type checks an a per-line basis is too onerous.