mirror of
https://github.com/oven-sh/bun
synced 2026-02-09 10:28:47 +00:00
5f1ca176cd
### What does this PR do? Fixes data loss when reading large amounts of data from subprocess pipes on Windows, a regression introduced by the libuv 1.51.0 upgrade in commite3783c244f. ### The Problem When piping large data through a subprocess on Windows (e.g., `process.stdin.pipe(process.stdout)`), Bun randomly loses ~73KB of data out of 1MB, receiving only ~974KB instead of the full 1048576 bytes. The subprocess correctly receives all 1MB on stdin, but the parent process loses data when reading from the subprocess stdout. ### Root Cause Analysis #### libuv 1.51.0 Change The libuv 1.51.0 upgrade (commit [libuv/libuv@727ee723](727ee7237e)) changed Windows pipe reading behavior: **Before:** libuv would call `PeekNamedPipe` to check available bytes, then read exactly that amount. **After:** libuv attempts immediate non-blocking reads (up to 65536 bytes) before falling back to async reads. If less data is available than requested, it returns what's available and signals `more=0`, causing the read loop to break. This optimization introduces **0-byte reads** when data isn't immediately available, which are delivered to Bun's read callback. #### The Race Condition When Bun's `WindowsBufferedReader` called `onRead(.drained)` for these 0-byte reads, it created a race condition. Debug logs clearly show the issue: **Error case (log.txt):** ``` Line 79-80: onStreamRead = 0 (drained) Line 81: filesink closes (stdin closes) Line 85: onStreamRead = 6024 ← Should be 74468! Line 89: onStreamRead = -4095 (EOF) ``` **Success case (success.log.txt):** ``` Line 79-80: onStreamRead = 0 (drained) Line 81: filesink closes (stdin closes) Line 85: onStreamRead = 74468 ← Full chunk! Line 89-90: onStreamRead = 0 (drained) Line 91: onStreamRead = 6024 Line 95: onStreamRead = -4095 (EOF) ``` When stdin closes while a 0-byte drained read is pending, the next read returns truncated data (6024 bytes instead of 74468 bytes). ### The Fix Two changes to `WindowsBufferedReader` in `src/io/PipeReader.zig`: #### 1. Ignore 0-byte reads (line 937-940) Don't call `onRead(.drained)` for 0-byte reads. Just return and let libuv queue the next read. This prevents the race condition that causes truncated reads. ```zig 0 => { // With libuv 1.51.0+, calling onRead(.drained) here causes a race condition // where subsequent reads return truncated data. Just ignore 0-byte reads. return; }, ``` #### 2. Defer `has_inflight_read` flag clearing (line 827-839) Clear the flag **after** the read callback completes, not before. This prevents libuv from starting a new overlapped read operation while we're still processing the current data buffer, which could cause memory corruption per the libuv commit message: > "Starting a new read after uv_read_cb returns causes memory corruption on the OVERLAPPED read_req if uv_read_stop+uv_read_start was called during the callback" ```zig const result = onReadChunkFn(this.parent, buf, hasMore); // Clear has_inflight_read after the callback completes this.flags.has_inflight_read = false; return result; ``` ### How to Test Run the modified test in `test/js/bun/spawn/spawn-stdin-readable-stream.test.ts`: ```js test("ReadableStream with very large chunked data", async () => { const chunkSize = 64 * 1024; // 64KB chunks const numChunks = 16; // 1MB total const chunk = Buffer.alloc(chunkSize, "x"); const stream = new ReadableStream({ pull(controller) { if (pushedChunks < numChunks) { controller.enqueue(chunk); pushedChunks++; } else { controller.close(); } }, }); await using proc = spawn({ cmd: [bunExe(), "-e", ` let length = 0; process.stdin.on('data', (data) => length += data.length); process.once('beforeExit', () => console.error(length)); process.stdin.pipe(process.stdout) `], stdin: stream, stdout: "pipe", env: bunEnv, }); const text = await proc.stdout.text(); expect(text.length).toBe(chunkSize * numChunks); // Should be 1048576 }); ``` **Before fix:** Randomly fails with ~974KB instead of 1MB **After fix:** Consistently passes with full 1MB Run ~100 times to verify the race condition is fixed. ### Related Issues This may also fix #23071 (Windows scripts hanging), though that issue needs separate verification. ### Why Draft? Marking as draft for Windows testing by the team. The fix is based on detailed debug log analysis showing the exact race condition, but needs verification on Windows CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude Bot <claude-bot@bun.sh> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Bun
Read the docs →
What is Bun?
Bun is an all-in-one toolkit for JavaScript and TypeScript apps. It ships as a single executable called bun.
At its core is the Bun runtime, a fast JavaScript runtime designed as a drop-in replacement for Node.js. It's written in Zig and powered by JavaScriptCore under the hood, dramatically reducing startup times and memory usage.
bun run index.tsx # TS and JSX supported out-of-the-box
The bun command-line tool also implements a test runner, script runner, and Node.js-compatible package manager. Instead of 1,000 node_modules for development, you only need bun. Bun's built-in tools are significantly faster than existing options and usable in existing Node.js projects with little to no changes.
bun test # run tests
bun run start # run the `start` script in `package.json`
bun install <pkg> # install a package
bunx cowsay 'Hello, world!' # execute a package
Install
Bun supports Linux (x64 & arm64), macOS (x64 & Apple Silicon) and Windows (x64).
Linux users — Kernel version 5.6 or higher is strongly recommended, but the minimum is 5.1.
x64 users — if you see "illegal instruction" or similar errors, check our CPU requirements
# with install script (recommended)
curl -fsSL https://bun.com/install | bash
# on windows
powershell -c "irm bun.com/install.ps1 | iex"
# with npm
npm install -g bun
# with Homebrew
brew tap oven-sh/bun
brew install bun
# with Docker
docker pull oven/bun
docker run --rm --init --ulimit memlock=-1:-1 oven/bun
Upgrade
To upgrade to the latest version of Bun, run:
bun upgrade
Bun automatically releases a canary build on every commit to main. To upgrade to the latest canary build, run:
bun upgrade --canary
Quick links
-
Intro
-
Templating
-
CLI
-
Runtime
-
Package manager
-
Bundler
-
Test runner
-
Package runner
-
API
- HTTP server (
Bun.serve) - WebSockets
- Workers
- Binary data
- Streams
- File I/O (
Bun.file) - import.meta
- SQLite (
bun:sqlite) - PostgreSQL (
Bun.sql) - Redis (
Bun.redis) - S3 Client (
Bun.s3) - FileSystemRouter
- TCP sockets
- UDP sockets
- Globals
- $ Shell
- Child processes (spawn)
- Transpiler (
Bun.Transpiler) - Hashing
- Colors (
Bun.color) - Console
- FFI (
bun:ffi) - C Compiler (
bun:fficc) - HTMLRewriter
- Testing (
bun:test) - Cookies (
Bun.Cookie) - Utils
- Node-API
- Glob (
Bun.Glob) - Semver (
Bun.semver) - DNS
- fetch API extensions
- HTTP server (
Guides
-
Binary
- Convert a Blob to a string
- Convert a Buffer to a blob
- Convert a Blob to a DataView
- Convert a Buffer to a string
- Convert a Blob to a ReadableStream
- Convert a Blob to a Uint8Array
- Convert a DataView to a string
- Convert a Uint8Array to a Blob
- Convert a Blob to an ArrayBuffer
- Convert an ArrayBuffer to a Blob
- Convert a Buffer to a Uint8Array
- Convert a Uint8Array to a Buffer
- Convert a Uint8Array to a string
- Convert a Buffer to an ArrayBuffer
- Convert an ArrayBuffer to a Buffer
- Convert an ArrayBuffer to a string
- Convert a Uint8Array to a DataView
- Convert a Buffer to a ReadableStream
- Convert a Uint8Array to an ArrayBuffer
- Convert an ArrayBuffer to a Uint8Array
- Convert an ArrayBuffer to an array of numbers
- Convert a Uint8Array to a ReadableStream
-
Ecosystem
- Use React and JSX
- Use EdgeDB with Bun
- Use Prisma with Bun
- Add Sentry to a Bun app
- Create a Discord bot
- Run Bun as a daemon with PM2
- Use Drizzle ORM with Bun
- Build an app with Nuxt and Bun
- Build an app with Qwik and Bun
- Build an app with Astro and Bun
- Build an app with Remix and Bun
- Build a frontend using Vite and Bun
- Build an app with Next.js and Bun
- Run Bun as a daemon with systemd
- Deploy a Bun application on Render
- Build an HTTP server using Hono and Bun
- Build an app with SvelteKit and Bun
- Build an app with SolidStart and Bun
- Build an HTTP server using Elysia and Bun
- Build an HTTP server using StricJS and Bun
- Containerize a Bun application with Docker
- Build an HTTP server using Express and Bun
- Use Neon Postgres through Drizzle ORM
- Server-side render (SSR) a React component
- Read and write data to MongoDB using Mongoose and Bun
- Use Neon's Serverless Postgres with Bun
-
HTMLRewriter
-
HTTP
- Hot reload an HTTP server
- Common HTTP server usage
- Write a simple HTTP server
- Configure TLS on an HTTP server
- Send an HTTP request using fetch
- Proxy HTTP requests using fetch()
- Start a cluster of HTTP servers
- Stream a file as an HTTP Response
- fetch with unix domain sockets in Bun
- Upload files via HTTP using FormData
- Streaming HTTP Server with Async Iterators
- Streaming HTTP Server with Node.js Streams
-
Install
- Add a dependency
- Add a Git dependency
- Add a peer dependency
- Add a trusted dependency
- Add a development dependency
- Add a tarball dependency
- Add an optional dependency
- Generate a yarn-compatible lockfile
- Configuring a monorepo using workspaces
- Install a package under a different name
- Install dependencies with Bun in GitHub Actions
- Using bun install with Artifactory
- Configure git to diff Bun's lockb lockfile
- Override the default npm registry for bun install
- Using bun install with an Azure Artifacts npm registry
- Migrate from npm install to bun install
- Configure a private registry for an organization scope with bun install
-
Process
-
Read file
-
Runtime
- Delete files
- Run a Shell Command
- Import a JSON file
- Import a TOML file
- Set a time zone in Bun
- Set environment variables
- Re-map import paths
- Delete directories
- Read environment variables
- Import a HTML file as text
- Install and run Bun in GitHub Actions
- Debugging Bun with the web debugger
- Install TypeScript declarations for Bun
- Debugging Bun with the VS Code extension
- Inspect memory usage using V8 heap snapshots
- Define and replace static globals & constants
- Codesign a single-file JavaScript executable on macOS
-
Streams
- Convert a ReadableStream to JSON
- Convert a ReadableStream to a Blob
- Convert a ReadableStream to a Buffer
- Convert a ReadableStream to a string
- Convert a ReadableStream to a Uint8Array
- Convert a ReadableStream to an array of chunks
- Convert a Node.js Readable to JSON
- Convert a ReadableStream to an ArrayBuffer
- Convert a Node.js Readable to a Blob
- Convert a Node.js Readable to a string
- Convert a Node.js Readable to an Uint8Array
- Convert a Node.js Readable to an ArrayBuffer
-
Test
- Spy on methods in
bun test - Bail early with the Bun test runner
- Mock functions in
bun test - Run tests in watch mode with Bun
- Use snapshot testing in
bun test - Skip tests with the Bun test runner
- Using Testing Library with Bun
- Update snapshots in
bun test - Run your tests with the Bun test runner
- Set the system time in Bun's test runner
- Set a per-test timeout with the Bun test runner
- Migrate from Jest to Bun's test runner
- Write browser DOM tests with Bun and happy-dom
- Mark a test as a "todo" with the Bun test runner
- Re-run tests multiple times with the Bun test runner
- Generate code coverage reports with the Bun test runner
- import, require, and test Svelte components with bun test
- Set a code coverage threshold with the Bun test runner
- Spy on methods in
-
Util
- Generate a UUID
- Hash a password
- Escape an HTML string
- Get the current Bun version
- Encode and decode base64 strings
- Compress and decompress data with gzip
- Sleep for a fixed number of milliseconds
- Detect when code is executed with Bun
- Check if two objects are deeply equal
- Compress and decompress data with DEFLATE
- Get the absolute path to the current entrypoint
- Get the directory of the current file
- Check if the current file is the entrypoint
- Get the file name of the current file
- Convert a file URL to an absolute path
- Convert an absolute path to a file URL
- Get the absolute path of the current file
- Get the path to an executable bin file
-
WebSocket
-
Write file
Contributing
Refer to the Project > Contributing guide to start contributing to Bun.
License
Refer to the Project > License page for information about Bun's licensing.