Commit Graph

162 Commits

Author SHA1 Message Date
Jarred Sumner
b3cfaab07f Fix: after pausing stdin, a subprocess should be able to read from stdin (#23341)
Fixes #23333, Fixes #13978

### What does this PR do?

### How did you verify your code works?

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: pfg <pfg@pfg.pw>
Co-authored-by: Zack Radisic <zack@theradisic.com>
2025-10-09 19:04:41 -07:00
robobun
5f1ca176cd fix(windows): prevent data loss in pipe reads after libuv 1.51.0 upgrade (#23340)
### What does this PR do?

Fixes data loss when reading large amounts of data from subprocess pipes
on Windows, a regression introduced by the libuv 1.51.0 upgrade in
commit e3783c244f.

### The Problem

When piping large data through a subprocess on Windows (e.g.,
`process.stdin.pipe(process.stdout)`), Bun randomly loses ~73KB of data
out of 1MB, receiving only ~974KB instead of the full 1048576 bytes.

The subprocess correctly receives all 1MB on stdin, but the parent
process loses data when reading from the subprocess stdout.

### Root Cause Analysis

#### libuv 1.51.0 Change

The libuv 1.51.0 upgrade (commit
[libuv/libuv@727ee723](727ee7237e))
changed Windows pipe reading behavior:

**Before:** libuv would call `PeekNamedPipe` to check available bytes,
then read exactly that amount.

**After:** libuv attempts immediate non-blocking reads (up to 65536
bytes) before falling back to async reads. If less data is available
than requested, it returns what's available and signals `more=0`,
causing the read loop to break.

This optimization introduces **0-byte reads** when data isn't
immediately available, which are delivered to Bun's read callback.

#### The Race Condition

When Bun's `WindowsBufferedReader` called `onRead(.drained)` for these
0-byte reads, it created a race condition. Debug logs clearly show the
issue:

**Error case (log.txt):**
```
Line 79-80: onStreamRead = 0 (drained)
Line 81:    filesink closes (stdin closes)
Line 85:    onStreamRead = 6024        ← Should be 74468!
Line 89:    onStreamRead = -4095 (EOF)
```

**Success case (success.log.txt):**
```
Line 79-80: onStreamRead = 0 (drained)
Line 81:    filesink closes (stdin closes)
Line 85:    onStreamRead = 74468       ← Full chunk!
Line 89-90: onStreamRead = 0 (drained)
Line 91:    onStreamRead = 6024
Line 95:    onStreamRead = -4095 (EOF)
```

When stdin closes while a 0-byte drained read is pending, the next read
returns truncated data (6024 bytes instead of 74468 bytes).

### The Fix

Two changes to `WindowsBufferedReader` in `src/io/PipeReader.zig`:

#### 1. Ignore 0-byte reads (line 937-940)

Don't call `onRead(.drained)` for 0-byte reads. Just return and let
libuv queue the next read. This prevents the race condition that causes
truncated reads.

```zig
0 => {
    // With libuv 1.51.0+, calling onRead(.drained) here causes a race condition
    // where subsequent reads return truncated data. Just ignore 0-byte reads.
    return;
},
```

#### 2. Defer `has_inflight_read` flag clearing (line 827-839)

Clear the flag **after** the read callback completes, not before. This
prevents libuv from starting a new overlapped read operation while we're
still processing the current data buffer, which could cause memory
corruption per the libuv commit message:

> "Starting a new read after uv_read_cb returns causes memory corruption
on the OVERLAPPED read_req if uv_read_stop+uv_read_start was called
during the callback"

```zig
const result = onReadChunkFn(this.parent, buf, hasMore);
// Clear has_inflight_read after the callback completes
this.flags.has_inflight_read = false;
return result;
```

### How to Test

Run the modified test in
`test/js/bun/spawn/spawn-stdin-readable-stream.test.ts`:

```js
test("ReadableStream with very large chunked data", async () => {
  const chunkSize = 64 * 1024; // 64KB chunks
  const numChunks = 16; // 1MB total
  const chunk = Buffer.alloc(chunkSize, "x");

  const stream = new ReadableStream({
    pull(controller) {
      if (pushedChunks < numChunks) {
        controller.enqueue(chunk);
        pushedChunks++;
      } else {
        controller.close();
      }
    },
  });

  await using proc = spawn({
    cmd: [bunExe(), "-e", `
      let length = 0;
      process.stdin.on('data', (data) => length += data.length);
      process.once('beforeExit', () => console.error(length));
      process.stdin.pipe(process.stdout)
    `],
    stdin: stream,
    stdout: "pipe",
    env: bunEnv,
  });

  const text = await proc.stdout.text();
  expect(text.length).toBe(chunkSize * numChunks); // Should be 1048576
});
```

**Before fix:** Randomly fails with ~974KB instead of 1MB  
**After fix:** Consistently passes with full 1MB

Run ~100 times to verify the race condition is fixed.

### Related Issues

This may also fix #23071 (Windows scripts hanging), though that issue
needs separate verification.

### Why Draft?

Marking as draft for Windows testing by the team. The fix is based on
detailed debug log analysis showing the exact race condition, but needs
verification on Windows CI.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-10-07 18:33:34 -07:00
Jarred Sumner
ec050e6d6e Fix windows memory leak in error case for opening tty & pipe (#23235)
### What does this PR do?

### How did you verify your code works?
2025-10-06 06:16:47 -07:00
Michael H
ba20670da3 implement pnpm migration (#22262)
### What does this PR do?

fixes #7157, fixes #14662

migrates pnpm-workspace.yaml data to package.json & converts
pnpm-lock.yml to bun.lock

---

### How did you verify your code works?

manually, tests and real world examples

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-09-27 00:45:29 -07:00
Meghan Denny
5a709a2dbf node:tty: use terminal VT mode on Windows (#21161)
mirrors: https://github.com/nodejs/node/pull/58358
2025-09-25 19:58:44 -07:00
taylor.fish
edf13bd91d Refactor BabyList (#22502)
(For internal tracking: fixes STAB-1129, STAB-1145, STAB-1146,
STAB-1150, STAB-1126, STAB-1147, STAB-1148, STAB-1149, STAB-1158)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-09-09 20:41:10 -07:00
taylor.fish
437e15bae5 Replace catch bun.outOfMemory() with safer alternatives (#22141)
Replace `catch bun.outOfMemory()`, which can accidentally catch
non-OOM-related errors, with either `bun.handleOom` or a manual `catch
|err| switch (err)`.

(For internal tracking: fixes STAB-1070)

---------

Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-08-26 12:50:25 -07:00
Zack Radisic
0845231a1e Fix pipeline stack errors on Windows (#21800)
### What does this PR do?

### How did you verify your code works?

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-14 18:03:26 -07:00
taylor.fish
41b1efe12c Rename disabled parameter in Output.scoped (#21769)
It's very confusing.

(For internal tracking: fixes STAB-977)
2025-08-11 20:19:34 -07:00
Meghan Denny
64361eb964 zig: delete deprecated bun.jsc.Maybe (#21327)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: taylor.fish <contact@taylor.fish>
2025-07-25 12:38:06 -07:00
taylor.fish
07cd45deae Refactor Zig imports and file structure (part 1) (#21270)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-07-22 17:51:38 -07:00
pfg
83760fc446 Sort imports in all files (#21119)
Co-authored-by: taylor.fish <contact@taylor.fish>
2025-07-21 13:26:47 -07:00
Zack Radisic
41d10ed01e Refactor shell and fix some bugs (#20476) 2025-06-19 18:47:00 -07:00
Jarred Sumner
8750f0b884 Add FileRoute for serving files (#20198)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
2025-06-10 19:41:21 -07:00
Jarred Sumner
eddee1b8cb *anyopaque -> mach_port_t (#20243) 2025-06-07 00:30:58 -07:00
Jarred Sumner
da5fc817d1 Change copy to a .len += 2025-06-04 00:25:57 -07:00
Jarred Sumner
1bd44e9ce7 Fixes #18239 (#20152) 2025-06-03 13:23:12 -07:00
Jarred Sumner
71c14fac7b Split EventLoop into many more files (#20134)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2025-06-01 20:43:30 -07:00
Jarred Sumner
390798c172 Fix memory leak in Bun.spawn (#20095)
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-05-31 20:06:22 -07:00
Dylan Conway
ce8767cdc8 add HTTPParser (#20049)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-05-31 16:21:08 -07:00
Jarred Sumner
3c37f25b65 Avoid allocating 16 KB to read a potentially empty buffer (#20101) 2025-05-31 14:00:51 -07:00
Meghan Denny
e0852fd651 fix memory leak when pipe Bun.spawn stdio is never read repeatedly (#20102)
Co-authored-by: nektro <5464072+nektro@users.noreply.github.com>
2025-05-31 11:16:49 -07:00
Jarred Sumner
3ea6133c46 CI: Remove unused top-level decls in formatter in zig (#19879)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-23 22:49:48 -07:00
Jarred Sumner
69be630aea WebKit Upgrade (#19839)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Ben Grant <ben@bun.sh>
Co-authored-by: 190n <7763597+190n@users.noreply.github.com>
2025-05-22 21:12:43 -07:00
pfg
00a3cbd977 more child-process (#18688)
Co-authored-by: pfgithub <6010774+pfgithub@users.noreply.github.com>
2025-05-06 22:12:24 -07:00
chloe caruso
3349c995b5 no usingnamespace, organize jsc namespace, enable -fincremental (#19122)
Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>
2025-04-22 16:34:15 -07:00
chloe caruso
7d7512076b remove more usingnamespace (#19042) 2025-04-17 19:04:05 -07:00
chloe caruso
4ec410e0d7 internal: make @import("bun") work in zig (#19096) 2025-04-17 12:32:47 -07:00
chloe caruso
903706dccf file descriptor rewrite (#18790) 2025-04-15 09:37:11 -07:00
Jarred Sumner
879fdd7ef6 Bump Zig again (#18948) 2025-04-11 19:13:20 -07:00
chloe caruso
c1dc5f1b73 remove some usingnamespaces (#18765) 2025-04-10 14:16:30 -07:00
Jarred Sumner
a8cc31f8c4 Split shell into more files (#18793) 2025-04-04 23:38:43 -07:00
pfg
d9c77be90d node child process maxbuf support (#18293) 2025-04-03 17:03:26 -07:00
Jarred Sumner
4806e84cc1 Revert "remove many usingnamespace, introduce new ref count and ref leak debugging tools (#18353)"
This reverts commit a199b85f2b. It does not compile on Windows.
2025-04-01 08:35:51 -07:00
chloe caruso
a199b85f2b remove many usingnamespace, introduce new ref count and ref leak debugging tools (#18353) 2025-03-31 17:17:38 -07:00
pfg
f4ab2e4986 Remove two usingnamespace & ptrCast on function pointer (#18486) 2025-03-26 14:41:14 -07:00
190n
de4182f305 chore: upgrade zig to 0.14.0 (#17820)
Co-authored-by: 190n <7763597+190n@users.noreply.github.com>
Co-authored-by: Zack Radisic <56137411+zackradisic@users.noreply.github.com>
Co-authored-by: pfg <pfg@pfg.pw>
Co-authored-by: pfgithub <6010774+pfgithub@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-03-14 22:13:31 -07:00
chloe caruso
2b97d61deb chore: remove some trivial usage of usingnamespace (#17268) 2025-02-11 19:38:52 -08:00
pfg
0c8658b350 Fix test-fs-promises-writefile.js on windows (#17053) 2025-02-05 22:12:52 -08:00
Minsoo Choo
fa502506e5 Fix formatting for extern "c" (#16983) 2025-02-02 21:34:58 -08:00
Meghan Denny
26d3688e53 zig: update to 0.14.0-dev (#16862)
Co-authored-by: nektro <5464072+nektro@users.noreply.github.com>
2025-02-01 01:11:02 -08:00
Dylan Conway
b098c9ed89 fix(fs): WriteStream pending write fastpath (#16856)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-01-31 05:14:41 -08:00
Jarred Sumner
0768baf605 Fix Windows EOF double free (#16750) 2025-01-25 03:55:28 -08:00
Don Isaac
0db90583e8 chore: remove unused fifo.zig (#16184) 2025-01-06 12:38:09 -08:00
Jarred Sumner
aada6f930f Fix heap snapshots memory usage stats. Introduce estimateDirectMemoryUsageOf function in "bun:jsc" (#15790) 2024-12-16 20:16:23 -08:00
Jarred Sumner
3ce6ffa6be Make git dependencies faster + further optimize bun install (#15771) 2024-12-14 19:42:23 -08:00
Meghan Denny
ca6013acef move .clang-format up a folder so it affects all our c/cpp files (#14400)
Co-authored-by: nektro <nektro@users.noreply.github.com>
2024-10-08 23:04:05 -07:00
Ashcon Partovi
30dc72c17b bun run clang-format (#14148)
Co-authored-by: Electroid <Electroid@users.noreply.github.com>
2024-09-24 20:39:29 -07:00
Ciro Spaciari
b1ca81a10d fix(node:net, node:tls) add named pipe support on node:net and node:tls modules (#13838) 2024-09-12 14:06:45 -07:00
Ciro Spaciari
eb8ed27a4a fix(ipc/subprocess) (#13414)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-08-20 05:19:23 -07:00