Commit Graph

94 Commits

Author SHA1 Message Date
pfg
05d0475c6c Update to zig 0.15.2 (#24204)
Fixes ENG-21287

Build times, from `bun run build && echo '//' >> src/main.zig && time
bun run build`

|Platform|0.14.1|0.15.2|Speedup|
|-|-|-|-|
|macos debug asan|126.90s|106.27s|1.19x|
|macos debug noasan|60.62s|50.85s|1.19x|
|linux debug asan|292.77s|241.45s|1.21x|
|linux debug noasan|146.58s|130.94s|1.12x|
|linux debug use_llvm=false|n/a|78.27s|1.87x|
|windows debug asan|177.13s|142.55s|1.24x|

Runtime performance:

- next build memory usage may have gone up by 5%. Otherwise seems the
same. Some code with writers may have gotten slower, especially one
instance of a counting writer and a few instances of unbuffered writers
that now have vtable overhead.
- File size reduced by 800kb (from 100.2mb to 99.4mb)

Improvements:

- `@export` hack is no longer needed for watch
- native x86_64 backend for linux builds faster. to use it, set use_llvm
false and no_link_obj false. also set `ASAN_OPTIONS=detect_leaks=0`
otherwise it will spam the output with tens of thousands of lines of
debug info errors. may need to use the zig lldb fork for debugging.
- zig test-obj, which we will be able to use for zig unit tests

Still an issue:

- false 'dependency loop' errors remain in watch mode
- watch mode crashes observed

Follow-up:

- [ ] search `comptime Writer: type` and `comptime W: type` and remove
- [ ] remove format_mode in our zig fork
- [ ] remove deprecated.zig autoFormatLabelFallback
- [ ] remove deprecated.zig autoFormatLabel
- [ ] remove deprecated.BufferedWriter and BufferedReader
- [ ] remove override_no_export_cpp_apis as it is no longer needed
- [ ] css Parser(W) -> Parser, and remove all the comptime writer: type
params
- [ ] remove deprecated writer fully

Files that add lines:

```
649     src/deprecated.zig
167     scripts/pack-codegen-for-zig-team.ts
54      scripts/cleartrace-impl.js
46      scripts/cleartrace.ts
43      src/windows.zig
18      src/fs.zig
17      src/bun.js/ConsoleObject.zig
16      src/output.zig
12      src/bun.js/test/debug.zig
12      src/bun.js/node/node_fs.zig
8       src/env_loader.zig
7       src/css/printer.zig
7       src/cli/init_command.zig
7       src/bun.js/node.zig
6       src/string/escapeRegExp.zig
6       src/install/PnpmMatcher.zig
5       src/bun.js/webcore/Blob.zig
4       src/crash_handler.zig
4       src/bun.zig
3       src/install/lockfile/bun.lock.zig
3       src/cli/update_interactive_command.zig
3       src/cli/pack_command.zig
3       build.zig
2       src/Progress.zig
2       src/install/lockfile/lockfile_json_stringify_for_debugging.zig
2       src/css/small_list.zig
2       src/bun.js/webcore/prompt.zig
1       test/internal/ban-words.test.ts
1       test/internal/ban-limits.json
1       src/watcher/WatcherTrace.zig
1       src/transpiler.zig
1       src/shell/builtin/cp.zig
1       src/js_printer.zig
1       src/io/PipeReader.zig
1       src/install/bin.zig
1       src/css/selectors/selector.zig
1       src/cli/run_command.zig
1       src/bun.js/RuntimeTranspilerStore.zig
1       src/bun.js/bindings/JSRef.zig
1       src/bake/DevServer.zig
```

Files that remove lines:

```
-1      src/test/recover.zig
-1      src/sql/postgres/SocketMonitor.zig
-1      src/sql/mysql/MySQLRequestQueue.zig
-1      src/sourcemap/CodeCoverage.zig
-1      src/css/values/color_js.zig
-1      src/compile_target.zig
-1      src/bundler/linker_context/convertStmtsForChunk.zig
-1      src/bundler/bundle_v2.zig
-1      src/bun.js/webcore/blob/read_file.zig
-1      src/ast/base.zig
-2      src/sql/postgres/protocol/ArrayList.zig
-2      src/shell/builtin/mkdir.zig
-2      src/install/PackageManager/patchPackage.zig
-2      src/install/PackageManager/PackageManagerDirectories.zig
-2      src/fmt.zig
-2      src/css/declaration.zig
-2      src/css/css_parser.zig
-2      src/collections/baby_list.zig
-2      src/bun.js/bindings/ZigStackFrame.zig
-2      src/ast/E.zig
-3      src/StandaloneModuleGraph.zig
-3      src/deps/picohttp.zig
-3      src/deps/libuv.zig
-3      src/btjs.zig
-4      src/threading/Futex.zig
-4      src/shell/builtin/touch.zig
-4      src/meta.zig
-4      src/install/lockfile.zig
-4      src/css/selectors/parser.zig
-5      src/shell/interpreter.zig
-5      src/css/error.zig
-5      src/bun.js/web_worker.zig
-5      src/bun.js.zig
-6      src/cli/test_command.zig
-6      src/bun.js/VirtualMachine.zig
-6      src/bun.js/uuid.zig
-6      src/bun.js/bindings/JSValue.zig
-9      src/bun.js/test/pretty_format.zig
-9      src/bun.js/api/BunObject.zig
-14     src/install/install_binding.zig
-14     src/fd.zig
-14     src/bun.js/node/path.zig
-14     scripts/pack-codegen-for-zig-team.sh
-17     src/bun.js/test/diff_format.zig
```

`git diff --numstat origin/main...HEAD | awk '{ print ($1-$2)"\t"$3 }' |
sort -rn`

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Meghan Denny <meghan@bun.com>
Co-authored-by: tayor.fish <contact@taylor.fish>
2025-11-10 14:38:26 -08:00
Jarred Sumner
767c61d355 Fix memory leaks & blocking syscall in Bun Shell (#23636)
## Summary

Fixes two critical bugs in Bun Shell:

1. **Memory leaks & incorrect GC reporting**: Shell objects weren't
reporting their memory usage to JavaScriptCore's garbage collector,
causing memory to accumulate unchecked. Also fixes a leak where
`ShellArgs` wasn't being freed in `Interpreter.finalize()`.

2. **Blocking I/O on macOS**: Fixes a bug where writing large amounts of
data (>1MB) to pipes would block the main thread on macOS. The issue:
`sendto()` with `MSG_NOWAIT` flag blocks on macOS despite the flag, so
we now avoid the socket fast path unless the socket is already
non-blocking.

## Changes

- Adds `memoryCost()` and `estimatedSize()` implementations across shell
AST nodes, interpreter, and I/O structures
- Reports estimated memory size to JavaScriptCore GC via
`vm.heap.reportExtraMemoryAllocated()`
- Fixes missing `this.args.deinit()` call in interpreter finalization
- Fixes `BabyList.memoryCost()` to return bytes, not element count
- Conditionally uses socket fast path in IOWriter based on platform and
socket state

## Test plan

- [x] New test: `shell-leak-args.test.ts` - validates memory doesn't
leak during parsing/execution
- [x] New test: `shell-blocking-pipe.test.ts` - validates large pipe
writes don't block the main thread
- [x] Existing shell tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
2025-10-19 22:17:19 -07:00
Jarred Sumner
bd88717ddc codegen: Add WriteBarrierEarlyInit support for classes with values and valuesArray (#23624)
## Summary

Adds comprehensive support to `generate-classes.ts` for JavaScript
classes that need both named WriteBarrier members (like callbacks) and a
dynamic array of JSValues, all properly tracked by the garbage
collector. This replaces error-prone manual `protect()/unprotect()`
calls with proper GC integration.

## Motivation

The shell interpreter was using `JSValue.protect()/unprotect()` to keep
JavaScript objects alive, which caused memory leaks when cleanup paths
didn't properly unprotect values. This is a common pattern that needed a
better solution.

## What Changed

### Code Generator (`generate-classes.ts`)

When a class has both `values: ["resolve", "reject"]` and `valuesArray:
true`:

**Generated C++ class gets:**
- `WTF::FixedVector<JSC::WriteBarrier<JSC::Unknown>> jsvalueArray`
member for dynamic array
- Individual `JSC::WriteBarrier<JSC::Unknown> m_resolve, m_reject`
members for named values
- 4 `create()` overloads covering all combinations:
  1. Basic: `create(vm, globalObject, structure, ptr)`
  2. Array only: `create(..., FixedVector<WriteBarrier<Unknown>>&&)`
  3. Named values: `create(..., JSValue resolve, JSValue reject)` 
  4. Both: `create(..., FixedVector&&, JSValue resolve, JSValue reject)`

**Constructor overloads using `WriteBarrierEarlyInit`:**
```cpp
JSShellInterpreter(VM& vm, Structure* structure, void* ptr, 
                   JSValue resolve, JSValue reject)
    : Base(vm, structure)
    , m_resolve(resolve, JSC::WriteBarrierEarlyInit)  // ← Key technique
    , m_reject(reject, JSC::WriteBarrierEarlyInit)
{
    m_ctx = ptr;
}
```

The `WriteBarrierEarlyInit` tag allows initializing WriteBarriers in the
constructor initializer list before the object is fully constructed,
which is required for proper GC integration.

**Extern C bridge functions:**
- `TypeName__createWithValues(globalObject, ptr, markedArgumentBuffer*)`
- `TypeName__createWithInitialValues(globalObject, ptr, resolve,
reject)`
- `TypeName__createWithValuesAndInitialValues(globalObject, ptr,
buffer*, resolve, reject)`

**Zig convenience wrappers:**
- `toJSWithValues(this, globalObject, markedArgumentBuffer)`
- `toJSWithInitialValues(this, globalObject, resolve, reject)`
- `toJSWithValuesAndInitialValues(this, globalObject, buffer, resolve,
reject)`

### Shell Interpreter Memory Leak Fix

**Before:**
```zig
const js_value = JSShellInterpreter.toJS(interpreter, globalThis);
resolve.protect();  // Manual reference counting
reject.protect();
// ... later in cleanup ...
resolve.unprotect();  // Easy to forget/miss in error paths
reject.unprotect();
```

**After:**
```zig
const js_value = Bun__createShellInterpreter(
    globalThis, 
    interpreter,
    parsed_shell_script,
    resolve,  // Stored with WriteBarrierEarlyInit
    reject,   // GC tracks automatically
);
// No manual memory management needed!
```

### Supporting Changes

- Added `MarkedArgumentBuffer.wrap()` helper in Zig for safe
MarkedArgumentBuffer usage
- Created `ShellBindings.cpp` with `Bun__createShellInterpreter()` using
the new API
- Removed all `protect()/unprotect()` calls from shell interpreter
- Applied pattern to both `ShellInterpreter` and `ShellArgs` classes

## Benefits

1. **No memory leaks**: GC tracks all references automatically
2. **Safer**: Cannot forget to unprotect values
3. **Cleaner code**: No manual reference counting
4. **Reusable**: Pattern works for any class needing to store JSValues
5. **Performance**: Same cost as manual protect/unprotect but safer

## Testing

Existing shell tests verify the functionality. The pattern is already
used throughout JavaScriptCore for similar cases (see
`JSWrappingFunction`, `AsyncContextFrame`, `JSModuleMock`, etc.)

## When to Use This Pattern

Use `values` + `valuesArray` + `WriteBarrierEarlyInit` when:
- Your C++ class needs to keep JavaScript values alive
- You have both known named callbacks AND dynamic arrays of values
- You want the GC to track references instead of manual
protect/unprotect
- Your class extends `JSDestructibleObject`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-10-13 19:15:38 -07:00
Dylan Conway
312a86fd43 fix writing UTF-16 with a trailing unpaired surrogate to process.stdout/stderr (#23444)
### What does this PR do?
Fixes `bun -p "process.stderr.write('Hello' +
String.fromCharCode(0xd800))"`.

Also fixes potential index out of bounds if there are many invalid
sequences.

This also affects `TextEncoder`.
### How did you verify your code works?
Added tests for edgecases

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-10-10 03:48:04 -07:00
Dylan Conway
8d28289407 fix(install): make negative workspace patterns work (#23229)
### What does this PR do?
It's common for monorepos to exclude portions of a large glob

```json
"workspaces": [
  "packages/**",
  "!packages/**/test/**",
  "!packages/**/template/**"
],
```

closes #4621 (note: patterns like `"packages/!(*-standalone)"` will need
to be written `"!packages/*-standalone"`)
### How did you verify your code works?
Manually tested https://github.com/opentiny/tiny-engine, and added a new
workspace test.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-10-04 00:31:47 -07:00
taylor.fish
edf13bd91d Refactor BabyList (#22502)
(For internal tracking: fixes STAB-1129, STAB-1145, STAB-1146,
STAB-1150, STAB-1126, STAB-1147, STAB-1148, STAB-1149, STAB-1158)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-09-09 20:41:10 -07:00
taylor.fish
437e15bae5 Replace catch bun.outOfMemory() with safer alternatives (#22141)
Replace `catch bun.outOfMemory()`, which can accidentally catch
non-OOM-related errors, with either `bun.handleOom` or a manual `catch
|err| switch (err)`.

(For internal tracking: fixes STAB-1070)

---------

Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-08-26 12:50:25 -07:00
Meghan Denny
5b972fa2b4 zig: ban not using .true and .false for js boolean literals (#21329)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Meghan Denny <meghan@bun.com>
2025-08-20 16:16:11 -07:00
taylor.fish
41b1efe12c Rename disabled parameter in Output.scoped (#21769)
It's very confusing.

(For internal tracking: fixes STAB-977)
2025-08-11 20:19:34 -07:00
Dylan Conway
24b7835ecd Fix shell lexer error message handling (#21419)
## Summary
- Fixed shell lexer to properly store error messages using TextRange
instead of direct string slices
- This prevents potential use-after-free issues when error messages are
accessed after the lexer's string pool might have been reallocated
- Added test coverage for shell syntax error reporting

## Changes
- Changed `LexError.msg` from `[]const u8` to `Token.TextRange` to store
indices into the string pool
- Added `TextRange.slice()` helper method for converting ranges back to
string slices
- Updated error message concatenation logic to use the new range-based
approach
- Added test to verify syntax errors are reported correctly

## Test plan
- [x] Added test case for invalid shell syntax error reporting
- [x] Existing shell tests continue to pass
- [x] Manual testing of various shell syntax errors

closes BAPI-2232

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-07-27 23:32:06 -07:00
aspizu
9bb2474adb fix: display of eq token in shell lexer (#21275)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Zack Radisic <56137411+zackradisic@users.noreply.github.com>
2025-07-25 12:47:48 -07:00
taylor.fish
07cd45deae Refactor Zig imports and file structure (part 1) (#21270)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-07-22 17:51:38 -07:00
pfg
83760fc446 Sort imports in all files (#21119)
Co-authored-by: taylor.fish <contact@taylor.fish>
2025-07-21 13:26:47 -07:00
Meghan Denny
875604a42b safety: a lot more exception checker progress (#20956) 2025-07-16 00:11:19 -07:00
Zack Radisic
ac61b1d471 Use better function names for bun.String (#20999) 2025-07-12 18:19:16 -07:00
Meghan Denny
f9712ce309 make node:buffer,zlib,stream,fs exception checker clear (#20494) 2025-06-25 18:36:08 -07:00
Zack Radisic
0a3ac50931 Refactor shell to use AllocationScope and fix memory issues (#20531)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <Jarred-Sumner@users.noreply.github.com>
2025-06-23 22:29:48 -07:00
Zack Radisic
41d10ed01e Refactor shell and fix some bugs (#20476) 2025-06-19 18:47:00 -07:00
190n
346e97dde2 fix bugs found by exception scope verification (#20285)
Co-authored-by: 190n <7763597+190n@users.noreply.github.com>
2025-06-18 23:08:19 -07:00
Jarred Sumner
407c4e800a Revert "add support for "workspaces.nohoist" and "workspaces.hoistingLimits" (#20124)"
This reverts commit 11070b8e16.
2025-06-03 23:51:03 -07:00
Dylan Conway
11070b8e16 add support for "workspaces.nohoist" and "workspaces.hoistingLimits" (#20124) 2025-06-03 23:44:09 -07:00
Jarred Sumner
3ea6133c46 CI: Remove unused top-level decls in formatter in zig (#19879)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-23 22:49:48 -07:00
pfg
a7b46ebbfe fastGet can throw (#19506)
Co-authored-by: Meghan Denny <meghan@bun.sh>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-05-14 22:14:20 -07:00
chloe caruso
3349c995b5 no usingnamespace, organize jsc namespace, enable -fincremental (#19122)
Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>
2025-04-22 16:34:15 -07:00
Jarred Sumner
0471254e4e Use Highway SIMD (#19134)
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2025-04-21 23:28:03 -07:00
Jarred Sumner
b117d14650 Avoid u3 (#19140) 2025-04-21 22:46:32 -07:00
chloe caruso
4ec410e0d7 internal: make @import("bun") work in zig (#19096) 2025-04-17 12:32:47 -07:00
chloe caruso
903706dccf file descriptor rewrite (#18790) 2025-04-15 09:37:11 -07:00
Jarred Sumner
879fdd7ef6 Bump Zig again (#18948) 2025-04-11 19:13:20 -07:00
Dylan Conway
78ee4a3e82 fix(shell): possible UAF when throwing a shell error (#18840) 2025-04-07 20:20:22 -07:00
Jarred Sumner
e75d226943 Make shell more reliable (#18794) 2025-04-05 02:45:25 -07:00
Jarred Sumner
a8cc31f8c4 Split shell into more files (#18793) 2025-04-04 23:38:43 -07:00
Jarred Sumner
11f2b5fb55 Run zig fmt 2025-04-03 12:04:24 -07:00
Jarred Sumner
4806e84cc1 Revert "remove many usingnamespace, introduce new ref count and ref leak debugging tools (#18353)"
This reverts commit a199b85f2b. It does not compile on Windows.
2025-04-01 08:35:51 -07:00
chloe caruso
a199b85f2b remove many usingnamespace, introduce new ref count and ref leak debugging tools (#18353) 2025-03-31 17:17:38 -07:00
Don Isaac
062a5b9bf8 fix(shell): remove unecessary allocations when printing errors (#17898) 2025-03-14 08:45:34 -07:00
Don Isaac
1ed87f4e83 fix: deadlock in Cow debug checks (#18173)
Co-authored-by: DonIsaac <22823424+DonIsaac@users.noreply.github.com>
2025-03-13 16:42:06 -07:00
Don Isaac
4ef7a43939 chore: add assertf and releaseAssert (#17859)
Co-authored-by: DonIsaac <22823424+DonIsaac@users.noreply.github.com>
Co-authored-by: chloe caruso <git@paperclover.net>
2025-03-04 12:50:59 -08:00
Meghan Denny
1574df835e zig: make JSValue.toBunString use JSError (#17648) 2025-02-25 13:04:44 -08:00
Meghan Denny
27c90786ca node: implement process.binding('fs') (#17480) 2025-02-20 22:15:01 -08:00
chloe caruso
2b97d61deb chore: remove some trivial usage of usingnamespace (#17268) 2025-02-11 19:38:52 -08:00
Minsoo Choo
fa502506e5 Fix formatting for extern "c" (#16983) 2025-02-02 21:34:58 -08:00
Meghan Denny
26d3688e53 zig: update to 0.14.0-dev (#16862)
Co-authored-by: nektro <5464072+nektro@users.noreply.github.com>
2025-02-01 01:11:02 -08:00
chloe caruso
834ad11d48 get node:fs tests passing part 1 (#16270) 2025-01-14 20:53:02 -08:00
Jarred Sumner
2465ccae53 Re-sync our Mutex implementation with zig stdlib (#16271) 2025-01-08 21:09:27 -08:00
Jarred Sumner
2b2ca3275c Improve stack overflow, show more properties in Error objects (#15985)
Co-authored-by: Dave Caruso <me@paperdave.net>
2024-12-25 21:47:13 -08:00
Jarred Sumner
d6b9c444c1 Rename src/bundler.zig to src/transpiler.zig (#15921)
Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>
2024-12-21 00:59:37 -08:00
Meghan Denny
2d441d868b zig: make throw use JSError (#15444) 2024-12-02 14:19:18 -08:00
Meghan Denny
663f00b62b zig: make throwOutOfMemory use JSError (#15413) 2024-11-26 12:58:43 -08:00
Meghan Denny
c434b2c191 zig: make throwPretty use JSError (#15410) 2024-11-25 18:08:42 -08:00