Commit Graph

311 Commits

Author SHA1 Message Date
Claude Bot
23016b2a82 feat: Add Express/Rails-style req.query getter to BunRequest
Implements a new `req.query` getter that parses URL query parameters into nested objects and arrays, matching the behavior of Express.js and Ruby on Rails.

## Features
- Parses simple parameters: `?name=john&age=30` → `{ name: "john", age: "30" }`
- Supports nested objects: `?user[name]=alice` → `{ user: { name: "alice" } }`
- Supports arrays: `?ids[]=1&ids[]=2` → `{ ids: ["1", "2"] }`
- Supports indexed arrays: `?items[0]=a&items[1]=b` → `{ items: ["a", "b"] }`
- Handles complex nesting: `?users[0][name]=alice` → `{ users: [{ name: "alice" }] }`

## Implementation
- Extracted parsing logic to BunRequestParams.cpp for maintainability
- Added PropertyCallback getter to JSBunRequest
- Uses null prototype objects to prevent prototype pollution
- Ignores __proto__ keys for security
- Only available when using routes with Bun.serve()

## Testing
- Added comprehensive test suite with 20 test cases
- Exposed parseQueryParams in bun:internal-for-testing for unit testing
- Tests cover edge cases including sparse arrays, type conflicts, and security

This brings Bun's request handling closer to Express/Rails conventions, making it easier for developers to migrate existing applications.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-02 14:15:00 +00:00
Jarred Sumner
de7c947161 bump webkit (#22256)
### What does this PR do?

### How did you verify your code works?

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-01 16:20:13 -07:00
Dylan Conway
fcaff77ed7 Implement Bun.YAML.stringify (#22183)
### What does this PR do?
This PR adds `Bun.YAML.stringify`. The stringifier will double quote
strings only when necessary (looks for keywords, numbers, or containing
non-printable or escaped characters). Anchors and aliases are detected
by object equality, and anchor name is chosen from property name, array
item, or the root collection.
```js
import { YAML } from "bun"

YAML.stringify(null) // null
YAML.stringify("hello YAML"); // "hello YAML"
YAML.stringify("123.456"); // "\"123.456\""

// anchors and aliases
const userInfo = { name: "bun" };
const obj = { user1: { userInfo }, user2: { userInfo } };
YAML.stringify(obj, null, 2);
// # output
// user1: 
//   userInfo: 
//     &userInfo
//     name: bun
// user2: 
//   userInfo: 
//     *userInfo

// will handle cycles
const obj = {};
obj.cycle = obj;
YAML.stringify(obj, null, 2);
// # output
// &root
// cycle:
//   *root

// default no space
const obj = { one: { two: "three" } };
YAML.stringify(obj);
// # output
// {one: {two: three}}
```

### How did you verify your code works?
Added tests for basic use and edgecases

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- New Features
- Added YAML.stringify to the YAML API, producing YAML from JavaScript
values with quoting, anchors, and indentation support.

- Improvements
- YAML.parse now accepts a wider range of inputs, including Buffer,
ArrayBuffer, TypedArrays, DataView, Blob/File, and SharedArrayBuffer,
with better error propagation and stack protection.

- Tests
- Extensive new tests for YAML.parse and YAML.stringify across data
types, edge cases, anchors/aliases, deep nesting, and round-trip
scenarios.

- Chores
- Added a YAML stringify benchmark script covering multiple libraries
and data shapes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-31 18:27:51 -07:00
Jarred Sumner
b93468ca48 Fix ESM <> CJS dual-package hazard determinism bug (#22231)
### What does this PR do?

Originally, we attempted to avoid the "dual package hazard" right before
we enqueue a parse task, but that code gets called in a
non-deterministic order. This meant that some of your modules would use
the right variant and some of them would not.

We have to instead do that in a separate pass, after all the files are
parsed.

The thing to watch out for with this PR is how it impacts the dev
server.

### How did you verify your code works?

Unskipped tests. Plus manual.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-08-30 02:50:35 -07:00
pfg
684f7ecd09 Hide private fields in typeInfo (#22228)
ebe0cdac31..e0b7c318f3

It hides it in typeInfo even in the current file, unlike private
declarations which are only hidden in other files.

This allows you to formatted print a type with private fields, but the
private fields are not shown. The alternative would be to allow
accessing private fields through `@field()` but that looked like it was
going to be more complicated (need to add an argument to
structFieldVal/structFieldPtr which are called by fieldVal/fieldPtr
which have 36 callsites)

---------

Co-authored-by: taylor.fish <contact@taylor.fish>
2025-08-29 13:55:51 -07:00
Jarred Sumner
fe8f8242fd Make BoundedArray more compact, shrink Data in sql from 32 bytes to 24 bytes (#22210)
### What does this PR do?

- Instead of storing `len` in `BoundedArray` as a `usize`, store it as
either a `u8` or ` u16` depending on the `buffer_capacity`
- Copy-paste `BoundedArray` from the standard library into Bun's
codebase as it was removed in
https://github.com/ziglang/zig/pull/24699/files#diff-cbd8cbbc17583cb9ea5cc0f711ce0ad447b446e62ea5ddbe29274696dce89e4f
and we will probably continue using it

### How did you verify your code works?

Ran `bun run zig:check`

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: taylor.fish <contact@taylor.fish>
2025-08-28 17:34:35 -07:00
Meghan Denny
eb7727819a node:util: move deprecate to internal file so its faster to import (#22197)
Co-authored-by: Meghan Denny <meghan@bun.com>
2025-08-28 15:05:52 -07:00
pfg
84604888e9 Private fields (#22189)
ZLS was tested manually and works with private fields (after restarting)

Zig diff:
d1a4e0b0dd..ebe0cdac31

ZLS diff:
15730e8e5d..3733f39c8d

Increases `zig build check` time by maybe 10ms?

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-28 04:24:04 -07:00
Jarred Sumner
a3fcfd3963 Bump WebKit (#22145)
### What does this PR do?

### How did you verify your code works?

---------

Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-26 17:38:15 -07:00
Alistair Smith
54b90213eb fix: support virtual entrypoints in onResolve() (#22144)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-26 16:51:41 -07:00
taylor.fish
437e15bae5 Replace catch bun.outOfMemory() with safer alternatives (#22141)
Replace `catch bun.outOfMemory()`, which can accidentally catch
non-OOM-related errors, with either `bun.handleOom` or a manual `catch
|err| switch (err)`.

(For internal tracking: fixes STAB-1070)

---------

Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-08-26 12:50:25 -07:00
Jarred Sumner
707fc4c3a2 Introduce Bun.secrets API (#21973)
This PR adds `Bun.secrets`, a new API for securely storing and
retrieving credentials using the operating system's native credential
storage locally. This helps developers avoid storing sensitive data in
plaintext config files.

```javascript
// Store a GitHub token securely
await Bun.secrets.set({
  service: "my-cli-tool",
  name: "github-token",
  value: "ghp_xxxxxxxxxxxxxxxxxxxx"
});

// Retrieve it when needed
const token = await Bun.secrets.get({
  service: "my-cli-tool",
  name: "github-token"
});

// Use with fallback to environment variable
const apiKey = await Bun.secrets.get({
  service: "my-app",
  name: "api-key"
}) || process.env.API_KEY;
```

Marking this as a draft because Linux and Windows have not been manually
tested yet. This API is only really meant for local development usecases
right now, but it would be nice if in the future to support adapters for
production or CI usecases.

### Core API
- `Bun.secrets.get({ service, name })` - Retrieve a stored credential
- `Bun.secrets.set({ service, name, value })` - Store or update a
credential
- `Bun.secrets.delete({ service, name })` - Delete a stored credential

### Platform Support
- **macOS**: Uses Keychain Services via Security.framework
- **Linux**: Uses libsecret (works with GNOME Keyring, KWallet, etc.)
- **Windows**: Uses Windows Credential Manager via advapi32.dll

### Implementation Highlights
- Non-blocking - all operations run on the threadpool
- Dynamic loading - no hard dependencies on system libraries
- Sensitive data is zeroed after use
- Consistent API across all platforms

## Use Cases

This API is particularly useful for:
- CLI tools that need to store authentication tokens
- Development tools that manage API keys
- Any tool that currently stores credentials in `~/.npmrc`,
`~/.aws/credentials` or in environment variables that're globally loaded

## Testing

Comprehensive test suite included with coverage for:
- Basic CRUD operations
- Empty strings and special characters
- Unicode support
- Concurrent operations
- Error handling

All tests pass on macOS. Linux and Windows implementations are complete
but would benefit from additional platform testing.

## Documentation

- Complete API documentation in `docs/api/secrets.md`
- TypeScript definitions with detailed JSDoc comments and examples

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-23 06:57:00 -07:00
Dylan Conway
8fad98ffdb Add Bun.YAML.parse and YAML imports (#22073)
### What does this PR do?
This PR adds builtin YAML parsing with `Bun.YAML.parse`
```js
import { YAML } from "bun";
const items = YAML.parse("- item1");
console.log(items); // [ "item1" ]
```

Also YAML imports work just like JSON and TOML imports
```js
import pkg from "./package.yaml"
console.log({ pkg }); // { pkg: { name: "pkg", version: "1.1.1" } }
```
### How did you verify your code works?
Added some tests for YAML imports and parsed values.

---------

Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-08-23 06:55:30 -07:00
Jarred Sumner
c342453065 Bump WebKit (#22072)
### What does this PR do?

### How did you verify your code works?
2025-08-23 00:31:53 -07:00
taylor.fish
7717693c70 Dev server refactoring, part 1 (mainly IncrementalGraph) (#22010)
* `IncrementalGraph(.client).File` packs its fields in a specific way to
save space, but it makes the struct hard to use and error-prone (e.g.,
untagged unions with tags stored in a separate `flags` struct). This PR
changes `File` to have a human-readable layout, but adds methods to
convert it to and from `File.Packed`, a packed version with the same
space efficiency as before.
* Reduce the need to pass the dev allocator to functions (e.g.,
`deinit`) by storing it as a struct field via the new `DevAllocator`
type. This type has no overhead in release builds, or when
`AllocationScope` is disabled.
* Use owned pointers in `PackedMap`.
* Use `bun.ptr.Shared` for `PackedMap` instead of the old
`bun.ptr.RefPtr`.
* Add `bun.ptr.ScopedOwned`, which is like `bun.ptr.Owned`, but can
store an `AllocationScope`. No overhead in release builds or when
`AllocationScope` is disabled.
* Reduce redundant allocators in `BundleV2`.
* Add owned pointer conversions to `MutableString`.
* Make `AllocationScope` behave like a pointer, so it can be moved
without invalidating allocations. This eliminates the need for
self-references.
* Change memory cost algorithm so it doesn't rely on “dedupe bits”.
These bits used to take advantage of padding but there is now no padding
in `PackedMap`.
* Replace `VoidFieldTypes` with `useAllFields`; this eliminates the need
for `voidFieldTypesDiscardHelper`.

(For internal tracking: fixes STAB-1035, STAB-1036, STAB-1037,
STAB-1038, STAB-1039, STAB-1040, STAB-1041, STAB-1042, STAB-1043,
STAB-1044, STAB-1045)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-22 23:04:58 -07:00
connerlphillippi
73fe9a4484 Add Windows code signing setup for x64 builds (#22022)
## Summary
- Implements automated Windows code signing for x64 and x64-baseline
builds
- Integrates DigiCert KeyLocker for secure certificate management
- Adds CI/CD pipeline support for signing during builds

## Changes
- Added `.buildkite/scripts/sign-windows.sh` script for automated
signing
- Updated CMake configurations to support signing workflow
- Modified build scripts to integrate signing step

## Testing
- Script tested locally with manual signing process
- Successfully signed test binaries at:
  - `C:\Builds\bun-windows-x64\bun.exe`
  - `C:\Builds\bun-windows-x64-baseline\bun.exe`

## References
Uses DigiCert KeyLocker tools for Windows signing

## Next Steps
- Validate Buildkite environment variables in CI
- Test full pipeline in CI environment

---------

Co-authored-by: Jarred Sumner <jarred@bun.sh>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-22 03:53:57 -07:00
Ciro Spaciari
ecbf103bf5 feat(MYSQL) Bun.SQL mysql support (#21968)
### What does this PR do?
Add MySQL support, Refactor will be in a followup PR
### How did you verify your code works?
A lot of tests

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: cirospaciari <6379399+cirospaciari@users.noreply.github.com>
2025-08-21 15:28:15 -07:00
Alistair Smith
efdbe3b54f bun install Security Scanner API (#21183)
### What does this PR do?

Fixes #22014

todo:
- [x] not spawn sync
- [x] better comm to subprocess (not stderr)
- [x] tty
- [x] more tests (also include some tests for the actual implementation
of a provider)
- [x] disable autoinstall?

Scanner template: https://github.com/oven-sh/security-scanner-template

<!-- **Please explain what your changes do**, example: -->

<!--

This adds a new flag --bail to bun test. When set, it will stop running
tests after the first failure. This is useful for CI environments where
you want to fail fast.

-->

---

- [x] Documentation or TypeScript types (it's okay to leave the rest
blank in this case)
- [x] Code changes

### How did you verify your code works?

<!-- **For code changes, please include automated tests**. Feel free to
uncomment the line below -->

<!-- I wrote automated tests -->

<!-- If JavaScript/TypeScript modules or builtins changed:

- [ ] I included a test for the new code, or existing tests cover it
- [ ] I ran my tests locally and they pass (`bun-debug test
test-file-name.test`)

-->

<!-- If Zig files changed:

- [ ] I checked the lifetime of memory allocated to verify it's (1)
freed and (2) only freed when it should be
- [ ] I included a test for the new code, or an existing test covers it
- [ ] JSValue used outside of the stack is either wrapped in a
JSC.Strong or is JSValueProtect'ed
- [ ] I wrote TypeScript/JavaScript tests and they pass locally
(`bun-debug test test-file-name.test`)
-->

<!-- If new methods, getters, or setters were added to a publicly
exposed class:

- [ ] I added TypeScript types for the new methods, getters, or setters
-->

<!-- If dependencies in tests changed:

- [ ] I made sure that specific versions of dependencies are used
instead of ranged or tagged versions
-->

<!-- If a new builtin ESM/CJS module was added:

- [ ] I updated Aliases in `module_loader.zig` to include the new module
- [ ] I added a test that imports the module
- [ ] I added a test that require() the module
-->


tests (bad currently)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan-conway@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-08-21 14:53:50 -07:00
taylor.fish
97495a86fe Add new shared pointer type (#21914)
Add a new reference-counted shared pointer type, `bun.ptr.Shared`.

Features:

* Can hold data of any type; doesn't require adding a `ref_count` field
* Reference count is an internal implementation detail; will never get
out of sync due to erroneous manipulation
* Supports weak pointers
* Supports optional pointers with no overhead (`Shared(?*T)` is the same
size as `Shared(*T)`)
* Has an atomic thread-safe version: `bun.ptr.AtomicShared`
* Defaults to `bun.default_allocator`, but can handle other allocators
as well, with both static and dynamic polymorphism

The following types are now deprecated and will eventually be removed:

* `bun.ptr.RefCount`
* `bun.ptr.ThreadSafeRefCount`
* `bun.ptr.RefPtr`
* `bun.ptr.WeakPtr`

(For internal tracking: fixes STAB-1011)
2025-08-20 17:44:25 -07:00
robobun
e7672b2d04 Add string fast path for postMessage and structuredClone (#21926)
## Summary

Implements a string fast path optimization for `postMessage` and
`structuredClone` operations that provides significant performance
improvements for string-only data transfer, along with various bug fixes
and infrastructure improvements.

## Key Performance Improvements

**postMessage with Workers:**
- **Small strings (11 chars):** ~5% faster (572ns vs 599ns)
- **Medium strings (14KB):** **~2.7x faster** (528ns vs 1.40μs) 
- **Large strings (3MB):** **~660x faster** (540ns vs 356μs)

**Compared to Node.js postMessage:**
- Similar performance for small strings
- Competitive for medium strings  
- **~455x faster** for large strings (540ns vs 245μs)

## Implementation Details

The optimization adds a **string fast path** that bypasses full
structured cloning serialization when:
- Input is a pure string (`value.isString()`)
- No transfer list or message ports are involved
- Not being stored persistently

### Core Changes

**String Thread-Safety Utilities (`BunString.cpp/h`):**
- `isCrossThreadShareable()` - Checks if string can be safely shared
across threads
- `toCrossThreadShareable()` - Converts strings to thread-safe form via
`isolatedCopy()`
- Handles edge cases: atoms, symbols, substring slices, external buffers

**Serialization Fast Path (`SerializedScriptValue.cpp`):**
- New `m_fastPathString` field stores string data directly
- Bypasses full object serialization machinery for pure strings
- Creates isolated copies for cross-thread safety

**Deserialization Fast Path:**
- Directly returns JSString from stored string data
- Avoids parsing serialized byte streams

**Updated Flags System (`JSValue.zig`, `Serialization.cpp`):**
- Replaces boolean `forTransfer` with structured `SerializedFlags`
- Supports `forCrossProcessTransfer` and `forStorage` distinctions

**Structured Clone Infrastructure:**
- Moved `structuredClone` implementation to dedicated
`StructuredClone.cpp`
- Added `jsFunctionStructuredCloneAdvanced` for testing with custom
flags
- Improved class serialization compatibility checks (`isForTransfer`,
`isForStorage`)

**IPC Improvements (`ipc.zig`):**
- Fixed race conditions in `SendQueue` by deferring cleanup to next tick
- Proper fd ownership handling with `bun.take()`
- Cached IPC serialize/parse functions for better performance

**BlockList Thread Safety Fixes (`BlockList.zig`):**
- Fixed potential deadlocks by moving mutex locking inside methods
- Added atomic `estimated_size` counter to avoid lock during GC
- Corrected pointer handling in comparison functions
- Improved GC safety in `rules()` method

## Benchmark Results

```
❯ bun-21926 bench/string-postmessage.mjs  # This branch
postMessage(11 chars string)  572.24 ns/iter
postMessage(14 KB string)     527.55 ns/iter  ← ~2.7x faster
postMessage(3 MB string)      539.70 ns/iter  ← ~660x faster

❯ bun-1.2.20 bench/string-postmessage.mjs  # Previous
postMessage(11 chars string)  598.76 ns/iter
postMessage(14 KB string)       1.40 µs/iter
postMessage(3 MB string)      356.38 µs/iter

❯ node bench/string-postmessage.mjs       # Node.js comparison  
postMessage(11 chars string)  569.63 ns/iter
postMessage(14 KB string)       1.46 µs/iter
postMessage(3 MB string)      245.46 µs/iter
```

**Key insight:** The fast path achieves **constant time performance**
regardless of string size (~540ns), while traditional serialization
scales linearly with data size.

## Test Coverage

**New Tests:**
- `test/js/web/structured-clone-fastpath.test.ts` - Fast path memory
usage validation
- `test/js/web/workers/structuredClone-classes.test.ts` - Comprehensive
class serialization tests
  - Tests ArrayBuffer transferability 
  - Tests BunFile cloning with storage/transfer restrictions
  - Tests net.BlockList cloning behavior
  - Validates different serialization contexts (default, worker, window)

**Enhanced Tests:**
- `test/js/web/workers/structured-clone.test.ts` - Multi-function
testing
- Tests `structuredClone`, `jscSerializeRoundtrip`, and cross-process
serialization
  - Validates consistency across different serialization paths
- `test/js/node/cluster.test.ts` - Better error handling and debugging

**Benchmarks:**
- `bench/string-postmessage.mjs` - Worker postMessage performance
comparison
- `bench/string-fastpath.mjs` - Fast path vs traditional serialization
comparison

## Bug Fixes

**BlockList Threading Issues:**
- Fixed potential deadlocks when multiple threads access BlockList
simultaneously
- Moved mutex locks inside methods rather than holding across entire
function calls
- Added atomic size tracking for GC compatibility
- Fixed comparison function pointer handling

**IPC Race Conditions:**
- Fixed race condition where `SendQueue._onAfterIPCClosed()` could be
called on wrong thread
- Deferred cleanup operations to next tick using task queue
- Improved file descriptor ownership with proper `bun.take()` usage

**Structured Clone Compatibility:**
- Enhanced class serialization with proper transfer/storage mode
checking
- Fixed edge cases where non-transferable objects were incorrectly
handled
- Added better error reporting for unsupported clone operations

## Technical Notes

- Thread safety ensured via `String.isolatedCopy()` for cross-VM
transfers
- Memory cost calculation updated to account for string references
- Maintains full compatibility with existing structured clone semantics
- Does not affect object serialization or transfer lists
- Proper cleanup and error handling throughout IPC pipeline

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Meghan Denny <meghan@bun.sh>
2025-08-20 00:25:00 -07:00
Alistair Smith
784271f85e SQLite in Bun.sql (#21640)
### What does this PR do?

Support sqlite in the Bun.sql API

Fixes #18951
Fixes #19701

### How did you verify your code works?

tests

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-08-19 23:15:53 -07:00
taylor.fish
ecd74ac14c Improve owned pointer types (#21908)
(For internal tracking: fixes STAB-1005, STAB-1006, STAB-1007,
STAB-1008, STAB-1009)
2025-08-15 19:05:25 -07:00
robobun
9fd5b20aa3 feat: Add WebKit text codec support for 24 additional encodings (#21835)
## Summary
This PR integrates WebKit's text codec implementations into Bun's
TextDecoder, adding support for 24 additional character encodings beyond
the native UTF-8, UTF-16, and Latin1.

Fixes https://github.com/oven-sh/bun/issues/11564

## What's New
### Supported Encodings (24 total)
- **11 single-byte encodings**: IBM866, ISO-8859-3/6/7/8/8-I, KOI8-U,
windows-874/1253/1255/1257
- **7 CJK encodings**: Big5, EUC-JP, ISO-2022-JP, Shift_JIS, EUC-KR,
GBK, GB18030
- **2 special encodings**: x-user-defined, replacement

### Implementation Details
- Integrated WebKit's text codec C++ implementations
- Generated static encoding tables from WHATWG spec (no ICU dependency)
- Created C++ wrapper for Zig/C++ interop
- All encoding aliases are supported (e.g., `sjis` → `shift_jis`)
- Proper whitespace trimming for encoding labels

## Testing
-  Added comprehensive tests for all supported encodings
-  Passes Web Platform Tests for single-byte decoders
-  Passes Web Platform Tests for encoding labels
-  All 2,227 tests pass

## Test Output
```
bun test v1.2.19 (9feaab47)
 2207 pass
 0 fail
 5012 expect() calls
Ran 2207 tests across 1 file. [899.00ms]
```

## Not Included
The following encodings were not added due to ICU data loading
constraints:
- ISO-8859-2, 4, 5, 10, 13, 14, 15, 16
- Windows-1250, 1251, 1254, 1256, 1258
- KOI8-R, macintosh, x-mac-cyrillic

## Example Usage
```javascript
// CJK encodings
const decoder = new TextDecoder("shift_jis");
const bytes = new Uint8Array([0x82, 0xb1, 0x82, 0xf1]);
console.log(decoder.decode(bytes)); // "こん"

// Single-byte encodings
const greekDecoder = new TextDecoder("iso-8859-7");
const greekBytes = new Uint8Array([0xC3, 0xe5, 0xe9, 0xdc]);
console.log(greekDecoder.decode(greekBytes)); // "Γειά"
```

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude <claude@anthropic.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-14 22:58:25 -07:00
Jarred Sumner
4fa69773a3 Introduce Bun.stripANSI (#21801)
### What does this PR do?

Introduce `Bun.stripANSI`, a SIMD-accelerated drop-in replacement for
the popular `"strip-ansi"` package.

`Bun.stripANSI` performs >10x faster and fixes several bugs in
`strip-ansi`, like [this long-standing
one](https://github.com/chalk/strip-ansi/issues/43).

### How did you verify your code works?

There are tests that check the output of `strip-ansi` matches
`Bun.stripANSI`. For cases where `strip-ansi`'s behavior is incorrect,
the expected value is manually provided.

---------

Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: taylor.fish <contact@taylor.fish>
2025-08-14 22:42:05 -07:00
robobun
a33de51419 Update WebKit commit to aa4997abc9126f5a7557c9ecb7e8104779d87ec4 (#21878)
## Summary
- Updates WebKit commit from `684d4551ce5f62683476409d7402424e0f6eafb5`
to `aa4997abc9126f5a7557c9ecb7e8104779d87ec4`
- Build completed successfully with no errors
- Verified functionality with hello world test

## Test plan
- [x] Build completed successfully
- [x] Hello world test passes with `bun bd`
- [x] No build errors encountered

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-14 21:11:57 -07:00
Jarred Sumner
7b31393d44 Don't run the "Date" header timer every second all the time (#21850)
### What does this PR do?

Only reschedule the Date header while there are in-flight incoming HTTP
requests.

Update the Date header if, at the time we reschedule it, it is now
stale.

Goal: don't wake up Bun's process on every second when we're idly doing
nothing.

| Metric | this branch | main |
|--------|--------------------------|-------------------|
| **task-clock** | **35.24 msec** 🟢 | **102.79 msec** |
| **context-switches** | 619 🟢 | 1,699 |
| **cpu-migrations** | 11 🟢| 35 |
| **page-faults** | 2,173 | 2,174 |
| **cpu_atom/instructions** | **109,904,685 (1.76 insn/cycle)** 🟢 |
**67,880,002 (0.55 insn/cycle)** |
| **cpu_core/instructions** | **87,183,124 (1.07 insn/cycle)** 🟢 |
**32,939,500 (0.44 insn/cycle)** |
| **cpu_atom/cycles** | 62,527,125 (1.774 GHz) 🔻 | 122,448,620 (1.191
GHz) |
| **cpu_core/cycles** | 81,651,366 (2.317 GHz) 🟢 | 75,584,111 (0.735
GHz) |
| **cpu_atom/branches** | 9,632,460 (273.338 M/sec) 🔻 | 12,119,616
(117.909 M/sec) |
| **cpu_core/branches** | 17,417,756 (494.259 M/sec) 🟢 | 6,901,859
(67.147 M/sec) |
| **cpu_atom/branch-misses** | 192,013 (1.99%) 🟢 | 1,735,446 (14.32%) |
| **cpu_core/branch-misses** | 473,567 (2.72%) 🟢 | 499,907 (7.24%) |
| **TopdownL1 (cpu_core)** | 31.4% backend_bound<br>11.7%
bad_speculation<br>36.0% frontend_bound 🔻<br>20.9% retiring<br>34.1%
bad_speculation<br>41.9% retiring<br>0.0% backend_bound<br>24.0%
frontend_bound 🔻 | 21.3% backend_bound<br>9.6% bad_speculation<br>56.2%
frontend_bound<br>12.9% retiring<br>-20.0% bad_speculation<br>55.2%
retiring<br>26.2% backend_bound<br>38.6% frontend_bound |
| **time elapsed** | 1000.0219 s | 1000.0107 s |
| **user time** | — | 0.042667 s |
| **sys time** | — | 0.060309 s |

### How did you verify your code works?

Added a test

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-14 15:39:09 -07:00
Jarred Sumner
fac5e71a0c Split subprocess into more files (#21842)
### What does this PR do?

Split subprocess into more files

### How did you verify your code works?

check

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-13 20:47:50 -07:00
pfg
bf24d1b527 Split expect.zig into one file per expect matcher (#21810)
That's 75 files and 955 extra lines of imports. Maybe too many files.
2025-08-13 20:26:58 -07:00
taylor.fish
a4555ee3df Add owned pointer types (#21776)
Add an owned pointer type—a wrapper around a pointer and an allocator.

`Owned(*Foo)` and `Owned([]Foo)` contain both the pointer/slice and the
allocator that was used to allocate it. Calling `deinit` on these types
first calls `Foo.deinit` and then frees the memory. This makes it easier
to remember to free the memory, and hard to accidentally free it with
the wrong allocator.

Optional pointers are also supported (`Owned(?*Foo)`, `Owned(?[]Foo)`),
and an unmanaged variant which doesn't store the allocator
(`Owned(*Foo).Unmanaged`) is available for cases where space efficiency
is a concern.

A `MaybeOwned` type is also provided for representing data that could be
owned or borrowed. If the data is owned, `MaybeOwned.deinit` works like
`Owned.deinit`; otherwise, it's a no-op.

(For internal tracking: fixes STAB-920, STAB-921)
2025-08-12 22:25:49 -07:00
taylor.fish
0c83ff3f7e Fix z_allocator implementation when use_mimalloc is false; make Bun compile with use_mimalloc false (#21771)
We can't use `std.heap.c_allocator` as `z_allocator`; it doesn't
zero-initialize the memory. This PR adds a fallback implementation.

This PR also makes Bun compile successfully with `use_mimalloc` set to
false. More work is likely necessary to make it function correctly in
this case, but it should at least compile.

(For internal tracking: fixes STAB-978, STAB-979)
2025-08-11 20:20:58 -07:00
taylor.fish
a57dee5721 Various safety improvements (safety.ThreadLock, stack traces, MimallocArena, RefCount, safety.alloc) (#21726)
* Move `DebugThreadLock` to `bun.safety`
* Enable in `ci_assert` builds, but store stack traces only in debug
builds
  * Reduce size of struct by making optional field non-optional
* Add `initLockedIfNonComptime` as a workaround for not being able to
call `initLocked` in comptime contexts
* Add `lockOrAssert` method to acquire the lock if unlocked, or else
assert that the current thread acquired the lock
* Add stack traces to `CriticalSection` and `AllocPtr` in debug builds
* Make `MimallocArena.init` infallible
* Make `MimallocArena.heap` non-nullable
* Rename `RefCount.active_counts` to `raw_count` and provide read-only
`get` method
* Add `bun.safety.alloc.assertEq` to assert that two allocators are
equal (avoiding comparison of undefined `ptr`s)

(For internal tracking: fixes STAB-917, STAB-918, STAB-962, STAB-963,
STAB-964, STAB-965)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-11 13:40:07 -07:00
robobun
3766f183e6 deps: bump WebKit to eb92990ae9e0a8df3141b8cf946a4f250393e213 (#21702)
## Summary
- Updates WebKit from 75f6499 to eb92990 (latest release from
oven-sh/webkit)
- This brings in the latest WebKit improvements and fixes

## Test plan
- [ ] Verify the build completes successfully
- [ ] Run existing test suite to ensure no regressions

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-09 05:00:46 -07:00
Jarred Sumner
46e1c5a0fa Downgrade mimalloc + set libc musl flag (#21684)
### What does this PR do?

### How did you verify your code works?

---------

Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-08 18:02:19 -07:00
Jarred Sumner
df61e88dc0 Fix potential crash in new Bun.Transpiler() (#21650)
### What does this PR do?

The `then` function in `transpiler.transform` can cause GC, which means
it can cause the `Transpiler` to become freed, which means that if that
same transpiler is in use by another run on the other thread, it could
have pointers to invalid memory.

Also, `ESMCondition` has unnecesasry memory allocations and there is a
very tiny memory leak in optionsFromLoaders

### How did you verify your code works?

Existing tests

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-07 15:16:44 -07:00
pfg
3652008b0d Update bun:test diff (#21158)
Fixes #6229 (Fixes BAPI-655): 

|before|<img width="806" height="84" alt="image"
src="https://github.com/user-attachments/assets/6d6c8628-40a8-4950-a7a4-8a85ee07a302"
/>|
|-|-|
|after|<img width="802" height="87" alt="image"
src="https://github.com/user-attachments/assets/c336a626-2b08-469e-aa73-676f43a0f176"
/>|

Fixes #21498 (Fixes BAPI-2240), Fixes #10852 (Fixes BAPI-743):

|before|after|
|-|-|
|<img width="474" height="147" alt="image"
src="https://github.com/user-attachments/assets/bf2225de-a573-4672-a095-f9ff359ec86c"
/>|<img width="283" height="226" alt="image"
src="https://github.com/user-attachments/assets/89cb0e45-b1b7-4dbb-9ddb-b9835baa4b74"
/>|
|<img width="279" height="176" alt="image"
src="https://github.com/user-attachments/assets/e9be7308-dc38-43d2-901c-c77ce4757a51"
/>|<img width="278" height="212" alt="image"
src="https://github.com/user-attachments/assets/8c29b385-a053-4606-9474-3e5c0e60278c"
/>|

Improves multiline string and long output

|before|after|
|-|-|
|<img width="537" height="897" alt="image"
src="https://github.com/user-attachments/assets/034800c5-ab22-4915-90d9-19831906bb2e"
/>|<img width="345" height="1016" alt="image"
src="https://github.com/user-attachments/assets/fa95339e-c136-4c7c-af94-5f11400836dd"
/>|

Improves long single line string output

|before|<img width="1903" height="191" alt="image"
src="https://github.com/user-attachments/assets/bae35c81-0566-4291-810e-e65dc0381aef"
/>|
|-|-|
|after|<img width="1905" height="123" alt="image"
src="https://github.com/user-attachments/assets/bf9f492a-1d52-4cfc-9b1b-c6544a072814"
/>|

Puts 'expected' before 'received' on object diffs. The new version
matches Jest and Vitest, and I find it more intuitive:

|before|after|
|-|-|
|<img width="344" height="221" alt="image"
src="https://github.com/user-attachments/assets/44d42655-c441-411e-9b67-c0db7a5dce08"
/>|<img width="342" height="293" alt="image"
src="https://github.com/user-attachments/assets/565e3934-a2a2-4f99-9d6f-b7df1905f933"
/>|

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-06 06:44:46 -07:00
Jarred Sumner
455f3a65b9 enable mimalloc simd (#21644)
### What does this PR do?

### How did you verify your code works?
2025-08-06 06:38:34 -07:00
Meghan Denny
4d301cc3c4 deps: bump WebKit (#21647)
642e2252f6...75f6499360
2025-08-06 06:35:55 -07:00
Jarred Sumner
ccbd3f3575 Update BuildMimalloc.cmake 2025-08-05 22:27:01 -07:00
pfg
a72d74e09a Split JS parser into multiple files (#20880)
Splits up js_parser.zig into multiple files. Also changes visitExprInOut
to use function calls rather than switch

Not ready:

- [ ] P.zig is ~70,000 tokens, still needs to get smaller
- [x] ~~measure zig build time before & after (is it slower?)~~ no
significant impact

---------

Co-authored-by: pfgithub <6010774+pfgithub@users.noreply.github.com>
2025-08-05 20:52:16 -07:00
Jarred Sumner
d3d08eeb2d CI: disable --icf=safe in debug builds and asan builds 2025-08-04 18:34:47 -07:00
Jarred Sumner
47727bdbe3 CI: Add ENABLE_ZIG_ASAN option to enable/disable asan for zig specifically with a value that inherits from ENABLE_ASAN 2025-08-04 18:27:15 -07:00
Jarred Sumner
4494353abf Split up some of sys.zig into more files (#21603) 2025-08-04 07:02:06 -07:00
github-actions[bot]
276eee74eb deps: update hdrhistogram to 0.11.8 (#21575)
## What does this PR do?

Updates hdrhistogram to version 0.11.8

Compare:
652d51bcc3...8dcce8f685

Auto-updated by [this
workflow](https://github.com/oven-sh/bun/actions/workflows/update-hdrhistogram.yml)

Co-authored-by: Jarred-Sumner <Jarred-Sumner@users.noreply.github.com>
2025-08-03 18:13:53 -07:00
github-actions[bot]
711de8a667 deps: update libarchive to v3.8.1 (#21574)
## What does this PR do?

Updates libarchive to version v3.8.1

Compare:
7118f97c26...9525f90ca4

Auto-updated by [this
workflow](https://github.com/oven-sh/bun/actions/workflows/update-libarchive.yml)

Co-authored-by: RiskyMH <RiskyMH@users.noreply.github.com>
2025-08-02 23:30:07 -07:00
Jarred Sumner
a5af485354 Refactor h2_frame_parser to use GC-visited fields (#21573)
### What does this PR do?

Instead of holding a strong for the options object passed with the
handlers, we make each of the callbacks kept alive by the handlers and
it detaches once the detachFromJS function is called.

This should fix #21570, which looks like it was caused by wrapper
functions for AsyncLocalStorage getting collected prematurely.

fixes #21254
fixes #21553
fixes #21422

### How did you verify your code works?

Ran test/js/node/http2/node-http2.test.js
2025-08-02 20:38:49 -07:00
Jarred Sumner
68d322f05f Fix mimalloc memory usage regression (#21550)
### What does this PR do?

### How did you verify your code works?

---------

Co-authored-by: taylor.fish <contact@taylor.fish>
2025-08-01 23:38:34 -07:00
Dylan Conway
53b24ace79 sync webkit (#21436)
### What does this PR do?

<!-- **Please explain what your changes do**, example: -->

<!--

This adds a new flag --bail to bun test. When set, it will stop running
tests after the first failure. This is useful for CI environments where
you want to fail fast.

-->

### How did you verify your code works?
ran fuzzy-wuzzy.test.ts
<!-- **For code changes, please include automated tests**. Feel free to
uncomment the line below -->

<!-- I wrote automated tests -->

<!-- If JavaScript/TypeScript modules or builtins changed:

- [ ] I included a test for the new code, or existing tests cover it
- [ ] I ran my tests locally and they pass (`bun-debug test
test-file-name.test`)

-->

<!-- If Zig files changed:

- [ ] I checked the lifetime of memory allocated to verify it's (1)
freed and (2) only freed when it should be
- [ ] I included a test for the new code, or an existing test covers it
- [ ] JSValue used outside of the stack is either wrapped in a
JSC.Strong or is JSValueProtect'ed
- [ ] I wrote TypeScript/JavaScript tests and they pass locally
(`bun-debug test test-file-name.test`)
-->

<!-- If new methods, getters, or setters were added to a publicly
exposed class:

- [ ] I added TypeScript types for the new methods, getters, or setters
-->

<!-- If dependencies in tests changed:

- [ ] I made sure that specific versions of dependencies are used
instead of ranged or tagged versions
-->

<!-- If a new builtin ESM/CJS module was added:

- [ ] I updated Aliases in `module_loader.zig` to include the new module
- [ ] I added a test that imports the module
- [ ] I added a test that require() the module
-->
2025-07-30 15:49:15 -07:00
taylor.fish
3de884f2c9 Add helper type to detect unsynchronized concurrent accesses of shared data (#21476)
Add a helper type to help detect race conditions. There's no performance
or memory use penalty in release builds.

Actually adding the type to various places will be left for future PRs.

(For internal tracking: fixes STAB-852)
2025-07-30 00:46:42 -07:00
Zack Radisic
3d6dda6901 Add asan checks to HiveArray (#21449) 2025-07-29 19:35:46 -07:00
Jarred Sumner
93f92658b3 Try mimalloc v3 (#17378)
(For internal tracking: fixes ENG-19852)

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Kai Tamkun <kai@tamkun.io>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: taylor.fish <contact@taylor.fish>
2025-07-29 18:07:15 -07:00