Commit Graph

1058 Commits

Author SHA1 Message Date
robobun
e7672b2d04 Add string fast path for postMessage and structuredClone (#21926)
## Summary

Implements a string fast path optimization for `postMessage` and
`structuredClone` operations that provides significant performance
improvements for string-only data transfer, along with various bug fixes
and infrastructure improvements.

## Key Performance Improvements

**postMessage with Workers:**
- **Small strings (11 chars):** ~5% faster (572ns vs 599ns)
- **Medium strings (14KB):** **~2.7x faster** (528ns vs 1.40μs) 
- **Large strings (3MB):** **~660x faster** (540ns vs 356μs)

**Compared to Node.js postMessage:**
- Similar performance for small strings
- Competitive for medium strings  
- **~455x faster** for large strings (540ns vs 245μs)

## Implementation Details

The optimization adds a **string fast path** that bypasses full
structured cloning serialization when:
- Input is a pure string (`value.isString()`)
- No transfer list or message ports are involved
- Not being stored persistently

### Core Changes

**String Thread-Safety Utilities (`BunString.cpp/h`):**
- `isCrossThreadShareable()` - Checks if string can be safely shared
across threads
- `toCrossThreadShareable()` - Converts strings to thread-safe form via
`isolatedCopy()`
- Handles edge cases: atoms, symbols, substring slices, external buffers

**Serialization Fast Path (`SerializedScriptValue.cpp`):**
- New `m_fastPathString` field stores string data directly
- Bypasses full object serialization machinery for pure strings
- Creates isolated copies for cross-thread safety

**Deserialization Fast Path:**
- Directly returns JSString from stored string data
- Avoids parsing serialized byte streams

**Updated Flags System (`JSValue.zig`, `Serialization.cpp`):**
- Replaces boolean `forTransfer` with structured `SerializedFlags`
- Supports `forCrossProcessTransfer` and `forStorage` distinctions

**Structured Clone Infrastructure:**
- Moved `structuredClone` implementation to dedicated
`StructuredClone.cpp`
- Added `jsFunctionStructuredCloneAdvanced` for testing with custom
flags
- Improved class serialization compatibility checks (`isForTransfer`,
`isForStorage`)

**IPC Improvements (`ipc.zig`):**
- Fixed race conditions in `SendQueue` by deferring cleanup to next tick
- Proper fd ownership handling with `bun.take()`
- Cached IPC serialize/parse functions for better performance

**BlockList Thread Safety Fixes (`BlockList.zig`):**
- Fixed potential deadlocks by moving mutex locking inside methods
- Added atomic `estimated_size` counter to avoid lock during GC
- Corrected pointer handling in comparison functions
- Improved GC safety in `rules()` method

## Benchmark Results

```
❯ bun-21926 bench/string-postmessage.mjs  # This branch
postMessage(11 chars string)  572.24 ns/iter
postMessage(14 KB string)     527.55 ns/iter  ← ~2.7x faster
postMessage(3 MB string)      539.70 ns/iter  ← ~660x faster

❯ bun-1.2.20 bench/string-postmessage.mjs  # Previous
postMessage(11 chars string)  598.76 ns/iter
postMessage(14 KB string)       1.40 µs/iter
postMessage(3 MB string)      356.38 µs/iter

❯ node bench/string-postmessage.mjs       # Node.js comparison  
postMessage(11 chars string)  569.63 ns/iter
postMessage(14 KB string)       1.46 µs/iter
postMessage(3 MB string)      245.46 µs/iter
```

**Key insight:** The fast path achieves **constant time performance**
regardless of string size (~540ns), while traditional serialization
scales linearly with data size.

## Test Coverage

**New Tests:**
- `test/js/web/structured-clone-fastpath.test.ts` - Fast path memory
usage validation
- `test/js/web/workers/structuredClone-classes.test.ts` - Comprehensive
class serialization tests
  - Tests ArrayBuffer transferability 
  - Tests BunFile cloning with storage/transfer restrictions
  - Tests net.BlockList cloning behavior
  - Validates different serialization contexts (default, worker, window)

**Enhanced Tests:**
- `test/js/web/workers/structured-clone.test.ts` - Multi-function
testing
- Tests `structuredClone`, `jscSerializeRoundtrip`, and cross-process
serialization
  - Validates consistency across different serialization paths
- `test/js/node/cluster.test.ts` - Better error handling and debugging

**Benchmarks:**
- `bench/string-postmessage.mjs` - Worker postMessage performance
comparison
- `bench/string-fastpath.mjs` - Fast path vs traditional serialization
comparison

## Bug Fixes

**BlockList Threading Issues:**
- Fixed potential deadlocks when multiple threads access BlockList
simultaneously
- Moved mutex locks inside methods rather than holding across entire
function calls
- Added atomic size tracking for GC compatibility
- Fixed comparison function pointer handling

**IPC Race Conditions:**
- Fixed race condition where `SendQueue._onAfterIPCClosed()` could be
called on wrong thread
- Deferred cleanup operations to next tick using task queue
- Improved file descriptor ownership with proper `bun.take()` usage

**Structured Clone Compatibility:**
- Enhanced class serialization with proper transfer/storage mode
checking
- Fixed edge cases where non-transferable objects were incorrectly
handled
- Added better error reporting for unsupported clone operations

## Technical Notes

- Thread safety ensured via `String.isolatedCopy()` for cross-VM
transfers
- Memory cost calculation updated to account for string references
- Maintains full compatibility with existing structured clone semantics
- Does not affect object serialization or transfer lists
- Proper cleanup and error handling throughout IPC pipeline

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Meghan Denny <meghan@bun.sh>
2025-08-20 00:25:00 -07:00
Jarred Sumner
9b363e4ef6 Skip this test for now 2025-08-19 21:51:25 -07:00
Michael H
49f33c948a fix regression in node:crypto with lowercase rsa-sha keys (#21812)
### What does this PR do?

there was a regression in 1.2.5 where it stopped supporting lowercase
veriants of the crypto keys. This broke the `mailauth` lib and proabibly
many more.

simple code:
```ts
import { sign, constants } from 'crypto';

const DUMMY_PRIVATE_KEY = `-----BEGIN PRIVATE KEY-----\r\nMIICeAIBADANBgkqhkiG9w0BAQEFAASCAmIwggJeAgEAAoGBAMx5bEJhDzwNBG1m\r\nmIYn/V1HMK9g8WTVaHym4F4iPcTdZ4RYUrMa/xOUwPMAfrOJdf3joSUFWBx3ZPdW\r\nhrvpqjmcmgoYDRJzZwVKJ1uqTko6Anm3gplWl6JP3nGOL9Vt5K5xAJWif5fHPfCx\r\nLA2p/SnJDNmcyOWURUCRVCDlZgJRAgMBAAECgYEAt8a+ZZ7EyY1NmGJo3dMdZnPw\r\nrwArlhw08CwwZorSB5mTS6Dym2W9MsU08nNUbVs0AIBRumtmOReaWK+dI1GtmsT+\r\n/5YOrE8aU9xcTgMzZjr9AjI9cSc5J9etqqTjUplKfC5Ay0WBhPlx66MPAcTsq/u/\r\nIdPYvhvgXuJm6X3oDP0CQQDllIopSYXW+EzfpsdTsY1dW+xKM90NA7hUFLbIExwc\r\nvL9dowJcNvPNtOOA8Zrt0guVz0jZU/wPYZhvAm2/ab93AkEA5AFCfcAXrfC2lnDe\r\n9G5x/DGaB5jAsQXi9xv+/QECyAN3wzSlQNAZO8MaNr2IUpKuqMfxl0sPJSsGjOMY\r\ne8aOdwJBAIM7U3aiVmU5bgfyN8J5ncsd/oWz+8mytK0rYgggFFPA+Mq3oWPA7cBK\r\nhDly4hLLnF+4K3Y/cbgBG7do9f8SnaUCQQCLvfXpqp0Yv4q4487SUwrLff8gns+i\r\n76+uslry5/azbeSuIIsUETcV+LsNR9bQfRRNX9ZDWv6aUid+nAU6f3R7AkAFoONM\r\nmr4hjSGiU1o91Duatf4tny1Hp/hw2VoZAb5zxAlMtMifDg4Aqg4XFgptST7IUzTN\r\nK3P7zdJ30gregvjI\r\n-----END PRIVATE KEY-----`;

sign('rsa-sha256', Buffer.from('message'), {
    key: DUMMY_PRIVATE_KEY,
    padding: constants.RSA_PKCS1_PSS_PADDING,
});
// would throw invalid digest
```

### How did you verify your code works?

made test

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-13 19:38:01 -07:00
pfg
408fda7ad2 Continue emitting 'readable' events after pausing stdin (#17690)
Fixes #21189

`.pause()` should unref but it should still continue to emit `readable`
events (although it should not send `data` events)

also stdin.unref() should not pause input, it should only prevent stdin
from keeping the process alive.

DRAFT:

- [x] ~~this causes a bug where `process.stdin.on("readable", () => {});
process.stdin.pause()` will allow the process to exit when it
shouldn't.~~ fixed

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-04 21:04:08 -07:00
Ciro Spaciari
ed6f099e5e fix(tls) fix ciphers (#21545)
### What does this PR do?
Uses same ciphers than node.js for compatibility and do the same error
checking on empty ciphers
Fixes https://github.com/oven-sh/bun/issues/9425
Fixes https://github.com/oven-sh/bun/issues/21518
Fixes https://github.com/oven-sh/bun/issues/19859
Fixes https://github.com/oven-sh/bun/issues/18980

You can see more about redis ciphers here
https://redis.io/docs/latest/operate/rs/security/encryption/tls/ciphers/
this should fix redis related ciphers issues
### How did you verify your code works?
Tests

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-04 19:42:40 -07:00
Ciro Spaciari
4bbe32fff8 fix(net/http2) fix socket internal timeout and owner_symbol check, fix padding support in http2 (#21263)
### What does this PR do?

<!-- **Please explain what your changes do**, example: -->

<!--

This adds a new flag --bail to bun test. When set, it will stop running
tests after the first failure. This is useful for CI environments where
you want to fail fast.

-->

- [ ] Documentation or TypeScript types (it's okay to leave the rest
blank in this case)
- [x] Code changes

### How did you verify your code works?
Tests added for padding support
Timeout of socket is being fired earlier due to backpressure or lack of
precision in usockets timers (now matchs node.js behavior).
Added check for owner_symbol so the error showed in
https://github.com/oven-sh/bun/issues/21055 is handled

<!-- **For code changes, please include automated tests**. Feel free to
uncomment the line below -->

<!-- I wrote automated tests -->

<!-- If JavaScript/TypeScript modules or builtins changed:

- [ ] I included a test for the new code, or existing tests cover it
- [ ] I ran my tests locally and they pass (`bun-debug test
test-file-name.test`)

-->

<!-- If Zig files changed:

- [ ] I checked the lifetime of memory allocated to verify it's (1)
freed and (2) only freed when it should be
- [ ] I included a test for the new code, or an existing test covers it
- [ ] JSValue used outside of the stack is either wrapped in a
JSC.Strong or is JSValueProtect'ed
- [ ] I wrote TypeScript/JavaScript tests and they pass locally
(`bun-debug test test-file-name.test`)
-->

<!-- If new methods, getters, or setters were added to a publicly
exposed class:

- [ ] I added TypeScript types for the new methods, getters, or setters
-->

<!-- If dependencies in tests changed:

- [ ] I made sure that specific versions of dependencies are used
instead of ranged or tagged versions
-->

<!-- If a new builtin ESM/CJS module was added:

- [ ] I updated Aliases in `module_loader.zig` to include the new module
- [ ] I added a test that imports the module
- [ ] I added a test that require() the module
-->

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-07-29 17:20:16 -07:00
Meghan Denny
bbdc3ae055 node: sync updated tests (#21147) 2025-07-25 19:14:22 -07:00
Meghan Denny
81e08d45d4 node: fix test-worker-uncaught-exception-async.js (#21150) 2025-07-25 19:13:48 -07:00
190n
1ab76610cf [STAB-861] Suppress known-benign core dumps in CI (#21321)
### What does this PR do?

- for these kinds of aborts which we test in CI, introduce a feature
flag to suppress core dumps and crash reporting only from that abort,
and set the flag when running the test:
    - libuv stub functions
- Node-API abort (used in particular when calling illegal functions
during finalizers)
    - passing `process.kill` its own PID
- core dumps are suppressed with `setrlimit`, and crash reporting with
the new `suppress_reporting` field. these suppressions are only engaged
right before crashing, so we won't ignore new kinds of crashes that come
up in these tests.
- for the test bindings used to test the crash handler in
`run-crash-handler.test.ts`, disables core dumps but does not disable
crash reporting (because crashes get reported to a server that the test
is running to make sure they are reported)
- fixes a panic when printing source code around an error containing
`\n\r`
- updates the code where we clone vendor tests to checkout the right tag
- adds `vendor/elysia/test/path/plugin.test.ts` to
no-validate-exceptions
- this failure was exposed by starting to test the version of elysia we
have been intending to test. the crash trace suggests it may be fixed by
#21307.
- makes dumping core or uploading a crash report count as a failing test
- this ensures we don't realize a crash has occurred if it happened in a
subprocess and the main test doesn't adequately check the exit code. to
spawn a subprocess you expect to fail, prefer `expect(code).toBe(1)`
over `expect(code).not.toBe(0)`. if you really expect multiple possible
erroneous exit codes, you might try `expect(signal).toBeNull()` to still
disallow crashes.

### How did you verify your code works?

Running affected tests on a Linux machine with core dumps set up and
checking no new ones appear.

https://buildkite.com/bun/bun/builds/21465 has no core dumps.
2025-07-25 16:22:04 -07:00
Meghan Denny
72a6278b3f Delete test/js/node/test/parallel/test-http-url.parse-https.request.js
flaky on macos ci atm
2025-07-24 19:06:38 -07:00
Meghan Denny
bd232189b4 node: fix test-http-server-keepalive-req-gc.js (#21333) 2025-07-24 13:46:50 -07:00
Meghan Denny
87df7527bb node: fix test-http-url.parse-https.request.js (#21335) 2025-07-24 13:44:55 -07:00
190n
a1f756fea9 Fix running bun test on multiple node:test tests (#19354)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-07-24 11:48:55 -07:00
Meghan Denny
6d2a0e30f5 test: node: revert the previous tmpdirName commit 2025-07-21 21:34:00 -07:00
Meghan Denny
d2a4fb8124 test: node: much more robust tmpdirName for use with --parallel 2025-07-21 15:56:32 -07:00
Meghan Denny
cc4f840e8b test: remove accidental uses of test.only (#20097) 2025-07-16 23:07:23 -07:00
Michael H
15b7cd8c18 node:fs.glob support array of excludes and globs (#20883) 2025-07-16 03:08:54 -07:00
Meghan Denny
875604a42b safety: a lot more exception checker progress (#20956) 2025-07-16 00:11:19 -07:00
Michael H
e8289cc3ab fix os.networkInterfaces where scope_id should be scopeid (#21026) 2025-07-14 02:48:20 -07:00
Jarred Sumner
2e02d9de28 Use ReadableStream.prototype.* in tests instead of new Response(...).* (#20937)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Alistair Smith <hi@alistair.sh>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-14 00:47:53 -07:00
Meghan Denny
dbd577cde6 ci: download exact version of node specifed (#20936)
Co-authored-by: nektro <5464072+nektro@users.noreply.github.com>
2025-07-12 00:53:52 -07:00
190n
c49547cfa4 Fix #20972 (#20974) 2025-07-11 15:23:29 -07:00
Meghan Denny
72b5c0885a test: fix process.test.js (#20932) 2025-07-09 23:02:43 -07:00
Jarred Sumner
454316ffc3 Implement "node:module"'s findSourceMap and SourceMap class (#20863)
Co-authored-by: Claude <claude@anthropic.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2025-07-07 23:08:12 -07:00
Kai Tamkun
05e8a6dd4d Add support for vm.constants.DONT_CONTEXTIFY (#20088) 2025-07-07 19:29:53 -07:00
Adam
7ba4b1d01e Fix: deprecated goo.gl links in snapshots raised in issue #20086 (#20424) 2025-07-05 00:58:42 -07:00
Michael H
0612dc7bd9 Support process.features.typescript (#20801) 2025-07-03 16:26:32 -07:00
Meghan Denny
01e2cb25e3 test: bump zlib/leak.test.ts for asan 2025-07-02 21:48:24 -07:00
Jarred Sumner
5a7b5ceb33 Fix several missing async context tracking callbacks (#20759)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-07-02 17:45:00 -07:00
Ben Grant
79284376ca Delete test-net-bytes-stats.js
Mistakenly re-added in #20659 after prior revert in #20693
2025-07-02 17:32:19 -07:00
Alistair Smith
452000a2ce Node.js test test-fs-watchfile.js (#20773) 2025-07-02 15:04:39 -07:00
190n
172aecb02e [publish images] Upgrade self-reported Node.js version from 22.6.0 to 24.3.0 (v2) (#20772)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Meghan Denny <meghan@bun.sh>
Co-authored-by: Ashcon Partovi <ashcon@partovi.net>
Co-authored-by: pfg <pfg@pfg.pw>
Co-authored-by: pfgithub <6010774+pfgithub@users.noreply.github.com>
Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-07-02 12:06:08 -07:00
Ben Grant
ea57037567 Revert "Upgrade self-reported Node.js version from 22.6.0 to 24.3.0 (#20659) [publish images]"
This reverts commit 80309e4d59. It breaks the Windows CI.
2025-07-02 09:40:32 -07:00
Jarred Sumner
80309e4d59 Upgrade self-reported Node.js version from 22.6.0 to 24.3.0 (#20659) [publish images]
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Meghan Denny <meghan@bun.sh>
Co-authored-by: Ashcon Partovi <ashcon@partovi.net>
Co-authored-by: pfg <pfg@pfg.pw>
Co-authored-by: pfgithub <6010774+pfgithub@users.noreply.github.com>
Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: Ben Grant <ben@bun.sh>
2025-07-02 00:03:05 -07:00
Jarred Sumner
fbe405fb89 Fail the test when no tests match the filter (#20749)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-07-01 16:03:19 -07:00
Meghan Denny
ca59ed04bd node: sync updated tests (#20625) 2025-06-30 14:52:50 -07:00
Jarred Sumner
034bcf2b57 Deflake
These tests should not have been marked as passing.

test/js/node/test/parallel/test-cluster-worker-kill-signal.js
test/js/node/test/parallel/test-child-process-prototype-tampering.mjs
2025-06-28 23:45:34 -07:00
Jarred Sumner
386743b508 Revert "node: fix test-net-bytes-stats.js" (#20693) 2025-06-27 17:38:18 -07:00
Meghan Denny
1789f92991 node: fix test-net-server-capture-rejection.js (#20646) 2025-06-27 14:42:52 -07:00
Meghan Denny
c863341bf4 node: fix test-net-connect-custom-lookup-non-string-address.mjs (#20648) 2025-06-27 14:42:25 -07:00
Meghan Denny
ea7068a531 node: fix test-net-bytes-stats.js (#20629) 2025-06-26 01:48:23 -07:00
Meghan Denny
4cf31f6a57 node: sync test-net-connect-options-invalid.js (#20607) 2025-06-24 22:46:56 -07:00
Meghan Denny
3f257a2905 node: fix test-timers-invalid-clear.js (#20624) 2025-06-24 22:39:20 -07:00
Jarred Sumner
90e3d6c898 Fix flushHeaders streaming (#20577)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
2025-06-24 18:48:23 -07:00
Meghan Denny
e8b652a5d9 node: fix test-net-better-error-messages-path.js (#20609) 2025-06-24 17:48:06 -07:00
Meghan Denny
354391a263 zig: socket: fix uaf in handleError, onOpen, onWritable (#20604) 2025-06-24 17:31:58 -07:00
Ciro Spaciari
17120cefdc fix(http2) fix remoteSettings not emitting on default settings (#20622) 2025-06-24 17:31:19 -07:00
Meghan Denny
7d9dd67586 node: fix test-net-connect-keepalive.js (#20610) 2025-06-24 02:56:50 -07:00
Meghan Denny
ccb0ed13c2 node: fix test-net-listen-fd0.js (#20611) 2025-06-24 02:56:40 -07:00
Meghan Denny
050a9cecb7 node: sync test-net-connect-no-arg.js (#20608) 2025-06-24 01:02:38 -07:00