mirror of
https://github.com/oven-sh/bun
synced 2026-02-10 10:58:56 +00:00
9d4ec5a03a263fb7d197bf1fcedc252ad2fa2568
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Meghan Denny
|
51ce3bc269 | [publish images] ci: ensure tests that require docker have it available (#22781) | ||
|
pfg
|
144c45229e |
bun.ptr.Shared.Lazy fixes cppbind change (#22753)
shared lazy:
- cloneWeak didn't incrementWeak. fixed
- exposes a public Optional so you can do `bun.ptr.Shared(*T).Optional`
- the doc comment for 'take' said it set self to null. but it did not.
fixed.
- upgrading a weak to a strong incremented the weak instead of
decrementing it. fixed.
- adds a new method unsafeGetStrongFromPointer. this is currently unused
but used in pfg/describe-2:
|
||
|
pfg
|
d2201eb1fe |
Rewrite test/describe, add test.concurrent (#22534)
# bun test Fixes #8768, Fixes #14624, Fixes #20100, Fixes #19875, Fixes #14135, Fixes #20980, Fixes #21830, Fixes #5738, Fixes #19758, Fixes #12782, Fixes #5585, Fixes #9548, Might fix 5996 # New features: ## Concurrent tests Concurrent tests allow running multiple async tests at the same time. ```ts // concurrent.test.ts test.concurrent("this takes a while 1", async () => { await Bun.sleep(1000); }); test.concurrent("this takes a while 2", async () => { await Bun.sleep(1000); }); test.concurrent("this takes a while 3", async () => { await Bun.sleep(1000); }); ``` Without `.concurrent`, this test file takes 3 seconds to run because each one has to wait for the one before it to finish before it can start. With `.concurrent`, this file takes 1 second because all three sleeps can run at once. ``` $> bun-after test concurrent concurrent.test.js: ✓ this takes a while 1 [1005.36ms] ✓ this takes a while 2 [1012.51ms] ✓ this takes a while 3 [1013.15ms] 3 pass 0 fail Ran 3 tests across 1 file. [1081.00ms] ``` To run all tests as concurrent, pass the `--concurrent` flag when running tests. Limitations: - concurrent tests cannot attribute `expect()` call counts to the test, meaning `expect.assertions()` does not function - concurrent tests cannot use `toMatchSnapshot`. `toMatchInlineSnapshot` is still supported. - `beforeAll`/`afterAll` will never be executed concurrently. `beforeEach`/`afterEach` will. ## Chaining Chaining multiple describe/test qualifiers is now allowed. Previously, it would fail. ```ts // chaining-test-qualifiers.test.ts test.failing.each([1, 2, 3])("each %i", async i => { throw new Error(i); }); ``` ``` $> bun-after test chaining-test-qualifiers a.test.js: ✓ each 1 ✓ each 2 ✓ each 3 ``` # Breaking changes: ## Describe ordering Previously, describe callbacks were called immediately. Now, they are deferred until the outer callback has finished running. The previous order matched Jest. The new order is similar to Vitest, but does not match exactly. ```ts // describe-ordering.test.ts describe("outer", () => { console.log("outer before"); describe("inner", () => { console.log("inner"); }); console.log("outer after"); }); ``` Before, this would print ``` $> bun-before test describe-ordering outer before inner outer after ``` Now, this will print ``` $> bun-after test describe-ordering outer before outer after inner ``` ## Test ordering Describes are no longer always called before tests. They are now in order. ```ts // test-ordering.test.ts test("one", () => {}); describe("scope", () => { test("two", () => {}); }); test("three", () => {}); ``` Before, this would print ``` $> bun-before test test-ordering ✓ scope > two ✓ one ✓ three ``` Now, this will print ``` $> bun-after test test-ordering ✓ one ✓ scope > two ✓ three ``` ## Preload hooks Previously, beforeAll in a preload ran before the first file and afterAll ran after the last file. Now, beforeAll will run at the start of each file and afterAll will run at the end of each file. This behaviour matches Jest and Vitest. ```ts // preload.ts beforeAll(() => console.log("preload: beforeAll")); afterAll(() => console.log("preload: afterAll")); ``` ```ts // preload-ordering-1.test.ts test("demonstration file 1", () => {}); ``` ```ts // preload-ordering-2.test.ts test("demonstration file 2", () => {}); ``` ``` $> bun-before test --preload=./preload preload-ordering preload-ordering-1.test.ts: preload: beforeAll ✓ demonstration file 1 preload-ordering-2.test.ts: ✓ demonstration file 2 preload: afterAll ``` ``` $> bun-after test --preload=./preload preload-ordering preload-ordering-1.test.ts: preload: beforeAll ✓ demonstration file 1 preload: afterAll preload-ordering-2.test.ts: preload: beforeAll ✓ demonstration file 2 preload: afterAll ``` ## Describe failures Current behaviour is that when an error is thrown inside a describe callback, none of the tests declared there will run. Now, describes declared inside will also not run. The new behaviour matches the behaviour of Jest and Vitest. ```ts // describe-failures.test.ts describe("erroring describe", () => { test("this test does not run because its describe failed", () => { expect(true).toBe(true); }); describe("inner describe", () => { console.log("does the inner describe callback get called?"); test("does the inner test run?", () => { expect(true).toBe(true); }); }); throw new Error("uh oh!"); }); ``` Before, the inner describe callback would be called and the inner test would run, although the outer test would not: ``` $> bun-before test describe-failures describe-failures.test.ts: does the inner describe callback get called? # Unhandled error between tests ------------------------------- 11 | throw new Error("uh oh!"); ^ error: uh oh! ------------------------------- ✓ erroring describe > inner describe > does the inner test run? 1 pass 0 fail 1 error 1 expect() calls Ran 1 test across 1 file. Exited with code [1] ``` Now, the inner describe callback is not called at all. ``` $> bun-after test describe-failures describe-failures.test.ts: # Unhandled error between tests ------------------------------- 11 | throw new Error("uh oh!"); ^ error: uh oh! ------------------------------- 0 pass 0 fail 1 error Ran 0 tests across 1 file. Exited with code [1] ``` ## Hook failures Previously, a beforeAll failure would skip subsequent beforeAll()s, the test, and the afterAll. Now, a beforeAll failure skips any subsequent beforeAll()s and the test, but not the afterAll. ```js beforeAll(() => { throw new Error("before all: uh oh!"); }); test("my test", () => { console.log("my test"); }); afterAll(() => console.log("after all")); ``` ``` $> bun-before test hook-failures Error: before all: uh oh! $> bun-after test hook-failures Error: before all: uh oh! after all ``` Previously, an async beforeEach failure would still allow the test to run. Now, an async beforeEach failure will prevent the test from running ```js beforeEach(() => { await 0; throw "uh oh!"; }); it("the test", async () => { console.log("does the test run?"); }); ``` ``` $> bun-before test async-beforeeach-failure does the test run? error: uh oh! uh oh! ✗ the test $> bun-after test async-beforeeach-failure error: uh oh! uh oh! ✗ the test ``` ## Hook timeouts Hooks will now time out, and can have their timeout configured in an options parameter ```js beforeAll(async () => { await Bun.sleep(1000); }, 500); test("my test", () => { console.log("ran my test"); }); ``` ``` $> bun-before test hook-timeouts ran my test Ran 1 test across 1 file. [1011.00ms] $> bun-after test hook-timeouts ✗ my test [501.15ms] ^ a beforeEach/afterEach hook timed out for this test. ``` ## Hook execution order beforeAll will now execute before the tests in the scope, rather than immediately when it is called. ```ts describe("d1", () => { beforeAll(() => { console.log("<d1>"); }); test("test", () => { console.log(" test"); }); afterAll(() => { console.log("</d1>"); }); }); describe("d2", () => { beforeAll(() => { console.log("<d2>"); }); test("test", () => { console.log(" test"); }); afterAll(() => { console.log("</d2>"); }); }); ``` ``` $> bun-before test ./beforeall-ordering.test.ts <d1> <d2> test </d1> test </d2> $> bun-after test ./beforeall-ordering.test.ts <d1> test </d1> <d2> test </d2> ``` ## test inside test test() inside test() now errors rather than silently failing. Support for this may be added in the future. ```ts test("outer", () => { console.log("outer"); test("inner", () => { console.log("inner"); }); }); ``` ``` $> bun-before test outer ✓ outer [0.06ms] 1 pass 0 fail Ran 1 test across 1 file. [8.00ms] $> bun-after test outer 1 | test("outer", () => { 2 | console.log("outer"); 3 | test("inner", () => { ^ error: Cannot call test() inside a test. Call it inside describe() instead. ✗ outer [0.71ms] 0 pass 1 fail ``` ## afterAll inside test afterAll inside a test is no longer allowed ```ts test("test 1", () => { afterAll(() => console.log("afterAll")); console.log("test 1"); }); test("test 2", () => { console.log("test 2"); }); ``` ``` $> bun-before test 1 ✓ test 1 [0.05ms] test 2 ✓ test 2 afterAll $> bun-after error: Cannot call afterAll() inside a test. Call it inside describe() instead. ✗ test 1 [1.00ms] test 2 ✓ test 2 [0.20ms] ``` # Only inside only Previously, an outer 'describe.only' would run all tests inside it even if there was an inner 'test.only'. Now, only the innermost only tests are executed. ```ts describe.only("outer", () => { test("one", () => console.log("should not run")); test.only("two", () => console.log("should run")); }); ``` ``` $> bun-before test should not run should run $> bun-after test should run ``` With no inner only, the outer only will still run all tests: ```ts describe.only("outer", () => { test("test 1", () => console.log("test 1 runs")); test("test 2", () => console.log("test 2 runs")); }); ``` # Potential follow-up work - [ ] for concurrent tests, display headers before console.log messages saying which test it is for - this will need async context or similar - refActiveExecutionEntry should also be able to know the current test even in test.concurrent - [ ] `test("rerun me", () => { console.log("run one time!"); });` `--rerun-each=3` <- this runs the first and third time but not the second time. fix. - [ ] should to cache the JSValue created from DoneCallback.callAsFunction - [ ] implement retry and rerun params for tests. - [ ] Remove finalizer on ScopeFunctions.zig by storing the data in 3 jsvalues passed in bind rather than using a custom class. We should also migrate off of the ClassGenerator for ScopeFunctions - [ ] support concurrent limit, how many concurrent tests are allowed to run at a time. ie `--concurrent-limit=25` - [ ] flag to run tests in random order - [ ] `test.failing` should have its own style in the same way `test.todo` passing marks as 'todo' insetead of 'passing'. right now it's `✓` which is confusing. - [ ] remove all instances of bun.jsc.Jest.Jest.current - [ ] test options should be in BunTestRoot - [ ] we will need one global still, stored in the globalobject/vm/?. but it should not be a Jest instance. - [ ] consider allowing test() inside test(), as well as afterEach and afterAll. could even allow describe() too. to do this we would switch from indices to pointers and they would be in a linked list. they would be allocated in memorypools for perf/locality. some special consideration is needed for making sure repeated tests lose their temporary items. this could also improve memory usage soomewhat. - [ ] consider using a jsc Bound Function rather than CallbackWithArgs. bound functions allow adding arguments and they are only one value for GC instead of many. and this removes our unnecessary three copies. - [ ] eliminate Strong.Safe. we should be using a C++ class instead. - [ ] consider modifying the junit reporter to print the whole describe tree at the end instead of trying to output as test results come in. and move it into its own file. - [ ] expect_call_count/expect_assertions is confusing. rename to `expect_calls`, `assert_expect_calls`. or something. - [ ] Should make line_no be an enum with a none option and a function to get if line nombers are enabled - [ ] looks like we don't need to use file_id anymore (remove `bun.jsc.Jest.Jest.runner.?.getOrPutFile(file_path).file_id;`, store the file path directly) - [ ] 'dot' test reporter like vitest? - [ ] `test.failing.if(false)` errors because it can't replace mode 'failing' with mode 'skip'. this should probably be allowed instead. - [ ] trigger timeout termination exception for `while(true) {}` - [ ] clean up unused callbacks. as soon as we advance to the next execution group, we can fully clean out the previous one. sometimes within an execution sequence we can do the same. - clean by swapping held values with undefined - [ ] structure cache for performance for donecallback/scopefunctions - [ ] consider migrating CallbackWithArgs to be a bound function. the length of the bound function can exclude the specified args. - [ ] setting both result and maybe_skip is not ideal, maybe there should be a function to do both at once? - [ ] try using a linked list rather than arraylist for describe/test children, see how it affects performance - [ ] consider a memory pool for describescope/executionentry. test if it improves performance. - [ ] consider making RefDataValue methods return the reason for failure rather than ?value. that way we can improve error messages. the reason could be a string or it could be a defined error set - [ ] instead of 'description orelse (unnamed)', let's have description default to 'unnamed' and not free it if it === the global that defines that - [ ] Add a phase before ordering results that inherits properties to the parents. (eg inherit only from the child and inherit has_callback from the child. and has_callback can be on describe/test individually rather than on base). then we won't have that happening in an init() function (terrible!) - [ ] this test was incidentally passing because resolves.pass() wasn't waiting for promise ``` test("fetching with Request object - issue #1527", async () => { const server = createServer((req, res) => { res.end(); }).listen(0); try { await once(server, "listening"); const body = JSON.stringify({ foo: "bar" }); const request = new Request(`http://localhost:${server.address().port}`, { method: "POST", body, }); expect(fetch(request)).resolves.pass(); } finally { server.closeAllConnections(); } }); ``` - [ ] the error "expect.assertions() is not supported in the describe phase, in concurrent tests, between tests, or after test execution has completed" is not very good. we should be able to identify which of those it is and print the right error for the context - [ ] consider: instead of storing weak pointers to BunTest, we can instead give the instance an id and check that it is correct when getting the current bun test instance from the ref - [ ] auto_killer: add three layers of auto_killer: - preload (includes file & test) - file (includes test) - test - that way at the end of the test, we kill the test processes. at the end of the file, we kill the file processes. at the end of all, we kill anything remaining. AsyncLocalStorage - store active_id & refdatavalue. active_id is a replacement for the above weak pointers thing. refdatavalue is for determining which test it is. this probably fits in 2×u64 - use for auto_killer so timeouts can kill even in concurrent tests --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
||
|
robobun
|
e7672b2d04 |
Add string fast path for postMessage and structuredClone (#21926)
## Summary Implements a string fast path optimization for `postMessage` and `structuredClone` operations that provides significant performance improvements for string-only data transfer, along with various bug fixes and infrastructure improvements. ## Key Performance Improvements **postMessage with Workers:** - **Small strings (11 chars):** ~5% faster (572ns vs 599ns) - **Medium strings (14KB):** **~2.7x faster** (528ns vs 1.40μs) - **Large strings (3MB):** **~660x faster** (540ns vs 356μs) **Compared to Node.js postMessage:** - Similar performance for small strings - Competitive for medium strings - **~455x faster** for large strings (540ns vs 245μs) ## Implementation Details The optimization adds a **string fast path** that bypasses full structured cloning serialization when: - Input is a pure string (`value.isString()`) - No transfer list or message ports are involved - Not being stored persistently ### Core Changes **String Thread-Safety Utilities (`BunString.cpp/h`):** - `isCrossThreadShareable()` - Checks if string can be safely shared across threads - `toCrossThreadShareable()` - Converts strings to thread-safe form via `isolatedCopy()` - Handles edge cases: atoms, symbols, substring slices, external buffers **Serialization Fast Path (`SerializedScriptValue.cpp`):** - New `m_fastPathString` field stores string data directly - Bypasses full object serialization machinery for pure strings - Creates isolated copies for cross-thread safety **Deserialization Fast Path:** - Directly returns JSString from stored string data - Avoids parsing serialized byte streams **Updated Flags System (`JSValue.zig`, `Serialization.cpp`):** - Replaces boolean `forTransfer` with structured `SerializedFlags` - Supports `forCrossProcessTransfer` and `forStorage` distinctions **Structured Clone Infrastructure:** - Moved `structuredClone` implementation to dedicated `StructuredClone.cpp` - Added `jsFunctionStructuredCloneAdvanced` for testing with custom flags - Improved class serialization compatibility checks (`isForTransfer`, `isForStorage`) **IPC Improvements (`ipc.zig`):** - Fixed race conditions in `SendQueue` by deferring cleanup to next tick - Proper fd ownership handling with `bun.take()` - Cached IPC serialize/parse functions for better performance **BlockList Thread Safety Fixes (`BlockList.zig`):** - Fixed potential deadlocks by moving mutex locking inside methods - Added atomic `estimated_size` counter to avoid lock during GC - Corrected pointer handling in comparison functions - Improved GC safety in `rules()` method ## Benchmark Results ``` ❯ bun-21926 bench/string-postmessage.mjs # This branch postMessage(11 chars string) 572.24 ns/iter postMessage(14 KB string) 527.55 ns/iter ← ~2.7x faster postMessage(3 MB string) 539.70 ns/iter ← ~660x faster ❯ bun-1.2.20 bench/string-postmessage.mjs # Previous postMessage(11 chars string) 598.76 ns/iter postMessage(14 KB string) 1.40 µs/iter postMessage(3 MB string) 356.38 µs/iter ❯ node bench/string-postmessage.mjs # Node.js comparison postMessage(11 chars string) 569.63 ns/iter postMessage(14 KB string) 1.46 µs/iter postMessage(3 MB string) 245.46 µs/iter ``` **Key insight:** The fast path achieves **constant time performance** regardless of string size (~540ns), while traditional serialization scales linearly with data size. ## Test Coverage **New Tests:** - `test/js/web/structured-clone-fastpath.test.ts` - Fast path memory usage validation - `test/js/web/workers/structuredClone-classes.test.ts` - Comprehensive class serialization tests - Tests ArrayBuffer transferability - Tests BunFile cloning with storage/transfer restrictions - Tests net.BlockList cloning behavior - Validates different serialization contexts (default, worker, window) **Enhanced Tests:** - `test/js/web/workers/structured-clone.test.ts` - Multi-function testing - Tests `structuredClone`, `jscSerializeRoundtrip`, and cross-process serialization - Validates consistency across different serialization paths - `test/js/node/cluster.test.ts` - Better error handling and debugging **Benchmarks:** - `bench/string-postmessage.mjs` - Worker postMessage performance comparison - `bench/string-fastpath.mjs` - Fast path vs traditional serialization comparison ## Bug Fixes **BlockList Threading Issues:** - Fixed potential deadlocks when multiple threads access BlockList simultaneously - Moved mutex locks inside methods rather than holding across entire function calls - Added atomic size tracking for GC compatibility - Fixed comparison function pointer handling **IPC Race Conditions:** - Fixed race condition where `SendQueue._onAfterIPCClosed()` could be called on wrong thread - Deferred cleanup operations to next tick using task queue - Improved file descriptor ownership with proper `bun.take()` usage **Structured Clone Compatibility:** - Enhanced class serialization with proper transfer/storage mode checking - Fixed edge cases where non-transferable objects were incorrectly handled - Added better error reporting for unsupported clone operations ## Technical Notes - Thread safety ensured via `String.isolatedCopy()` for cross-VM transfers - Memory cost calculation updated to account for string references - Maintains full compatibility with existing structured clone semantics - Does not affect object serialization or transfer lists - Proper cleanup and error handling throughout IPC pipeline --------- Co-authored-by: Jarred Sumner <jarred@jarredsumner.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Meghan Denny <meghan@bun.sh> |
||
|
pfg
|
7c4c360431 |
Make getIfPropertyValueExistsImpl accept a slice (#21554)
Previously it accepted `property: anytype` but now it's `[]const u8` because that was the only allowed value, so it makes it easier to see what type it accepts in autocomplete. Also updates the doc comment, switches it to use ZIG_EXPORT, and updates the cppbind doc comment |
||
|
pfg
|
40bff9fea8 |
Support functions in cppbind (#21439)
Fixes #21434: Makes sure 'bun install' is executed before running |
||
|
pfg
|
60faa8696f |
Auto cpp->zig bindings (#20881)
Co-authored-by: pfgithub <6010774+pfgithub@users.noreply.github.com> Co-authored-by: Ben Grant <ben@bun.sh> Co-authored-by: Claude <noreply@anthropic.com> |