Commit Graph

40 Commits

Author SHA1 Message Date
Meghan Denny
5702b39ef1 runtime: implement CompressionStream/DecompressionStream (#24757)
Closes https://github.com/oven-sh/bun/issues/1723
Closes https://github.com/oven-sh/bun/pull/22214
Closes https://github.com/oven-sh/bun/pull/24241

also supports the `"brotli"` and `"zstd"` formats

<img width="1244" height="547" alt="image"
src="https://github.com/user-attachments/assets/aecf4489-29ad-411d-9f6b-3bee50ed1b27"
/>
2025-11-20 17:14:37 -08:00
Jarred Sumner
e9e9ca4ffd Enable minify.keepNames in JS builtins 2025-10-18 16:24:16 -07:00
taylor.fish
f14f3b03bb Add new bindings generator; port SSLConfig (#23169)
Add a new generator for JS → Zig bindings. The bulk of the conversion is
done in C++, after which the data is transformed into an FFI-safe
representation, passed to Zig, and then finally transformed into
idiomatic Zig types.

In its current form, the new bindings generator supports:

* Signed and unsigned integers
* Floats (plus a “finite” variant that disallows NaN and infinities)
* Strings
* ArrayBuffer (accepts ArrayBuffer, TypedArray, or DataView)
* Blob
* Optional types
* Nullable types (allows null, whereas Optional only allows undefined)
* Arrays
* User-defined string enumerations
* User-defined unions (fields can optionally be named to provide a
better experience in Zig)
* Null and undefined, for use in unions (can more efficiently represent
optional/nullable unions than wrapping a union in an optional)
* User-defined dictionaries (arbitrary key-value pairs; expects a JS
object and parses it into a struct)
* Default values for dictionary members
* Alternative names for dictionary members (e.g., to support both
`serverName` and `servername` without taking up twice the space)
* Descriptive error messages
* Automatic `fromJS` functions in Zig for dictionaries
* Automatic `deinit` functions for the generated Zig types

Although this bindings generator has many features not present in
`bindgen.ts`, it does not yet implement all of `bindgen.ts`'s
functionality, so for the time being, it has been named `bindgenv2`, and
its configuration is specified in `.bindv2.ts` files. Once all
`bindgen.ts`'s functionality has been incorporated, it will be renamed.

This PR ports `SSLConfig` to use the new bindings generator; see
`SSLConfig.bindv2.ts`.

(For internal tracking: fixes STAB-1319, STAB-1322, STAB-1323,
STAB-1324)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Alistair Smith <hi@alistair.sh>
2025-10-03 17:10:28 -07:00
Jarred Sumner
00490199f1 bun feedback (#22710)
### What does this PR do?

### How did you verify your code works?

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-26 03:47:26 -07:00
Meghan Denny
45760cd53c ci: instrument being able to run leaksanitizer (#21142)
tests not in `test/no-validate-leaksan.txt` will run with leaksanitizer
in CI
leaks documented in `test/leaksan.supp` will not cause a test failure

-- notes about leaksanitizer

- will not catch garbage collected objects accumulated during
long-running processes
- will not catch js objects (eg a strong held to a promise)
- will catch native calls to `malloc` not `free`d
- will catch allocations made in C, Zig, C++, libc, dependencies,
dlopen'd

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-09-19 02:06:02 -07:00
Ciro Spaciari
beea7180f3 refactor(MySQL) (#22619)
### What does this PR do?

### How did you verify your code works?

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-13 14:52:19 -07:00
robobun
e7672b2d04 Add string fast path for postMessage and structuredClone (#21926)
## Summary

Implements a string fast path optimization for `postMessage` and
`structuredClone` operations that provides significant performance
improvements for string-only data transfer, along with various bug fixes
and infrastructure improvements.

## Key Performance Improvements

**postMessage with Workers:**
- **Small strings (11 chars):** ~5% faster (572ns vs 599ns)
- **Medium strings (14KB):** **~2.7x faster** (528ns vs 1.40μs) 
- **Large strings (3MB):** **~660x faster** (540ns vs 356μs)

**Compared to Node.js postMessage:**
- Similar performance for small strings
- Competitive for medium strings  
- **~455x faster** for large strings (540ns vs 245μs)

## Implementation Details

The optimization adds a **string fast path** that bypasses full
structured cloning serialization when:
- Input is a pure string (`value.isString()`)
- No transfer list or message ports are involved
- Not being stored persistently

### Core Changes

**String Thread-Safety Utilities (`BunString.cpp/h`):**
- `isCrossThreadShareable()` - Checks if string can be safely shared
across threads
- `toCrossThreadShareable()` - Converts strings to thread-safe form via
`isolatedCopy()`
- Handles edge cases: atoms, symbols, substring slices, external buffers

**Serialization Fast Path (`SerializedScriptValue.cpp`):**
- New `m_fastPathString` field stores string data directly
- Bypasses full object serialization machinery for pure strings
- Creates isolated copies for cross-thread safety

**Deserialization Fast Path:**
- Directly returns JSString from stored string data
- Avoids parsing serialized byte streams

**Updated Flags System (`JSValue.zig`, `Serialization.cpp`):**
- Replaces boolean `forTransfer` with structured `SerializedFlags`
- Supports `forCrossProcessTransfer` and `forStorage` distinctions

**Structured Clone Infrastructure:**
- Moved `structuredClone` implementation to dedicated
`StructuredClone.cpp`
- Added `jsFunctionStructuredCloneAdvanced` for testing with custom
flags
- Improved class serialization compatibility checks (`isForTransfer`,
`isForStorage`)

**IPC Improvements (`ipc.zig`):**
- Fixed race conditions in `SendQueue` by deferring cleanup to next tick
- Proper fd ownership handling with `bun.take()`
- Cached IPC serialize/parse functions for better performance

**BlockList Thread Safety Fixes (`BlockList.zig`):**
- Fixed potential deadlocks by moving mutex locking inside methods
- Added atomic `estimated_size` counter to avoid lock during GC
- Corrected pointer handling in comparison functions
- Improved GC safety in `rules()` method

## Benchmark Results

```
❯ bun-21926 bench/string-postmessage.mjs  # This branch
postMessage(11 chars string)  572.24 ns/iter
postMessage(14 KB string)     527.55 ns/iter  ← ~2.7x faster
postMessage(3 MB string)      539.70 ns/iter  ← ~660x faster

❯ bun-1.2.20 bench/string-postmessage.mjs  # Previous
postMessage(11 chars string)  598.76 ns/iter
postMessage(14 KB string)       1.40 µs/iter
postMessage(3 MB string)      356.38 µs/iter

❯ node bench/string-postmessage.mjs       # Node.js comparison  
postMessage(11 chars string)  569.63 ns/iter
postMessage(14 KB string)       1.46 µs/iter
postMessage(3 MB string)      245.46 µs/iter
```

**Key insight:** The fast path achieves **constant time performance**
regardless of string size (~540ns), while traditional serialization
scales linearly with data size.

## Test Coverage

**New Tests:**
- `test/js/web/structured-clone-fastpath.test.ts` - Fast path memory
usage validation
- `test/js/web/workers/structuredClone-classes.test.ts` - Comprehensive
class serialization tests
  - Tests ArrayBuffer transferability 
  - Tests BunFile cloning with storage/transfer restrictions
  - Tests net.BlockList cloning behavior
  - Validates different serialization contexts (default, worker, window)

**Enhanced Tests:**
- `test/js/web/workers/structured-clone.test.ts` - Multi-function
testing
- Tests `structuredClone`, `jscSerializeRoundtrip`, and cross-process
serialization
  - Validates consistency across different serialization paths
- `test/js/node/cluster.test.ts` - Better error handling and debugging

**Benchmarks:**
- `bench/string-postmessage.mjs` - Worker postMessage performance
comparison
- `bench/string-fastpath.mjs` - Fast path vs traditional serialization
comparison

## Bug Fixes

**BlockList Threading Issues:**
- Fixed potential deadlocks when multiple threads access BlockList
simultaneously
- Moved mutex locks inside methods rather than holding across entire
function calls
- Added atomic size tracking for GC compatibility
- Fixed comparison function pointer handling

**IPC Race Conditions:**
- Fixed race condition where `SendQueue._onAfterIPCClosed()` could be
called on wrong thread
- Deferred cleanup operations to next tick using task queue
- Improved file descriptor ownership with proper `bun.take()` usage

**Structured Clone Compatibility:**
- Enhanced class serialization with proper transfer/storage mode
checking
- Fixed edge cases where non-transferable objects were incorrectly
handled
- Added better error reporting for unsupported clone operations

## Technical Notes

- Thread safety ensured via `String.isolatedCopy()` for cross-VM
transfers
- Memory cost calculation updated to account for string references
- Maintains full compatibility with existing structured clone semantics
- Does not affect object serialization or transfer lists
- Proper cleanup and error handling throughout IPC pipeline

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Meghan Denny <meghan@bun.sh>
2025-08-20 00:25:00 -07:00
Alistair Smith
784271f85e SQLite in Bun.sql (#21640)
### What does this PR do?

Support sqlite in the Bun.sql API

Fixes #18951
Fixes #19701

### How did you verify your code works?

tests

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-08-19 23:15:53 -07:00
Alistair Smith
04883a8bdc revert fe28e00d53.
This reverts commit fe28e00d53.
2025-08-05 16:10:29 -07:00
Alistair Smith
fe28e00d53 feat: add Bun.SQL API with initial SQLite support 2025-08-05 16:04:11 -07:00
Meghan Denny
809992229f node:net rework (#18962)
Co-authored-by: nektro <5464072+nektro@users.noreply.github.com>
2025-05-28 17:04:37 -07:00
Jarred Sumner
14b439a115 Fix formatters not running in CI + delete unnecessary files (#19433) 2025-05-08 23:22:16 -07:00
Jarred Sumner
032713c58c Fix several lints (#19121) 2025-04-19 05:41:34 -07:00
Meghan Denny
2bbdf4f950 codegen: fix this ModuleLoader enum (#18769) 2025-04-03 19:49:06 -07:00
chloe caruso
c29933f823 implement require.extensions attempt 2 (#18686) 2025-04-01 14:31:16 -07:00
Jarred Sumner
f38d35f7c9 Revert #18562 #18478 (#18610) 2025-03-28 20:23:49 -07:00
chloe caruso
70ddfb55e6 implement require.extensions (#18478) 2025-03-27 14:58:24 -07:00
Don Isaac
75144ab881 fix: reflect-metadata import order (#18086)
Co-authored-by: DonIsaac <22823424+DonIsaac@users.noreply.github.com>
2025-03-26 16:19:45 -07:00
chloe caruso
b5cbf16cb8 module pr 2 (#18266) 2025-03-20 00:45:44 -07:00
Meghan Denny
2024fa09d7 js: fix many typescript errors (#18272)
Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>
2025-03-19 22:39:24 -07:00
Jarred Sumner
46b2a58c25 Small improvements to internal types 2025-03-19 19:26:13 -07:00
chloe caruso
66cf62c3c4 dev server: rewrite HMRModule, support sync esm + hot.accept (#17954)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-03-07 15:12:16 -08:00
Meghan Denny
2d481e7bcb fix more node:stream (#16385)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-01-16 22:40:39 -08:00
Jarred Sumner
522f2b91a0 Introduce experimental support for on-demand bundling via HTML imports and Bun.serve() (#16395) 2025-01-15 01:00:23 -08:00
Minsoo Choo
9019aaf7d4 fix: update username (#16405) 2025-01-14 21:15:23 -08:00
Don Isaac
0372ca5c0a test(node): get test-assert.js working (#15698)
Co-authored-by: Don Isaac <don@bun.sh>
Co-authored-by: DonIsaac <DonIsaac@users.noreply.github.com>
2025-01-10 00:45:43 +00:00
Jarred Sumner
da54e81955 Support bundling HTML files and their js, css, and assets in Bun.build and bun build (#15940) 2024-12-23 11:04:38 -08:00
Don Isaac
2d5ea4993f fix(codegen): better error messages for internals using module.exports (#15687) 2024-12-10 15:10:21 -08:00
dave caruso
b39632c921 feat: new binding generator (#15638) 2024-12-10 12:43:17 -08:00
dave caruso
32ddf343ee bake: csr, streaming ssr, serve integration, safer jsvalue functions, &more (#14900)
Co-authored-by: paperdave <paperdave@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-11-13 18:19:12 -08:00
Jarred Sumner
8d7d58606b Add generator for $ERR_* as fake private globals (#13843)
Co-authored-by: Jarred-Sumner <Jarred-Sumner@users.noreply.github.com>
2024-09-10 16:07:22 -07:00
Jarred Sumner
cd6785771e run prettier and add back format action (#13722) 2024-09-03 21:32:52 -07:00
Jarred Sumner
b0018465cc WebKit upgrade (#12246)
Co-authored-by: Jarred-Sumner <Jarred-Sumner@users.noreply.github.com>
2024-07-01 23:27:59 -07:00
Jarred Sumner
f00772e98d GitHub actions 2024-05-16 01:35:28 -07:00
Jarred Sumner
4cbd215d55 Fix regression with TOML & JSONC (#10573) 2024-04-27 02:58:44 -07:00
dave caruso
d712254128 internal: remove secret hidden internals and introduce new way to call native code from js (#8166)
* oooooh magic

* stuff

* run format

* ok

* yippee

* run the formatter back

* finish things up

* fix webkit

* more

* [autofix.ci] apply automated fixes

* fix compile

* fix compilation on windows, it seems to not work though :(

* update

* a

* v

* ok

* [autofix.ci] apply automated fixes

* OOPS

* bump bun to reduce ci bugs

* a

* js2native is done!

* improve array binding

* rebase

* some final stuff

* wasi fixes

* os

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2024-03-29 21:47:11 -07:00
dave caruso
360bbb4dea fix(windows): fix directory cache regression "expected to end with a trailing slash" (#9144)
* okaaaaaaaay

* Revert "resolver: fix debug mode crash in test/bundler/bun-build-api.test.ts (#9140)"

This reverts commit 331d079dad.

* correctly fix the cache bust bug

this was introduced a couple of commits ago in my random fixes,
where i put the wrong fix to another directory caching bug.

i still stand by the assertion in place despite it causing many people
issues. it's precense will prevent subtle module resolutions failures.

* add an extra comment

* fix building a release build locally

* add a better test case for 3216

* staging

* fix mac issues

* ok

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-02-28 14:51:14 -08:00
dave caruso
01d41838c5 fix(builtin-bundler): do not replace globals when extending classes (#8557)
* fix(builtin-bundler): do not replace globals when extending classes

* format
2024-01-29 11:07:39 -08:00
Jarred Sumner
83f2432da6 Update prettier (#8198)
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2024-01-15 23:47:13 -08:00
Jarred Sumner
7485c7c7cb feat: Windows + CMake Build System (#4410)
* Prepare for windows event loop

* More progress

* Update libuv.zig

* wip

* Make compiling each dependency a shell script

* Bump mimalloc

* Add the build scripts

* Update settings.json

* Fix a bunch of compiler warnings

* Remove more warnings

* more warnings

* cmake works

* Update JSSQLStatement.h

* put it in the zig file

* Fix usockets warnings

* Fixup

* Fix one of the compiler errors

* chunk

* draw the rest of the owl

* theres more

* Rename Process -> BunProcess

Works around a Windows issue

* Add musl polyfill for memmem on Windows

* More

* 12 mb

* Fix getenvZ

* fix variosu issues

* Add fast-ish path for bun install on Windows

* Update windows.zig

* Update windows.zig

* Fix build issue

* it works

* hmmm

* Rename file

* Fixups

* Update wtf-bindings.cpp

* Update src/bun.js/bindings/headers-handwritten.h

Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>

* further!

* more

* Update .gitignore

* hm

* quite a lot of fixes

* Update CMakeLists.txt

* zig fmt

* Many more things are starting to work.

* reb

* regenaret

* Update JSSink.h

* fixup

* fetch works

* Bun.serve() and much of the event loop works now

* Make require() work

* bun install progress

* more things work

* use less std.os

* Fixes

* small fixes

* Bump

* Bummp

* Undo that change

* We have to bump the version of Debian because libarchive has a higher minimum requirement

* ok

* some clenaup

* windows

* Update bun.zig

* fixup

* avoid duplicate symbols

* avoid undefined symbols

* bump

* Remove issue template for install

It's not used, and use the bug issue instead.

* Add types for cp and cpSync

* Add types for watchFile and unwatchFile

* Add bun-types to 'bun fmt' script

* Update nodejs compat docs cp/cpSync/watchFile/unwatchFile (#4525)

* feat(fetch) rejectUnauthorized and checkServerIdentity (#4514)

* enable root certs on fetch

* rebase

* fix lookup

* some fixes and improvements

* fmt

* more fixes

* more fixes

* check detached onHandshake

* fix promise case

* fix cert non-Native

* add fetch tls tests

* more one test

* churn

* Update feature_flags.zig

* Update response.zig

* Revert "avoid undefined symbols"

This reverts commit ca835b726f.

* Revert "avoid duplicate symbols"

This reverts commit 4ac6ca8700.

* Update feature_flags.zig

* Set permissions

* more

* Update mimalloc

* Fix sqlite test failures

* Fix some test failures

* Make sure we remove libusockets is removed

* hm

* [dave]: fix webcrypto crash

* bump

* Update index.ts

* windows zig compiles

* cmake on mac works

* progress

* yay

* bun run build

* fix

* ok

* oops

* asdfasfdafdsafda

* fghjkl

* git ignore

* wow

* Process -> BunProcess

* hmm

* blah

* finalize merge

* now it only has linker errors on mac

* sdfadsf

* g

* getting farther

* sxdcvbnmk,

* adfhjskfjdhkas

* a

* fgh

* update build dot zig

* asdfg

* theoretical -DCANARY flag we can use

* asdf

* cool

* okay

* colorterm

* New build workflow

* Fix script

* Use sudo

* More sudo

* Tweak dependencies

* Another sudo attempt

* Tweak script

* 16.0 -> 16

* Tweak script

* Tweak script

* Tweak script

* Tweak script

* Tweak script

* bun install

* ssh into github actions

* add more to ssh

* Fix postinstal

* Skip llvm

* New dockerfile

* Build

* More changes to Dockerfile

* chaos chaos chaos

* okay

* a

* more cmake nonsense

* add unified sources code (does not work)

* stuff

* prepare for CI builds

* ok

* yay

* yeah

* make this more stable simply by trying again if it fails, 5 times, then lose. it fixes the stability issue i was running into L O L

* messing with ci

* x

* a

* clean dependencies before build

* oops

* this is not going to work but its closer

* not gonna work either

* a

* a

* did i do it

* a

* a

* work around weird fs+Bun.build issues

* properly pass debug flag correctly

* idk im sorry

* lose

* maybe

* run the tests please

* a

* fix zlib script

* a

* hi

* prevent stupid ci issue

* i totally didnt leave in a syntax error on cmakelists

* a

* lol

* relax

* 😭

* a

* SO SILLY

* 😡 one line mistake

* one character diff

* fix linking symbols missing

* work on dependency scripts

* does this work now?

* fix mac build

* a

* bump!

* woops

* add macos baseline build

* .

* fix sqlite and also enable $assert/$debug support in builtin functions

* okay

* oops

* zig upgrade lol

* Merge

* Fix spawn test issue

* Set a timeout

* yeah

* etc

* mi

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>
Co-authored-by: Ashcon Partovi <ashcon@partovi.net>
Co-authored-by: Birk Skyum <74932975+birkskyum@users.noreply.github.com>
Co-authored-by: dave caruso <me@paperdave.net>
2023-10-27 01:51:56 -07:00