Commit Graph

67 Commits

Author SHA1 Message Date
robobun
43c46b1f77 fix(FormData): throw error instead of assertion failure on very large input (#25006)
## Summary

- Fix crash in `FormData.from()` when called with very large ArrayBuffer
input
- Add length check in C++ `toString` function against both Bun's
synthetic limit and WebKit's `String::MaxLength`
- For UTF-8 tagged strings, use simdutf to calculate actual UTF-16
length only when byte length exceeds the limit

## Root Cause

When `FormData.from()` was called with a very large ArrayBuffer (e.g.,
`new Uint32Array(913148244)` = ~3.6GB), the code would crash with:

```
ASSERTION FAILED: data.size() <= MaxLength
vendor/WebKit/Source/WTF/wtf/text/StringImpl.h(886)
```

The `toString()` function in `helpers.h` was only checking against
`Bun__stringSyntheticAllocationLimit` (which defaults to ~4GB), but not
against WebKit's `String::MaxLength` (INT32_MAX, ~2GB). When the input
exceeded `String::MaxLength`, `createWithoutCopying()` would fail with
an assertion.

## Changes

1. **helpers.h**: Added `|| str.len > WTF::String::MaxLength` checks to
all three code paths in `toString()`:
- UTF-8 tagged pointer path (with simdutf length calculation only when
needed)
   - External pointer path
   - Non-copying creation path

2. **url.zig**: Reverted the incorrect Zig-side check (UTF-8 byte length
!= UTF-16 character length)

## Test plan

- [x] Added test that verifies FormData.from with oversized input
doesn't crash
- [x] Verified original crash case now returns empty FormData instead of
crashing:
  ```js
  const v3 = new Uint32Array(913148244);
  FormData.from(v3); // No longer crashes
  ```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-11-26 13:46:08 -08:00
pfg
05d0475c6c Update to zig 0.15.2 (#24204)
Fixes ENG-21287

Build times, from `bun run build && echo '//' >> src/main.zig && time
bun run build`

|Platform|0.14.1|0.15.2|Speedup|
|-|-|-|-|
|macos debug asan|126.90s|106.27s|1.19x|
|macos debug noasan|60.62s|50.85s|1.19x|
|linux debug asan|292.77s|241.45s|1.21x|
|linux debug noasan|146.58s|130.94s|1.12x|
|linux debug use_llvm=false|n/a|78.27s|1.87x|
|windows debug asan|177.13s|142.55s|1.24x|

Runtime performance:

- next build memory usage may have gone up by 5%. Otherwise seems the
same. Some code with writers may have gotten slower, especially one
instance of a counting writer and a few instances of unbuffered writers
that now have vtable overhead.
- File size reduced by 800kb (from 100.2mb to 99.4mb)

Improvements:

- `@export` hack is no longer needed for watch
- native x86_64 backend for linux builds faster. to use it, set use_llvm
false and no_link_obj false. also set `ASAN_OPTIONS=detect_leaks=0`
otherwise it will spam the output with tens of thousands of lines of
debug info errors. may need to use the zig lldb fork for debugging.
- zig test-obj, which we will be able to use for zig unit tests

Still an issue:

- false 'dependency loop' errors remain in watch mode
- watch mode crashes observed

Follow-up:

- [ ] search `comptime Writer: type` and `comptime W: type` and remove
- [ ] remove format_mode in our zig fork
- [ ] remove deprecated.zig autoFormatLabelFallback
- [ ] remove deprecated.zig autoFormatLabel
- [ ] remove deprecated.BufferedWriter and BufferedReader
- [ ] remove override_no_export_cpp_apis as it is no longer needed
- [ ] css Parser(W) -> Parser, and remove all the comptime writer: type
params
- [ ] remove deprecated writer fully

Files that add lines:

```
649     src/deprecated.zig
167     scripts/pack-codegen-for-zig-team.ts
54      scripts/cleartrace-impl.js
46      scripts/cleartrace.ts
43      src/windows.zig
18      src/fs.zig
17      src/bun.js/ConsoleObject.zig
16      src/output.zig
12      src/bun.js/test/debug.zig
12      src/bun.js/node/node_fs.zig
8       src/env_loader.zig
7       src/css/printer.zig
7       src/cli/init_command.zig
7       src/bun.js/node.zig
6       src/string/escapeRegExp.zig
6       src/install/PnpmMatcher.zig
5       src/bun.js/webcore/Blob.zig
4       src/crash_handler.zig
4       src/bun.zig
3       src/install/lockfile/bun.lock.zig
3       src/cli/update_interactive_command.zig
3       src/cli/pack_command.zig
3       build.zig
2       src/Progress.zig
2       src/install/lockfile/lockfile_json_stringify_for_debugging.zig
2       src/css/small_list.zig
2       src/bun.js/webcore/prompt.zig
1       test/internal/ban-words.test.ts
1       test/internal/ban-limits.json
1       src/watcher/WatcherTrace.zig
1       src/transpiler.zig
1       src/shell/builtin/cp.zig
1       src/js_printer.zig
1       src/io/PipeReader.zig
1       src/install/bin.zig
1       src/css/selectors/selector.zig
1       src/cli/run_command.zig
1       src/bun.js/RuntimeTranspilerStore.zig
1       src/bun.js/bindings/JSRef.zig
1       src/bake/DevServer.zig
```

Files that remove lines:

```
-1      src/test/recover.zig
-1      src/sql/postgres/SocketMonitor.zig
-1      src/sql/mysql/MySQLRequestQueue.zig
-1      src/sourcemap/CodeCoverage.zig
-1      src/css/values/color_js.zig
-1      src/compile_target.zig
-1      src/bundler/linker_context/convertStmtsForChunk.zig
-1      src/bundler/bundle_v2.zig
-1      src/bun.js/webcore/blob/read_file.zig
-1      src/ast/base.zig
-2      src/sql/postgres/protocol/ArrayList.zig
-2      src/shell/builtin/mkdir.zig
-2      src/install/PackageManager/patchPackage.zig
-2      src/install/PackageManager/PackageManagerDirectories.zig
-2      src/fmt.zig
-2      src/css/declaration.zig
-2      src/css/css_parser.zig
-2      src/collections/baby_list.zig
-2      src/bun.js/bindings/ZigStackFrame.zig
-2      src/ast/E.zig
-3      src/StandaloneModuleGraph.zig
-3      src/deps/picohttp.zig
-3      src/deps/libuv.zig
-3      src/btjs.zig
-4      src/threading/Futex.zig
-4      src/shell/builtin/touch.zig
-4      src/meta.zig
-4      src/install/lockfile.zig
-4      src/css/selectors/parser.zig
-5      src/shell/interpreter.zig
-5      src/css/error.zig
-5      src/bun.js/web_worker.zig
-5      src/bun.js.zig
-6      src/cli/test_command.zig
-6      src/bun.js/VirtualMachine.zig
-6      src/bun.js/uuid.zig
-6      src/bun.js/bindings/JSValue.zig
-9      src/bun.js/test/pretty_format.zig
-9      src/bun.js/api/BunObject.zig
-14     src/install/install_binding.zig
-14     src/fd.zig
-14     src/bun.js/node/path.zig
-14     scripts/pack-codegen-for-zig-team.sh
-17     src/bun.js/test/diff_format.zig
```

`git diff --numstat origin/main...HEAD | awk '{ print ($1-$2)"\t"$3 }' |
sort -rn`

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Meghan Denny <meghan@bun.com>
Co-authored-by: tayor.fish <contact@taylor.fish>
2025-11-10 14:38:26 -08:00
Meghan Denny
0eb470fd88 zig: handle termination exception from promise fulfullment/rejection (#23285) 2025-10-14 19:48:25 -07:00
taylor.fish
41b1efe12c Rename disabled parameter in Output.scoped (#21769)
It's very confusing.

(For internal tracking: fixes STAB-977)
2025-08-11 20:19:34 -07:00
taylor.fish
07cd45deae Refactor Zig imports and file structure (part 1) (#21270)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-07-22 17:51:38 -07:00
pfg
83760fc446 Sort imports in all files (#21119)
Co-authored-by: taylor.fish <contact@taylor.fish>
2025-07-21 13:26:47 -07:00
Zack Radisic
ac61b1d471 Use better function names for bun.String (#20999) 2025-07-12 18:19:16 -07:00
Jarred Sumner
3ea6133c46 CI: Remove unused top-level decls in formatter in zig (#19879)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-23 22:49:48 -07:00
Jarred Sumner
bfd12eeeba [bun install] Do not prefetch dns for npm registry if proxy (#19749) 2025-05-19 12:35:16 -07:00
Dylan Conway
a0819e9d02 fix more node:timers tests (#19432)
Co-authored-by: 190n <ben@bun.sh>
2025-05-02 20:50:02 -07:00
chloe caruso
3349c995b5 no usingnamespace, organize jsc namespace, enable -fincremental (#19122)
Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>
2025-04-22 16:34:15 -07:00
chloe caruso
4ec410e0d7 internal: make @import("bun") work in zig (#19096) 2025-04-17 12:32:47 -07:00
Ciro Spaciari
79afefa488 fix(s3) Support Virtual Hosted-Style endpoints (#17292)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-02-13 19:47:53 -08:00
Meghan Denny
26d3688e53 zig: update to 0.14.0-dev (#16862)
Co-authored-by: nektro <5464072+nektro@users.noreply.github.com>
2025-02-01 01:11:02 -08:00
Meghan Denny
676e8d1632 zig: delete is_bindgen (#16858) 2025-01-28 23:51:24 -08:00
Ciro Spaciari
fe4176e403 feat(s3) s3 client (#15740)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: cirospaciari <cirospaciari@users.noreply.github.com>
2024-12-28 17:46:22 -08:00
Jarred Sumner
2b2ca3275c Improve stack overflow, show more properties in Error objects (#15985)
Co-authored-by: Dave Caruso <me@paperdave.net>
2024-12-25 21:47:13 -08:00
Meghan Denny
c3f63bcdc4 zig: make throwInvalidArguments use JSError (#15305) 2024-11-21 16:19:13 -08:00
Meghan Denny
0e3e33072b zig: rename CallFrame.arguments to .arguments_old to free up decl name (#15296) 2024-11-20 16:18:56 -08:00
Meghan Denny
1efab7f42d zig: JSValue: make .get and .toSliceOrNull use JSError (#15270) 2024-11-20 13:26:41 -08:00
Meghan Denny
173f67d81e zig: make throwError use JSError (#15267) 2024-11-19 22:21:02 -08:00
Meghan Denny
fdd8d35845 allow zig js host functions to return JSError (#15120) 2024-11-13 21:11:56 -08:00
Jarred Sumner
1a6ead667b Introduce bun --fetch-preconnect <url> ./my-script.ts (#12698) 2024-07-22 03:41:59 -07:00
Jarred Sumner
b0018465cc WebKit upgrade (#12246)
Co-authored-by: Jarred-Sumner <Jarred-Sumner@users.noreply.github.com>
2024-07-01 23:27:59 -07:00
Jarred Sumner
6a756bf979 Implement blob: URLs in fetch, import, and Worker (#11537)
Co-authored-by: Jarred-Sumner <Jarred-Sumner@users.noreply.github.com>
Co-authored-by: dave caruso <me@paperdave.net>
2024-06-05 18:49:03 -07:00
Jarred Sumner
1191bf0c15 Delete dead test code (#10405)
* Delete dead test code

* Apply formatting changes

---------

Co-authored-by: Jarred-Sumner <Jarred-Sumner@users.noreply.github.com>
2024-04-20 21:25:10 -07:00
Jarred Sumner
688844b472 refactor: ban std.debug.assert (#10168)
* Ban `std.debug.assert`

* Create .clangd

* Update lint.yml

* Update linter.ts

* update

* lint

* Update linter.ts

* Update linter.ts

* update

* Update linter.ts

* update

* Update linter.ts

* more

* Update install.zig

* words

* Remove UB
2024-04-11 17:52:29 -07:00
Meghan Denny
5712ba8b3b debug: introduce bun.callmod_inline (#9001) 2024-02-19 14:43:42 -08:00
Meghan Denny
be0896e204 remove while loops where a for would be more efficient (#8131)
* remove while loops where a for would be more efficient

* this needs to be a stack copy

* this can use the better loop

* this was translated wrong

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2024-01-13 19:13:54 -08:00
Dylan Conway
98d253d9bb fix(install): incremental support for windows (#7991)
* update for windows

* use correct sizes

* mode

* fchmod and correct error checking

* some progress

* walker_skippable.zig for windows

* no segfault

* comptime only

* normalize

* create files in .bin

* bun.sys.write

* string formatters in bun.fmt, diriterator enum

* isAllASCII comptime

* update more

* some cleanup

* fix 1

* fix 2

* simlink

* cast

* remove normalize

* to zig err

* update fchmod

* fixup

* fix running bin files

* --bun for running binaries

* --bun for scripts

* remove binary linking

* update

* remove todo

* todos

* fix

* simlink

* cast

* more cast
2024-01-11 22:00:47 -08:00
dave caruso
369e3022e4 chore: upgrade zig to 0.12.0-dev.1828+225fe6ddb (#7671)
* chore: upgrade zig to 0.12.0-dev.1828+225fe6ddb

* open as iterable

* fix building identifier cache

* fix windows build

* fix linux build

* fix linux build
2023-12-16 00:14:15 -08:00
Jarred Sumner
70e57a7da0 Fixes #4876 (#7391) 2023-12-01 22:54:40 +01:00
Jarred Sumner
5692f82aaf Delete bun_dev_http_server.zig (#7283)
* Rename http_client_async -> http

* Delete bun_dev_http_server.zig

* update these

* [uws] Add method for getting SO_ERROR

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-11-24 18:21:56 -08:00
Jarred Sumner
7a4e0158d6 Lots of stuff (#7027)
* Use debug mode by default

* Enable build with assertions enabled

* Update cli.zig

* Update bun-linux-build.yml

* Fixes

* Fix `ASSERT_ENABLED`

* try this

* Update Dockerfile

* mimalloc debug

* Update CMakeLists.txt

* `Bun.deepMatch` - fix assertion failures

cc @dylan-conway, looks like we need to use `putDirectMayBeIndex` and check for `isCell` more carefully.

* Object.create support in code generator and callbacks wrapper

* Remove unused file

* zig upgrade

* zls

* Fix various errors

* Support `BuiltinAccessor` in create_hash_table script

* Fix assertion failure in `process.mainModule`

* Fix assertion failure in `onerror`

* Fix assertion failure when creating a Worker

* Fix asssertion failure when loading lots of files in bun test

* Fix assertion failure when termating a `Worker`

* Add helper for converting BunString to a WTFString

* Fix assertion failure in notifyNeedTermination

* Add more debug logs in `bun test`

* Fix compiler warning in usockets

* Fix assertion failure with `Worker` termination (another)

* Fix assertion failure in `coerceToInt64`

* Fix assertion failure in `BroadcastChannel`

* Fix assertion failure in `Headers.prototype.getAll`

* Fixes #7067

* Add heap analyzer label for CommonJS modules

* Fix assertion failure in module.require && module.require.resolve

* Remove unused code

* Fix assertion failure in debugger

* Fix crash in debugger

* Fix assertion failures in bun:sqlite

* Bump zig

* Bump WebKit

* Fix assertion failure in JSPromise::reject && JSInternalPromise::reject

* Fix assertion failure in ReadableStream::cancel

* Fix assertion failure in AsyncContextFrame::create

* Fix assertion failure in bun:sqlite

* Fix assertion failure in mocks

* Fix assertion failure in ServerWebSocket.close

* Fix assertion failure in N-API with subclasses

* [napi] Make promises cheaper

* undo

* Don't check for exceptions in ObjectInitializationScope

* Add separate entry point for test runner that doesn't generate code

* Don't deref builtin code

* Fix preload test

* Fix assertion failure in memoryUsage()

* Fix preload test, part 2

* Ensure that the env map for a Worker is empty after it is used

* The pointer for the Arena allocator used in parsing should not change

* Terminate thread on exit

* Start to implement scriptExecutionStatus

* Update worker.test.ts

* Fix Dirent.name setter

* Update settings.json

* Fix assertion failure in node:http

* Use correct value for `JSFinalObject::maxInlineCapacity`

* JSFinalObject::maxInlineCapacity x2

* Don't strip when assertions are enabled

* Make `m_wasTerminated` atomic

* Preserve directives in the transpiler

cc @ctjlewis

* Workaround assertion failure in ServerWebSocket.sendBinary and ServerWebSocket.sendText

* windows

* Buffer lockfile serialization in-memory

* PR feedback

* PR feedback

* PR feedback

* Windows

* quotes

* Update CMakeLists.txt

* Update bun-linux-build.yml

* Update bun-linux-build.yml

* Move this code to BunString.cpp

* Update BunString.cpp

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-11-13 22:10:09 -08:00
Ashcon Partovi
f0cbc79d8b Implement server.url (#7043)
* Implement `server.url`

* Fix memory leak
2023-11-10 23:00:27 -08:00
Ashcon Partovi
d5d9fc4684 Fix websocket upgrade (#6564)
* Remove ancient changelog

* Fix `Host` header excluding port in WebSocket upgrade

* `byteSlice()`

* Revert `byteSlice()`
2023-10-19 22:24:45 -07:00
Ashcon Partovi
e31ed84b1b Fix request.url having incorrect port
Fixes #6443
2023-10-16 15:14:15 -07:00
saurabh
9a90d90966 fix: form data content type (#6380)
* fix: form data content type

* fix: condition if no extension for file
2023-10-10 20:08:32 -07:00
Jarred Sumner
7a1ebec26f Support file: URLs in fetch (#3858)
* Support file: URLs in `fetch`

* Update url.zig

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-07-28 15:44:05 -07:00
Jarred Sumner
636cec03e1 Use WebKit's URL parser in fetch() and bun install (#3730)
* Use WebKit's URL parser in fetch() and `bun install`

* Allocate less memory

* Fix test

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-07-21 23:27:28 -07:00
Jarred Sumner
dd46c11273 Support streams in response.formData() & request.formData, introduce Bun.readableStreamToFormData() (#3697)
* codegen

* FormData.from

* Fixes #3225

* Introduce `Bun.readableStreamToFormData`

* Update bun.d.ts

* Add examples

* add

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-07-19 23:12:06 -07:00
Dylan Conway
661355546a zig upgrade (#3667)
* upgrade

* more fixes

* Bump Zig

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-07-18 01:20:20 -07:00
Ciro Spaciari
95ddfcc437 fix query without slash (#3547) 2023-07-06 12:01:16 -07:00
Dylan Conway
5fa13625a1 upgrade zig to v0.11.0-dev.3737+9eb008717 (#3374)
* progress

* finish `@memset/@memcpy` update

* Update build.zig

* change `@enumToInt` to `@intFromEnum` and friends

* update zig versions

* it was 1

* add link to issue

* add `compileError` reminder

* fix merge

* format

* upgrade to llvm 16

* Revert "upgrade to llvm 16"

This reverts commit cc930ceb1c.

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-06-21 23:38:18 -07:00
Jarred Sumner
557aac6a34 Support FormData & file uploads in fetch body (#3123)
* Fixes #2264

* fixup

* Don't leak HTTP headers

* Include the mime type. It's cleaner

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-05-31 00:43:59 -07:00
Ciro Spaciari
c6c21eeba7 added some improvements on server (#2803)
* added some improvements on server

* undo unintended change

* clean data handler before end calls

* refactor

* make ctx.resp nullable

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2023-05-08 14:10:40 -07:00
Jarred Sumner
ea47ed0ee5 Fix broken autocomplete 2023-04-19 00:01:27 -07:00
Jarred Sumner
7bd6a1f86d Remove usages of void{} in favor of {}
See https://github.com/ziglang/zig/issues/15213
2023-04-09 05:39:05 -07:00
Ciro Spaciari
ac970146b4 allows Bun.serve to listen on IPV6 (#2400) 2023-03-15 14:30:31 -07:00
Alex Lam S.L
e887a064fb prefer bun.copy() over std.mem.copy() (#2152) 2023-02-24 05:53:26 -08:00