Commit Graph

152 Commits

Author SHA1 Message Date
CountBleck
2956281845 Support WebAssembly.{instantiate,compile}Streaming() (#20503)
### What does this PR do?

<!-- **Please explain what your changes do** -->

This PR should fix #14219 and implement
`WebAssembly.instantiateStreaming()` and
`WebAssembly.compileStreaming()`.

This is a mixture of WebKit's implementation (using a helper,
`handleResponseOnStreamingAction`, also containing a fast-path for
blobs) and some of Node.js's validation (error messages) and its
builtin-based strategy to consume chunks from streams.

`src/bun.js/bindings/GlobalObject.zig` has a helper function
(`getBodyStreamOrBytesForWasmStreaming`), called by C++, to validate the
response (like
[Node.js](214e4db60e/lib/internal/wasm_web_api.js)
does) and to extract the data from the response, either as a slice/span
(if we can get the data synchronously), or as a `ReadableStream` body
(if the data is still pending or if it is a file/S3 `Blob`).

In C++, `handleResponseOnStreamingAction` is called by
`compileStreaming` and `instantiateStreaming` on the
`JSC::GlobalObjectMethodTable`, just like in
[WebKit](97ee3c598a/Source/WebCore/bindings/js/JSDOMGlobalObject.cpp (L517)).
It calls the aforementioned Zig helper for validation and getting the
response data. The data is then fed into `JSC::Wasm::StreamingCompiler`.

If the data is received as a `ReadableStream`, then we call a JS builtin
in `WasmStreaming.ts` to iterate over each chunk of the stream, like
[Node.js](214e4db60e/lib/internal/wasm_web_api.js (L50-L52))
does. The `JSC::Wasm::StreamingCompiler` is passed into JS through a new
wrapper object, `WebCore::WasmStreamingCompiler`, like
[Node.js](214e4db60e/src/node_wasm_web_api.h)
does. It has `addBytes`, `finalize`, `error`, and (unused) `cancel`
methods to mirror the underlying JSC class.

(If there's a simpler way to do this, please let me know...that would be
very much appreciated)

- [x] Code changes

### How did you verify your code works?

<!-- **For code changes, please include automated tests**. Feel free to
uncomment the line below -->

I wrote automated tests (`test/js/web/fetch/wasm-streaming.test`).

<!-- If JavaScript/TypeScript modules or builtins changed: -->

- [x] I included a test for the new code, or existing tests cover it
- [x] I ran my tests locally and they pass (`bun-debug test
test/js/web/fetch/wasm-streaming.test`)

<!-- If Zig files changed: -->

- [x] I checked the lifetime of memory allocated to verify it's (1)
freed and (2) only freed when it should be (NOTE: consumed `AnyBlob`
bodies are freed, and all other allocations are in C++ and either GCed
or ref-counted)
- [x] I included a test for the new code, or an existing test covers it
(NOTE: via JS/TS unit test)
- [x] JSValue used outside of the stack is either wrapped in a
JSC.Strong or is JSValueProtect'ed (NOTE: N/A, JSValue never used
outside the stack)
- [x] I wrote TypeScript/JavaScript tests and they pass locally
(`bun-debug test test/js/web/fetch/wasm-streaming.test`)

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-07-28 11:59:45 -07:00
Jarred Sumner
89aae0bdc0 Add flag to disable sql auto pipelining (#21067)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-07-15 01:13:35 -07:00
jarred-sumner-bot
5fe0c034e2 fix: respect user-provided Connection header in fetch() requests (#21049)
Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: jarred-sumner-bot <220441119+jarred-sumner-bot@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <Jarred-Sumner@users.noreply.github.com>
2025-07-14 20:53:46 -07:00
Jarred Sumner
2e02d9de28 Use ReadableStream.prototype.* in tests instead of new Response(...).* (#20937)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Alistair Smith <hi@alistair.sh>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-14 00:47:53 -07:00
Meghan Denny
9bc559e09f console: fix printing of Response(Bun.file()) (#20933) 2025-07-11 11:37:44 -07:00
Meghan Denny
5763a8e533 node:zlib: add zstd (#20313)
Co-authored-by: nektro <5464072+nektro@users.noreply.github.com>
2025-06-12 14:20:28 -07:00
Ciro Spaciari
ef9ea8ae1c fix(fetch) ignore trailers and add trailer tests (#19854) 2025-05-22 20:17:21 -07:00
Jarred Sumner
14b439a115 Fix formatters not running in CI + delete unnecessary files (#19433) 2025-05-08 23:22:16 -07:00
190n
754032c9ff skip flaky fetch test (#19437) 2025-05-02 20:49:15 -07:00
pfg
ea681fa9ec test-assert-typedarray-deepequal (#19285) 2025-04-25 23:36:07 -07:00
Ciro Spaciari
575d2c40a8 fix(server) Fix empty stream response (#18707) 2025-04-01 19:08:04 -07:00
Ciro Spaciari
39cf0906d1 fix(fetch) handle aborted connection inside start (#18512) 2025-03-26 20:52:49 -07:00
Kai Tamkun
60acfb17f0 node:http compatibility (options.lookup) (#18395)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-03-24 23:49:02 -07:00
Ciro Spaciari
1656bca9ab improve(fetch) (#18187) 2025-03-24 23:24:16 -07:00
Kai Tamkun
4a0e982bb2 node:http improvements (#17093)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: Pham Minh Triet <92496972+Nanome203@users.noreply.github.com>
Co-authored-by: snwy <snwy@snwy.me>
Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
Co-authored-by: cirospaciari <cirospaciari@users.noreply.github.com>
Co-authored-by: Ben Grant <ben@bun.sh>
2025-03-10 20:19:29 -07:00
Shlomo
506ea28b36 feat: load full certificate bundles from NODE_EXTRA_CA_CERTS (#16782) 2025-02-12 11:42:31 -08:00
Jarred Sumner
77be87b0a7 Fix process.stdin.ref (#16767) 2025-01-26 03:51:16 -08:00
Jarred Sumner
75a95aa5fa Rewrite the internal Web Stream native bindings to use less memory (#16349) 2025-01-26 00:04:39 -08:00
chloe caruso
834ad11d48 get node:fs tests passing part 1 (#16270) 2025-01-14 20:53:02 -08:00
Don Isaac
81ecf7556c ci: repair lint setup and run it in CI (#15720)
Co-authored-by: Don Isaac <don@bun.sh>
2025-01-08 07:12:18 +00:00
Meghan Denny
e1cfea4925 node: fix the rest of test-process (#16026) 2025-01-06 14:30:36 -08:00
Jarred Sumner
7b06872abb Deflake fetch tests (#16000) 2024-12-27 14:07:41 -08:00
Don Isaac
960b2b2c11 perf(node:util): fast path for extractedSplitNewLines (#15838)
Co-authored-by: Don Isaac <don@bun.sh>
Co-authored-by: DonIsaac <DonIsaac@users.noreply.github.com>
2024-12-19 23:42:18 +00:00
Ciro Spaciari
fcca2cc398 fix(fetch) fix redirect + Connection: close (#15623) 2024-12-06 15:06:11 -08:00
Ciro Spaciari
dd5c40dab7 fix(node:http) fix node:http chunked encoding on server and add chunked encoding support on the client (#15579)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-12-04 17:58:21 -08:00
pfg
78b495aff5 fix \uFFFF printing regression (#15330)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-11-21 22:01:27 -08:00
Jarred Sumner
5fc53353fb Allow disabling keep-alive (#14569)
Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
2024-10-14 16:58:42 -07:00
Meghan Denny
a5006a13a8 fetch-tcp-stress.test.ts: todo failing on macos ci (#14514) 2024-10-14 12:48:42 -07:00
Jarred Sumner
1458fcca4a Run formatter 2024-09-06 18:13:08 -07:00
Jarred Sumner
d38fc909e3 Support ReadableStream in request.clone & response.clone() (#13744) 2024-09-05 17:55:59 -07:00
Jarred Sumner
a9cf463eeb Introduce fast path for buffered ReadableStream (#13704)
Co-authored-by: Jarred-Sumner <Jarred-Sumner@users.noreply.github.com>
2024-09-04 04:16:47 -07:00
Jarred Sumner
cd6785771e run prettier and add back format action (#13722) 2024-09-03 21:32:52 -07:00
Jarred Sumner
12174e0577 Call cancel on ReadableStream when Bun.serve() response is aborted (#13687) 2024-09-02 09:40:03 -07:00
Jarred Sumner
5f6015bb79 Fixes #13657 (#13667) 2024-09-01 18:21:34 -07:00
Ciro Spaciari
adb54f1849 fix(Server) handle requestIP after async call (#13532)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-08-30 19:13:53 -07:00
Ciro Spaciari
bd3c258af4 fix(FormData, Bun.file()) FormData can append file slices, Bun.file(..).slice(..).text() works as expected (#13580)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-08-30 00:53:40 -07:00
Jarred Sumner
bd2eb40a39 Fix oom handling in Bun.file (#13603) 2024-08-29 18:54:33 -07:00
Jarred Sumner
5a525d3042 Deflake test/js/web/fetch/fetch-tcp-stress.test.ts (#13537) 2024-08-25 21:24:36 -07:00
Ciro Spaciari
df9d18659c revert d8ac4c59ff 2024-08-25 19:43:24 -07:00
Ciro Spaciari
d8ac4c59ff deflaky 2024-08-25 19:35:35 -07:00
Ciro Spaciari
5a108c5027 fix(fetch) always make sure that abort tracker is cleaned, revise ref count (#13459) 2024-08-23 17:08:57 -07:00
Jarred Sumner
766a9cf4f2 Fix some argument validation issues in fetch() (#13337) 2024-08-15 23:14:59 -07:00
Jarred Sumner
3a245dd248 upgrade webkit (#13192)
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Zack Radisic <zack@theradisic.com>
2024-08-12 23:17:17 -07:00
Jarred Sumner
7aaf935711 Handle OOM better (#13142) 2024-08-09 00:43:54 -07:00
Ciro Spaciari
077ee55211 fix fast shutdown (#13156) 2024-08-08 15:54:54 -07:00
Ciro Spaciari
c527058f14 implement NODE_EXTRA_CA_CERTS (#13150) 2024-08-07 13:21:52 -07:00
Jarred Sumner
c5c55c7ce4 Fixes #13049 (#13069) 2024-08-04 19:50:53 -07:00
Jarred Sumner
43326b0b2d Fixes #12894 (#13067) 2024-08-04 19:49:49 -07:00
Ciro Spaciari
c552cb40d1 fix(server/tls/streams) fix onReadFile, streams, avoid shutdown on fatal errors, ensure ssl loop data and server ref count refactor (#12979) 2024-08-03 01:41:18 -07:00
Ciro Spaciari
1c648063fa fix(tls/socket/fetch) shutdown fix + ref counted context (#12925)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-07-30 22:41:54 -07:00