Commit Graph

31 Commits

Author SHA1 Message Date
pfg
05d0475c6c Update to zig 0.15.2 (#24204)
Fixes ENG-21287

Build times, from `bun run build && echo '//' >> src/main.zig && time
bun run build`

|Platform|0.14.1|0.15.2|Speedup|
|-|-|-|-|
|macos debug asan|126.90s|106.27s|1.19x|
|macos debug noasan|60.62s|50.85s|1.19x|
|linux debug asan|292.77s|241.45s|1.21x|
|linux debug noasan|146.58s|130.94s|1.12x|
|linux debug use_llvm=false|n/a|78.27s|1.87x|
|windows debug asan|177.13s|142.55s|1.24x|

Runtime performance:

- next build memory usage may have gone up by 5%. Otherwise seems the
same. Some code with writers may have gotten slower, especially one
instance of a counting writer and a few instances of unbuffered writers
that now have vtable overhead.
- File size reduced by 800kb (from 100.2mb to 99.4mb)

Improvements:

- `@export` hack is no longer needed for watch
- native x86_64 backend for linux builds faster. to use it, set use_llvm
false and no_link_obj false. also set `ASAN_OPTIONS=detect_leaks=0`
otherwise it will spam the output with tens of thousands of lines of
debug info errors. may need to use the zig lldb fork for debugging.
- zig test-obj, which we will be able to use for zig unit tests

Still an issue:

- false 'dependency loop' errors remain in watch mode
- watch mode crashes observed

Follow-up:

- [ ] search `comptime Writer: type` and `comptime W: type` and remove
- [ ] remove format_mode in our zig fork
- [ ] remove deprecated.zig autoFormatLabelFallback
- [ ] remove deprecated.zig autoFormatLabel
- [ ] remove deprecated.BufferedWriter and BufferedReader
- [ ] remove override_no_export_cpp_apis as it is no longer needed
- [ ] css Parser(W) -> Parser, and remove all the comptime writer: type
params
- [ ] remove deprecated writer fully

Files that add lines:

```
649     src/deprecated.zig
167     scripts/pack-codegen-for-zig-team.ts
54      scripts/cleartrace-impl.js
46      scripts/cleartrace.ts
43      src/windows.zig
18      src/fs.zig
17      src/bun.js/ConsoleObject.zig
16      src/output.zig
12      src/bun.js/test/debug.zig
12      src/bun.js/node/node_fs.zig
8       src/env_loader.zig
7       src/css/printer.zig
7       src/cli/init_command.zig
7       src/bun.js/node.zig
6       src/string/escapeRegExp.zig
6       src/install/PnpmMatcher.zig
5       src/bun.js/webcore/Blob.zig
4       src/crash_handler.zig
4       src/bun.zig
3       src/install/lockfile/bun.lock.zig
3       src/cli/update_interactive_command.zig
3       src/cli/pack_command.zig
3       build.zig
2       src/Progress.zig
2       src/install/lockfile/lockfile_json_stringify_for_debugging.zig
2       src/css/small_list.zig
2       src/bun.js/webcore/prompt.zig
1       test/internal/ban-words.test.ts
1       test/internal/ban-limits.json
1       src/watcher/WatcherTrace.zig
1       src/transpiler.zig
1       src/shell/builtin/cp.zig
1       src/js_printer.zig
1       src/io/PipeReader.zig
1       src/install/bin.zig
1       src/css/selectors/selector.zig
1       src/cli/run_command.zig
1       src/bun.js/RuntimeTranspilerStore.zig
1       src/bun.js/bindings/JSRef.zig
1       src/bake/DevServer.zig
```

Files that remove lines:

```
-1      src/test/recover.zig
-1      src/sql/postgres/SocketMonitor.zig
-1      src/sql/mysql/MySQLRequestQueue.zig
-1      src/sourcemap/CodeCoverage.zig
-1      src/css/values/color_js.zig
-1      src/compile_target.zig
-1      src/bundler/linker_context/convertStmtsForChunk.zig
-1      src/bundler/bundle_v2.zig
-1      src/bun.js/webcore/blob/read_file.zig
-1      src/ast/base.zig
-2      src/sql/postgres/protocol/ArrayList.zig
-2      src/shell/builtin/mkdir.zig
-2      src/install/PackageManager/patchPackage.zig
-2      src/install/PackageManager/PackageManagerDirectories.zig
-2      src/fmt.zig
-2      src/css/declaration.zig
-2      src/css/css_parser.zig
-2      src/collections/baby_list.zig
-2      src/bun.js/bindings/ZigStackFrame.zig
-2      src/ast/E.zig
-3      src/StandaloneModuleGraph.zig
-3      src/deps/picohttp.zig
-3      src/deps/libuv.zig
-3      src/btjs.zig
-4      src/threading/Futex.zig
-4      src/shell/builtin/touch.zig
-4      src/meta.zig
-4      src/install/lockfile.zig
-4      src/css/selectors/parser.zig
-5      src/shell/interpreter.zig
-5      src/css/error.zig
-5      src/bun.js/web_worker.zig
-5      src/bun.js.zig
-6      src/cli/test_command.zig
-6      src/bun.js/VirtualMachine.zig
-6      src/bun.js/uuid.zig
-6      src/bun.js/bindings/JSValue.zig
-9      src/bun.js/test/pretty_format.zig
-9      src/bun.js/api/BunObject.zig
-14     src/install/install_binding.zig
-14     src/fd.zig
-14     src/bun.js/node/path.zig
-14     scripts/pack-codegen-for-zig-team.sh
-17     src/bun.js/test/diff_format.zig
```

`git diff --numstat origin/main...HEAD | awk '{ print ($1-$2)"\t"$3 }' |
sort -rn`

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Meghan Denny <meghan@bun.com>
Co-authored-by: tayor.fish <contact@taylor.fish>
2025-11-10 14:38:26 -08:00
robobun
72490281e5 fix: handle empty chunked gzip responses correctly (#22360)
## Summary
Fixes #18413 - Empty chunked gzip responses were causing `Decompression
error: ShortRead`

## The Issue
When a server sends an empty response with `Content-Encoding: gzip` and
`Transfer-Encoding: chunked`, Bun was throwing a `ShortRead` error. This
occurred because the code was checking if `avail_in == 0` (no input
data) and immediately returning an error, without attempting to
decompress what could be a valid empty gzip stream.

## The Fix
Instead of checking `avail_in == 0` before calling `inflate()`, we now:
1. Always call `inflate()` even when `avail_in == 0` 
2. Check the return code from `inflate()`
3. If it returns `BufError` with `avail_in == 0`, then we truly need
more data and return `ShortRead`
4. If it returns `StreamEnd`, it was a valid empty gzip stream and we
finish successfully

This approach correctly distinguishes between "no data yet" and "valid
empty gzip stream".

## Why This Works
- A valid empty gzip stream still has headers and trailers (~20 bytes)
- The zlib `inflate()` function can handle empty streams correctly  
- `BufError` with `avail_in == 0` specifically means "need more input
data"

## Test Plan
 Added regression test in `test/regression/issue/18413.test.ts`
covering:
- Empty chunked gzip response
- Empty non-chunked gzip response  
- Empty chunked response without gzip

 Verified all existing gzip-related tests still pass
 Tested with the original failing case from the issue

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-09-03 18:57:39 -07:00
taylor.fish
07cd45deae Refactor Zig imports and file structure (part 1) (#21270)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-07-22 17:51:38 -07:00
pfg
83760fc446 Sort imports in all files (#21119)
Co-authored-by: taylor.fish <contact@taylor.fish>
2025-07-21 13:26:47 -07:00
Meghan Denny
5763a8e533 node:zlib: add zstd (#20313)
Co-authored-by: nektro <5464072+nektro@users.noreply.github.com>
2025-06-12 14:20:28 -07:00
Jarred Sumner
3ea6133c46 CI: Remove unused top-level decls in formatter in zig (#19879)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-23 22:49:48 -07:00
chloe caruso
4ec410e0d7 internal: make @import("bun") work in zig (#19096) 2025-04-17 12:32:47 -07:00
Meghan Denny
80db770521 rework node:zlib to match internal js api and properly support async writes (#14079)
Co-authored-by: nektro <nektro@users.noreply.github.com>
2024-09-25 15:55:53 -07:00
Meghan Denny
8cd515f533 node:zlib: move deflate and gzip into native code (#11770)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-09-06 18:49:19 -07:00
Jarred Sumner
a2ae28d158 Add named allocator 2024-07-28 21:27:08 -07:00
dave caruso
5137213f86 heavy revision on heap_breakdown's safety (#12445) 2024-07-09 14:29:00 -07:00
Jarred Sumner
1191bf0c15 Delete dead test code (#10405)
* Delete dead test code

* Apply formatting changes

---------

Co-authored-by: Jarred-Sumner <Jarred-Sumner@users.noreply.github.com>
2024-04-20 21:25:10 -07:00
dave caruso
c99d7ed331 feat: overhaul the crash handler (#10203)
* some work

* linux things

* linux things

* feat: tracestrings on Windows

* bwaa

* more work on the crash handler

* okay

* adgadsgbcxcv

* ya

* dsafds

* a

* wuh

* a

* bru h

* ok

* yay

* window

* alright

* oops

* yeah

* a

* a

* OOM handling

* fix on window
2024-04-17 15:32:25 -07:00
Jarred Sumner
492b2d5b76 Implement Brotli support in fetch() (#7839)
* Implement Brotli support in fetch()

* Use @panic

* Update src/http.zig

Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>

* Update src/http.zig

Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>

* Fix redirect logic

* Allow extremely long redirect URLs

* Update fetch.test.ts

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
2023-12-31 06:19:08 -08:00
Jarred Sumner
7485c7c7cb feat: Windows + CMake Build System (#4410)
* Prepare for windows event loop

* More progress

* Update libuv.zig

* wip

* Make compiling each dependency a shell script

* Bump mimalloc

* Add the build scripts

* Update settings.json

* Fix a bunch of compiler warnings

* Remove more warnings

* more warnings

* cmake works

* Update JSSQLStatement.h

* put it in the zig file

* Fix usockets warnings

* Fixup

* Fix one of the compiler errors

* chunk

* draw the rest of the owl

* theres more

* Rename Process -> BunProcess

Works around a Windows issue

* Add musl polyfill for memmem on Windows

* More

* 12 mb

* Fix getenvZ

* fix variosu issues

* Add fast-ish path for bun install on Windows

* Update windows.zig

* Update windows.zig

* Fix build issue

* it works

* hmmm

* Rename file

* Fixups

* Update wtf-bindings.cpp

* Update src/bun.js/bindings/headers-handwritten.h

Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>

* further!

* more

* Update .gitignore

* hm

* quite a lot of fixes

* Update CMakeLists.txt

* zig fmt

* Many more things are starting to work.

* reb

* regenaret

* Update JSSink.h

* fixup

* fetch works

* Bun.serve() and much of the event loop works now

* Make require() work

* bun install progress

* more things work

* use less std.os

* Fixes

* small fixes

* Bump

* Bummp

* Undo that change

* We have to bump the version of Debian because libarchive has a higher minimum requirement

* ok

* some clenaup

* windows

* Update bun.zig

* fixup

* avoid duplicate symbols

* avoid undefined symbols

* bump

* Remove issue template for install

It's not used, and use the bug issue instead.

* Add types for cp and cpSync

* Add types for watchFile and unwatchFile

* Add bun-types to 'bun fmt' script

* Update nodejs compat docs cp/cpSync/watchFile/unwatchFile (#4525)

* feat(fetch) rejectUnauthorized and checkServerIdentity (#4514)

* enable root certs on fetch

* rebase

* fix lookup

* some fixes and improvements

* fmt

* more fixes

* more fixes

* check detached onHandshake

* fix promise case

* fix cert non-Native

* add fetch tls tests

* more one test

* churn

* Update feature_flags.zig

* Update response.zig

* Revert "avoid undefined symbols"

This reverts commit ca835b726f.

* Revert "avoid duplicate symbols"

This reverts commit 4ac6ca8700.

* Update feature_flags.zig

* Set permissions

* more

* Update mimalloc

* Fix sqlite test failures

* Fix some test failures

* Make sure we remove libusockets is removed

* hm

* [dave]: fix webcrypto crash

* bump

* Update index.ts

* windows zig compiles

* cmake on mac works

* progress

* yay

* bun run build

* fix

* ok

* oops

* asdfasfdafdsafda

* fghjkl

* git ignore

* wow

* Process -> BunProcess

* hmm

* blah

* finalize merge

* now it only has linker errors on mac

* sdfadsf

* g

* getting farther

* sxdcvbnmk,

* adfhjskfjdhkas

* a

* fgh

* update build dot zig

* asdfg

* theoretical -DCANARY flag we can use

* asdf

* cool

* okay

* colorterm

* New build workflow

* Fix script

* Use sudo

* More sudo

* Tweak dependencies

* Another sudo attempt

* Tweak script

* 16.0 -> 16

* Tweak script

* Tweak script

* Tweak script

* Tweak script

* Tweak script

* bun install

* ssh into github actions

* add more to ssh

* Fix postinstal

* Skip llvm

* New dockerfile

* Build

* More changes to Dockerfile

* chaos chaos chaos

* okay

* a

* more cmake nonsense

* add unified sources code (does not work)

* stuff

* prepare for CI builds

* ok

* yay

* yeah

* make this more stable simply by trying again if it fails, 5 times, then lose. it fixes the stability issue i was running into L O L

* messing with ci

* x

* a

* clean dependencies before build

* oops

* this is not going to work but its closer

* not gonna work either

* a

* a

* did i do it

* a

* a

* work around weird fs+Bun.build issues

* properly pass debug flag correctly

* idk im sorry

* lose

* maybe

* run the tests please

* a

* fix zlib script

* a

* hi

* prevent stupid ci issue

* i totally didnt leave in a syntax error on cmakelists

* a

* lol

* relax

* 😭

* a

* SO SILLY

* 😡 one line mistake

* one character diff

* fix linking symbols missing

* work on dependency scripts

* does this work now?

* fix mac build

* a

* bump!

* woops

* add macos baseline build

* .

* fix sqlite and also enable $assert/$debug support in builtin functions

* okay

* oops

* zig upgrade lol

* Merge

* Fix spawn test issue

* Set a timeout

* yeah

* etc

* mi

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>
Co-authored-by: Ashcon Partovi <ashcon@partovi.net>
Co-authored-by: Birk Skyum <74932975+birkskyum@users.noreply.github.com>
Co-authored-by: dave caruso <me@paperdave.net>
2023-10-27 01:51:56 -07:00
Jarred Sumner
25cf4016d3 Fix memory leak in fetch() (#6350)
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-10-06 20:14:44 -07:00
Ciro Spaciari
f1afd58338 fix zlib deflate on fetch (#4483)
* fix zlib deflate on fetch

* mention issue on test

* more tests

* oops
2023-09-04 17:37:47 -07:00
Ciro Spaciari
6e4a1f2918 make pending_response and metdata life cycle more clear and make fetch more reliable (#4331)
* make pending_response and metdata life cycle more clear

* typo

* WIP: memory investigation

* check zlib and fix zlib

* use state allocator for metadata

* remove postBodyProcess

* undo some test things

* fix race condition

* fix removing compressed header

* some extra checks

* remove arenas on zlib and comment repoter.assert because of toOwnedSliceZ

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2023-08-28 07:21:46 -07:00
Ciro Spaciari
9027484ae1 fetch(stream) add stream support for compressed and uncompressed data (#4127)
* streams non compressed data in 64kb chunks (at least)

* fmt

* wip remove pause

* fix default streaming and buffering

* fix atomic lags

* fix size

* make chunked encoding work again (WIP streaming chunked)

* WIP: chunked encoding streaming

* fix end of streamings

* working streaming + compression

* add fixes + tests

* fmt + fix proxy

* fix oopsies

* codegen after merge

* fmt + fixes

* more fixes

* more fixes and logs

* avoid double free

* check empty before pop

* check empty on pop

* fix copy to real when complete

* remove unnecessary logs

* better has_schedule_callback swap, body locked size helper, remove isEmpty from unbounded_queue pop

* fix response ref, fix body_size

* add deflate support, fix error throw, add more tests

* codegen after merge

* remove logs, add connection close test

* fix macOS build

* fix redirect error option

* make body_size more clear

* support new Reponse(response)

* toString DOMWrapper objects properly instead of supporting response in Response constructor

* ignore headers with no name, add more tests

* oops

* handle transform with fetch

* add gz image stream test

* remove duplicate test

* fix missing chunk on macOS under pressure

* oops include all OS

* some fixes

* compare buffers instead of sizes

* refactor err.err and protect it
2023-08-21 20:30:34 -07:00
Dylan Conway
661355546a zig upgrade (#3667)
* upgrade

* more fixes

* Bump Zig

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-07-18 01:20:20 -07:00
Jarred Sumner
bdb1b7124a Fix crash in CJS (#3294)
* Fix crash in CJS

* Add std.heap.ArenaAllocator

* Use our arena allocator

* Reduce JS parser memory usage and make HMR faster

* Write some comments

* fix test failure & clean up this code

* Update javascript.zig

* make arena usage safer

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-06-13 09:15:05 -07:00
Jarred Sumner
ea47ed0ee5 Fix broken autocomplete 2023-04-19 00:01:27 -07:00
Dylan Conway
3f04f8d0a6 Upgrade Zig (#2151)
* fixup

* Upgrade Zig

* Remove bad assertion

* strings

* bump

* mode -> optimize

* optimize

* Linux build

* Update bindgen.zig
2023-02-23 23:57:19 -08:00
Jarred Sumner
c0dd284136 Upgrade to latest Zig (#1610)
* @min and @max

* builtins and some trivial ones

* Most of them

* more

* more!

* More Progress

* wip

* Update tagged_pointer.zig

* Update http_client_async.zig

* Most of the iterable dir changes

* alright

* Remove usages of deprecated formatters

* 📷

* fmt

* Update shimmer.zig

* wip

* wip

* wip

* progress

* more

* Latest

* stuck on error

* latest

* workaround stage2

* wip

* Update string_immutable.zig

* wip

* Migrate `Dirent` and `require("fs')` to use JSC<>Zig bindings

* Fix build errors

* Fixup most of the test failures

* Fix `make headers`

* Fix "outside package path" error

* Fixup aligned alloc

* Add missing file

* linux

* More linux fixes

* use latest peechy

* Fix transpiler test failure

* Forgot about these

* Fixup test failure

* Update node-timers.test.ts

* [node:htt] Fix `undefined is not an object` error

Fixes https://github.com/oven-sh/bun/issues/1618

* Update http.exports.js

* Make this test less flaky

* fix hashes

* Fix hex formatting and zls issues

* Download zig version

* Update Dockerfile

* Update Dockerfile

* Update uws

* Update Dockerfile

* Set llvm version

* Update README.md

* Update uws

* Update Dockerfile

* Update io_linux.zig

* Update bun.zig

* Log output

* workaround strange @cInclude error

* Make ffi tests better

* Don't use cImport

* Update c.zig

* Update c-bindings.cpp

* call setOutputDir

* Update Dockerfile

* Use a longer name

* latest

* Update serve.test.ts

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Veikka Tuominen <git@vexu.eu>
2022-12-28 00:51:22 -08:00
Jarred Sumner
b230e7a73a [fetch] Fix sporadic data corruption bug in HTTP client and add fast path
- This removes memory pooling from the HTTP client which sometimes caused invalid memory to be written to the response body.
- This adds a fast path for small HTTP/HTTPS responses that makes it a single memory allocation for the response body, instead of copying & allocating a temporary buffer

cc @Electroid
2022-11-19 22:34:57 -08:00
Jarred Sumner
93b663e43d New Zig <> C++ bindings generator. +20% faster HTTP server 2022-08-17 07:37:14 -07:00
sno2
f5d896542a fix: update build files to latest Zig version 2022-07-09 18:25:25 -07:00
Jarred Sumner
bdf28e42f6 [bun.js] Support gzipSync, gunzipSync, inflateSync, deflateSync 2022-05-07 01:38:26 -07:00
Jarred Sumner
e75c711c68 Upgrade to latest Zig, remove dependency on patched version of Zig (#96)
* Prepare to upgrade zig

* zig fmt

* AllocGate

* Update data_url.zig

* wip

* few files

* just headers now?

* I think everything works?

* Update mimalloc

* Update hash_map.zig

* Perf improvements to compensate for Allocgate

* Bump

* 📷

* Update bun.lockb

* Less branching

* [js parser] Slightly reduce memory usage

* Update js_parser.zig

* WIP remove unused

* [JS parser] WIP support for `with` keyword

* Remove more dead code

* Fix all the build errors!

* cleanup

* Move `network_thread` up

* Bump peechy

* Update README.md
2021-12-30 21:12:32 -08:00
Jarred Sumner
809c1e46d8 [npm install] starting to look good! 2021-12-16 19:18:51 -08:00
Jarred Sumner
0db7af6166 [fetch] Add support for gzip & deflate to the http client
Powered by Cloudflare's zlib fork
2021-10-10 23:41:18 -07:00