Commit Graph

39 Commits

Author SHA1 Message Date
pfg
05d0475c6c Update to zig 0.15.2 (#24204)
Fixes ENG-21287

Build times, from `bun run build && echo '//' >> src/main.zig && time
bun run build`

|Platform|0.14.1|0.15.2|Speedup|
|-|-|-|-|
|macos debug asan|126.90s|106.27s|1.19x|
|macos debug noasan|60.62s|50.85s|1.19x|
|linux debug asan|292.77s|241.45s|1.21x|
|linux debug noasan|146.58s|130.94s|1.12x|
|linux debug use_llvm=false|n/a|78.27s|1.87x|
|windows debug asan|177.13s|142.55s|1.24x|

Runtime performance:

- next build memory usage may have gone up by 5%. Otherwise seems the
same. Some code with writers may have gotten slower, especially one
instance of a counting writer and a few instances of unbuffered writers
that now have vtable overhead.
- File size reduced by 800kb (from 100.2mb to 99.4mb)

Improvements:

- `@export` hack is no longer needed for watch
- native x86_64 backend for linux builds faster. to use it, set use_llvm
false and no_link_obj false. also set `ASAN_OPTIONS=detect_leaks=0`
otherwise it will spam the output with tens of thousands of lines of
debug info errors. may need to use the zig lldb fork for debugging.
- zig test-obj, which we will be able to use for zig unit tests

Still an issue:

- false 'dependency loop' errors remain in watch mode
- watch mode crashes observed

Follow-up:

- [ ] search `comptime Writer: type` and `comptime W: type` and remove
- [ ] remove format_mode in our zig fork
- [ ] remove deprecated.zig autoFormatLabelFallback
- [ ] remove deprecated.zig autoFormatLabel
- [ ] remove deprecated.BufferedWriter and BufferedReader
- [ ] remove override_no_export_cpp_apis as it is no longer needed
- [ ] css Parser(W) -> Parser, and remove all the comptime writer: type
params
- [ ] remove deprecated writer fully

Files that add lines:

```
649     src/deprecated.zig
167     scripts/pack-codegen-for-zig-team.ts
54      scripts/cleartrace-impl.js
46      scripts/cleartrace.ts
43      src/windows.zig
18      src/fs.zig
17      src/bun.js/ConsoleObject.zig
16      src/output.zig
12      src/bun.js/test/debug.zig
12      src/bun.js/node/node_fs.zig
8       src/env_loader.zig
7       src/css/printer.zig
7       src/cli/init_command.zig
7       src/bun.js/node.zig
6       src/string/escapeRegExp.zig
6       src/install/PnpmMatcher.zig
5       src/bun.js/webcore/Blob.zig
4       src/crash_handler.zig
4       src/bun.zig
3       src/install/lockfile/bun.lock.zig
3       src/cli/update_interactive_command.zig
3       src/cli/pack_command.zig
3       build.zig
2       src/Progress.zig
2       src/install/lockfile/lockfile_json_stringify_for_debugging.zig
2       src/css/small_list.zig
2       src/bun.js/webcore/prompt.zig
1       test/internal/ban-words.test.ts
1       test/internal/ban-limits.json
1       src/watcher/WatcherTrace.zig
1       src/transpiler.zig
1       src/shell/builtin/cp.zig
1       src/js_printer.zig
1       src/io/PipeReader.zig
1       src/install/bin.zig
1       src/css/selectors/selector.zig
1       src/cli/run_command.zig
1       src/bun.js/RuntimeTranspilerStore.zig
1       src/bun.js/bindings/JSRef.zig
1       src/bake/DevServer.zig
```

Files that remove lines:

```
-1      src/test/recover.zig
-1      src/sql/postgres/SocketMonitor.zig
-1      src/sql/mysql/MySQLRequestQueue.zig
-1      src/sourcemap/CodeCoverage.zig
-1      src/css/values/color_js.zig
-1      src/compile_target.zig
-1      src/bundler/linker_context/convertStmtsForChunk.zig
-1      src/bundler/bundle_v2.zig
-1      src/bun.js/webcore/blob/read_file.zig
-1      src/ast/base.zig
-2      src/sql/postgres/protocol/ArrayList.zig
-2      src/shell/builtin/mkdir.zig
-2      src/install/PackageManager/patchPackage.zig
-2      src/install/PackageManager/PackageManagerDirectories.zig
-2      src/fmt.zig
-2      src/css/declaration.zig
-2      src/css/css_parser.zig
-2      src/collections/baby_list.zig
-2      src/bun.js/bindings/ZigStackFrame.zig
-2      src/ast/E.zig
-3      src/StandaloneModuleGraph.zig
-3      src/deps/picohttp.zig
-3      src/deps/libuv.zig
-3      src/btjs.zig
-4      src/threading/Futex.zig
-4      src/shell/builtin/touch.zig
-4      src/meta.zig
-4      src/install/lockfile.zig
-4      src/css/selectors/parser.zig
-5      src/shell/interpreter.zig
-5      src/css/error.zig
-5      src/bun.js/web_worker.zig
-5      src/bun.js.zig
-6      src/cli/test_command.zig
-6      src/bun.js/VirtualMachine.zig
-6      src/bun.js/uuid.zig
-6      src/bun.js/bindings/JSValue.zig
-9      src/bun.js/test/pretty_format.zig
-9      src/bun.js/api/BunObject.zig
-14     src/install/install_binding.zig
-14     src/fd.zig
-14     src/bun.js/node/path.zig
-14     scripts/pack-codegen-for-zig-team.sh
-17     src/bun.js/test/diff_format.zig
```

`git diff --numstat origin/main...HEAD | awk '{ print ($1-$2)"\t"$3 }' |
sort -rn`

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Meghan Denny <meghan@bun.com>
Co-authored-by: tayor.fish <contact@taylor.fish>
2025-11-10 14:38:26 -08:00
Michael H
ba20670da3 implement pnpm migration (#22262)
### What does this PR do?

fixes #7157, fixes #14662

migrates pnpm-workspace.yaml data to package.json & converts
pnpm-lock.yml to bun.lock

---

### How did you verify your code works?

manually, tests and real world examples

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-09-27 00:45:29 -07:00
Dylan Conway
084eeb945e fix(install): serialize updated workspaces versions correctly for bun.lockb (#22932)
### What does this PR do?
This change was missing after changing semver core numbers to use u64.

Also fixes potentially serializing uninitialized bytes from resolution
unions.
### How did you verify your code works?
Added a test for migrating a bun.lockb with most features used.
2025-09-24 02:42:57 -07:00
Dylan Conway
285143dc66 fix(install): change semver core numbers to u64 (#22889)
### What does this PR do?
Sometimes packages will use very large numbers exceeding max u32 for
major/minor/patch (usually patch). This pr changes each core number in
bun to u64.

Because we serialize package information to disk for the binary lockfile
and package manifests, this pr bumps the version of each. We don't need
to change anything other than the version for serialized package
manifests because they will invalidate and save the new version. For old
binary lockfiles, this pr adds logic for migrating to the new version.
Even if there are no changes, migrating will always save the new
lockfile. Unfortunately means there will be a one time invisible diff
for binary lockfile users, but this is better than installs failing to
work.

fixes #22881
fixes #21793
fixes #16041
fixes #22891

resolves BUN-7MX, BUN-R4Q, BUN-WRB

### How did you verify your code works?
Manually, and added a test for migrating from an older binary lockfile.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-22 19:28:26 -07:00
taylor.fish
07cd45deae Refactor Zig imports and file structure (part 1) (#21270)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-07-22 17:51:38 -07:00
pfg
83760fc446 Sort imports in all files (#21119)
Co-authored-by: taylor.fish <contact@taylor.fish>
2025-07-21 13:26:47 -07:00
Dylan Conway
f24e8cb98a implement "nodeLinker": "isolated" in bun install (#20440)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-07-09 00:19:57 -07:00
Jarred Sumner
3ea6133c46 CI: Remove unused top-level decls in formatter in zig (#19879)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-23 22:49:48 -07:00
Dylan Conway
98ee30eccf Implement catalogs in bun install (#19809)
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Jarred-Sumner <Jarred-Sumner@users.noreply.github.com>
2025-05-20 23:03:21 -07:00
chloe caruso
4ec410e0d7 internal: make @import("bun") work in zig (#19096) 2025-04-17 12:32:47 -07:00
Jarred Sumner
aac951bd47 Move semver-related structs into their own files (#16987) 2025-02-02 00:20:45 -08:00
Meghan Denny
160bf9d563 zig: make install.Resolution.init() not use anytype (#16852) 2025-01-28 18:08:10 -08:00
Dylan Conway
b55ca429c7 Implement text-based lockfile (#15705) 2024-12-11 05:05:49 -08:00
Jarred Sumner
cacbaba524 Make signal.abort() from native code fast (#13064) 2024-08-05 15:50:36 -07:00
Dylan Conway
3e4c0918a4 fix(install): invalidate manifest cache on registry change (#11606) 2024-06-05 18:44:36 -07:00
Dylan Conway
5a81dc8e33 fix(install): fix dependency install order (#10240)
* packages wait for parent trees before install

* use `directoryExistsAt`

* missing increment

* fix faccessat

* swap destination and target

* update

* force

* only create destination dir before installing package

* fix windows symlink/junction

* increment, false on extracting

* done
2024-04-15 18:29:34 -07:00
Dylan Conway
bf0e5a82f7 fix(windows): install bugfixes for workspaces, tarballs, and relative paths (#8813)
* more tests

* update migration.zig

* fix up paths

* update tests

* update tests

* test

* test

* update registry tests

* comments

* early exit if stream is invalid

* dont pass invalid_fd to openFileAtWindows

* fix merge

* misc crash fix

* make this use optional pointers instead of 0xaa

* ensure absolute paths are propagated properly

* package.json expects forward slash

* this assert was invalid

* add panic checks

* pass bun-remove

* more panic checks

* test: pass bun-add

* querying these hangs outside bun too

* fix compile error from merge conflict resolution

* use compileError instead of comptime unreachable

* tidy

* bunx: check for the .exe bin extension

* bunx: another route to make cache path if it doesnt exist

* install: another case of FolderResolution.getOrPut expecting absolute path

* fix a bun install crash

* dont print zig stack trace if bun install fails

* test: pass bun-link

* test: bunx: add more expects

* test: bun-install-registry: pass

* test: bun-install: pass

* test: bun-pm: pass

* fix merge main error

* fix posix tests

* fix last failing test in bun-install.test.ts
symlink difference between platforms

* bun-install-registry.test.ts fix

* bun-run.test.ts: remove stray console log

---------

Co-authored-by: Meghan Denny <meghan@bun.sh>
Co-authored-by: Meghan Denny <hello@nektro.net>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-03-18 22:34:15 -07:00
Dylan Conway
e45ece05d9 improve bun pm trust output (#9371)
* fix recursion

* accurate count of blocked package scripts

* update

* fix merge

* reset pipereaders, set bin linker error correctly

* more pretty

* small changes

* e

* update tests

* this one too

* bun.start_time, iterate package_ids, remove put
2024-03-12 19:50:46 -07:00
Dylan Conway
d37fbbd4e0 fix(install): lifecycle script changes (#8943)
* empty trustedDependencies

* tests

* handle edgecases with default trusted dependencies

* could be zero length

* --trusted and skipped scripts

* resolver too

* second run --trusted

* --trust, better formatting

* more tests

* --trusted applies to dep deps, more tests

* progress

* fix build

* fix crash, make it look good, comments

* alphabetize, verbose log

* feature flag

* update lockfile

* update skipped text

* check update requests first

* be more careful with inline strings

* only with scripts

* fix tests, todo tests

* fix another test

* fix merge

* fix fix merge

* check binding.gyp for tarball and git resolutions

* remove dead code

* debug assert

* move newline printing

* use enum for `__has_install_script`

* oops

* clone packages

* Update src/install/install.zig

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-03-07 19:22:21 -08:00
Dylan Conway
4f98336f86 fix(windows): fix installing non-ascii paths and make normalizeBuf generic (#8608)
* comptime type normalizeStringBuf

* delete

* revert

* revert revert

* revert

* remove unused

* remove unnecessary assert

* add comment

* remove normalize, need ../

* use Output.err

* update error message

* generic T suffix

* fix windows build

* more fix build

* more fix build

* mkdiratZ

* update test

* [autofix.ci] apply automated fixes

* update snapshot again

* fix merge

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2024-01-31 22:44:23 -08:00
Meghan Denny
8733748eb6 remove bun.assertDefined, comparing against undefined is UB (#8137) 2024-01-12 18:16:04 -08:00
Dylan Conway
351e47355a add default trustedDependencies and run lifecycle scripts during installation (#7132)
* [install] allow parallel execution of `postinstall` scripts

- fixes spurious exit code 42 from `spawn()`

* postinstall to a pipe

* feat(install): include top 500 packages as defaults for postinstall

* windows newline handling*

*i did not test it

* stuff

* cool

* a

* fix merge

* set `has_trusted_dependencies`

* fix a bunch of tests

* fix merge

* remove `PackageManager`

* remove commented code

* change to function

* Update lockfile.zig

* run scripts if added to `trustedDependencies` after install

* packages without `resolved` properties

* node-gyp scripts

* node-gyp script in the root

* another test

* git deps run prepare scripts

* fix merge

* run lifecycle scripts during installation

* Update lockfile.zig

* always increment

* 🏗️

* update tests

* tickWIthoutIdle

* const uws

* loop forwards through trees

* single buffer bitset list

* tag.isGit

* windows path separators

* `bun.sys.read` and enable/disable buffering

* fix test and waiter thread

* waiter thread and tests

* Update bun-install-registry.test.ts

* workspace exclude `preprepare` and `postprepare`

* Create esbuild.test.ts

* make sure length is the same

* remove deferred binlinks, add estrella test

* test with another version

* address some comments

* remove .verdaccio-db.json

* ooops

* fix build

* use `pid` to wait

* dont register pid_poll when using waiter thread

* stress test

* free

* fix failing tests

* fix linux crash, snapshot stress test

* oops

* concurrent scripts

* activate as soon as possible

* test

* delete stress test packages

* remove unused packages

* comment stress test and maybe fix segfault

* delete snapshot

* fix assertion

* use cpu_count * 2 for default concurrent scripts

* gear emoji

* add --concurrent-scripts to docs

* more docs

---------

Co-authored-by: alexlamsl <alexlamsl@gmail.com>
Co-authored-by: dave caruso <me@paperdave.net>
Co-authored-by: Dylan Conway <33744874+MilesWright7@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2023-12-11 22:08:25 -08:00
dave caruso
1bf28e0d77 feat(install): automatically migrate package-lock.json to bun.lockb (#6352)
* work so far

* stuff

* a

* basics work

* stuff

* yoo

* build lockfile

* correct

* f

* a

* install fixture havent tested

* i made it worse

* lol

* be more reasonable

* make the test easier to pass because bun install doesn't handle obscure lockfile edge cases :/

* a

* works now

* ok

* a

* a

* cool

* nah

* fix stuff

* l

* a

* idfk

* LAME

* prettier errors

* does this fix tests?

* Add more safety checks to Integrity

* Add another check

* More careful lifetime handling

* Fix linux debugger issue

* a

* tmp dir and snapshot test

---------

Co-authored-by: Jarred SUmner <jarred@jarredsumner.com>
2023-10-11 02:27:07 -07:00
Jarred Sumner
3f463786a5 Correctly fix #5888 (#6000)
* Update our MultiArrayList

* More consistent CI

* Fix serialization issue

* Update bun.lockb

* Update bun.lockb

* fixup

* Fix `Bin{}`

* `bun update` should not cache the manifest

* Make bun install tests more consistent

* This differs by platform evidently

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-09-24 17:43:59 -07:00
dave caruso
1e998c1bf2 fix(install): ensure all lockfile structs do not have undefined padding (#4401)
* padding sucks

* this assertion is already done elsewhere

* remove test. will be covered alex's pr i believe?

* fix webkit submodule

* fix uws submodule
2023-09-05 14:25:19 -07:00
Dylan Conway
5fa13625a1 upgrade zig to v0.11.0-dev.3737+9eb008717 (#3374)
* progress

* finish `@memset/@memcpy` update

* Update build.zig

* change `@enumToInt` to `@intFromEnum` and friends

* update zig versions

* it was 1

* add link to issue

* add `compileError` reminder

* fix merge

* format

* upgrade to llvm 16

* Revert "upgrade to llvm 16"

This reverts commit cc930ceb1c.

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-06-21 23:38:18 -07:00
Alex Lam S.L
c60d7db178 [install] support git dependencies (#2094) 2023-02-17 14:48:57 -08:00
Alex Lam S.L
5a19f8434b [WIP] fix corner cases with aliases dependencies (#2000)
* fix corner cases with aliases dependencies

* in-memory convert legacy `bun.lockb`
2023-02-06 22:24:34 -08:00
Jarred Sumner
468226fc90 be more careful 2023-01-25 01:52:04 -08:00
Jarred Sumner
274ae5de1b [bun install] Make it print workspace: instead of workspace:// 2023-01-16 18:57:04 -08:00
Alex Lam S.L
7fa023b8b5 support installation of NPM workspaces (#1764) 2023-01-14 07:14:48 -08:00
Jarred Sumner
c0dd284136 Upgrade to latest Zig (#1610)
* @min and @max

* builtins and some trivial ones

* Most of them

* more

* more!

* More Progress

* wip

* Update tagged_pointer.zig

* Update http_client_async.zig

* Most of the iterable dir changes

* alright

* Remove usages of deprecated formatters

* 📷

* fmt

* Update shimmer.zig

* wip

* wip

* wip

* progress

* more

* Latest

* stuck on error

* latest

* workaround stage2

* wip

* Update string_immutable.zig

* wip

* Migrate `Dirent` and `require("fs')` to use JSC<>Zig bindings

* Fix build errors

* Fixup most of the test failures

* Fix `make headers`

* Fix "outside package path" error

* Fixup aligned alloc

* Add missing file

* linux

* More linux fixes

* use latest peechy

* Fix transpiler test failure

* Forgot about these

* Fixup test failure

* Update node-timers.test.ts

* [node:htt] Fix `undefined is not an object` error

Fixes https://github.com/oven-sh/bun/issues/1618

* Update http.exports.js

* Make this test less flaky

* fix hashes

* Fix hex formatting and zls issues

* Download zig version

* Update Dockerfile

* Update Dockerfile

* Update uws

* Update Dockerfile

* Set llvm version

* Update README.md

* Update uws

* Update Dockerfile

* Update io_linux.zig

* Update bun.zig

* Log output

* workaround strange @cInclude error

* Make ffi tests better

* Don't use cImport

* Update c.zig

* Update c-bindings.cpp

* call setOutputDir

* Update Dockerfile

* Use a longer name

* latest

* Update serve.test.ts

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Veikka Tuominen <git@vexu.eu>
2022-12-28 00:51:22 -08:00
Jarred Sumner
cc2c50ad09 [bun install] Implement bun link 2022-08-07 23:23:34 -07:00
Jarred Sumner
a16dcbbb5b [bun install] Support private registries & private scopes 2022-08-05 22:41:43 -07:00
luke miles
7f33846005 copy: replace Bun with bun (#99)
Most CLI tools have the style convention of referring to themselves in
lowercase. It is, after all, the name that users type in when using
the tool. This PR maintains that convention in bun.

"Drop the uppercase B, it's cleaner"
2022-01-02 03:09:51 -08:00
Jarred Sumner
0c7b35d0ab [bun install] Support file: protocol for installing folders as packages 2021-12-19 14:38:32 -08:00
Jarred Sumner
e97106c770 [bun install] order() and fmt() 2021-12-16 19:18:51 -08:00
Jarred Sumner
339fe00dfe CLI flags 2021-12-16 19:18:51 -08:00
Jarred Sumner
41c60fb848 [bun install] move around some code 2021-12-16 19:18:51 -08:00