138 Commits

Author SHA1 Message Date
Jarred Sumner
528620e9ae Add postinstall optimizer with native binlink support and script skipping (#24283)
## Summary

This PR introduces a new postinstall optimization system that
significantly reduces the need to run lifecycle scripts for certain
packages by intelligently handling their requirements at install time.

## Key Features

### 1. Native Binlink Optimization

When packages like `esbuild` ship platform-specific binaries as optional
dependencies, we now:
- Detect the native binlink pattern (enabled by default for `esbuild`)
- Find the matching platform-specific dependency based on target CPU/OS
- Link binaries directly from the platform-specific package (e.g.,
`@esbuild/darwin-arm64`)
- Fall back gracefully if the platform-specific package isn't found

**Result**: No postinstall scripts needed for esbuild and similar
packages.

### 2. Lifecycle Script Skipping

For packages like `sharp` that run heavy postinstall scripts:
- Skip lifecycle scripts entirely (enabled by default for `sharp`)
- Prevents downloading large binaries or compiling native code
unnecessarily
- Reduces install time and potential failures in restricted environments

## Configuration

Both features can be configured via `package.json`:

```json
{
  "nativeDependencies": ["esbuild", "my-custom-package"],
  "ignoreScripts": ["sharp", "another-package"]
}
```

Set to empty arrays to disable defaults:
```json
{
  "nativeDependencies": [],
  "ignoreScripts": []
}
```

Environment variable overrides:
- `BUN_FEATURE_FLAG_DISABLE_NATIVE_DEPENDENCY_LINKER=1` - disable native
binlink
- `BUN_FEATURE_FLAG_DISABLE_IGNORE_SCRIPTS=1` - disable script ignoring

## Implementation Details

### Core Components

- **`postinstall_optimizer.zig`**: New file containing the optimizer
logic
- `PostinstallOptimizer` enum with `native_binlink` and `ignore`
variants
  - `List` type to track optimization strategies per package hash
  - Defaults for `esbuild` (native binlink) and `sharp` (ignore)
  
- **`Bin.Linker` changes**: Extended to support separate target paths
  - `target_node_modules_path`: Where to find the actual binary
  - `target_package_name`: Name of the package containing the binary
  - Fallback logic when native binlink optimization fails

### Modified Components

- **PackageInstaller.zig**: Checks optimizer before:
  - Enqueueing lifecycle scripts
  - Linking binaries (with platform-specific package resolution)
  
- **isolated_install/Installer.zig**: Similar checks for isolated linker
mode
  - `maybeReplaceNodeModulesPath()` resolves platform-specific packages
  - Retry logic without optimization on failure

- **Lockfile**: Added `postinstall_optimizer` field to persist
configuration

## Changes Included

- Updated `esbuild` from 0.21.5 to 0.25.11 (testing with latest)
- VS Code launch config updates for debugging install with new flags
- New feature flags in `env_var.zig`

## Test Plan

- [x] Existing install tests pass
- [ ] Test esbuild install without postinstall scripts running
- [ ] Test sharp install with scripts skipped
- [ ] Test custom package.json configuration
- [ ] Test fallback when platform-specific package not found
- [ ] Test feature flag overrides

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Native binlink optimization: installs platform-specific binaries when
available, with a safe retry fallback and verbose logging option.
* Per-package postinstall controls to optionally skip lifecycle scripts.
* New feature flags to disable native binlink optimization and to
disable lifecycle-script ignoring.

* **Tests**
* End-to-end tests and test packages added to validate native binlink
behavior across install scenarios and linker modes.

* **Documentation**
  * Bench README and sample app migrated to a Next.js-based setup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-11-03 20:36:22 -08:00
Dylan Conway
f770b1b1c7 fix(install): fix optional peer resolving (#24272)
### What does this PR do?
Allows optional peers to resolve to package if possible.

Optional peers aren't auto-installed, but they should still be given a
chance to resolve. If they're always left unresolved it's possible for
multiple dependencies on the same package to result in different peer
resolutions when they should be the same. For example, this bug this
could cause monorepos using elysia to have corrupt node_modules because
there might be more than one copy of elysia in `node_modules/.bun` (or
more than the expected number of copies).

fixes #23725
most likely fixes #23895

fixes ENG-21411

### How did you verify your code works?
Added a test for optional peers and non-optional peers that would
previously trigger this bug.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Improved resolution of optional peer dependencies during isolated
installations, with better propagation across package hierarchies.

* **Tests**
* Added comprehensive test suite covering optional peer dependency
scenarios in isolated workspaces.
* Added test fixtures for packages with peer and optional peer
dependencies.
* Enhanced lockfile migration test verification using snapshot-based
assertions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-11-01 22:38:36 -07:00
Dylan Conway
6c8635da63 fix(install): isolated installs with transitive self dependencies (#23222)
### What does this PR do?
Packages with self dependencies at a different version were colliding
with the current version in the store node_modules. This pr nests them
in another node_modules

Example:
self-dep@1.0.2 has a dependency on self-dep@1.0.1.

self-dep@1.0.2 is placed here in:
`./node_modules/.bun/self-dep@1.0.2/node_modules/self-dep`

and it's self-dep dependency symlink is now placed in:

`./node_modules/.bun/self-dep@1.0.2/node_modules/self-dep/node_modules/self-dep`

fixes #22681
### How did you verify your code works?
Manually tested the linked issue is working, and added a test

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-10-04 02:59:47 -07:00
Dylan Conway
4580e11fc3 [PKG-517] fix(install): --linker=isolated should spawn scripts on the main thread (#21425)
### What does this PR do?
Fixes thread safety issues due to file poll code being not thread safe.
<!-- **Please explain what your changes do**, example: -->

<!--

This adds a new flag --bail to bun test. When set, it will stop running
tests after the first failure. This is useful for CI environments where
you want to fail fast.

-->

### How did you verify your code works?
Added tests for lifecycle scripts. The tests are unlikely to reproduce
the bug, but we'll know if it actually fixes the issue if
`test/package.json` doesn't show in flaky tests anymore.
<!-- **For code changes, please include automated tests**. Feel free to
uncomment the line below -->

<!-- I wrote automated tests -->

<!-- If JavaScript/TypeScript modules or builtins changed:

- [ ] I included a test for the new code, or existing tests cover it
- [ ] I ran my tests locally and they pass (`bun-debug test
test-file-name.test`)

-->

<!-- If Zig files changed:

- [ ] I checked the lifetime of memory allocated to verify it's (1)
freed and (2) only freed when it should be
- [ ] I included a test for the new code, or an existing test covers it
- [ ] JSValue used outside of the stack is either wrapped in a
JSC.Strong or is JSValueProtect'ed
- [ ] I wrote TypeScript/JavaScript tests and they pass locally
(`bun-debug test test-file-name.test`)
-->

<!-- If new methods, getters, or setters were added to a publicly
exposed class:

- [ ] I added TypeScript types for the new methods, getters, or setters
-->

<!-- If dependencies in tests changed:

- [ ] I made sure that specific versions of dependencies are used
instead of ranged or tagged versions
-->

<!-- If a new builtin ESM/CJS module was added:

- [ ] I updated Aliases in `module_loader.zig` to include the new module
- [ ] I added a test that imports the module
- [ ] I added a test that require() the module
-->

---------

Co-authored-by: taylor.fish <contact@taylor.fish>
2025-07-28 12:29:47 -07:00
Jarred Sumner
2e02d9de28 Use ReadableStream.prototype.* in tests instead of new Response(...).* (#20937)
Co-authored-by: Jarred-Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Alistair Smith <hi@alistair.sh>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-14 00:47:53 -07:00
Dylan Conway
f24e8cb98a implement "nodeLinker": "isolated" in bun install (#20440)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-07-09 00:19:57 -07:00
nobkd
6edc3a9900 remove audit from bun pm help (#20167) 2025-06-03 14:37:03 -07:00
Alistair Smith
9e329ee605 bun pm audit (#19855) 2025-05-23 22:31:12 -07:00
Dylan Conway
73bcff9d01 fix 16842 (#16952) 2025-01-31 22:39:30 -08:00
Dylan Conway
c130df6c58 start verdaccio in multiple test files (#16118) 2025-01-03 08:21:00 -08:00
Dylan Conway
5058bd3913 handle bundle(d)Dependencies in bun install (#16055) 2024-12-31 13:40:55 -08:00
Dylan Conway
45ca9e08c3 fix(install): peer/dev/optional = false lockfile fix (#15874)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-12-20 00:34:21 -08:00
Dylan Conway
d5f1f2f8ad Use the same hoisting logic for text lockfile (#15778) 2024-12-16 16:37:46 -08:00
Jarred Sumner
c218bffd94 Add "bin" field to bun.lock (#15763)
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2024-12-14 22:52:17 -08:00
Dylan Conway
3df39f4bb7 bun.lock: fix --frozen-lockfile and resolving extra dependencies (#15748) 2024-12-13 22:40:12 -08:00
Dylan Conway
8b3b1442fd bun.lock workspace sorting and comma bugfix (#15739) 2024-12-12 19:33:44 -08:00
Robert Shuford
7eb8a3feae Fixes #14433 - global .npmrc not using auth (#15539) 2024-12-04 12:37:18 -08:00
Dylan Conway
d7ed9c673e add a --config test for bun install (#15546) 2024-12-04 12:36:10 -08:00
Meghan Denny
82cb82d828 pm: add some missing npm_ env vars (#14786)
Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>
2024-11-22 15:13:32 -08:00
Jarred Sumner
5bcaf32ba3 Fix lockfile print crash (#15332) 2024-11-22 02:07:11 -08:00
Dylan Conway
910efec0b7 fix auto-install on windows when symlinks aren't available (#15182) 2024-11-16 00:43:12 -08:00
Dylan Conway
8f5eab3c84 fix(install): package-lock.json migration and non-existent cafile fixes (#15059) 2024-11-08 14:34:44 -08:00
Dylan Conway
62881ee36b Redact secrets in bunfig.toml and npmrc logs (#14919) 2024-10-31 18:44:24 -07:00
Dylan Conway
6933208790 fix(install): only globally link requested packages (#12506)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-10-31 18:21:04 -07:00
Dylan Conway
489890deb1 fix(install): check cached package.jsons (#14899) 2024-10-29 18:55:52 -07:00
Meghan Denny
5237869101 bun-install-registry.test.ts: remove ini format hint here (#14803) 2024-10-26 16:51:19 -07:00
Meghan Denny
50d80a805d pm: fix weird package.json formatting after install (#14801) 2024-10-26 01:36:25 -07:00
Dylan Conway
9621b641a1 update test/bun.lockb (#14746) 2024-10-25 00:03:52 -07:00
Dylan Conway
247456b675 fix(install): continue install if optional postinstall fails (#14783) 2024-10-23 21:58:53 -07:00
Eckhardt (Kaizen) Dreyer
3db0191409 fix(install): Skip optional dependencies if false in bunfig.toml (#14629) 2024-10-22 11:55:10 -07:00
Ashcon Partovi
7bb39023b8 Merge queue (#14639) 2024-10-18 01:14:42 +00:00
Meghan Denny
709cd95c30 test: use isWindows from harness (#14577) 2024-10-14 21:19:09 -07:00
Dylan Conway
d3323c84bb fix(publish): missing bins bugfix (#14488)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2024-10-11 21:28:47 -07:00
Dylan Conway
5fd0a61ae2 CA support for bun install (#14416) 2024-10-11 13:16:26 -07:00
Dylan Conway
c071415664 add bun pm whoami (#14387) 2024-10-07 17:36:14 -07:00
Dylan Conway
b2cb3603e2 fix(publish): ignore npm-notice when x-local-cache exists (#14352) 2024-10-04 23:36:34 -07:00
Dylan Conway
f307d2a6ef test(publish): ci names in user-agent (#14328) 2024-10-04 00:15:22 -07:00
Dylan Conway
39b1c0111e add docs for bun publish (#14327) 2024-10-03 03:41:11 -07:00
Dylan Conway
f374ae6db1 add bun publish (#14215) 2024-10-02 20:47:22 -07:00
Meghan Denny
25083a4252 pm: print command name to stdout (#14266) 2024-10-02 02:24:37 -07:00
Robert Shuford
5e97fb8d97 Support reading from $HOME/.npmrc (#13990) 2024-09-26 14:41:28 -07:00
Dylan Conway
7ef0f04acd fix #13942 (#13943) 2024-09-14 01:22:28 -04:00
Dylan Conway
749632f125 fix(spawn): update cwd before searching for executable to run (#13845)
Co-authored-by: dylan-conway <dylan-conway@users.noreply.github.com>
2024-09-11 16:58:24 -07:00
Dylan Conway
4a58a97fa0 fix sleep tests in bun-install-registry.test.ts (#13874) 2024-09-10 22:28:57 -07:00
Jarred Sumner
cd6785771e run prettier and add back format action (#13722) 2024-09-03 21:32:52 -07:00
Dylan Conway
f3ed9eac4a fix(outdated): match scoped names with * (#13610) 2024-08-29 16:06:41 -07:00
Dylan Conway
ba2ea6fbb2 add --filter and package pattern arguments to bun outdated (#13557) 2024-08-27 00:18:27 -07:00
Dylan Conway
1a9307da08 bun outdated docs (#13497)
Co-authored-by: Zack Radisic <zack@theradisic.com>
2024-08-23 23:11:52 -07:00
Dylan Conway
6415cc3e92 implement bun outdated (#13461)
Co-authored-by: Zack Radisic <zack@theradisic.com>
2024-08-23 02:15:13 -07:00
Dylan Conway
5f7b96b58f fix(install): optional peer dependency bugfix (#12485)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: dylan-conway <dylan-conway@users.noreply.github.com>
2024-07-10 20:04:32 -07:00