Files
bun.sh/test/cli/install/registry/packages/create-native-binlink-packages.ts
Jarred Sumner 528620e9ae Add postinstall optimizer with native binlink support and script skipping (#24283)
## Summary

This PR introduces a new postinstall optimization system that
significantly reduces the need to run lifecycle scripts for certain
packages by intelligently handling their requirements at install time.

## Key Features

### 1. Native Binlink Optimization

When packages like `esbuild` ship platform-specific binaries as optional
dependencies, we now:
- Detect the native binlink pattern (enabled by default for `esbuild`)
- Find the matching platform-specific dependency based on target CPU/OS
- Link binaries directly from the platform-specific package (e.g.,
`@esbuild/darwin-arm64`)
- Fall back gracefully if the platform-specific package isn't found

**Result**: No postinstall scripts needed for esbuild and similar
packages.

### 2. Lifecycle Script Skipping

For packages like `sharp` that run heavy postinstall scripts:
- Skip lifecycle scripts entirely (enabled by default for `sharp`)
- Prevents downloading large binaries or compiling native code
unnecessarily
- Reduces install time and potential failures in restricted environments

## Configuration

Both features can be configured via `package.json`:

```json
{
  "nativeDependencies": ["esbuild", "my-custom-package"],
  "ignoreScripts": ["sharp", "another-package"]
}
```

Set to empty arrays to disable defaults:
```json
{
  "nativeDependencies": [],
  "ignoreScripts": []
}
```

Environment variable overrides:
- `BUN_FEATURE_FLAG_DISABLE_NATIVE_DEPENDENCY_LINKER=1` - disable native
binlink
- `BUN_FEATURE_FLAG_DISABLE_IGNORE_SCRIPTS=1` - disable script ignoring

## Implementation Details

### Core Components

- **`postinstall_optimizer.zig`**: New file containing the optimizer
logic
- `PostinstallOptimizer` enum with `native_binlink` and `ignore`
variants
  - `List` type to track optimization strategies per package hash
  - Defaults for `esbuild` (native binlink) and `sharp` (ignore)
  
- **`Bin.Linker` changes**: Extended to support separate target paths
  - `target_node_modules_path`: Where to find the actual binary
  - `target_package_name`: Name of the package containing the binary
  - Fallback logic when native binlink optimization fails

### Modified Components

- **PackageInstaller.zig**: Checks optimizer before:
  - Enqueueing lifecycle scripts
  - Linking binaries (with platform-specific package resolution)
  
- **isolated_install/Installer.zig**: Similar checks for isolated linker
mode
  - `maybeReplaceNodeModulesPath()` resolves platform-specific packages
  - Retry logic without optimization on failure

- **Lockfile**: Added `postinstall_optimizer` field to persist
configuration

## Changes Included

- Updated `esbuild` from 0.21.5 to 0.25.11 (testing with latest)
- VS Code launch config updates for debugging install with new flags
- New feature flags in `env_var.zig`

## Test Plan

- [x] Existing install tests pass
- [ ] Test esbuild install without postinstall scripts running
- [ ] Test sharp install with scripts skipped
- [ ] Test custom package.json configuration
- [ ] Test fallback when platform-specific package not found
- [ ] Test feature flag overrides

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Native binlink optimization: installs platform-specific binaries when
available, with a safe retry fallback and verbose logging option.
* Per-package postinstall controls to optionally skip lifecycle scripts.
* New feature flags to disable native binlink optimization and to
disable lifecycle-script ignoring.

* **Tests**
* End-to-end tests and test packages added to validate native binlink
behavior across install scenarios and linker modes.

* **Documentation**
  * Bench README and sample app migrated to a Next.js-based setup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
2025-11-03 20:36:22 -08:00

156 lines
4.8 KiB
TypeScript

#!/usr/bin/env bun
/**
* This script creates test packages for native binlink optimization testing.
* It creates:
* - test-native-binlink: main package with a bin that exits with code 1
* - test-native-binlink-target: platform-specific package with bin that exits with code 0
*/
import { $ } from "bun";
import { mkdir, writeFile } from "fs/promises";
import { join } from "path";
const packagesDir = import.meta.dir;
// Main package that should NOT be used
const mainPkgDir = join(packagesDir, "test-native-binlink-tmp");
await mkdir(mainPkgDir, { recursive: true });
await mkdir(join(mainPkgDir, "bin"), { recursive: true });
await writeFile(
join(mainPkgDir, "package.json"),
JSON.stringify(
{
name: "test-native-binlink",
version: "1.0.0",
bin: {
"test-binlink-cmd": "./bin/main.js",
},
optionalDependencies: {
"test-native-binlink-target": "1.0.0",
},
},
null,
2,
),
);
await writeFile(
join(mainPkgDir, "bin", "main.js"),
`#!/usr/bin/env node
console.log("ERROR: Using main package bin, not platform-specific!");
process.exit(1);
`,
);
// Create package structure for tarball
const mainTarDir = join(mainPkgDir, "package");
await mkdir(mainTarDir, { recursive: true });
await mkdir(join(mainTarDir, "bin"), { recursive: true });
await $`cp ${join(mainPkgDir, "package.json")} ${mainTarDir}/`;
await $`cp ${join(mainPkgDir, "bin", "main.js")} ${join(mainTarDir, "bin")}/`;
// Create tarball
await mkdir(join(packagesDir, "test-native-binlink"), { recursive: true });
await $`cd ${mainPkgDir} && tar -czf ${join(packagesDir, "test-native-binlink", "test-native-binlink-1.0.0.tgz")} package`;
// Platform-specific package
const targetPkgDir = join(packagesDir, "test-native-binlink-target-tmp");
await mkdir(targetPkgDir, { recursive: true });
await mkdir(join(targetPkgDir, "bin"), { recursive: true });
await writeFile(
join(targetPkgDir, "package.json"),
JSON.stringify(
{
name: "test-native-binlink-target",
version: "1.0.0",
os: ["darwin", "linux", "win32"],
cpu: ["arm64", "x64"],
},
null,
2,
),
);
// Use the SAME filename as the main package!
await writeFile(
join(targetPkgDir, "bin", "main.js"),
`#!/usr/bin/env node
console.log("SUCCESS: Using platform-specific bin (test-native-binlink-target)");
process.exit(0);
`,
);
// Create package structure for tarball
const targetTarDir = join(targetPkgDir, "package");
await mkdir(targetTarDir, { recursive: true });
await mkdir(join(targetTarDir, "bin"), { recursive: true });
await $`cp ${join(targetPkgDir, "package.json")} ${targetTarDir}/`;
await $`cp ${join(targetPkgDir, "bin", "main.js")} ${join(targetTarDir, "bin")}/`;
// Create tarball
await mkdir(join(packagesDir, "test-native-binlink-target"), { recursive: true });
await $`cd ${targetPkgDir} && tar -czf ${join(packagesDir, "test-native-binlink-target", "test-native-binlink-target-1.0.0.tgz")} package`;
// Create package.json for verdaccio registry with proper integrity hashes
for (const pkgName of ["test-native-binlink", "test-native-binlink-target"]) {
const version = "1.0.0";
const tarballName = `${pkgName}-${version}.tgz`;
const tarballPath = join(packagesDir, pkgName, tarballName);
// Calculate SHA512 integrity hash
const tarballFile = Bun.file(tarballPath);
const tarballBytes = await tarballFile.arrayBuffer();
const hash = new Bun.CryptoHasher("sha512");
hash.update(tarballBytes);
const integrity = `sha512-${Buffer.from(hash.digest()).toString("base64")}`;
// Calculate SHA1 shasum
const sha1Hash = new Bun.CryptoHasher("sha1");
sha1Hash.update(tarballBytes);
const shasum = Buffer.from(sha1Hash.digest()).toString("hex");
await writeFile(
join(packagesDir, pkgName, "package.json"),
JSON.stringify(
{
_id: pkgName,
name: pkgName,
"dist-tags": {
latest: version,
},
versions: {
[version]: {
name: pkgName,
version,
_id: `${pkgName}@${version}`,
bin: pkgName === "test-native-binlink" ? { "test-binlink-cmd": "./bin/main.js" } : undefined,
optionalDependencies:
pkgName === "test-native-binlink"
? {
"test-native-binlink-target": "1.0.0",
}
: undefined,
os: pkgName === "test-native-binlink-target" ? ["darwin", "linux", "win32"] : undefined,
cpu: pkgName === "test-native-binlink-target" ? ["arm64", "x64"] : undefined,
dist: {
integrity,
shasum,
tarball: `http://localhost:4873/${pkgName}/-/${tarballName}`,
},
},
},
},
null,
2,
),
);
}
// Clean up temp directories
await $`rm -rf ${mainPkgDir}`;
await $`rm -rf ${targetPkgDir}`;
console.log("✅ Created native binlink test packages");