Files
bun.sh/docs/cli/bun-install.md
Dylan Conway 351e47355a add default trustedDependencies and run lifecycle scripts during installation (#7132)
* [install] allow parallel execution of `postinstall` scripts

- fixes spurious exit code 42 from `spawn()`

* postinstall to a pipe

* feat(install): include top 500 packages as defaults for postinstall

* windows newline handling*

*i did not test it

* stuff

* cool

* a

* fix merge

* set `has_trusted_dependencies`

* fix a bunch of tests

* fix merge

* remove `PackageManager`

* remove commented code

* change to function

* Update lockfile.zig

* run scripts if added to `trustedDependencies` after install

* packages without `resolved` properties

* node-gyp scripts

* node-gyp script in the root

* another test

* git deps run prepare scripts

* fix merge

* run lifecycle scripts during installation

* Update lockfile.zig

* always increment

* 🏗️

* update tests

* tickWIthoutIdle

* const uws

* loop forwards through trees

* single buffer bitset list

* tag.isGit

* windows path separators

* `bun.sys.read` and enable/disable buffering

* fix test and waiter thread

* waiter thread and tests

* Update bun-install-registry.test.ts

* workspace exclude `preprepare` and `postprepare`

* Create esbuild.test.ts

* make sure length is the same

* remove deferred binlinks, add estrella test

* test with another version

* address some comments

* remove .verdaccio-db.json

* ooops

* fix build

* use `pid` to wait

* dont register pid_poll when using waiter thread

* stress test

* free

* fix failing tests

* fix linux crash, snapshot stress test

* oops

* concurrent scripts

* activate as soon as possible

* test

* delete stress test packages

* remove unused packages

* comment stress test and maybe fix segfault

* delete snapshot

* fix assertion

* use cpu_count * 2 for default concurrent scripts

* gear emoji

* add --concurrent-scripts to docs

* more docs

---------

Co-authored-by: alexlamsl <alexlamsl@gmail.com>
Co-authored-by: dave caruso <me@paperdave.net>
Co-authored-by: Dylan Conway <33744874+MilesWright7@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2023-12-11 22:08:25 -08:00

256 lines
10 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
### `bun install`
bun install is a fast package manager & npm client.
bun install can be configured via `bunfig.toml`, environment variables, and CLI flags.
#### Configuring `bun install` with `bunfig.toml`
`bunfig.toml` is searched for in the following paths on `bun install`, `bun remove`, and `bun add`:
1. `$XDG_CONFIG_HOME/.bunfig.toml` or `$HOME/.bunfig.toml`
2. `./bunfig.toml`
If both are found, the results are merged together.
Configuring with `bunfig.toml` is optional. Bun tries to be zero configuration in general, but that's not always possible.
```toml
# Using scoped packages with bun install
[install.scopes]
# Scope name The value can be a URL string or an object
"@mybigcompany" = { token = "123456", url = "https://registry.mybigcompany.com" }
# URL is optional and falls back to the default registry
# The "@" in the scope is optional
mybigcompany2 = { token = "123456" }
# Environment variables can be referenced as a string that starts with $ and it will be replaced
mybigcompany3 = { token = "$npm_config_token" }
# Setting username and password turns it into a Basic Auth header by taking base64("username:password")
mybigcompany4 = { username = "myusername", password = "$npm_config_password", url = "https://registry.yarnpkg.com/" }
# You can set username and password in the registry URL. This is the same as above.
mybigcompany5 = "https://username:password@registry.yarnpkg.com/"
# You can set a token for a registry URL:
mybigcompany6 = "https://:$NPM_CONFIG_TOKEN@registry.yarnpkg.com/"
[install]
# Default registry
# can be a URL string or an object
registry = "https://registry.yarnpkg.com/"
# as an object
#registry = { url = "https://registry.yarnpkg.com/", token = "123456" }
# Install for production? This is the equivalent to the "--production" CLI argument
production = false
# Disallow changes to lockfile? This is the equivalent to the "--frozen-lockfile" CLI argument
frozenLockfile = false
# Don't actually install
dryRun = true
# Install optionalDependencies (default: true)
optional = true
# Install local devDependencies (default: true)
dev = true
# Install peerDependencies (default: true)
peer = true
# Max number of concurrent lifecycle scripts (default: (cpu count or GOMAXPROCS) x2)
concurrentScripts = 16
# When using `bun install -g`, install packages here
globalDir = "~/.bun/install/global"
# When using `bun install -g`, link package bins here
globalBinDir = "~/.bun/bin"
# cache-related configuration
[install.cache]
# The directory to use for the cache
dir = "~/.bun/install/cache"
# Don't load from the global cache.
# Note: Bun may still write to node_modules/.cache
disable = false
# Always resolve the latest versions from the registry
disableManifest = false
# Lockfile-related configuration
[install.lockfile]
# Print a yarn v1 lockfile
# Note: it does not load the lockfile, it just converts bun.lockb into a yarn.lock
print = "yarn"
# Save the lockfile to disk
save = true
```
If it's easier to read as TypeScript types:
```ts
export interface Root {
install: Install;
}
export interface Install {
scopes: Scopes;
registry: Registry;
production: boolean;
frozenLockfile: boolean;
dryRun: boolean;
optional: boolean;
dev: boolean;
peer: boolean;
globalDir: string;
globalBinDir: string;
cache: Cache;
lockfile: Lockfile;
logLevel: "debug" | "error" | "warn";
}
type Registry =
| string
| {
url?: string;
token?: string;
username?: string;
password?: string;
};
type Scopes = Record<string, Registry>;
export interface Cache {
dir: string;
disable: boolean;
disableManifest: boolean;
}
export interface Lockfile {
print?: "yarn";
save: boolean;
}
```
## Configuring with environment variables
Environment variables have a higher priority than `bunfig.toml`.
| Name | Description |
| -------------------------------- | ------------------------------------------------------------- |
| BUN_CONFIG_REGISTRY | Set an npm registry (default: <https://registry.npmjs.org>) |
| BUN_CONFIG_TOKEN | Set an auth token (currently does nothing) |
| BUN_CONFIG_YARN_LOCKFILE | Save a Yarn v1-style yarn.lock |
| BUN_CONFIG_LINK_NATIVE_BINS | Point `bin` in package.json to a platform-specific dependency |
| BUN_CONFIG_SKIP_SAVE_LOCKFILE | Dont save a lockfile |
| BUN_CONFIG_SKIP_LOAD_LOCKFILE | Dont load a lockfile |
| BUN_CONFIG_SKIP_INSTALL_PACKAGES | Dont install any packages |
Bun always tries to use the fastest available installation method for the target platform. On macOS, thats `clonefile` and on Linux, thats `hardlink`. You can change which installation method is used with the `--backend` flag. When unavailable or on error, `clonefile` and `hardlink` fallsback to a platform-specific implementation of copying files.
Bun stores installed packages from npm in `~/.bun/install/cache/${name}@${version}`. Note that if the semver version has a `build` or a `pre` tag, it is replaced with a hash of that value instead. This is to reduce the chances of errors from long file paths, but unfortunately complicates figuring out where a package was installed on disk.
When the `node_modules` folder exists, before installing, Bun checks if the `"name"` and `"version"` in `package/package.json` in the expected node_modules folder matches the expected `name` and `version`. This is how it determines whether it should install. It uses a custom JSON parser which stops parsing as soon as it finds `"name"` and `"version"`.
When a `bun.lockb` doesnt exist or `package.json` has changed dependencies, tarballs are downloaded & extracted eagerly while resolving.
When a `bun.lockb` exists and `package.json` hasnt changed, Bun downloads missing dependencies lazily. If the package with a matching `name` & `version` already exists in the expected location within `node_modules`, Bun wont attempt to download the tarball.
## Platform-specific dependencies?
bun stores normalized `cpu` and `os` values from npm in the lockfile, along with the resolved packages. It skips downloading, extracting, and installing packages disabled for the current target at runtime. This means the lockfile wont change between platforms/architectures even if the packages ultimately installed do change.
## Peer dependencies?
Peer dependencies are handled similarly to yarn. `bun install` will automatically install peer dependencies. If the dependency is marked optional in `peerDependenciesMeta`, an existing dependency will be chosen if possible.
## Lockfile
`bun.lockb` is Buns binary lockfile format.
## Why is it binary?
In a word: Performance. Buns lockfile saves & loads incredibly quickly, and saves a lot more data than what is typically inside lockfiles.
## How do I inspect it?
For now, the easiest thing is to run `bun install -y`. That prints a Yarn v1-style yarn.lock file.
## What does the lockfile store?
Packages, metadata for those packages, the hoisted install order, dependencies for each package, what packages those dependencies resolved to, an integrity hash (if available), what each package was resolved to and which version (or equivalent).
## Why is it fast?
It uses linear arrays for all data. [Packages](https://github.com/oven-sh/bun/blob/be03fc273a487ac402f19ad897778d74b6d72963/src/install/install.zig#L1825) are referenced by an auto-incrementing integer ID or a hash of the package name. Strings longer than 8 characters are de-duplicated. Prior to saving on disk, the lockfile is garbage-collected & made deterministic by walking the package tree and cloning the packages in dependency order.
## Cache
To delete the cache:
```bash
$ rm -rf ~/.bun/install/cache
```
## Platform-specific backends
`bun install` uses different system calls to install dependencies depending on the platform. This is a performance optimization. You can force a specific backend with the `--backend` flag.
**`hardlink`** is the default backend on Linux. Benchmarking showed it to be the fastest on Linux.
```bash
$ rm -rf node_modules
$ bun install --backend hardlink
```
**`clonefile`** is the default backend on macOS. Benchmarking showed it to be the fastest on macOS. It is only available on macOS.
```bash
$ rm -rf node_modules
$ bun install --backend clonefile
```
**`clonefile_each_dir`** is similar to `clonefile`, except it clones each file individually per directory. It is only available on macOS and tends to perform slower than `clonefile`. Unlike `clonefile`, this does not recursively clone subdirectories in one system call.
```bash
$ rm -rf node_modules
$ bun install --backend clonefile_each_dir
```
**`copyfile`** is the fallback used when any of the above fail, and is the slowest. on macOS, it uses `fcopyfile()` and on linux it uses `copy_file_range()`.
```bash
$ rm -rf node_modules
$ bun install --backend copyfile
```
**`symlink`** is typically only used for `file:` dependencies (and eventually `link:`) internally. To prevent infinite loops, it skips symlinking the `node_modules` folder.
If you install with `--backend=symlink`, Node.js won't resolve node_modules of dependencies unless each dependency has its own node_modules folder or you pass `--preserve-symlinks` to `node`. See [Node.js documentation on `--preserve-symlinks`](https://nodejs.org/api/cli.html#--preserve-symlinks).
```bash
$ rm -rf node_modules
$ bun install --backend symlink
$ node --preserve-symlinks ./my-file.js # https://nodejs.org/api/cli.html#--preserve-symlinks
```
Bun's runtime does not currently expose an equivalent of `--preserve-symlinks`, though the code for it does exist.
## npm registry metadata
bun uses a binary format for caching NPM registry responses. This loads much faster than JSON and tends to be smaller on disk.
You will see these files in `~/.bun/install/cache/*.npm`. The filename pattern is `${hash(packageName)}.npm`. Its a hash so that extra directories dont need to be created for scoped packages.
Bun's usage of `Cache-Control` ignores `Age`. This improves performance, but means bun may be about 5 minutes out of date to receive the latest package version metadata from npm.