Files
bun.sh/docs/runtime/shell.md
Zack Radisic 1b1760a9c9 feat: Bun shell (#7748)
* bring in shell impl

* add `$` to global bun scope

* Expose shell parse func on debug mode

* Expose lex tokens and add tests

* refactor parser to have better ast

* assigns and export

* pipeline kinda working

* Decouple Subprocess spawning code from JS stuff

* Subprocess works!

* Conditional execution

* Support JS objects in template expression

* More complete redirection

* Properly drain stdin/stdout/stderr and fix crash from deallocating JSC memory

* Return errors in parser

* Support command substitution

* wip brace expansion stuff

* Rearrange some files

* expansion wip

* Brace expansion working

* wip brace expansion

* refactor brace algorithm

* wip brace expansion on shell

* fix brace expansion

* Working nested brace expansion!

* brace expansion in shell variable assignment only set the last expanded

* stuff

* Small little perf things

* benchmark and test and stuff

* stuff

* fix nested braces but its also kinda broken

* attempt to fix complicated nested braces

* test

* Use fast tokenized algorithm for non nested braces, use parsed algorithm for nested braces

* fix nested braces one and for all

* small stuff

* Not sure if that made a difference

* revert that

* good speed optimization

* rip

* Environment variables, builtin/native shell cmds

* Fix tests

* Support `cd`, `pwd`, add boilerplate for glob expansion

* Support `which`

* Support `rm`

* wip

* wip

* escaping and abstract shell char iterator

* strpool unicode

* Brace expansion support unicode, disallow invalid surrogates in shell script

* shell choose ascii or unicode lexer depending on input

* fix bugs write tests

* kinda start async stuff

* HOLY SHIT big refactor of Subprocess

woops forget to commit this

...and this

* HOLY MOLY it works

* Refactor some stuff, start eval word expansion

* interpret all the nodes

* stuff

* stuff

* stuff

* kind of works but doesnt

* Buffered output works

* no need to heap allocate autosizer

* Fix bug

* Fix some stuff

* unprotect

* move out dummy shell thing

* Bring back assignments

* create expansion state so it can be non blocking for expansions that need IO (glob, cmd subst)

* glob back in action

* Setup builtin non blocking IO commands and implement export

big issue is control flow is really fcked up here need to fix that

* make Cmd state machine use a loop so control flow is a bit more clear

* rename stuff

* move that

* Implement the echo builtin again

* implement cd again but non blocking io

* Fix ls and use proper write function to prevent blockign writes

* Implement which

* holy moly big port std.fs.deleteTree

* fix compile errors

* Okay that works

* rm works thatsnice

* damn

* split it out

* rm async implementation

* fix rm bug for nested

* Work on files as well

* prevent root from being deleted

* rm error handling

* oops

* pwd and fix some script exec bug

* Implement `mv`

* stub out mv to work accross filesystems

* move it around

* woops

* boilerplate for ls and options

* more boilerplate

* stuff

* that got lost in merge

* upgrade shell stuff zig 0.12.0-dev.1828+225fe6ddb

* Implement basic ls

* smol cleanup

* Fix stream, response redirect stdin

* No longer need spawn to be abstract

* Custom promise

* move around some stuff

* shell promise returns shell output

* make tht work for builtins

* refactor IO abstractions to work with JS or mini event loop

* woops

* scaffolding for refactor

* refactor builtins to make event loop refactor easier

* Fix parsing edge case on assignments, fix expansion on cmd assignments

* change subproc to work with any event loop

* Finish refactoring subproc

* move global abstraction out

* big refactor boys

* holy moly: integrate into cli and fix allll the compile errors

* okay works in bun run now

* actually tick the event loop lol

* Fix more stuff

* Support comments

* Fix some tests

* delete that

* Properly report errors when failing to spawn command

* fix a whole bunch of tests

* fix a whole bunch of tests again

* .

* Fix rm

* Fix some exit code bugs, write force rm from deno, fix ls stderr

* fix `rm -d`

* fix `rm -d`

* Fix boolean logic

* error on subshells (e.g. `true && (echo hi && echo lol)`)

* Move out shell state from interpreter struct

* Cmd substitution supports arbitrary script, not just a single cmd

* Some escaping/quotation tests

* Fix stuff add more tests:

- cmd substitution quotations
- escape backticks in single quots

* ALOT of stuff:

- fix proper subshell inheritance of env for cmd subst
- fix: was wrong, assignments don't run in subshell in conditionals
- fix lexing chained vars `$VAR$VAR`
- more tests

* Fix subtle bugs

* Fix crazy redirect to arraybuffer bug

* more crazy echo edgecases

* Proper lexer errors instead of just panicking lol

* yoops

* Proper parsing errors

* Errors for bun run shell script

* Fix redirecting to file

* More test fix bugs yay

* Fix redirect on builtins

* Open redirection fds with O_TRUNC

* Fix lexing invalid variables and add ability to change cwd from JS api

* yoops

* Fix `.cwd()`

* `$PWD` and fix redirection bugs

* `$PWD` and fix redirection bugs

* Get rid of  some `FIXME`s

* throw errors in some places instead of panicking

* Print some errors to stderr

* Get rid of some more panics again

* Handle errors on glob

* pwd test

* `.env()`

* copy-on-write abstraction

* Reference counted env strings + fix some tests

* deinit cwd

* Put commands into a pipeline properly

* deinit Expansion and Assigns properly

* comments

* Comments

* Make it compile

* Update types

* [autofix.ci] apply automated fixes

* Only one WaiterThread

* Fix lifetimes and clean up interface

* Update shell.ts

* Add lazy test

* Remove some dead code

* Update shell.zig

* Fix memory leak

* Fix crash with empty braces

* [autofix.ci] apply automated fixes

* Linux build + bun.sh

* Update subproc.zig

* Update interpreter.zig

* Update interpreter.zig

* Fix some stuff that broke

* Fix Windows compile errors

* Fix some fd leaks

* Fix ls

* Fix a bunch of stuff

* Fix quiet

* Update leak tests fix rm bug

* More reproducible tests

* [autofix.ci] apply automated fixes

* more mem leak tests

* [autofix.ci] apply automated fixes

* Fix merge conflict

* Fix test not actually using temp directory

* Update bunshell.test.ts

* Shell instance

* Capture async context

* Increase test timeouts

* [autofix.ci] apply automated fixes

* Escape

* [autofix.ci] apply automated fixes

* Fix crash

* Add more methods

* [autofix.ci] apply automated fixes

* Fix leak

* Treat file(path) blobs as a file path string

* Create bunshell-file.test.ts

* Support Blob input

* Fix leak + organize imports

* doc

* Update shell.md

* Update shell.md

* Update shell.md

* Update shell.md

* Update CMakeLists.txt

* Fix segfault by cloning error path so it's not freed by arena

* deinit ShellErr

* Delete dead code

* fix really stupid segfault

* don't deinit shell ls task in event loop

* Fix ls bug

* Fix tests

* make truly lazy

* allow more things in the shell substitution and escape whitespace

* Fix newline and exit when finishing shell in `bun run`

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2024-01-19 17:30:57 -08:00

8.9 KiB

Bun Shell makes shell scripting with JavaScript & TypeScript fun. It's a cross-platform bash-like shell with seamless JavaScript interop.

{% callout type="note" %} Alpha-quality software: Bun Shell is an unstable API still under development. If you have feature requests or run into bugs, please open an issue. There may be breaking changes in the future. {% /callout %}

Quickstart:

import { $ } from "bun";

const response = await fetch("https://example.com");
const buffer = Buffer.alloc(100);

// Use Response as stdin.
await $`echo < ${response} > wc -c`; // 120

Features:

  • Cross-platform: works on Windows, Linux & macOS. Instead of rimraf or cross-env', you can use Bun Shell without installing extra dependencies. Common shell commands like ls, cd, rm are implemented natively.
  • Familiar: Bun Shell is a bash-like shell, supporting redirection, pipes, environment variables and more.
  • Globs: Glob patterns are supported natively, including **, *, {expansion}, and more.
  • Template literals: Template literals are used to execute shell commands. This allows for easy interpolation of variables and expressions.
  • Safety: Bun Shell escapes all strings by default, preventing shell injection attacks.
  • JavaScript interop: Use Response, ArrayBuffer, Blob, Bun.file(path) and other JavaScript objects as stdin, stdout, and stderr.

Getting started

The simplest shell command is echo. To run it, use the $ template literal tag:

import { $ } from "bun";

await $`echo "Hello World!"`; // Hello World!

By default, shell commands print to stdout. To quiet the output, call .quiet():

import { $ } from "bun";

await $`echo "Hello World!"`.quiet(); // No output

What if you want to access the output of the command as text? Use .text():

import { $ } from "bun";

// .text() automatically calls .quiet() for you
const welcome = await $`echo "Hello World!"`.text();

console.log(welcome); // Hello World!\n

To get stdout, stderr, and the exit code, use await or .run:

import { $ } from "bun";

const { stdout, stderr, exitCode } = await $`echo "Hello World!"`.quiet();

console.log(stdout); // Buffer(6) [ 72, 101, 108, 108, 111, 32 ]
console.log(stderr); // Buffer(0) []
console.log(exitCode); // 0

Redirection

Bun Shell supports redirection with <, >, and | operators.

To JavaScript objects (>)

To redirect stdout to a JavaScript object, use the > operator:

import { $ } from "bun";

const buffer = Buffer.alloc(100);
const result = await $`echo "Hello World!" > ${buffer}`;

console.log(result.exitCode); // 0
console.log(buffer.toString()); // Hello World!\n

The following JavaScript objects are supported for redirection to:

  • Buffer, Uint8Array, Uint16Array, Uint32Array, Int8Array, Int16Array, Int32Array, Float32Array, Float64Array, ArrayBuffer, SharedArrayBuffer (writes to the underlying buffer)
  • Bun.file(path), Bun.file(fd) (writes to the file)

From JavaScript objects (<)

To redirect the output from JavaScript objects to stdin, use the < operator:

import { $, file } from "bun";

const response = new Response("hello i am a response body");

const result = await $`cat < ${response}`.text();

console.log(result); // hello i am a response body

The following JavaScript objects are supported for redirection from:

  • Buffer, Uint8Array, Uint16Array, Uint32Array, Int8Array, Int16Array, Int32Array, Float32Array, Float64Array, ArrayBuffer, SharedArrayBuffer (reads from the underlying buffer)
  • Bun.file(path), Bun.file(fd) (reads from the file)
  • Response (reads from the body)

Piping (|)

Like in bash, you can pipe the output of one command to another:

import { $ } from "bun";

const result = await $`echo "Hello World!" | wc -w`.text();

console.log(result); // 2\n

You can also pipe with JavaScript objects:

import { $ } from "bun";

const response = new Response("hello i am a response body");

const result = await $`cat < ${response} | wc -w`.text();

console.log(result); // 6\n

Environment variables

Environment variables can be set like in bash:

import { $ } from "bun";

await $`FOO=foo bun -e 'console.log(process.env.FOO)'`; // foo\n

You can use string interpolation to set environment variables:

import { $ } from "bun";

const foo = "bar123";

await $`FOO=${foo + "456"} bun -e 'console.log(process.env.FOO)'`; // bar123456\n

Input is escaped by default, preventing shell injection attacks:

import { $ } from "bun";

const foo = "bar123; rm -rf /tmp";

await $`FOO=${foo} bun -e 'console.log(process.env.FOO)'`; // bar123; rm -rf /tmp\n

Changing the environment variables

By default, process.env is used as the environment variables for all commands.

You can change the environment variables for a single command by calling .env():

import { $ } from "bun";

await $`echo $FOO`.env({ ...process.env, FOO: "bar" }); // bar

You can change the default environment variables for all commands by calling $.env:

import { $ } from "bun";

$.env({ FOO: "bar" });

// the globally-set $FOO
await $`echo $FOO`; // bar

// the locally-set $FOO
await $`echo $FOO`.env({ FOO: "baz" }); // baz

You can reset the environment variables to the default by calling $.env() with no arguments:

import { $ } from "bun";

$.env({ FOO: "bar" });

// the globally-set $FOO
await $`echo $FOO`; // bar

// the locally-set $FOO
await $`echo $FOO`.env(undefined); // ""

Changing the working directory

You can change the working directory of a command by passing a string to .cwd():

import { $ } from "bun";

await $`pwd`.cwd("/tmp"); // /tmp

You can change the default working directory for all commands by calling $.cwd:

import { $ } from "bun";

$.cwd("/tmp");

// the globally-set working directory
await $`pwd`; // /tmp

// the locally-set working directory
await $`pwd`.cwd("/"); // /

Reading output

To read the output of a command as a string, use .text():

import { $ } from "bun";

const result = await $`echo "Hello World!"`.text();

console.log(result); // Hello World!\n

Reading output as JSON

To read the output of a command as JSON, use .json():

import { $ } from "bun";

const result = await $`echo '{"foo": "bar"}'`.json();

console.log(result); // { foo: "bar" }

Reading output line-by-line

To read the output of a command line-by-line, use .lines():

import { $ } from "bun";

for await (let line of $`echo "Hello World!"`.lines()) {
  console.log(line); // Hello World!
}

You can also use .lines() on a completed command:

import { $ } from "bun";

const search = "bun";
const iterator = await $`cat list.txt | grep ${search}`.lines();
if (iterator.exitCode !== 0) {
  throw new Error("oh no");
}
for await (let line of iterator) {
  console.log(line);
}

Reading output as a Blob

To read the output of a command as a Blob, use .blob():

import { $ } from "bun";

const result = await $`echo "Hello World!"`.blob();

console.log(result); // Blob(13) { size: 13, type: "text/plain" }

Builtin Commands

For cross-platform compatibility, Bun Shell implements a set of builtin commands, in addition to reading commands from the PATH environment variable.

  • cd: change the working directory
  • ls: list files in a directory
  • rm: remove files and directories
  • echo: print text
  • pwd: print the working directory
  • bun: run bun in bun

Partially implemented:

  • mv: move files and directories (missing cross-device support)

Not implemented yet, but planned:

  • mkdir: create directories
  • cp: copy files and directories
  • cat: concatenate files

Utilities

Bun Shell also implements a set of utilities for working with shells.

$.braces (brace expansion)

This function implements simple brace expansion for shell commands:

import { $ } from "bun";

await $.braces(`echo {1,2,3}`);
// => ["echo 1", "echo 2", "echo 3"]

$.raw (unescaped strings)

For security purposes, Bun Shell escapes input by default. If you need to disable that, this function returns a string that is not escaped by Bun Shell:

import { $ } from "bun";

await $`echo ${$.raw("Hello World!")}`;
// => Hello World!

Standalone usage

You can use Bun Shell to run simple shell script files on your computer.

To do that, run any file with bun that ends with .bun.sh:

$ echo "echo Hello World!" > script.bun.sh
$ bun ./script.bun.sh
> Hello World!

On Windows, Bun Shell is used automatically to run .sh files when using Bun:

$ echo "echo Hello World!" > script.sh
# On windows, .bun.sh is not needed, just .sh
$ bun ./script.sh
> Hello World!

Credits

Large parts of this API were inspired by zx and dax. Thank you to the authors of those projects.