### What does this PR do?
It's common for monorepos to exclude portions of a large glob
```json
"workspaces": [
"packages/**",
"!packages/**/test/**",
"!packages/**/template/**"
],
```
closes#4621 (note: patterns like `"packages/!(*-standalone)"` will need
to be written `"!packages/*-standalone"`)
### How did you verify your code works?
Manually tested https://github.com/opentiny/tiny-engine, and added a new
workspace test.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* Make it work with abs patterns
* add tests
* fixes
* Make all the ported fast-glob tests work for absolute path patterns, fix some stuff
* fix scan test
* remove TestBuilder in scan.test.ts
---------
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
* Skeleton code for `bun run --workspace`
* Update run_command.zig
* implement directory traversal to find workspace root
* finish --workspace implementation
* clean up changes in run_command.zig
* add workspace tests, update harness to handle nested dirs
* [autofix.ci] apply automated fixes
* basic filtering
* [autofix.ci] apply automated fixes
* working filter without patterns
* update tests, filter mostly working
* simplify package name parsing, commit tests
* support filter even without workspace setup
* move filter arg handling to separate source file
* use bun.sys.chdir, match root package for scripts
* fix exit code handling
* ignore node_modules and directories starting with . in --filter
* progress converting --filter to use iterators
* convert filtering to use iterators
* cleanup
* implement DirEntry access method for glob (currently crashing)
* cleanup and fixes
* run js files in subprocess when filter flag passed
* clean up dead code
* fix fd leak in run_command.zig
* [autofix.ci] apply automated fixes
* fix issues after merge
* use posix-spawn in runBinary, fix resource PATH variable resource leak
* move filter argument to runtime category
* fix test harness
* add js and binary tests to filter-workspace
* [autofix.ci] apply automated fixes
* fix compile after merge
* [autofix.ci] apply automated fixes
* clean up filter-workspace test
* [autofix.ci] apply automated fixes
* fixes to running binaries
* fix actually setting cwd_override
* windows fixes
* address some review comments
* handle malformed JSON
* add various tests
* [autofix.ci] apply automated fixes
* update docs for filter
* [autofix.ci] apply automated fixes
* reset tinycc commit
* filtered run prototype
* make pretty
* implement abort handler (not working)
* make prettier
* prep for windows
* windows path and printing fixes
* implement log-style output (not tui)
* fix issues when logging to file
* revert a bunch of unecessary changes
* cleanup
* implement dependency order execution
* detect circular dependencies, fix cancel hang
* Fix `$PATH`
* ignore dep order on loop, stream on linux, sort pkgs
* support pre and post scripts
* add more filter tests, print elapsed time
* enable 'bun --filter' without run
* fix harness after merge
* [autofix.ci] apply automated fixes
* print number of scripts we're waiting for
* update docs, fix windows build
* fix tests on windows
* [autofix.ci] apply automated fixes
* fix uninitialized memory
* use terminal synchronized update sequences
* Add skip list
* Preallocate
* Use current bun in tests
---------
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
* bring in shell impl
* add `$` to global bun scope
* Expose shell parse func on debug mode
* Expose lex tokens and add tests
* refactor parser to have better ast
* assigns and export
* pipeline kinda working
* Decouple Subprocess spawning code from JS stuff
* Subprocess works!
* Conditional execution
* Support JS objects in template expression
* More complete redirection
* Properly drain stdin/stdout/stderr and fix crash from deallocating JSC memory
* Return errors in parser
* Support command substitution
* wip brace expansion stuff
* Rearrange some files
* expansion wip
* Brace expansion working
* wip brace expansion
* refactor brace algorithm
* wip brace expansion on shell
* fix brace expansion
* Working nested brace expansion!
* brace expansion in shell variable assignment only set the last expanded
* stuff
* Small little perf things
* benchmark and test and stuff
* stuff
* fix nested braces but its also kinda broken
* attempt to fix complicated nested braces
* test
* Use fast tokenized algorithm for non nested braces, use parsed algorithm for nested braces
* fix nested braces one and for all
* small stuff
* Not sure if that made a difference
* revert that
* good speed optimization
* rip
* Environment variables, builtin/native shell cmds
* Fix tests
* Support `cd`, `pwd`, add boilerplate for glob expansion
* Support `which`
* Support `rm`
* wip
* wip
* escaping and abstract shell char iterator
* strpool unicode
* Brace expansion support unicode, disallow invalid surrogates in shell script
* shell choose ascii or unicode lexer depending on input
* fix bugs write tests
* kinda start async stuff
* HOLY SHIT big refactor of Subprocess
woops forget to commit this
...and this
* HOLY MOLY it works
* Refactor some stuff, start eval word expansion
* interpret all the nodes
* stuff
* stuff
* stuff
* kind of works but doesnt
* Buffered output works
* no need to heap allocate autosizer
* Fix bug
* Fix some stuff
* unprotect
* move out dummy shell thing
* Bring back assignments
* create expansion state so it can be non blocking for expansions that need IO (glob, cmd subst)
* glob back in action
* Setup builtin non blocking IO commands and implement export
big issue is control flow is really fcked up here need to fix that
* make Cmd state machine use a loop so control flow is a bit more clear
* rename stuff
* move that
* Implement the echo builtin again
* implement cd again but non blocking io
* Fix ls and use proper write function to prevent blockign writes
* Implement which
* holy moly big port std.fs.deleteTree
* fix compile errors
* Okay that works
* rm works thatsnice
* damn
* split it out
* rm async implementation
* fix rm bug for nested
* Work on files as well
* prevent root from being deleted
* rm error handling
* oops
* pwd and fix some script exec bug
* Implement `mv`
* stub out mv to work accross filesystems
* move it around
* woops
* boilerplate for ls and options
* more boilerplate
* stuff
* that got lost in merge
* upgrade shell stuff zig 0.12.0-dev.1828+225fe6ddb
* Implement basic ls
* smol cleanup
* Fix stream, response redirect stdin
* No longer need spawn to be abstract
* Custom promise
* move around some stuff
* shell promise returns shell output
* make tht work for builtins
* refactor IO abstractions to work with JS or mini event loop
* woops
* scaffolding for refactor
* refactor builtins to make event loop refactor easier
* Fix parsing edge case on assignments, fix expansion on cmd assignments
* change subproc to work with any event loop
* Finish refactoring subproc
* move global abstraction out
* big refactor boys
* holy moly: integrate into cli and fix allll the compile errors
* okay works in bun run now
* actually tick the event loop lol
* Fix more stuff
* Support comments
* Fix some tests
* delete that
* Properly report errors when failing to spawn command
* fix a whole bunch of tests
* fix a whole bunch of tests again
* .
* Fix rm
* Fix some exit code bugs, write force rm from deno, fix ls stderr
* fix `rm -d`
* fix `rm -d`
* Fix boolean logic
* error on subshells (e.g. `true && (echo hi && echo lol)`)
* Move out shell state from interpreter struct
* Cmd substitution supports arbitrary script, not just a single cmd
* Some escaping/quotation tests
* Fix stuff add more tests:
- cmd substitution quotations
- escape backticks in single quots
* ALOT of stuff:
- fix proper subshell inheritance of env for cmd subst
- fix: was wrong, assignments don't run in subshell in conditionals
- fix lexing chained vars `$VAR$VAR`
- more tests
* Fix subtle bugs
* Fix crazy redirect to arraybuffer bug
* more crazy echo edgecases
* Proper lexer errors instead of just panicking lol
* yoops
* Proper parsing errors
* Errors for bun run shell script
* Fix redirecting to file
* More test fix bugs yay
* Fix redirect on builtins
* Open redirection fds with O_TRUNC
* Fix lexing invalid variables and add ability to change cwd from JS api
* yoops
* Fix `.cwd()`
* `$PWD` and fix redirection bugs
* `$PWD` and fix redirection bugs
* Get rid of some `FIXME`s
* throw errors in some places instead of panicking
* Print some errors to stderr
* Get rid of some more panics again
* Handle errors on glob
* pwd test
* `.env()`
* copy-on-write abstraction
* Reference counted env strings + fix some tests
* deinit cwd
* Put commands into a pipeline properly
* deinit Expansion and Assigns properly
* comments
* Comments
* Make it compile
* Update types
* [autofix.ci] apply automated fixes
* Only one WaiterThread
* Fix lifetimes and clean up interface
* Update shell.ts
* Add lazy test
* Remove some dead code
* Update shell.zig
* Fix memory leak
* Fix crash with empty braces
* [autofix.ci] apply automated fixes
* Linux build + bun.sh
* Update subproc.zig
* Update interpreter.zig
* Update interpreter.zig
* Fix some stuff that broke
* Fix Windows compile errors
* Fix some fd leaks
* Fix ls
* Fix a bunch of stuff
* Fix quiet
* Update leak tests fix rm bug
* More reproducible tests
* [autofix.ci] apply automated fixes
* more mem leak tests
* [autofix.ci] apply automated fixes
* Fix merge conflict
* Fix test not actually using temp directory
* Update bunshell.test.ts
* Shell instance
* Capture async context
* Increase test timeouts
* [autofix.ci] apply automated fixes
* Escape
* [autofix.ci] apply automated fixes
* Fix crash
* Add more methods
* [autofix.ci] apply automated fixes
* Fix leak
* Treat file(path) blobs as a file path string
* Create bunshell-file.test.ts
* Support Blob input
* Fix leak + organize imports
* doc
* Update shell.md
* Update shell.md
* Update shell.md
* Update shell.md
* Update CMakeLists.txt
* Fix segfault by cloning error path so it's not freed by arena
* deinit ShellErr
* Delete dead code
* fix really stupid segfault
* don't deinit shell ls task in event loop
* Fix ls bug
* Fix tests
* make truly lazy
* allow more things in the shell substitution and escape whitespace
* Fix newline and exit when finishing shell in `bun run`
---------
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* all: make bun.FileDescriptor a unique type
# Conflicts:
# src/bun.js/api/bun/socket.zig
# src/bun.js/api/bun/spawn.zig
# src/deps/uws.zig
* need this change too
* convert another bun.FD.asDir call
* more fixes + linux
* fix this size
* accidentally flipped these fd numbers
* more fixes + windows
# Conflicts:
# src/deps/uws.zig
* dont include type name in serialization
* add another TODO
* resolve review comments
* address more pr comments
* build fixes post-rebase
* another rebase fix
---------
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
* chore: upgrade zig to 0.12.0-dev.1828+225fe6ddb
* open as iterable
* fix building identifier cache
* fix windows build
* fix linux build
* fix linux build
* idk why that was left out
* No need for stat call, add text for broken symlinks
* tests for crashes
* Fix test
* Add some checks to limit max open file descriptors
* Update glob.zig
* Update glob.zig
* Update scan.test.ts
---------
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
* initial glob impl
* Add `Bun.globMatch`
* Glob boilerplate setup
* Experiment with Rust glob implementation
* Rust impl is slow revert
* Setup glob walking
* Basic glob walker working
* Fix segfault
* Recursive directory traversal
* Fix glob match non-ascii
* Make faster lil bit
* use arena
* ASCII fast path
* Experiment with packed codepoint cursor
Results in ~4% perf boost if the glob pattern needs to create/manipulate cursors (for example when the pattern uses braces)
* Try converting to u32 array
Made it pretty slow
* Lazily create codepoint buffer
* Different walk algorithm
* Fast path optimizations
* Add `dot` option to `Glob`
* .
* Fix some bugs
* Fix bug, clean up lil bit
* Windows fix
* Non absolute paths
* use specific version of fast-glob for benchmarks and tests
* .
* Fix some stuff
* Fix more stuff
* Add `hasPendingActivity()` to glob
* accident
* Symlinks
* fast-glob e2e tests
* remove
* woops
* Fix relative paths
* Fix absolute
* add test for `onlyFiles`
* Fix invalid surrogate pairs problem
* Rename: `match/matchSync` -> `scan/scanSync` and `matchString` -> `match`
* forgot to close cwd fd
* Update types
* Add stress test
* Port `micromatch` / `glob-match` / `globlin` tests
* fix stale reference arena thing
* stupid bug
* Add builtins to classes code generator and add `Glob.scanIter()`
* all iterables
* generate fixtures, remove from git
* fix test
* Fix
* woops on test
* Fix stuff
licenses
license
`has_pending_activity` to usize
cwd threadSafe fix atomic compile errors
`GlobWalker` own `cwd`
Fix windows path and absolute test
stuff
* Fixes
* Fix stuff
* Use Syscall.close
* Use private symbols for underlying scan functions to preevent misuse
* Update types
* Fix build for zig
* Fix tests
* Fix more tests
* Prevent these tests from GC'ing too much
* Make this benchmark work in Node and Bun
* Fix memory leak
* Add leak test
* Fix windows
* comment about arena allocator use for glob walker
* Make leak test run in separate process
* Iterator api for glob
* GlobWalker.Iterator
* fix leak test
* Remove old impl
* filter functions wip start
* stuff
* wip lockfile use glob
* glob working with lockfile
* revert lockfile changes
* Update bun.lockb
* Manually set to cwd to prevent test failing on linux CI
---------
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
