* WIP sync close (shows ref count bug in stream)
* fix closing on PipeWriter and PipeReader
* remove old todos
* join
* Some shell changes
at least it compiles
* fix some compile errors
* fix ref/unref server on windows
* actually use the ref count in this places
* make windows compile again
* more tests passing
* Make shell compile again
* Slowly remove some `@panic("TODO SHELL")`
* Eliminate `@panic("TODO SHELL")` for BufferedWriter
* Holy cleansing of `@panic("TODO SHELL")`
at least it compiles now
* Okay now the shell compiles, but segfaults
* Fix compiler errors
* more stable stream and now Content-Range pass
* make windows compile again
* revert stuff until the fix is actually ready
* revert onDone thing
* Fix buffered writer for shell
* Fix buffered writer + shell/subproc.zig and windows build
* Fix for #8982 got lost in the merge
* Actually buffer subproc output
* Fix some stuff shell
* oops
* fix context deinit
* fix renderMissing
* shell: Fix array buffer
* more stable streams (#9053)
fix stream ref counting
* wip
* Remove `@panic("TODO")` on shell event loop tasks and Redirect open flags got lost in merge
* Support redirects
* fixes
cc @cirospaciari
* Update ReadableStreamInternals.ts
* Fix spurious error
* Update stream.js
* leak
* Fix UAF
cc @cirospaciari
* Fix memory leaks
* HOLY FUCK big refactor
* misc cleanup
* shell: Fix a bunch of tests
* clean up
* gitignore: fix ending newline
* get windows compiling again
* tidy
* hide linker warn with icu
* closeIfPossible
* Better leak test
* Fix forgetting to decrement reference count
* Update stdio.zig
* Fix shell windows build
* Stupid unreachable
* Woops
* basic echo hi works on windows
* Fix flaky test on Windows
* Fix windows regression in Bun.main (#9156)
* Fix windows regression in Bun.main
* Handle invalid handles
* Fix flaky test
* Better launch config
* Fixup
* Make this test less flaky on Windows
* Fixup
* Cygwin
* Support signal codes in subprocess.kill(), resolve file path
* Treat null as ignore
* Ignore carriage returns
* Fixup
* shell: Fix IOWriter bug
* shell: Use custom `open()`/`openat()`
* windows shell subproc works
* zack commit
* I think I understand WindowsStreamingWriter
* fix thing
* why were we doing this in tests
* shell: Fix rm
* shell: Add rm -rf node_modules/ test
* shell: use `.runAsTest()` in some places to make it easier to determine which test failed
* [autofix.ci] apply automated fixes
* woopsie
* Various changes
* Fix
* shell: abstract output task logic
* shell: mkdir builtin
* fixup
* stuff
* shell: Make writing length of 0 in IOWriter immediately resolve
* shell: Implement `touch`
* shell: basic `cat` working
* Make it compile on windows
* shell: Fix IOReader bug
* [autofix.ci] apply automated fixes
* fix windows kill on subprocess/process
* fix dns tests to match behavior on windows (same as nodejs)
* fix windows ci
* again
* move `close_handle` to flags in `PipeWriter` and fix shell hanging
* Fix `ls` not giving non-zero exit code on error
* Handle edgecase in is_atty
* Fix writer.flush() when there's no data
* Fix some tests
* Disable uv_unref on uv_process_t on Windows, for now.
* fix writer.end
* fix stdout.write
* fix child-process on win32
* Make this test less flaky on Windows
* Add assertion
* Make these the same
* Make it pass on windows
* Don't commit
* Log the test name
* Make this test less flaky on windows
* Make this test less flaky on windows
* Print which test is taking awhile in the runner
* fixups
* Fixups
* Add some assertions
* Bring back test concurrency
* shell: bring back redirect stdin
* make it compile again cc @zackradisic
* initialize env map with capacity
* some fixes
* cleanup
* oops
* fix leak, fix done
* fix unconsumedPromises on events
* always run expect
* Update child_process.test.ts
* fix reading special files
* Fix a test
* Deflake this test
* Make these comparisons easier
* Won't really fix it but slightly cleaner
* Update serve.test.ts
* Make the checks for if the body is already used more resilient
* Move this to the harness
* Make this test not hang in development
* Fix this test
* Make the logs better
* zero init some things
* Make this test better
* Fix readSocket
* Parallelize this test
* Handle EPipe and avoid big data
* This was a mistake
* Fix a bunch of things
* Fix memory leak
* Avoid sigpipe + optimize + delete dead code
* Make this take less time
* Make it bigger
* Remove some redundant code
* Update process.zig
* Merge and hopefully don't breka things along teh way
* Silence build warning
* Uncomment on posix
* Skip test on windows
* windows
* Cleanup test
* Update
* Deflake
* always
* less flaky test
* [autofix.ci] apply automated fixes
* logs
* fix uaf on shell IOReader
* stuff to make it work with mini event loop
* fix 2 double free scenarios, support redirections on windows
* shell: Make `1>&2` and `2>&1` work with libuv
* yoops
* Partial fix
* Partial fix
* fix build
* fix build
* ok
* Make a couple shell tests pass
* More logging
* fix
* fix
* Fix build issue
* more tests pass
* Deflake
* Deflake
* Use Output.panic instead of garbled text
* Formatting
* Introduce `bun.sys.File`, use it for `Output.Source.StreamType`, fix nested Output.scoped() calls, use Win32 `ReadFile` API for reading when it's not a libuv file descriptor.
This lets us avoid the subtle usages of `unreachable` in std.os when writing to stdout/stderr.
Previously, we were initializing the libuv loop immediately at launch due to checking for the existence of a bun build --compile'd executable. When the file descriptor is not from libuv, it's just overhead to use libuv
cc @paperdave, please tell me if Iany of that is incorrect or if you think this is a bad idea.
* Fix closing undefined memory file descriptors in spawn
cc @zackradisic
* pause instead of close
* Fix poorly-written test
* We don't need big numbers for this test
* sad workaround
* fixup
* Clearer error handling for this test
* Fix incorrect test
@electroid when ReadableStream isn't closed, hanging is the correct behavior when consuming buffered data. We cannot know if the buffered data is finished if the stream never closes.
* Fix build
* Remove known failing on windows
* Deflake
* Mark no longer failing
* show all the failing tests
* Sort the list of tests
* fix argument handling
* dont show "posix_spawn" as an error code on windows
* make bun-upgrade.test.ts pass on windows
* fix bunx and bun create again sorry
* a
* fix invalidexe because we should not be running javascript files as if they were exes
* Concurrency in test runner + better logging
* Revert "fix invalidexe because we should not be running javascript files as if they were exes"
This reverts commit da47cf8247.
* WIP: Unix fixes (#9322)
* wip
* [autofix.ci] apply automated fixes
* wip 2
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
* Update runner.node.mjs
* Update runner.node.mjs
* Document some environment variables
* shell: Make `Response` work with builtins
* Make it compile
* make pwd test pass
* [autofix.ci] apply automated fixes
* Fix printing garbage for source code previews
* Update javascript.zig
* Fix posix test failures
* Fix signal dispatch
cc @paperdave. Signals can be run from any thread. This causes an assertion failure when the receiving thread happens to not be the main thread. Easiest to reproduce on linux when you spawn 100 short-lived processes at once.
* windows
---------
Co-authored-by: cirospaciari <ciro.spaciari@gmail.com>
Co-authored-by: Zack Radisic <56137411+zackradisic@users.noreply.github.com>
Co-authored-by: Zack Radisic <zackradisic@Zacks-MBP-2.attlocal.net>
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Meghan Denny <meghan@bun.sh>
Co-authored-by: Zack Radisic <zack@theradisic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: dave caruso <me@paperdave.net>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
* node:http: emit 'socket' event after calling http.request()
* add reference links for why this is how it is
* add guards to not waste time
* add a regression test
* use test harness port number
* node:url implement domainToASCII and domainToUnicode
* fix arg checks
* remove unneeded use of WTF::Vector
* tidy
* throw a js error if icu fails
* add a ton more tests, fix ascii guard, upconvert latin1
* even more tests
* add a comment for this guard
* use ASSERT_NOT_REACHED() instead of raise(SIGABRT)
* Add test for ensuring the 'readable' event is emitted on end
* Run emitReadable on nextTick instead of as microtask
* perf: Store intermediate variables
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
---------
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
* windows: implement bun.isWritable
* windows: pass test/cli/run/as-node.test.ts
C:\Users\dave\AppData\Local\Temp\bun-node-a2ae984c3\node.exe is a hardlink on windows so it will not resolve to C:\bun\build\bun-debug.exe
skip the first param since that is not the behavior this test is supposed to be testing
* windows: pass test/js/node/dns/node-dns.test.js
* windows: pass test/js/node/process/process.test.js
* windows: pass test/js/web/streams/streams.test.js
* windows: pass test/js/workerd/html-rewriter.test.js
Closes#8459
* windows: fix node:util.inspect
* windows: these pass now
* windows: pass test/js/node/stream/node-stream.test.js
* disable http sendfile on windows
* use url.origin here
* more sendfile removal
* windows: pass test/js/web/websocket/websocket.test.js
* test/js/deno/performance/performance.test.ts is flaky, come back to it
---------
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
* Add fs.exists[util.promisify.custom]
fs.exists doesn't follow the error-first-callback convention, so it
needs a custom implementation for util.promisify.
* Simplify
---------
Co-authored-by: John-David Dalton <john.david.dalton@gmail.com>
* Fix#8403
* Throw on error by default
* Add the shell promise utilities to `ShellOutput` and `ShellError`
* Fix tests
* [autofix.ci] apply automated fixes
* Fix memleak
* [autofix.ci] apply automated fixes
* Woops
* `Bun.gc(true)` in fd leak test
* fd leak test should check if `fd <= baseline`
* wtf
* oob check
* [autofix.ci] apply automated fixes
* Fix double free
* Fix#8550
* increase mem threshold for linux
* Requested changes and make not throw on by default
* [autofix.ci] apply automated fixes
* more requested changes
* Do destructuring in function definition
* delete
* Change shell output test to enable throwing
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* yay!!!!!!
* [autofix.ci] apply automated fixes
* ok
* do not use reflect here
* ok
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix sys.dup on windows and make console-iterator work
* fix(windows): fix console async iterator to not include the \r
* fix kill on windows again. and report the right errno
---------
Co-authored-by: dave caruso <me@paperdave.net>
* bring in shell impl
* add `$` to global bun scope
* Expose shell parse func on debug mode
* Expose lex tokens and add tests
* refactor parser to have better ast
* assigns and export
* pipeline kinda working
* Decouple Subprocess spawning code from JS stuff
* Subprocess works!
* Conditional execution
* Support JS objects in template expression
* More complete redirection
* Properly drain stdin/stdout/stderr and fix crash from deallocating JSC memory
* Return errors in parser
* Support command substitution
* wip brace expansion stuff
* Rearrange some files
* expansion wip
* Brace expansion working
* wip brace expansion
* refactor brace algorithm
* wip brace expansion on shell
* fix brace expansion
* Working nested brace expansion!
* brace expansion in shell variable assignment only set the last expanded
* stuff
* Small little perf things
* benchmark and test and stuff
* stuff
* fix nested braces but its also kinda broken
* attempt to fix complicated nested braces
* test
* Use fast tokenized algorithm for non nested braces, use parsed algorithm for nested braces
* fix nested braces one and for all
* small stuff
* Not sure if that made a difference
* revert that
* good speed optimization
* rip
* Environment variables, builtin/native shell cmds
* Fix tests
* Support `cd`, `pwd`, add boilerplate for glob expansion
* Support `which`
* Support `rm`
* wip
* wip
* escaping and abstract shell char iterator
* strpool unicode
* Brace expansion support unicode, disallow invalid surrogates in shell script
* shell choose ascii or unicode lexer depending on input
* fix bugs write tests
* kinda start async stuff
* HOLY SHIT big refactor of Subprocess
woops forget to commit this
...and this
* HOLY MOLY it works
* Refactor some stuff, start eval word expansion
* interpret all the nodes
* stuff
* stuff
* stuff
* kind of works but doesnt
* Buffered output works
* no need to heap allocate autosizer
* Fix bug
* Fix some stuff
* unprotect
* move out dummy shell thing
* Bring back assignments
* create expansion state so it can be non blocking for expansions that need IO (glob, cmd subst)
* glob back in action
* Setup builtin non blocking IO commands and implement export
big issue is control flow is really fcked up here need to fix that
* make Cmd state machine use a loop so control flow is a bit more clear
* rename stuff
* move that
* Implement the echo builtin again
* implement cd again but non blocking io
* Fix ls and use proper write function to prevent blockign writes
* Implement which
* holy moly big port std.fs.deleteTree
* fix compile errors
* Okay that works
* rm works thatsnice
* damn
* split it out
* rm async implementation
* fix rm bug for nested
* Work on files as well
* prevent root from being deleted
* rm error handling
* oops
* pwd and fix some script exec bug
* Implement `mv`
* stub out mv to work accross filesystems
* move it around
* woops
* boilerplate for ls and options
* more boilerplate
* stuff
* that got lost in merge
* upgrade shell stuff zig 0.12.0-dev.1828+225fe6ddb
* Implement basic ls
* smol cleanup
* Fix stream, response redirect stdin
* No longer need spawn to be abstract
* Custom promise
* move around some stuff
* shell promise returns shell output
* make tht work for builtins
* refactor IO abstractions to work with JS or mini event loop
* woops
* scaffolding for refactor
* refactor builtins to make event loop refactor easier
* Fix parsing edge case on assignments, fix expansion on cmd assignments
* change subproc to work with any event loop
* Finish refactoring subproc
* move global abstraction out
* big refactor boys
* holy moly: integrate into cli and fix allll the compile errors
* okay works in bun run now
* actually tick the event loop lol
* Fix more stuff
* Support comments
* Fix some tests
* delete that
* Properly report errors when failing to spawn command
* fix a whole bunch of tests
* fix a whole bunch of tests again
* .
* Fix rm
* Fix some exit code bugs, write force rm from deno, fix ls stderr
* fix `rm -d`
* fix `rm -d`
* Fix boolean logic
* error on subshells (e.g. `true && (echo hi && echo lol)`)
* Move out shell state from interpreter struct
* Cmd substitution supports arbitrary script, not just a single cmd
* Some escaping/quotation tests
* Fix stuff add more tests:
- cmd substitution quotations
- escape backticks in single quots
* ALOT of stuff:
- fix proper subshell inheritance of env for cmd subst
- fix: was wrong, assignments don't run in subshell in conditionals
- fix lexing chained vars `$VAR$VAR`
- more tests
* Fix subtle bugs
* Fix crazy redirect to arraybuffer bug
* more crazy echo edgecases
* Proper lexer errors instead of just panicking lol
* yoops
* Proper parsing errors
* Errors for bun run shell script
* Fix redirecting to file
* More test fix bugs yay
* Fix redirect on builtins
* Open redirection fds with O_TRUNC
* Fix lexing invalid variables and add ability to change cwd from JS api
* yoops
* Fix `.cwd()`
* `$PWD` and fix redirection bugs
* `$PWD` and fix redirection bugs
* Get rid of some `FIXME`s
* throw errors in some places instead of panicking
* Print some errors to stderr
* Get rid of some more panics again
* Handle errors on glob
* pwd test
* `.env()`
* copy-on-write abstraction
* Reference counted env strings + fix some tests
* deinit cwd
* Put commands into a pipeline properly
* deinit Expansion and Assigns properly
* comments
* Comments
* Make it compile
* Update types
* [autofix.ci] apply automated fixes
* Only one WaiterThread
* Fix lifetimes and clean up interface
* Update shell.ts
* Add lazy test
* Remove some dead code
* Update shell.zig
* Fix memory leak
* Fix crash with empty braces
* [autofix.ci] apply automated fixes
* Linux build + bun.sh
* Update subproc.zig
* Update interpreter.zig
* Update interpreter.zig
* Fix some stuff that broke
* Fix Windows compile errors
* Fix some fd leaks
* Fix ls
* Fix a bunch of stuff
* Fix quiet
* Update leak tests fix rm bug
* More reproducible tests
* [autofix.ci] apply automated fixes
* more mem leak tests
* [autofix.ci] apply automated fixes
* Fix merge conflict
* Fix test not actually using temp directory
* Update bunshell.test.ts
* Shell instance
* Capture async context
* Increase test timeouts
* [autofix.ci] apply automated fixes
* Escape
* [autofix.ci] apply automated fixes
* Fix crash
* Add more methods
* [autofix.ci] apply automated fixes
* Fix leak
* Treat file(path) blobs as a file path string
* Create bunshell-file.test.ts
* Support Blob input
* Fix leak + organize imports
* doc
* Update shell.md
* Update shell.md
* Update shell.md
* Update shell.md
* Update CMakeLists.txt
* Fix segfault by cloning error path so it's not freed by arena
* deinit ShellErr
* Delete dead code
* fix really stupid segfault
* don't deinit shell ls task in event loop
* Fix ls bug
* Fix tests
* make truly lazy
* allow more things in the shell substitution and escape whitespace
* Fix newline and exit when finishing shell in `bun run`
---------
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
