Commit Graph

18 Commits

Author SHA1 Message Date
Meghan Denny
e209ae81dd meta: ensure there's a single 'bun' import per file in zig (#10137)
* meta: ensure there's a single 'bun' import per file in zig

* undo this change in codegen
2024-04-09 22:41:07 -07:00
Jarred Sumner
00f27fbeec Get bunx tests to pass on Windows (#9729)
* Get bunx tests to pass on Windows

* wip

* WIP

* wip

* wip

* ads

* asdf

* makeOpenPath

* almost revert

* fix build

* enoent

* fix bun install git repos

* cleanup

* use custom zig stdlib from submodule

* update dockerfile to copy zig stdlib sources

* fix dockerfile, update gitmodules

* fix dockerfile

* fix build

* fix build

* fix symlinkat

* fix build

* fix build

* Remove usages of unreachable

* Fixup

* Fixup

* wip

* fixup

* Fix one of the bugs

* asd

* Normalize BUN_INSTALL_CACHE_DIR var

* Set iterable to false when we're about to delete

* Update bun.zig

* I still can't repro this outside CI

* i think that fixes it?

* fix posix compile

* factor out directory creation

* update all install methods to use InstallDirState

* move walker creation to init function

* fix error cleanup

* fix posix compile

* all install tests pass locally

* cleanup

* [autofix.ci] apply automated fixes

* Fix posix regressions

---------

Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Meghan Denny <hello@nektro.net>
Co-authored-by: Georgijs Vilums <georgijs.vilums@gmail.com>
Co-authored-by: Georgijs <48869301+gvilums@users.noreply.github.com>
Co-authored-by: Georgijs Vilums <georgijs@bun.sh>
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2024-04-03 20:53:28 -07:00
Georgijs
1a695f13bf Async file copying on windows (#8649)
* rework and simplify file opening in windows

* fix tests

* symlink tests pass

* update error handling

* remove outdated normalization check

* fix mac build

* apply suggested fixes

* fix path tests

* remove debug print

* fix windows compile
2024-02-03 22:33:35 -08:00
dave caruso
bc7e7027e4 windows: some random things (#8521)
* swaggin

* fix(bun_shim_impl): support exe names that are shorter than four characters

* hi

* a
2024-01-29 10:50:24 -08:00
Zack Radisic
1b1760a9c9 feat: Bun shell (#7748)
* bring in shell impl

* add `$` to global bun scope

* Expose shell parse func on debug mode

* Expose lex tokens and add tests

* refactor parser to have better ast

* assigns and export

* pipeline kinda working

* Decouple Subprocess spawning code from JS stuff

* Subprocess works!

* Conditional execution

* Support JS objects in template expression

* More complete redirection

* Properly drain stdin/stdout/stderr and fix crash from deallocating JSC memory

* Return errors in parser

* Support command substitution

* wip brace expansion stuff

* Rearrange some files

* expansion wip

* Brace expansion working

* wip brace expansion

* refactor brace algorithm

* wip brace expansion on shell

* fix brace expansion

* Working nested brace expansion!

* brace expansion in shell variable assignment only set the last expanded

* stuff

* Small little perf things

* benchmark and test and stuff

* stuff

* fix nested braces but its also kinda broken

* attempt to fix complicated nested braces

* test

* Use fast tokenized algorithm for non nested braces, use parsed algorithm for nested braces

* fix nested braces one and for all

* small stuff

* Not sure if that made a difference

* revert that

* good speed optimization

* rip

* Environment variables, builtin/native shell cmds

* Fix tests

* Support `cd`, `pwd`, add boilerplate for glob expansion

* Support `which`

* Support `rm`

* wip

* wip

* escaping and abstract shell char iterator

* strpool unicode

* Brace expansion support unicode, disallow invalid surrogates in shell script

* shell choose ascii or unicode lexer depending on input

* fix bugs write tests

* kinda start async stuff

* HOLY SHIT big refactor of Subprocess

woops forget to commit this

...and this

* HOLY MOLY it works

* Refactor some stuff, start eval word expansion

* interpret all the nodes

* stuff

* stuff

* stuff

* kind of works but doesnt

* Buffered output works

* no need to heap allocate autosizer

* Fix bug

* Fix some stuff

* unprotect

* move out dummy shell thing

* Bring back assignments

* create expansion state so it can be non blocking for expansions that need IO (glob, cmd subst)

* glob back in action

* Setup builtin non blocking IO commands and implement export

big issue is control flow is really fcked up here need to fix that

* make Cmd state machine use a loop so control flow is a bit more clear

* rename stuff

* move that

* Implement the echo builtin again

* implement cd again but non blocking io

* Fix ls and use proper write function to prevent blockign writes

* Implement which

* holy moly big port std.fs.deleteTree

* fix compile errors

* Okay that works

* rm works thatsnice

* damn

* split it out

* rm async implementation

* fix rm bug for nested

* Work on files as well

* prevent root from being deleted

* rm error handling

* oops

* pwd and fix some script exec bug

* Implement `mv`

* stub out mv to work accross filesystems

* move it around

* woops

* boilerplate for ls and options

* more boilerplate

* stuff

* that got lost in merge

* upgrade shell stuff zig 0.12.0-dev.1828+225fe6ddb

* Implement basic ls

* smol cleanup

* Fix stream, response redirect stdin

* No longer need spawn to be abstract

* Custom promise

* move around some stuff

* shell promise returns shell output

* make tht work for builtins

* refactor IO abstractions to work with JS or mini event loop

* woops

* scaffolding for refactor

* refactor builtins to make event loop refactor easier

* Fix parsing edge case on assignments, fix expansion on cmd assignments

* change subproc to work with any event loop

* Finish refactoring subproc

* move global abstraction out

* big refactor boys

* holy moly: integrate into cli and fix allll the compile errors

* okay works in bun run now

* actually tick the event loop lol

* Fix more stuff

* Support comments

* Fix some tests

* delete that

* Properly report errors when failing to spawn command

* fix a whole bunch of tests

* fix a whole bunch of tests again

* .

* Fix rm

* Fix some exit code bugs, write force rm from deno, fix ls stderr

* fix `rm -d`

* fix `rm -d`

* Fix boolean logic

* error on subshells (e.g. `true && (echo hi && echo lol)`)

* Move out shell state from interpreter struct

* Cmd substitution supports arbitrary script, not just a single cmd

* Some escaping/quotation tests

* Fix stuff add more tests:

- cmd substitution quotations
- escape backticks in single quots

* ALOT of stuff:

- fix proper subshell inheritance of env for cmd subst
- fix: was wrong, assignments don't run in subshell in conditionals
- fix lexing chained vars `$VAR$VAR`
- more tests

* Fix subtle bugs

* Fix crazy redirect to arraybuffer bug

* more crazy echo edgecases

* Proper lexer errors instead of just panicking lol

* yoops

* Proper parsing errors

* Errors for bun run shell script

* Fix redirecting to file

* More test fix bugs yay

* Fix redirect on builtins

* Open redirection fds with O_TRUNC

* Fix lexing invalid variables and add ability to change cwd from JS api

* yoops

* Fix `.cwd()`

* `$PWD` and fix redirection bugs

* `$PWD` and fix redirection bugs

* Get rid of  some `FIXME`s

* throw errors in some places instead of panicking

* Print some errors to stderr

* Get rid of some more panics again

* Handle errors on glob

* pwd test

* `.env()`

* copy-on-write abstraction

* Reference counted env strings + fix some tests

* deinit cwd

* Put commands into a pipeline properly

* deinit Expansion and Assigns properly

* comments

* Comments

* Make it compile

* Update types

* [autofix.ci] apply automated fixes

* Only one WaiterThread

* Fix lifetimes and clean up interface

* Update shell.ts

* Add lazy test

* Remove some dead code

* Update shell.zig

* Fix memory leak

* Fix crash with empty braces

* [autofix.ci] apply automated fixes

* Linux build + bun.sh

* Update subproc.zig

* Update interpreter.zig

* Update interpreter.zig

* Fix some stuff that broke

* Fix Windows compile errors

* Fix some fd leaks

* Fix ls

* Fix a bunch of stuff

* Fix quiet

* Update leak tests fix rm bug

* More reproducible tests

* [autofix.ci] apply automated fixes

* more mem leak tests

* [autofix.ci] apply automated fixes

* Fix merge conflict

* Fix test not actually using temp directory

* Update bunshell.test.ts

* Shell instance

* Capture async context

* Increase test timeouts

* [autofix.ci] apply automated fixes

* Escape

* [autofix.ci] apply automated fixes

* Fix crash

* Add more methods

* [autofix.ci] apply automated fixes

* Fix leak

* Treat file(path) blobs as a file path string

* Create bunshell-file.test.ts

* Support Blob input

* Fix leak + organize imports

* doc

* Update shell.md

* Update shell.md

* Update shell.md

* Update shell.md

* Update CMakeLists.txt

* Fix segfault by cloning error path so it's not freed by arena

* deinit ShellErr

* Delete dead code

* fix really stupid segfault

* don't deinit shell ls task in event loop

* Fix ls bug

* Fix tests

* make truly lazy

* allow more things in the shell substitution and escape whitespace

* Fix newline and exit when finishing shell in `bun run`

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2024-01-19 17:30:57 -08:00
Jarred Sumner
a36355cefb Some Windows tweaks (#8118)
* Some windows tweaks

* Make this pub

* Update bundle_v2.zig

* Fix woopsie

* Make this error better

* Add assertion for using allocator for HTTP client from another thread.

* Do fewer copies in readdir()

* Fix crash/failing tests

* Update tests

* Fix windows build

* Update loop.c

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2024-01-15 17:33:33 -08:00
Dylan Conway
98d253d9bb fix(install): incremental support for windows (#7991)
* update for windows

* use correct sizes

* mode

* fchmod and correct error checking

* some progress

* walker_skippable.zig for windows

* no segfault

* comptime only

* normalize

* create files in .bin

* bun.sys.write

* string formatters in bun.fmt, diriterator enum

* isAllASCII comptime

* update more

* some cleanup

* fix 1

* fix 2

* simlink

* cast

* remove normalize

* to zig err

* update fchmod

* fixup

* fix running bin files

* --bun for running binaries

* --bun for scripts

* remove binary linking

* update

* remove todo

* todos

* fix

* simlink

* cast

* more cast
2024-01-11 22:00:47 -08:00
dave caruso
369e3022e4 chore: upgrade zig to 0.12.0-dev.1828+225fe6ddb (#7671)
* chore: upgrade zig to 0.12.0-dev.1828+225fe6ddb

* open as iterable

* fix building identifier cache

* fix windows build

* fix linux build

* fix linux build
2023-12-16 00:14:15 -08:00
dave caruso
441612917d windows: more windows stuff (#6938)
* fix(win/upgrade): do not show powershell expand-archive info while upgrading

* start working bun run

* experiment: `bun.new`

* you can now bun run

* Update src/install/install.zig

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>

* Update src/install/install.zig

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>

* stuff

* fix stuff

* fix this

* farther but not really

* sadfs

* path hell

not sure how much worse or better this makes things. its a mess. windows path handlign is a mess aaaaaaaaaaaaaaaa

* path.resolve bs

* remove old build system stuff from pr

* a

* fix some path.parse/join cases

* path closer not perfect

* normalize and join tests tests  done

* paths

* implement path.relative

* ,

* stuff

* assert

* fix compile

* hate

* the code isnt great

* stuff

* housekeeping for build system

* blah

* explain windows sitaution in docs

* some progress? not much though

* zig compiler crashes here

* fix

* yippee

* ok

* a

* ala wala

* fix builds on stuff

* clean

* the tests now run

* a

* aa

* dedupe uv event loop

* fix fs test accuracy

* stuff

* [autofix.ci] apply automated fixes

* huge updat e

* [autofix.ci] apply automated fixes

* url

* [autofix.ci] apply automated fixes

* start windows spawnSync

* [autofix.ci] apply automated fixes

* add --webkit for update submodules

* add better err message for `bun setup`

* fix unix platform build

* .

* [autofix.ci] apply automated fixes

* un-upgrade libarchive

* z

* asdfghj

* wrk

* todo -> panic

* ok

* a

* [autofix.ci] apply automated fixes

* fix build scripts l ol

* dfghj

* fa

* [autofix.ci] apply automated fixes

* aaaa

* a

* l

* [autofix.ci] apply automated fixes

* more logs

* [autofix.ci] apply automated fixes

* j

* fix init_command

* CORE DUMP HELL

* i swear im being pranked by the github actions gods

* fadsjkfdshjkhjkdfsahjkdfshjksdafjkhhjkfdsahfsdkjhfsdjkahf

* thanks IAS

* this is the correct fix

* personal review

* ddisablbe these

* revisions!

* ok

* fix submodule

* stuff

* fix libarchive

* [autofix.ci] apply automated fixes

* stuff

* [autofix.ci] apply automated fixes

* a

* fix addressToJS on windows

* make dns async again

* dx: add flag to update submodules ps1 to clone webkit

* dns error case for libuv

* dx improvements on windows

* newline

* obvious fix

* install steps

* extra note

* fix fs test

* Update building-windows.md

* fix builtins bundler to support \r\n line endnigs

* better

* some windows stuff

* a

* a

* a

* aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

* [autofix.ci] apply automated fixes

* bunfile text works

* fix build on the mac

* hellooooooooooo

* install steps

* ci for baseline?

* fix

* aaa

* wow

* install script revamp

* bug

* OK

* ok

* aaaaaaaaaaaaaa

* okay

* fix the node test runner lol

* fix napi stuff

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: cirospaciari <ciro.spaciai@gmail.com>
Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com>
2023-12-14 16:56:33 -08:00
Zack Radisic
6ba4e950cc Add Bun.Glob (#6861)
* initial glob impl

* Add `Bun.globMatch`

* Glob boilerplate setup

* Experiment with Rust glob implementation

* Rust impl is slow revert

* Setup glob walking

* Basic glob walker working

* Fix segfault

* Recursive directory traversal

* Fix glob match non-ascii

* Make faster lil bit

* use arena

* ASCII fast path

* Experiment with packed codepoint cursor

Results in ~4% perf boost if the glob pattern needs to create/manipulate cursors (for example when the pattern uses braces)

* Try converting to u32 array

Made it pretty slow

* Lazily create codepoint buffer

* Different walk algorithm

* Fast path optimizations

* Add `dot` option to `Glob`

* .

* Fix some bugs

* Fix bug, clean up lil bit

* Windows fix

* Non absolute paths

* use specific version of fast-glob for benchmarks and tests

* .

* Fix some stuff

* Fix more stuff

* Add `hasPendingActivity()` to glob

* accident

* Symlinks

* fast-glob e2e tests

* remove

* woops

* Fix relative paths

* Fix absolute

* add test for `onlyFiles`

* Fix invalid surrogate pairs problem

* Rename: `match/matchSync` -> `scan/scanSync` and `matchString` -> `match`

* forgot to close cwd fd

* Update types

* Add stress test

* Port `micromatch` / `glob-match` / `globlin` tests

* fix stale reference arena thing

* stupid bug

* Add builtins to classes code generator and add `Glob.scanIter()`

* all iterables

* generate fixtures, remove from git

* fix test

* Fix

* woops on test

* Fix stuff

licenses

license

`has_pending_activity` to usize

cwd threadSafe fix atomic compile errors

`GlobWalker` own `cwd`

Fix windows path and absolute test

stuff

* Fixes

* Fix stuff

* Use Syscall.close

* Use private symbols for underlying scan functions to preevent misuse

* Update types

* Fix build for zig

* Fix tests

* Fix more tests

* Prevent these tests from GC'ing too much

* Make this benchmark work in Node and Bun

* Fix memory leak

* Add leak test

* Fix windows

* comment about arena allocator use for glob walker

* Make leak test run in separate process

* Iterator api for glob

* GlobWalker.Iterator

* fix leak test

* Remove old impl

* filter functions wip start

* stuff

* wip lockfile use glob

* glob working with lockfile

* revert lockfile changes

* Update bun.lockb

* Manually set to cwd to prevent test failing on linux CI

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-11-22 14:26:09 -08:00
Jarred Sumner
e2a17344dc just kernel32 things (#4354)
* just kernel32 things

* more

* Update linux_c.zig

* Update windows_c.zig

* Add workaround

Workaround https://github.com/ziglang/zig/issues/16980

* Rename http.zig to bun_dev_http_server.zig

* Rename usages

* more

* more

* more

* thanks tigerbeetle

* Rename `JSC.Node.Syscall` -> `bun.sys`

* more

* woops

* more!

* hmm

* it says there are only 37 errors, but that's not true

* populate argv

* it says 32 errors!

* 24 errors

* fix regular build

* 12 left!

* Still 12 left!

* more

* 2 errors left...

* 1 more error

* Add link to Tigerbeetle

* Fix the remainign error

* Fix test timeout

* Update syscall.zig

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-08-28 04:39:16 -07:00
Dylan Conway
661355546a zig upgrade (#3667)
* upgrade

* more fixes

* Bump Zig

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-07-18 01:20:20 -07:00
Dylan Conway
5fa13625a1 upgrade zig to v0.11.0-dev.3737+9eb008717 (#3374)
* progress

* finish `@memset/@memcpy` update

* Update build.zig

* change `@enumToInt` to `@intFromEnum` and friends

* update zig versions

* it was 1

* add link to issue

* add `compileError` reminder

* fix merge

* format

* upgrade to llvm 16

* Revert "upgrade to llvm 16"

This reverts commit cc930ceb1c.

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-06-21 23:38:18 -07:00
Jarred Sumner
fdb7940c4e Fix a bunch of bugs (#3352)
* Fix a bunch of bugs

* undo that one

* Fix crash in readdir()

* woops

* woops

* Add comment

* ✂️

* Make `readlink()` and `realpath` use much less memory

* Update BunString.cpp

* woopsie

* Unnecessary

* Don't commit these

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-06-18 10:47:42 -07:00
Jarred Sumner
ea47ed0ee5 Fix broken autocomplete 2023-04-19 00:01:27 -07:00
Jarred Sumner
c0dd284136 Upgrade to latest Zig (#1610)
* @min and @max

* builtins and some trivial ones

* Most of them

* more

* more!

* More Progress

* wip

* Update tagged_pointer.zig

* Update http_client_async.zig

* Most of the iterable dir changes

* alright

* Remove usages of deprecated formatters

* 📷

* fmt

* Update shimmer.zig

* wip

* wip

* wip

* progress

* more

* Latest

* stuck on error

* latest

* workaround stage2

* wip

* Update string_immutable.zig

* wip

* Migrate `Dirent` and `require("fs')` to use JSC<>Zig bindings

* Fix build errors

* Fixup most of the test failures

* Fix `make headers`

* Fix "outside package path" error

* Fixup aligned alloc

* Add missing file

* linux

* More linux fixes

* use latest peechy

* Fix transpiler test failure

* Forgot about these

* Fixup test failure

* Update node-timers.test.ts

* [node:htt] Fix `undefined is not an object` error

Fixes https://github.com/oven-sh/bun/issues/1618

* Update http.exports.js

* Make this test less flaky

* fix hashes

* Fix hex formatting and zls issues

* Download zig version

* Update Dockerfile

* Update Dockerfile

* Update uws

* Update Dockerfile

* Set llvm version

* Update README.md

* Update uws

* Update Dockerfile

* Update io_linux.zig

* Update bun.zig

* Log output

* workaround strange @cInclude error

* Make ffi tests better

* Don't use cImport

* Update c.zig

* Update c-bindings.cpp

* call setOutputDir

* Update Dockerfile

* Use a longer name

* latest

* Update serve.test.ts

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Veikka Tuominen <git@vexu.eu>
2022-12-28 00:51:22 -08:00
Jarred Sumner
b631ea7d72 import everything from "bun" where possible 2022-11-29 02:42:56 -08:00
Jarred Sumner
729d445b68 change the directory structure 2022-06-22 23:21:48 -07:00