Commit Graph

129 Commits

Author SHA1 Message Date
Jarred Sumner
4b5233fc3a feat(fetch) rejectUnauthorized and checkServerIdentity (#4514)
* enable root certs on fetch

* rebase

* fix lookup

* some fixes and improvements

* fmt

* more fixes

* more fixes

* check detached onHandshake

* fix promise case

* fix cert non-Native

* add fetch tls tests

* more one test
2023-09-06 22:33:55 -07:00
Jarred Sumner
e44911d4eb fetch works 2023-09-05 22:19:48 -08:00
Jarred Sumner
b9e5758a86 Merge branch 'main' into jarred/prepare-for-libuv 2023-09-05 19:26:45 -08:00
Ciro Spaciari
d268097ded fix SSL proxy tunneling on fetch (#4510) 2023-09-05 15:21:34 -07:00
Jarred Sumner
4b1d1a272a Merge branch 'main' into jarred/prepare-for-libuv 2023-09-05 05:11:46 -08:00
Ciro Spaciari
f1afd58338 fix zlib deflate on fetch (#4483)
* fix zlib deflate on fetch

* mention issue on test

* more tests

* oops
2023-09-04 17:37:47 -07:00
Ciro Spaciari
2d80f94eda fix(HTMLRewriter) buffer response before transform (#4418)
* html rewriter response buffering

* pipe the data when marked as used

* fix empty response

* add some fetch tests

* deinit parent stream

* fix decompression

* keep byte_reader alive

* update builds

* remove nonsense

* was not nonsense after all

* protect tmp ret value from GC, fix readable strong ref deinit/init

* fmt

* if we detach the stream we cannot update the fetch stream

* detach checking source

* more tests, progress with javascript and Direct sink

* drop support for pure readable stream for now

* more fixes

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2023-09-04 12:26:49 -07:00
Jarred Sumner
3775081e5a More progress 2023-08-31 02:11:04 -07:00
Ciro Spaciari
908018a4df fix(http/https) disable decompress on http/https client (#4399)
* disable decompress on http/https module

* make js again
2023-08-30 13:50:09 -03:00
Ciro Spaciari
6e4a1f2918 make pending_response and metdata life cycle more clear and make fetch more reliable (#4331)
* make pending_response and metdata life cycle more clear

* typo

* WIP: memory investigation

* check zlib and fix zlib

* use state allocator for metadata

* remove postBodyProcess

* undo some test things

* fix race condition

* fix removing compressed header

* some extra checks

* remove arenas on zlib and comment repoter.assert because of toOwnedSliceZ

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2023-08-28 07:21:46 -07:00
Jarred Sumner
e2a17344dc just kernel32 things (#4354)
* just kernel32 things

* more

* Update linux_c.zig

* Update windows_c.zig

* Add workaround

Workaround https://github.com/ziglang/zig/issues/16980

* Rename http.zig to bun_dev_http_server.zig

* Rename usages

* more

* more

* more

* thanks tigerbeetle

* Rename `JSC.Node.Syscall` -> `bun.sys`

* more

* woops

* more!

* hmm

* it says there are only 37 errors, but that's not true

* populate argv

* it says 32 errors!

* 24 errors

* fix regular build

* 12 left!

* Still 12 left!

* more

* 2 errors left...

* 1 more error

* Add link to Tigerbeetle

* Fix the remainign error

* Fix test timeout

* Update syscall.zig

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-08-28 04:39:16 -07:00
Jarred Sumner
8a48e8bb0b Report extra memory more (#4289)
* Report memory allocated in fetch

* Memory size reporting to `Headers`

* Fixup memory reporting allocator

* Make these tests do more

* cleanup some of this

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-08-24 19:39:00 -07:00
Jarred Sumner
c60385716b Bunch of streams fixes (#4251)
* Update WebKit

* Don't do async hooks things when async hooks are not enabled

* Smarter scheduling of event loop tasks with the http server

* less exciting approach

* Bump WebKit

* Another approach

* Fix body-stream tests

* Fixes #1886

* Fix UAF in fetch body streaming

* Missing from commit

* Fix leak

* Fix the other leak

* Fix test

* Fix crash

* missing duperef

* Make this code clearer

* Ignore empty chunks

* Fixes #3969

* Delete flaky test

* Update bun-linux-build.yml

* Fix memory issue

* fix result body, and .done status before the last callback, dont touch headers after sent once

* refactor HTTPClientResult

* less flasky corrupted test

* oops

* fix mutex invalid state

* fix onProgressUpdate deinit/unlock

* fix onProgressUpdate deinit/unlock

* oops

* remove verbose

* fix posible null use

* avoid http null

* metadata can still be used onReject after toResponse

* dont leak task.http

* fix flask tests

* less flask close tests

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: cirospaciari <ciro.spaciari@gmail.com>
2023-08-23 14:05:05 -07:00
Ciro Spaciari
9027484ae1 fetch(stream) add stream support for compressed and uncompressed data (#4127)
* streams non compressed data in 64kb chunks (at least)

* fmt

* wip remove pause

* fix default streaming and buffering

* fix atomic lags

* fix size

* make chunked encoding work again (WIP streaming chunked)

* WIP: chunked encoding streaming

* fix end of streamings

* working streaming + compression

* add fixes + tests

* fmt + fix proxy

* fix oopsies

* codegen after merge

* fmt + fixes

* more fixes

* more fixes and logs

* avoid double free

* check empty before pop

* check empty on pop

* fix copy to real when complete

* remove unnecessary logs

* better has_schedule_callback swap, body locked size helper, remove isEmpty from unbounded_queue pop

* fix response ref, fix body_size

* add deflate support, fix error throw, add more tests

* codegen after merge

* remove logs, add connection close test

* fix macOS build

* fix redirect error option

* make body_size more clear

* support new Reponse(response)

* toString DOMWrapper objects properly instead of supporting response in Response constructor

* ignore headers with no name, add more tests

* oops

* handle transform with fetch

* add gz image stream test

* remove duplicate test

* fix missing chunk on macOS under pressure

* oops include all OS

* some fixes

* compare buffers instead of sizes

* refactor err.err and protect it
2023-08-21 20:30:34 -07:00
Ciro Spaciari
d9f162ff95 clean declare structs (#3929) 2023-08-02 12:58:44 -07:00
Jarred Sumner
ec2cf38ad8 [fetch] Ignore invalid Content-Encoding header (#3834)
Fixes #3802

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-07-26 20:24:36 -07:00
Dylan Conway
661355546a zig upgrade (#3667)
* upgrade

* more fixes

* Bump Zig

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-07-18 01:20:20 -07:00
dave caruso
c21fadf9bc set content-length 0 in some cases (#3503) 2023-07-02 20:24:15 -07:00
Dylan Conway
5fa13625a1 upgrade zig to v0.11.0-dev.3737+9eb008717 (#3374)
* progress

* finish `@memset/@memcpy` update

* Update build.zig

* change `@enumToInt` to `@intFromEnum` and friends

* update zig versions

* it was 1

* add link to issue

* add `compileError` reminder

* fix merge

* format

* upgrade to llvm 16

* Revert "upgrade to llvm 16"

This reverts commit cc930ceb1c.

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-06-21 23:38:18 -07:00
Jarred Sumner
4c01454376 Make uploading files with fetch()fast (#3125)
* Make file uploads fast

* Add benchmark

* Update README.md

* defaults

* print

* prettier

* smaller

* fix(path) fix parse behavior (#3134)

* Add macro docs (#3139)

* Add macro doc

* Updates

* Tweaks

* Update doc

* Update macro serialization doc

* Update macro doc

* `--no-macros` flag, disable macros in node_modules

* invert base/filename internally (#3141)

* always false

* Fix broken test

* Add a test sendfile() test with large file

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
2023-05-31 17:20:30 -07:00
Jarred Sumner
b6007a860e [internal] Add more debug logs for uws 2023-05-21 14:30:36 -07:00
Jarred Sumner
b33f20d51b [internal] Make AbortSIgnal usage slightly safer 2023-05-21 14:29:02 -07:00
Ciro Spaciari
6070e135e9 fix proxy status return (#2860) 2023-05-12 08:06:17 -03:00
Ciro Spaciari
3530cfac68 fix redirect: \'manual\' and setTimeout on node:http request (#2848) 2023-05-11 14:25:21 -07:00
Ciro Spaciari
f9831e1f6f feat(fetch) add redirect: 'error' support (#2845)
* add redirect: 'error' support

* fix typo

* fix typo

* refactor FetchRedirect enum

* fix FetchRedirect

* updated
2023-05-10 23:55:39 -07:00
Jarred Sumner
ea47ed0ee5 Fix broken autocomplete 2023-04-19 00:01:27 -07:00
Jarred Sumner
f6a4516d66 Upgrade Zig (#2656)
* Upgrade Zig

* Don't add `d` files in github actions

* Revert "Don't add `d` files in github actions"

This reverts commit 446e2dd674.

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-04-14 03:58:38 -07:00
Ciro Spaciari
6baedd27bc feat(tls.Server) basic support (cert, key, rejectUnauthorized, requestCert, ca) #2412 (#2552)
* cherry picked

* add StringOrBuffer parameter

* Format and adds types

* update uws

* fix tests

* more types

* fix typing

* add timeouts, clean some stuff

* move tests to describe

* fixes SSL hostname when Host is provided

* cleanup on tests

* change 127.0.0.1 to 0.0.0.0

* try another listening parameter

* test timings and update uws

* remove unnecessary comment

* move listening event around

* always await Bun.connect

* do not fail if the tests already passed when using Bun.connect

* regenerate classes

* undo generated classes

* generate classes

* fix merge

---------

Co-authored-by: cirospaciari <cirospaciari@MiWiFi-RA82-srv.cirospaciari>
2023-04-07 04:02:05 -07:00
Ciro Spaciari
d8c467be42 fix(fetch.proxy) fix proxy authentication (#2554)
* fix proxy authentication

* add auth tests

* remove unused
2023-04-05 17:48:18 -07:00
Alex Lam S.L
9b0f12883c [install] reduce parallel HTTP requests under heavy load (#2536)
* [install] reduce parallel HTTP requests under heavy load

* make `max_simultaneous_requests` atomic
2023-04-02 22:32:19 -07:00
Alex Lam S.L
67422785cc [install] fix connection error detection (#2307) 2023-03-05 08:09:17 -08:00
Jarred Sumner
9388b3f825 Add a zig fmt action (#2277)
* Add a zig fmt action

* add failing file

* Setup prettier better

* Update prettier-fmt.yml

* Fail on error

* Update prettier-fmt.yml

* boop

* boop2

* tar.gz

* Update zig-fmt.yml

* Update zig-fmt.yml

* Update zig-fmt.yml

* Update zig-fmt.yml

* Update zig-fmt.yml

* boop

* Update prettier-fmt.yml

* tag

* newlines

* multiline

* fixup

* Update zig-fmt.yml

* update it

* fixup

* both

* w

* Update prettier-fmt.yml

* prettier all the things

* Update package.json

* zig fmt

*  

* bump

* .

* quotes

* fix prettier ignore

* once more

* Update prettier-fmt.yml

* Update fallback.ts

* consistentcy

---------

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
2023-03-02 19:02:10 -08:00
Ciro Spaciari
0afb1693d3 fix(fetch.signal) capture socket and shutdown on abort signal (#2143)
* capture socket and shutdown on abort signal

* queue shutdown tasks

* little cleanup

* change shutdown strategy

* use fetchSwapRemove on fetch shutdown

* use fetchSwapRemove on fetch shutdown

* fix formatting, remove unused property
2023-02-27 16:10:03 -08:00
Alex Lam S.L
26df7ca892 fix compiler_rt linking failure on x86_64 (#2163)
- minor code tweaks
2023-02-24 14:51:20 -08:00
Alex Lam S.L
e887a064fb prefer bun.copy() over std.mem.copy() (#2152) 2023-02-24 05:53:26 -08:00
Dylan Conway
3f04f8d0a6 Upgrade Zig (#2151)
* fixup

* Upgrade Zig

* Remove bad assertion

* strings

* bump

* mode -> optimize

* optimize

* Linux build

* Update bindgen.zig
2023-02-23 23:57:19 -08:00
Jarred Sumner
0143eccb27 Fix flaky request.signal implementation 2023-02-22 21:29:43 -08:00
Ciro Spaciari
24d624b176 feat(Request.signal) Initial support for signal in Request + fetch and Request + Bun.serve (#2097)
* add fetch abort signal

* get aborted (still segfaults)

* bidings.zig u0 error

* still GC/memory error

* fix start crash

* fix AbortSignal fromJS

* change fromJS to obj.as

* addAbortSignalEventListenner

* handle abort types, and add tests

* fix tests

* add custom reason test

* merge 2 substring methods, use MAKE_STATIC_STRING_IMPL

* fix create AbortError and TimeoutError, move globalThis and exception creation to main thread

* fix tests and rebuild headers

* no need to check with substring reason is already an exception

* no need to check with substring reason is already an exception

* fix dumb error inverting conditions for check reason

* fix custom reason behavior

* Request signal

* remove package-lock.json

* Remove JSC.Strong from Request signal

* fix globals for fetch abort signal

* more tests, clone signal crashs

* fix AbortSignal.toJS

* fix toJS bidings for AbortSignal

* add streaming tests

* fix abortion before connecting

* fix tests and segfault

* add fetch testing abort after finish

* fix signal handler cleanup

* support signal event Bun.serve

* pull tests (failing)

* remove unsupported test

* formating

* fix server Request.signal, fix cleanNativeBindings

* add direct tests

* more pull tests

* fix stream tests

* fix fetch, pending onAborted fix in HTTPServerWritable

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2023-02-22 19:27:25 -08:00
Ciro Spaciari
597053ea91 feat(fetch) AbortSignal (#2019)
* add fetch abort signal

* get aborted (still segfaults)

* bidings.zig u0 error

* still GC/memory error

* fix start crash

* fix AbortSignal fromJS

* change fromJS to obj.as

* addAbortSignalEventListenner

* handle abort types, and add tests

* fix tests

* add custom reason test

* merge 2 substring methods, use MAKE_STATIC_STRING_IMPL

* fix create AbortError and TimeoutError, move globalThis and exception creation to main thread

* fix tests and rebuild headers

* no need to check with substring reason is already an exception

* no need to check with substring reason is already an exception

* fix dumb error inverting conditions for check reason

* fix custom reason behavior
2023-02-15 14:20:40 -08:00
Jarred Sumner
99d2d6e607 Support query string parameters in module resolution 2023-01-21 23:11:31 -08:00
Ciro Spaciari
c00fadab9e feat(cli): Support HTTPS_PROXY, HTTP_PROXY, and NO_PROXY #1440 (#1814)
* WIP: http_proxy implemented, first steps for https_proxy #1440

* add HTTP_PROXY support to upgrade_command and WIP: tunneling

* WIP async handshake, stuck on WANT_READ, try to defer  + check

* create and upgrade with proxy working, TLS and non-TLS to proxy TLS working

* bun install/upgrade/create working with http(s)_proxy #1440

* add NO_PROXY support #1440

* remove commented code and add TODO

* fix getHttpProxy no_proxy

* fix formatting

* refactor catch and getHttpProxy, fix empty strngs in env for proxy

* allow optimization for handleResponseBody
2023-01-17 17:47:01 -08:00
Jarred SUmner
828fd0cfea Fix debug mode crash 2023-01-13 11:55:46 -08:00
Jarred Sumner
c99059db9f Remove usages of std.x 2023-01-03 18:53:40 -08:00
Jarred Sumner
c0dd284136 Upgrade to latest Zig (#1610)
* @min and @max

* builtins and some trivial ones

* Most of them

* more

* more!

* More Progress

* wip

* Update tagged_pointer.zig

* Update http_client_async.zig

* Most of the iterable dir changes

* alright

* Remove usages of deprecated formatters

* 📷

* fmt

* Update shimmer.zig

* wip

* wip

* wip

* progress

* more

* Latest

* stuck on error

* latest

* workaround stage2

* wip

* Update string_immutable.zig

* wip

* Migrate `Dirent` and `require("fs')` to use JSC<>Zig bindings

* Fix build errors

* Fixup most of the test failures

* Fix `make headers`

* Fix "outside package path" error

* Fixup aligned alloc

* Add missing file

* linux

* More linux fixes

* use latest peechy

* Fix transpiler test failure

* Forgot about these

* Fixup test failure

* Update node-timers.test.ts

* [node:htt] Fix `undefined is not an object` error

Fixes https://github.com/oven-sh/bun/issues/1618

* Update http.exports.js

* Make this test less flaky

* fix hashes

* Fix hex formatting and zls issues

* Download zig version

* Update Dockerfile

* Update Dockerfile

* Update uws

* Update Dockerfile

* Set llvm version

* Update README.md

* Update uws

* Update Dockerfile

* Update io_linux.zig

* Update bun.zig

* Log output

* workaround strange @cInclude error

* Make ffi tests better

* Don't use cImport

* Update c.zig

* Update c-bindings.cpp

* call setOutputDir

* Update Dockerfile

* Use a longer name

* latest

* Update serve.test.ts

Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com>
Co-authored-by: Veikka Tuominen <git@vexu.eu>
2022-12-28 00:51:22 -08:00
Jarred Sumner
714ce7619d [internal] Make string comparisons faster 2022-12-04 06:53:53 -08:00
Jarred Sumner
9f4d7690ac [fetch] Treat 302 like 303 2022-12-01 21:15:34 -08:00
Jarred Sumner
5854d39525 [fetch] Implement redirect: "manual" 2022-12-01 21:09:43 -08:00
Jarred Sumner
b631ea7d72 import everything from "bun" where possible 2022-11-29 02:42:56 -08:00
Jarred Sumner
a4b67ccbff fix protocol relative urls again 2022-11-20 07:03:50 -08:00
Jarred Sumner
003237b086 Typo 2022-11-20 01:45:57 -08:00