Commit Graph

26 Commits

Author SHA1 Message Date
Jarred Sumner
b6b3626c14 fix(bindings): handle errors from String.toJS() for oversized strings (#26213)
## Summary

- When a string exceeds `WTF::String::MaxLength` (~4GB),
`bun.String.createUninitialized()` returns a `.Dead` tag
- The C++ layer now properly throws `ERR_STRING_TOO_LONG` when this
happens
- Updated `String.toJS()` in Zig to return `bun.JSError!jsc.JSValue`
instead of just `jsc.JSValue`
- Updated ~40 Zig caller files to handle the error with `try`
- C++ callers updated with `RETURN_IF_EXCEPTION` checks

## Test plan

- [x] `bun bd test test/js/node/buffer.test.js` - 449 tests pass
- [x] `bun bd
test/js/node/test/parallel/test-buffer-tostring-rangeerror.js` - passes

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
2026-01-21 13:01:25 -08:00
Ciro Spaciari
5d3f37d7ae feat(s3): add Content-Encoding header support for S3 uploads (#26149)
## Summary

Add support for setting the `Content-Encoding` header in S3 `.write()`
and `.writer()` calls, following the same pattern as
`Content-Disposition`.

This allows users to specify the encoding of uploaded content:

```typescript
// With .write()
await s3file.write("compressed data", { contentEncoding: "gzip" });

// With .writer()
const writer = s3file.writer({ contentEncoding: "gzip" });
writer.write("compressed data");
await writer.end();

// With bucket.write()
await bucket.write("key", data, { contentEncoding: "br" });
```

## Implementation

- Extended `SignedHeaders.Key` from 6 bits to 7 bits (64→128
combinations) to accommodate the new header
- Added `content_encoding` to `S3CredentialsWithOptions`, `SignOptions`,
and `SignResult` structs
- Updated `CanonicalRequest` format strings to include
`content-encoding` in AWS SigV4 signing
- Added `getContentEncoding()` method to `Headers` for fetch-based S3
uploads
- Expanded `_headers` array from 9 to 10 elements
- Pass `content_encoding` through all S3 upload paths (upload,
uploadStream, writableStream)

## Test plan

- Added tests for "should be able to set content-encoding" 
- Added tests for "should be able to set content-encoding in writer"
- Tests verify the Content-Encoding header is properly set on uploaded
objects via presigned URL fetch
- All 4 new tests pass with `bun bd test` and fail with
`USE_SYSTEM_BUN=1` (confirming the feature is new)

## Changelog

> Describe your changes in 1-2 sentences. These will be featured on
[bun.sh/blog](https://bun.sh/blog) and Bun's release notes.

Added `contentEncoding` option to S3 `.write()` and `.writer()` methods,
allowing users to set the `Content-Encoding` header when uploading
objects.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-15 18:09:33 -08:00
Dylan Conway
959169dfaf feat(archive): change API to constructor-based with S3 support (#25940)
## Summary
- Change Archive API from `Bun.Archive.from(data)` to `new
Bun.Archive(data, options?)`
- Change compression options from `{ gzip: true }` to `{ compress:
"gzip", level?: number }`
- Default to no compression when no options provided
- Use `{ compress: "gzip" }` to enable gzip compression (level 6 by
default)
- Add Archive support for S3 and local file writes via `Bun.write()`

## New API

```typescript
// Create archive - defaults to uncompressed tar
const archive = new Bun.Archive({
  "hello.txt": "Hello, World!",
  "data.json": JSON.stringify({ foo: "bar" }),
});

// Enable gzip compression
const compressed = new Bun.Archive(files, { compress: "gzip" });

// Gzip with custom level (1-12)
const maxCompression = new Bun.Archive(files, { compress: "gzip", level: 12 });

// Write to local file
await Bun.write("archive.tar", archive);           // uncompressed by default
await Bun.write("archive.tar.gz", compressed);     // gzipped

// Write to S3
await client.write("archive.tar.gz", compressed);          // S3Client.write()
await Bun.write("s3://bucket/archive.tar.gz", compressed); // S3 URL
await s3File.write(compressed);                            // s3File.write()

// Get bytes/blob (uses compression setting from constructor)
const bytes = await archive.bytes();
const blob = await archive.blob();
```

## TypeScript Types

```typescript
type ArchiveCompression = "gzip";

type ArchiveOptions = {
  compress?: "gzip";
  level?: number;  // 1-12, default 6 when gzip enabled
};
```

## Test plan
- [x] 98 archive tests pass
- [x] S3 integration tests updated to new API
- [x] TypeScript types updated
- [x] Documentation updated with new examples

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2026-01-12 14:54:21 -08:00
Ciro Spaciari
f25ea59683 feat(s3): add Content-Disposition support for S3 uploads (#25363)
### What does this PR do?
- Add `contentDisposition` option to S3 file uploads to control the
`Content-Disposition` HTTP header
- Support passing `contentDisposition` through all S3 upload paths
(simple uploads, multipart uploads, and streaming uploads)
- Add TypeScript types for the new option
Fixes https://github.com/oven-sh/bun/issues/25362
### How did you verify your code works?
Test
2025-12-08 15:30:20 -08:00
Meghan Denny
51ce3bc269 [publish images] ci: ensure tests that require docker have it available (#22781) 2025-09-25 19:03:22 -07:00
Jarred Sumner
0b549321e9 Start using test.concurrent in our tests (#22823)
### What does this PR do?

### How did you verify your code works?

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Bot <claude-bot@bun.sh>
2025-09-22 05:30:34 -07:00
robobun
f8aed4826b Migrate all Docker usage to unified docker-compose infrastructure (#22740)
## Summary

This PR migrates all Docker container usage in tests from individual
`docker run` commands to a centralized Docker Compose setup. This makes
tests run **10x faster**, eliminates port conflicts, and provides a much
better developer experience.

## What is Docker Compose?

Docker Compose is a tool for defining and running multi-container Docker
applications. Instead of each test file managing its own containers with
complex `docker run` commands, we define all services once in a YAML
file and Docker Compose handles the orchestration.

## The Problem (Before)

```javascript
// Each test file managed its own container
const container = await Bun.spawn({
  cmd: ["docker", "run", "-d", "-p", "0:5432", "postgres:15"],
  // ... complex setup
});
```

**Issues:**
- Each test started its own container (30+ seconds for PostgreSQL tests)
- Containers were killed after each test (wasteful!)
- Random port conflicts between tests
- No coordination between test suites
- Docker configuration scattered across dozens of test files

## The Solution (After)

```javascript
// All tests share managed containers
const pg = await dockerCompose.ensure("postgres_plain");
// Container starts only if needed, returns connection info
```

**Benefits:**
- Containers start once and stay running (3 seconds for PostgreSQL tests
- **10x faster!**)
- Automatic port management (no conflicts)
- All services defined in one place
- Lazy loading (services only start when needed)
- Same setup locally and in CI

## What Changed

### New Infrastructure
- `test/docker/docker-compose.yml` - Defines all test services
- `test/docker/index.ts` - TypeScript API for managing services  
- `test/docker/README.md` - Comprehensive documentation
- Configuration files and init scripts for services

### Services Migrated

| Service | Status | Tests |
|---------|--------|--------|
| PostgreSQL (plain, TLS, auth) |  | All passing |
| MySQL (plain, native_password, TLS) |  | All passing |
| S3/MinIO |  | 276 passing |
| Redis/Valkey |  | 25/26 passing* |
| Autobahn WebSocket |  | 517 available |

*One Redis test was already broken before migration (reconnection test
times out)

### Key Features

- **Dynamic Ports**: Docker assigns available ports automatically (no
conflicts!)
- **Unix Sockets**: Proxy support for PostgreSQL and Redis Unix domain
sockets
- **Persistent Data**: Volumes for services that need data to survive
restarts
- **Health Checks**: Proper readiness detection for all services
- **Backward Compatible**: Fallback to old Docker method if needed

## Performance Improvements

| Test Suite | Before | After | Improvement |
|------------|--------|-------|-------------|
| PostgreSQL | ~30s | ~3s | **10x faster** |
| MySQL | ~25s | ~3s | **8x faster** |
| Redis | ~20s | ~2s | **10x faster** |

The improvements come from container reuse - containers start once and
stay running instead of starting/stopping for each test.

## How to Use

```typescript
import * as dockerCompose from "../../docker/index.ts";

test("database test", async () => {
  // Ensure service is running (starts if needed)
  const pg = await dockerCompose.ensure("postgres_plain");
  
  // Connect using provided info
  const client = new PostgresClient({
    host: pg.host,
    port: pg.ports[5432],  // Mapped to random available port
  });
});
```

## Testing

All affected test suites have been run and verified:
- `bun test test/js/sql/sql.test.ts` 
- `bun test test/js/sql/sql-mysql*.test.ts` 
- `bun test test/js/bun/s3/s3.test.ts` 
- `bun test test/js/valkey/valkey.test.ts` 
- `bun test test/js/web/websocket/autobahn.test.ts` 

## Documentation

Comprehensive documentation added in `test/docker/README.md` including:
- Detailed explanation of Docker Compose for beginners
- Architecture overview
- Usage examples
- Debugging guide
- Migration guide for adding new services

## Notes

- The Redis reconnection test that's skipped was already broken before
this migration. It's a pre-existing issue with the Redis client's
reconnection logic, not related to Docker changes.
- All tests that were passing before continue to pass after migration.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <claude@anthropic.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-09-19 04:20:58 -07:00
Ciro Spaciari
e4dd780c2a fix(s3) ref before calling unlink (#21317)
Co-authored-by: Meghan Denny <meghan@bun.sh>
2025-07-23 22:23:58 -07:00
Ciro Spaciari
93cc51c974 fix(s3) handle slashs at end and start of bucket name (#20997) 2025-07-15 16:52:47 -07:00
Ciro Spaciari
75902e6a21 fix(s3) fix loading http endpoint from env (#20876) 2025-07-07 19:24:32 -07:00
Ciro Spaciari
964f2a8941 fix(fetch/s3) Handle backpressure when upload large bodies (#20481)
Co-authored-by: cirospaciari <6379399+cirospaciari@users.noreply.github.com>
2025-06-27 20:52:46 -07:00
Jarred Sumner
14b439a115 Fix formatters not running in CI + delete unnecessary files (#19433) 2025-05-08 23:22:16 -07:00
Ciro Spaciari
79afefa488 fix(s3) Support Virtual Hosted-Style endpoints (#17292)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-02-13 19:47:53 -08:00
Ciro Spaciari
25f6cbd471 fix(s3) fix queue and multipart flow (#16890)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-01-31 05:19:23 -08:00
Jarred Sumner
75a95aa5fa Rewrite the internal Web Stream native bindings to use less memory (#16349) 2025-01-26 00:04:39 -08:00
Inqnuam
98c7b8452d feat(s3Client): add support for AWS S3 Object Storage Class (#16617) 2025-01-23 05:11:41 -08:00
Ciro Spaciari
5d98e64fd6 make s3 a default client (#16574) 2025-01-21 22:08:52 -08:00
Ciro Spaciari
e0e4a270a8 fix(S3) if the part is too small and is not ended wait for more data before sending (#16453) 2025-01-18 10:47:53 -08:00
Ciro Spaciari
6cdcb1c867 fix(test) enable minio in s3.test.ts (#16298) 2025-01-16 18:39:35 -08:00
Ciro Spaciari
8c04818a80 fix(s3) fix slice with offset 0 (#16400) 2025-01-14 22:38:15 -08:00
Ciro Spaciari
81fce29fd9 S3: refactor + S3Client static method (#16198) 2025-01-06 23:52:19 -08:00
Ciro Spaciari
034f776047 WIP: S3 improvements (#16167) 2025-01-04 19:57:35 -08:00
Jarred Sumner
8a469cce7e Default to "auto" instead of "us-east-1" 2025-01-04 06:21:32 -08:00
Ciro Spaciari
a53f2e6aaa fix test on windows (#16151) 2025-01-04 01:22:48 -08:00
Jarred Sumner
d5fc928ca8 S3 cleanup (#16039)
Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
2025-01-03 19:11:48 -08:00
Ciro Spaciari
fe4176e403 feat(s3) s3 client (#15740)
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: cirospaciari <cirospaciari@users.noreply.github.com>
2024-12-28 17:46:22 -08:00