Commit Graph

13 Commits

Author SHA1 Message Date
Marko Vejnovic
67647c3522 test(valkey): Improvements to valkey IPC interlock (#23252)
### What does this PR do?

Adds a stronger IPC interlock in the failing subscriber test.

### How did you verify your code works?

Hopefully CI.
2025-10-05 05:07:59 -07:00
Ciro Spaciari
d3ce459f0e fix(valkey/redis) fix tls (includes pub/sub) (#22981)
### What does this PR do?
Fix tls property not being properly set
Fixes https://github.com/oven-sh/bun/issues/22186
### How did you verify your code works?
Tests + Manually test with upstash using `rediss` protocol and tls: true
options

---------

Co-authored-by: Marko Vejnovic <marko.vejnovic@hotmail.com>
Co-authored-by: Marko Vejnovic <marko@bun.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-26 18:57:06 -07:00
Marko Vejnovic
17b503b389 Redis PUB/SUB 2.0 (#22568)
### What does this PR do?

**This PR is created because [the previous PR I
opened](https://github.com/oven-sh/bun/pull/21728) had some concerning
issues.** Thanks @Jarred-Sumner for the help.

The goal of this PR is to introduce PUB/SUB functionality to the
built-in Redis client. Based on the fact that the current Redis API does
not appear to have compatibility with `io-redis` or `redis-node`, I've
decided to do away with existing APIs and API compatibility with these
existing libraries.

I have decided to base my implementation on the [`redis-node` pub/sub
API](https://github.com/redis/node-redis/blob/master/docs/pub-sub.md).

#### Random Things That Happened

- [x] Refactored the build scripts so that `valgrind` can be disabled.
- [x] Added a `numeric` namespace in `harness.ts` with useful
mathematical libraries.
- [x] Added a mechanism in `cppbind.ts` to disable static assertions
(specifically to allow `check_slow` even when returning a `JSValue`).
Implemented via `// NOLINT[NEXTLINE]?\(.*\)` macros.
- [x] Fixed inconsistencies in error handling of `JSMap`.

### How did you verify your code works?

I've written a set of unit tests to hopefully catch the major use-cases
of this feature. They all appear to pass.


#### Future Improvements

I would have a lot more confidence in our Redis implementation if we
tested it with a test suite running over a network which emulates a high
network failure rate. There are large amounts of edge cases that are
worthwhile to grab, but I think we can roll that out in a future PR.

### Future Tasks

- [ ] Tests over flaky network
- [ ] Use the custom private members over `_<member>`.

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-09-26 03:06:18 -07:00
Meghan Denny
51ce3bc269 [publish images] ci: ensure tests that require docker have it available (#22781) 2025-09-25 19:03:22 -07:00
robobun
f8aed4826b Migrate all Docker usage to unified docker-compose infrastructure (#22740)
## Summary

This PR migrates all Docker container usage in tests from individual
`docker run` commands to a centralized Docker Compose setup. This makes
tests run **10x faster**, eliminates port conflicts, and provides a much
better developer experience.

## What is Docker Compose?

Docker Compose is a tool for defining and running multi-container Docker
applications. Instead of each test file managing its own containers with
complex `docker run` commands, we define all services once in a YAML
file and Docker Compose handles the orchestration.

## The Problem (Before)

```javascript
// Each test file managed its own container
const container = await Bun.spawn({
  cmd: ["docker", "run", "-d", "-p", "0:5432", "postgres:15"],
  // ... complex setup
});
```

**Issues:**
- Each test started its own container (30+ seconds for PostgreSQL tests)
- Containers were killed after each test (wasteful!)
- Random port conflicts between tests
- No coordination between test suites
- Docker configuration scattered across dozens of test files

## The Solution (After)

```javascript
// All tests share managed containers
const pg = await dockerCompose.ensure("postgres_plain");
// Container starts only if needed, returns connection info
```

**Benefits:**
- Containers start once and stay running (3 seconds for PostgreSQL tests
- **10x faster!**)
- Automatic port management (no conflicts)
- All services defined in one place
- Lazy loading (services only start when needed)
- Same setup locally and in CI

## What Changed

### New Infrastructure
- `test/docker/docker-compose.yml` - Defines all test services
- `test/docker/index.ts` - TypeScript API for managing services  
- `test/docker/README.md` - Comprehensive documentation
- Configuration files and init scripts for services

### Services Migrated

| Service | Status | Tests |
|---------|--------|--------|
| PostgreSQL (plain, TLS, auth) |  | All passing |
| MySQL (plain, native_password, TLS) |  | All passing |
| S3/MinIO |  | 276 passing |
| Redis/Valkey |  | 25/26 passing* |
| Autobahn WebSocket |  | 517 available |

*One Redis test was already broken before migration (reconnection test
times out)

### Key Features

- **Dynamic Ports**: Docker assigns available ports automatically (no
conflicts!)
- **Unix Sockets**: Proxy support for PostgreSQL and Redis Unix domain
sockets
- **Persistent Data**: Volumes for services that need data to survive
restarts
- **Health Checks**: Proper readiness detection for all services
- **Backward Compatible**: Fallback to old Docker method if needed

## Performance Improvements

| Test Suite | Before | After | Improvement |
|------------|--------|-------|-------------|
| PostgreSQL | ~30s | ~3s | **10x faster** |
| MySQL | ~25s | ~3s | **8x faster** |
| Redis | ~20s | ~2s | **10x faster** |

The improvements come from container reuse - containers start once and
stay running instead of starting/stopping for each test.

## How to Use

```typescript
import * as dockerCompose from "../../docker/index.ts";

test("database test", async () => {
  // Ensure service is running (starts if needed)
  const pg = await dockerCompose.ensure("postgres_plain");
  
  // Connect using provided info
  const client = new PostgresClient({
    host: pg.host,
    port: pg.ports[5432],  // Mapped to random available port
  });
});
```

## Testing

All affected test suites have been run and verified:
- `bun test test/js/sql/sql.test.ts` 
- `bun test test/js/sql/sql-mysql*.test.ts` 
- `bun test test/js/bun/s3/s3.test.ts` 
- `bun test test/js/valkey/valkey.test.ts` 
- `bun test test/js/web/websocket/autobahn.test.ts` 

## Documentation

Comprehensive documentation added in `test/docker/README.md` including:
- Detailed explanation of Docker Compose for beginners
- Architecture overview
- Usage examples
- Debugging guide
- Migration guide for adding new services

## Notes

- The Redis reconnection test that's skipped was already broken before
this migration. It's a pre-existing issue with the Redis client's
reconnection logic, not related to Docker changes.
- All tests that were passing before continue to pass after migration.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <claude@anthropic.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
2025-09-19 04:20:58 -07:00
Jarred Sumner
6611983038 Revert "Redis PUB/SUB (#21728)"
This reverts commit dc3c8f79c4.
2025-09-09 23:31:07 -07:00
Marko Vejnovic
dc3c8f79c4 Redis PUB/SUB (#21728)
### What does this PR do?

The goal of this PR is to introduce PUB/SUB functionality to the
built-in Redis client. Based on the fact that the current Redis API does
not appear to have compatibility with `io-redis` or `redis-node`, I've
decided to do away with existing APIs and API compatibility with these
existing libraries.

I have decided to base my implementation on the [`redis-node` pub/sub
API](https://github.com/redis/node-redis/blob/master/docs/pub-sub.md).



### How did you verify your code works?

I've written a set of unit tests to hopefully catch the major use-cases
of this feature. They all appear to pass:

<img width="368" height="71" alt="image"
src="https://github.com/user-attachments/assets/36527386-c8fe-47f6-b69a-a11d4b614fa0"
/>


#### Future Improvements

I would have a lot more confidence in our Redis implementation if we
tested it with a test suite running over a network which emulates a high
network failure rate. There are large amounts of edge cases that are
worthwhile to grab, but I think we can roll that out in a future PR.

### Future Tasks

- [ ] Tests over flaky network
- [ ] Use the custom private members over `_<member>`.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Co-authored-by: Alistair Smith <hi@alistair.sh>
2025-09-09 22:13:25 -07:00
Marko Vejnovic
5a75bcde13 (#19041): Enable connecting to different databases within Redis (#22385)
### What does this PR do?

Enable connecting to different databases for Redis.

### How did you verify your code works?

Unit tests were added.

### Credits

Thank you very much @HeyItsBATMAN for your original PR. I've made
extremely slight changes to your PR. I apologize for it taking so long
to review and merge your PR.

---------

Co-authored-by: Kai Niebes <kai.niebes@outlook.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-04 14:25:22 -07:00
Marko Vejnovic
e3e8d15263 Fix redis reconnecting (#21724)
### What does this PR do?

This PR fixes https://github.com/oven-sh/bun/issues/19131.

I am not 100% certain that this fix is correct as I am still nebulous
regarding some decisions I've made in this PR. I'll try to provide my
reasoning and would love to be proven wrong:

#### Re-authentication

- The `is_authenticated` flag needs to be reset to false. When the
lifecycle reaches a point of attempting to connect, it sends out a
`HELLO 3`, and receives a response. `handleResponse()` is fired and does
not correctly handle it because there is a guard at the top of the
function:

```zig
if (!this.flags.is_authenticated) {
    this.handleHelloResponse(value);

    // We've handled the HELLO response without consuming anything from the command queue
    return;
}
```

Rather, it treats this packet as a regular data packet and complains
that it doesn't have a promise to associate it to. By resetting the
`is_authenticated` flag to false, we guarantee that we handle the `HELLO
3` packet as an authentication packet.

It also seems to make semantic sense since dropping a connection implies
you dropped authentication.

#### Retry Attempts

I've deleted the `retry_attempts = 0` in `reconnect()` because I noticed
that we would never actually re-attempt to reconnect after the first
attempt. Specifically, I was expecting `valkey.zig:459` to potentially
fire multiple times, but it only ever fired once. Removing this reset to
zero caused successful reattempts (in my case 3 of them).

```zig
        debug("reconnect in {d}ms (attempt {d}/{d})", .{ delay_ms, this.retry_attempts, this.max_retries });
```

I'm still iffy on whether this is necessary, but I think it makes sense.
```zig
        this.client.retry_attempts = 0
```

### How did you verify your code works?

I have added a small unit test. I have compared mainline `bun`, which
fails that test, to this fix, which passes the test.

---------

Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
2025-08-22 12:08:42 -07:00
Meghan Denny
b34bab745b test: handle docker exiting with a signal (#21512) 2025-07-30 23:11:17 -07:00
Jarred Sumner
f677ac322c Rename redis.disconnect to redis.close (#18880) 2025-04-08 19:59:08 -07:00
Jarred Sumner
03f3a59ff5 Always disable stop if necessary timer (#18882) 2025-04-08 19:17:20 -07:00
Jarred Sumner
ec87a27d87 Introduce Bun.redis - a builtin Redis client for Bun (#18812) 2025-04-08 03:34:00 -07:00