Claude Bot
aa5ea829a2
feat(repl): add shell command support and fix async event loop handling
...
- Add $`command` syntax for running shell commands in REPL
- Transform $`...` to await Bun.$`...` for shell execution
- Fix event loop handling for async operations by calling autoTick()
- Now properly awaits Bun.sleep(), setTimeout(), and Bun.$ operations
- Add tests for shell commands and Bun.sleep
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 22:11:14 +00:00
Claude Bot
4815c10efe
fix(repl): address fourth round of CodeRabbit review comments
...
- Fix BracketDepth.template tracking: decrement template count when
closing template literal and check template depth in needsContinuation()
- Fix reverseSearch: show proper "not yet implemented" message instead
of confusing prompt that suggests waiting for input
- Fix evalDirect: await promises like eval() does for consistent async
behavior
- Fix getHistoryPath: use std.fs.path.join instead of manual string
formatting for cross-platform correctness
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 22:00:59 +00:00
Claude Bot
eb804bd8e0
fix more review feedback issues
...
- Fix History.next() underflow guard for empty history
- Fix saveHistory to resolve relative paths to absolute
- Fix completion suffix bounds check for property completions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 21:45:33 +00:00
Claude Bot
702e8448d6
address additional code review feedback
...
- Fix .install command prefix parsing (.i alias used wrong offset)
- Fix loadFile to resolve paths to absolute paths first
- Remove silent fallback in saveToFile, emit warning instead
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 21:26:55 +00:00
Claude Bot
be79d67802
address code review feedback for REPL
...
- Delete .bun_repl_history and add to .gitignore
- Fix memory leak in getCompletions for property names
- Use vm.waitForPromise instead of manual event loop polling
- Add proper cleanup for stores and arena on init failure
- Remove manual timeout from test helper
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 21:11:55 +00:00
autofix-ci[bot]
f5018d71df
[autofix.ci] apply automated fixes
2026-01-20 20:53:57 +00:00
Claude Bot
cf159ae2fb
test(repl): add comprehensive tests for bun repl
...
Add 41 tests covering:
- REPL commands (.exit, .q, .help)
- Expression evaluation (numbers, strings, objects, arrays)
- Variable persistence across lines
- Function and class definitions
- Async/await support
- Built-in APIs (Bun, process, fetch, URL, etc.)
- Error handling (syntax and runtime errors)
- Modern JS features (destructuring, spread, arrow functions, etc.)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 20:51:57 +00:00
Claude Bot
261f429fd1
feat(repl): add JSC-based autocomplete for object properties
...
The REPL now provides intelligent autocomplete for object properties
by dynamically querying the JSC runtime. When typing `obj.` and pressing
Tab, the REPL will show available properties from the actual object.
Features:
- Property completion for any object (e.g., `Bun.`, `console.`)
- Navigates nested paths (e.g., `Bun.file.`)
- Includes both own properties and prototype chain
Also adds tests for class persistence and destructuring to verify
the full AST transforms work correctly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 20:44:24 +00:00
Claude Bot
018358d8a1
feat(repl): persist history to ~/.bun_repl_history
...
The REPL now saves command history to the user's home directory
($HOME/.bun_repl_history or %USERPROFILE%\.bun_repl_history on Windows).
History is automatically loaded when the REPL starts and saved on exit.
The .save command can still be used to save history to a custom file.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 20:35:31 +00:00
Claude Bot
73cd4e92bb
feat(repl): integrate full transpiler with replMode for AST transforms
...
This replaces the simple string-based let/const->var transform with
Bun's full transpiler in replMode, enabling:
- Proper hoisting of let/const/var/function/class declarations
- Top-level await support via async IIFE wrapping
- Result capture in { value: expr } wrapper
- Object literal disambiguation (wrapping { } in parens)
- Class and function declarations that persist across lines
The REPL now properly awaits promises by running the event loop
until they resolve, allowing `await Promise.resolve(42)` to work.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 20:28:02 +00:00
Claude Bot
e6f08624b1
test(repl): add comprehensive tests for bun repl
...
Add tests covering:
- Basic expression evaluation
- Bun globals availability
- Variable persistence (let, const, function)
- REPL commands (.help, .timing)
- Error handling
- Object and array display
- Multi-statement lines
Note: Top-level await test is skipped until the full REPL transforms
are integrated, which will wrap code containing await in an async IIFE.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 18:40:35 +00:00
Claude Bot
efa6eab698
feat(repl): add simple REPL transforms for let/const persistence
...
Add a simple string-based transform that converts top-level `let` and
`const` declarations to `var`, which allows variables to persist across
REPL lines since `var` declarations become properties of the global
object in sloppy mode.
This is a temporary solution until we can properly integrate the
AST-based repl_transforms.zig module with the transpiler pipeline.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 18:34:12 +00:00
Claude Bot
d0bf75f88f
feat(repl): implement native Zig REPL for bun repl
...
This implements a new REPL in Zig that replaces the previous bunx-based
REPL. The new REPL provides:
- Interactive line editing with cursor movement and history navigation
- Syntax highlighting using QuickAndDirtyJavaScriptSyntaxHighlighter
- Multi-line input with bracket matching detection
- REPL commands (.help, .exit, .clear, .load, .save, .editor, .timing)
- JavaScript evaluation using the global eval() function
- Pretty-printed output for evaluated expressions
- Support for both TTY (interactive) and non-TTY (piped) input
The REPL properly initializes the JSC VirtualMachine with the API lock
held, enabling safe JavaScript execution. It uses Bun's transpiler
infrastructure and output formatting.
TODO:
- Implement REPL transforms for variable persistence across lines
- Integrate package installation
- Add shell mode integration
- Implement JSC-based autocomplete
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-20 18:26:35 +00:00
Jarred Sumner
3536e422e9
Merge branch 'main' into jarred/repl-mode
2026-01-19 14:09:41 -08:00
Jarred Sumner
8be7c161d0
fix: handle spread/rest patterns in convertBindingToExpr
...
- For arrays: wrap last element in E.Spread when has_spread is true
- For objects: set property.kind to .spread when is_spread flag is set
- Copy flags and is_single_line from binding to match Binding.toExpr
Claude-Generated-By: Claude Code (cli/claude-opus-4-5=100%)
Claude-Steers: 0
Claude-Permission-Prompts: 0
Claude-Escapes: 0
Claude-Plan:
<claude-plan>
# REPL Transform Fixes and Node.js Parity
## Current Status
The basic `replMode` option is implemented. This plan covers fixes and parity with Node.js REPL.
## Issues to Fix
### 1. Value Wrapper Has Extra Parentheses (CRITICAL)
**Current output:**
```js
({
__proto__: null,
value: 42
});
```
**Expected behavior (per Node.js):**
- For **non-async expressions**: Node.js returns `null` (no transform) - the REPL evaluates the expression directly
- For **async expressions**: `(async () => { return { __proto__: null, value: (expr) } })()`
**Solution:**
1. For non-async expressions: Don't wrap in `{ value: expr }` - just return the expression as-is
2. For async expressions: The `{ __proto__: null, value: expr }` is already inside the function after `return`, so no outer parens needed
3. Add inner parens around the expression value for clarity: `{ __proto__: null, value: (expr) }`
### 2. Object Literal Disambiguation (CRITICAL)
**Input:** `{a: 1}` or `{foo: await fetch()}`
**Current:** Parsed as block with labeled statement, NOT object literal
**Solution:** Pre-check input at transpiler layer:
- If code starts with `{` and doesn't end with `;`, try parsing as `(_=CODE)`
- If valid, wrap input as `(CODE)` before processing
- This matches Node.js approach in `repl.js` line 411-414
### 3. Class Declarations Don't Persist to VM Context
**Current:** Uses `let ClassName;` hoisting - doesn't become vm context property
**Node.js behavior:** Also uses `let` - this is a known limitation in Node.js too!
Looking at Node.js `await.js` line 31-37:
```js
ClassDeclaration(node, state, c) {
state.prepend(node, `${node.id.name}=`);
ArrayPrototypePush(state.hoistedDeclarationStatements, `let ${node.id.name}; `);
}
```
**Decision:** Use `var` instead of `let` for class hoisting. This makes classes persist to vm context, matching user expectations for REPL behavior. (Different from Node.js which uses `let`)
---
## Implementation Plan
## Usage Example
```typescript
// REPL tool implementation
const transpiler = new Bun.Transpiler({
loader: "tsx",
replMode: true, // NEW OPTION
});
// For each REPL input line:
const transformed = transpiler.transformSync(userInput);
// Execute in persistent VM context
const result = vm.runInContext(transformed, replContext);
// result.value contains the expression result (wrapped to prevent auto-await)
console.log(result.value);
```
## Design Decisions
- **Value wrapper**: Use `{ value: expr }` wrapper like Node.js to prevent auto-awaiting Promise results
- **Static imports**: Keep static imports as-is (Bun handles them natively)
- **Scope**: Full Node.js REPL transform parity
---
## Node.js REPL Transform Behavior (Reference)
From `vendor/node/lib/internal/repl/await.js`:
### 1. Object Literal Detection
```javascript
// {a:1} → ({a:1}) when starts with { and no trailing ;
if (/^\s*{/.test(code) && !/;\s*$/.test(code)) {
code = `(${code})`;
}
```
### 2. Top-Level Await Transform
```javascript
// Input: await x
// Output: (async () => { return { value: (await x) } })()
// Input: var x = await 1
// Output: var x; (async () => { void (x = await 1) })()
// Input: const x = await 1
// Output: let x; (async () => { void (x = await 1) })() // const→let
// Input: function foo() {} (with await somewhere)
// Output: var foo; (async () => { this.foo = foo; function foo() {} })()
// Input: class Foo {} (with await somewhere)
// Output: let Foo; (async () => { Foo=class Foo {} })()
```
### 3. Transform Skipping
Returns `null` (no transform) when:
- No `await` expression present at top level
- Top-level `return` statement exists
- Code is inside async functions/arrow functions/class methods
---
## Implementation Plan
### Fix 1: Remove Extra Parentheses from Value Wrapper
**Problem:** The printer adds `()` around objects at statement start to disambiguate from blocks.
**Solution:** Always use an IIFE wrapper (sync or async) so the object is after `return`:
```js
// Non-async expression (current - BAD)
({ __proto__: null, value: 42 });
// Non-async expression (fixed - GOOD)
(() => { return { __proto__: null, value: 42 } })()
// Non-async with hoisting (fixed - GOOD)
var x;
(() => { void (x = 1); return { __proto__: null, value: x } })()
// Async (already correct)
var x;
(async () => { void (x = await 1); return { __proto__: null, value: x } })()
```
**Files to modify:**
1. `src/ast/P.zig` - `applyReplValueWrapper()` function
**Changes:**
- Remove the simple `{ value: expr }` wrapper approach
- Always use `applyReplAsyncTransform()` style IIFE wrapping, but with `is_async = false` for non-async code
- This ensures the object is always after `return`, avoiding the parentheses issue
- Hoisting still works for both cases
### Fix 2: Object Literal Disambiguation
**File:** `src/bun.js/api/JSTranspiler.zig` - Before parsing
Add pre-processing check:
```zig
// In transformSync, before parsing:
if (config.repl_mode) {
// Check if input looks like object literal: starts with { and doesn't end with ;
if (startsWithBrace(source) and !endsWithSemicolon(source)) {
// Try parsing as expression by wrapping: _=(CODE)
// If valid, wrap input as (CODE)
source = wrapAsExpression(source);
}
}
```
This matches Node.js `isObjectLiteral()` check in `repl/utils.js:786-789`:
```js
function isObjectLiteral(code) {
return /^\s*{/.test(code) && !/;\s*$/.test(code);
}
```
### Fix 3: Class Declaration Persistence
**File:** `src/ast/P.zig` - `applyReplAsyncTransform()` in the class handling section
Change from:
```zig
// let Foo; (hoisted)
try hoisted_stmts.append(p.s(S.Local{ .kind = .k_let, ... }));
```
To:
```zig
// var Foo; (hoisted) - use var so it becomes context property
try hoisted_stmts.append(p.s(S.Local{ .kind = .k_var, ... }));
// Also add: this.Foo = Foo; assignment after class declaration
```
---
## Node.js REPL Test Cases to Match
From `vendor/node/test/parallel/test-repl-preprocess-top-level-await.js`:
| Input | Expected Output |
|-------|-----------------|
| `await 0` | `(async () => { return { value: (await 0) } })()` |
| `var a = await 1` | `var a; (async () => { void (a = await 1) })()` |
| `let a = await 1` | `let a; (async () => { void (a = await 1) })()` |
| `const a = await 1` | `let a; (async () => { void (a = await 1) })()` |
| `await 0; function foo() {}` | `var foo; (async () => { await 0; this.foo = foo; function foo() {} })()` |
| `await 0; class Foo {}` | `let Foo; (async () => { await 0; Foo=class Foo {} })()` |
| `var {a} = {a:1}, [b] = [1]` | `var a, b; (async () => { void ( ({a} = {a:1}), ([b] = [1])) })()` |
---
## Files to Modify
| File | Changes |
|------|---------|
| `src/ast/P.zig` | Fix value wrapper format, fix class hoisting to use var |
| `src/bun.js/api/JSTranspiler.zig` | Add object literal pre-check |
| `src/ast/js_printer.zig` | May need to check object literal printing |
| `test/js/bun/transpiler/repl-transform.test.ts` | Update tests for exact Node.js parity |
---
## Verification
1. Run Node.js preprocess test cases through Bun's transpiler
2. Verify output matches Node.js exactly (or functionally equivalent)
3. Test with vm.runInContext for variable persistence
4. Test object literal inputs: `{a: 1}`, `{foo: await bar()}`
---
## DEPRECATED - Previous Implementation (Already Done)
### 1. Add `replMode` to Bun.Transpiler API
**File**: `src/bun.js/api/JSTranspiler.zig`
Add to `Config` struct (around line 27-44):
```zig
pub const Config = struct {
// ... existing fields ...
repl_mode: bool = false,
```
Parse the option in `Config.fromJS()` (around line 420-430):
```zig
if (try object.getBooleanLoose(globalThis, "replMode")) |flag| {
this.repl_mode = flag;
}
```
Apply the option in `constructor()` (around line 714-721):
```zig
transpiler.options.repl_mode = config.repl_mode;
```
### 2. Add Feature Flag to Runtime
**File**: `src/runtime.zig` (in `Runtime.Features`)
```zig
/// REPL mode: transforms code for interactive evaluation
/// - Wraps lone object literals `{...}` in parentheses
/// - Hoists variable declarations for REPL persistence
/// - Wraps last expression in { value: expr } for result capture
/// - Assigns functions to context for persistence
repl_mode: bool = false,
```
### 3. Add to BundleOptions
**File**: `src/options.zig`
Add to `BundleOptions` struct:
```zig
repl_mode: bool = false,
```
### 4. Implement REPL Transforms in Parser
**File**: `src/ast/P.zig`
#### 4a. Object Literal Detection (Parser-Level)
In REPL mode, the parser should prefer interpreting ambiguous `{...}` as object literals instead of blocks.
**Location**: `src/ast/parseStmt.zig` in statement parsing
When `repl_mode` is true and the parser sees `{` at the start of a statement:
1. Try parsing as expression statement (object literal) first
2. If that fails, fall back to block statement
This is similar to how JavaScript engines handle REPL input. The parser already has the infrastructure to do this - we just need to change the precedence in REPL mode.
```zig
// In parseStmt when repl_mode is true and we see '{'
if (p.options.features.repl_mode and p.token.tag == .t_open_brace) {
// Try parsing as expression first
const saved_state = p.saveState();
if (p.tryParseExpressionStatement()) |expr_stmt| {
return expr_stmt;
}
p.restoreState(saved_state);
// Fall back to block statement
return p.parseBlockStatement();
}
```
This handles:
- `{a: 1}` → parsed as object literal expression
- `{a: 1, b: 2}` → parsed as object literal expression
- `{ let x = 1; }` → fails as expression, parsed as block
- `{ label: break label; }` → fails as expression (break not valid in object), parsed as block
#### 4b. REPL Transform Pass (in toAST after visiting)
Add a new function `applyReplTransforms()` that:
1. **Detect if transform is needed**: Walk AST to check for top-level `await`
2. **Skip transform when**:
- No `await` at top level
- Top-level `return` statement exists
3. **When transform IS needed**:
- Wrap entire code in `(async () => { ... })()`
- Hoist variable declarations outside the async wrapper
- Convert `const` to `let` for persistence
- Wrap last expression in `return { value: (expr) }`
- Handle function declarations (assign to `this`)
- Handle class declarations (hoist as `let`)
**Key Logic:**
```zig
fn applyReplTransforms(p: *Parser, stmts: []Stmt) ![]Stmt {
// 1. Check for top-level await
const has_await = p.hasTopLevelAwait(stmts);
const has_return = p.hasTopLevelReturn(stmts);
if (!has_await or has_return) {
// Just wrap last expression, no async wrapper needed
return p.wrapLastExpression(stmts);
}
// 2. Collect declarations to hoist
var hoisted = std.ArrayList(Stmt).init(p.allocator);
var inner_stmts = std.ArrayList(Stmt).init(p.allocator);
for (stmts) |stmt| {
switch (stmt.data) {
.s_local => |local| {
// Hoist declaration, convert const→let
try hoisted.append(p.createHoistedDecl(local));
// Add assignment expression to inner
try inner_stmts.append(p.createAssignmentExpr(local));
},
.s_function => |func| {
// var foo; (hoisted)
try hoisted.append(p.createVarDecl(func.name));
// this.foo = foo; function foo() {} (inner)
try inner_stmts.append(p.createThisAssignment(func.name));
try inner_stmts.append(stmt);
},
.s_class => |class| {
// let Foo; (hoisted)
try hoisted.append(p.createLetDecl(class.name));
// Foo = class Foo {} (inner)
try inner_stmts.append(p.createClassAssignment(class));
},
else => try inner_stmts.append(stmt),
}
}
// 3. Wrap last expression in return { value: expr }
p.wrapLastExpressionWithReturn(&inner_stmts);
// 4. Create async IIFE: (async () => { ...inner... })()
const async_iife = p.createAsyncIIFE(inner_stmts.items);
// 5. Combine: hoisted declarations + async IIFE
try hoisted.append(async_iife);
return hoisted.toOwnedSlice();
}
```
### 5. TypeScript Type Definitions
**File**: `packages/bun-types/bun.d.ts`
Add to `TranspilerOptions` interface (around line 1748):
```typescript
interface TranspilerOptions {
// ... existing options ...
/**
* Enable REPL mode transforms:
* - Wraps object literals in parentheses
* - Hoists declarations for REPL persistence
* - Wraps last expression in { value: expr } for result capture
* - Wraps code with await in async IIFE
*/
replMode?: boolean;
}
```
---
## Files to Modify
| File | Changes |
|------|---------|
| `src/bun.js/api/JSTranspiler.zig` | Add `repl_mode` to Config, parse from JS, apply to transpiler |
| `src/runtime.zig` | Add `repl_mode: bool` to `Runtime.Features` |
| `src/options.zig` | Add `repl_mode: bool` to `BundleOptions` |
| `src/ast/P.zig` | REPL transform pass in `toAST()` |
| `src/ast/parseStmt.zig` | Object literal vs block disambiguation in REPL mode |
| `packages/bun-types/bun.d.ts` | Add `replMode?: boolean` to `TranspilerOptions` |
---
## Test Cases
Create test file: `test/js/bun/transpiler/repl-transform.test.ts`
### Part 1: Transform Output Tests (Unit Tests)
Test exact transformation output matches expected patterns:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("Bun.Transpiler replMode - Transform Output", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Based on Node.js test-repl-preprocess-top-level-await.js
const testCases: [string, string | null][] = [
// No await = null (no async transform, but still expression capture)
['0', null],
// Basic await
['await 0', '(async () => { return { value: (await 0) } })()'],
['await 0;', '(async () => { return { value: (await 0) }; })()'],
['(await 0)', '(async () => { return ({ value: (await 0) }) })()'],
// No transform for await inside async functions
['async function foo() { await 0; }', null],
['async () => await 0', null],
['class A { async method() { await 0 } }', null],
// Top-level return = no transform
['await 0; return 0;', null],
// Multiple await - last one gets return wrapper
['await 1; await 2;', '(async () => { await 1; return { value: (await 2) }; })()'],
// Variable hoisting - var
['var a = await 1', 'var a; (async () => { void (a = await 1) })()'],
// Variable hoisting - let
['let a = await 1', 'let a; (async () => { void (a = await 1) })()'],
// Variable hoisting - const becomes let
['const a = await 1', 'let a; (async () => { void (a = await 1) })()'],
// For loop with var - hoist var
['for (var i = 0; i < 1; ++i) { await i }',
'var i; (async () => { for (void (i = 0); i < 1; ++i) { await i } })()'],
// For loop with let - no hoist
['for (let i = 0; i < 1; ++i) { await i }',
'(async () => { for (let i = 0; i < 1; ++i) { await i } })()'],
// Destructuring with var
['var {a} = {a:1}, [b] = [1], {c:{d}} = {c:{d: await 1}}',
'var a, b, d; (async () => { void ( ({a} = {a:1}), ([b] = [1]), ({c:{d}} = {c:{d: await 1}})) })()'],
// Destructuring with let
['let [a, b, c] = await ([1, 2, 3])',
'let a, b, c; (async () => { void ([a, b, c] = await ([1, 2, 3])) })()'],
// Function declarations - assign to this
['await 0; function foo() {}',
'var foo; (async () => { await 0; this.foo = foo; function foo() {} })()'],
// Class declarations - hoist as let
['await 0; class Foo {}',
'let Foo; (async () => { await 0; Foo=class Foo {} })()'],
// Nested scopes
['if (await true) { var a = 1; }',
'var a; (async () => { if (await true) { void (a = 1); } })()'],
['if (await true) { let a = 1; }',
'(async () => { if (await true) { let a = 1; } })()'],
// Mixed declarations
['var a = await 1; let b = 2; const c = 3;',
'var a; let b; let c; (async () => { void (a = await 1); void (b = 2); void (c = 3); })()'],
// for await
['for await (var i of asyncIterable) { i; }',
'var i; (async () => { for await (i of asyncIterable) { i; } })()'],
// for-of with var
['for (var i of [1,2,3]) { await 1; }',
'var i; (async () => { for (i of [1,2,3]) { await 1; } })()'],
// for-in with var
['for (var i in {x:1}) { await 1 }',
'var i; (async () => { for (i in {x:1}) { await 1 } })()'],
// Spread in destructuring
['var { ...rest } = await {}',
'var rest; (async () => { void ({ ...rest } = await {}) })()'],
];
for (const [input, expected] of testCases) {
test(`transform: ${input.slice(0, 40)}...`, () => {
const result = transpiler.transformSync(input);
if (expected === null) {
// No async transform expected, but expression capture may still happen
expect(result).not.toMatch(/^\(async/);
} else {
expect(result.trim()).toBe(expected);
}
});
}
// Object literal detection - parser handles this automatically in REPL mode
describe("object literal vs block disambiguation", () => {
test("{a: 1} parsed as object literal", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{a: 1}");
const result = await vm.runInContext(code, ctx);
// Should evaluate to object, not undefined (block with label)
expect(result.value).toEqual({ a: 1 });
});
test("{a: 1, b: 2} parsed as object literal", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{a: 1, b: 2}");
const result = await vm.runInContext(code, ctx);
expect(result.value).toEqual({ a: 1, b: 2 });
});
test("{ let x = 1; x } parsed as block", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{ let x = 1; x }");
const result = await vm.runInContext(code, ctx);
// Block returns last expression value
expect(result.value).toBe(1);
});
test("{ x: 1; y: 2 } parsed as block with labels", async () => {
// Semicolons make this a block with labeled statements, not object
const ctx = vm.createContext({});
const code = transpiler.transformSync("{ x: 1; y: 2 }");
const result = await vm.runInContext(code, ctx);
// Block with labels returns last value
expect(result.value).toBe(2);
});
});
});
```
### Part 2: Variable Persistence Tests (Integration with node:vm)
Test that variables persist across multiple REPL evaluations:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL Variable Persistence", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Helper to run multiple REPL lines in sequence
async function runReplSession(lines: string[], context?: object) {
const ctx = vm.createContext(context ?? { console });
const results: any[] = [];
for (const line of lines) {
const transformed = transpiler.transformSync(line);
const result = await vm.runInContext(transformed, ctx);
results.push(result?.value ?? result);
}
return { results, context: ctx };
}
test("var persists across lines", async () => {
const { results, context } = await runReplSession([
"var x = 10",
"x + 5",
"x = 20",
"x",
]);
expect(results[1]).toBe(15); // x + 5
expect(results[3]).toBe(20); // x after reassignment
expect(context.x).toBe(20); // x visible in context
});
test("let persists across lines (hoisted)", async () => {
const { results } = await runReplSession([
"let y = await Promise.resolve(100)",
"y * 2",
]);
expect(results[1]).toBe(200);
});
test("const becomes let, can be reassigned in later lines", async () => {
const { results } = await runReplSession([
"const z = await Promise.resolve(5)",
"z",
// In REPL, const becomes let, so next line can redeclare
"z = 10", // This works because const→let
"z",
]);
expect(results[1]).toBe(5);
expect(results[3]).toBe(10);
});
test("function declarations persist", async () => {
const { results, context } = await runReplSession([
"await 1; function add(a, b) { return a + b; }",
"add(2, 3)",
"function multiply(a, b) { return a * b; }", // no await
"multiply(4, 5)",
]);
expect(results[1]).toBe(5);
expect(results[3]).toBe(20);
expect(typeof context.add).toBe("function");
expect(typeof context.multiply).toBe("function");
});
test("class declarations persist", async () => {
const { results, context } = await runReplSession([
"await 1; class Counter { constructor() { this.count = 0; } inc() { this.count++; } }",
"const c = new Counter()",
"c.inc(); c.inc(); c.count",
]);
expect(results[2]).toBe(2);
expect(typeof context.Counter).toBe("function");
});
test("complex session with mixed declarations", async () => {
const { results } = await runReplSession([
"var total = 0",
"async function addAsync(n) { return total += await Promise.resolve(n); }",
"await addAsync(10)",
"await addAsync(20)",
"total",
]);
expect(results[2]).toBe(10);
expect(results[3]).toBe(30);
expect(results[4]).toBe(30);
});
test("destructuring assignment persists", async () => {
const { results, context } = await runReplSession([
"var { a, b } = await Promise.resolve({ a: 1, b: 2 })",
"a + b",
"var [x, y, z] = [10, 20, 30]",
"x + y + z",
]);
expect(results[1]).toBe(3);
expect(results[3]).toBe(60);
expect(context.a).toBe(1);
expect(context.x).toBe(10);
});
});
```
### Part 3: eval() Scoping Semantics Tests
Test that REPL behaves like eval() with proper scoping:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL eval() Scoping Semantics", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
test("var hoists to global context", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("var globalVar = 42");
await vm.runInContext(code, ctx);
expect(ctx.globalVar).toBe(42);
});
test("let/const hoisted for REPL but scoped correctly", async () => {
const ctx = vm.createContext({});
// With await, let is hoisted outside async wrapper
const code1 = transpiler.transformSync("let x = await 1");
await vm.runInContext(code1, ctx);
expect(ctx.x).toBe(1);
// Without await, let behavior depends on implementation
const code2 = transpiler.transformSync("let y = 2");
await vm.runInContext(code2, ctx);
// y should still be accessible in REPL context
});
test("block-scoped let does NOT leak", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("if (await true) { let blockScoped = 1; }");
await vm.runInContext(code, ctx);
// blockScoped should NOT be visible in context
expect(ctx.blockScoped).toBeUndefined();
});
test("function in block hoists with var (sloppy mode)", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("if (await true) { function blockFn() { return 42; } }");
await vm.runInContext(code, ctx);
// In sloppy mode, function in block hoists to function scope
expect(typeof ctx.blockFn).toBe("function");
expect(ctx.blockFn()).toBe(42);
});
test("this binding in function declarations", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("await 1; function greet() { return 'hello'; }");
await vm.runInContext(code, ctx);
// Function should be assigned to this (context) for REPL persistence
expect(ctx.greet()).toBe("hello");
});
test("async function expression captures result", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("await (async () => { return 42; })()");
const result = await vm.runInContext(code, ctx);
expect(result.value).toBe(42);
});
test("Promise result NOT auto-awaited due to { value: } wrapper", async () => {
const ctx = vm.createContext({});
// Without wrapper, result would be auto-awaited
// With { value: } wrapper, we get the Promise object
const code = transpiler.transformSync("await Promise.resolve(Promise.resolve(42))");
const result = await vm.runInContext(code, ctx);
// The inner Promise should be in value, not auto-resolved
expect(result.value).toBeInstanceOf(Promise);
expect(await result.value).toBe(42);
});
});
```
### Part 4: Edge Cases and Error Handling
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL Edge Cases", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
test("empty input", () => {
const result = transpiler.transformSync("");
expect(result).toBe("");
});
test("whitespace only", () => {
const result = transpiler.transformSync(" \n\t ");
expect(result.trim()).toBe("");
});
test("comment only", () => {
const result = transpiler.transformSync("// just a comment");
expect(result).toContain("// just a comment");
});
test("multiline input", () => {
const input = `
var x = await 1;
var y = await 2;
x + y
`;
const result = transpiler.transformSync(input);
expect(result).toContain("var x");
expect(result).toContain("var y");
expect(result).toContain("async");
});
test("TypeScript syntax", () => {
const input = "const x: number = await Promise.resolve(42)";
const result = transpiler.transformSync(input);
expect(result).not.toContain(": number"); // Types stripped
expect(result).toContain("let x");
});
test("JSX in REPL", () => {
const input = "await Promise.resolve(<div>Hello</div>)";
const result = transpiler.transformSync(input);
expect(result).toContain("async");
});
test("import expression (dynamic)", () => {
// Dynamic imports should work fine
const input = "await import('fs')";
const result = transpiler.transformSync(input);
expect(result).toContain("import");
});
test("nested await expressions", () => {
const input = "await (await Promise.resolve(Promise.resolve(1)))";
const result = transpiler.transformSync(input);
expect(result).toContain("{ value:");
});
test("for-await-of", () => {
const input = `
async function* gen() { yield 1; yield 2; }
for await (const x of gen()) { console.log(x); }
`;
const result = transpiler.transformSync(input);
expect(result).toContain("var gen");
expect(result).toContain("for await");
});
});
```
---
## Verification Plan
### 1. Build and Basic Tests
```bash
# Build Bun with changes
bun bd
# Run the REPL transform tests
bun bd test test/js/bun/transpiler/repl-transform.test.ts
```
### 2. Manual Transform Output Verification
```typescript
// test-repl-manual.ts
const t = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Object literal
console.log("Object literal:");
console.log(t.transformSync("{a: 1}"));
// Expected: contains "({a: 1})"
// Basic await
console.log("\nBasic await:");
console.log(t.transformSync("await 0"));
// Expected: (async () => { return { value: (await 0) } })()
// Variable hoisting
console.log("\nVar hoisting:");
console.log(t.transformSync("var x = await 1"));
// Expected: var x; (async () => { void (x = await 1) })()
// const → let
console.log("\nConst to let:");
console.log(t.transformSync("const x = await 1"));
// Expected: let x; (async () => { void (x = await 1) })()
// Function hoisting
console.log("\nFunction:");
console.log(t.transformSync("await 0; function foo() {}"));
// Expected: var foo; (async () => { await 0; this.foo = foo; function foo() {} })()
```
### 3. Full REPL Session Simulation
```typescript
// test-repl-session.ts
import vm from "node:vm";
const t = new Bun.Transpiler({ loader: "tsx", replMode: true });
const ctx = vm.createContext({ console, Promise });
async function repl(code: string) {
const transformed = t.transformSync(code);
console.log(`> ${code}`);
console.log(`[transformed]: ${transformed}`);
const result = await vm.runInContext(transformed, ctx);
console.log(`= ${JSON.stringify(result?.value ?? result)}\n`);
return result?.value ?? result;
}
// Test session
await repl("var counter = 0");
await repl("function increment() { return ++counter; }");
await repl("increment()"); // Should be 1
await repl("increment()"); // Should be 2
await repl("counter"); // Should be 2
await repl("const data = await Promise.resolve({ x: 10, y: 20 })");
await repl("data.x + data.y"); // Should be 30
await repl("class Point { constructor(x, y) { this.x = x; this.y = y; } }");
await repl("const p = new Point(3, 4)");
await repl("Math.sqrt(p.x**2 + p.y**2)"); // Should be 5
```
### 4. Verify No Regressions
```bash
# Run existing transpiler tests
bun bd test test/js/bun/transpiler/
# Run existing vm tests
bun bd test test/js/node/vm/
```
### 5. Cross-check with Node.js (Optional)
Compare transform outputs with Node.js's `processTopLevelAwait`:
```typescript
// Compare a few key transforms with Node.js output
const cases = [
"await 0",
"var x = await 1",
"await 0; function foo() {}",
];
// Verify Bun output matches Node.js patterns
```
</claude-plan>
2026-01-19 11:07:58 -08:00
Jarred Sumner
a47f1555d6
refactor: address code review feedback
...
- Clarify JSDoc for replMode in bun.d.ts
- Consolidate applySyncTransform and applyAsyncTransform into transformWithHoisting
- Fix b_missing to return E.Missing in createBindingAssignment
- Add documentation comment for repl_mode in options.zig
- Remove unnecessary async modifiers from sync test callbacks
Claude-Generated-By: Claude Code (cli/claude-opus-4-5=100%)
Claude-Steers: 0
Claude-Permission-Prompts: 0
Claude-Escapes: 0
Claude-Plan:
<claude-plan>
# REPL Transform Fixes and Node.js Parity
## Current Status
The basic `replMode` option is implemented. This plan covers fixes and parity with Node.js REPL.
## Issues to Fix
### 1. Value Wrapper Has Extra Parentheses (CRITICAL)
**Current output:**
```js
({
__proto__: null,
value: 42
});
```
**Expected behavior (per Node.js):**
- For **non-async expressions**: Node.js returns `null` (no transform) - the REPL evaluates the expression directly
- For **async expressions**: `(async () => { return { __proto__: null, value: (expr) } })()`
**Solution:**
1. For non-async expressions: Don't wrap in `{ value: expr }` - just return the expression as-is
2. For async expressions: The `{ __proto__: null, value: expr }` is already inside the function after `return`, so no outer parens needed
3. Add inner parens around the expression value for clarity: `{ __proto__: null, value: (expr) }`
### 2. Object Literal Disambiguation (CRITICAL)
**Input:** `{a: 1}` or `{foo: await fetch()}`
**Current:** Parsed as block with labeled statement, NOT object literal
**Solution:** Pre-check input at transpiler layer:
- If code starts with `{` and doesn't end with `;`, try parsing as `(_=CODE)`
- If valid, wrap input as `(CODE)` before processing
- This matches Node.js approach in `repl.js` line 411-414
### 3. Class Declarations Don't Persist to VM Context
**Current:** Uses `let ClassName;` hoisting - doesn't become vm context property
**Node.js behavior:** Also uses `let` - this is a known limitation in Node.js too!
Looking at Node.js `await.js` line 31-37:
```js
ClassDeclaration(node, state, c) {
state.prepend(node, `${node.id.name}=`);
ArrayPrototypePush(state.hoistedDeclarationStatements, `let ${node.id.name}; `);
}
```
**Decision:** Use `var` instead of `let` for class hoisting. This makes classes persist to vm context, matching user expectations for REPL behavior. (Different from Node.js which uses `let`)
---
## Implementation Plan
## Usage Example
```typescript
// REPL tool implementation
const transpiler = new Bun.Transpiler({
loader: "tsx",
replMode: true, // NEW OPTION
});
// For each REPL input line:
const transformed = transpiler.transformSync(userInput);
// Execute in persistent VM context
const result = vm.runInContext(transformed, replContext);
// result.value contains the expression result (wrapped to prevent auto-await)
console.log(result.value);
```
## Design Decisions
- **Value wrapper**: Use `{ value: expr }` wrapper like Node.js to prevent auto-awaiting Promise results
- **Static imports**: Keep static imports as-is (Bun handles them natively)
- **Scope**: Full Node.js REPL transform parity
---
## Node.js REPL Transform Behavior (Reference)
From `vendor/node/lib/internal/repl/await.js`:
### 1. Object Literal Detection
```javascript
// {a:1} → ({a:1}) when starts with { and no trailing ;
if (/^\s*{/.test(code) && !/;\s*$/.test(code)) {
code = `(${code})`;
}
```
### 2. Top-Level Await Transform
```javascript
// Input: await x
// Output: (async () => { return { value: (await x) } })()
// Input: var x = await 1
// Output: var x; (async () => { void (x = await 1) })()
// Input: const x = await 1
// Output: let x; (async () => { void (x = await 1) })() // const→let
// Input: function foo() {} (with await somewhere)
// Output: var foo; (async () => { this.foo = foo; function foo() {} })()
// Input: class Foo {} (with await somewhere)
// Output: let Foo; (async () => { Foo=class Foo {} })()
```
### 3. Transform Skipping
Returns `null` (no transform) when:
- No `await` expression present at top level
- Top-level `return` statement exists
- Code is inside async functions/arrow functions/class methods
---
## Implementation Plan
### Fix 1: Remove Extra Parentheses from Value Wrapper
**Problem:** The printer adds `()` around objects at statement start to disambiguate from blocks.
**Solution:** Always use an IIFE wrapper (sync or async) so the object is after `return`:
```js
// Non-async expression (current - BAD)
({ __proto__: null, value: 42 });
// Non-async expression (fixed - GOOD)
(() => { return { __proto__: null, value: 42 } })()
// Non-async with hoisting (fixed - GOOD)
var x;
(() => { void (x = 1); return { __proto__: null, value: x } })()
// Async (already correct)
var x;
(async () => { void (x = await 1); return { __proto__: null, value: x } })()
```
**Files to modify:**
1. `src/ast/P.zig` - `applyReplValueWrapper()` function
**Changes:**
- Remove the simple `{ value: expr }` wrapper approach
- Always use `applyReplAsyncTransform()` style IIFE wrapping, but with `is_async = false` for non-async code
- This ensures the object is always after `return`, avoiding the parentheses issue
- Hoisting still works for both cases
### Fix 2: Object Literal Disambiguation
**File:** `src/bun.js/api/JSTranspiler.zig` - Before parsing
Add pre-processing check:
```zig
// In transformSync, before parsing:
if (config.repl_mode) {
// Check if input looks like object literal: starts with { and doesn't end with ;
if (startsWithBrace(source) and !endsWithSemicolon(source)) {
// Try parsing as expression by wrapping: _=(CODE)
// If valid, wrap input as (CODE)
source = wrapAsExpression(source);
}
}
```
This matches Node.js `isObjectLiteral()` check in `repl/utils.js:786-789`:
```js
function isObjectLiteral(code) {
return /^\s*{/.test(code) && !/;\s*$/.test(code);
}
```
### Fix 3: Class Declaration Persistence
**File:** `src/ast/P.zig` - `applyReplAsyncTransform()` in the class handling section
Change from:
```zig
// let Foo; (hoisted)
try hoisted_stmts.append(p.s(S.Local{ .kind = .k_let, ... }));
```
To:
```zig
// var Foo; (hoisted) - use var so it becomes context property
try hoisted_stmts.append(p.s(S.Local{ .kind = .k_var, ... }));
// Also add: this.Foo = Foo; assignment after class declaration
```
---
## Node.js REPL Test Cases to Match
From `vendor/node/test/parallel/test-repl-preprocess-top-level-await.js`:
| Input | Expected Output |
|-------|-----------------|
| `await 0` | `(async () => { return { value: (await 0) } })()` |
| `var a = await 1` | `var a; (async () => { void (a = await 1) })()` |
| `let a = await 1` | `let a; (async () => { void (a = await 1) })()` |
| `const a = await 1` | `let a; (async () => { void (a = await 1) })()` |
| `await 0; function foo() {}` | `var foo; (async () => { await 0; this.foo = foo; function foo() {} })()` |
| `await 0; class Foo {}` | `let Foo; (async () => { await 0; Foo=class Foo {} })()` |
| `var {a} = {a:1}, [b] = [1]` | `var a, b; (async () => { void ( ({a} = {a:1}), ([b] = [1])) })()` |
---
## Files to Modify
| File | Changes |
|------|---------|
| `src/ast/P.zig` | Fix value wrapper format, fix class hoisting to use var |
| `src/bun.js/api/JSTranspiler.zig` | Add object literal pre-check |
| `src/ast/js_printer.zig` | May need to check object literal printing |
| `test/js/bun/transpiler/repl-transform.test.ts` | Update tests for exact Node.js parity |
---
## Verification
1. Run Node.js preprocess test cases through Bun's transpiler
2. Verify output matches Node.js exactly (or functionally equivalent)
3. Test with vm.runInContext for variable persistence
4. Test object literal inputs: `{a: 1}`, `{foo: await bar()}`
---
## DEPRECATED - Previous Implementation (Already Done)
### 1. Add `replMode` to Bun.Transpiler API
**File**: `src/bun.js/api/JSTranspiler.zig`
Add to `Config` struct (around line 27-44):
```zig
pub const Config = struct {
// ... existing fields ...
repl_mode: bool = false,
```
Parse the option in `Config.fromJS()` (around line 420-430):
```zig
if (try object.getBooleanLoose(globalThis, "replMode")) |flag| {
this.repl_mode = flag;
}
```
Apply the option in `constructor()` (around line 714-721):
```zig
transpiler.options.repl_mode = config.repl_mode;
```
### 2. Add Feature Flag to Runtime
**File**: `src/runtime.zig` (in `Runtime.Features`)
```zig
/// REPL mode: transforms code for interactive evaluation
/// - Wraps lone object literals `{...}` in parentheses
/// - Hoists variable declarations for REPL persistence
/// - Wraps last expression in { value: expr } for result capture
/// - Assigns functions to context for persistence
repl_mode: bool = false,
```
### 3. Add to BundleOptions
**File**: `src/options.zig`
Add to `BundleOptions` struct:
```zig
repl_mode: bool = false,
```
### 4. Implement REPL Transforms in Parser
**File**: `src/ast/P.zig`
#### 4a. Object Literal Detection (Parser-Level)
In REPL mode, the parser should prefer interpreting ambiguous `{...}` as object literals instead of blocks.
**Location**: `src/ast/parseStmt.zig` in statement parsing
When `repl_mode` is true and the parser sees `{` at the start of a statement:
1. Try parsing as expression statement (object literal) first
2. If that fails, fall back to block statement
This is similar to how JavaScript engines handle REPL input. The parser already has the infrastructure to do this - we just need to change the precedence in REPL mode.
```zig
// In parseStmt when repl_mode is true and we see '{'
if (p.options.features.repl_mode and p.token.tag == .t_open_brace) {
// Try parsing as expression first
const saved_state = p.saveState();
if (p.tryParseExpressionStatement()) |expr_stmt| {
return expr_stmt;
}
p.restoreState(saved_state);
// Fall back to block statement
return p.parseBlockStatement();
}
```
This handles:
- `{a: 1}` → parsed as object literal expression
- `{a: 1, b: 2}` → parsed as object literal expression
- `{ let x = 1; }` → fails as expression, parsed as block
- `{ label: break label; }` → fails as expression (break not valid in object), parsed as block
#### 4b. REPL Transform Pass (in toAST after visiting)
Add a new function `applyReplTransforms()` that:
1. **Detect if transform is needed**: Walk AST to check for top-level `await`
2. **Skip transform when**:
- No `await` at top level
- Top-level `return` statement exists
3. **When transform IS needed**:
- Wrap entire code in `(async () => { ... })()`
- Hoist variable declarations outside the async wrapper
- Convert `const` to `let` for persistence
- Wrap last expression in `return { value: (expr) }`
- Handle function declarations (assign to `this`)
- Handle class declarations (hoist as `let`)
**Key Logic:**
```zig
fn applyReplTransforms(p: *Parser, stmts: []Stmt) ![]Stmt {
// 1. Check for top-level await
const has_await = p.hasTopLevelAwait(stmts);
const has_return = p.hasTopLevelReturn(stmts);
if (!has_await or has_return) {
// Just wrap last expression, no async wrapper needed
return p.wrapLastExpression(stmts);
}
// 2. Collect declarations to hoist
var hoisted = std.ArrayList(Stmt).init(p.allocator);
var inner_stmts = std.ArrayList(Stmt).init(p.allocator);
for (stmts) |stmt| {
switch (stmt.data) {
.s_local => |local| {
// Hoist declaration, convert const→let
try hoisted.append(p.createHoistedDecl(local));
// Add assignment expression to inner
try inner_stmts.append(p.createAssignmentExpr(local));
},
.s_function => |func| {
// var foo; (hoisted)
try hoisted.append(p.createVarDecl(func.name));
// this.foo = foo; function foo() {} (inner)
try inner_stmts.append(p.createThisAssignment(func.name));
try inner_stmts.append(stmt);
},
.s_class => |class| {
// let Foo; (hoisted)
try hoisted.append(p.createLetDecl(class.name));
// Foo = class Foo {} (inner)
try inner_stmts.append(p.createClassAssignment(class));
},
else => try inner_stmts.append(stmt),
}
}
// 3. Wrap last expression in return { value: expr }
p.wrapLastExpressionWithReturn(&inner_stmts);
// 4. Create async IIFE: (async () => { ...inner... })()
const async_iife = p.createAsyncIIFE(inner_stmts.items);
// 5. Combine: hoisted declarations + async IIFE
try hoisted.append(async_iife);
return hoisted.toOwnedSlice();
}
```
### 5. TypeScript Type Definitions
**File**: `packages/bun-types/bun.d.ts`
Add to `TranspilerOptions` interface (around line 1748):
```typescript
interface TranspilerOptions {
// ... existing options ...
/**
* Enable REPL mode transforms:
* - Wraps object literals in parentheses
* - Hoists declarations for REPL persistence
* - Wraps last expression in { value: expr } for result capture
* - Wraps code with await in async IIFE
*/
replMode?: boolean;
}
```
---
## Files to Modify
| File | Changes |
|------|---------|
| `src/bun.js/api/JSTranspiler.zig` | Add `repl_mode` to Config, parse from JS, apply to transpiler |
| `src/runtime.zig` | Add `repl_mode: bool` to `Runtime.Features` |
| `src/options.zig` | Add `repl_mode: bool` to `BundleOptions` |
| `src/ast/P.zig` | REPL transform pass in `toAST()` |
| `src/ast/parseStmt.zig` | Object literal vs block disambiguation in REPL mode |
| `packages/bun-types/bun.d.ts` | Add `replMode?: boolean` to `TranspilerOptions` |
---
## Test Cases
Create test file: `test/js/bun/transpiler/repl-transform.test.ts`
### Part 1: Transform Output Tests (Unit Tests)
Test exact transformation output matches expected patterns:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("Bun.Transpiler replMode - Transform Output", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Based on Node.js test-repl-preprocess-top-level-await.js
const testCases: [string, string | null][] = [
// No await = null (no async transform, but still expression capture)
['0', null],
// Basic await
['await 0', '(async () => { return { value: (await 0) } })()'],
['await 0;', '(async () => { return { value: (await 0) }; })()'],
['(await 0)', '(async () => { return ({ value: (await 0) }) })()'],
// No transform for await inside async functions
['async function foo() { await 0; }', null],
['async () => await 0', null],
['class A { async method() { await 0 } }', null],
// Top-level return = no transform
['await 0; return 0;', null],
// Multiple await - last one gets return wrapper
['await 1; await 2;', '(async () => { await 1; return { value: (await 2) }; })()'],
// Variable hoisting - var
['var a = await 1', 'var a; (async () => { void (a = await 1) })()'],
// Variable hoisting - let
['let a = await 1', 'let a; (async () => { void (a = await 1) })()'],
// Variable hoisting - const becomes let
['const a = await 1', 'let a; (async () => { void (a = await 1) })()'],
// For loop with var - hoist var
['for (var i = 0; i < 1; ++i) { await i }',
'var i; (async () => { for (void (i = 0); i < 1; ++i) { await i } })()'],
// For loop with let - no hoist
['for (let i = 0; i < 1; ++i) { await i }',
'(async () => { for (let i = 0; i < 1; ++i) { await i } })()'],
// Destructuring with var
['var {a} = {a:1}, [b] = [1], {c:{d}} = {c:{d: await 1}}',
'var a, b, d; (async () => { void ( ({a} = {a:1}), ([b] = [1]), ({c:{d}} = {c:{d: await 1}})) })()'],
// Destructuring with let
['let [a, b, c] = await ([1, 2, 3])',
'let a, b, c; (async () => { void ([a, b, c] = await ([1, 2, 3])) })()'],
// Function declarations - assign to this
['await 0; function foo() {}',
'var foo; (async () => { await 0; this.foo = foo; function foo() {} })()'],
// Class declarations - hoist as let
['await 0; class Foo {}',
'let Foo; (async () => { await 0; Foo=class Foo {} })()'],
// Nested scopes
['if (await true) { var a = 1; }',
'var a; (async () => { if (await true) { void (a = 1); } })()'],
['if (await true) { let a = 1; }',
'(async () => { if (await true) { let a = 1; } })()'],
// Mixed declarations
['var a = await 1; let b = 2; const c = 3;',
'var a; let b; let c; (async () => { void (a = await 1); void (b = 2); void (c = 3); })()'],
// for await
['for await (var i of asyncIterable) { i; }',
'var i; (async () => { for await (i of asyncIterable) { i; } })()'],
// for-of with var
['for (var i of [1,2,3]) { await 1; }',
'var i; (async () => { for (i of [1,2,3]) { await 1; } })()'],
// for-in with var
['for (var i in {x:1}) { await 1 }',
'var i; (async () => { for (i in {x:1}) { await 1 } })()'],
// Spread in destructuring
['var { ...rest } = await {}',
'var rest; (async () => { void ({ ...rest } = await {}) })()'],
];
for (const [input, expected] of testCases) {
test(`transform: ${input.slice(0, 40)}...`, () => {
const result = transpiler.transformSync(input);
if (expected === null) {
// No async transform expected, but expression capture may still happen
expect(result).not.toMatch(/^\(async/);
} else {
expect(result.trim()).toBe(expected);
}
});
}
// Object literal detection - parser handles this automatically in REPL mode
describe("object literal vs block disambiguation", () => {
test("{a: 1} parsed as object literal", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{a: 1}");
const result = await vm.runInContext(code, ctx);
// Should evaluate to object, not undefined (block with label)
expect(result.value).toEqual({ a: 1 });
});
test("{a: 1, b: 2} parsed as object literal", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{a: 1, b: 2}");
const result = await vm.runInContext(code, ctx);
expect(result.value).toEqual({ a: 1, b: 2 });
});
test("{ let x = 1; x } parsed as block", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{ let x = 1; x }");
const result = await vm.runInContext(code, ctx);
// Block returns last expression value
expect(result.value).toBe(1);
});
test("{ x: 1; y: 2 } parsed as block with labels", async () => {
// Semicolons make this a block with labeled statements, not object
const ctx = vm.createContext({});
const code = transpiler.transformSync("{ x: 1; y: 2 }");
const result = await vm.runInContext(code, ctx);
// Block with labels returns last value
expect(result.value).toBe(2);
});
});
});
```
### Part 2: Variable Persistence Tests (Integration with node:vm)
Test that variables persist across multiple REPL evaluations:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL Variable Persistence", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Helper to run multiple REPL lines in sequence
async function runReplSession(lines: string[], context?: object) {
const ctx = vm.createContext(context ?? { console });
const results: any[] = [];
for (const line of lines) {
const transformed = transpiler.transformSync(line);
const result = await vm.runInContext(transformed, ctx);
results.push(result?.value ?? result);
}
return { results, context: ctx };
}
test("var persists across lines", async () => {
const { results, context } = await runReplSession([
"var x = 10",
"x + 5",
"x = 20",
"x",
]);
expect(results[1]).toBe(15); // x + 5
expect(results[3]).toBe(20); // x after reassignment
expect(context.x).toBe(20); // x visible in context
});
test("let persists across lines (hoisted)", async () => {
const { results } = await runReplSession([
"let y = await Promise.resolve(100)",
"y * 2",
]);
expect(results[1]).toBe(200);
});
test("const becomes let, can be reassigned in later lines", async () => {
const { results } = await runReplSession([
"const z = await Promise.resolve(5)",
"z",
// In REPL, const becomes let, so next line can redeclare
"z = 10", // This works because const→let
"z",
]);
expect(results[1]).toBe(5);
expect(results[3]).toBe(10);
});
test("function declarations persist", async () => {
const { results, context } = await runReplSession([
"await 1; function add(a, b) { return a + b; }",
"add(2, 3)",
"function multiply(a, b) { return a * b; }", // no await
"multiply(4, 5)",
]);
expect(results[1]).toBe(5);
expect(results[3]).toBe(20);
expect(typeof context.add).toBe("function");
expect(typeof context.multiply).toBe("function");
});
test("class declarations persist", async () => {
const { results, context } = await runReplSession([
"await 1; class Counter { constructor() { this.count = 0; } inc() { this.count++; } }",
"const c = new Counter()",
"c.inc(); c.inc(); c.count",
]);
expect(results[2]).toBe(2);
expect(typeof context.Counter).toBe("function");
});
test("complex session with mixed declarations", async () => {
const { results } = await runReplSession([
"var total = 0",
"async function addAsync(n) { return total += await Promise.resolve(n); }",
"await addAsync(10)",
"await addAsync(20)",
"total",
]);
expect(results[2]).toBe(10);
expect(results[3]).toBe(30);
expect(results[4]).toBe(30);
});
test("destructuring assignment persists", async () => {
const { results, context } = await runReplSession([
"var { a, b } = await Promise.resolve({ a: 1, b: 2 })",
"a + b",
"var [x, y, z] = [10, 20, 30]",
"x + y + z",
]);
expect(results[1]).toBe(3);
expect(results[3]).toBe(60);
expect(context.a).toBe(1);
expect(context.x).toBe(10);
});
});
```
### Part 3: eval() Scoping Semantics Tests
Test that REPL behaves like eval() with proper scoping:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL eval() Scoping Semantics", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
test("var hoists to global context", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("var globalVar = 42");
await vm.runInContext(code, ctx);
expect(ctx.globalVar).toBe(42);
});
test("let/const hoisted for REPL but scoped correctly", async () => {
const ctx = vm.createContext({});
// With await, let is hoisted outside async wrapper
const code1 = transpiler.transformSync("let x = await 1");
await vm.runInContext(code1, ctx);
expect(ctx.x).toBe(1);
// Without await, let behavior depends on implementation
const code2 = transpiler.transformSync("let y = 2");
await vm.runInContext(code2, ctx);
// y should still be accessible in REPL context
});
test("block-scoped let does NOT leak", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("if (await true) { let blockScoped = 1; }");
await vm.runInContext(code, ctx);
// blockScoped should NOT be visible in context
expect(ctx.blockScoped).toBeUndefined();
});
test("function in block hoists with var (sloppy mode)", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("if (await true) { function blockFn() { return 42; } }");
await vm.runInContext(code, ctx);
// In sloppy mode, function in block hoists to function scope
expect(typeof ctx.blockFn).toBe("function");
expect(ctx.blockFn()).toBe(42);
});
test("this binding in function declarations", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("await 1; function greet() { return 'hello'; }");
await vm.runInContext(code, ctx);
// Function should be assigned to this (context) for REPL persistence
expect(ctx.greet()).toBe("hello");
});
test("async function expression captures result", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("await (async () => { return 42; })()");
const result = await vm.runInContext(code, ctx);
expect(result.value).toBe(42);
});
test("Promise result NOT auto-awaited due to { value: } wrapper", async () => {
const ctx = vm.createContext({});
// Without wrapper, result would be auto-awaited
// With { value: } wrapper, we get the Promise object
const code = transpiler.transformSync("await Promise.resolve(Promise.resolve(42))");
const result = await vm.runInContext(code, ctx);
// The inner Promise should be in value, not auto-resolved
expect(result.value).toBeInstanceOf(Promise);
expect(await result.value).toBe(42);
});
});
```
### Part 4: Edge Cases and Error Handling
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL Edge Cases", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
test("empty input", () => {
const result = transpiler.transformSync("");
expect(result).toBe("");
});
test("whitespace only", () => {
const result = transpiler.transformSync(" \n\t ");
expect(result.trim()).toBe("");
});
test("comment only", () => {
const result = transpiler.transformSync("// just a comment");
expect(result).toContain("// just a comment");
});
test("multiline input", () => {
const input = `
var x = await 1;
var y = await 2;
x + y
`;
const result = transpiler.transformSync(input);
expect(result).toContain("var x");
expect(result).toContain("var y");
expect(result).toContain("async");
});
test("TypeScript syntax", () => {
const input = "const x: number = await Promise.resolve(42)";
const result = transpiler.transformSync(input);
expect(result).not.toContain(": number"); // Types stripped
expect(result).toContain("let x");
});
test("JSX in REPL", () => {
const input = "await Promise.resolve(<div>Hello</div>)";
const result = transpiler.transformSync(input);
expect(result).toContain("async");
});
test("import expression (dynamic)", () => {
// Dynamic imports should work fine
const input = "await import('fs')";
const result = transpiler.transformSync(input);
expect(result).toContain("import");
});
test("nested await expressions", () => {
const input = "await (await Promise.resolve(Promise.resolve(1)))";
const result = transpiler.transformSync(input);
expect(result).toContain("{ value:");
});
test("for-await-of", () => {
const input = `
async function* gen() { yield 1; yield 2; }
for await (const x of gen()) { console.log(x); }
`;
const result = transpiler.transformSync(input);
expect(result).toContain("var gen");
expect(result).toContain("for await");
});
});
```
---
## Verification Plan
### 1. Build and Basic Tests
```bash
# Build Bun with changes
bun bd
# Run the REPL transform tests
bun bd test test/js/bun/transpiler/repl-transform.test.ts
```
### 2. Manual Transform Output Verification
```typescript
// test-repl-manual.ts
const t = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Object literal
console.log("Object literal:");
console.log(t.transformSync("{a: 1}"));
// Expected: contains "({a: 1})"
// Basic await
console.log("\nBasic await:");
console.log(t.transformSync("await 0"));
// Expected: (async () => { return { value: (await 0) } })()
// Variable hoisting
console.log("\nVar hoisting:");
console.log(t.transformSync("var x = await 1"));
// Expected: var x; (async () => { void (x = await 1) })()
// const → let
console.log("\nConst to let:");
console.log(t.transformSync("const x = await 1"));
// Expected: let x; (async () => { void (x = await 1) })()
// Function hoisting
console.log("\nFunction:");
console.log(t.transformSync("await 0; function foo() {}"));
// Expected: var foo; (async () => { await 0; this.foo = foo; function foo() {} })()
```
### 3. Full REPL Session Simulation
```typescript
// test-repl-session.ts
import vm from "node:vm";
const t = new Bun.Transpiler({ loader: "tsx", replMode: true });
const ctx = vm.createContext({ console, Promise });
async function repl(code: string) {
const transformed = t.transformSync(code);
console.log(`> ${code}`);
console.log(`[transformed]: ${transformed}`);
const result = await vm.runInContext(transformed, ctx);
console.log(`= ${JSON.stringify(result?.value ?? result)}\n`);
return result?.value ?? result;
}
// Test session
await repl("var counter = 0");
await repl("function increment() { return ++counter; }");
await repl("increment()"); // Should be 1
await repl("increment()"); // Should be 2
await repl("counter"); // Should be 2
await repl("const data = await Promise.resolve({ x: 10, y: 20 })");
await repl("data.x + data.y"); // Should be 30
await repl("class Point { constructor(x, y) { this.x = x; this.y = y; } }");
await repl("const p = new Point(3, 4)");
await repl("Math.sqrt(p.x**2 + p.y**2)"); // Should be 5
```
### 4. Verify No Regressions
```bash
# Run existing transpiler tests
bun bd test test/js/bun/transpiler/
# Run existing vm tests
bun bd test test/js/node/vm/
```
### 5. Cross-check with Node.js (Optional)
Compare transform outputs with Node.js's `processTopLevelAwait`:
```typescript
// Compare a few key transforms with Node.js output
const cases = [
"await 0",
"var x = await 1",
"await 0; function foo() {}",
];
// Verify Bun output matches Node.js patterns
```
</claude-plan>
2026-01-19 00:35:49 -08:00
Jarred Sumner
84ef7598fd
fix: hoist all declarations as var for context property persistence
...
In REPL mode, var declarations at the top level become properties of
the vm context object, while let/const do not. Change all hoisting to
use var so that all variables are accessible via context.varName.
Claude-Generated-By: Claude Code (cli/claude-opus-4-5=100%)
Claude-Steers: 0
Claude-Permission-Prompts: 0
Claude-Escapes: 0
Claude-Plan:
<claude-plan>
# REPL Transform Fixes and Node.js Parity
## Current Status
The basic `replMode` option is implemented. This plan covers fixes and parity with Node.js REPL.
## Issues to Fix
### 1. Value Wrapper Has Extra Parentheses (CRITICAL)
**Current output:**
```js
({
__proto__: null,
value: 42
});
```
**Expected behavior (per Node.js):**
- For **non-async expressions**: Node.js returns `null` (no transform) - the REPL evaluates the expression directly
- For **async expressions**: `(async () => { return { __proto__: null, value: (expr) } })()`
**Solution:**
1. For non-async expressions: Don't wrap in `{ value: expr }` - just return the expression as-is
2. For async expressions: The `{ __proto__: null, value: expr }` is already inside the function after `return`, so no outer parens needed
3. Add inner parens around the expression value for clarity: `{ __proto__: null, value: (expr) }`
### 2. Object Literal Disambiguation (CRITICAL)
**Input:** `{a: 1}` or `{foo: await fetch()}`
**Current:** Parsed as block with labeled statement, NOT object literal
**Solution:** Pre-check input at transpiler layer:
- If code starts with `{` and doesn't end with `;`, try parsing as `(_=CODE)`
- If valid, wrap input as `(CODE)` before processing
- This matches Node.js approach in `repl.js` line 411-414
### 3. Class Declarations Don't Persist to VM Context
**Current:** Uses `let ClassName;` hoisting - doesn't become vm context property
**Node.js behavior:** Also uses `let` - this is a known limitation in Node.js too!
Looking at Node.js `await.js` line 31-37:
```js
ClassDeclaration(node, state, c) {
state.prepend(node, `${node.id.name}=`);
ArrayPrototypePush(state.hoistedDeclarationStatements, `let ${node.id.name}; `);
}
```
**Decision:** Use `var` instead of `let` for class hoisting. This makes classes persist to vm context, matching user expectations for REPL behavior. (Different from Node.js which uses `let`)
---
## Implementation Plan
## Usage Example
```typescript
// REPL tool implementation
const transpiler = new Bun.Transpiler({
loader: "tsx",
replMode: true, // NEW OPTION
});
// For each REPL input line:
const transformed = transpiler.transformSync(userInput);
// Execute in persistent VM context
const result = vm.runInContext(transformed, replContext);
// result.value contains the expression result (wrapped to prevent auto-await)
console.log(result.value);
```
## Design Decisions
- **Value wrapper**: Use `{ value: expr }` wrapper like Node.js to prevent auto-awaiting Promise results
- **Static imports**: Keep static imports as-is (Bun handles them natively)
- **Scope**: Full Node.js REPL transform parity
---
## Node.js REPL Transform Behavior (Reference)
From `vendor/node/lib/internal/repl/await.js`:
### 1. Object Literal Detection
```javascript
// {a:1} → ({a:1}) when starts with { and no trailing ;
if (/^\s*{/.test(code) && !/;\s*$/.test(code)) {
code = `(${code})`;
}
```
### 2. Top-Level Await Transform
```javascript
// Input: await x
// Output: (async () => { return { value: (await x) } })()
// Input: var x = await 1
// Output: var x; (async () => { void (x = await 1) })()
// Input: const x = await 1
// Output: let x; (async () => { void (x = await 1) })() // const→let
// Input: function foo() {} (with await somewhere)
// Output: var foo; (async () => { this.foo = foo; function foo() {} })()
// Input: class Foo {} (with await somewhere)
// Output: let Foo; (async () => { Foo=class Foo {} })()
```
### 3. Transform Skipping
Returns `null` (no transform) when:
- No `await` expression present at top level
- Top-level `return` statement exists
- Code is inside async functions/arrow functions/class methods
---
## Implementation Plan
### Fix 1: Remove Extra Parentheses from Value Wrapper
**Problem:** The printer adds `()` around objects at statement start to disambiguate from blocks.
**Solution:** Always use an IIFE wrapper (sync or async) so the object is after `return`:
```js
// Non-async expression (current - BAD)
({ __proto__: null, value: 42 });
// Non-async expression (fixed - GOOD)
(() => { return { __proto__: null, value: 42 } })()
// Non-async with hoisting (fixed - GOOD)
var x;
(() => { void (x = 1); return { __proto__: null, value: x } })()
// Async (already correct)
var x;
(async () => { void (x = await 1); return { __proto__: null, value: x } })()
```
**Files to modify:**
1. `src/ast/P.zig` - `applyReplValueWrapper()` function
**Changes:**
- Remove the simple `{ value: expr }` wrapper approach
- Always use `applyReplAsyncTransform()` style IIFE wrapping, but with `is_async = false` for non-async code
- This ensures the object is always after `return`, avoiding the parentheses issue
- Hoisting still works for both cases
### Fix 2: Object Literal Disambiguation
**File:** `src/bun.js/api/JSTranspiler.zig` - Before parsing
Add pre-processing check:
```zig
// In transformSync, before parsing:
if (config.repl_mode) {
// Check if input looks like object literal: starts with { and doesn't end with ;
if (startsWithBrace(source) and !endsWithSemicolon(source)) {
// Try parsing as expression by wrapping: _=(CODE)
// If valid, wrap input as (CODE)
source = wrapAsExpression(source);
}
}
```
This matches Node.js `isObjectLiteral()` check in `repl/utils.js:786-789`:
```js
function isObjectLiteral(code) {
return /^\s*{/.test(code) && !/;\s*$/.test(code);
}
```
### Fix 3: Class Declaration Persistence
**File:** `src/ast/P.zig` - `applyReplAsyncTransform()` in the class handling section
Change from:
```zig
// let Foo; (hoisted)
try hoisted_stmts.append(p.s(S.Local{ .kind = .k_let, ... }));
```
To:
```zig
// var Foo; (hoisted) - use var so it becomes context property
try hoisted_stmts.append(p.s(S.Local{ .kind = .k_var, ... }));
// Also add: this.Foo = Foo; assignment after class declaration
```
---
## Node.js REPL Test Cases to Match
From `vendor/node/test/parallel/test-repl-preprocess-top-level-await.js`:
| Input | Expected Output |
|-------|-----------------|
| `await 0` | `(async () => { return { value: (await 0) } })()` |
| `var a = await 1` | `var a; (async () => { void (a = await 1) })()` |
| `let a = await 1` | `let a; (async () => { void (a = await 1) })()` |
| `const a = await 1` | `let a; (async () => { void (a = await 1) })()` |
| `await 0; function foo() {}` | `var foo; (async () => { await 0; this.foo = foo; function foo() {} })()` |
| `await 0; class Foo {}` | `let Foo; (async () => { await 0; Foo=class Foo {} })()` |
| `var {a} = {a:1}, [b] = [1]` | `var a, b; (async () => { void ( ({a} = {a:1}), ([b] = [1])) })()` |
---
## Files to Modify
| File | Changes |
|------|---------|
| `src/ast/P.zig` | Fix value wrapper format, fix class hoisting to use var |
| `src/bun.js/api/JSTranspiler.zig` | Add object literal pre-check |
| `src/ast/js_printer.zig` | May need to check object literal printing |
| `test/js/bun/transpiler/repl-transform.test.ts` | Update tests for exact Node.js parity |
---
## Verification
1. Run Node.js preprocess test cases through Bun's transpiler
2. Verify output matches Node.js exactly (or functionally equivalent)
3. Test with vm.runInContext for variable persistence
4. Test object literal inputs: `{a: 1}`, `{foo: await bar()}`
---
## DEPRECATED - Previous Implementation (Already Done)
### 1. Add `replMode` to Bun.Transpiler API
**File**: `src/bun.js/api/JSTranspiler.zig`
Add to `Config` struct (around line 27-44):
```zig
pub const Config = struct {
// ... existing fields ...
repl_mode: bool = false,
```
Parse the option in `Config.fromJS()` (around line 420-430):
```zig
if (try object.getBooleanLoose(globalThis, "replMode")) |flag| {
this.repl_mode = flag;
}
```
Apply the option in `constructor()` (around line 714-721):
```zig
transpiler.options.repl_mode = config.repl_mode;
```
### 2. Add Feature Flag to Runtime
**File**: `src/runtime.zig` (in `Runtime.Features`)
```zig
/// REPL mode: transforms code for interactive evaluation
/// - Wraps lone object literals `{...}` in parentheses
/// - Hoists variable declarations for REPL persistence
/// - Wraps last expression in { value: expr } for result capture
/// - Assigns functions to context for persistence
repl_mode: bool = false,
```
### 3. Add to BundleOptions
**File**: `src/options.zig`
Add to `BundleOptions` struct:
```zig
repl_mode: bool = false,
```
### 4. Implement REPL Transforms in Parser
**File**: `src/ast/P.zig`
#### 4a. Object Literal Detection (Parser-Level)
In REPL mode, the parser should prefer interpreting ambiguous `{...}` as object literals instead of blocks.
**Location**: `src/ast/parseStmt.zig` in statement parsing
When `repl_mode` is true and the parser sees `{` at the start of a statement:
1. Try parsing as expression statement (object literal) first
2. If that fails, fall back to block statement
This is similar to how JavaScript engines handle REPL input. The parser already has the infrastructure to do this - we just need to change the precedence in REPL mode.
```zig
// In parseStmt when repl_mode is true and we see '{'
if (p.options.features.repl_mode and p.token.tag == .t_open_brace) {
// Try parsing as expression first
const saved_state = p.saveState();
if (p.tryParseExpressionStatement()) |expr_stmt| {
return expr_stmt;
}
p.restoreState(saved_state);
// Fall back to block statement
return p.parseBlockStatement();
}
```
This handles:
- `{a: 1}` → parsed as object literal expression
- `{a: 1, b: 2}` → parsed as object literal expression
- `{ let x = 1; }` → fails as expression, parsed as block
- `{ label: break label; }` → fails as expression (break not valid in object), parsed as block
#### 4b. REPL Transform Pass (in toAST after visiting)
Add a new function `applyReplTransforms()` that:
1. **Detect if transform is needed**: Walk AST to check for top-level `await`
2. **Skip transform when**:
- No `await` at top level
- Top-level `return` statement exists
3. **When transform IS needed**:
- Wrap entire code in `(async () => { ... })()`
- Hoist variable declarations outside the async wrapper
- Convert `const` to `let` for persistence
- Wrap last expression in `return { value: (expr) }`
- Handle function declarations (assign to `this`)
- Handle class declarations (hoist as `let`)
**Key Logic:**
```zig
fn applyReplTransforms(p: *Parser, stmts: []Stmt) ![]Stmt {
// 1. Check for top-level await
const has_await = p.hasTopLevelAwait(stmts);
const has_return = p.hasTopLevelReturn(stmts);
if (!has_await or has_return) {
// Just wrap last expression, no async wrapper needed
return p.wrapLastExpression(stmts);
}
// 2. Collect declarations to hoist
var hoisted = std.ArrayList(Stmt).init(p.allocator);
var inner_stmts = std.ArrayList(Stmt).init(p.allocator);
for (stmts) |stmt| {
switch (stmt.data) {
.s_local => |local| {
// Hoist declaration, convert const→let
try hoisted.append(p.createHoistedDecl(local));
// Add assignment expression to inner
try inner_stmts.append(p.createAssignmentExpr(local));
},
.s_function => |func| {
// var foo; (hoisted)
try hoisted.append(p.createVarDecl(func.name));
// this.foo = foo; function foo() {} (inner)
try inner_stmts.append(p.createThisAssignment(func.name));
try inner_stmts.append(stmt);
},
.s_class => |class| {
// let Foo; (hoisted)
try hoisted.append(p.createLetDecl(class.name));
// Foo = class Foo {} (inner)
try inner_stmts.append(p.createClassAssignment(class));
},
else => try inner_stmts.append(stmt),
}
}
// 3. Wrap last expression in return { value: expr }
p.wrapLastExpressionWithReturn(&inner_stmts);
// 4. Create async IIFE: (async () => { ...inner... })()
const async_iife = p.createAsyncIIFE(inner_stmts.items);
// 5. Combine: hoisted declarations + async IIFE
try hoisted.append(async_iife);
return hoisted.toOwnedSlice();
}
```
### 5. TypeScript Type Definitions
**File**: `packages/bun-types/bun.d.ts`
Add to `TranspilerOptions` interface (around line 1748):
```typescript
interface TranspilerOptions {
// ... existing options ...
/**
* Enable REPL mode transforms:
* - Wraps object literals in parentheses
* - Hoists declarations for REPL persistence
* - Wraps last expression in { value: expr } for result capture
* - Wraps code with await in async IIFE
*/
replMode?: boolean;
}
```
---
## Files to Modify
| File | Changes |
|------|---------|
| `src/bun.js/api/JSTranspiler.zig` | Add `repl_mode` to Config, parse from JS, apply to transpiler |
| `src/runtime.zig` | Add `repl_mode: bool` to `Runtime.Features` |
| `src/options.zig` | Add `repl_mode: bool` to `BundleOptions` |
| `src/ast/P.zig` | REPL transform pass in `toAST()` |
| `src/ast/parseStmt.zig` | Object literal vs block disambiguation in REPL mode |
| `packages/bun-types/bun.d.ts` | Add `replMode?: boolean` to `TranspilerOptions` |
---
## Test Cases
Create test file: `test/js/bun/transpiler/repl-transform.test.ts`
### Part 1: Transform Output Tests (Unit Tests)
Test exact transformation output matches expected patterns:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("Bun.Transpiler replMode - Transform Output", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Based on Node.js test-repl-preprocess-top-level-await.js
const testCases: [string, string | null][] = [
// No await = null (no async transform, but still expression capture)
['0', null],
// Basic await
['await 0', '(async () => { return { value: (await 0) } })()'],
['await 0;', '(async () => { return { value: (await 0) }; })()'],
['(await 0)', '(async () => { return ({ value: (await 0) }) })()'],
// No transform for await inside async functions
['async function foo() { await 0; }', null],
['async () => await 0', null],
['class A { async method() { await 0 } }', null],
// Top-level return = no transform
['await 0; return 0;', null],
// Multiple await - last one gets return wrapper
['await 1; await 2;', '(async () => { await 1; return { value: (await 2) }; })()'],
// Variable hoisting - var
['var a = await 1', 'var a; (async () => { void (a = await 1) })()'],
// Variable hoisting - let
['let a = await 1', 'let a; (async () => { void (a = await 1) })()'],
// Variable hoisting - const becomes let
['const a = await 1', 'let a; (async () => { void (a = await 1) })()'],
// For loop with var - hoist var
['for (var i = 0; i < 1; ++i) { await i }',
'var i; (async () => { for (void (i = 0); i < 1; ++i) { await i } })()'],
// For loop with let - no hoist
['for (let i = 0; i < 1; ++i) { await i }',
'(async () => { for (let i = 0; i < 1; ++i) { await i } })()'],
// Destructuring with var
['var {a} = {a:1}, [b] = [1], {c:{d}} = {c:{d: await 1}}',
'var a, b, d; (async () => { void ( ({a} = {a:1}), ([b] = [1]), ({c:{d}} = {c:{d: await 1}})) })()'],
// Destructuring with let
['let [a, b, c] = await ([1, 2, 3])',
'let a, b, c; (async () => { void ([a, b, c] = await ([1, 2, 3])) })()'],
// Function declarations - assign to this
['await 0; function foo() {}',
'var foo; (async () => { await 0; this.foo = foo; function foo() {} })()'],
// Class declarations - hoist as let
['await 0; class Foo {}',
'let Foo; (async () => { await 0; Foo=class Foo {} })()'],
// Nested scopes
['if (await true) { var a = 1; }',
'var a; (async () => { if (await true) { void (a = 1); } })()'],
['if (await true) { let a = 1; }',
'(async () => { if (await true) { let a = 1; } })()'],
// Mixed declarations
['var a = await 1; let b = 2; const c = 3;',
'var a; let b; let c; (async () => { void (a = await 1); void (b = 2); void (c = 3); })()'],
// for await
['for await (var i of asyncIterable) { i; }',
'var i; (async () => { for await (i of asyncIterable) { i; } })()'],
// for-of with var
['for (var i of [1,2,3]) { await 1; }',
'var i; (async () => { for (i of [1,2,3]) { await 1; } })()'],
// for-in with var
['for (var i in {x:1}) { await 1 }',
'var i; (async () => { for (i in {x:1}) { await 1 } })()'],
// Spread in destructuring
['var { ...rest } = await {}',
'var rest; (async () => { void ({ ...rest } = await {}) })()'],
];
for (const [input, expected] of testCases) {
test(`transform: ${input.slice(0, 40)}...`, () => {
const result = transpiler.transformSync(input);
if (expected === null) {
// No async transform expected, but expression capture may still happen
expect(result).not.toMatch(/^\(async/);
} else {
expect(result.trim()).toBe(expected);
}
});
}
// Object literal detection - parser handles this automatically in REPL mode
describe("object literal vs block disambiguation", () => {
test("{a: 1} parsed as object literal", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{a: 1}");
const result = await vm.runInContext(code, ctx);
// Should evaluate to object, not undefined (block with label)
expect(result.value).toEqual({ a: 1 });
});
test("{a: 1, b: 2} parsed as object literal", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{a: 1, b: 2}");
const result = await vm.runInContext(code, ctx);
expect(result.value).toEqual({ a: 1, b: 2 });
});
test("{ let x = 1; x } parsed as block", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{ let x = 1; x }");
const result = await vm.runInContext(code, ctx);
// Block returns last expression value
expect(result.value).toBe(1);
});
test("{ x: 1; y: 2 } parsed as block with labels", async () => {
// Semicolons make this a block with labeled statements, not object
const ctx = vm.createContext({});
const code = transpiler.transformSync("{ x: 1; y: 2 }");
const result = await vm.runInContext(code, ctx);
// Block with labels returns last value
expect(result.value).toBe(2);
});
});
});
```
### Part 2: Variable Persistence Tests (Integration with node:vm)
Test that variables persist across multiple REPL evaluations:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL Variable Persistence", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Helper to run multiple REPL lines in sequence
async function runReplSession(lines: string[], context?: object) {
const ctx = vm.createContext(context ?? { console });
const results: any[] = [];
for (const line of lines) {
const transformed = transpiler.transformSync(line);
const result = await vm.runInContext(transformed, ctx);
results.push(result?.value ?? result);
}
return { results, context: ctx };
}
test("var persists across lines", async () => {
const { results, context } = await runReplSession([
"var x = 10",
"x + 5",
"x = 20",
"x",
]);
expect(results[1]).toBe(15); // x + 5
expect(results[3]).toBe(20); // x after reassignment
expect(context.x).toBe(20); // x visible in context
});
test("let persists across lines (hoisted)", async () => {
const { results } = await runReplSession([
"let y = await Promise.resolve(100)",
"y * 2",
]);
expect(results[1]).toBe(200);
});
test("const becomes let, can be reassigned in later lines", async () => {
const { results } = await runReplSession([
"const z = await Promise.resolve(5)",
"z",
// In REPL, const becomes let, so next line can redeclare
"z = 10", // This works because const→let
"z",
]);
expect(results[1]).toBe(5);
expect(results[3]).toBe(10);
});
test("function declarations persist", async () => {
const { results, context } = await runReplSession([
"await 1; function add(a, b) { return a + b; }",
"add(2, 3)",
"function multiply(a, b) { return a * b; }", // no await
"multiply(4, 5)",
]);
expect(results[1]).toBe(5);
expect(results[3]).toBe(20);
expect(typeof context.add).toBe("function");
expect(typeof context.multiply).toBe("function");
});
test("class declarations persist", async () => {
const { results, context } = await runReplSession([
"await 1; class Counter { constructor() { this.count = 0; } inc() { this.count++; } }",
"const c = new Counter()",
"c.inc(); c.inc(); c.count",
]);
expect(results[2]).toBe(2);
expect(typeof context.Counter).toBe("function");
});
test("complex session with mixed declarations", async () => {
const { results } = await runReplSession([
"var total = 0",
"async function addAsync(n) { return total += await Promise.resolve(n); }",
"await addAsync(10)",
"await addAsync(20)",
"total",
]);
expect(results[2]).toBe(10);
expect(results[3]).toBe(30);
expect(results[4]).toBe(30);
});
test("destructuring assignment persists", async () => {
const { results, context } = await runReplSession([
"var { a, b } = await Promise.resolve({ a: 1, b: 2 })",
"a + b",
"var [x, y, z] = [10, 20, 30]",
"x + y + z",
]);
expect(results[1]).toBe(3);
expect(results[3]).toBe(60);
expect(context.a).toBe(1);
expect(context.x).toBe(10);
});
});
```
### Part 3: eval() Scoping Semantics Tests
Test that REPL behaves like eval() with proper scoping:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL eval() Scoping Semantics", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
test("var hoists to global context", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("var globalVar = 42");
await vm.runInContext(code, ctx);
expect(ctx.globalVar).toBe(42);
});
test("let/const hoisted for REPL but scoped correctly", async () => {
const ctx = vm.createContext({});
// With await, let is hoisted outside async wrapper
const code1 = transpiler.transformSync("let x = await 1");
await vm.runInContext(code1, ctx);
expect(ctx.x).toBe(1);
// Without await, let behavior depends on implementation
const code2 = transpiler.transformSync("let y = 2");
await vm.runInContext(code2, ctx);
// y should still be accessible in REPL context
});
test("block-scoped let does NOT leak", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("if (await true) { let blockScoped = 1; }");
await vm.runInContext(code, ctx);
// blockScoped should NOT be visible in context
expect(ctx.blockScoped).toBeUndefined();
});
test("function in block hoists with var (sloppy mode)", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("if (await true) { function blockFn() { return 42; } }");
await vm.runInContext(code, ctx);
// In sloppy mode, function in block hoists to function scope
expect(typeof ctx.blockFn).toBe("function");
expect(ctx.blockFn()).toBe(42);
});
test("this binding in function declarations", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("await 1; function greet() { return 'hello'; }");
await vm.runInContext(code, ctx);
// Function should be assigned to this (context) for REPL persistence
expect(ctx.greet()).toBe("hello");
});
test("async function expression captures result", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("await (async () => { return 42; })()");
const result = await vm.runInContext(code, ctx);
expect(result.value).toBe(42);
});
test("Promise result NOT auto-awaited due to { value: } wrapper", async () => {
const ctx = vm.createContext({});
// Without wrapper, result would be auto-awaited
// With { value: } wrapper, we get the Promise object
const code = transpiler.transformSync("await Promise.resolve(Promise.resolve(42))");
const result = await vm.runInContext(code, ctx);
// The inner Promise should be in value, not auto-resolved
expect(result.value).toBeInstanceOf(Promise);
expect(await result.value).toBe(42);
});
});
```
### Part 4: Edge Cases and Error Handling
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL Edge Cases", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
test("empty input", () => {
const result = transpiler.transformSync("");
expect(result).toBe("");
});
test("whitespace only", () => {
const result = transpiler.transformSync(" \n\t ");
expect(result.trim()).toBe("");
});
test("comment only", () => {
const result = transpiler.transformSync("// just a comment");
expect(result).toContain("// just a comment");
});
test("multiline input", () => {
const input = `
var x = await 1;
var y = await 2;
x + y
`;
const result = transpiler.transformSync(input);
expect(result).toContain("var x");
expect(result).toContain("var y");
expect(result).toContain("async");
});
test("TypeScript syntax", () => {
const input = "const x: number = await Promise.resolve(42)";
const result = transpiler.transformSync(input);
expect(result).not.toContain(": number"); // Types stripped
expect(result).toContain("let x");
});
test("JSX in REPL", () => {
const input = "await Promise.resolve(<div>Hello</div>)";
const result = transpiler.transformSync(input);
expect(result).toContain("async");
});
test("import expression (dynamic)", () => {
// Dynamic imports should work fine
const input = "await import('fs')";
const result = transpiler.transformSync(input);
expect(result).toContain("import");
});
test("nested await expressions", () => {
const input = "await (await Promise.resolve(Promise.resolve(1)))";
const result = transpiler.transformSync(input);
expect(result).toContain("{ value:");
});
test("for-await-of", () => {
const input = `
async function* gen() { yield 1; yield 2; }
for await (const x of gen()) { console.log(x); }
`;
const result = transpiler.transformSync(input);
expect(result).toContain("var gen");
expect(result).toContain("for await");
});
});
```
---
## Verification Plan
### 1. Build and Basic Tests
```bash
# Build Bun with changes
bun bd
# Run the REPL transform tests
bun bd test test/js/bun/transpiler/repl-transform.test.ts
```
### 2. Manual Transform Output Verification
```typescript
// test-repl-manual.ts
const t = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Object literal
console.log("Object literal:");
console.log(t.transformSync("{a: 1}"));
// Expected: contains "({a: 1})"
// Basic await
console.log("\nBasic await:");
console.log(t.transformSync("await 0"));
// Expected: (async () => { return { value: (await 0) } })()
// Variable hoisting
console.log("\nVar hoisting:");
console.log(t.transformSync("var x = await 1"));
// Expected: var x; (async () => { void (x = await 1) })()
// const → let
console.log("\nConst to let:");
console.log(t.transformSync("const x = await 1"));
// Expected: let x; (async () => { void (x = await 1) })()
// Function hoisting
console.log("\nFunction:");
console.log(t.transformSync("await 0; function foo() {}"));
// Expected: var foo; (async () => { await 0; this.foo = foo; function foo() {} })()
```
### 3. Full REPL Session Simulation
```typescript
// test-repl-session.ts
import vm from "node:vm";
const t = new Bun.Transpiler({ loader: "tsx", replMode: true });
const ctx = vm.createContext({ console, Promise });
async function repl(code: string) {
const transformed = t.transformSync(code);
console.log(`> ${code}`);
console.log(`[transformed]: ${transformed}`);
const result = await vm.runInContext(transformed, ctx);
console.log(`= ${JSON.stringify(result?.value ?? result)}\n`);
return result?.value ?? result;
}
// Test session
await repl("var counter = 0");
await repl("function increment() { return ++counter; }");
await repl("increment()"); // Should be 1
await repl("increment()"); // Should be 2
await repl("counter"); // Should be 2
await repl("const data = await Promise.resolve({ x: 10, y: 20 })");
await repl("data.x + data.y"); // Should be 30
await repl("class Point { constructor(x, y) { this.x = x; this.y = y; } }");
await repl("const p = new Point(3, 4)");
await repl("Math.sqrt(p.x**2 + p.y**2)"); // Should be 5
```
### 4. Verify No Regressions
```bash
# Run existing transpiler tests
bun bd test test/js/bun/transpiler/
# Run existing vm tests
bun bd test test/js/node/vm/
```
### 5. Cross-check with Node.js (Optional)
Compare transform outputs with Node.js's `processTopLevelAwait`:
```typescript
// Compare a few key transforms with Node.js output
const cases = [
"await 0",
"var x = await 1",
"await 0; function foo() {}",
];
// Verify Bun output matches Node.js patterns
```
</claude-plan>
2026-01-19 00:32:55 -08:00
autofix-ci[bot]
b001c9afc0
[autofix.ci] apply automated fixes
2026-01-19 08:26:29 +00:00
Jarred Sumner
5a0705348b
feat(transpiler): add replMode option for REPL transforms
...
Add a new `replMode` option to Bun.Transpiler that transforms code for
interactive REPL evaluation:
- Wraps expressions in `{ value: expr }` for result capture
- Uses sync/async IIFE wrappers to avoid parentheses around objects
- Hoists var/let/const declarations for persistence across REPL lines
- Converts const to let for REPL mutability
- Hoists function declarations with this.funcName assignment
- Hoists class declarations with var for vm context persistence
- Auto-detects object literals (starting with { without trailing ;)
This enables building a Node.js-compatible REPL using Bun.Transpiler
with vm.runInContext for persistent variable scope.
Usage:
```typescript
const transpiler = new Bun.Transpiler({
loader: "tsx",
replMode: true,
});
const transformed = transpiler.transformSync(userInput);
const result = await vm.runInContext(transformed, context);
console.log(result.value);
```
REPL transforms are extracted into separate repl_transforms.zig module.
Claude-Generated-By: Claude Code (cli/claude-opus-4-5=100%)
Claude-Steers: 8
Claude-Permission-Prompts: 0
Claude-Escapes: 0
Claude-Plan:
<claude-plan>
# REPL Transform Fixes and Node.js Parity
## Current Status
The basic `replMode` option is implemented. This plan covers fixes and parity with Node.js REPL.
## Issues to Fix
### 1. Value Wrapper Has Extra Parentheses (CRITICAL)
**Current output:**
```js
({
__proto__: null,
value: 42
});
```
**Expected behavior (per Node.js):**
- For **non-async expressions**: Node.js returns `null` (no transform) - the REPL evaluates the expression directly
- For **async expressions**: `(async () => { return { __proto__: null, value: (expr) } })()`
**Solution:**
1. For non-async expressions: Don't wrap in `{ value: expr }` - just return the expression as-is
2. For async expressions: The `{ __proto__: null, value: expr }` is already inside the function after `return`, so no outer parens needed
3. Add inner parens around the expression value for clarity: `{ __proto__: null, value: (expr) }`
### 2. Object Literal Disambiguation (CRITICAL)
**Input:** `{a: 1}` or `{foo: await fetch()}`
**Current:** Parsed as block with labeled statement, NOT object literal
**Solution:** Pre-check input at transpiler layer:
- If code starts with `{` and doesn't end with `;`, try parsing as `(_=CODE)`
- If valid, wrap input as `(CODE)` before processing
- This matches Node.js approach in `repl.js` line 411-414
### 3. Class Declarations Don't Persist to VM Context
**Current:** Uses `let ClassName;` hoisting - doesn't become vm context property
**Node.js behavior:** Also uses `let` - this is a known limitation in Node.js too!
Looking at Node.js `await.js` line 31-37:
```js
ClassDeclaration(node, state, c) {
state.prepend(node, `${node.id.name}=`);
ArrayPrototypePush(state.hoistedDeclarationStatements, `let ${node.id.name}; `);
}
```
**Decision:** Use `var` instead of `let` for class hoisting. This makes classes persist to vm context, matching user expectations for REPL behavior. (Different from Node.js which uses `let`)
---
## Implementation Plan
## Usage Example
```typescript
// REPL tool implementation
const transpiler = new Bun.Transpiler({
loader: "tsx",
replMode: true, // NEW OPTION
});
// For each REPL input line:
const transformed = transpiler.transformSync(userInput);
// Execute in persistent VM context
const result = vm.runInContext(transformed, replContext);
// result.value contains the expression result (wrapped to prevent auto-await)
console.log(result.value);
```
## Design Decisions
- **Value wrapper**: Use `{ value: expr }` wrapper like Node.js to prevent auto-awaiting Promise results
- **Static imports**: Keep static imports as-is (Bun handles them natively)
- **Scope**: Full Node.js REPL transform parity
---
## Node.js REPL Transform Behavior (Reference)
From `vendor/node/lib/internal/repl/await.js`:
### 1. Object Literal Detection
```javascript
// {a:1} → ({a:1}) when starts with { and no trailing ;
if (/^\s*{/.test(code) && !/;\s*$/.test(code)) {
code = `(${code})`;
}
```
### 2. Top-Level Await Transform
```javascript
// Input: await x
// Output: (async () => { return { value: (await x) } })()
// Input: var x = await 1
// Output: var x; (async () => { void (x = await 1) })()
// Input: const x = await 1
// Output: let x; (async () => { void (x = await 1) })() // const→let
// Input: function foo() {} (with await somewhere)
// Output: var foo; (async () => { this.foo = foo; function foo() {} })()
// Input: class Foo {} (with await somewhere)
// Output: let Foo; (async () => { Foo=class Foo {} })()
```
### 3. Transform Skipping
Returns `null` (no transform) when:
- No `await` expression present at top level
- Top-level `return` statement exists
- Code is inside async functions/arrow functions/class methods
---
## Implementation Plan
### Fix 1: Remove Extra Parentheses from Value Wrapper
**Problem:** The printer adds `()` around objects at statement start to disambiguate from blocks.
**Solution:** Always use an IIFE wrapper (sync or async) so the object is after `return`:
```js
// Non-async expression (current - BAD)
({ __proto__: null, value: 42 });
// Non-async expression (fixed - GOOD)
(() => { return { __proto__: null, value: 42 } })()
// Non-async with hoisting (fixed - GOOD)
var x;
(() => { void (x = 1); return { __proto__: null, value: x } })()
// Async (already correct)
var x;
(async () => { void (x = await 1); return { __proto__: null, value: x } })()
```
**Files to modify:**
1. `src/ast/P.zig` - `applyReplValueWrapper()` function
**Changes:**
- Remove the simple `{ value: expr }` wrapper approach
- Always use `applyReplAsyncTransform()` style IIFE wrapping, but with `is_async = false` for non-async code
- This ensures the object is always after `return`, avoiding the parentheses issue
- Hoisting still works for both cases
### Fix 2: Object Literal Disambiguation
**File:** `src/bun.js/api/JSTranspiler.zig` - Before parsing
Add pre-processing check:
```zig
// In transformSync, before parsing:
if (config.repl_mode) {
// Check if input looks like object literal: starts with { and doesn't end with ;
if (startsWithBrace(source) and !endsWithSemicolon(source)) {
// Try parsing as expression by wrapping: _=(CODE)
// If valid, wrap input as (CODE)
source = wrapAsExpression(source);
}
}
```
This matches Node.js `isObjectLiteral()` check in `repl/utils.js:786-789`:
```js
function isObjectLiteral(code) {
return /^\s*{/.test(code) && !/;\s*$/.test(code);
}
```
### Fix 3: Class Declaration Persistence
**File:** `src/ast/P.zig` - `applyReplAsyncTransform()` in the class handling section
Change from:
```zig
// let Foo; (hoisted)
try hoisted_stmts.append(p.s(S.Local{ .kind = .k_let, ... }));
```
To:
```zig
// var Foo; (hoisted) - use var so it becomes context property
try hoisted_stmts.append(p.s(S.Local{ .kind = .k_var, ... }));
// Also add: this.Foo = Foo; assignment after class declaration
```
---
## Node.js REPL Test Cases to Match
From `vendor/node/test/parallel/test-repl-preprocess-top-level-await.js`:
| Input | Expected Output |
|-------|-----------------|
| `await 0` | `(async () => { return { value: (await 0) } })()` |
| `var a = await 1` | `var a; (async () => { void (a = await 1) })()` |
| `let a = await 1` | `let a; (async () => { void (a = await 1) })()` |
| `const a = await 1` | `let a; (async () => { void (a = await 1) })()` |
| `await 0; function foo() {}` | `var foo; (async () => { await 0; this.foo = foo; function foo() {} })()` |
| `await 0; class Foo {}` | `let Foo; (async () => { await 0; Foo=class Foo {} })()` |
| `var {a} = {a:1}, [b] = [1]` | `var a, b; (async () => { void ( ({a} = {a:1}), ([b] = [1])) })()` |
---
## Files to Modify
| File | Changes |
|------|---------|
| `src/ast/P.zig` | Fix value wrapper format, fix class hoisting to use var |
| `src/bun.js/api/JSTranspiler.zig` | Add object literal pre-check |
| `src/ast/js_printer.zig` | May need to check object literal printing |
| `test/js/bun/transpiler/repl-transform.test.ts` | Update tests for exact Node.js parity |
---
## Verification
1. Run Node.js preprocess test cases through Bun's transpiler
2. Verify output matches Node.js exactly (or functionally equivalent)
3. Test with vm.runInContext for variable persistence
4. Test object literal inputs: `{a: 1}`, `{foo: await bar()}`
---
## DEPRECATED - Previous Implementation (Already Done)
### 1. Add `replMode` to Bun.Transpiler API
**File**: `src/bun.js/api/JSTranspiler.zig`
Add to `Config` struct (around line 27-44):
```zig
pub const Config = struct {
// ... existing fields ...
repl_mode: bool = false,
```
Parse the option in `Config.fromJS()` (around line 420-430):
```zig
if (try object.getBooleanLoose(globalThis, "replMode")) |flag| {
this.repl_mode = flag;
}
```
Apply the option in `constructor()` (around line 714-721):
```zig
transpiler.options.repl_mode = config.repl_mode;
```
### 2. Add Feature Flag to Runtime
**File**: `src/runtime.zig` (in `Runtime.Features`)
```zig
/// REPL mode: transforms code for interactive evaluation
/// - Wraps lone object literals `{...}` in parentheses
/// - Hoists variable declarations for REPL persistence
/// - Wraps last expression in { value: expr } for result capture
/// - Assigns functions to context for persistence
repl_mode: bool = false,
```
### 3. Add to BundleOptions
**File**: `src/options.zig`
Add to `BundleOptions` struct:
```zig
repl_mode: bool = false,
```
### 4. Implement REPL Transforms in Parser
**File**: `src/ast/P.zig`
#### 4a. Object Literal Detection (Parser-Level)
In REPL mode, the parser should prefer interpreting ambiguous `{...}` as object literals instead of blocks.
**Location**: `src/ast/parseStmt.zig` in statement parsing
When `repl_mode` is true and the parser sees `{` at the start of a statement:
1. Try parsing as expression statement (object literal) first
2. If that fails, fall back to block statement
This is similar to how JavaScript engines handle REPL input. The parser already has the infrastructure to do this - we just need to change the precedence in REPL mode.
```zig
// In parseStmt when repl_mode is true and we see '{'
if (p.options.features.repl_mode and p.token.tag == .t_open_brace) {
// Try parsing as expression first
const saved_state = p.saveState();
if (p.tryParseExpressionStatement()) |expr_stmt| {
return expr_stmt;
}
p.restoreState(saved_state);
// Fall back to block statement
return p.parseBlockStatement();
}
```
This handles:
- `{a: 1}` → parsed as object literal expression
- `{a: 1, b: 2}` → parsed as object literal expression
- `{ let x = 1; }` → fails as expression, parsed as block
- `{ label: break label; }` → fails as expression (break not valid in object), parsed as block
#### 4b. REPL Transform Pass (in toAST after visiting)
Add a new function `applyReplTransforms()` that:
1. **Detect if transform is needed**: Walk AST to check for top-level `await`
2. **Skip transform when**:
- No `await` at top level
- Top-level `return` statement exists
3. **When transform IS needed**:
- Wrap entire code in `(async () => { ... })()`
- Hoist variable declarations outside the async wrapper
- Convert `const` to `let` for persistence
- Wrap last expression in `return { value: (expr) }`
- Handle function declarations (assign to `this`)
- Handle class declarations (hoist as `let`)
**Key Logic:**
```zig
fn applyReplTransforms(p: *Parser, stmts: []Stmt) ![]Stmt {
// 1. Check for top-level await
const has_await = p.hasTopLevelAwait(stmts);
const has_return = p.hasTopLevelReturn(stmts);
if (!has_await or has_return) {
// Just wrap last expression, no async wrapper needed
return p.wrapLastExpression(stmts);
}
// 2. Collect declarations to hoist
var hoisted = std.ArrayList(Stmt).init(p.allocator);
var inner_stmts = std.ArrayList(Stmt).init(p.allocator);
for (stmts) |stmt| {
switch (stmt.data) {
.s_local => |local| {
// Hoist declaration, convert const→let
try hoisted.append(p.createHoistedDecl(local));
// Add assignment expression to inner
try inner_stmts.append(p.createAssignmentExpr(local));
},
.s_function => |func| {
// var foo; (hoisted)
try hoisted.append(p.createVarDecl(func.name));
// this.foo = foo; function foo() {} (inner)
try inner_stmts.append(p.createThisAssignment(func.name));
try inner_stmts.append(stmt);
},
.s_class => |class| {
// let Foo; (hoisted)
try hoisted.append(p.createLetDecl(class.name));
// Foo = class Foo {} (inner)
try inner_stmts.append(p.createClassAssignment(class));
},
else => try inner_stmts.append(stmt),
}
}
// 3. Wrap last expression in return { value: expr }
p.wrapLastExpressionWithReturn(&inner_stmts);
// 4. Create async IIFE: (async () => { ...inner... })()
const async_iife = p.createAsyncIIFE(inner_stmts.items);
// 5. Combine: hoisted declarations + async IIFE
try hoisted.append(async_iife);
return hoisted.toOwnedSlice();
}
```
### 5. TypeScript Type Definitions
**File**: `packages/bun-types/bun.d.ts`
Add to `TranspilerOptions` interface (around line 1748):
```typescript
interface TranspilerOptions {
// ... existing options ...
/**
* Enable REPL mode transforms:
* - Wraps object literals in parentheses
* - Hoists declarations for REPL persistence
* - Wraps last expression in { value: expr } for result capture
* - Wraps code with await in async IIFE
*/
replMode?: boolean;
}
```
---
## Files to Modify
| File | Changes |
|------|---------|
| `src/bun.js/api/JSTranspiler.zig` | Add `repl_mode` to Config, parse from JS, apply to transpiler |
| `src/runtime.zig` | Add `repl_mode: bool` to `Runtime.Features` |
| `src/options.zig` | Add `repl_mode: bool` to `BundleOptions` |
| `src/ast/P.zig` | REPL transform pass in `toAST()` |
| `src/ast/parseStmt.zig` | Object literal vs block disambiguation in REPL mode |
| `packages/bun-types/bun.d.ts` | Add `replMode?: boolean` to `TranspilerOptions` |
---
## Test Cases
Create test file: `test/js/bun/transpiler/repl-transform.test.ts`
### Part 1: Transform Output Tests (Unit Tests)
Test exact transformation output matches expected patterns:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("Bun.Transpiler replMode - Transform Output", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Based on Node.js test-repl-preprocess-top-level-await.js
const testCases: [string, string | null][] = [
// No await = null (no async transform, but still expression capture)
['0', null],
// Basic await
['await 0', '(async () => { return { value: (await 0) } })()'],
['await 0;', '(async () => { return { value: (await 0) }; })()'],
['(await 0)', '(async () => { return ({ value: (await 0) }) })()'],
// No transform for await inside async functions
['async function foo() { await 0; }', null],
['async () => await 0', null],
['class A { async method() { await 0 } }', null],
// Top-level return = no transform
['await 0; return 0;', null],
// Multiple await - last one gets return wrapper
['await 1; await 2;', '(async () => { await 1; return { value: (await 2) }; })()'],
// Variable hoisting - var
['var a = await 1', 'var a; (async () => { void (a = await 1) })()'],
// Variable hoisting - let
['let a = await 1', 'let a; (async () => { void (a = await 1) })()'],
// Variable hoisting - const becomes let
['const a = await 1', 'let a; (async () => { void (a = await 1) })()'],
// For loop with var - hoist var
['for (var i = 0; i < 1; ++i) { await i }',
'var i; (async () => { for (void (i = 0); i < 1; ++i) { await i } })()'],
// For loop with let - no hoist
['for (let i = 0; i < 1; ++i) { await i }',
'(async () => { for (let i = 0; i < 1; ++i) { await i } })()'],
// Destructuring with var
['var {a} = {a:1}, [b] = [1], {c:{d}} = {c:{d: await 1}}',
'var a, b, d; (async () => { void ( ({a} = {a:1}), ([b] = [1]), ({c:{d}} = {c:{d: await 1}})) })()'],
// Destructuring with let
['let [a, b, c] = await ([1, 2, 3])',
'let a, b, c; (async () => { void ([a, b, c] = await ([1, 2, 3])) })()'],
// Function declarations - assign to this
['await 0; function foo() {}',
'var foo; (async () => { await 0; this.foo = foo; function foo() {} })()'],
// Class declarations - hoist as let
['await 0; class Foo {}',
'let Foo; (async () => { await 0; Foo=class Foo {} })()'],
// Nested scopes
['if (await true) { var a = 1; }',
'var a; (async () => { if (await true) { void (a = 1); } })()'],
['if (await true) { let a = 1; }',
'(async () => { if (await true) { let a = 1; } })()'],
// Mixed declarations
['var a = await 1; let b = 2; const c = 3;',
'var a; let b; let c; (async () => { void (a = await 1); void (b = 2); void (c = 3); })()'],
// for await
['for await (var i of asyncIterable) { i; }',
'var i; (async () => { for await (i of asyncIterable) { i; } })()'],
// for-of with var
['for (var i of [1,2,3]) { await 1; }',
'var i; (async () => { for (i of [1,2,3]) { await 1; } })()'],
// for-in with var
['for (var i in {x:1}) { await 1 }',
'var i; (async () => { for (i in {x:1}) { await 1 } })()'],
// Spread in destructuring
['var { ...rest } = await {}',
'var rest; (async () => { void ({ ...rest } = await {}) })()'],
];
for (const [input, expected] of testCases) {
test(`transform: ${input.slice(0, 40)}...`, () => {
const result = transpiler.transformSync(input);
if (expected === null) {
// No async transform expected, but expression capture may still happen
expect(result).not.toMatch(/^\(async/);
} else {
expect(result.trim()).toBe(expected);
}
});
}
// Object literal detection - parser handles this automatically in REPL mode
describe("object literal vs block disambiguation", () => {
test("{a: 1} parsed as object literal", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{a: 1}");
const result = await vm.runInContext(code, ctx);
// Should evaluate to object, not undefined (block with label)
expect(result.value).toEqual({ a: 1 });
});
test("{a: 1, b: 2} parsed as object literal", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{a: 1, b: 2}");
const result = await vm.runInContext(code, ctx);
expect(result.value).toEqual({ a: 1, b: 2 });
});
test("{ let x = 1; x } parsed as block", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("{ let x = 1; x }");
const result = await vm.runInContext(code, ctx);
// Block returns last expression value
expect(result.value).toBe(1);
});
test("{ x: 1; y: 2 } parsed as block with labels", async () => {
// Semicolons make this a block with labeled statements, not object
const ctx = vm.createContext({});
const code = transpiler.transformSync("{ x: 1; y: 2 }");
const result = await vm.runInContext(code, ctx);
// Block with labels returns last value
expect(result.value).toBe(2);
});
});
});
```
### Part 2: Variable Persistence Tests (Integration with node:vm)
Test that variables persist across multiple REPL evaluations:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL Variable Persistence", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Helper to run multiple REPL lines in sequence
async function runReplSession(lines: string[], context?: object) {
const ctx = vm.createContext(context ?? { console });
const results: any[] = [];
for (const line of lines) {
const transformed = transpiler.transformSync(line);
const result = await vm.runInContext(transformed, ctx);
results.push(result?.value ?? result);
}
return { results, context: ctx };
}
test("var persists across lines", async () => {
const { results, context } = await runReplSession([
"var x = 10",
"x + 5",
"x = 20",
"x",
]);
expect(results[1]).toBe(15); // x + 5
expect(results[3]).toBe(20); // x after reassignment
expect(context.x).toBe(20); // x visible in context
});
test("let persists across lines (hoisted)", async () => {
const { results } = await runReplSession([
"let y = await Promise.resolve(100)",
"y * 2",
]);
expect(results[1]).toBe(200);
});
test("const becomes let, can be reassigned in later lines", async () => {
const { results } = await runReplSession([
"const z = await Promise.resolve(5)",
"z",
// In REPL, const becomes let, so next line can redeclare
"z = 10", // This works because const→let
"z",
]);
expect(results[1]).toBe(5);
expect(results[3]).toBe(10);
});
test("function declarations persist", async () => {
const { results, context } = await runReplSession([
"await 1; function add(a, b) { return a + b; }",
"add(2, 3)",
"function multiply(a, b) { return a * b; }", // no await
"multiply(4, 5)",
]);
expect(results[1]).toBe(5);
expect(results[3]).toBe(20);
expect(typeof context.add).toBe("function");
expect(typeof context.multiply).toBe("function");
});
test("class declarations persist", async () => {
const { results, context } = await runReplSession([
"await 1; class Counter { constructor() { this.count = 0; } inc() { this.count++; } }",
"const c = new Counter()",
"c.inc(); c.inc(); c.count",
]);
expect(results[2]).toBe(2);
expect(typeof context.Counter).toBe("function");
});
test("complex session with mixed declarations", async () => {
const { results } = await runReplSession([
"var total = 0",
"async function addAsync(n) { return total += await Promise.resolve(n); }",
"await addAsync(10)",
"await addAsync(20)",
"total",
]);
expect(results[2]).toBe(10);
expect(results[3]).toBe(30);
expect(results[4]).toBe(30);
});
test("destructuring assignment persists", async () => {
const { results, context } = await runReplSession([
"var { a, b } = await Promise.resolve({ a: 1, b: 2 })",
"a + b",
"var [x, y, z] = [10, 20, 30]",
"x + y + z",
]);
expect(results[1]).toBe(3);
expect(results[3]).toBe(60);
expect(context.a).toBe(1);
expect(context.x).toBe(10);
});
});
```
### Part 3: eval() Scoping Semantics Tests
Test that REPL behaves like eval() with proper scoping:
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL eval() Scoping Semantics", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
test("var hoists to global context", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("var globalVar = 42");
await vm.runInContext(code, ctx);
expect(ctx.globalVar).toBe(42);
});
test("let/const hoisted for REPL but scoped correctly", async () => {
const ctx = vm.createContext({});
// With await, let is hoisted outside async wrapper
const code1 = transpiler.transformSync("let x = await 1");
await vm.runInContext(code1, ctx);
expect(ctx.x).toBe(1);
// Without await, let behavior depends on implementation
const code2 = transpiler.transformSync("let y = 2");
await vm.runInContext(code2, ctx);
// y should still be accessible in REPL context
});
test("block-scoped let does NOT leak", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("if (await true) { let blockScoped = 1; }");
await vm.runInContext(code, ctx);
// blockScoped should NOT be visible in context
expect(ctx.blockScoped).toBeUndefined();
});
test("function in block hoists with var (sloppy mode)", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("if (await true) { function blockFn() { return 42; } }");
await vm.runInContext(code, ctx);
// In sloppy mode, function in block hoists to function scope
expect(typeof ctx.blockFn).toBe("function");
expect(ctx.blockFn()).toBe(42);
});
test("this binding in function declarations", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("await 1; function greet() { return 'hello'; }");
await vm.runInContext(code, ctx);
// Function should be assigned to this (context) for REPL persistence
expect(ctx.greet()).toBe("hello");
});
test("async function expression captures result", async () => {
const ctx = vm.createContext({});
const code = transpiler.transformSync("await (async () => { return 42; })()");
const result = await vm.runInContext(code, ctx);
expect(result.value).toBe(42);
});
test("Promise result NOT auto-awaited due to { value: } wrapper", async () => {
const ctx = vm.createContext({});
// Without wrapper, result would be auto-awaited
// With { value: } wrapper, we get the Promise object
const code = transpiler.transformSync("await Promise.resolve(Promise.resolve(42))");
const result = await vm.runInContext(code, ctx);
// The inner Promise should be in value, not auto-resolved
expect(result.value).toBeInstanceOf(Promise);
expect(await result.value).toBe(42);
});
});
```
### Part 4: Edge Cases and Error Handling
```typescript
import { expect, test, describe } from "bun:test";
import vm from "node:vm";
describe("REPL Edge Cases", () => {
const transpiler = new Bun.Transpiler({ loader: "tsx", replMode: true });
test("empty input", () => {
const result = transpiler.transformSync("");
expect(result).toBe("");
});
test("whitespace only", () => {
const result = transpiler.transformSync(" \n\t ");
expect(result.trim()).toBe("");
});
test("comment only", () => {
const result = transpiler.transformSync("// just a comment");
expect(result).toContain("// just a comment");
});
test("multiline input", () => {
const input = `
var x = await 1;
var y = await 2;
x + y
`;
const result = transpiler.transformSync(input);
expect(result).toContain("var x");
expect(result).toContain("var y");
expect(result).toContain("async");
});
test("TypeScript syntax", () => {
const input = "const x: number = await Promise.resolve(42)";
const result = transpiler.transformSync(input);
expect(result).not.toContain(": number"); // Types stripped
expect(result).toContain("let x");
});
test("JSX in REPL", () => {
const input = "await Promise.resolve(<div>Hello</div>)";
const result = transpiler.transformSync(input);
expect(result).toContain("async");
});
test("import expression (dynamic)", () => {
// Dynamic imports should work fine
const input = "await import('fs')";
const result = transpiler.transformSync(input);
expect(result).toContain("import");
});
test("nested await expressions", () => {
const input = "await (await Promise.resolve(Promise.resolve(1)))";
const result = transpiler.transformSync(input);
expect(result).toContain("{ value:");
});
test("for-await-of", () => {
const input = `
async function* gen() { yield 1; yield 2; }
for await (const x of gen()) { console.log(x); }
`;
const result = transpiler.transformSync(input);
expect(result).toContain("var gen");
expect(result).toContain("for await");
});
});
```
---
## Verification Plan
### 1. Build and Basic Tests
```bash
# Build Bun with changes
bun bd
# Run the REPL transform tests
bun bd test test/js/bun/transpiler/repl-transform.test.ts
```
### 2. Manual Transform Output Verification
```typescript
// test-repl-manual.ts
const t = new Bun.Transpiler({ loader: "tsx", replMode: true });
// Object literal
console.log("Object literal:");
console.log(t.transformSync("{a: 1}"));
// Expected: contains "({a: 1})"
// Basic await
console.log("\nBasic await:");
console.log(t.transformSync("await 0"));
// Expected: (async () => { return { value: (await 0) } })()
// Variable hoisting
console.log("\nVar hoisting:");
console.log(t.transformSync("var x = await 1"));
// Expected: var x; (async () => { void (x = await 1) })()
// const → let
console.log("\nConst to let:");
console.log(t.transformSync("const x = await 1"));
// Expected: let x; (async () => { void (x = await 1) })()
// Function hoisting
console.log("\nFunction:");
console.log(t.transformSync("await 0; function foo() {}"));
// Expected: var foo; (async () => { await 0; this.foo = foo; function foo() {} })()
```
### 3. Full REPL Session Simulation
```typescript
// test-repl-session.ts
import vm from "node:vm";
const t = new Bun.Transpiler({ loader: "tsx", replMode: true });
const ctx = vm.createContext({ console, Promise });
async function repl(code: string) {
const transformed = t.transformSync(code);
console.log(`> ${code}`);
console.log(`[transformed]: ${transformed}`);
const result = await vm.runInContext(transformed, ctx);
console.log(`= ${JSON.stringify(result?.value ?? result)}\n`);
return result?.value ?? result;
}
// Test session
await repl("var counter = 0");
await repl("function increment() { return ++counter; }");
await repl("increment()"); // Should be 1
await repl("increment()"); // Should be 2
await repl("counter"); // Should be 2
await repl("const data = await Promise.resolve({ x: 10, y: 20 })");
await repl("data.x + data.y"); // Should be 30
await repl("class Point { constructor(x, y) { this.x = x; this.y = y; } }");
await repl("const p = new Point(3, 4)");
await repl("Math.sqrt(p.x**2 + p.y**2)"); // Should be 5
```
### 4. Verify No Regressions
```bash
# Run existing transpiler tests
bun bd test test/js/bun/transpiler/
# Run existing vm tests
bun bd test test/js/node/vm/
```
### 5. Cross-check with Node.js (Optional)
Compare transform outputs with Node.js's `processTopLevelAwait`:
```typescript
// Compare a few key transforms with Node.js output
const cases = [
"await 0",
"var x = await 1",
"await 0; function foo() {}",
];
// Verify Bun output matches Node.js patterns
```
</claude-plan>
2026-01-19 00:22:36 -08:00