Compare commits

...

49 Commits

Author SHA1 Message Date
Zack Radisic
e89abda3a5 more tests 2024-12-05 14:50:43 -08:00
Zack Radisic
de7f500658 Only set the NapiModuleMeta when it is a bundler plugin and properly check that the input to napiModule in JS is indeed a bun native plugihn 2024-12-05 14:40:42 -08:00
Zack Radisic
68c2607cb2 Use translate C instead of @cImport 2024-12-04 21:57:41 -08:00
Zack Radisic
f82eba73aa Make BUN_PLUGIN_NAME optional 2024-12-04 20:23:10 -08:00
Zack Radisic
277abc3fb7 lol 2024-12-04 19:50:12 -08:00
Zack Radisic
2414fc0cc5 Don't commit generated files 2024-12-04 19:43:57 -08:00
Zack Radisic
b2392f0e10 Update the Rust bindings a bit 2024-12-04 19:42:05 -08:00
Zack Radisic
c7b63a2250 Nicer/better plugin name for errors 2024-12-04 18:30:47 -08:00
Zack Radisic
5744d60a92 Resolve some comments 2024-12-04 17:06:22 -08:00
Zack Radisic
7d1e892299 Update docs for Rust bindihngs to bundler native plugin 2024-12-04 15:16:21 -08:00
Zack Radisic
a8d17e7ffa stupid windows dll export thing 2024-12-03 18:15:29 -08:00
Zack Radisic
de17576198 okay plz work lol 2024-12-03 17:48:51 -08:00
Zack Radisic
caf0925ebd use other syntax 2024-12-03 17:24:00 -08:00
Zack Radisic
40489a120f msvc does not support stdatomic.h 2024-12-03 17:06:27 -08:00
Zack Radisic
84dc4429fc woops 2024-12-03 16:27:16 -08:00
Zack Radisic
cd4d6f335b lol 2024-12-03 16:23:54 -08:00
Zack Radisic
bbf66ee2be Fix cpp compile on windows 2024-12-03 16:12:50 -08:00
Zack Radisic
a7ffd65fbc Forgot to commit this 2024-12-03 16:07:33 -08:00
Zack Radisic
ea4c4a1478 Attempt to fix windows build again 2024-12-03 16:02:02 -08:00
Zack Radisic
7c16775e60 Attempt to fix windows build 2024-12-03 15:56:19 -08:00
Zack Radisic
cca03b4707 replace strnstr with strtr 2024-12-03 15:27:24 -08:00
Zack Radisic
a962731c77 Fix include path 2024-12-03 01:48:47 -08:00
zackradisic
ca861131ab bun run clang-format 2024-12-03 09:36:21 +00:00
Zack Radisic
160c8f81f8 Merge conflicts 2024-12-03 01:34:20 -08:00
Zack Radisic
a4e8c02351 More changes 2024-12-03 01:19:09 -08:00
Zack Radisic
3059f9e98e update stuff 2024-12-02 19:38:04 -08:00
Zack Radisic
d702005fb1 not necessasry 2024-12-02 17:58:52 -08:00
Zack Radisic
dd62274145 more tests 2024-12-02 17:57:27 -08:00
Zack Radisic
4e2fcbb257 have name in crash handler 2024-11-27 18:38:39 -08:00
Zack Radisic
def9582019 update crash handler to log when a native plugin has crashed 2024-11-27 17:48:07 -08:00
Zack Radisic
7b9716043b Handle Windows invalid file paths 2024-11-26 19:27:19 -08:00
Zack Radisic
f54c14bccd More stuff 2024-11-26 19:06:34 -08:00
Zack Radisic
7c1c682426 yoops forgot to commit this 2024-11-26 14:41:32 -08:00
Zack Radisic
b0b1494e25 Test compilation ctx being freed. 2024-11-26 14:40:30 -08:00
Zack Radisic
f7826f91bb Use headers and test for versioning 2024-11-26 14:01:08 -08:00
Zack Radisic
6a4dd91e95 More tests 2024-11-25 22:56:20 -08:00
Zack Radisic
70afaaafc8 Make it not explode when filter regex is used concurrently 2024-11-25 20:46:50 -08:00
Zack Radisic
86ee27f395 Merge branch 'main' into jarred/native-plugin 2024-11-25 13:54:37 -08:00
Zack Radisic
c337e7783d wip 2024-11-21 17:16:52 -08:00
Zack Radisic
e9b32dfa91 add plugin test 2024-11-21 17:08:34 -08:00
Zack Radisic
d85401996a Make symbol come from napi module 2024-11-21 15:13:20 -08:00
Zack Radisic
ec05dc7eae Merge branch 'main' into jarred/native-plugin 2024-11-19 13:35:11 -08:00
Zack Radisic
7a76b1322a wip 2024-11-19 12:27:22 -08:00
Zack Radisic
8500e8c4ba wip stuff 2024-11-18 12:33:31 -08:00
Zack Radisic
b48c79b8cc wip external in C api plugin 2024-11-18 12:31:52 -08:00
Jarred Sumner
091e331356 Update bun.h 2024-11-03 03:50:59 -08:00
Jarred Sumner
c8c304143e Add proof of concept plugin 2024-11-03 03:49:04 -08:00
Jarred Sumner
2595745658 Create bun.h 2024-11-03 00:45:42 -07:00
Jarred Sumner
2042d82e21 Introduce native plugins in Bun.build, starting with onBeforeParse hook 2024-11-03 00:31:47 -07:00
61 changed files with 4310 additions and 137 deletions

View File

@@ -414,6 +414,15 @@ pub fn addBunObject(b: *Build, opts: *BunBuildOptions) *Compile {
}
addInternalPackages(b, obj, opts);
obj.root_module.addImport("build_options", opts.buildOptionsModule(b));
const translate_plugin_api = b.addTranslateC(.{
.root_source_file = b.path("./packages/bun-native-bundler-plugin-api/bundler_plugin.h"),
.target = opts.target,
.optimize = opts.optimize,
.link_libc = true,
});
obj.root_module.addImport("bun-native-bundler-plugin-api", translate_plugin_api.createModule());
return obj;
}

View File

@@ -355,7 +355,7 @@ Bun.build({
{% /callout %}
## Lifecycle callbacks
## Lifecycle hooks
Plugins can register callbacks to be run at various points in the lifecycle of a bundle:
@@ -363,6 +363,8 @@ Plugins can register callbacks to be run at various points in the lifecycle of a
- [`onResolve()`](#onresolve): Run before a module is resolved
- [`onLoad()`](#onload): Run before a module is loaded.
### Reference
A rough overview of the types (please refer to Bun's `bun.d.ts` for the full type definitions):
```ts
@@ -603,3 +605,98 @@ plugin({
```
Note that the `.defer()` function currently has the limitation that it can only be called once per `onLoad` callback.
## Native plugins
{% callout %}
**NOTE** — This is an advanced and experiemental API recommended for plugin developers who are familiar with systems programming and the C ABI. Use with caution.
{% /callout %}
One of the reasons why Bun's bundler is so fast is that it is written in native code and leverages multi-threading to load and parse modules in parallel.
However, one limitation of plugins written in JavaScript is that JavaScript itself is single-threaded.
Native plugins are written as [NAPI](/docs/node-api) modules and can be run on multiple threads. This allows native plugins to run much faster than JavaScript plugins.
In addition, native plugins can skip unnecessary work such as the UTF-8 -> UTF-16 conversion needed to pass strings to JavaScript.
These are the following lifecycle hooks which are available to native plugins:
- [`onBeforeParse()`](#onbeforeparse): Called on any thread before a file is parsed by Bun's bundler.
### Creating a native plugin
Native plugins are NAPI modules which expose lifecycle hooks as C ABI functions.
To create a native plugin, you must export a C ABI function which matches the signature of the native lifecycle hook you want to implement.
#### Example: Rust with napi-rs
First initialize a napi project (see [here](https://napi.rs/docs/introduction/getting-started) for a more comprehensive guide).
Then install Bun's official safe plugin wrapper crate:
```bash
cargo add bun-native-plugin
```
Now you can export an `extern "C" fn` which is the implementation of your plugin:
```rust
#[no_mangle]
extern "C" fn on_before_parse_impl(
args: *const bun_native_plugin::sys::OnBeforeParseArguments,
result: *mut bun_native_plugin::sys::OnBeforeParseResult,
) {
let args = unsafe { &*args };
let result = unsafe { &mut *result };
let mut handle = match bun_native_plugin::OnBeforeParse::from_raw(args, result) {
Ok(handle) => handle,
Err(_) => {
return;
}
};
let source_code = match handle.input_source_code() {
Ok(source_code) => source_code,
Err(_) => {
handle.log_error("Fetching source code failed!");
return;
}
};
let loader = handle.output_loader();
handle.set_output_source_code(source_code.replace("foo", "bar"), loader);
```
Use napi-rs to compile the plugin to a `.node` file, then you can `require()` it from JS and use it:
```js
await Bun.build({
entrypoints: ["index.ts"],
setup(build) {
const myNativePlugin = require("./path/to/plugin.node");
build.onBeforeParse(
{ filter: /\.ts/ },
{ napiModule: myNativePlugin, symbol: "on_before_parse_impl" },
);
},
});
```
### `onBeforeParse`
```ts
onBeforeParse(
args: { filter: RegExp; namespace?: string },
callback: { napiModule: NapiModule; symbol: string; external?: unknown },
): void;
```
This lifecycle callback is run immediately before a file is parsed by Bun's bundler.
As input, it receives the file's contents and can optionally return new source code.
This callback can be called from any thread and so the napi module implementation must be thread-safe.

View File

@@ -0,0 +1,5 @@
[target.aarch64-unknown-linux-musl]
linker = "aarch64-linux-musl-gcc"
rustflags = ["-C", "target-feature=-crt-static"]
[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "target-feature=+crt-static"]

202
packages/bun-build-mdx-rs/.gitignore vendored Normal file
View File

@@ -0,0 +1,202 @@
# Created by https://www.toptal.com/developers/gitignore/api/node
# Edit at https://www.toptal.com/developers/gitignore?templates=node
### Node ###
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
lerna-debug.log*
# Diagnostic reports (https://nodejs.org/api/report.html)
report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
# Runtime data
pids
*.pid
*.seed
*.pid.lock
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
*.lcov
# nyc test coverage
.nyc_output
# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# Bower dependency directory (https://bower.io/)
bower_components
# node-waf configuration
.lock-wscript
# Compiled binary addons (https://nodejs.org/api/addons.html)
build/Release
# Dependency directories
node_modules/
jspm_packages/
# TypeScript v1 declaration files
typings/
# TypeScript cache
*.tsbuildinfo
# Optional npm cache directory
.npm
# Optional eslint cache
.eslintcache
# Microbundle cache
.rpt2_cache/
.rts2_cache_cjs/
.rts2_cache_es/
.rts2_cache_umd/
# Optional REPL history
.node_repl_history
# Output of 'npm pack'
*.tgz
# Yarn Integrity file
.yarn-integrity
# dotenv environment variables file
.env
.env.test
# parcel-bundler cache (https://parceljs.org/)
.cache
# Next.js build output
.next
# Nuxt.js build / generate output
.nuxt
dist
# Gatsby files
.cache/
# Comment in the public line in if your project uses Gatsby and not Next.js
# https://nextjs.org/blog/next-9-1#public-directory-support
# public
# vuepress build output
.vuepress/dist
# Serverless directories
.serverless/
# FuseBox cache
.fusebox/
# DynamoDB Local files
.dynamodb/
# TernJS port file
.tern-port
# Stores VSCode versions used for testing VSCode extensions
.vscode-test
# End of https://www.toptal.com/developers/gitignore/api/node
# Created by https://www.toptal.com/developers/gitignore/api/macos
# Edit at https://www.toptal.com/developers/gitignore?templates=macos
### macOS ###
# General
.DS_Store
.AppleDouble
.LSOverride
# Icon must end with two
Icon
# Thumbnails
._*
# Files that might appear in the root of a volume
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
.TemporaryItems
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
### macOS Patch ###
# iCloud generated files
*.icloud
# End of https://www.toptal.com/developers/gitignore/api/macos
# Created by https://www.toptal.com/developers/gitignore/api/windows
# Edit at https://www.toptal.com/developers/gitignore?templates=windows
### Windows ###
# Windows thumbnail cache files
Thumbs.db
Thumbs.db:encryptable
ehthumbs.db
ehthumbs_vista.db
# Dump file
*.stackdump
# Folder config file
[Dd]esktop.ini
# Recycle Bin used on file shares
$RECYCLE.BIN/
# Windows Installer files
*.cab
*.msi
*.msix
*.msm
*.msp
# Windows shortcuts
*.lnk
# End of https://www.toptal.com/developers/gitignore/api/windows
#Added by cargo
/target
Cargo.lock
.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions
*.node
dist/
index.js
index.d.ts

View File

@@ -0,0 +1,13 @@
target
Cargo.lock
.cargo
.github
npm
.eslintrc
.prettierignore
rustfmt.toml
yarn.lock
*.node
.yarn
__test__
renovate.json

View File

@@ -0,0 +1,21 @@
[package]
edition = "2021"
name = "bun-mdx-rs"
version = "0.0.0"
[lib]
crate-type = ["cdylib"]
[dependencies]
# Default enable napi4 feature, see https://nodejs.org/api/n-api.html#node-api-version-matrix
napi = { version = "2.12.2", default-features = false, features = ["napi4"] }
napi-derive = "2.12.2"
mdxjs = "0.2.11"
bun-native-plugin = { path = "../bun-native-plugin-rs" }
[build-dependencies]
napi-build = "2.0.1"
[profile.release]
lto = true
strip = "symbols"

View File

@@ -0,0 +1,34 @@
# bun-build-mdx-rs
This is a proof of concept for using a third-party native addon in `Bun.build()`.
This uses `mdxjs-rs` to convert MDX to JSX.
TODO: **This needs to be built & published to npm.**
## Building locally:
```sh
cargo build --release
```
```js
import { build } from "bun";
import mdx from "./index.js";
// TODO: This needs to be prebuilt for the current platform
// Probably use a napi-rs template for this
import addon from "./target/release/libmdx_bun.dylib" with { type: "file" };
const results = await build({
entrypoints: ["./hello.jsx"],
plugins: [mdx({ addon })],
minify: true,
outdir: "./dist",
define: {
"process.env.NODE_ENV": JSON.stringify("production"),
},
});
console.log(results);
```

View File

@@ -0,0 +1,7 @@
import test from 'ava'
import { sum } from '../index.js'
test('sum from native', (t) => {
t.is(sum(1, 2), 3)
})

View File

@@ -0,0 +1,5 @@
extern crate napi_build;
fn main() {
napi_build::setup();
}

View File

@@ -0,0 +1,6 @@
import page1 from "./page1.mdx";
import page2 from "./page2.mdx";
import page3 from "./page3.mdx";
import page4 from "./page4.mdx";
console.log(page1, page2, page3, page4);

View File

@@ -0,0 +1,11 @@
# Hello World
This is a sample MDX file that demonstrates various MDX features.
## Components
You can use JSX components directly in MDX:
<Button onClick={() => alert("Hello!")}>Click me</Button>
## Code Blocks

View File

@@ -0,0 +1,11 @@
# Hello World
This is a sample MDX file that demonstrates various MDX features.
## Components
You can use JSX components directly in MDX:
<Button onClick={() => alert("Hello!")}>Click me</Button>
## Code Blocks

View File

@@ -0,0 +1,11 @@
# Hello World
This is a sample MDX file that demonstrates various MDX features.
## Components
You can use JSX components directly in MDX:
<Button onClick={() => alert("Hello!")}>Click me</Button>
## Code Blocks

View File

@@ -0,0 +1,11 @@
# Hello World
This is a sample MDX file that demonstrates various MDX features.
## Components
You can use JSX components directly in MDX:
<Button onClick={() => alert("Hello!")}>Click me</Button>
## Code Blocks

View File

@@ -0,0 +1,3 @@
# `bun-mdx-rs-darwin-arm64`
This is the **aarch64-apple-darwin** binary for `bun-mdx-rs`

View File

@@ -0,0 +1,18 @@
{
"name": "bun-mdx-rs-darwin-arm64",
"version": "0.0.0",
"os": [
"darwin"
],
"cpu": [
"arm64"
],
"main": "bun-mdx-rs.darwin-arm64.node",
"files": [
"bun-mdx-rs.darwin-arm64.node"
],
"license": "MIT",
"engines": {
"node": ">= 10"
}
}

View File

@@ -0,0 +1,3 @@
# `bun-mdx-rs-darwin-x64`
This is the **x86_64-apple-darwin** binary for `bun-mdx-rs`

View File

@@ -0,0 +1,18 @@
{
"name": "bun-mdx-rs-darwin-x64",
"version": "0.0.0",
"os": [
"darwin"
],
"cpu": [
"x64"
],
"main": "bun-mdx-rs.darwin-x64.node",
"files": [
"bun-mdx-rs.darwin-x64.node"
],
"license": "MIT",
"engines": {
"node": ">= 10"
}
}

View File

@@ -0,0 +1,3 @@
# `bun-mdx-rs-linux-arm64-gnu`
This is the **aarch64-unknown-linux-gnu** binary for `bun-mdx-rs`

View File

@@ -0,0 +1,21 @@
{
"name": "bun-mdx-rs-linux-arm64-gnu",
"version": "0.0.0",
"os": [
"linux"
],
"cpu": [
"arm64"
],
"main": "bun-mdx-rs.linux-arm64-gnu.node",
"files": [
"bun-mdx-rs.linux-arm64-gnu.node"
],
"license": "MIT",
"engines": {
"node": ">= 10"
},
"libc": [
"glibc"
]
}

View File

@@ -0,0 +1,3 @@
# `bun-mdx-rs-linux-arm64-musl`
This is the **aarch64-unknown-linux-musl** binary for `bun-mdx-rs`

View File

@@ -0,0 +1,21 @@
{
"name": "bun-mdx-rs-linux-arm64-musl",
"version": "0.0.0",
"os": [
"linux"
],
"cpu": [
"arm64"
],
"main": "bun-mdx-rs.linux-arm64-musl.node",
"files": [
"bun-mdx-rs.linux-arm64-musl.node"
],
"license": "MIT",
"engines": {
"node": ">= 10"
},
"libc": [
"musl"
]
}

View File

@@ -0,0 +1,3 @@
# `bun-mdx-rs-linux-x64-gnu`
This is the **x86_64-unknown-linux-gnu** binary for `bun-mdx-rs`

View File

@@ -0,0 +1,21 @@
{
"name": "bun-mdx-rs-linux-x64-gnu",
"version": "0.0.0",
"os": [
"linux"
],
"cpu": [
"x64"
],
"main": "bun-mdx-rs.linux-x64-gnu.node",
"files": [
"bun-mdx-rs.linux-x64-gnu.node"
],
"license": "MIT",
"engines": {
"node": ">= 10"
},
"libc": [
"glibc"
]
}

View File

@@ -0,0 +1,3 @@
# `bun-mdx-rs-linux-x64-musl`
This is the **x86_64-unknown-linux-musl** binary for `bun-mdx-rs`

View File

@@ -0,0 +1,21 @@
{
"name": "bun-mdx-rs-linux-x64-musl",
"version": "0.0.0",
"os": [
"linux"
],
"cpu": [
"x64"
],
"main": "bun-mdx-rs.linux-x64-musl.node",
"files": [
"bun-mdx-rs.linux-x64-musl.node"
],
"license": "MIT",
"engines": {
"node": ">= 10"
},
"libc": [
"musl"
]
}

View File

@@ -0,0 +1,3 @@
# `bun-mdx-rs-win32-x64-msvc`
This is the **x86_64-pc-windows-msvc** binary for `bun-mdx-rs`

View File

@@ -0,0 +1,18 @@
{
"name": "bun-mdx-rs-win32-x64-msvc",
"version": "0.0.0",
"os": [
"win32"
],
"cpu": [
"x64"
],
"main": "bun-mdx-rs.win32-x64-msvc.node",
"files": [
"bun-mdx-rs.win32-x64-msvc.node"
],
"license": "MIT",
"engines": {
"node": ">= 10"
}
}

View File

@@ -0,0 +1,37 @@
{
"name": "bun-mdx-rs",
"version": "0.0.0",
"main": "index.js",
"types": "index.d.ts",
"napi": {
"name": "bun-mdx-rs",
"triples": {
"additional": [
"aarch64-apple-darwin",
"aarch64-unknown-linux-gnu",
"aarch64-unknown-linux-musl",
"x86_64-unknown-linux-musl"
]
}
},
"license": "MIT",
"devDependencies": {
"@napi-rs/cli": "^2.18.4",
"ava": "^6.0.1"
},
"ava": {
"timeout": "3m"
},
"engines": {
"node": ">= 10"
},
"scripts": {
"artifacts": "napi artifacts",
"build": "napi build --platform --release",
"build:debug": "napi build --platform",
"prepublishOnly": "napi prepublish -t npm",
"test": "ava",
"universal": "napi universal",
"version": "napi version"
}
}

View File

@@ -0,0 +1,2 @@
tab_spaces = 2
edition = "2021"

View File

@@ -0,0 +1,55 @@
use bun_native_plugin::{define_bun_plugin, BunLoader, OnBeforeParse};
use mdxjs::{compile, Options as CompileOptions};
use napi_derive::napi;
#[macro_use]
extern crate napi;
define_bun_plugin!("bun-mdx-rs");
#[no_mangle]
pub extern "C" fn bun_mdx_rs(
args: *const bun_native_plugin::sys::OnBeforeParseArguments,
result: *mut bun_native_plugin::sys::OnBeforeParseResult,
) {
let args = unsafe { &*args };
let mut handle = match OnBeforeParse::from_raw(args, result) {
Ok(handle) => handle,
Err(_) => {
return;
}
};
let source_str = match handle.input_source_code() {
Ok(source_str) => source_str,
Err(_) => {
handle.log_error("Failed to fetch source code");
return;
}
};
let mut options = CompileOptions::gfm();
// Leave it as JSX for Bun to handle
options.jsx = true;
let path = match handle.path() {
Ok(path) => path,
Err(e) => {
handle.log_error(&format!("Failed to get path: {:?}", e));
return;
}
};
options.filepath = Some(path.to_string());
match compile(&source_str, &options) {
Ok(compiled) => {
handle.set_output_source_code(compiled, BunLoader::BUN_LOADER_JSX);
}
Err(_) => {
handle.log_error("Failed to compile MDX");
return;
}
}
}

View File

@@ -0,0 +1,73 @@
#ifndef BUN_NATIVE_BUNDLER_PLUGIN_API_H
#define BUN_NATIVE_BUNDLER_PLUGIN_API_H
#include <stddef.h>
#include <stdint.h>
typedef enum {
BUN_LOADER_JSX = 0,
BUN_LOADER_JS = 1,
BUN_LOADER_TS = 2,
BUN_LOADER_TSX = 3,
BUN_LOADER_CSS = 4,
BUN_LOADER_FILE = 5,
BUN_LOADER_JSON = 6,
BUN_LOADER_TOML = 7,
BUN_LOADER_WASM = 8,
BUN_LOADER_NAPI = 9,
BUN_LOADER_BASE64 = 10,
BUN_LOADER_DATAURL = 11,
BUN_LOADER_TEXT = 12,
} BunLoader;
const BunLoader BUN_LOADER_MAX = BUN_LOADER_TEXT;
typedef struct BunLogOptions {
size_t __struct_size;
const uint8_t *message_ptr;
size_t message_len;
const uint8_t *path_ptr;
size_t path_len;
const uint8_t *source_line_text_ptr;
size_t source_line_text_len;
int8_t level;
int line;
int lineEnd;
int column;
int columnEnd;
} BunLogOptions;
typedef struct {
size_t __struct_size;
void *bun;
const uint8_t *path_ptr;
size_t path_len;
const uint8_t *namespace_ptr;
size_t namespace_len;
uint8_t default_loader;
void *external;
} OnBeforeParseArguments;
typedef struct OnBeforeParseResult {
size_t __struct_size;
uint8_t *source_ptr;
size_t source_len;
uint8_t loader;
int (*fetchSourceCode)(const OnBeforeParseArguments *args,
struct OnBeforeParseResult *result);
void *plugin_source_code_context;
void (*free_plugin_source_code_context)(void *ctx);
void (*log)(const OnBeforeParseArguments *args, BunLogOptions *options);
} OnBeforeParseResult;
typedef enum {
BUN_LOG_LEVEL_VERBOSE = 0,
BUN_LOG_LEVEL_DEBUG = 1,
BUN_LOG_LEVEL_INFO = 2,
BUN_LOG_LEVEL_WARN = 3,
BUN_LOG_LEVEL_ERROR = 4,
} BunLogLevel;
const BunLogLevel BUN_LOG_MAX = BUN_LOG_LEVEL_ERROR;
#endif // BUN_NATIVE_BUNDLER_PLUGIN_API_H

View File

@@ -0,0 +1 @@
target/

286
packages/bun-native-plugin-rs/Cargo.lock generated Normal file
View File

@@ -0,0 +1,286 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 3
[[package]]
name = "aho-corasick"
version = "1.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916"
dependencies = [
"memchr",
]
[[package]]
name = "bindgen"
version = "0.70.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f49d8fed880d473ea71efb9bf597651e77201bdd4893efe54c9e5d65ae04ce6f"
dependencies = [
"bitflags",
"cexpr",
"clang-sys",
"itertools",
"log",
"prettyplease",
"proc-macro2",
"quote",
"regex",
"rustc-hash",
"shlex",
"syn",
]
[[package]]
name = "bitflags"
version = "2.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b048fb63fd8b5923fc5aa7b340d8e156aec7ec02f0c78fa8a6ddc2613f6f71de"
[[package]]
name = "bun-native-plugin"
version = "0.1.0"
dependencies = [
"bindgen",
]
[[package]]
name = "cexpr"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6fac387a98bb7c37292057cffc56d62ecb629900026402633ae9160df93a8766"
dependencies = [
"nom",
]
[[package]]
name = "cfg-if"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
[[package]]
name = "clang-sys"
version = "1.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b023947811758c97c59bf9d1c188fd619ad4718dcaa767947df1cadb14f39f4"
dependencies = [
"glob",
"libc",
"libloading",
]
[[package]]
name = "either"
version = "1.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "60b1af1c220855b6ceac025d3f6ecdd2b7c4894bfe9cd9bda4fbb4bc7c0d4cf0"
[[package]]
name = "glob"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2fabcfbdc87f4758337ca535fb41a6d701b65693ce38287d856d1674551ec9b"
[[package]]
name = "itertools"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "413ee7dfc52ee1a4949ceeb7dbc8a33f2d6c088194d9f922fb8318faf1f01186"
dependencies = [
"either",
]
[[package]]
name = "libc"
version = "0.2.166"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2ccc108bbc0b1331bd061864e7cd823c0cab660bbe6970e66e2c0614decde36"
[[package]]
name = "libloading"
version = "0.8.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4979f22fdb869068da03c9f7528f8297c6fd2606bc3a4affe42e6a823fdb8da4"
dependencies = [
"cfg-if",
"windows-targets",
]
[[package]]
name = "log"
version = "0.4.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7a70ba024b9dc04c27ea2f0c0548feb474ec5c54bba33a7f72f873a39d07b24"
[[package]]
name = "memchr"
version = "2.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78ca9ab1a0babb1e7d5695e3530886289c18cf2f87ec19a575a0abdce112e3a3"
[[package]]
name = "minimal-lexical"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
[[package]]
name = "nom"
version = "7.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
dependencies = [
"memchr",
"minimal-lexical",
]
[[package]]
name = "prettyplease"
version = "0.2.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "64d1ec885c64d0457d564db4ec299b2dae3f9c02808b8ad9c3a089c591b18033"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro2"
version = "1.0.92"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37d3544b3f2748c54e147655edb5025752e2303145b5aefb3c3ea2c78b973bb0"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b5b9d34b8991d19d98081b46eacdd8eb58c6f2b201139f7c5f643cc155a633af"
dependencies = [
"proc-macro2",
]
[[package]]
name = "regex"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b544ef1b4eac5dc2db33ea63606ae9ffcfac26c1416a2806ae0bf5f56b201191"
dependencies = [
"aho-corasick",
"memchr",
"regex-automata",
"regex-syntax",
]
[[package]]
name = "regex-automata"
version = "0.4.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "809e8dc61f6de73b46c85f4c96486310fe304c434cfa43669d7b40f711150908"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b15c43186be67a4fd63bee50d0303afffcef381492ebe2c5d87f324e1b8815c"
[[package]]
name = "rustc-hash"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
[[package]]
name = "shlex"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
[[package]]
name = "syn"
version = "2.0.89"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44d46482f1c1c87acd84dea20c1bf5ebff4c757009ed6bf19cfd36fb10e92c4e"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "adb9e6ca4f869e1180728b7950e35922a7fc6397f7b641499e8f3ef06e50dc83"
[[package]]
name = "windows-targets"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973"
dependencies = [
"windows_aarch64_gnullvm",
"windows_aarch64_msvc",
"windows_i686_gnu",
"windows_i686_gnullvm",
"windows_i686_msvc",
"windows_x86_64_gnu",
"windows_x86_64_gnullvm",
"windows_x86_64_msvc",
]
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3"
[[package]]
name = "windows_aarch64_msvc"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469"
[[package]]
name = "windows_i686_gnu"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b"
[[package]]
name = "windows_i686_gnullvm"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66"
[[package]]
name = "windows_i686_msvc"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66"
[[package]]
name = "windows_x86_64_gnu"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d"
[[package]]
name = "windows_x86_64_msvc"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"

View File

@@ -0,0 +1,7 @@
[package]
name = "bun-native-plugin"
version = "0.1.0"
edition = "2021"
[build-dependencies]
bindgen = "0.70.1"

View File

@@ -0,0 +1,248 @@
> ⚠️ Note: This is an advanced and experimental API recommended only for plugin developers who are familiar with systems proramming and the C ABI. Use with caution.
# Bun Native Plugins
This crate provides a Rustified wrapper over the Bun's native bundler plugin C API.
Some advantages to _native_ bundler plugins as opposed to regular ones implemented in JS:
- Native plugins take full advantage of Bun's parallelized bundler pipeline and run on multiple threads at the same time
- Unlike JS, native plugins don't need to do the UTF-8 <-> UTF-16 source code string conversions
What are native bundler plugins exactly? Precisely, they are NAPI modules which expose a C ABI function which implement a plugin lifecycle hook.
The currently supported lifecycle hooks are:
- `onBeforeParse` (called immediately before a file is parsed, allows you to modify the source code of the file)
## Getting started
Since native bundler plugins are NAPI modules, the easiest way to get started is to create a new [napi-rs](https://github.com/napi-rs/napi-rs) project:
```bash
bun add -g @napi-rs/cli
napi new
```
Then install this crate:
```bash
cargo add bun-native-plugin
```
Now, inside the `lib.rs` file, expose a C ABI function which has the same function signature as the plugin lifecycle hook that you want to implement.
For example, implementing `onBeforeParse`:
```rs
use bun_native_plugin::{define_bun_plugin, OnBeforeParse};
use napi_derive::napi;
/// Define with the name of the plugin
define_bun_plugin!("replace-foo-with-bar");
/// This is necessary for napi-rs to compile this into a proper NAPI module
#[napi]
pub fn register_bun_plugin() {}
/// Use `no_mangle` so that we can reference this symbol by name later
/// when registering this native plugin in JS.
///
/// Here we'll create a dummy plugin which replaces all occurences of
/// `foo` with `bar`
#[no_mangle]
pub extern "C" fn on_before_parse_plugin_impl(
args: *const bun_native_plugin::sys::OnBeforeParseArguments,
result: *mut bun_native_plugin::sys::OnBeforeParseResult,
) {
let args = unsafe { &*args };
// This returns a handle which is a safe wrapper over the raw
// C API.
let mut handle = OnBeforeParse::from_raw(args, result) {
Ok(handle) => handle,
Err(_) => {
// `OnBeforeParse::from_raw` handles error logging
// so it fine to return here.
return;
}
};
let input_source_code = match handle.input_source_code() {
Ok(source_str) => source_str,
Err(_) => {
// If we encounter an error, we must log it so that
// Bun knows this plugin failed.
handle.log_error("Failed to fetch source code!");
return;
}
};
let loader = handle.output_loader();
let output_source_code = source_str.replace("foo", "bar");
handle.set_output_source_code(output_source_code, loader);
}
```
Then compile this NAPI module. If you using napi-rs, the `package.json` should have a `build` script you can run:
```bash
bun run build
```
This will produce a `.node` file in the project directory.
With the compiled NAPI module, you can now register the plugin from JS:
```js
const result = await Bun.build({
entrypoints: ["index.ts"],
plugins: [
{
name: "replace-foo-with-bar",
setup(build) {
const napiModule = require("path/to/napi_module.node");
// Register the `onBeforeParse` hook to run on all `.ts` files.
// We tell it to use function we implemented inside of our `lib.rs` code.
build.onBeforeParse(
{ filter: /\.ts/ },
{ napiModule, symbol: "on_before_parse_plugin_impl" },
);
},
},
],
});
```
## Very important information
### Error handling and panics
It is highly recommended to avoid panicking as this will crash the runtime. Instead, you must handle errors and log them:
```rs
let input_source_code = match handle.input_source_code() {
Ok(source_str) => source_str,
Err(_) => {
// If we encounter an error, we must log it so that
// Bun knows this plugin failed.
handle.log_error("Failed to fetch source code!");
return;
}
};
```
### Passing state to and from JS: `External`
One way to communicate data from your plugin and JS and vice versa is through the NAPI's [External](https://napi.rs/docs/concepts/external) type.
An External in NAPI is like an opaque pointer to data that can be passed to and from JS. Inside your NAPI module, you can retrieve
the pointer and modify the data.
As an example that extends our getting started example above, let's say you wanted to count the number of `foo`'s that the native plugin encounters.
You would expose a NAPI module function which creates this state. Recall that state in native plugins must be threadsafe. This usually means
that your state must be `Sync`:
```rs
struct PluginState {
foo_count: std::sync::atomic::AtomicU32,
}
#[napi]
pub fn create_plugin_state() -> External<PluginState> {
let external = External::new(PluginState {
foo_count: 0,
});
external
}
#[napi]
pub fn get_foo_count(plugin_state: External<PluginState>) -> u32 {
let plugin_state: &PluginState = &plugin_state;
plugin_state.foo_count.load(std::sync::atomic::Ordering::Relaxed)
}
```
When you register your plugin from Javascript, you call the napi module function to create the external and then pass it:
```js
const napiModule = require("path/to/napi_module.node");
const pluginState = napiModule.createPluginState();
const result = await Bun.build({
entrypoints: ["index.ts"],
plugins: [
{
name: "replace-foo-with-bar",
setup(build) {
build.onBeforeParse(
{ filter: /\.ts/ },
{
napiModule,
symbol: "on_before_parse_plugin_impl",
// pass our NAPI external which contains our plugin state here
external: pluginState,
},
);
},
},
],
});
console.log("Total `foo`s encountered: ", pluginState.getFooCount());
```
Finally, from the native implementation of your plugin, you can extract the external:
```rs
pub extern "C" fn on_before_parse_plugin_impl(
args: *const bun_native_plugin::sys::OnBeforeParseArguments,
result: *mut bun_native_plugin::sys::OnBeforeParseResult,
) {
let args = unsafe { &*args };
let mut handle = OnBeforeParse::from_raw(args, result) {
Ok(handle) => handle,
Err(_) => {
// `OnBeforeParse::from_raw` handles error logging
// so it fine to return here.
return;
}
};
let plugin_state: &PluginState =
// This operation is only safe if you pass in an external when registering the plugin.
// If you don't, this could lead to a segfault or access of undefined memory.
match unsafe { handle.external().and_then(|state| state.ok_or(Error::Unknown)) } {
Ok(state) => state,
Err(_) => {
handle.log_error("Failed to get external!");
return;
}
};
// Fetch our source code again
let input_source_code = match handle.input_source_code() {
Ok(source_str) => source_str,
Err(_) => {
handle.log_error("Failed to fetch source code!");
return;
}
};
// Count the number of `foo`s and add it to our state
let foo_count = source_code.matches("foo").count() as u32;
plugin_state.foo_count.fetch_add(foo_count, std::sync::atomic::Ordering::Relaxed);
}
```
### Concurrency
Your `extern "C"` plugin function can be called _on any thread_ at _any time_ and _multiple times at once_.
Therefore, you must design any state management to be threadsafe

View File

@@ -0,0 +1,20 @@
use std::path::PathBuf;
fn main() {
println!("cargo:rustc-link-search=./headers");
let bindings = bindgen::Builder::default()
.header("wrapper.h")
// Add absolute path to headers directory
.clang_arg("-I./headers")
.parse_callbacks(Box::new(bindgen::CargoCallbacks))
.rustified_enum("BunLogLevel")
.rustified_enum("BunLoader")
.generate()
.expect("Unable to generate bindings");
let out_path = PathBuf::from(std::env::var("OUT_DIR").unwrap());
bindings
.write_to_file(out_path.join("bindings.rs"))
.expect("Couldn't write bindings!");
}

View File

@@ -0,0 +1,6 @@
import { join } from "node:path";
const dirname = join(import.meta.dir, "../", "bun-native-bundler-plugin-api");
await Bun.$`rm -rf headers`;
await Bun.$`mkdir -p headers`;
await Bun.$`cp -R ${dirname} headers/bun-native-bundler-plugin-api`;

View File

@@ -0,0 +1,79 @@
#ifndef BUN_NATIVE_BUNDLER_PLUGIN_API_H
#define BUN_NATIVE_BUNDLER_PLUGIN_API_H
#include <stddef.h>
#include <stdint.h>
typedef enum {
BUN_LOADER_JSX = 0,
BUN_LOADER_JS = 1,
BUN_LOADER_TS = 2,
BUN_LOADER_TSX = 3,
BUN_LOADER_CSS = 4,
BUN_LOADER_FILE = 5,
BUN_LOADER_JSON = 6,
BUN_LOADER_TOML = 7,
BUN_LOADER_WASM = 8,
BUN_LOADER_NAPI = 9,
BUN_LOADER_BASE64 = 10,
BUN_LOADER_DATAURL = 11,
BUN_LOADER_TEXT = 12,
BUN_LOADER_BUNSH = 13,
BUN_LOADER_SQLITE = 14,
BUN_LOADER_SQLITE_EMBEDDED = 15
} BunLoader;
const BunLoader BUN_LOADER_MAX = BUN_LOADER_SQLITE_EMBEDDED;
typedef struct BunLogOptions {
size_t __struct_size;
const uint8_t* message_ptr;
size_t message_len;
const uint8_t* path_ptr;
size_t path_len;
const uint8_t* source_line_text_ptr;
size_t source_line_text_len;
int8_t level;
int line;
int lineEnd;
int column;
int columnEnd;
} BunLogOptions;
typedef struct {
size_t __struct_size;
void* bun;
const uint8_t* path_ptr;
size_t path_len;
const uint8_t* namespace_ptr;
size_t namespace_len;
uint8_t default_loader;
void *external;
} OnBeforeParseArguments;
typedef struct OnBeforeParseResult {
size_t __struct_size;
uint8_t* source_ptr;
size_t source_len;
uint8_t loader;
int (*fetchSourceCode)(
const OnBeforeParseArguments* args,
struct OnBeforeParseResult* result
);
void* plugin_source_code_context;
void (*free_plugin_source_code_context)(void* ctx);
void (*log)(const OnBeforeParseArguments* args, BunLogOptions* options);
} OnBeforeParseResult;
typedef enum {
BUN_LOG_LEVEL_VERBOSE = 0,
BUN_LOG_LEVEL_DEBUG = 1,
BUN_LOG_LEVEL_INFO = 2,
BUN_LOG_LEVEL_WARN = 3,
BUN_LOG_LEVEL_ERROR = 4,
} BunLogLevel;
const BunLogLevel BUN_LOG_MAX = BUN_LOG_LEVEL_ERROR;
#endif // BUN_NATIVE_BUNDLER_PLUGIN_API_H

View File

@@ -0,0 +1,627 @@
//! > ⚠️ Note: This is an advanced and experimental API recommended only for plugin developers who are familiar with systems proramming and the C ABI. Use with caution.
//!
//! # Bun Native Plugins
//!
//! This crate provides a Rustified wrapper over the Bun's native bundler plugin C API.
//!
//! Some advantages to _native_ bundler plugins as opposed to regular ones implemented in JS:
//!
//! - Native plugins take full advantage of Bun's parallelized bundler pipeline and run on multiple threads at the same time
//! - Unlike JS, native plugins don't need to do the UTF-8 <-> UTF-16 source code string conversions
//!
//! What are native bundler plugins exactly? Precisely, they are NAPI modules which expose a C ABI function which implement a plugin lifecycle hook.
//!
//! The currently supported lifecycle hooks are:
//!
//! - `onBeforeParse` (called immediately before a file is parsed, allows you to modify the source code of the file)
//!
//! ## Getting started
//!
//! Since native bundler plugins are NAPI modules, the easiest way to get started is to create a new [napi-rs](https://github.com/napi-rs/napi-rs) project:
//!
//! ```bash
//! bun add -g @napi-rs/cli
//! napi new
//! ```
//!
//! Then install this crate:
//!
//! ```bash
//! cargo add bun-native-plugin
//! ```
//!
//! Now, inside the `lib.rs` file, expose a C ABI function which has the same function signature as the plugin lifecycle hook that you want to implement.
//!
//! For example, implementing `onBeforeParse`:
//!
//! ```rust
//! use bun_native_plugin::{OnBeforeParse};
//!
//! /// This is necessary for napi-rs to compile this into a proper NAPI module
//! #[napi]
//! pub fn register_bun_plugin() {}
//!
//! /// Use `no_mangle` so that we can reference this symbol by name later
//! /// when registering this native plugin in JS.
//! ///
//! /// Here we'll create a dummy plugin which replaces all occurences of
//! /// `foo` with `bar`
//! #[no_mangle]
//! pub extern "C" fn on_before_parse_plugin_impl(
//! args: *const bun_native_plugin::sys::OnBeforeParseArguments,
//! result: *mut bun_native_plugin::sys::OnBeforeParseResult,
//! ) {
//! let args = unsafe { &*args };
//! let result = unsafe { &mut *result };
//!
//! // This returns a handle which is a safe wrapper over the raw
//! // C API.
//! let mut handle = OnBeforeParse::from_raw(args, result) {
//! Ok(handle) => handle,
//! Err(_) => {
//! // `OnBeforeParse::from_raw` handles error logging
//! // so it fine to return here.
//! return;
//! }
//! };
//!
//! let input_source_code = match handle.input_source_code() {
//! Ok(source_str) => source_str,
//! Err(_) => {
//! // If we encounter an error, we must log it so that
//! // Bun knows this plugin failed.
//! handle.log_error("Failed to fetch source code!");
//! return;
//! }
//! };
//!
//! let loader = handle.output_loader();
//! let output_source_code = source_str.replace("foo", "bar");
//! handle.set_output_source_code(output_source_code, loader);
//! }
//! ```
//!
//! Then compile this NAPI module. If you using napi-rs, the `package.json` should have a `build` script you can run:
//!
//! ```bash
//! bun run build
//! ```
//!
//! This will produce a `.node` file in the project directory.
//!
//! With the compiled NAPI module, you can now register the plugin from JS:
//!
//! ```js
//! const result = await Bun.build({
//! entrypoints: ["index.ts"],
//! plugins: [
//! {
//! name: "replace-foo-with-bar",
//! setup(build) {
//! const napiModule = require("path/to/napi_module.node");
//!
//! // Register the `onBeforeParse` hook to run on all `.ts` files.
//! // We tell it to use function we implemented inside of our `lib.rs` code.
//! build.onBeforeParse(
//! { filter: /\.ts/ },
//! { napiModule, symbol: "on_before_parse_plugin_impl" },
//! );
//! },
//! },
//! ],
//! });
//! ```
//!
//! ## Very important information
//!
//! ### Error handling and panics
//!
//! It is highly recommended to avoid panicking as this will crash the runtime. Instead, you must handle errors and log them:
//!
//! ```rust
//! let input_source_code = match handle.input_source_code() {
//! Ok(source_str) => source_str,
//! Err(_) => {
//! // If we encounter an error, we must log it so that
//! // Bun knows this plugin failed.
//! handle.log_error("Failed to fetch source code!");
//! return;
//! }
//! };
//! ```
//!
//! ### Passing state to and from JS: `External`
//!
//! One way to communicate data from your plugin and JS and vice versa is through the NAPI's [External](https://napi.rs/docs/concepts/external) type.
//!
//! An External in NAPI is like an opaque pointer to data that can be passed to and from JS. Inside your NAPI module, you can retrieve
//! the pointer and modify the data.
//!
//! As an example that extends our getting started example above, let's say you wanted to count the number of `foo`'s that the native plugin encounters.
//!
//! You would expose a NAPI module function which creates this state. Recall that state in native plugins must be threadsafe. This usually means
//! that your state must be `Sync`:
//!
//! ```rust
//! struct PluginState {
//! foo_count: std::sync::atomic::AtomicU32,
//! }
//!
//! #[napi]
//! pub fn create_plugin_state() -> External<PluginState> {
//! let external = External::new(PluginState {
//! foo_count: 0,
//! });
//!
//! external
//! }
//!
//!
//! #[napi]
//! pub fn get_foo_count(plugin_state: External<PluginState>) -> u32 {
//! let plugin_state: &PluginState = &plugin_state;
//! plugin_state.foo_count.load(std::sync::atomic::Ordering::Relaxed)
//! }
//! ```
//!
//! When you register your plugin from Javascript, you call the napi module function to create the external and then pass it:
//!
//! ```js
//! const napiModule = require("path/to/napi_module.node");
//! const pluginState = napiModule.createPluginState();
//!
//! const result = await Bun.build({
//! entrypoints: ["index.ts"],
//! plugins: [
//! {
//! name: "replace-foo-with-bar",
//! setup(build) {
//! build.onBeforeParse(
//! { filter: /\.ts/ },
//! {
//! napiModule,
//! symbol: "on_before_parse_plugin_impl",
//! // pass our NAPI external which contains our plugin state here
//! external: pluginState,
//! },
//! );
//! },
//! },
//! ],
//! });
//!
//! console.log("Total `foo`s encountered: ", pluginState.getFooCount());
//! ```
//!
//! Finally, from the native implementation of your plugin, you can extract the external:
//!
//! ```rust
//! pub extern "C" fn on_before_parse_plugin_impl(
//! args: *const bun_native_plugin::sys::OnBeforeParseArguments,
//! result: *mut bun_native_plugin::sys::OnBeforeParseResult,
//! ) {
//! let args = unsafe { &*args };
//! let result = unsafe { &mut *result };
//!
//! let mut handle = OnBeforeParse::from_raw(args, result) {
//! Ok(handle) => handle,
//! Err(_) => {
//! // `OnBeforeParse::from_raw` handles error logging
//! // so it fine to return here.
//! return;
//! }
//! };
//!
//! let plugin_state: &PluginState =
//! // This operation is only safe if you pass in an external when registering the plugin.
//! // If you don't, this could lead to a segfault or access of undefined memory.
//! match unsafe { handle.external().and_then(|state| state.ok_or(Error::Unknown)) } {
//! Ok(state) => state,
//! Err(_) => {
//! handle.log_error("Failed to get external!");
//! return;
//! }
//! };
//!
//!
//! // Fetch our source code again
//! let input_source_code = match handle.input_source_code() {
//! Ok(source_str) => source_str,
//! Err(_) => {
//! handle.log_error("Failed to fetch source code!");
//! return;
//! }
//! };
//!
//! // Count the number of `foo`s and add it to our state
//! let foo_count = source_code.matches("foo").count() as u32;
//! plugin_state.foo_count.fetch_add(foo_count, std::sync::atomic::Ordering::Relaxed);
//! }
//! ```
//!
//! ### Concurrency
//!
//! Your `extern "C"` plugin function can be called _on any thread_ at _any time_ and _multiple times at once_.
//!
//! Therefore, you must design any state management to be threadsafe
#![allow(non_upper_case_globals)]
#![allow(non_camel_case_types)]
#![allow(non_snake_case)]
#[repr(transparent)]
pub struct BunPluginName(*const c_char);
impl BunPluginName {
pub const fn new(ptr: *const c_char) -> Self {
Self(ptr)
}
}
#[macro_export]
macro_rules! define_bun_plugin {
($name:expr) => {
pub static BUN_PLUGIN_NAME_STRING: &str = $name;
#[no_mangle]
pub static BUN_PLUGIN_NAME: bun_native_plugin::BunPluginName =
bun_native_plugin::BunPluginName::new(BUN_PLUGIN_NAME_STRING.as_ptr() as *const _);
#[napi]
fn bun_plugin_register() {}
};
}
unsafe impl Sync for BunPluginName {}
use std::{
any::TypeId,
borrow::Cow,
cell::UnsafeCell,
ffi::{c_char, c_void},
str::Utf8Error,
};
pub mod sys {
include!(concat!(env!("OUT_DIR"), "/bindings.rs"));
}
#[repr(C)]
pub struct TaggedObject<T> {
type_id: TypeId,
pub(crate) object: Option<T>,
}
struct SourceCodeContext {
source_ptr: *mut u8,
source_len: usize,
source_cap: usize,
}
extern "C" fn free_plugin_source_code_context(ctx: *mut c_void) {
// SAFETY: The ctx pointer is a pointer to the `SourceCodeContext` struct we allocated.
unsafe {
drop(Box::from_raw(ctx as *mut SourceCodeContext));
}
}
impl Drop for SourceCodeContext {
fn drop(&mut self) {
if !self.source_ptr.is_null() {
// SAFETY: These fields come from a `String` that we allocated.
unsafe {
drop(String::from_raw_parts(
self.source_ptr,
self.source_len,
self.source_cap,
));
}
}
}
}
pub type BunLogLevel = sys::BunLogLevel;
pub type BunLoader = sys::BunLoader;
fn get_from_raw_str<'a>(ptr: *const u8, len: usize) -> Result<Cow<'a, str>> {
let slice: &'a [u8] = unsafe { std::slice::from_raw_parts(ptr, len) };
// Windows allows invalid UTF-16 strings in the filesystem. These get converted to WTF-8 in Zig.
// Meaning the string may contain invalid UTF-8, we'll have to use the safe checked version.
#[cfg(target_os = "windows")]
{
std::str::from_utf8(slice)
.map(Into::into)
.or_else(|_| Ok(String::from_utf8_lossy(slice)))
}
#[cfg(not(target_os = "windows"))]
{
// SAFETY: The source code comes from Zig, which uses UTF-8, so this should be safe.
std::str::from_utf8(slice)
.map(Into::into)
.or_else(|_| Ok(String::from_utf8_lossy(slice)))
}
}
#[derive(Debug, Clone)]
pub enum Error {
Utf8(Utf8Error),
IncompatiblePluginVersion,
ExternalTypeMismatch,
Unknown,
}
pub type Result<T> = std::result::Result<T, Error>;
impl From<Utf8Error> for Error {
fn from(value: Utf8Error) -> Self {
Self::Utf8(value)
}
}
/// A safe handle for the arguments + result struct for the
/// `OnBeforeParse` bundler lifecycle hook.
///
/// This struct acts as a safe wrapper around the raw C API structs
/// (`sys::OnBeforeParseArguments`/`sys::OnBeforeParseResult`) needed to
/// implement the `OnBeforeParse` bundler lifecycle hook.
///
/// To initialize this struct, see the `from_raw` method.
pub struct OnBeforeParse<'a> {
args_raw: &'a sys::OnBeforeParseArguments,
result_raw: *mut sys::OnBeforeParseResult,
compilation_context: *mut SourceCodeContext,
}
impl<'a> OnBeforeParse<'a> {
/// Initialize this struct from references to their raw counterparts.
///
/// This function will do a versioning check to ensure that the plugin
/// is compatible with the current version of Bun. If the plugin is not
/// compatible, it will log an error and return an error result.
///
/// # Example
/// ```rust
/// extern "C" fn on_before_parse_impl(args: *const sys::OnBeforeParseArguments, result: *mut sys::OnBeforeParseResult) {
/// let args = unsafe { &*args };
/// let result = unsafe { &mut *result };
/// let handle = match OnBeforeParse::from_raw(args, result) {
/// Ok(handle) => handle,
/// Err(()) => return,
/// };
/// }
/// ```
pub fn from_raw(
args: &'a sys::OnBeforeParseArguments,
result: *mut sys::OnBeforeParseResult,
) -> Result<Self> {
if args.__struct_size < std::mem::size_of::<sys::OnBeforeParseArguments>()
|| unsafe { (*result).__struct_size } < std::mem::size_of::<sys::OnBeforeParseResult>()
{
let message = "This plugin is not compatible with the current version of Bun.";
let mut log_options = sys::BunLogOptions {
__struct_size: std::mem::size_of::<sys::BunLogOptions>(),
message_ptr: message.as_ptr(),
message_len: message.len(),
path_ptr: args.path_ptr,
path_len: args.path_len,
source_line_text_ptr: std::ptr::null(),
source_line_text_len: 0,
level: BunLogLevel::BUN_LOG_LEVEL_ERROR as i8,
line: 0,
lineEnd: 0,
column: 0,
columnEnd: 0,
};
// SAFETY: The `log` function pointer is guaranteed to be valid by the Bun runtime.
unsafe {
((*result).log.unwrap())(args, &mut log_options);
}
return Err(Error::IncompatiblePluginVersion);
}
Ok(Self {
args_raw: args,
result_raw: result,
compilation_context: std::ptr::null_mut() as *mut _,
})
}
pub fn path(&self) -> Result<Cow<'_, str>> {
get_from_raw_str(self.args_raw.path_ptr, self.args_raw.path_len)
}
pub fn namespace(&self) -> Result<Cow<'_, str>> {
get_from_raw_str(self.args_raw.namespace_ptr, self.args_raw.namespace_len)
}
/// Get the external object from the `OnBeforeParse` arguments.
///
/// The external object is set by the plugin definition inside of JS:
/// ```js
/// await Bun.build({
/// plugins: [
/// {
/// name: "my-plugin",
/// setup(builder) {
/// const native_plugin = require("./native_plugin.node");
/// const external = native_plugin.createExternal();
/// builder.external({ napiModule: native_plugin, symbol: 'onBeforeParse', external });
/// },
/// },
/// ],
/// });
/// ```
///
/// The external object must be created from NAPI for this function to be safe!
///
/// This function will return an error if the external object is not a
/// valid tagged object for the given type.
///
/// This function will return `Ok(None)` if there is no external object
/// set.
///
/// # Example
/// The code to create the external from napi-rs:
/// ```rs
/// #[no_mangle]
/// #[napi]
/// pub fn create_my_external() -> External<MyStruct> {
/// let external = External::new(MyStruct::new());
///
/// external
/// }
/// ```
///
/// The code to extract the external:
/// ```rust
/// let external = match handle.external::<MyStruct>() {
/// Ok(Some(external)) => external,
/// _ => {
/// handle.log_error("Could not get external object.");
/// return;
/// },
/// };
/// ```
pub unsafe fn external<T: 'static + Sync>(&self) -> Result<Option<&'static T>> {
if self.args_raw.external.is_null() {
return Ok(None);
}
let external: *mut TaggedObject<T> = self.args_raw.external as *mut TaggedObject<T>;
unsafe {
if (*external).type_id != TypeId::of::<T>() {
return Err(Error::ExternalTypeMismatch);
}
Ok((*external).object.as_ref())
}
}
/// The same as [`crate::bun_native_plugin::OnBeforeParse::external`], but returns a mutable reference.
///
/// This is unsafe as you must ensure that no other invocation of the plugin
/// simultaneously holds a mutable reference to the external.
pub unsafe fn external_mut<T: 'static + Sync>(&mut self) -> Result<Option<&mut T>> {
if self.args_raw.external.is_null() {
return Ok(None);
}
let external: *mut TaggedObject<T> = self.args_raw.external as *mut TaggedObject<T>;
unsafe {
if (*external).type_id != TypeId::of::<T>() {
return Err(Error::ExternalTypeMismatch);
}
Ok((*external).object.as_mut())
}
}
/// Get the input source code for the current file.
///
/// On Windows, this function may return an `Err(Error::Utf8(...))` if the
/// source code contains invalid UTF-8.
pub fn input_source_code(&self) -> Result<Cow<'_, str>> {
let fetch_result = unsafe {
((*self.result_raw).fetchSourceCode.unwrap())(self.args_raw, self.result_raw)
};
if fetch_result != 0 {
Err(Error::Unknown)
} else {
// SAFETY: We don't hand out mutable references to `result_raw` so dereferencing here is safe.
unsafe {
get_from_raw_str((*self.result_raw).source_ptr, (*self.result_raw).source_len)
}
}
}
/// Set the output source code for the current file.
pub fn set_output_source_code(&mut self, source: String, loader: BunLoader) {
let source_cap = source.capacity();
let source = source.leak();
let source_ptr = source.as_mut_ptr();
let source_len = source.len();
if self.compilation_context.is_null() {
self.compilation_context = Box::into_raw(Box::new(SourceCodeContext {
source_ptr,
source_len,
source_cap,
}));
// SAFETY: We don't hand out mutable references to `result_raw` so dereferencing it is safe.
unsafe {
(*self.result_raw).plugin_source_code_context =
self.compilation_context as *mut c_void;
(*self.result_raw).free_plugin_source_code_context =
Some(free_plugin_source_code_context);
}
} else {
unsafe {
// SAFETY: If we're here we know that `compilation_context` is not null.
let context = &mut *self.compilation_context;
drop(String::from_raw_parts(
context.source_ptr,
context.source_len,
context.source_cap,
));
context.source_ptr = source_ptr;
context.source_len = source_len;
context.source_cap = source_cap;
}
}
// SAFETY: We don't hand out mutable references to `result_raw` so dereferencing it is safe.
unsafe {
(*self.result_raw).loader = loader as u8;
(*self.result_raw).source_ptr = source_ptr;
(*self.result_raw).source_len = source_len;
}
}
/// Set the output loader for the current file.
pub fn set_output_loader(&self, loader: BunLogLevel) {
// SAFETY: We don't hand out mutable references to `result_raw` so dereferencing it is safe.
unsafe {
(*self.result_raw).loader = loader as u8;
}
}
/// Get the output loader for the current file.
pub fn output_loader(&self) -> BunLoader {
unsafe { std::mem::transmute((*self.result_raw).loader as u32) }
}
/// Log an error message.
pub fn log_error(&self, message: &str) {
self.log(message, BunLogLevel::BUN_LOG_LEVEL_ERROR)
}
/// Log a message with the given level.
pub fn log(&self, message: &str, level: BunLogLevel) {
let mut log_options = sys::BunLogOptions {
__struct_size: std::mem::size_of::<sys::BunLogOptions>(),
message_ptr: message.as_ptr(),
message_len: message.len(),
path_ptr: self.args_raw.path_ptr,
path_len: self.args_raw.path_len,
source_line_text_ptr: std::ptr::null(),
source_line_text_len: 0,
level: level as i8,
line: 0,
lineEnd: 0,
column: 0,
columnEnd: 0,
};
unsafe {
((*self.result_raw).log.unwrap())(self.args_raw, &mut log_options);
}
}
}

View File

@@ -0,0 +1 @@
#include <bun-native-bundler-plugin-api/bundler_plugin.h>

View File

@@ -14,6 +14,7 @@
* This module aliases `globalThis.Bun`.
*/
declare module "bun" {
import type { FFIFunctionCallableSymbol } from "bun:ffi";
import type { Encoding as CryptoEncoding } from "crypto";
import type { CipherNameAndProtocol, EphemeralKeyInfo, PeerCertificate } from "tls";
interface Env {
@@ -3881,7 +3882,7 @@ declare module "bun" {
defer: () => Promise<void>;
}
type OnLoadResult = OnLoadResultSourceCode | OnLoadResultObject | undefined;
type OnLoadResult = OnLoadResultSourceCode | OnLoadResultObject | undefined | void;
type OnLoadCallback = (args: OnLoadArgs) => OnLoadResult | Promise<OnLoadResult>;
type OnStartCallback = () => void | Promise<void>;
@@ -3931,7 +3932,30 @@ declare module "bun" {
args: OnResolveArgs,
) => OnResolveResult | Promise<OnResolveResult | undefined | null> | undefined | null;
type FFIFunctionCallable = Function & {
// Making a nominally typed function so that the user must get it from dlopen
readonly __ffi_function_callable: typeof FFIFunctionCallableSymbol;
};
interface PluginBuilder {
/**
* Register a callback which will be invoked when bundling starts.
* @example
* ```ts
* Bun.plugin({
* setup(builder) {
* builder.onStart(() => {
* console.log("bundle just started!!")
* });
* },
* });
* ```
*/
onStart(callback: OnStartCallback): void;
onBeforeParse(
constraints: PluginConstraints,
callback: { napiModule: unknown; symbol: string; external?: unknown | undefined },
): void;
/**
* Register a callback to load imports with a specific import specifier
* @param constraints The constraints to apply the plugin to
@@ -3964,20 +3988,6 @@ declare module "bun" {
* ```
*/
onResolve(constraints: PluginConstraints, callback: OnResolveCallback): void;
/**
* Register a callback which will be invoked when bundling starts.
* @example
* ```ts
* Bun.plugin({
* setup(builder) {
* builder.onStart(() => {
* console.log("bundle just started!!")
* });
* },
* });
* ```
*/
onStart(callback: OnStartCallback): void;
/**
* The config object passed to `Bun.build` as is. Can be mutated.
*/

View File

@@ -566,17 +566,21 @@ declare module "bun:ffi" {
type ToFFIType<T extends FFITypeOrString> = T extends FFIType ? T : T extends string ? FFITypeStringToType[T] : never;
const FFIFunctionCallableSymbol: unique symbol;
type ConvertFns<Fns extends Symbols> = {
[K in keyof Fns]: (
...args: Fns[K]["args"] extends infer A extends readonly FFITypeOrString[]
? { [L in keyof A]: FFITypeToArgsType[ToFFIType<A[L]>] }
: // eslint-disable-next-line @definitelytyped/no-single-element-tuple-type
[unknown] extends [Fns[K]["args"]]
? []
: never
) => [unknown] extends [Fns[K]["returns"]] // eslint-disable-next-line @definitelytyped/no-single-element-tuple-type
? undefined
: FFITypeToReturnsType[ToFFIType<NonNullable<Fns[K]["returns"]>>];
[K in keyof Fns]: {
(
...args: Fns[K]["args"] extends infer A extends readonly FFITypeOrString[]
? { [L in keyof A]: FFITypeToArgsType[ToFFIType<A[L]>] }
: // eslint-disable-next-line @definitelytyped/no-single-element-tuple-type
[unknown] extends [Fns[K]["args"]]
? []
: never
): [unknown] extends [Fns[K]["returns"]] // eslint-disable-next-line @definitelytyped/no-single-element-tuple-type
? undefined
: FFITypeToReturnsType[ToFFIType<NonNullable<Fns[K]["returns"]>>];
__ffi_function_callable: typeof FFIFunctionCallableSymbol;
};
};
/**

View File

@@ -78,6 +78,7 @@ pub const JSBundler = struct {
experimental_css: bool = false,
css_chunking: bool = false,
drop: bun.StringSet = bun.StringSet.init(bun.default_allocator),
has_any_on_before_parse: bool = false,
pub const List = bun.StringArrayHashMapUnmanaged(Config);
@@ -858,6 +859,25 @@ pub const JSBundler = struct {
return plugin;
}
extern fn JSBundlerPlugin__callOnBeforeParsePlugins(
*Plugin,
bun_context: *anyopaque,
namespace: *const String,
path: *const String,
on_before_parse_args: ?*anyopaque,
on_before_parse_result: ?*anyopaque,
should_continue: *i32,
) i32;
pub fn callOnBeforeParsePlugins(this: *Plugin, ctx: *anyopaque, namespace: *const String, path: *const String, on_before_parse_args: ?*anyopaque, on_before_parse_result: ?*anyopaque, should_continue: *i32) i32 {
return JSBundlerPlugin__callOnBeforeParsePlugins(this, ctx, namespace, path, on_before_parse_args, on_before_parse_result, should_continue);
}
extern fn JSBundlerPlugin__hasOnBeforeParsePlugins(*Plugin) i32;
pub fn hasOnBeforeParsePlugins(this: *Plugin) bool {
return JSBundlerPlugin__hasOnBeforeParsePlugins(this) != 0;
}
extern fn JSBundlerPlugin__tombstone(*Plugin) void;
pub fn deinit(this: *Plugin) void {
JSC.markBinding(@src());

View File

@@ -823,6 +823,7 @@ pub const FFI = struct {
bun.cast(JSC.JSHostFunctionPtr, compiled.ptr),
false,
true,
function.symbol_from_dynamic_library,
);
compiled.js_function = cb;
obj.put(globalThis, &str, cb);
@@ -1178,6 +1179,7 @@ pub const FFI = struct {
bun.cast(JSC.JSHostFunctionPtr, compiled.ptr),
false,
true,
function.symbol_from_dynamic_library,
);
compiled.js_function = cb;
obj.put(global, &str, cb);
@@ -1280,6 +1282,7 @@ pub const FFI = struct {
bun.cast(JSC.JSHostFunctionPtr, compiled.ptr),
false,
true,
function.symbol_from_dynamic_library,
);
compiled.js_function = cb;

View File

@@ -40,6 +40,7 @@
#include "ErrorCode.h"
#include "napi_handle_scope.h"
#include "napi_external.h"
#ifndef WIN32
#include <errno.h>
@@ -359,6 +360,8 @@ JSC_DEFINE_HOST_FUNCTION(Process_functionDlopen,
void* handle = dlopen(utf8.data(), RTLD_LAZY);
#endif
globalObject->m_pendingNapiModuleDlopenHandle = handle;
Bun__process_dlopen_count++;
if (!handle) {
@@ -425,10 +428,25 @@ JSC_DEFINE_HOST_FUNCTION(Process_functionDlopen,
EncodedJSValue exportsValue = JSC::JSValue::encode(exports);
JSC::JSValue resultValue = JSValue::decode(napi_register_module_v1(globalObject, exportsValue));
if (auto resultObject = resultValue.getObject()) {
// If this is a native bundler plugin we want to store the handle from dlopen
// as we are going to call `dlsym()` on it later to get the plugin implementation.
const char** pointer_to_plugin_name = (const char**)dlsym(handle, "BUN_PLUGIN_NAME");
if (pointer_to_plugin_name) {
// TODO: think about the finalizer here
// currently we do not dealloc napi modules so we don't have to worry about it right now
auto* meta = new Bun::NapiModuleMeta(globalObject->m_pendingNapiModuleDlopenHandle);
Bun::NapiExternal* napi_external = Bun::NapiExternal::create(vm, globalObject->NapiExternalStructure(), meta, nullptr, nullptr);
bool success = resultObject->putDirect(vm, WebCore::builtinNames(vm).napiDlopenHandlePrivateName(), napi_external, JSC::PropertyAttribute::DontDelete | JSC::PropertyAttribute::ReadOnly);
ASSERT(success);
}
}
RETURN_IF_EXCEPTION(scope, {});
globalObject->m_pendingNapiModuleAndExports[0].clear();
globalObject->m_pendingNapiModuleAndExports[1].clear();
globalObject->m_pendingNapiModuleDlopenHandle = nullptr;
// https://github.com/nodejs/node/blob/2eff28fb7a93d3f672f80b582f664a7c701569fb/src/node_api.cc#L734-L742
// https://github.com/oven-sh/bun/issues/1288

View File

@@ -1,5 +1,7 @@
#include "JSBundlerPlugin.h"
#include "BunProcess.h"
#include "../../../packages/bun-native-bundler-plugin-api/bundler_plugin.h"
#include "headers-handwritten.h"
#include <JavaScriptCore/CatchScope.h>
#include <JavaScriptCore/JSGlobalObject.h>
@@ -11,6 +13,7 @@
#include <JavaScriptCore/JSObjectInlines.h>
#include <wtf/text/WTFString.h>
#include <JavaScriptCore/JSCInlines.h>
#include "JSFFIFunction.h"
#include <JavaScriptCore/ObjectConstructor.h>
#include <JavaScriptCore/SubspaceInlines.h>
@@ -23,10 +26,18 @@
#include <JavaScriptCore/LazyPropertyInlines.h>
#include <JavaScriptCore/VMTrapsInlines.h>
#include <JavaScriptCore/YarrMatchingContextHolder.h>
#include "ErrorCode.h"
#include "napi_external.h"
#include <JavaScriptCore/Strong.h>
#include <JavaScriptCore/JSPromise.h>
#if OS(WINDOWS)
#include <windows.h>
#endif
namespace Bun {
extern "C" int OnBeforeParsePlugin__isDone(void* context);
#define WRAP_BUNDLER_PLUGIN(argName) jsDoubleNumber(bitwise_cast<double>(reinterpret_cast<uintptr_t>(argName)))
#define UNWRAP_BUNDLER_PLUGIN(callFrame) reinterpret_cast<void*>(bitwise_cast<uintptr_t>(callFrame->argument(0).asDouble()))
@@ -41,16 +52,18 @@ JSC_DECLARE_HOST_FUNCTION(jsBundlerPluginFunction_addFilter);
JSC_DECLARE_HOST_FUNCTION(jsBundlerPluginFunction_addError);
JSC_DECLARE_HOST_FUNCTION(jsBundlerPluginFunction_onLoadAsync);
JSC_DECLARE_HOST_FUNCTION(jsBundlerPluginFunction_onResolveAsync);
JSC_DECLARE_HOST_FUNCTION(jsBundlerPluginFunction_onBeforeParse);
JSC_DECLARE_HOST_FUNCTION(jsBundlerPluginFunction_generateDeferPromise);
void BundlerPlugin::NamespaceList::append(JSC::VM& vm, JSC::RegExp* filter, String& namespaceString)
void BundlerPlugin::NamespaceList::append(JSC::VM& vm, JSC::RegExp* filter, String& namespaceString, unsigned& index)
{
auto* nsGroup = group(namespaceString);
auto* nsGroup = group(namespaceString, index);
if (nsGroup == nullptr) {
namespaces.append(namespaceString);
groups.append(Vector<Yarr::RegularExpression> {});
nsGroup = &groups.last();
index = namespaces.size() - 1;
}
Yarr::RegularExpression regex(
@@ -60,62 +73,48 @@ void BundlerPlugin::NamespaceList::append(JSC::VM& vm, JSC::RegExp* filter, Stri
nsGroup->append(WTFMove(regex));
}
bool BundlerPlugin::anyMatchesCrossThread(JSC::VM& vm, const BunString* namespaceStr, const BunString* path, bool isOnLoad)
static bool anyMatchesForNamespace(JSC::VM& vm, BundlerPlugin::NamespaceList& list, const BunString* namespaceStr, const BunString* path)
{
constexpr bool usesPatternContextBuffer = false;
if (isOnLoad) {
if (this->onLoad.fileNamespace.isEmpty() && this->onLoad.namespaces.isEmpty())
return false;
// Avoid unnecessary string copies
auto namespaceString = namespaceStr ? namespaceStr->toWTFString(BunString::ZeroCopy) : String();
if (list.fileNamespace.isEmpty() && list.namespaces.isEmpty())
return false;
auto* group = this->onLoad.group(namespaceString);
if (group == nullptr) {
return false;
}
// Avoid unnecessary string copies
auto namespaceString = namespaceStr ? namespaceStr->toWTFString(BunString::ZeroCopy) : String();
unsigned index = 0;
auto* group = list.group(namespaceString, index);
if (group == nullptr) {
return false;
}
auto& filters = *group;
auto pathString = path->toWTFString(BunString::ZeroCopy);
auto& filters = *group;
auto pathString = path->toWTFString(BunString::ZeroCopy);
for (auto& filter : filters) {
Yarr::MatchingContextHolder regExpContext(vm, usesPatternContextBuffer, nullptr, Yarr::MatchFrom::CompilerThread);
if (filter.match(pathString) > -1) {
return true;
}
}
} else {
if (this->onResolve.fileNamespace.isEmpty() && this->onResolve.namespaces.isEmpty())
return false;
// Avoid unnecessary string copies
auto namespaceString = namespaceStr ? namespaceStr->toWTFString(BunString::ZeroCopy) : String();
auto* group = this->onResolve.group(namespaceString);
if (group == nullptr) {
return false;
}
auto pathString = path->toWTFString(BunString::ZeroCopy);
auto& filters = *group;
for (auto& filter : filters) {
Yarr::MatchingContextHolder regExpContext(vm, usesPatternContextBuffer, nullptr, Yarr::MatchFrom::CompilerThread);
if (filter.match(pathString) > -1) {
return true;
}
for (auto& filter : filters) {
Yarr::MatchingContextHolder regExpContext(vm, usesPatternContextBuffer, nullptr, Yarr::MatchFrom::CompilerThread);
if (filter.match(pathString) > -1) {
return true;
}
}
return false;
}
bool BundlerPlugin::anyMatchesCrossThread(JSC::VM& vm, const BunString* namespaceStr, const BunString* path, bool isOnLoad)
{
if (isOnLoad) {
return anyMatchesForNamespace(vm, this->onLoad, namespaceStr, path);
} else {
return anyMatchesForNamespace(vm, this->onResolve, namespaceStr, path);
}
}
static const HashTableValue JSBundlerPluginHashTable[] = {
{ "addFilter"_s, static_cast<unsigned>(JSC::PropertyAttribute::Function | JSC::PropertyAttribute::ReadOnly | JSC::PropertyAttribute::DontDelete), NoIntrinsic, { HashTableValue::NativeFunctionType, jsBundlerPluginFunction_addFilter, 3 } },
{ "addError"_s, static_cast<unsigned>(JSC::PropertyAttribute::Function | JSC::PropertyAttribute::ReadOnly | JSC::PropertyAttribute::DontDelete), NoIntrinsic, { HashTableValue::NativeFunctionType, jsBundlerPluginFunction_addError, 3 } },
{ "onLoadAsync"_s, static_cast<unsigned>(JSC::PropertyAttribute::Function | JSC::PropertyAttribute::ReadOnly | JSC::PropertyAttribute::DontDelete), NoIntrinsic, { HashTableValue::NativeFunctionType, jsBundlerPluginFunction_onLoadAsync, 3 } },
{ "onResolveAsync"_s, static_cast<unsigned>(JSC::PropertyAttribute::Function | JSC::PropertyAttribute::ReadOnly | JSC::PropertyAttribute::DontDelete), NoIntrinsic, { HashTableValue::NativeFunctionType, jsBundlerPluginFunction_onResolveAsync, 4 } },
{ "onBeforeParse"_s, static_cast<unsigned>(JSC::PropertyAttribute::Function | JSC::PropertyAttribute::ReadOnly | JSC::PropertyAttribute::DontDelete), NoIntrinsic, { HashTableValue::NativeFunctionType, jsBundlerPluginFunction_onBeforeParse, 4 } },
{ "generateDeferPromise"_s, static_cast<unsigned>(JSC::PropertyAttribute::Function | JSC::PropertyAttribute::ReadOnly | JSC::PropertyAttribute::DontDelete), NoIntrinsic, { HashTableValue::NativeFunctionType, jsBundlerPluginFunction_generateDeferPromise, 0 } },
};
@@ -163,20 +162,12 @@ public:
/// These are the user implementation of the plugin callbacks
JSC::LazyProperty<JSBundlerPlugin, JSC::JSFunction> onLoadFunction;
JSC::LazyProperty<JSBundlerPlugin, JSC::JSFunction> onResolveFunction;
JSC::LazyProperty<JSBundlerPlugin, JSC::JSFunction> moduleFunction;
JSC::LazyProperty<JSBundlerPlugin, JSC::JSFunction> setupFunction;
JSC::JSGlobalObject* m_globalObject;
private:
JSBundlerPlugin(
JSC::VM& vm,
JSC::JSGlobalObject* global,
JSC::Structure* structure,
void* config,
BunPluginTarget target,
JSBundlerPluginAddErrorCallback addError,
JSBundlerPluginOnLoadAsyncCallback onLoadAsync,
JSBundlerPluginOnResolveAsyncCallback onResolveAsync)
JSBundlerPlugin(JSC::VM& vm, JSC::JSGlobalObject* global, JSC::Structure* structure, void* config, BunPluginTarget target,
JSBundlerPluginAddErrorCallback addError, JSBundlerPluginOnLoadAsyncCallback onLoadAsync, JSBundlerPluginOnResolveAsyncCallback onResolveAsync)
: JSC::JSNonFinalObject(vm, structure)
, plugin(BundlerPlugin(config, target, addError, onLoadAsync, onResolveAsync))
, m_globalObject(global)
@@ -213,18 +204,196 @@ JSC_DEFINE_HOST_FUNCTION(jsBundlerPluginFunction_addFilter, (JSC::JSGlobalObject
namespaceStr = String();
}
bool isOnLoad = callFrame->argument(2).toNumber(globalObject) == 1;
uint32_t isOnLoad = callFrame->argument(2).toUInt32(globalObject);
auto& vm = globalObject->vm();
unsigned index = 0;
if (isOnLoad) {
thisObject->plugin.onLoad.append(vm, regExp->regExp(), namespaceStr);
thisObject->plugin.onLoad.append(vm, regExp->regExp(), namespaceStr, index);
} else {
thisObject->plugin.onResolve.append(vm, regExp->regExp(), namespaceStr);
thisObject->plugin.onResolve.append(vm, regExp->regExp(), namespaceStr, index);
}
return JSC::JSValue::encode(JSC::jsUndefined());
}
static JSBundlerPluginNativeOnBeforeParseCallback nativeCallbackFromJS(JSC::JSGlobalObject* globalObject, JSC::JSValue value)
{
if (auto* fn = jsDynamicCast<JSFFIFunction*>(value)) {
return reinterpret_cast<JSBundlerPluginNativeOnBeforeParseCallback>(fn->symbolFromDynamicLibrary);
}
if (auto* object = value.getObject()) {
if (auto callbackValue = object->getIfPropertyExists(globalObject, JSC::Identifier::fromString(globalObject->vm(), String("native"_s)))) {
if (auto* fn = jsDynamicCast<JSFFIFunction*>(callbackValue)) {
return reinterpret_cast<JSBundlerPluginNativeOnBeforeParseCallback>(fn->symbolFromDynamicLibrary);
}
}
}
return nullptr;
}
void BundlerPlugin::NativePluginList::append(JSC::VM& vm, JSC::RegExp* filter, String& namespaceString, JSBundlerPluginNativeOnBeforeParseCallback callback, const char* name, NapiExternal* external)
{
unsigned index = 0;
{
auto* nsGroup = group(namespaceString, index);
if (nsGroup == nullptr) {
namespaces.append(namespaceString);
groups.append(Vector<NativeFilterRegexp> {});
nsGroup = &groups.last();
index = namespaces.size() - 1;
}
Yarr::RegularExpression regex(
StringView(filter->pattern()),
filter->flags());
NativeFilterRegexp nativeFilterRegexp = std::make_pair(regex, std::make_shared<std::mutex>());
nsGroup->append(nativeFilterRegexp);
}
if (index == std::numeric_limits<unsigned>::max()) {
this->fileCallbacks.append(NativePluginCallback {
callback,
external,
name,
});
} else {
if (this->namespaceCallbacks.size() <= index) {
this->namespaceCallbacks.grow(index + 1);
}
this->namespaceCallbacks[index].append(NativePluginCallback { callback, external, name });
}
}
extern "C" void CrashHandler__setInsideNativePlugin(const char* plugin_name);
int BundlerPlugin::NativePluginList::call(JSC::VM& vm, BundlerPlugin* plugin, int* shouldContinue, void* bunContextPtr, const BunString* namespaceStr, const BunString* pathString, void* onBeforeParseArgs, void* onBeforeParseResult)
{
unsigned index = 0;
const auto* group = this->group(namespaceStr->toWTFString(BunString::ZeroCopy), index);
if (group == nullptr) {
return -1;
}
const auto& callbacks = index == std::numeric_limits<unsigned>::max() ? this->fileCallbacks : this->namespaceCallbacks[index];
ASSERT_WITH_MESSAGE(callbacks.size() == group->size(), "Number of callbacks and filters must match");
if (callbacks.isEmpty()) {
return -1;
}
int count = 0;
constexpr bool usesPatternContextBuffer = false;
const WTF::String& path = pathString->toWTFString(BunString::ZeroCopy);
for (size_t i = 0, total = callbacks.size(); i < total && *shouldContinue; ++i) {
Yarr::MatchingContextHolder regExpContext(vm, usesPatternContextBuffer, nullptr, Yarr::MatchFrom::CompilerThread);
// Need to lock the mutex to access the regular expression
{
std::lock_guard<std::mutex> lock(*group->at(i).second);
if (group->at(i).first.match(path) > -1) {
Bun::NapiExternal* external = callbacks[i].external;
if (external) {
((OnBeforeParseArguments*)(onBeforeParseArgs))->external = external->value();
}
JSBundlerPluginNativeOnBeforeParseCallback callback = callbacks[i].callback;
const char* name = callbacks[i].name ? callbacks[i].name : "<unknown>";
CrashHandler__setInsideNativePlugin(name);
callback(onBeforeParseArgs, onBeforeParseResult);
CrashHandler__setInsideNativePlugin(nullptr);
count++;
}
}
if (OnBeforeParsePlugin__isDone(bunContextPtr)) {
return count;
}
}
return count;
}
JSC_DEFINE_HOST_FUNCTION(jsBundlerPluginFunction_onBeforeParse, (JSC::JSGlobalObject * globalObject, JSC::CallFrame* callFrame))
{
auto& vm = globalObject->vm();
auto scope = DECLARE_THROW_SCOPE(vm);
JSBundlerPlugin* thisObject = jsCast<JSBundlerPlugin*>(callFrame->thisValue());
if (thisObject->plugin.tombstoned) {
return JSC::JSValue::encode(JSC::jsUndefined());
}
// Clone the regexp so we don't have to worry about it being used concurrently with the JS thread.
// TODO: Should we have a regexp object for every thread in the thread pool? Then we could avoid using
// a mutex to synchronize access to the same regexp from multiple threads.
JSC::RegExpObject* jsRegexp = jsCast<JSC::RegExpObject*>(callFrame->argument(0));
RegExp* reggie = jsRegexp->regExp();
RegExp* newRegexp = RegExp::create(vm, reggie->pattern(), reggie->flags());
WTF::String namespaceStr = callFrame->argument(1).toWTFString(globalObject);
RETURN_IF_EXCEPTION(scope, {});
if (namespaceStr == "file"_s) {
namespaceStr = String();
}
JSC::JSValue node_addon = callFrame->argument(2);
if (!node_addon.isObject()) {
Bun::throwError(globalObject, scope, ErrorCode::ERR_INVALID_ARG_TYPE, "Expected node_addon (2nd argument) to be an object"_s);
return {};
}
JSC::JSValue on_before_parse_symbol_js = callFrame->argument(3);
if (!on_before_parse_symbol_js.isString()) {
Bun::throwError(globalObject, scope, ErrorCode::ERR_INVALID_ARG_TYPE, "Expected on_before_parse_symbol (3rd argument) to be a string"_s);
return {};
}
WTF::String on_before_parse_symbol = on_before_parse_symbol_js.toWTFString(globalObject);
// The dlopen *void handle is attached to the node_addon as a NapiExternal
Bun::NapiExternal* napi_external = jsDynamicCast<Bun::NapiExternal*>(node_addon.getObject()->get(globalObject, WebCore::builtinNames(vm).napiDlopenHandlePrivateName()));
if (UNLIKELY(!napi_external)) {
Bun::throwError(globalObject, scope, ErrorCode::ERR_INVALID_ARG_TYPE, "Expected node_addon (2nd argument) to have a napiDlopenHandle property"_s);
return {};
}
Bun::NapiModuleMeta* meta = (Bun::NapiModuleMeta*)napi_external->value();
void* dlopen_handle = meta->dlopenHandle;
CString utf8 = on_before_parse_symbol.utf8();
#if OS(WINDOWS)
void* on_before_parse_symbol_ptr = GetProcAddress((HMODULE)dlopen_handle, utf8.data());
const char** native_plugin_name = (const char**)GetProcAddress((HMODULE)dlopen_handle, "BUN_PLUGIN_NAME");
#else
void* on_before_parse_symbol_ptr = dlsym(dlopen_handle, utf8.data());
const char** native_plugin_name = (const char**)dlsym(dlopen_handle, "BUN_PLUGIN_NAME");
#endif
if (!on_before_parse_symbol_ptr) {
Bun::throwError(globalObject, scope, ErrorCode::ERR_INVALID_ARG_TYPE, makeString("Could not find the symbol \""_s, on_before_parse_symbol, "\" in the given napi module."_s));
return {};
}
JSBundlerPluginNativeOnBeforeParseCallback callback = reinterpret_cast<JSBundlerPluginNativeOnBeforeParseCallback>(on_before_parse_symbol_ptr);
JSC::JSValue external = callFrame->argument(4);
NapiExternal* externalPtr = nullptr;
if (!external.isUndefinedOrNull()) {
externalPtr = jsDynamicCast<Bun::NapiExternal*>(external);
if (UNLIKELY(!externalPtr)) {
Bun::throwError(globalObject, scope, ErrorCode::ERR_INVALID_ARG_TYPE, "Expected external (3rd argument) to be a NAPI external"_s);
return {};
}
}
thisObject->plugin.onBeforeParse.append(vm, newRegexp, namespaceStr, callback, native_plugin_name ? *native_plugin_name : nullptr, externalPtr);
return JSC::JSValue::encode(JSC::jsUndefined());
}
JSC_DEFINE_HOST_FUNCTION(jsBundlerPluginFunction_addError, (JSC::JSGlobalObject * globalObject, JSC::CallFrame* callFrame))
{
JSBundlerPlugin* thisObject = jsCast<JSBundlerPlugin*>(callFrame->thisValue());
@@ -473,6 +642,23 @@ extern "C" void JSBundlerPlugin__tombstone(Bun::JSBundlerPlugin* plugin)
plugin->plugin.tombstone();
}
extern "C" int JSBundlerPlugin__callOnBeforeParsePlugins(
Bun::JSBundlerPlugin* plugin,
void* bunContextPtr,
const BunString* namespaceStr,
const BunString* pathString,
OnBeforeParseArguments* onBeforeParseArgs,
void* onBeforeParseResult,
int* shouldContinue)
{
return plugin->plugin.onBeforeParse.call(plugin->vm(), &plugin->plugin, shouldContinue, bunContextPtr, namespaceStr, pathString, onBeforeParseArgs, onBeforeParseResult);
}
extern "C" int JSBundlerPlugin__hasOnBeforeParsePlugins(Bun::JSBundlerPlugin* plugin)
{
return plugin->plugin.onBeforeParse.namespaceCallbacks.size() > 0 || plugin->plugin.onBeforeParse.fileCallbacks.size() > 0;
}
extern "C" JSC::JSGlobalObject* JSBundlerPlugin__globalObject(Bun::JSBundlerPlugin* plugin)
{
return plugin->m_globalObject;

View File

@@ -3,14 +3,14 @@
#include "root.h"
#include "headers-handwritten.h"
#include <JavaScriptCore/JSGlobalObject.h>
#include <JavaScriptCore/Strong.h>
#include <JavaScriptCore/RegularExpression.h>
#include "helpers.h"
#include "napi_external.h"
#include <JavaScriptCore/Yarr.h>
typedef void (*JSBundlerPluginAddErrorCallback)(void*, void*, JSC::EncodedJSValue, JSC::EncodedJSValue);
typedef void (*JSBundlerPluginOnLoadAsyncCallback)(void*, void*, JSC::EncodedJSValue, JSC::EncodedJSValue);
typedef void (*JSBundlerPluginOnResolveAsyncCallback)(void*, void*, JSC::EncodedJSValue, JSC::EncodedJSValue, JSC::EncodedJSValue);
typedef void (*JSBundlerPluginNativeOnBeforeParseCallback)(void*, void*);
namespace Bun {
@@ -18,22 +18,24 @@ using namespace JSC;
class BundlerPlugin final {
public:
class NamespaceList final {
class NamespaceList {
public:
Vector<Yarr::RegularExpression> fileNamespace = {};
Vector<String> namespaces = {};
Vector<Vector<Yarr::RegularExpression>> groups = {};
BunPluginTarget target { BunPluginTargetBun };
Vector<Yarr::RegularExpression>* group(const String& namespaceStr)
Vector<Yarr::RegularExpression>* group(const String& namespaceStr, unsigned& index)
{
if (namespaceStr.isEmpty()) {
index = std::numeric_limits<unsigned>::max();
return &fileNamespace;
}
size_t length = namespaces.size();
for (size_t i = 0; i < length; i++) {
if (namespaces[i] == namespaceStr) {
index = i;
return &groups[i];
}
}
@@ -41,7 +43,56 @@ public:
return nullptr;
}
void append(JSC::VM& vm, JSC::RegExp* filter, String& namespaceString);
void append(JSC::VM& vm, JSC::RegExp* filter, String& namespaceString, unsigned& index);
};
/// In native plugins, the regular expression could be called concurrently on multiple threads.
/// Therefore, we need a mutex to synchronize access.
typedef std::pair<Yarr::RegularExpression, std::shared_ptr<std::mutex>> NativeFilterRegexp;
struct NativePluginCallback {
JSBundlerPluginNativeOnBeforeParseCallback callback;
Bun::NapiExternal* external;
/// This refers to the string exported in the native plugin under
/// the symbol BUN_PLUGIN_NAME
///
/// Right now we do not close NAPI modules opened with dlopen and
/// so we do not worry about lifetimes right now.
const char* name;
};
class NativePluginList {
public:
using PerNamespaceCallbackList = Vector<NativePluginCallback>;
Vector<NativeFilterRegexp> fileNamespace = {};
Vector<String> namespaces = {};
Vector<Vector<NativeFilterRegexp>> groups = {};
BunPluginTarget target { BunPluginTargetBun };
PerNamespaceCallbackList fileCallbacks = {};
Vector<PerNamespaceCallbackList> namespaceCallbacks = {};
int call(JSC::VM& vm, BundlerPlugin* plugin, int* shouldContinue, void* bunContextPtr, const BunString* namespaceStr, const BunString* pathString, void* onBeforeParseArgs, void* onBeforeParseResult);
void append(JSC::VM& vm, JSC::RegExp* filter, String& namespaceString, JSBundlerPluginNativeOnBeforeParseCallback callback, const char* name, NapiExternal* external);
Vector<NativeFilterRegexp>* group(const String& namespaceStr, unsigned& index)
{
if (namespaceStr.isEmpty()) {
index = std::numeric_limits<unsigned>::max();
return &fileNamespace;
}
size_t length = namespaces.size();
for (size_t i = 0; i < length; i++) {
if (namespaces[i] == namespaceStr) {
index = i;
return &groups[i];
}
}
return nullptr;
}
};
public:
@@ -59,6 +110,7 @@ public:
NamespaceList onLoad = {};
NamespaceList onResolve = {};
NativePluginList onBeforeParse = {};
BunPluginTarget target { BunPluginTargetBrowser };
Vector<Strong<JSPromise>> deferredPromises = {};

View File

@@ -113,7 +113,7 @@ extern "C" void Bun__untrackFFIFunction(Zig::GlobalObject* globalObject, JSC::En
{
globalObject->untrackFFIFunction(JSC::jsCast<JSC::JSFunction*>(JSC::JSValue::decode(function)));
}
extern "C" JSC::EncodedJSValue Bun__CreateFFIFunctionValue(Zig::GlobalObject* globalObject, const ZigString* symbolName, unsigned argCount, Zig::FFIFunction functionPointer, bool strong, bool addPtrField)
extern "C" JSC::EncodedJSValue Bun__CreateFFIFunctionValue(Zig::GlobalObject* globalObject, const ZigString* symbolName, unsigned argCount, Zig::FFIFunction functionPointer, bool strong, bool addPtrField, void* symbolFromDynamicLibrary)
{
if (addPtrField) {
auto* function = Zig::JSFFIFunction::createForFFI(globalObject->vm(), globalObject, argCount, symbolName != nullptr ? Zig::toStringCopy(*symbolName) : String(), reinterpret_cast<Bun::CFFIFunction>(functionPointer));
@@ -121,8 +121,8 @@ extern "C" JSC::EncodedJSValue Bun__CreateFFIFunctionValue(Zig::GlobalObject* gl
// We should only expose the "ptr" field when it's a JSCallback for bun:ffi.
// Not for internal usages of this function type.
// We should also consider a separate JSFunction type for our usage to not have this branch in the first place...
function->putDirect(vm, JSC::Identifier::fromString(vm, String(MAKE_STATIC_STRING_IMPL("ptr"))), JSC::jsNumber(bitwise_cast<double>(functionPointer)), JSC::PropertyAttribute::ReadOnly | 0);
function->putDirect(vm, JSC::Identifier::fromString(vm, String("ptr"_s)), JSC::jsNumber(bitwise_cast<double>(functionPointer)), JSC::PropertyAttribute::ReadOnly | 0);
function->symbolFromDynamicLibrary = symbolFromDynamicLibrary;
return JSC::JSValue::encode(function);
}

View File

@@ -87,6 +87,7 @@ public:
#endif
void* dataPtr;
void* symbolFromDynamicLibrary { nullptr };
private:
JSFFIFunction(VM&, NativeExecutable*, JSGlobalObject*, Structure*, CFFIFunction&&);

View File

@@ -386,6 +386,10 @@ public:
// When a napi module initializes on dlopen, we need to know what the value is
mutable JSC::WriteBarrier<Unknown> m_pendingNapiModuleAndExports[2];
// This is the result of dlopen()ing a napi module.
// We will add it to the resulting napi value.
void* m_pendingNapiModuleDlopenHandle = nullptr;
// The handle scope where all new NAPI values will be created. You must not pass any napi_values
// back to a NAPI function without putting them in the handle scope, as the NAPI function may
// move them off the stack which will cause them to get collected if not in the handle scope.

View File

@@ -3509,7 +3509,7 @@ pub const JSGlobalObject = opaque {
// when querying from JavaScript, 'func.len'
comptime argument_count: u32,
) JSValue {
return NewRuntimeFunction(global, ZigString.static(display_name), argument_count, toJSHostFunction(function), false, false);
return NewRuntimeFunction(global, ZigString.static(display_name), argument_count, toJSHostFunction(function), false, false, null);
}
pub usingnamespace @import("ErrorCode").JSGlobalObjectExtensions;
@@ -6774,6 +6774,7 @@ const private = struct {
functionPointer: JSHostFunctionPtr,
strong: bool,
add_ptr_field: bool,
inputFunctionPtr: ?*anyopaque,
) JSValue;
pub extern fn Bun__untrackFFIFunction(
@@ -6793,9 +6794,9 @@ pub fn NewFunction(
strong: bool,
) JSValue {
if (@TypeOf(functionPointer) == JSC.JSHostFunctionType) {
return NewRuntimeFunction(globalObject, symbolName, argCount, functionPointer, strong, false);
return NewRuntimeFunction(globalObject, symbolName, argCount, functionPointer, strong, false, null);
}
return NewRuntimeFunction(globalObject, symbolName, argCount, toJSHostFunction(functionPointer), strong, false);
return NewRuntimeFunction(globalObject, symbolName, argCount, toJSHostFunction(functionPointer), strong, false, null);
}
pub fn createCallback(
@@ -6807,7 +6808,7 @@ pub fn createCallback(
if (@TypeOf(functionPointer) == JSC.JSHostFunctionType) {
return NewRuntimeFunction(globalObject, symbolName, argCount, functionPointer, false, false);
}
return NewRuntimeFunction(globalObject, symbolName, argCount, toJSHostFunction(functionPointer), false, false);
return NewRuntimeFunction(globalObject, symbolName, argCount, toJSHostFunction(functionPointer), false, false, null);
}
pub fn NewRuntimeFunction(
@@ -6817,9 +6818,10 @@ pub fn NewRuntimeFunction(
functionPointer: JSHostFunctionPtr,
strong: bool,
add_ptr_property: bool,
inputFunctionPtr: ?*anyopaque,
) JSValue {
JSC.markBinding(@src());
return private.Bun__CreateFFIFunctionValue(globalObject, symbolName, argCount, functionPointer, strong, add_ptr_property);
return private.Bun__CreateFFIFunctionValue(globalObject, symbolName, argCount, functionPointer, strong, add_ptr_property, inputFunctionPtr);
}
pub fn getFunctionData(function: JSValue) ?*anyopaque {

View File

@@ -66,6 +66,7 @@
#include "CommonJSModuleRecord.h"
#include "wtf/text/ASCIIFastPath.h"
#include "JavaScriptCore/WeakInlines.h"
#include <JavaScriptCore/BuiltinNames.h>
// #include <iostream>
using namespace JSC;
@@ -1004,6 +1005,16 @@ extern "C" void napi_module_register(napi_module* mod)
return;
}
auto* meta = new Bun::NapiModuleMeta(globalObject->m_pendingNapiModuleDlopenHandle);
// TODO: think about the finalizer here
Bun::NapiExternal* napi_external = Bun::NapiExternal::create(vm, globalObject->NapiExternalStructure(), meta, nullptr, nullptr);
bool success = resultValue.getObject()->putDirect(vm, WebCore::builtinNames(vm).napiDlopenHandlePrivateName(), napi_external, JSC::PropertyAttribute::DontDelete | JSC::PropertyAttribute::ReadOnly);
ASSERT(success);
globalObject->m_pendingNapiModuleDlopenHandle = nullptr;
// https://github.com/nodejs/node/blob/2eff28fb7a93d3f672f80b582f664a7c701569fb/src/node_api.cc#L734-L742
// https://github.com/oven-sh/bun/issues/1288
if (!scope.exception() && strongExportsObject && strongExportsObject.get() != resultValue) {
@@ -2541,7 +2552,8 @@ extern "C" napi_status napi_get_value_external(napi_env env, napi_value value,
return napi_invalid_arg;
}
auto* external = jsDynamicCast<Bun::NapiExternal*>(toJS(value));
JSValue jsval = toJS(value);
auto* external = jsDynamicCast<Bun::NapiExternal*>(jsval);
if (UNLIKELY(!external)) {
return napi_invalid_arg;
}

View File

@@ -12,6 +12,11 @@ namespace Bun {
using namespace JSC;
using namespace WebCore;
typedef struct {
/// The result of call to dlopen to load the module
void* dlopenHandle;
} NapiModuleMeta;
class NapiExternal : public JSC::JSDestructibleObject {
using Base = JSC::JSDestructibleObject;

View File

@@ -387,6 +387,9 @@ pub const BundleV2 = struct {
/// See the comment in `Chunk.OutputPiece`
unique_key: u64 = 0,
dynamic_import_entry_points: std.AutoArrayHashMap(Index.Int, void) = undefined,
has_on_parse_plugins: bool = false,
finalizers: std.ArrayListUnmanaged(CacheEntry.External) = .{},
drain_defer_task: DeferredBatchTask = .{},
@@ -438,6 +441,10 @@ pub const BundleV2 = struct {
};
}
pub fn hasOnParsePlugins(this: *const BundleV2) bool {
return this.has_on_parse_plugins;
}
/// Same semantics as bundlerForTarget for `path_to_source_index_map`
pub inline fn pathToSourceIndexMap(this: *BundleV2, target: options.Target) *PathToSourceIndexMap {
return if (!this.bundler.options.server_components)
@@ -1490,6 +1497,7 @@ pub const BundleV2 = struct {
pub fn processFilesToCopy(this: *BundleV2, reachable_files: []const Index) !void {
if (this.graph.estimated_file_loader_count > 0) {
const file_allocators = this.graph.input_files.items(.allocator);
const unique_key_for_additional_files = this.graph.input_files.items(.unique_key_for_additional_file);
const content_hashes_for_additional_files = this.graph.input_files.items(.content_hash_for_additional_file);
const sources = this.graph.input_files.items(.source);
@@ -1526,7 +1534,7 @@ pub const BundleV2 = struct {
additional_output_files.append(options.OutputFile.init(.{
.data = .{ .buffer = .{
.data = source.contents,
.allocator = bun.default_allocator,
.allocator = file_allocators[index],
} },
.size = source.contents.len,
.output_path = std.fmt.allocPrint(bun.default_allocator, "{}", .{template}) catch bun.outOfMemory(),
@@ -1571,8 +1579,9 @@ pub const BundleV2 = struct {
.task = JSBundleCompletionTask.TaskCompletion.init(completion),
};
if (plugins) |plugin|
if (plugins) |plugin| {
plugin.setConfig(completion);
}
// Ensure this exists before we spawn the thread to prevent any race
// conditions from creating two
@@ -2064,6 +2073,16 @@ pub const BundleV2 = struct {
}
pub fn deinit(this: *BundleV2) void {
{
// We do this first to make it harder for any dangling pointers to data to be used in there.
var on_parse_finalizers = this.finalizers;
this.finalizers = .{};
for (on_parse_finalizers.items) |finalizer| {
finalizer.call();
}
on_parse_finalizers.deinit(bun.default_allocator);
}
defer this.graph.ast.deinit(bun.default_allocator);
defer this.graph.input_files.deinit(bun.default_allocator);
if (this.graph.pool.workers_assignments.count() > 0) {
@@ -2755,6 +2774,19 @@ pub const BundleV2 = struct {
pub fn onParseTaskComplete(parse_result: *ParseTask.Result, this: *BundleV2) void {
const trace = tracer(@src(), "onParseTaskComplete");
defer trace.end();
if (parse_result.external.function != null) {
const source = switch (parse_result.value) {
inline .empty, .err => |data| data.source_index.get(),
.success => |val| val.source.index.get(),
};
const loader: Loader = this.graph.input_files.items(.loader)[source];
if (!loader.shouldCopyForBundling(this.bundler.options.experimental_css)) {
this.finalizers.append(bun.default_allocator, parse_result.external) catch bun.outOfMemory();
} else {
this.graph.input_files.items(.allocator)[source] = ExternalFreeFunctionAllocator.create(@ptrCast(parse_result.external.function.?), parse_result.external.ctx.?);
}
}
defer bun.default_allocator.destroy(parse_result);
const graph = &this.graph;
@@ -3268,6 +3300,7 @@ pub const ParseTask = struct {
path: Fs.Path,
secondary_path_for_commonjs_interop: ?Fs.Path = null,
contents_or_fd: ContentsOrFd,
external: CacheEntry.External = .{},
side_effects: _resolver.SideEffects,
loader: ?Loader = null,
jsx: options.JSX.Pragma,
@@ -3291,6 +3324,10 @@ pub const ParseTask = struct {
ctx: *BundleV2,
value: Value,
watcher_data: WatcherData,
/// This is used for native onBeforeParsePlugins to store
/// a function pointer and context pointer to free the
/// returned source code by the plugin.
external: CacheEntry.External = .{},
pub const Value = union(enum) {
success: Success,
@@ -3724,23 +3761,17 @@ pub const ParseTask = struct {
return ast;
}
fn run(
fn getCodeForParseTaskWithoutPlugins(
task: *ParseTask,
this: *ThreadPool.Worker,
step: *ParseTask.Result.Error.Step,
log: *Logger.Log,
) anyerror!Result.Success {
const allocator = this.allocator;
var data = this.data;
var bundler = &data.bundler;
errdefer bundler.resetStore();
var resolver: *Resolver = &bundler.resolver;
var file_path = task.path;
step.* = .read_file;
const loader = task.loader orelse file_path.loader(&bundler.options.loaders) orelse options.Loader.file;
var entry: CacheEntry = switch (task.contents_or_fd) {
bundler: *Bundler,
resolver: *Resolver,
allocator: std.mem.Allocator,
file_path: *Fs.Path,
loader: Loader,
experimental_css: bool,
) !CacheEntry {
return switch (task.contents_or_fd) {
.fd => |contents| brk: {
const trace = tracer(@src(), "readFile");
defer trace.end();
@@ -3751,7 +3782,7 @@ pub const ParseTask = struct {
switch (file) {
.code => |code| break :brk .{ .contents = code },
.import => |path| {
file_path = Fs.Path.init(path);
file_path.* = Fs.Path.init(path);
break :lookup_builtin;
},
}
@@ -3764,7 +3795,7 @@ pub const ParseTask = struct {
}
break :brk resolver.caches.fs.readFileWithAllocator(
if (loader.shouldCopyForBundling(this.ctx.bundler.options.experimental_css))
if (loader.shouldCopyForBundling(experimental_css))
// The OutputFile will own the memory for the contents
bun.default_allocator
else
@@ -3808,6 +3839,342 @@ pub const ParseTask = struct {
.fd = bun.invalid_fd,
},
};
}
fn getCodeForParseTask(
task: *ParseTask,
log: *Logger.Log,
bundler: *Bundler,
resolver: *Resolver,
allocator: std.mem.Allocator,
file_path: *Fs.Path,
loader: *Loader,
experimental_css: bool,
from_plugin: *bool,
) !CacheEntry {
const might_have_on_parse_plugins = brk: {
if (task.source_index.isRuntime()) break :brk false;
const plugin = task.ctx.plugins orelse break :brk false;
if (!plugin.hasOnBeforeParsePlugins()) break :brk false;
if (strings.eqlComptime(file_path.namespace, "node")) {
break :brk false;
}
break :brk true;
};
if (!might_have_on_parse_plugins) {
return getCodeForParseTaskWithoutPlugins(task, log, bundler, resolver, allocator, file_path, loader.*, experimental_css);
}
var should_continue_running: i32 = 1;
var ctx = OnBeforeParsePlugin{
.task = task,
.log = log,
.bundler = bundler,
.resolver = resolver,
.allocator = allocator,
.file_path = file_path,
.loader = loader,
.experimental_css = experimental_css,
.deferred_error = null,
.should_continue_running = &should_continue_running,
};
return try ctx.run(task.ctx.plugins.?, from_plugin);
}
const OnBeforeParsePlugin = struct {
task: *ParseTask,
log: *Logger.Log,
bundler: *Bundler,
resolver: *Resolver,
allocator: std.mem.Allocator,
file_path: *Fs.Path,
loader: *Loader,
experimental_css: bool,
deferred_error: ?anyerror = null,
should_continue_running: *i32,
result: ?*OnBeforeParseResult = null,
const headers = @import("bun-native-bundler-plugin-api");
comptime {
bun.assert(@sizeOf(OnBeforeParseArguments) == @sizeOf(headers.OnBeforeParseArguments));
bun.assert(@alignOf(OnBeforeParseArguments) == @alignOf(headers.OnBeforeParseArguments));
bun.assert(@sizeOf(BunLogOptions) == @sizeOf(headers.BunLogOptions));
bun.assert(@alignOf(BunLogOptions) == @alignOf(headers.BunLogOptions));
bun.assert(@sizeOf(OnBeforeParseResult) == @sizeOf(headers.OnBeforeParseResult));
bun.assert(@alignOf(OnBeforeParseResult) == @alignOf(headers.OnBeforeParseResult));
bun.assert(@sizeOf(BunLogOptions) == @sizeOf(headers.BunLogOptions));
bun.assert(@alignOf(BunLogOptions) == @alignOf(headers.BunLogOptions));
}
const OnBeforeParseArguments = extern struct {
struct_size: usize = @sizeOf(OnBeforeParseArguments),
context: *OnBeforeParsePlugin,
path_ptr: [*]const u8 = "",
path_len: usize = 0,
namespace_ptr: [*]const u8 = "file",
namespace_len: usize = "file".len,
default_loader: Loader = .file,
external: ?*void = null,
};
const BunLogOptions = extern struct {
struct_size: usize = @sizeOf(BunLogOptions),
message_ptr: ?[*]const u8 = null,
message_len: usize = 0,
path_ptr: ?[*]const u8 = null,
path_len: usize = 0,
source_line_text_ptr: ?[*]const u8 = null,
source_line_text_len: usize = 0,
level: Logger.Log.Level = .err,
line: i32 = 0,
column: i32 = 0,
line_end: i32 = 0,
column_end: i32 = 0,
pub fn sourceLineText(this: *const BunLogOptions) string {
if (this.source_line_text_ptr) |ptr| {
if (this.source_line_text_len > 0) {
return ptr[0..this.source_line_text_len];
}
}
return "";
}
pub fn path(this: *const BunLogOptions) string {
if (this.path_ptr) |ptr| {
if (this.path_len > 0) {
return ptr[0..this.path_len];
}
}
return "";
}
pub fn message(this: *const BunLogOptions) string {
if (this.message_ptr) |ptr| {
if (this.message_len > 0) {
return ptr[0..this.message_len];
}
}
return "";
}
pub fn append(this: *const BunLogOptions, log: *Logger.Log, namespace: string) void {
const allocator = log.msgs.allocator;
const source_line_text = this.sourceLineText();
const location = Logger.Location.init(
this.path(),
namespace,
@max(this.line, -1),
@max(this.column, -1),
@max(this.column_end - this.column, 0),
if (source_line_text.len > 0) allocator.dupe(u8, source_line_text) catch bun.outOfMemory() else null,
null,
);
var msg = Logger.Msg{ .data = .{ .location = location, .text = allocator.dupe(u8, this.message()) catch bun.outOfMemory() } };
switch (this.level) {
.err => msg.kind = .err,
.warn => msg.kind = .warn,
.verbose => msg.kind = .verbose,
.debug => msg.kind = .debug,
else => {},
}
if (msg.kind == .err) {
log.errors += 1;
} else if (msg.kind == .warn) {
log.warnings += 1;
}
log.addMsg(msg) catch bun.outOfMemory();
}
pub fn logFn(
args_: ?*OnBeforeParseArguments,
log_options_: ?*BunLogOptions,
) callconv(.C) void {
const args = args_ orelse return;
const log_options = log_options_ orelse return;
log_options.append(args.context.log, args.context.file_path.namespace);
}
};
const OnBeforeParseResultWrapper = struct {
original_source: ?[]const u8 = null,
loader: Loader,
impl: OnBeforeParseResult,
};
const OnBeforeParseResult = extern struct {
struct_size: usize = @sizeOf(OnBeforeParseResult),
source_ptr: ?[*]const u8 = null,
source_len: usize = 0,
loader: Loader,
fetch_source_code_fn: *const fn (*const OnBeforeParseArguments, *OnBeforeParseResult) callconv(.C) i32 = &fetchSourceCode,
user_context: ?*anyopaque = null,
free_user_context: ?*const fn (?*anyopaque) callconv(.C) void = null,
log: *const fn (
args_: ?*OnBeforeParseArguments,
log_options_: ?*BunLogOptions,
) callconv(.C) void = &BunLogOptions.logFn,
};
pub fn fetchSourceCode(args: *const OnBeforeParseArguments, result: *OnBeforeParseResult) callconv(.C) i32 {
debug("fetchSourceCode", .{});
const this = args.context;
if (this.log.errors > 0 or this.deferred_error != null or this.should_continue_running.* != 1) {
return 1;
}
if (result.source_ptr != null) {
return 0;
}
const entry = getCodeForParseTaskWithoutPlugins(
this.task,
this.log,
this.bundler,
this.resolver,
this.allocator,
this.file_path,
result.loader,
this.experimental_css,
) catch |err| {
this.deferred_error = err;
this.should_continue_running.* = 0;
return 1;
};
result.source_ptr = entry.contents.ptr;
result.source_len = entry.contents.len;
result.free_user_context = null;
result.user_context = null;
return 0;
}
pub export fn OnBeforeParsePlugin__isDone(this: *OnBeforeParsePlugin) i32 {
if (this.should_continue_running.* != 1) {
return 1;
}
const result = this.result orelse return 1;
if (result.source_ptr != null) {
return 1;
}
return 0;
}
pub fn run(this: *OnBeforeParsePlugin, plugin: *JSC.API.JSBundler.Plugin, from_plugin: *bool) !CacheEntry {
var args = OnBeforeParseArguments{
.context = this,
.path_ptr = this.file_path.text.ptr,
.path_len = this.file_path.text.len,
.default_loader = this.loader.*,
};
if (this.file_path.namespace.len > 0) {
args.namespace_ptr = this.file_path.namespace.ptr;
args.namespace_len = this.file_path.namespace.len;
}
var result = OnBeforeParseResult{
.loader = this.loader.*,
};
this.result = &result;
const count = plugin.callOnBeforeParsePlugins(
this,
if (bun.strings.eqlComptime(this.file_path.namespace, "file"))
&bun.String.empty
else
&bun.String.init(this.file_path.namespace),
&bun.String.init(this.file_path.text),
&args,
&result,
this.should_continue_running,
);
if (comptime Environment.enable_logs)
debug("callOnBeforeParsePlugins({s}:{s}) = {d}", .{ this.file_path.namespace, this.file_path.text, count });
if (count > 0) {
if (this.deferred_error) |err| {
if (result.free_user_context) |free_user_context| {
free_user_context(result.user_context);
}
return err;
}
// If the plugin sets the `free_user_context` function pointer, it _must_ set the `user_context` pointer.
// Otherwise this is just invalid behavior.
if (result.user_context == null and result.free_user_context != null) {
var msg = Logger.Msg{ .data = .{ .location = null, .text = bun.default_allocator.dupe(
u8,
"Native plugin set the `free_plugin_source_code_context` field without setting the `plugin_source_code_context` field.",
) catch bun.outOfMemory() } };
msg.kind = .err;
args.context.log.errors += 1;
args.context.log.addMsg(msg) catch bun.outOfMemory();
return error.InvalidNativePlugin;
}
if (this.log.errors > 0) {
if (result.free_user_context) |free_user_context| {
free_user_context(result.user_context);
}
return error.SyntaxError;
}
if (result.source_ptr) |ptr| {
if (result.free_user_context != null) {
this.task.external = CacheEntry.External{
.ctx = result.user_context,
.function = result.free_user_context,
};
}
from_plugin.* = true;
this.loader.* = result.loader;
return CacheEntry{
.contents = ptr[0..result.source_len],
.external = .{
.ctx = result.user_context,
.function = result.free_user_context,
},
};
}
}
return try getCodeForParseTaskWithoutPlugins(this.task, this.log, this.bundler, this.resolver, this.allocator, this.file_path, this.loader.*, this.experimental_css);
}
};
fn run(
task: *ParseTask,
this: *ThreadPool.Worker,
step: *ParseTask.Result.Error.Step,
log: *Logger.Log,
) anyerror!Result.Success {
const allocator = this.allocator;
var data = this.data;
var bundler = &data.bundler;
errdefer bundler.resetStore();
var resolver: *Resolver = &bundler.resolver;
var file_path = task.path;
step.* = .read_file;
var loader = task.loader orelse file_path.loader(&bundler.options.loaders) orelse options.Loader.file;
var contents_came_from_plugin: bool = false;
var entry = try getCodeForParseTask(task, log, bundler, resolver, allocator, &file_path, &loader, this.ctx.bundler.options.experimental_css, &contents_came_from_plugin);
// WARNING: Do not change the variant of `task.contents_or_fd` from
// `.fd` to `.contents` (or back) after this point!
@@ -4033,6 +4400,7 @@ pub const ParseTask = struct {
.ctx = this.ctx,
.task = undefined,
.value = value,
.external = this.external,
.watcher_data = .{
.fd = if (this.contents_or_fd == .fd) this.contents_or_fd.fd.file else bun.invalid_fd,
.dir_fd = if (this.contents_or_fd == .fd) this.contents_or_fd.fd.dir else bun.invalid_fd,
@@ -4511,6 +4879,7 @@ pub const Graph = struct {
source: Logger.Source,
loader: options.Loader = options.Loader.file,
side_effects: _resolver.SideEffects,
allocator: std.mem.Allocator = bun.default_allocator,
additional_files: BabyList(AdditionalFile) = .{},
unique_key_for_additional_file: string = "",
content_hash_for_additional_file: u64 = 0,
@@ -15256,3 +15625,38 @@ pub fn generateUniqueKey() u64 {
}
return key;
}
const ExternalFreeFunctionAllocator = struct {
free_callback: *const fn (ctx: *anyopaque) void,
context: *anyopaque,
const vtable: std.mem.Allocator.VTable = .{
.alloc = &alloc,
.free = &free,
.resize = &resize,
};
pub fn create(free_callback: *const fn (ctx: *anyopaque) void, context: *anyopaque) std.mem.Allocator {
return .{
.ptr = bun.create(bun.default_allocator, ExternalFreeFunctionAllocator, .{
.free_callback = free_callback,
.context = context,
}),
.vtable = &vtable,
};
}
fn alloc(_: *anyopaque, _: usize, _: u8, _: usize) ?[*]u8 {
return null;
}
fn resize(_: *anyopaque, _: []u8, _: u8, _: usize, _: usize) bool {
return false;
}
fn free(ext_free_function: *anyopaque, _: []u8, _: u8, _: usize) void {
const info: *ExternalFreeFunctionAllocator = @alignCast(@ptrCast(ext_free_function));
info.free_callback(info.context);
bun.default_allocator.destroy(info);
}
};

View File

@@ -47,9 +47,23 @@ pub const Fs = struct {
pub const Entry = struct {
contents: string,
fd: StoredFileDescriptorType = bun.invalid_fd,
external: External = .{},
pub const External = struct {
ctx: ?*anyopaque = null,
function: ?*const fn (?*anyopaque) callconv(.C) void = null,
pub fn call(this: *const @This()) void {
if (this.function) |func| {
func(this.ctx);
}
}
};
pub fn deinit(entry: *Entry, allocator: std.mem.Allocator) void {
if (entry.contents.len > 0) {
if (entry.external.function) |func| {
func(entry.external.ctx);
} else if (entry.contents.len > 0) {
allocator.free(entry.contents);
entry.contents = "";
}

View File

@@ -50,6 +50,12 @@ var panic_mutex = std.Thread.Mutex{};
/// This is used to catch and handle panics triggered by the panic handler.
threadlocal var panic_stage: usize = 0;
threadlocal var inside_native_plugin: ?[*:0]const u8 = null;
export fn CrashHandler__setInsideNativePlugin(name: ?[*:0]const u8) callconv(.C) void {
inside_native_plugin = name;
}
/// This can be set by various parts of the codebase to indicate a broader
/// action being taken. It is printed when a crash happens, which can help
/// narrow down what the bug is. Example: "Crashed while parsing /path/to/file.js"
@@ -226,6 +232,18 @@ pub fn crashHandler(
writer.writeAll("=" ** 60 ++ "\n") catch std.posix.abort();
printMetadata(writer) catch std.posix.abort();
if (inside_native_plugin) |name| {
const native_plugin_name = name;
const fmt =
\\
\\Bun has encountered a crash while running the <red><d>"{s}"<r> native plugin.
\\
\\This indicates either a bug in the native plugin or in Bun.
\\
;
writer.print(Output.prettyFmt(fmt, true), .{native_plugin_name}) catch std.posix.abort();
}
} else {
if (Output.enable_ansi_colors) {
writer.writeAll(Output.prettyFmt("<red>", true)) catch std.posix.abort();
@@ -308,7 +326,17 @@ pub fn crashHandler(
} else {
writer.writeAll(Output.prettyFmt(": ", true)) catch std.posix.abort();
}
if (reason == .out_of_memory) {
if (inside_native_plugin) |name| {
const native_plugin_name = name;
writer.print(Output.prettyFmt(
\\Bun has encountered a crash while running the <red><d>"{s}"<r> native plugin.
\\
\\To send a redacted crash report to Bun's team,
\\please file a GitHub issue using the link below:
\\
\\
, true), .{native_plugin_name}) catch std.posix.abort();
} else if (reason == .out_of_memory) {
writer.writeAll(
\\Bun has ran out of memory.
\\

View File

@@ -254,6 +254,7 @@ using namespace JSC;
macro(writeRequests) \
macro(writing) \
macro(written) \
macro(napiDlopenHandle) \
BUN_ADDITIONAL_BUILTIN_NAMES(macro)
// --- END of BUN_COMMON_PRIVATE_IDENTIFIERS_EACH_PROPERTY_NAME ---

View File

@@ -1,11 +1,4 @@
import type {
BuildConfig,
BunPlugin,
OnLoadCallback,
OnResolveCallback,
PluginBuilder,
PluginConstraints,
} from "bun";
import type { BuildConfig, BunPlugin, OnLoadCallback, OnResolveCallback, PluginBuilder, PluginConstraints } from "bun";
type AnyFunction = (...args: any[]) => any;
interface BundlerPlugin {
@@ -46,6 +39,8 @@ interface PluginBuilderExt extends PluginBuilder {
esbuild: any;
}
type BeforeOnParseExternal = unknown;
export function runSetupFunction(
this: BundlerPlugin,
setup: Setup,
@@ -57,14 +52,31 @@ export function runSetupFunction(
this.promises = promises;
var onLoadPlugins = new Map<string, [RegExp, AnyFunction][]>();
var onResolvePlugins = new Map<string, [RegExp, AnyFunction][]>();
var onBeforeParsePlugins = new Map<
string,
[RegExp, napiModule: unknown, symbol: string, external?: undefined | unknown][]
>();
function validate(filterObject: PluginConstraints, callback, map) {
function validate(filterObject: PluginConstraints, callback, map, symbol, external) {
if (!filterObject || !$isObject(filterObject)) {
throw new TypeError('Expected an object with "filter" RegExp');
}
if (!callback || !$isCallable(callback)) {
throw new TypeError("callback must be a function");
let isOnBeforeParse = false;
if (map === onBeforeParsePlugins) {
isOnBeforeParse = true;
// TODO: how to check if it a napi module here?
if (!callback || !$isObject(callback) || !callback.$napiDlopenHandle) {
throw new TypeError("onBeforeParse `napiModule` must be a Napi module which exports the `BUN_PLUGIN_SYMBOL`");
}
if (typeof symbol !== "string") {
throw new TypeError("onBeforeParse `symbol` must be a string");
}
} else {
if (!callback || !$isCallable(callback)) {
throw new TypeError("lmao callback must be a function");
}
}
var { filter, namespace = "file" } = filterObject;
@@ -92,23 +104,30 @@ export function runSetupFunction(
var callbacks = map.$get(namespace);
if (!callbacks) {
map.$set(namespace, [[filter, callback]]);
map.$set(namespace, [isOnBeforeParse ? [filter, callback, symbol, external] : [filter, callback]]);
} else {
$arrayPush(callbacks, [filter, callback]);
$arrayPush(callbacks, isOnBeforeParse ? [filter, callback, symbol, external] : [filter, callback]);
}
}
function onLoad(filterObject, callback) {
validate(filterObject, callback, onLoadPlugins);
validate(filterObject, callback, onLoadPlugins, undefined, undefined);
}
function onResolve(filterObject, callback) {
validate(filterObject, callback, onResolvePlugins);
validate(filterObject, callback, onResolvePlugins, undefined, undefined);
}
function onBeforeParse(
filterObject,
{ napiModule, external, symbol }: { napiModule: unknown; symbol: string; external?: undefined | unknown },
) {
validate(filterObject, napiModule, onBeforeParsePlugins, symbol, external);
}
const self = this;
function onStart(callback) {
if(isBake) {
if (isBake) {
throw new TypeError("onStart() is not supported in Bake yet");
}
if (!$isCallable(callback)) {
@@ -126,7 +145,8 @@ export function runSetupFunction(
const processSetupResult = () => {
var anyOnLoad = false,
anyOnResolve = false;
anyOnResolve = false,
anyOnBeforeParse = false;
for (var [namespace, callbacks] of onLoadPlugins.entries()) {
for (var [filter] of callbacks) {
@@ -142,6 +162,13 @@ export function runSetupFunction(
}
}
for (let [namespace, callbacks] of onBeforeParsePlugins.entries()) {
for (let [filter, addon, symbol, external] of callbacks) {
this.onBeforeParse(filter, namespace, addon, symbol, external);
anyOnBeforeParse = true;
}
}
if (anyOnResolve) {
var onResolveObject = this.onResolve;
if (!onResolveObject) {
@@ -189,6 +216,7 @@ export function runSetupFunction(
onEnd: notImplementedIssueFn(2771, "On-end callbacks"),
onLoad,
onResolve,
onBeforeParse,
onStart,
resolve: notImplementedIssueFn(2771, "build.resolve()"),
module: () => {
@@ -335,7 +363,14 @@ export function runOnResolvePlugins(this: BundlerPlugin, specifier, inputNamespa
}
}
export function runOnLoadPlugins(this: BundlerPlugin, internalID, path, namespace, defaultLoaderId, isServerSide: boolean) {
export function runOnLoadPlugins(
this: BundlerPlugin,
internalID,
path,
namespace,
defaultLoaderId,
isServerSide: boolean,
) {
const LOADERS_MAP = $LoaderLabelToId;
const loaderName = $LoaderIdToLabel[defaultLoaderId];
@@ -376,15 +411,15 @@ export function runOnLoadPlugins(this: BundlerPlugin, internalID, path, namespac
}
var { contents, loader = defaultLoader } = result as any;
if ((loader as any) === 'object') {
if (!('exports' in result)) {
if ((loader as any) === "object") {
if (!("exports" in result)) {
throw new TypeError('onLoad plugin returning loader: "object" must have "exports" property');
}
try {
contents = JSON.stringify(result.exports);
loader = 'json';
loader = "json";
} catch (e) {
throw new TypeError('When using Bun.build, onLoad plugin must return a JSON-serializable object: ' + e) ;
throw new TypeError("When using Bun.build, onLoad plugin must return a JSON-serializable object: " + e);
}
}

View File

@@ -0,0 +1,679 @@
import { BunFile, Loader, plugin } from "bun";
import { afterEach, beforeAll, beforeEach, describe, expect, it } from "bun:test";
import path, { dirname, join, resolve } from "path";
import source from "./native_plugin.cc" with { type: "file" };
import notAPlugin from "./not_native_plugin.cc" with { type: "file" };
import bundlerPluginHeader from "../../packages/bun-native-bundler-plugin-api/bundler_plugin.h" with { type: "file" };
import { bunEnv, bunExe, tempDirWithFiles } from "harness";
import { itBundled } from "bundler/expectBundled";
describe("native-plugins", async () => {
const cwd = process.cwd();
let tempdir: string = "";
let outdir: string = "";
beforeAll(async () => {
const files = {
"bun-native-bundler-plugin-api/bundler_plugin.h": await Bun.file(bundlerPluginHeader).text(),
"plugin.cc": await Bun.file(source).text(),
"not_a_plugin.cc": await Bun.file(notAPlugin).text(),
"package.json": JSON.stringify({
"name": "fake-plugin",
"module": "index.ts",
"type": "module",
"devDependencies": {
"@types/bun": "latest",
},
"peerDependencies": {
"typescript": "^5.0.0",
},
"scripts": {
"build:napi": "node-gyp configure && node-gyp build",
},
"dependencies": {
"node-gyp": "10.2.0",
},
}),
"index.ts": /* ts */ `import values from "./stuff.ts";
import json from "./lmao.json";
const many_foo = ["foo","foo","foo","foo","foo","foo","foo"]
const many_bar = ["bar","bar","bar","bar","bar","bar","bar"]
const many_baz = ["baz","baz","baz","baz","baz","baz","baz"]
console.log(JSON.stringify(json));
values;`,
"stuff.ts": `export default { foo: "bar", baz: "baz" }`,
"lmao.json": ``,
"binding.gyp": /* gyp */ `{
"targets": [
{
"target_name": "xXx123_foo_counter_321xXx",
"sources": [ "plugin.cc" ],
"include_dirs": [ "." ]
},
{
"target_name": "not_a_plugin",
"sources": [ "not_a_plugin.cc" ],
"include_dirs": [ "." ]
}
]
}`,
};
tempdir = tempDirWithFiles("native-plugins", files);
outdir = path.join(tempdir, "dist");
console.log("tempdir", tempdir);
process.chdir(tempdir);
await Bun.$`${bunExe()} i && ${bunExe()} build:napi`.env(bunEnv).cwd(tempdir);
});
afterEach(async () => {
await Bun.$`rm -rf ${outdir}`;
process.chdir(cwd);
});
it("works in a basic case", async () => {
await Bun.$`${bunExe()} i && ${bunExe()} build:napi`.env(bunEnv).cwd(tempdir);
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
const external = napiModule.createExternal();
const result = await Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "xXx123_foo_counter_321xXx",
setup(build) {
build.onBeforeParse({ filter: /\.ts/ }, { napiModule, symbol: "plugin_impl", external });
build.onLoad({ filter: /lmao\.json/ }, async ({ defer }) => {
await defer();
const count = napiModule.getFooCount(external);
return {
contents: JSON.stringify({ fooCount: count }),
loader: "json",
};
});
},
},
],
});
if (!result.success) console.log(result);
expect(result.success).toBeTrue();
const output = await Bun.$`${bunExe()} run dist/index.js`.cwd(tempdir).json();
expect(output).toStrictEqual({ fooCount: 9 });
const compilationCtxFreedCount = await napiModule.getCompilationCtxFreedCount(external);
expect(compilationCtxFreedCount).toBe(2);
});
it("doesn't explode when there are a lot of concurrent files", async () => {
// Generate 100 json files
const files: [filepath: string, var_name: string][] = await Promise.all(
Array.from({ length: 100 }, async (_, i) => {
await Bun.write(path.join(tempdir, "json_files", `lmao${i}.json`), `{}`);
return [`import json${i} from "./json_files/lmao${i}.json"`, `json${i}`];
}),
);
// Append the imports to index.ts
const prelude = /* ts */ `import values from "./stuff.ts"
const many_foo = ["foo","foo","foo","foo","foo","foo","foo"]
`;
await Bun.$`echo ${prelude} > index.ts`;
await Bun.$`echo ${files.map(([fp]) => fp).join("\n")} >> index.ts`;
await Bun.$`echo ${files.map(([, varname]) => `console.log(JSON.stringify(${varname}))`).join("\n")} >> index.ts`;
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
const external = napiModule.createExternal();
const result = await Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "xXx123_foo_counter_321xXx",
setup(build) {
build.onBeforeParse({ filter: /\.ts/ }, { napiModule, symbol: "plugin_impl", external });
build.onLoad({ filter: /\.json/ }, async ({ defer, path }) => {
await defer();
const count = napiModule.getFooCount(external);
return {
contents: JSON.stringify({ fooCount: count }),
loader: "json",
};
});
},
},
],
});
if (!result.success) console.log(result);
console.log(result);
expect(result.success).toBeTrue();
const output = await Bun.$`${bunExe()} run dist/index.js`.cwd(tempdir).text();
const outputJsons = output
.trim()
.split("\n")
.map(s => JSON.parse(s));
for (const json of outputJsons) {
expect(json).toStrictEqual({ fooCount: 9 });
}
const compilationCtxFreedCount = await napiModule.getCompilationCtxFreedCount(external);
expect(compilationCtxFreedCount).toBe(2);
});
// We clone the RegExp object in the C++ code so this test ensures that there
// is no funny business regarding the filter regular expression and multiple
// threads
it("doesn't explode when there are a lot of concurrent files AND the filter regex is used on the JS thread", async () => {
const filter = /\.ts/;
// Generate 100 json files
const files: [filepath: string, var_name: string][] = await Promise.all(
Array.from({ length: 100 }, async (_, i) => {
await Bun.write(path.join(tempdir, "json_files", `lmao${i}.json`), `{}`);
return [`import json${i} from "./json_files/lmao${i}.json"`, `json${i}`];
}),
);
// Append the imports to index.ts
const prelude = /* ts */ `import values from "./stuff.ts"
const many_foo = ["foo","foo","foo","foo","foo","foo","foo"]
`;
await Bun.$`echo ${prelude} > index.ts`;
await Bun.$`echo ${files.map(([fp]) => fp).join("\n")} >> index.ts`;
await Bun.$`echo ${files.map(([, varname]) => `console.log(JSON.stringify(${varname}))`).join("\n")} >> index.ts`;
await Bun.$`echo '(() => values)();' >> index.ts`;
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
const external = napiModule.createExternal();
const resultPromise = Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "xXx123_foo_counter_321xXx",
setup(build) {
build.onBeforeParse({ filter }, { napiModule, symbol: "plugin_impl", external });
build.onLoad({ filter: /\.json/ }, async ({ defer, path }) => {
await defer();
const count = napiModule.getFooCount(external);
return {
contents: JSON.stringify({ fooCount: count }),
loader: "json",
};
});
},
},
],
});
// Now saturate this thread with uses of the filter regex to test that nothing bad happens
// when the JS thread and the bundler thread use regexes concurrently
let dummy = 0;
for (let i = 0; i < 10000; i++) {
// Match the filter regex on some dummy string
dummy += filter.test("foo") ? 1 : 0;
}
const result = await resultPromise;
if (!result.success) console.log(result);
expect(result.success).toBeTrue();
const output = await Bun.$`${bunExe()} run dist/index.js`.cwd(tempdir).text();
const outputJsons = output
.trim()
.split("\n")
.map(s => JSON.parse(s));
for (const json of outputJsons) {
expect(json).toStrictEqual({ fooCount: 9 });
}
const compilationCtxFreedCount = await napiModule.getCompilationCtxFreedCount(external);
expect(compilationCtxFreedCount).toBe(2);
});
it("doesn't explode when passing invalid external", async () => {
const filter = /\.ts/;
// Generate 100 json files
const files: [filepath: string, var_name: string][] = await Promise.all(
Array.from({ length: 100 }, async (_, i) => {
await Bun.write(path.join(tempdir, "json_files", `lmao${i}.json`), `{}`);
return [`import json${i} from "./json_files/lmao${i}.json"`, `json${i}`];
}),
);
// Append the imports to index.ts
const prelude = /* ts */ `import values from "./stuff.ts"
const many_foo = ["foo","foo","foo","foo","foo","foo","foo"]
`;
await Bun.$`echo ${prelude} > index.ts`;
await Bun.$`echo ${files.map(([fp]) => fp).join("\n")} >> index.ts`;
await Bun.$`echo ${files.map(([, varname]) => `console.log(JSON.stringify(${varname}))`).join("\n")} >> index.ts`;
const resultPromise = Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "xXx123_foo_counter_321xXx",
setup(build) {
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
const external = undefined;
build.onBeforeParse({ filter }, { napiModule, symbol: "plugin_impl", external });
build.onLoad({ filter: /\.json/ }, async ({ defer, path }) => {
await defer();
let count = 0;
try {
count = napiModule.getFooCount(external);
} catch (e) {}
return {
contents: JSON.stringify({ fooCount: count }),
loader: "json",
};
});
},
},
],
});
const result = await resultPromise;
if (!result.success) console.log(result);
expect(result.success).toBeTrue();
const output = await Bun.$`${bunExe()} run dist/index.js`.cwd(tempdir).text();
const outputJsons = output
.trim()
.split("\n")
.map(s => JSON.parse(s));
for (const json of outputJsons) {
expect(json).toStrictEqual({ fooCount: 0 });
}
});
it("works when logging an error", async () => {
const filter = /\.ts/;
const prelude = /* ts */ `import values from "./stuff.ts"
const many_foo = ["foo","foo","foo","foo","foo","foo","foo"]
`;
await Bun.$`echo ${prelude} > index.ts`;
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
const external = napiModule.createExternal();
const resultPromise = Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "xXx123_foo_counter_321xXx",
setup(build) {
napiModule.setThrowsErrors(external, true);
build.onBeforeParse({ filter }, { napiModule, symbol: "plugin_impl", external });
build.onLoad({ filter: /\.json/ }, async ({ defer, path }) => {
await defer();
let count = 0;
try {
count = napiModule.getFooCount(external);
} catch (e) {}
return {
contents: JSON.stringify({ fooCount: count }),
loader: "json",
};
});
},
},
],
});
const result = await resultPromise;
if (result.success) console.log(result);
expect(result.success).toBeFalse();
const log = result.logs[0];
expect(log.message).toContain("Throwing an error");
expect(log.level).toBe("error");
const compilationCtxFreedCount = await napiModule.getCompilationCtxFreedCount(external);
expect(compilationCtxFreedCount).toBe(0);
});
it("works with versioning", async () => {
const filter = /\.ts/;
const prelude = /* ts */ `import values from "./stuff.ts"
const many_foo = ["foo","foo","foo","foo","foo","foo","foo"]
`;
await Bun.$`echo ${prelude} > index.ts`;
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
const external = napiModule.createExternal();
const resultPromise = Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "xXx123_foo_counter_321xXx",
setup(build) {
build.onBeforeParse({ filter }, { napiModule, symbol: "incompatible_version_plugin_impl", external });
build.onLoad({ filter: /\.json/ }, async ({ defer, path }) => {
await defer();
let count = 0;
try {
count = napiModule.getFooCount(external);
} catch (e) {}
return {
contents: JSON.stringify({ fooCount: count }),
loader: "json",
};
});
},
},
],
});
const result = await resultPromise;
if (result.success) console.log(result);
expect(result.success).toBeFalse();
const log = result.logs[0];
expect(log.message).toContain("This plugin is built for a newer version of Bun than the one currently running.");
expect(log.level).toBe("error");
const compilationCtxFreedCount = await napiModule.getCompilationCtxFreedCount(external);
expect(compilationCtxFreedCount).toBe(0);
});
// don't know how to reliably test this on windows
it.skipIf(process.platform === "win32")("prints name when plugin crashes", async () => {
const prelude = /* ts */ `import values from "./stuff.ts"
const many_foo = ["foo","foo","foo","foo","foo","foo","foo"]
`;
await Bun.$`echo ${prelude} > index.ts`;
const build_code = /* ts */ `
import * as path from "path";
const tempdir = process.env.BUN_TEST_TEMP_DIR;
const filter = /\.ts/;
const resultPromise = await Bun.build({
outdir: "dist",
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "xXx123_foo_counter_321xXx",
setup(build) {
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
const external = napiModule.createExternal();
napiModule.setWillCrash(external, true);
build.onBeforeParse({ filter }, { napiModule, symbol: "plugin_impl", external });
build.onLoad({ filter: /\.json/ }, async ({ defer, path }) => {
await defer();
let count = 0;
try {
count = napiModule.getFooCount(external);
} catch (e) {}
return {
contents: JSON.stringify({ fooCount: count }),
loader: "json",
};
});
},
},
],
});
console.log(resultPromise);
`;
await Bun.$`echo ${build_code} > build.ts`;
const { stdout, stderr } = await Bun.$`BUN_TEST_TEMP_DIR=${tempdir} ${bunExe()} run build.ts`.throws(false);
const errorString = stderr.toString();
expect(errorString).toContain('\x1b[31m\x1b[2m"native_plugin_test"\x1b[0m');
});
it("detects when plugin sets function pointer but does not user context pointer", async () => {
const filter = /\.ts/;
const prelude = /* ts */ `import values from "./stuff.ts"
const many_foo = ["foo","foo","foo","foo","foo","foo","foo"]
`;
await Bun.$`echo ${prelude} > index.ts`;
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
const external = napiModule.createExternal();
const resultPromise = Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "xXx123_foo_counter_321xXx",
setup(build) {
build.onBeforeParse({ filter }, { napiModule, symbol: "plugin_impl_bad_free_function_pointer", external });
build.onLoad({ filter: /\.json/ }, async ({ defer, path }) => {
await defer();
let count = 0;
try {
count = napiModule.getFooCount(external);
} catch (e) {}
return {
contents: JSON.stringify({ fooCount: count }),
loader: "json",
};
});
},
},
],
});
const result = await resultPromise;
if (result.success) console.log(result);
expect(result.success).toBeFalse();
const log = result.logs[0];
expect(log.message).toContain(
"Native plugin set the `free_plugin_source_code_context` field without setting the `plugin_source_code_context` field.",
);
expect(log.level).toBe("error");
const compilationCtxFreedCount = await napiModule.getCompilationCtxFreedCount(external);
expect(compilationCtxFreedCount).toBe(0);
});
it("should fail gracefully when passing something that is NOT a bunler plugin", async () => {
const not_plugins = [require(path.join(tempdir, "build/Release/not_a_plugin.node")), 420, "hi", {}];
for (const napiModule of not_plugins) {
try {
await Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "not_a_plugin",
setup(build) {
build.onBeforeParse({ filter: /\.ts/ }, { napiModule, symbol: "plugin_impl" });
},
},
],
});
expect.unreachable();
} catch (e) {
expect(e.toString()).toContain(
"onBeforeParse `napiModule` must be a Napi module which exports the `BUN_PLUGIN_SYMBOL`",
);
}
}
});
it("should fail gracefully when can't find the symbol", async () => {
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
try {
await Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "not_a_plugin",
setup(build) {
build.onBeforeParse({ filter: /\.ts/ }, { napiModule, symbol: "OOGA_BOOGA_420" });
},
},
],
});
expect.unreachable();
} catch (e) {
expect(e.toString()).toContain('TypeError: Could not find the symbol "OOGA_BOOGA_420" in the given napi module.');
}
});
it("should use result of the first plugin that runs and doesn't execute the others", async () => {
const filter = /\.ts/;
const prelude = /* ts */ `import values from "./stuff.ts"
import json from "./lmao.json";
const many_foo = ["foo","foo","foo","foo","foo","foo","foo"]
const many_bar = ["bar","bar","bar","bar","bar","bar","bar"]
const many_baz = ["baz","baz","baz","baz","baz","baz","baz"]
console.log(JSON.stringify(json))
`;
await Bun.$`echo ${prelude} > index.ts`;
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
const external = napiModule.createExternal();
const resultPromise = Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "xXx123_foo_counter_321xXx",
setup(build) {
build.onBeforeParse({ filter }, { napiModule, symbol: "plugin_impl", external });
build.onBeforeParse({ filter }, { napiModule, symbol: "plugin_impl_bar", external });
build.onBeforeParse({ filter }, { napiModule, symbol: "plugin_impl_baz", external });
build.onLoad({ filter: /\.json/ }, async ({ defer, path }) => {
await defer();
let fooCount = 0;
let barCount = 0;
let bazCount = 0;
try {
fooCount = napiModule.getFooCount(external);
barCount = napiModule.getBarCount(external);
bazCount = napiModule.getBazCount(external);
} catch (e) {}
return {
contents: JSON.stringify({ fooCount, barCount, bazCount }),
loader: "json",
};
});
},
},
],
});
const result = await resultPromise;
if (result.success) console.log(result);
expect(result.success).toBeTrue();
const output = await Bun.$`${bunExe()} run dist/index.js`.cwd(tempdir).json();
expect(output).toStrictEqual({ fooCount: 9, barCount: 0, bazCount: 0 });
const compilationCtxFreedCount = await napiModule.getCompilationCtxFreedCount(external);
expect(compilationCtxFreedCount).toBe(2);
});
type AdditionalFile = {
name: string;
contents: BunFile | string;
loader: Loader;
};
const additional_files: AdditionalFile[] = [
{
name: "bun.png",
contents: await Bun.file(path.join(import.meta.dir, "../integration/sharp/bun.png")),
loader: "file",
},
{
name: "index.js",
contents: /* ts */ `console.log('HELLO FRIENDS')`,
loader: "js",
},
{
name: "index.ts",
contents: /* ts */ `console.log('HELLO FRIENDS')`,
loader: "ts",
},
{
name: "lmao.jsx",
contents: /* ts */ `console.log('HELLO FRIENDS')`,
loader: "jsx",
},
{
name: "lmao.tsx",
contents: /* ts */ `console.log('HELLO FRIENDS')`,
loader: "tsx",
},
{
name: "lmao.toml",
contents: /* toml */ `foo = "bar"`,
loader: "toml",
},
{
name: "lmao.text",
contents: "HELLO FRIENDS",
loader: "text",
},
];
for (const { name, contents, loader } of additional_files) {
it(`works with ${loader} loader`, async () => {
await Bun.$`echo ${contents} > ${name}`;
const source = /* ts */ `import foo from "./${name}";
console.log(foo);`;
await Bun.$`echo ${source} > index.ts`;
const result = await Bun.build({
outdir,
entrypoints: [path.join(tempdir, "index.ts")],
plugins: [
{
name: "test",
setup(build) {
const ext = name.split(".").pop()!;
const napiModule = require(path.join(tempdir, "build/Release/xXx123_foo_counter_321xXx.node"));
// Construct regexp to match the file extension
const filter = new RegExp(`\\.${ext}$`);
build.onBeforeParse({ filter }, { napiModule, symbol: "plugin_impl" });
},
},
],
});
expect(result.success).toBeTrue();
});
}
});

View File

@@ -0,0 +1,651 @@
/*
Dummy plugin which counts the occurences of the word "foo" in the source code,
replacing it with "boo".
It stores the number of occurences in the External struct.
*/
#include <atomic>
#include <bun-native-bundler-plugin-api/bundler_plugin.h>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <node_api.h>
#ifdef _WIN32
#define BUN_PLUGIN_EXPORT __declspec(dllexport)
#else
#define BUN_PLUGIN_EXPORT
#include <signal.h>
#include <unistd.h>
#endif
BUN_PLUGIN_EXPORT const char *BUN_PLUGIN_NAME = "native_plugin_test";
struct External {
std::atomic<size_t> foo_count;
std::atomic<size_t> bar_count;
std::atomic<size_t> baz_count;
// For testing logging error logic
std::atomic<bool> throws_an_error;
// For testing crash reporting
std::atomic<bool> simulate_crash;
std::atomic<size_t> compilation_ctx_freed_count;
};
struct CompilationCtx {
const char *source_ptr;
size_t source_len;
std::atomic<size_t> *free_counter;
};
CompilationCtx *compilation_ctx_new(const char *source_ptr, size_t source_len,
std::atomic<size_t> *free_counter) {
CompilationCtx *ctx = new CompilationCtx;
ctx->source_ptr = source_ptr;
ctx->source_len = source_len;
ctx->free_counter = free_counter;
return ctx;
}
void compilation_ctx_free(CompilationCtx *ctx) {
printf("Freed compilation ctx!\n");
if (ctx->free_counter != nullptr) {
ctx->free_counter->fetch_add(1);
}
free((void *)ctx->source_ptr);
delete ctx;
}
void log_error(const OnBeforeParseArguments *args,
const OnBeforeParseResult *result, BunLogLevel level,
const char *message, size_t message_len) {
BunLogOptions options;
options.message_ptr = (uint8_t *)message;
options.message_len = message_len;
options.path_ptr = args->path_ptr;
options.path_len = args->path_len;
options.source_line_text_ptr = nullptr;
options.source_line_text_len = 0;
options.level = (int8_t)level;
options.line = 0;
options.lineEnd = 0;
options.column = 0;
options.columnEnd = 0;
(result->log)(args, &options);
}
extern "C" BUN_PLUGIN_EXPORT void
plugin_impl_with_needle(const OnBeforeParseArguments *args,
OnBeforeParseResult *result, const char *needle) {
// if (args->__struct_size < sizeof(OnBeforeParseArguments)) {
// log_error(args, result, BUN_LOG_LEVEL_ERROR, "Invalid
// OnBeforeParseArguments struct size", sizeof("Invalid
// OnBeforeParseArguments struct size") - 1); return;
// }
if (args->external) {
External *external = (External *)args->external;
if (external->throws_an_error.load()) {
log_error(args, result, BUN_LOG_LEVEL_ERROR, "Throwing an error",
sizeof("Throwing an error") - 1);
return;
} else if (external->simulate_crash.load()) {
#ifndef _WIN32
raise(SIGSEGV);
#endif
}
}
int fetch_result = result->fetchSourceCode(args, result);
if (fetch_result != 0) {
printf("FUCK\n");
exit(1);
}
size_t needle_len = strlen(needle);
int needle_count = 0;
const char *end = (const char *)result->source_ptr + result->source_len;
char *cursor = (char *)strstr((const char *)result->source_ptr, needle);
while (cursor != nullptr) {
needle_count++;
cursor += needle_len;
if (cursor + needle_len < end) {
cursor = (char *)strstr((const char *)cursor, needle);
} else
break;
}
if (needle_count > 0) {
char *new_source = (char *)malloc(result->source_len);
if (new_source == nullptr) {
printf("FUCK\n");
exit(1);
}
memcpy(new_source, result->source_ptr, result->source_len);
cursor = strstr(new_source, needle);
while (cursor != nullptr) {
cursor[0] = 'q';
cursor += 3;
if (cursor + 3 < end) {
cursor = (char *)strstr((const char *)cursor, needle);
} else
break;
}
std::atomic<size_t> *free_counter = nullptr;
if (args->external) {
External *external = (External *)args->external;
std::atomic<size_t> *needle_atomic_value = nullptr;
if (strcmp(needle, "foo") == 0) {
needle_atomic_value = &external->foo_count;
} else if (strcmp(needle, "bar") == 0) {
needle_atomic_value = &external->bar_count;
} else if (strcmp(needle, "baz") == 0) {
needle_atomic_value = &external->baz_count;
}
printf("FUCK: %d %s\n", needle_count, needle);
needle_atomic_value->fetch_add(needle_count);
free_counter = &external->compilation_ctx_freed_count;
}
result->source_ptr = (uint8_t *)new_source;
result->source_len = result->source_len;
result->plugin_source_code_context =
compilation_ctx_new(new_source, result->source_len, free_counter);
result->free_plugin_source_code_context =
(void (*)(void *))compilation_ctx_free;
} else {
result->source_ptr = nullptr;
result->source_len = 0;
result->loader = 0;
}
}
extern "C" BUN_PLUGIN_EXPORT void
plugin_impl(const OnBeforeParseArguments *args, OnBeforeParseResult *result) {
plugin_impl_with_needle(args, result, "foo");
}
extern "C" BUN_PLUGIN_EXPORT void
plugin_impl_bar(const OnBeforeParseArguments *args,
OnBeforeParseResult *result) {
plugin_impl_with_needle(args, result, "bar");
}
extern "C" BUN_PLUGIN_EXPORT void
plugin_impl_baz(const OnBeforeParseArguments *args,
OnBeforeParseResult *result) {
plugin_impl_with_needle(args, result, "baz");
}
extern "C" void finalizer(napi_env env, void *data, void *hint) {
External *external = (External *)data;
if (external != nullptr) {
delete external;
}
}
napi_value create_external(napi_env env, napi_callback_info info) {
napi_status status;
// Allocate the External struct
External *external = new External();
if (external == nullptr) {
napi_throw_error(env, nullptr, "Failed to allocate memory");
return nullptr;
}
external->foo_count = 0;
external->compilation_ctx_freed_count = 0;
// Create the external wrapper
napi_value result;
status = napi_create_external(env, external, finalizer, nullptr, &result);
if (status != napi_ok) {
delete external;
napi_throw_error(env, nullptr, "Failed to create external");
return nullptr;
}
return result;
}
napi_value set_will_crash(napi_env env, napi_callback_info info) {
napi_status status;
External *external;
size_t argc = 1;
napi_value args[1];
status = napi_get_cb_info(env, info, &argc, args, nullptr, nullptr);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to parse arguments");
return nullptr;
}
if (argc < 1) {
napi_throw_error(env, nullptr, "Wrong number of arguments");
return nullptr;
}
status = napi_get_value_external(env, args[0], (void **)&external);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to get external");
return nullptr;
}
bool throws;
status = napi_get_value_bool(env, args[0], &throws);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to get boolean value");
return nullptr;
}
external->simulate_crash.store(throws);
return nullptr;
}
napi_value set_throws_errors(napi_env env, napi_callback_info info) {
napi_status status;
External *external;
size_t argc = 1;
napi_value args[1];
status = napi_get_cb_info(env, info, &argc, args, nullptr, nullptr);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to parse arguments");
return nullptr;
}
if (argc < 1) {
napi_throw_error(env, nullptr, "Wrong number of arguments");
return nullptr;
}
status = napi_get_value_external(env, args[0], (void **)&external);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to get external");
return nullptr;
}
bool throws;
status = napi_get_value_bool(env, args[0], &throws);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to get boolean value");
return nullptr;
}
external->throws_an_error.store(throws);
return nullptr;
}
napi_value get_compilation_ctx_freed_count(napi_env env,
napi_callback_info info) {
napi_status status;
External *external;
size_t argc = 1;
napi_value args[1];
status = napi_get_cb_info(env, info, &argc, args, nullptr, nullptr);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to parse arguments");
return nullptr;
}
if (argc < 1) {
napi_throw_error(env, nullptr, "Wrong number of arguments");
return nullptr;
}
status = napi_get_value_external(env, args[0], (void **)&external);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to get external");
return nullptr;
}
napi_value result;
status = napi_create_int32(env, external->compilation_ctx_freed_count.load(),
&result);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to create array");
return nullptr;
}
return result;
}
napi_value get_foo_count(napi_env env, napi_callback_info info) {
napi_status status;
External *external;
size_t argc = 1;
napi_value args[1];
status = napi_get_cb_info(env, info, &argc, args, nullptr, nullptr);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to parse arguments");
return nullptr;
}
if (argc < 1) {
napi_throw_error(env, nullptr, "Wrong number of arguments");
return nullptr;
}
status = napi_get_value_external(env, args[0], (void **)&external);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to get external");
return nullptr;
}
size_t foo_count = external->foo_count.load();
if (foo_count > INT32_MAX) {
napi_throw_error(env, nullptr,
"Too many foos! This probably means undefined memory or "
"heap corruption.");
return nullptr;
}
napi_value result;
status = napi_create_int32(env, foo_count, &result);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to create array");
return nullptr;
}
return result;
}
napi_value get_bar_count(napi_env env, napi_callback_info info) {
napi_status status;
External *external;
size_t argc = 1;
napi_value args[1];
status = napi_get_cb_info(env, info, &argc, args, nullptr, nullptr);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to parse arguments");
return nullptr;
}
if (argc < 1) {
napi_throw_error(env, nullptr, "Wrong number of arguments");
return nullptr;
}
status = napi_get_value_external(env, args[0], (void **)&external);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to get external");
return nullptr;
}
size_t bar_count = external->bar_count.load();
if (bar_count > INT32_MAX) {
napi_throw_error(env, nullptr,
"Too many bars! This probably means undefined memory or "
"heap corruption.");
return nullptr;
}
napi_value result;
status = napi_create_int32(env, bar_count, &result);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to create array");
return nullptr;
}
return result;
}
napi_value get_baz_count(napi_env env, napi_callback_info info) {
napi_status status;
External *external;
size_t argc = 1;
napi_value args[1];
status = napi_get_cb_info(env, info, &argc, args, nullptr, nullptr);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to parse arguments");
return nullptr;
}
if (argc < 1) {
napi_throw_error(env, nullptr, "Wrong number of arguments");
return nullptr;
}
status = napi_get_value_external(env, args[0], (void **)&external);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to get external");
return nullptr;
}
size_t baz_count = external->baz_count.load();
if (baz_count > INT32_MAX) {
napi_throw_error(env, nullptr,
"Too many bazs! This probably means undefined memory or "
"heap corruption.");
return nullptr;
}
napi_value result;
status = napi_create_int32(env, baz_count, &result);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to create array");
return nullptr;
}
return result;
}
napi_value Init(napi_env env, napi_value exports) {
napi_status status;
napi_value fn_get_foo_count;
napi_value fn_get_bar_count;
napi_value fn_get_baz_count;
napi_value fn_get_compilation_ctx_freed_count;
napi_value fn_create_external;
napi_value fn_set_throws_errors;
napi_value fn_set_will_crash;
// Register get_foo_count function
status = napi_create_function(env, nullptr, 0, get_foo_count, nullptr,
&fn_get_foo_count);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to create get_names function");
return nullptr;
}
status =
napi_set_named_property(env, exports, "getFooCount", fn_get_foo_count);
if (status != napi_ok) {
napi_throw_error(env, nullptr,
"Failed to add get_names function to exports");
return nullptr;
}
// Register get_bar_count function
status = napi_create_function(env, nullptr, 0, get_bar_count, nullptr,
&fn_get_bar_count);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to create get_names function");
return nullptr;
}
status =
napi_set_named_property(env, exports, "getBarCount", fn_get_bar_count);
if (status != napi_ok) {
napi_throw_error(env, nullptr,
"Failed to add get_names function to exports");
return nullptr;
}
// Register get_baz_count function
status = napi_create_function(env, nullptr, 0, get_baz_count, nullptr,
&fn_get_baz_count);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to create get_names function");
return nullptr;
}
status =
napi_set_named_property(env, exports, "getBazCount", fn_get_baz_count);
if (status != napi_ok) {
napi_throw_error(env, nullptr,
"Failed to add get_names function to exports");
return nullptr;
}
// Register get_compilation_ctx_freed_count function
status =
napi_create_function(env, nullptr, 0, get_compilation_ctx_freed_count,
nullptr, &fn_get_compilation_ctx_freed_count);
if (status != napi_ok) {
napi_throw_error(
env, nullptr,
"Failed to create get_compilation_ctx_freed_count function");
return nullptr;
}
status = napi_set_named_property(env, exports, "getCompilationCtxFreedCount",
fn_get_compilation_ctx_freed_count);
if (status != napi_ok) {
napi_throw_error(
env, nullptr,
"Failed to add get_compilation_ctx_freed_count function to exports");
return nullptr;
}
// Register set_throws_errors function
status = napi_create_function(env, nullptr, 0, set_throws_errors, nullptr,
&fn_set_throws_errors);
if (status != napi_ok) {
napi_throw_error(env, nullptr,
"Failed to create set_throws_errors function");
return nullptr;
}
status = napi_set_named_property(env, exports, "setThrowsErrors",
fn_set_throws_errors);
if (status != napi_ok) {
napi_throw_error(env, nullptr,
"Failed to add set_throws_errors function to exports");
return nullptr;
}
// Register set_will_crash function
status = napi_create_function(env, nullptr, 0, set_will_crash, nullptr,
&fn_set_will_crash);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to create set_will_crash function");
return nullptr;
}
status =
napi_set_named_property(env, exports, "setWillCrash", fn_set_will_crash);
if (status != napi_ok) {
napi_throw_error(env, nullptr,
"Failed to add set_will_crash function to exports");
return nullptr;
}
// Register create_external function
status = napi_create_function(env, nullptr, 0, create_external, nullptr,
&fn_create_external);
if (status != napi_ok) {
napi_throw_error(env, nullptr, "Failed to create create_external function");
return nullptr;
}
status = napi_set_named_property(env, exports, "createExternal",
fn_create_external);
if (status != napi_ok) {
napi_throw_error(env, nullptr,
"Failed to add create_external function to exports");
return nullptr;
}
return exports;
}
struct NewOnBeforeParseArguments {
size_t __struct_size;
void *bun;
const uint8_t *path_ptr;
size_t path_len;
const uint8_t *namespace_ptr;
size_t namespace_len;
uint8_t default_loader;
void *external;
size_t new_field_one;
size_t new_field_two;
size_t new_field_three;
};
struct NewOnBeforeParseResult {
size_t __struct_size;
uint8_t *source_ptr;
size_t source_len;
uint8_t loader;
int (*fetchSourceCode)(const NewOnBeforeParseArguments *args,
struct NewOnBeforeParseResult *result);
void *plugin_source_code_context;
void (*free_plugin_source_code_context)(void *ctx);
void (*log)(const NewOnBeforeParseArguments *args, BunLogOptions *options);
size_t new_field_one;
size_t new_field_two;
size_t new_field_three;
};
void new_log_error(const NewOnBeforeParseArguments *args,
const NewOnBeforeParseResult *result, BunLogLevel level,
const char *message, size_t message_len) {
BunLogOptions options;
options.message_ptr = (uint8_t *)message;
options.message_len = message_len;
options.path_ptr = args->path_ptr;
options.path_len = args->path_len;
options.source_line_text_ptr = nullptr;
options.source_line_text_len = 0;
options.level = (int8_t)level;
options.line = 0;
options.lineEnd = 0;
options.column = 0;
options.columnEnd = 0;
(result->log)(args, &options);
}
extern "C" BUN_PLUGIN_EXPORT void
incompatible_version_plugin_impl(const NewOnBeforeParseArguments *args,
NewOnBeforeParseResult *result) {
if (args->__struct_size < sizeof(NewOnBeforeParseArguments)) {
const char *msg = "This plugin is built for a newer version of Bun than "
"the one currently running.";
new_log_error(args, result, BUN_LOG_LEVEL_ERROR, msg, strlen(msg));
return;
}
if (result->__struct_size < sizeof(NewOnBeforeParseResult)) {
const char *msg = "This plugin is built for a newer version of Bun than "
"the one currently running.";
new_log_error(args, result, BUN_LOG_LEVEL_ERROR, msg, strlen(msg));
return;
}
}
struct RandomUserContext {
const char *foo;
size_t bar;
};
extern "C" BUN_PLUGIN_EXPORT void random_user_context_free(void *ptr) {
free(ptr);
}
extern "C" BUN_PLUGIN_EXPORT void
plugin_impl_bad_free_function_pointer(const OnBeforeParseArguments *args,
OnBeforeParseResult *result) {
// Intentionally not setting the context here:
// result->plugin_source_code_context = ctx;
result->free_plugin_source_code_context = random_user_context_free;
}
NAPI_MODULE(NODE_GYP_MODULE_NAME, Init)