-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
Is there an existing issue for this?
- I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
- I have reviewed the documentation https://docs.sentry.io/
- I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases
How do you use Sentry?
Sentry Saas (sentry.io)
Which SDK are you using?
@sentry/node
SDK Version
10.36.0
Framework Version
No response
Link to Sentry event
No response
Reproduction Example/SDK Setup
/**
- Minimal reproduction: V8 heap corruption with @sentry/profiling-node + worker_threads
- This script demonstrates that nodeProfilingIntegration() causes V8 heap corruption
- when used in worker threads under OTel span allocation pressure.
- The native CPU profiler addon samples the V8 heap from a separate OS thread.
- When multiple worker threads rapidly allocate short-lived objects (Map copies in
- SentryContextManager.with(), Set allocations in addChildSpanToSpan()), the
- profiler's concurrent heap access corrupts V8's internal structures.
- Expected: crashes within a few runs (~40% crash rate per run)
- Control: set DISABLE_PROFILING=1 to verify 0% crash rate without profiling
- Dependencies:
- npm install @sentry/node @sentry/profiling-node @sentry/opentelemetry \
-
@opentelemetry/api @opentelemetry/sdk-trace-node - Usage:
- node sentry_profiling_crash_repro.js
- Environment:
- Node v22+ (tested on v24.11.1, macOS arm64)
- @sentry/profiling-node 10.36.0
*/
const { Worker, isMainThread, workerData, parentPort } = require('worker_threads')
// ────────────────────────────────────────────────────────────────────────────
// MAIN THREAD — spawns 3 worker threads and monitors for crashes
// ────────────────────────────────────────────────────────────────────────────
if (isMainThread) {
const NUM_WORKERS = 3
console.log('=== Sentry Profiling + Worker Threads Crash Repro ===')
console.log(Node ${process.version} (${process.arch}, ${process.platform}))
console.log(PID: ${process.pid})
console.log(Profiling: ${process.env.DISABLE_PROFILING ? 'DISABLED' : 'ENABLED'})
console.log(Spawning ${NUM_WORKERS} worker threads...\n)
let crashCount = 0
const workers = []
for (let i = 0; i < NUM_WORKERS; i++) {
const worker = new Worker(__filename, {
workerData: { workerId: i },
})
worker.on('message', (msg) => {
console.log(` [worker-${i}] ${msg}`)
})
worker.on('error', (err) => {
console.error(` [worker-${i}] ERROR: ${err.message}`)
})
worker.on('exit', (code) => {
if (code !== 0) {
crashCount++
console.error(` [worker-${i}] CRASHED (exit code ${code})`)
} else {
console.log(` [worker-${i}] exited cleanly`)
}
})
workers.push(worker)
}
Promise.all(workers.map((w) => new Promise((resolve) => w.on('exit', resolve)))).then(() => {
console.log(\nDone. ${crashCount}/${NUM_WORKERS} workers crashed.)
if (crashCount > 0) {
console.log('V8 heap corruption reproduced!')
} else {
console.log('No crashes this run. Try running again (crash rate ~40%).')
}
process.exit(crashCount > 0 ? 1 : 0)
})
} else {
// ──────────────────────────────────────────────────────────────────────────
// WORKER THREAD — initializes Sentry + profiling and creates rapid OTel spans
// ──────────────────────────────────────────────────────────────────────────
const { workerId } = workerData
const Sentry = require('@sentry/node')
const { nodeProfilingIntegration } = require('@sentry/profiling-node')
const { SentryPropagator, SentrySampler, SentrySpanProcessor } = require('@sentry/opentelemetry')
const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node')
const { context, trace } = require('@opentelemetry/api')
// Initialize Sentry — profiling is the key ingredient that triggers the crash.
// The native CPU profiler addon runs a sampling thread that concurrently reads
// V8 heap structures while the main thread is allocating/GCing objects.
const integrations = []
if (!process.env.DISABLE_PROFILING) {
integrations.push(nodeProfilingIntegration())
}
const sentryClient = Sentry.init({
// Use a dummy DSN — we don't need to actually send events
dsn: 'https://examplePublicKey@o0.ingest.sentry.io/0',
environment: 'crash-repro',
integrations,
tracesSampleRate: 1.0,
profileLifecycle: 'trace',
profileSessionSampleRate: 1.0,
skipOpenTelemetrySetup: true,
})
// Register OTel provider with Sentry's context manager and span processor.
// SentryContextManager.with() copies the context Map 4 times per call.
// SentrySpanProcessor.onSpanStart() calls addChildSpanToSpan() which creates new Set().
// These short-lived allocations are what the profiler's sampling thread races against.
const provider = new NodeTracerProvider({
sampler: new SentrySampler(sentryClient),
spanProcessors: [new SentrySpanProcessor()],
})
provider.register({
propagator: new SentryPropagator(),
contextManager: new Sentry.SentryContextManager(),
})
const tracer = trace.getTracerProvider().getTracer('repro')
const ITERATIONS = 2000
const CONCURRENT_SPANS = 5
async function createSpanBurst(id) {
// Each burst creates nested spans, triggering:
// - SentryContextManager.with() → Map copies (4× per startActiveSpan)
// - addChildSpanToSpan() → new Set() per child span
return tracer.startActiveSpan(parent-${id}, async (parentSpan) => {
// Yield to event loop so setImmediate callbacks can interleave
await new Promise((r) => setImmediate(r))
const children = Array.from({ length: 3 }, (_, i) =>
tracer.startActiveSpan(`child-${id}-${i}`, async (childSpan) => {
await new Promise((r) => setImmediate(r))
childSpan.end()
})
)
await Promise.all(children)
parentSpan.end()
})
}
async function run() {
parentPort.postMessage(Starting ${ITERATIONS} iterations x ${CONCURRENT_SPANS} concurrent spans)
const start = performance.now()
for (let i = 0; i < ITERATIONS; i++) {
// Launch multiple concurrent span bursts to maximize allocation pressure
const bursts = Array.from({ length: CONCURRENT_SPANS }, (_, j) =>
createSpanBurst(workerId * ITERATIONS + i * CONCURRENT_SPANS + j)
)
await Promise.all(bursts)
if ((i + 1) % 500 === 0) {
const elapsed = Math.round(performance.now() - start)
const rss = Math.round(process.memoryUsage().rss / 1024 / 1024)
parentPort.postMessage(`${i + 1}/${ITERATIONS} | ${elapsed}ms | RSS: ${rss}MB`)
}
}
const elapsed = Math.round(performance.now() - start)
parentPort.postMessage(`Completed in ${elapsed}ms`)
await provider.shutdown()
await Sentry.close()
}
run().catch((err) => {
console.error(Worker ${workerId} error:, err)
process.exit(1)
})
}
Steps to Reproduce
Run the repro script as described above.
Expected Result
No segfault.
Actual Result
V8 segfault.
Additional Context
When using Sentry with nodeProfilingIntegration() and BullMQ with worker threads (docs) we experience frequent V8 segfaults. I asked Claude to debug this issue, and it came up with the analysis below, as well as the repro script. I can confirm that removing nodeProfilingIntegration() fixes the issue.
V8 heap corruption when using nodeProfilingIntegration() with worker threads and OTel spans
Description
Using nodeProfilingIntegration() inside worker_threads causes V8 heap corruption under moderate OpenTelemetry span allocation pressure. The native CPU profiler addon's sampling thread appears to race with V8's GC, corrupting internal heap structures (hidden class descriptors, Map/Set internals).
Disabling nodeProfilingIntegration() completely eliminates the crashes with zero code changes otherwise.
Reproduction
Prerequisites
npm install @sentry/node @sentry/profiling-node @sentry/opentelemetry \
@opentelemetry/api @opentelemetry/sdk-trace-nodeRepro script
Save the attached sentry_profiling_crash_repro.js and run:
node sentry_profiling_crash_repro.jsThe script spawns 3 worker threads. Each thread:
- Initializes Sentry with
nodeProfilingIntegration() - Registers an OTel
NodeTracerProviderwithSentrySpanProcessorandSentryContextManager - Rapidly creates nested spans via
startActiveSpan(2000 iterations x 5 concurrent bursts)
Expected result: all workers complete without errors.
Actual result: one or more workers crash (~40% crash rate per run). Running the script 3-5 times will almost certainly trigger at least one crash.
Verification that profiling is the cause
DISABLE_PROFILING=1 node sentry_profiling_crash_repro.jsWith profiling disabled: 0% crash rate across dozens of runs.
Crash signatures observed
Multiple distinct crash types, all pointing to heap corruption:
1. V8 descriptor check failure (most common)
# Fatal error in , line 0
# Check failed: descriptor_number.as_int() < number_of_descriptors().
Exit code 134 (SIGABRT). The JS stack shows new Set() in addChildSpanToSpan receiving corrupted data.
2. Map/Set internal confusion
TypeError: Method Map.prototype.set called on incompatible receiver [object Map Iterator]
at MapIterator.set (<anonymous>)
at new Map (<anonymous>)
at new BaseContext (@opentelemetry/api/context.ts:42)
at SentryContextManager.with (contextManager.ts:78)
V8 confuses a Map with a Map Iterator — a clear sign of corrupted heap object metadata.
3. GC filler map check
# Check failed: !IsFreeSpaceOrFillerMap(map).
V8 encounters a freed/filler object where a live object should be.
4. Corrupted function pointers
TypeError: immediate._onImmediate is not a function
A setImmediate callback's function reference has been corrupted.
Analysis
The root cause appears to be:
- The native CPU profiler addon (
@sentry-internal/node-cpu-profiler) runs V8'sCpuProfilerwhich samples heap data from a separate OS thread. SentryContextManager.with()performs 4new Map(parentMap)copies perstartActiveSpancall, andaddChildSpanToSpan()allocatesnew Set([childSpan])per span — creating significant short-lived allocation pressure.- With 3+ concurrent worker threads each running their own V8 isolate with profiling, the profiler's sampling thread accesses heap structures concurrently with the main thread's GC, corrupting V8's internal object layout.
This is consistent with known classes of bugs where native addons perform concurrent heap access without proper synchronization with V8's GC.
Environment
- Node.js: v22.14.0, v24.11.1 (reproduced on both)
- OS: macOS 15 (arm64), also observed on Linux x86_64 (production)
- @sentry/node: 10.36.0
- @sentry/profiling-node: 10.36.0
- @sentry/opentelemetry: 10.36.0
- @opentelemetry/sdk-trace-node: 2.6.0
Workaround
Remove nodeProfilingIntegration() from the Sentry initialization in any code that runs inside worker threads. This eliminates the native addon's sampling thread and the race condition entirely.
Priority
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it.
Metadata
Metadata
Assignees
Fields
Give feedbackProjects
Status