Skip to content

docs: add SAFETY comments to all unsafe blocks in shuffle spark_unsafe module#3603

Merged
andygrove merged 1 commit intoapache:mainfrom
andygrove:add-safety-comments-shuffle
Feb 26, 2026
Merged

docs: add SAFETY comments to all unsafe blocks in shuffle spark_unsafe module#3603
andygrove merged 1 commit intoapache:mainfrom
andygrove:add-safety-comments-shuffle

Conversation

@andygrove
Copy link
Member

Summary

  • Add // SAFETY: comments to all 25 existing unsafe blocks in the spark_unsafe shuffle module (row.rs, list.rs, map.rs)
  • Add a # Safety documentation section to the SparkUnsafeObject trait documenting invariants for implementors
  • No functional changes

All unsafe in this module falls into one category: reading/writing JVM-allocated Spark UnsafeRow/UnsafeArray/UnsafeMap memory via raw pointers received through JNI. The safety invariants are:

  • Pointers come from the JVM and remain valid for the batch lifetime (guaranteed by JNI pinning)
  • Memory layout follows Spark's documented UnsafeRow/UnsafeArray format
  • Index bounds are enforced by the caller (schema-driven iteration)

Test plan

  • Documentation-only change, no functional impact
  • Verified cargo check passes

🤖 Generated with Claude Code

…e module

Document the safety invariants for all 25 existing unsafe blocks across
row.rs, list.rs, and map.rs. Also add a Safety section to the
SparkUnsafeObject trait documenting the requirements for implementors.
Copy link
Member

@wForget wForget left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @andygrove

@andygrove andygrove merged commit c916849 into apache:main Feb 26, 2026
132 of 133 checks passed
@andygrove andygrove deleted the add-safety-comments-shuffle branch February 26, 2026 02:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants