Fix mimalloc init re-entrancy on Windows (TLS recursion + page-map)

[jinpzhanAMD]

PR Description:

Problem

On Windows, accessing __declspec(thread) storage can trigger on-demand TLS initialization (__dyn_tls_init) which may run user TLS constructors that allocate memory. During mimalloc process initialization, this can cause recursive allocations before global state (like the page map) is ready, leading to assertion failures.

Root Causes

1. TLS recursion: mi_prim_get_default_heap() accesses TLS, which can trigger __dyn_tls_init and user constructors
2. Page-map race: Recursive allocations during mi_process_init() can occur before _mi_page_map is initialized
3. Non-blocking initialization: mi_atomic_once is non-blocking, allowing other threads to observe incomplete initialization

Solution

This PR addresses the issues by:

1. TLS recursion guard (MI_TLS_RECURSE_GUARD): Prevents TLS access before mi_process_init completes on Windows
2. Proper initialization synchronization: Rewrote mi_process_init() with atomic state tracking (0=not started, 1=in progress, 2=done)
3. Re-entrancy support: Allows safe recursion on the initializing thread while blocking other threads until completion
4. Deferred page-map assertion: Moved assertion after initialization check to avoid false positives

Changes

• include/mimalloc/prim.h: Added MI_TLS_RECURSE_GUARD definition for Windows
• src/init.c: Rewrote mi_process_init() with proper synchronization (48 lines)
• src/page-map.c: Adjusted assertion order in _mi_page_map_register()

Testing

Tested on Windows with applications that trigger TLS initialization during mimalloc init. No more assertion failures or crashes.

Compatibility

• Windows-specific fix, no impact on other platforms
• Maintains existing API and behavior
• No breaking changes

[jinpzhanAMD]

@microsoft-github-policy-service agree [company="{AMD}"]

[jinpzhanAMD]

@microsoft-github-policy-service agree company="AMD"

[daanx]

Hi @jinpzhanAMD -- thanks for the PR; this is tricky stuff.

1. Normally on Windows we use MI_TLS_MODEL_DYNAMIC_WIN32 which works without issue -- is there any particular reason to use MI_TLS_MODEL_THREAD_LOCAL ?

2. For the other issue (the PR should perhaps be split?) to guard in mi_process_init against multiple threads trying to run the routine: we usually ensure that only one thread runs this as it is called early during the process start; and in particular when using dynamic override. Did you observe multiple threads executing this concurrently in practice?

[jinpzhanAMD]

Hi @daanx,

Thank you for the review. I need to rethink this PR as it was based on the code prior to the TLS refactor implemented in commit a7f84f3. The issue this PR addressed has been resolved in the latest release (v3.2.6), which includes that commit.

However, we've now encountered a different issue with v3.2.6. Could you please review this PR first: #1202?

Thank you!

[daanx]

Thank you!