Add fallback implementation of `PyCriticalSection_BeginMutex` for Python 3.13t by XuehaiPan · Pull Request #5981 · pybind/pybind11

XuehaiPan · 2026-02-02T17:16:45Z

Description

Suggested changelog entry:

Placeholder.

…hon 3.13t

colesbury · 2026-02-02T18:00:44Z

include/pybind11/detail/internals.h

+#    if defined(PY_VERSION_HEX) && PY_VERSION_HEX >= 0x030E00C1 // 3.14.0rc1
        PyCriticalSection_BeginMutex(&cs, &mutex.mutex);
+#    else
+        _PyCriticalSection_BeginMutex(_PyThreadState_GET(), &cs, &mutex.mutex);


I don't think this will compile on 3.13 without additional changes. _PyCriticalSection_BeginMutex is in an inline function in an internal-only header (pycore_critical_section.h).

I think we should use PyAPI_FUNC(void) _PyCriticalSection_BeginSlow(PyCriticalSection *c, PyMutex *m); instead. It's not inline, but you'll need to declare it above.

Also, we'll still have the race condition with py::make_key_iterator in 3.13 due to the critical section implementation behaving differently in 3.13, but otherwise it should preserve compatibility.

Also, we'll still have the race condition with py::make_key_iterator in 3.13 due to the critical section implementation behaving differently in 3.13, but otherwise it should preserve compatibility.

I think that's OK. We just want to limp through enough to make the pybind11 v3.0.2 release without breaking what worked with Python 3.13t before.

include/pybind11/detail/internals.h

`_PyCriticalSection_BeginSlow` is a private CPython function not exported on Linux. For Python < 3.14.0rc1, use direct `mutex.lock()`/`mutex.unlock()` instead of critical section APIs.

henryiii · 2026-02-05T15:53:33Z

I think this hangs. I think we probably should do nothing on 3.13t, rather than trying to also have a lock?

… < 3.14

colesbury · 2026-02-05T16:28:21Z

What was the issue with _PyCriticalSection_BeginSlow?

include/pybind11/detail/internals.h

Refactor pycritical_section into a unified class with internal version checks instead of using a type alias fallback. Skip locking in make_iterator_impl for Python < 3.14.0rc1 to avoid deadlock during type registration, as pycritical_section cannot release the mutex during Python callbacks without PyCriticalSection_BeginMutex.

XuehaiPan · 2026-02-05T18:17:55Z

What was the issue with _PyCriticalSection_BeginSlow?

There is a linkage issue on Linux.

Skylion007 · 2026-02-05T19:10:35Z

tests/test_multiple_interpreters.py

 @pytest.mark.skipif(
    sys.platform.startswith("emscripten"), reason="Requires loadable modules"
 )
+@pytest.mark.xfail(env.MUSLLINUX, reason="Flaky on musllinux", strict=False)


Is this a known errata with the current fix?

Xref: #5972 (comment)

@XuehaiPan could you please add the URL to the reason?

colesbury · 2026-02-05T19:11:28Z

You would need to copy the forward declaration from Python (PyAPI_FUNC(void)). I don't think extern "C" is always sufficient because of symbol visibility:

https://github.com/python/cpython/blob/60403a5409ff2c3f3b07dd2ca91a7a3e096839c7/Include/internal/pycore_critical_section.h#L91-L92

But if you get the mutex working, that seems fine too

rwgk

@colesbury @XuehaiPan question about this comment:

#5981 (comment)

I'm not sure if that means we should still do something, or if we're fine with this PR as-is? (it looks good to me, with my rather superficial specific background)

rwgk · 2026-02-08T22:46:56Z

tests/test_multiple_interpreters.py

 @pytest.mark.skipif(
    sys.platform.startswith("emscripten"), reason="Requires loadable modules"
 )
+@pytest.mark.xfail(env.MUSLLINUX, reason="Flaky on musllinux", strict=False)


@XuehaiPan could you please add the URL to the reason?

XuehaiPan · 2026-02-09T10:35:48Z

I'm not sure if that means we should still do something, or if we're fine with this PR as-is? (it looks good to me, with my rather superficial specific background)

Neither PyAPI_FUNC(void) nor extern "C" is working (#5981 (comment)).

$ cmake --build build --parallel --target pytest
...
[100%] Built target pybind11_tests
Traceback (most recent call last):
  File "/home/PanXuehai/Projects/pybind11/tests/conftest.py", line 26, in <module>
    import pybind11_tests
ImportError: /home/PanXuehai/Projects/pybind11/build/tests/pybind11_tests.cpython-313t-x86_64-linux-gnu.so: undefined symbol: _ZN8pybind116detail28_PyCriticalSection_BeginSlowEP17PyCriticalSectionP7PyMutex
ImportError while loading conftest '/home/PanXuehai/Projects/pybind11/tests/conftest.py'.
../../tests/conftest.py:26: in <module>
    import pybind11_tests
E   ImportError: /home/PanXuehai/Projects/pybind11/build/tests/pybind11_tests.cpython-313t-x86_64-linux-gnu.so: undefined symbol: _ZN8pybind116detail28_PyCriticalSection_BeginSlowEP17PyCriticalSectionP7PyMutex
gmake[3]: *** [tests/CMakeFiles/pytest.dir/build.make:76: tests/CMakeFiles/pytest] Error 4
gmake[2]: *** [CMakeFiles/Makefile2:601: tests/CMakeFiles/pytest.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:608: tests/CMakeFiles/pytest.dir/rule] Error 2
gmake: *** [Makefile:281: pytest] Error 2

Add failback implementation of PyCriticalSection_BeginMutex for Pyt…

d569c4c

…hon 3.13t

XuehaiPan mentioned this pull request Feb 2, 2026

Fix race condition with py::make_key_iterator in free threading #5971

Merged

Add comment for Python version

5e7d423

colesbury reviewed Feb 2, 2026

View reviewed changes

Use _PyCriticalSection_BeginSlow

9de6d1a

XuehaiPan changed the title ~~Add failback implementation of PyCriticalSection_BeginMutex for Python 3.13t~~ Add fallback implementation of PyCriticalSection_BeginMutex for Python 3.13t Feb 2, 2026

Add forward declaration

00bcf66

colesbury reviewed Feb 2, 2026

View reviewed changes

include/pybind11/detail/internals.h Outdated Show resolved Hide resolved

XuehaiPan added 5 commits February 3, 2026 04:13

Fix forward declaration

225cb46

Remove always true condition defined(PY_VERSION_HEX)

0e6ffd3

Merge remote-tracking branch 'upstream/master' into fix-py313t

862aa00

Detect musllinux

24db64d

Add manylinux test

c17212a

XuehaiPan requested a review from henryiii as a code owner February 3, 2026 11:46

Use direct mutex locking for Python 3.13t

3d4af60

`_PyCriticalSection_BeginSlow` is a private CPython function not exported on Linux. For Python < 3.14.0rc1, use direct `mutex.lock()`/`mutex.unlock()` instead of critical section APIs.

XuehaiPan requested review from colesbury and rwgk February 3, 2026 13:10

XuehaiPan added 4 commits February 4, 2026 19:50

Empty commit to trigger CI

358d3c7

Empty commit to trigger CI

fea12dc

Empty commit to trigger CI

1ef8cae

Run apt update before apt install

f7a002c

XuehaiPan mentioned this pull request Feb 5, 2026

Revert internals destruction and add test for internals recreation #5972

Merged

XuehaiPan added 2 commits February 5, 2026 17:33

Remove unnecessary prefix

3f82060

Add manylinux test with Python 3.13t

f20f126

Simplify pycritical_section with std::unique_lock fallback for Python…

89b2879

… < 3.14

colesbury reviewed Feb 5, 2026

View reviewed changes

include/pybind11/detail/internals.h Outdated Show resolved Hide resolved

Skylion007 reviewed Feb 5, 2026

View reviewed changes

henryiii approved these changes Feb 5, 2026

View reviewed changes

rwgk reviewed Feb 8, 2026

View reviewed changes

XuehaiPan added 2 commits February 9, 2026 18:31

Merge remote-tracking branch 'upstream/master' into fix-py313t

afb4a2a

Add reference for xfail message

3095f2f

XuehaiPan requested review from colesbury and rwgk February 9, 2026 10:37

Conversation

XuehaiPan commented Feb 2, 2026

Description

Suggested changelog entry:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

henryiii commented Feb 5, 2026

Uh oh!

colesbury commented Feb 5, 2026

Uh oh!

Uh oh!

XuehaiPan commented Feb 5, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

colesbury commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rwgk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

XuehaiPan commented Feb 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

colesbury commented Feb 5, 2026 •

edited

Loading