`itertools.zip_longest` stores NULL in GC-tracked `ittuple` when iterators exhaust

[devdanzin]

Crash report

What happened?

itertools.zip_longest stores NULL in its internal ittuple (the tuple of iterators) when an iterator is exhausted, at Modules/itertoolsmodule.c:3847,3877:

PyTuple_SET_ITEM(lz->ittuple, i, NULL);
Py_DECREF(it);

The ittuple is GC-tracked and visited by zip_longest_traverse. When exposed via gc.get_referents(), the tuple contains NULL items. On builds without the PyUnicodeWriter_WriteRepr NULL fix (gh-146056), repr() of this tuple segfaults. On builds with the fix, it displays <NULL>.

This is the same class of bug as gh-146056 (NULL items in GC-tracked containers), but in itertools.zip_longest rather than xml.etree.ElementTree.

Reproducer

import itertools
import gc

zl = itertools.zip_longest(iter([1]), iter([1, 2, 3]))
next(zl)  # (1, 1) — both iterators alive
next(zl)  # (None, 2) — first iterator exhausted, ittuple[0] set to NULL

# Expose the internal ittuple via gc.get_referents
refs = gc.get_referents(zl)
for r in refs:
    if isinstance(r, tuple):
        print(repr(r))  # Segfault on builds without gh-146056 fix
                         # Shows (<NULL>, <list_iterator ...>) on builds with it

Suggested fix

Replace NULL with Py_None as the sentinel for exhausted iterators. Py_None can never be a valid iterator in the ittuple (passing None as an iterable to zip_longest fails at PyObject_GetIter in zip_longest_new).

--- a/Modules/itertoolsmodule.c
+++ b/Modules/itertoolsmodule.c
@@ -3832,7 +3832,7 @@
         for (i=0 ; i < tuplesize ; i++) {
             it = PyTuple_GET_ITEM(lz->ittuple, i);
-            if (it == NULL) {
+            if (it == Py_None) {
                 item = Py_NewRef(lz->fillvalue);
             } else {
@@ -3844,7 +3844,7 @@
                     } else {
                         item = Py_NewRef(lz->fillvalue);
-                        PyTuple_SET_ITEM(lz->ittuple, i, NULL);
+                        PyTuple_SET_ITEM(lz->ittuple, i, Py_NewRef(Py_None));
                         Py_DECREF(it);
                     }

The same change applies to both the result-reuse path (line ~3835/3847) and the new-result path (line ~3865/3877) — 4 lines total.

Found with assistance from Claude Code, while investigating a related crash reported by @zhuyifei1999.

CPython versions tested on:

CPython main branch, 3.15, 3.14

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.15.0a7+ free-threading build (heads/main:4d0e8ee649c, Mar 29 2026, 23:20:29) [Clang 21.1.2 (2ubuntu6)]

[serhiy-storchaka]

As you know, segfault was fixed in #146056. So what is the issue again?

[devdanzin]

I assumed having NULL accessible wouldn't be desirable. Seems to be the case:

import itertools
import gc

zl = itertools.zip_longest(iter([1]), iter([1, 2, 3]))
next(zl)  # (1, 1) — both iterators alive
next(zl)  # (None, 2) — first iterator exhausted, ittuple[0] set to NULL

# Expose the internal ittuple via gc.get_referents
refs = gc.get_referents(zl)
for r in refs:
    if isinstance(r, tuple):
        print(repr(r))
        break  
print(r[0])

Backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x0000555555caa089 in _Py_atomic_load_uint32_relaxed (obj=0xc) at ./Include/cpython/pyatomic_gcc.h:367
367     { return __atomic_load_n(obj, __ATOMIC_RELAXED); }

#0  0x0000555555caa089 in _Py_atomic_load_uint32_relaxed (obj=0xc) at ./Include/cpython/pyatomic_gcc.h:367
#1  Py_INCREF (op=0x0) at ./Include/refcount.h:267
#2  _Py_NewRef (obj=0x0) at ./Include/refcount.h:529
#3  tuple_item (op=op@entry=0x7bffb69bc790, i=i@entry=0) at Objects/tupleobject.c:429
#4  0x0000555555caa5d2 in tuple_subscript (op=0x7bffb69bc790, item=0x55555693c378 <_PyRuntime+30392>) at Objects/tupleobject.c:886
#5  0x0000555555a5c6fc in PyObject_GetItem (o=0x7bffb69bc790, key=0x55555693c378 <_PyRuntime+30392>) at Objects/abstract.c:163
#6  0x0000555555ee14e2 in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at Python/generated_cases.c.h:65
#7  0x0000555555ea8d04 in _PyEval_EvalFrame (tstate=0x555556997d60 <_PyRuntime+405664>, frame=0x7e8ff6fe0220, throwflag=0) at ./Include/internal/pycore_ceval.h:118
#8  _PyEval_Vector (tstate=tstate@entry=0x555556997d60 <_PyRuntime+405664>, func=func@entry=0x7bffb64c3e30, locals=locals@entry=0x7bffb670ca90, args=args@entry=0x0,
    argcount=argcount@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:2179
#9  0x0000555555ea80db in PyEval_EvalCode (co=co@entry=0x7bffb63e7a90, globals=globals@entry=0x7bffb670ca90, locals=locals@entry=0x7bffb670ca90) at Python/ceval.c:686
#10 0x00005555560c6ca0 in run_eval_code_obj (tstate=tstate@entry=0x555556997d60 <_PyRuntime+405664>, co=co@entry=0x7bffb63e7a90, globals=0x7bffb670ca90, locals=locals@entry=0x7bffb670ca90)
    at Python/pythonrun.c:1369
#11 0x00005555560c6322 in run_mod (mod=mod@entry=0x7bffb67dcd58, filename=filename@entry=0x7bffb64d0590, globals=globals@entry=0x7bffb670ca90, locals=locals@entry=0x7bffb670ca90,
    flags=0x7fffffffba50, arena=arena@entry=0x7bffb60508b0, interactive_src=0x0, generate_new_source=0) at Python/pythonrun.c:1472
#12 0x00005555560c002d in pyrun_file (fp=fp@entry=0x7d4ff6fe3000, filename=filename@entry=0x7bffb64d0590, start=start@entry=257, globals=globals@entry=0x7bffb670ca90,
    locals=locals@entry=0x7bffb670ca90, closeit=closeit@entry=1, flags=0x7fffffffba50) at Python/pythonrun.c:1296
#13 0x00005555560bda8f in _PyRun_SimpleFileObject (fp=0x7d4ff6fe3000, filename=0x7bffb64d0590, closeit=closeit@entry=1, flags=flags@entry=0x7fffffffba50) at Python/pythonrun.c:518
#14 0x00005555560bce01 in _PyRun_AnyFileObject (fp=fp@entry=0x7d4ff6fe3000, filename=filename@entry=0x7bffb64d0590, closeit=closeit@entry=1, flags=flags@entry=0x7fffffffba50)
    at Python/pythonrun.c:81
#15 0x000055555613f6ab in pymain_run_file_obj (program_name=0x7bffb64d0610, filename=0x7bffb64d0590, skip_source_first_line=0) at Modules/main.c:410
#16 pymain_run_file (config=config@entry=0x555556961590 <_PyRuntime+182480>) at Modules/main.c:429
#17 0x000055555613d601 in pymain_run_python (exitcode=<optimized out>) at Modules/main.c:714
#18 Py_RunMain () at Modules/main.c:795
#19 0x000055555613e6b0 in pymain_main (args=<optimized out>) at Modules/main.c:825
#20 0x000055555613e828 in Py_BytesMain (argc=<optimized out>, argv=0x7fffffffbee8) at Modules/main.c:849
#21 0x00007ffff7c2a575 in __libc_start_call_main (main=main@entry=0x5555558e6c70 <main>, argc=argc@entry=2, argv=argv@entry=0x7fffffffbee8) at ../sysdeps/nptl/libc_start_call_main.h:58
#22 0x00007ffff7c2a628 in __libc_start_main_impl (main=0x5555558e6c70 <main>, argc=2, argv=0x7fffffffbee8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
    stack_end=0x7fffffffbed8) at ../csu/libc-start.c:360
#23 0x00005555557fd525 in _start ()

Edit: accessing NULL in the list that contains it also crashes using the reproducer in #146056. So it seems any container displaying a NULL could be susceptible to this, right?

[manueljacob]

The segfault for the code in #148181 (comment) was fixed by commit 0f2246b, but the code in #148181 (comment) still causes a segfault on current main (commit a0c57a8).