feat: support the Python stable ABI (abi3 and abi3t) - #6198

If this is better in smaller parts, there are some good candidates for that, including two bug fixes and a nice performance speedup:

🤖 AI text below 🤖

Yes. Beyond the method-descriptor patch, there are two real bug fixes and one large feature that do not depend on Py_LIMITED_API, plus a pile of prep refactors.

Bug fixes (strongest candidates)

• type_is_managed_by_our_internals() cross-module fix (ad10b98, include/pybind11/detail/cpp_conduit.h). Master compares tp_new against this module's pybind11_object_new, which fails for types whose base came from another module. The branch switches to the registry lookup PyPy already used. No test was added on the branch, so it needs one before it ships alone.
• pybind11_traverse/pybind11_clear under PYBIND11_BACKWARD_COMPATIBILITY_TP_DICTOFFSET on 3.13+ (part of 4d65cb2). Master unconditionally calls PyObject_VisitManagedDict/PyObject_ClearManagedDict on 3.13+, even when that macro puts the dict at tp_dictoffset and the type has no Py_TPFLAGS_MANAGED_DICT. The dict is then never visited or cleared by the GC, at best. I did not run this on master, so verify with a 3.13 build and that macro before filing.

Feature, independent of the stable ABI

• Spec-based type creation / PyType_FromMetaclass default on 3.12+ (fbc9d96, 8f47ff2, c05d08d, a5f341a, 04459a5, e669ac1, 4d65cb2). PYBIND11_TYPE_CREATION_VIA_SPEC is a separate switch from Py_LIMITED_API and already has its own CI job. It is big, but it is the natural first PR: the stable ABI PR then shrinks to "the spec path, plus limited-API gating".

Method descriptor / instancemethod (the one you named)

• 73cd296 and 721fabc give pybind11's own instancemethod type vectorcall and Py_TPFLAGS_METHOD_DESCRIPTOR. On the branch that type exists only under Py_LIMITED_API; master uses PyInstanceMethod_Type, which lacks the flag. A standalone PR has to make the own type unconditional. If you do that, 973e9ef comes along for free: the metaclass tp_getattro can be dropped everywhere, which removes the slow MRO walk on class attribute reads, and the interned-name type_lookup() speedup applies to regular builds too.

Prep refactors, low value alone but they shrink the main diff

• Traceback walk by attribute in error_already_set::what() (683f376).
• PyThreadState_GetInterpreter() instead of tstate->interp, with the PyPy fallback (5c50c3d, part of e0bc493).
• get_bases()/get_mro()/get_tp_name()/tp_name_is_one_of() helpers (d1af605, 1865f0b) and the PyTuple_GET_ITEM-style macro replacements (cdbc9b4).
• value_and_holder::type_is_exact() replacing the repeated Py_TYPE(v_h.inst) == v_h.type->type in init.h and pybind11.h.
• The cross-module test modules factored into an exec function (test-only).

Not separable

• The chrono per-TU fix (88fb22a) fixes helpers the branch itself introduced; master inlines the macros, so it does not apply.
• SABIModule export (573d161), eval.h, numpy.h, complex.h, the TLS/critical-section changes, and the conftest compile-time version logic are stable-ABI-only.

@henryiii this is the first pass reviewing with

codex GPT-6.1-Sol ultra:

Rechecked against 75be228. All six findings remain after the rebase:

1. [P1] Pin registry candidates before acquiring the internals lock.
   get_object_handle() retains registry iterators and raw instances across values_and_holders(), which acquires the internals lock through all_type_info(). Under abi3t, contention suspends the outer registry critical section, allowing another thread to erase entries or rehash the shard. py::get_override() can then resume with invalid iterators or freed wrappers. Please collect strong candidates under the registry lock and inspect them afterward, as find_registered_python_instance() already does, retaining the selected wrapper through override lookup.

2. [P2] Keep dictionary-offset helpers available for non-limited builds.
   Defining PYBIND11_BACKWARD_COMPATIBILITY_TP_DICTOFFSET in a normal CPython 3.12+ build now makes including pybind11.h fail to compile. The new spec path calls type_dictoffset and type_basicsize, but both helpers exist only under Py_LIMITED_API. I reproduced this with CPython 3.14.4 headers; the same include compiles without that macro. Please provide non-limited equivalents or retain direct field access for this configuration.

3. [P2] Check base metaclasses before selecting the spec path.
   The fallback condition checks only the requested/default metaclass. Binding Base with py::metaclass(Meta), where Python defines class Meta(type): pass, selects the legacy path. Subsequently binding py::class_<Derived, Base>(m, "Derived") selects the spec path with the default pybind11_type. PyType_FromMetaclass() then raises a metaclass conflict; the previous PyType_Ready path accepts this binding. Please consider the base metaclasses when deciding whether to fall back.

4. [P2] Recognize Windows free-threaded Python suffixes.
   The fallback detection matches cpython-…t, whereas Windows Python 3.15t uses cp315t-win_amd64 / .cp315t-win_amd64.pyd. Since Python_FREE_THREADED was added in CMake 4.4, CMake 3.30–4.3 misdetects this interpreter as GIL-enabled. Default STABLE_ABI consequently targets 3.12 and fails compilation because free-threaded stable ABI requires 3.15.

5. [P2] Preserve FindPython paths for Windows abi3t consumers.
   With add_subdirectory(pybind11) and Python discovery inside that subdirectory, Python_INCLUDE_DIRS and Python_SABI_LIBRARY_DIRS remain scoped there. A caller using GIL-enabled Windows Python 3.15+, PYBIND11_ABI3T=ON, and pybind11_add_module(... STABLE_ABI ...) reaches the manual setup with empty paths. find_library(... NO_DEFAULT_PATH REQUIRED) fails even when python3t.lib is installed. Please obtain these paths from the global imported target or preserve them for callers.

6. [P2] Include the second chrono translation unit when selecting chrono tests.
   -DPYBIND11_TEST_OVERRIDE=test_chrono excludes the independently listed test_chrono_second_tu.cpp, while test_chrono.py unconditionally imports its submodule. The extension builds, but pytest fails collection with ImportError. Please include the helper translation unit whenever chrono is selected.

Finding 2 was reproduced through compilation at this revision. The others are based on code inspection; no test suites were run.

I'm going to revisit this soon, I don't agree with all the review fixes I applied. "Pin registry candidates before acquiring the internals lock" specifically is wrong, we should not be returning an object instead of a handle from get_object_handle.

codex GPT-6 on behalf of Ralf:

I prepared ten suggested-fix commits atop 1ea0ea0, one per issue, each with regression coverage. The clean series is in #6225, an open DO NOT MERGE PR used for CI validation.

1. 80ba2963fdd6 — Keep the selected wrapper owned throughout override lookup; cover classic and smart-holder lifetimes.
2. 585af4233d22 — Bypass metaclass overrides when reading structural type metadata.
3. 9621441ff9f7 — Preserve inherited dictionaries and Python-slot GC traversal, including cycles.
4. cd1fee7d2bc4 — Keep bound and fallback class layouts consistent in dictionary-offset compatibility mode.
5. 5791328f55a2 — Initialize datetime and MethodType caches with GIL-safe once handling.
6. e0cc3e72515e — Bind __complex__ descriptors with Python special-method semantics.
7. def221569a2f — Expose Python headers through the public CMake target, including Windows abi3t consumers.
8. 1f513f5e0016 — Keep WITHOUT_SOABI modules importable by applying Python prefix/suffix rules.
9. 3416a32beaa8 — Derive filenames and Windows export symbols from the effective explicit ABI macros.
10. 61f504dc713e — Encode the numeric limited-API version in the link guard and reject incompatible precompiled CMake target reuse.

Full CI is green on both initial validation #6224 and clean series #6225: each has 97 passing checks, two expected skips, and all three external status contexts passing.

The clean series is tree-identical to the initial validation PR #6224, with CI follow-ups folded into the corresponding commits. Local full default/free-threaded and abi3 suites passed, as did C++11 precompiled checks and the same abi3t binary on GIL-enabled and free-threaded Python 3.15. Independent review found no remaining significant findings.

These commits are offered for adoption or adaptation in the listed order. Neither validation PR is intended for merging.

1. Converts get_object_handle from retuning a handle to returning an object! It's literally in the name. I think this is just the AI being pedantic; it's a really slim edge case with multiple threads and one of them dropping a vector while the other reads it. If we really want to plug the hole, we need to introduce a new function. The test here is circular. I already reverted Claude's attempt to add this earlier.
2. Another technically correct but pedantic change, with a costly solution; this is a huge cost on every lookup for no real gain. CPython reads tp_bases and non-limited pybind11 does too.
3. This is a bug fix for master, not related to stable ABI. Also a bit costly, adds a registry lookup per base.
4. Looks good, though it can be made a little simpler.
5. Good.
6. Correct though minimal. No cost other than more lines, so let's take it.
7. Yep.
8. Good, and 6 lines.
9. Overbuilt, but core fix is good. Tests is more expensive than needed.
10. We don't need an 80+ line CMake test for this, but core is good.

So I'll adopt 4-10, with some modifications.

🤖 AI text below 🤖

Cherry-picked unchanged (5 to 8) with Ralf's authorship. I left the <pybind11/gil_safe_call_once.h> include as is since class.h already uses angle brackets.

Rewritten (4, 9, 10), each noting the source commit in the message:

• 4: common.h now defaults PYBIND11_TYPE_CREATION_VIA_SPEC to 0 when PYBIND11_BACKWARD_COMPATIBILITY_TP_DICTOFFSET is user-defined, and errors if both are set explicitly. The class-inl.h condition is untouched. Kept the docs sentence and the test. Note the test passes on every path, so it only guards the layout mix, not the macro case. No CI job defines that macro.
• 9: the explicit macros are a list, validated in a loop, with abi3t set if any is Py_TARGET_ABI3T. Kept the unit test; dropped the extra real-build parametrization.
• 10: header, docs, and CMake same-tree check adopted. The 83-line precompile_guard.cmake test and its subproject were dropped. I verified by hand that the mangled guard symbol differs between 0x030C0000 and 0x030D0000 and is identical for 0x030C0000 and 0x030c0000.

🤖 AI text below 🤖

Downstream testing

I built 12 downstream projects against this branch (macOS arm64) in three ways:

1. A regular build. If tests failed, I compared with the project's pinned or released pybind11.
2. An abi3 module, built on CPython 3.12. I installed the same wheel or module on 3.12, 3.13, 3.14 and 3.15 and ran the project's tests.
3. An abi3t module, built on free-threaded 3.15.0rc3t. I ran it on 3.15t and on GIL-enabled 3.15.

I also ran abi3audit on each module and found no violations.

"No regressions" means some tests failed, but the same tests also failed with released pybind11. The causes were outside pybind11: Hypothesis, solver versions, and LaTeX or Tk in the environment.

Some test runs had one or two extra limits:

• hnswlib's abi3 wheel did not run on 3.13.
• For cvxpy, several solvers have no 3.15 or 3.15t wheels, so some tests could not run. The failures were the same as with a regular build in the same environment.

Other checks:

• pybind11's own Eigen tests passed under abi3, abi3t and PYBIND11_ABI3T=ON (159 tests each).
• Under abi3t, weakref.getweakrefcount() is one higher, as the docs say. No downstream test depended on it.
• Multithreaded stress tests passed on 3.15t with the GIL off for iminuit, matplotlib, OpenTimelineIO, cvxpy and dlib.

What I changed on this branch:

• PYBIND11_PLUGIN now fails under abi3t with a clear static_assert. Before, users got a confusing overload error.
• An empty PYBIND11_STABLE_ABI_VERSION now means the default. scikit-build-core before 0.11 does not set SKBUILD_SABI_VERSION, so the documented snippet gave an empty value.
• Doc notes:
  • Development.SABIModule when a project calls find_package(Python) itself;
  • the meson-python setup;
  • the bdist_wheel tag for setuptools;
  • the .abi3-<platform>.so / .abi3t-<platform>.so suffixes;
  • build abi3 wheels with the Python version you target, and abi3t wheels on free-threaded Python;
  • pass Python_EXECUTABLE when a setup.py runs CMake;
  • every module in a stable-ABI wheel must use the stable ABI;
  • numpy>=2.5 for abi3t;
  • export PyModExport_* when you restrict exported symbols;
  • Py_GIL_DISABLED is always defined under abi3t.

I also found a problem that this PR did not cause: Pybind11Extension changes the caller's argument lists in place. The fix is in #6226.

To close the loop here, while reading

An important baseline detail: the default PyType_Spec / PyType_FromMetaclass() path for ordinary CPython 3.12+ builds already landed through #6231.

above it crossed my mind to ask about the performance impact of #6231. Codex jumped straight to a basic benchmark experiment. The results are shown here:

• feat: create types with PyType_FromMetaclass on CPython 3.12+ #6231 (comment)

— codex GPT-6 on behalf of Ralf —

One scope observation, separate from the type-creation switch already landed in #6231: the remaining changes in this PR also affect some ordinary CPython builds, even with the Stable ABI disabled.

Many existing fast paths are retained, but there are shared implementation changes. For example:

• Bytearray-to-std::string conversion now uses PyByteArray_AsString() and PyByteArray_Size() instead of their direct-access macros, without a Limited API conditional.
• Base-type slot forwarding uses cached PyType_GetSlot() results on ordinary CPython 3.10+ too. The lookup happens once per helper; subsequent calls use the cached function pointer.

This is a source-level observation; I haven’t benchmarked these changes. If users report noticeable slowdowns, we can optimize the affected paths in follow-up PRs.

What is the "Stable ABI"? — I have ideas about it, but I never got to looking / understanding systematically.

It's a limited subset of CPython's API (Limited API). If you use that, you can build a ABI that CPython commits to keeping in future versions (Stable ABI). Practical (massive) benefit: No longer have to build wheels for every version of Python. Can reduce a massive build matrix into a small one. Less storage on PyPI (which projects often have to request size increases for!).

Why does using the Stable ABI (apparently) impact runtime performance?

It is more abstracted from the implementation so it can be stable across future implementations. Most applications don't spend a lot of time in the binding layer, so this is fine most of the time; I'd recommend projects measure. I didn't see any measurable regressions in the projects I tested (at least after a couple of rounds of optimizations on this PR). And it's even not always negative, as in this PR we take a different, not quite backward compatible path to get better performance in one case with stable ABI than without. :)

The other two comments were probably better answered by the AI.

Here's a Fable 5.2 review:

🤖 AI text below 🤖

The review found 10 issues in origin/master...HEAD (61 commits, 72 files). Nothing was built or run. I read the code for each finding: 9 match the code, and I did not confirm the CMake one (8).

Behavior changes and bugs

1. include/pybind11/pybind11.h:870: install_buffer_funcs now checks PyType_GetSlot(type, Py_bf_getbuffer) and not tp_as_buffer. On Python 3.9, PyType_GetSlot reads the type's own as_buffer, not the inherited pointer. A Derived class that calls def_buffer without py::buffer_protocol(), with a Base that has the annotation, now throws at import. It worked before, 3.10+ is not affected, and no test covers it.
2. include/pybind11/complex.h:88: the stable-ABI caster calls interned_name("__complex__") on each load and puts the new reference in a non-owning handle. Each conversion of a non-complex object leaks one reference. All other callers keep the result in a function-local static.
3. include/pybind11/detail/internals.h:243: under Py_LIMITED_API, get_thread_state_unchecked() returns PyGILState_GetThisThreadState(). That value is non-null when the GIL is not held, and it can be from a different interpreter. The callers are gil-inl.h:48,61, subinterpreter.h:402 and internals.h:261. PyInterpreterState_Get() is in the stable ABI and gives the current interpreter.
4. include/pybind11/pytypes.h:548: stable-ABI get_tp_name() adds the module if type_is_managed_by_our_internals() is true. That is also true for Python subclasses after all_type_info() caches them. The name of class D(m.Base) changes from D to __main__.D after the first D(). Each call on a heap type also takes the internals lock.
5. include/pybind11/detail/class-inl.h:1136: under Py_LIMITED_API, set_published_type_names() does not restore tp_name after it sets __name__. The type.__name__ setter overwrites tp_name, so the comment there is wrong for names that contain a dot. py::class_<T>(m, "a.b") gets tp_name == "a.b" with no module.

Lower risk

6. include/pybind11/pytypes.h:514: get_tp_name() builds a std::string from tp_name, and the null checks in error_fetch_and_normalize are gone. A null tp_name is now undefined behavior and not a pybind11_fail. This needs an exception type that is not ready, which is rare.
7. include/pybind11/pytypes.h:1354: get_staticmethod_type() reads staticmethod from the current frame's builtins (PyEval_GetBuiltins()) and caches it in a plain static. A frame with a custom __builtins__ can make the lookup fail or cache the wrong object. The review says the other three type caches use gil_safe_call_once_and_store and a module import.
8. tools/pybind11NewTools.cmake:471 (not confirmed): the default STABLE_ABI MODULE path links pybind11::module in addition to USE_SABI. The review says this puts python3.lib and python3XY.lib on the Windows link line, and only link order keeps the .pyd version-independent. The review also says CI runs the abi3 forward test only on Linux.

Efficiency and cleanup

9. include/pybind11/pytypes.h:2752: stable-ABI type_lookup() uses PyObject_GetItem for targets below 3.13. Each miss raises and clears a KeyError, one for each MRO class. A call to the mappingproxy get method prevents the exceptions.
10. include/pybind11/detail/class.h:57-59: static_property_dict_ptr, pybind11_static_property_traverse and pybind11_static_property_clear are declared but have no definition and no caller. Remove them.

Local origin/master is stale, so the reviewed diff also contains merged commit #6248.

Applied fixes:
🤖 AI text below 🤖

• Finding 5 has no real fix: the stable ABI cannot write tp_name, so the difference for names with a dot stays.
• Finding 7 still uses PyImport_ImportModule: I first used an import that bypasses builtins.__import__, but tests/test_cmake_build/test.py needs the types import to go through that hook. A frame whose builtins have no __import__ still makes the lookup fail.
• Finding 8, precompiled library: it now calls _pybind11_stable_abi_setup(), the same setup as a stable-ABI module. This changes the Windows link line, so Windows CI must confirm it.

Chrono.h change summary:

🤖 AI text below 🤖

The chrono.h changes let the std::chrono casters build under Py_LIMITED_API. The datetime C API (datetime.h, PyDateTimeAPI, and the PyDateTime_* macros) is not part of the stable ABI. The fix puts every datetime operation behind a small set of detail:: helpers. Each helper has two builds: one for the limited API, and one for the full API.

The helper layer

The casters now call only these helpers:

PYBIND11_DATETIME_ACCESSOR generates the 16 field accessors from one table, so both builds share the same list. The macro is #undef'd after use.

The caster bodies (duration_caster, and the time_point caster's load/cast) change mechanically: each macro call becomes a call to the matching helper. The conversion logic does not change.

Limited API details

• One-time init with gil_safe_call_once_and_store. Importing datetime runs Python code, and that code can release the GIL. A plain static local or std::call_once could then deadlock with another thread. gil_safe_call_once_and_store sets a safe lock order. The 4 type objects and each field's descriptor getter are kept for the life of the process. They are never decref'd.
• Field reads go through the base type's descriptor. For example, datetime.datetime.year.__get__(obj), not obj.year. A subclass that overrides year with a property therefore does not change the result. This matches the C macros, which read the struct fields directly.
  • date.year is inherited by datetime, so the datetime_year/month/day getters also work for plain date objects. The is_date branch depends on this.
• Cost. Each field read is a Python call, so one datetime load makes about 9 calls. This is much slower than the macros, which only read struct fields. This is the normal cost of the stable ABI.
• Error behavior. If the datetime import fails, error_already_set is now raised from load/cast. The old code continued with a null PyDateTimeAPI. In the limited build, make_datetime and datetime_from_timestamp throw on failure because object::operator() throws. Thus the existing if (!dt) throw error_already_set(); checks cannot trigger there. They are harmless, and they are still needed for the full-API build.

Full API detail: static inline

PyDateTimeAPI is a static variable, so each translation unit has its own copy. Before this change, each caster did its own import check and then used its own copy, so the pointer was always set. With the code split into helpers, an inline (external linkage) datetime_import() from TU A could be paired with an is_timedelta() from TU B. B's pointer was never set, so that would cause a null dereference. With static inline, each TU calls its own helpers, so its import and its checks always use the same pointer.

The only other change in the full-API path is the #include <datetime.h>. It is now skipped when Py_LIMITED_API is defined.

Result of giving the chrono.h changes a close look (LGTM):

codex GPT-6.1-Sol ultra:

1. Subclass behavior: preserve the stored datetime value.

   CPython’s existing datetime macros accept subclasses and extract their stored fields. A subclass can expose different values through ordinary Python attribute access. CPython documentation

   For example:

   class Weird(datetime.datetime):
       @property
       def year(self):
           return 2001
   
   value = Weird(2020, 1, 2)
   value.year                              # 2001
   datetime.datetime.year.__get__(value)    # 2020

   Replacing the C macro with value.year would therefore change behavior. The implementation instead obtains the base type’s descriptor, caches its __get__, and calls that getter with the supplied object. That preserves the stored value even when a subclass overrides the attribute. I checked this directly for datetime, time, and timedelta subclasses. Getter implementation

   System-clock conversion also constructs a plain datetime before calling timestamp(), so a subclass’s overridden timestamp() cannot intervene. The PR has regression coverage for both overridden fields and overridden timestamp(). Field test, timestamp handling

   This part looks deliberate and appropriate: the extra indirection preserves existing CPython semantics.

2. Cache initialization: avoid imports and descriptor lookup on every conversion.

   There are two cache layers:

   • One holds the four datetime types.
   • Sixteen field accessors each lazily cache a descriptor getter. Date conversions reuse the datetime year/month/day getters.

   The retained Python references keep those objects alive. Subsequent field reads reuse the cached getter. Cache implementation

   Initialization uses the existing gil_safe_call_once_and_store helper. This matters because importing Python code can release the GIL. With a naïve static initializer, another thread could acquire the GIL and then wait for that initializer, preventing the initializing thread from reacquiring the GIL.

   The helper establishes the safe lock order: release the GIL before waiting on std::call_once, then reacquire it inside the initializer. Its process-lifetime storage also avoids Python-object destruction during C++ static teardown. Helper implementation

   The maintenance benefit is that chrono reuses this machinery rather than implementing its own initialization protocol.

3. Repeated Python calls: caching removes lookup costs, but field reads remain more expensive.

   Each cached getter invocation still produces a Python integer, which is converted to C++ int. The callee is normally implemented in C, but Python’s calling and value-conversion machinery still has to run.

   Counting explicit callable invocations in the usual warm stable-ABI paths:

   Conversion	Calls
   timedelta → C++ duration	3 field getters
   datetime → C++ system-clock time point	8 getters + constructor + timestamp() = 10
   C++ system-clock time point → datetime	fromtimestamp() + 7 getters + constructor = 9
   C++ duration → timedelta	1 constructor

   These are totals for that implementation; some work, including the input timestamp() call, is shared with the full-API path. Windows’ near-epoch shortcut omits some calls. Conversion code

   Thus, the overhead is concrete, although its actual timing remains unmeasured here. Occasional conversions and a tight conversion loop can experience very different application-level effects.

I’d retain the support: the subclass semantics are preserved and initialization reuses established machinery.

Result of giving the numpy.h changes a close look (LGTM):

codex GPT-6.1-Sol ultra:

Yes. The key distinction is existing knowledge of NumPy’s layouts versus the new mechanism for finding those layouts when Python’s object header is opaque.

1. What pybind11 already does

   numpy.h contains small C++ structs that mirror NumPy’s array and dtype layouts. These let pybind11 read fields such as the data pointer, dimensions, strides, and dtype metadata without requiring NumPy headers when compiling an extension.

   This layout knowledge predates abi3t. So does the compatibility handling for NumPy 1.x versus 2.x, whose dtype layouts differ. Proxy definitions

   The maintenance obligation to track NumPy’s layouts therefore already existed.

2. What abi3t changes

   With an ordinary build, pybind11 knows the Python object header layout from the headers it compiled against. An abi3t binary must accommodate both GIL-enabled and free-threaded interpreters, whose headers can differ. It cannot calculate where NumPy’s fields begin by assuming a particular sizeof(PyObject).

   The solution is to let the loaded NumPy implementation calculate that address:

   • _PyArray_GET_ITEM_DATA locates an array’s metadata.
   • _PyDataType_GET_ITEM_DATA locates a dtype’s metadata.

   These are NumPy-exported C-API functions. Their implementations calculate the appropriate native offsets. Pybind11 then uses proxies that omit the Python header. NumPy implementations, pybind11 accessors

   One naming detail: _PyArray_GET_ITEM_DATA returns the metadata structure, whose data member points to the array’s element bytes.

   Build mode	Access strategy
   Ordinary full API, including native free-threaded builds	Existing header-containing proxies
   abi3	Existing header-containing proxies
   abi3t	NumPy accessor → headerless proxy

   This branch follows the build target, including when an abi3t module runs on a GIL-enabled interpreter. abi3t configuration

3. The new NumPy dependency is explicit and checked

   Those two C-API entries were introduced in NumPy 2.5. Pybind11 checks NumPy’s runtime C-API feature version against 0x16 before fetching them. Older NumPy produces an explicit error. API definitions, runtime guard

   There are two distinct version boundaries: the accessor API dates from 2.5, while supported Python 3.15 combinations start with NumPy 2.5.2. Release notes

   CPython’s Stable ABI and NumPy’s ABI remain separate compatibility contracts. This bridge removes the need to know the Python header’s size; pybind11 still depends on NumPy’s C API and field layouts. NumPy ABI guidance

4. Structured scalars need a separate storage-access solution

   This concerns structured numpy.void scalars—records corresponding to registered C++ structs. The converter first checks that the object is a numpy.void instance or subclass and has an equivalent dtype.

   Previously, it obtained the record bytes directly from the scalar’s obval field. That struct access is unavailable under abi3t, so the new path calls NumPy’s base numpy.void buffer getter. Converter

   Calling the base getter is essential: a subclass’s __buffer__ can supply unrelated bytes or temporary storage. The base getter exposes the scalar’s existing record storage.

   Why does the pointer survive releasing the buffer export? NumPy’s getter returns scalar->obval and adds a reference to the scalar. Cleanup drops that extra export reference; the original Python argument still keeps the scalar—and any backing array—alive during conversion and the bound call. This reasoning depends on NumPy’s storage behavior, rather than a general buffer-protocol guarantee about pointers after release. NumPy getter

   Your included fix supplies the matching cleanup: use the base release slot, then decrement the export owner. Generic PyBuffer_Release() could invoke a subclass’s __release_buffer__ even though its acquisition hook was bypassed. In NumPy 2.5.4, the base void type has no release callback. Base slots

   The regression test makes this observable: the scalar stores 2, its overridden buffer supplies 7, and conversion returns 20 while invoking neither subclass hook. It first verifies that an ordinary memoryview really does invoke both hooks. Regression test

The parallel with chrono is preserving stored-value semantics while bypassing subclass overrides. Here, NumPy-specific layout knowledge remains, while NumPy itself supplies the runtime-correct starting address. The implementation looks consistent with those goals.

This is preliminary: tracking here, but only as expandable details:

Regarding point 5 in my previous comment:

I just pushed commit dabeb4a.

Sharing a follow-on prompt/response:

My prompt

I want to look slightly beyond the horizon: is there a indication why the size/offset calculation was left as an implementation detail?

Where my mind is going:

C++ std::complex left .real and .imag struct layout as an implementation detail.
A long time ago, balancing practicality with purity, I wrote code that assumed the layout in memory is (e.g.) double parts[2]
It's obviously a massive simplification, and still works 25 years later; most likely it'll still work another 25 years later.

Do we have a situation like that with the size/offset calculation, or should we be more worried?

codex GPT-6.1-Sol ultra

Yes—there is a concrete design reason, and it makes me more cautious about expecting this convention to last another 25 years.

In the original PEP discussion, Mark Shannon argued that the API should describe how to obtain a class’s data without prescribing its location. Petr Viktorin explicitly agreed: the function bodies illustrated current CPython, while the API should accommodate a future object-layout redesign. Mark’s critique, Petr’s response.

The motivating example includes fixed-size classes with multiple inheritance. If two extension classes each contribute a C struct, Python could choose where those structs belong in a combined subclass. A base class’s data would then potentially have different offsets in different derived classes. Petr confirmed that the API should permit Python to make that placement decision. This concern extends beyond variable-sized objects such as tuples. Multiple-inheritance example and response.

Our calculation nevertheless has substantial practical robustness:

offset = base_basicsize - base_type_data_size;

It calculates the offset at runtime, so a different object-header size or alignment does not, by itself, break it. Its consequential assumptions are:

1. The base’s data region occupies the end of that base’s instance layout.
2. That region retains the same offset in every derived object.

There is an actual proposal that challenges those assumptions. The Grand Unified Python Object Layout proposal allows some extension structs before the core object and explicitly allows a base’s struct to move within derived objects. Whether such a redesign would affect existing PEP 697 types remains unsettled.

There is also some recent interest: in April 2026, Petr expressed an intention to prepare a PEP for Python 3.16. I found no published PEP or accepted implementation in the primary sources checked. That comment establishes interest, rather than a release commitment.

My engineering judgment: retaining the implementation remains a defensible practical choice, but I would justify it as a deliberate dependency on current CPython behavior, with a concrete upstream design direction to watch. I would place less confidence in its lasting for decades than your std::complex example.

The stable-ABI context adds weight: if a future Python breaks this assumption, updating pybind11 would not repair already distributed wheels compiled with the old calculation; those wheels would need rebuilding. That makes the dependency relevant to the forward-compatibility promise, even though we have no demonstrated failure today.

One interesting wrinkle in your analogy: today’s C++ specification explicitly guarantees the real/imaginary array-style access you describe. Your practical choice ultimately acquired a formal guarantee. Current C++ draft.