Add Python bindings build using bzlmod #1764
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uses the newly started
@nanobind_bazel
project to build nanobind extensions. This means that we can drop all in-tree custom build defs and build files for nanobind and the C++ Python headers.Additionally, the temporary WORKSPACE overwrite hack naturally goes away due to the WORKSPACE system being obsolete.
The git override might be a little shaky, but I have not released
nanobind-bazel
on the BCR yet, so this will have to do for now as a proof of concept.Backstory & explanations
Before,
google_benchmark
had a custom Python binding extension in-tree (bindings/python/google_benchmark/build_defs.bzl
), which was defined back in the pybind11 days (and possibly even predatingpybind11_bazel
, which also contains these defs).Since bindings are now built using nanobind, I decided to make use of nanobind-bazel, a self-developed counterpart for easy building of nanobind extensions with Bazel. Its main strength is that it also includes flags for optionally building with size optimizations enabled, targeting the Python stable ABI, and platform-specific compiler and linker options.
The main benefit for GBM is that all of these aspects are now in
nanobind_bazel
's hands, which relieves maintainers from having to adjust those optimizations (and if necessary, implement them as patch files to use in agit_override
).Another reason why this has become feasible now is that instead of having to monkey-patch the Python include path, newer versions of
@rules_python
include the@rules_python//python/cc:current_py_cc_headers
target, which gives the header files for the currently configured Python toolchain. The targeted Python interpreter can be given via a build flag, which is done in the build command insetup.py
.For our (admittedly clunky) way of building the Python wheels, this means that the
.so
files are built with a hermetic interpreter, which looks a bit sketchy, but ultimately does not break anything as the C API is promised stable in a given minor version of Python.Currently, the Python stable ABI is targeted starting from Python 3.12, which means that Python users of Google Benchmark will enjoy stability across Python minor versions going forward. A quick audit of the
_benchmark.so
extension on my local machine shows no ABI violations, but adding such a check into CI might be a good idea.