//python:defs.bzl

Core rules for building Python projects.

rule current_py_toolchain(name)

This rule exists so that the current python toolchain can be used in the toolchains attribute of other rules, such as genrule. It allows exposing a python toolchain after toolchain resolution has happened, to a rule which expects a concrete implementation of a toolchain, rather than a toolchain_type which could be resolved to that toolchain.

Changed in version 1.4.0: From now on, we also expose $(PYTHON2_ROOTPATH) and $(PYTHON3_ROOTPATH) which are runfiles locations equivalents of $(PYTHON2) and `$(PYTHON3) respectively.

Attributes:

  • name – (Name)

    A unique name for this target.

    mandatory

py_binary(**attrs)

Creates an executable Python program.

This is the public macro wrapping the underlying rule. Args are forwarded on as-is unless otherwise specified. See the underlying py_binary rule for detailed attribute documentation.

This macro affects the following args:

  • python_version: cannot be PY2

  • srcs_version: cannot be PY2 or PY2ONLY

  • tags: May have special marker values added, if not already present.

Changed in version VERSION_NEXT_FEATURE: The PYTHONBREAKPOINT environment variable is inherited. Use in combination with --debugger to customize the debugger available and used.

Args:
rule py_import(name, deps=[], srcs=[])

This rule allows the use of Python packages as dependencies.

It imports the given .egg file(s), which might be checked in source files, fetched externally as with http_file, or produced as outputs of other rules.

It may be used like a py_library, in the deps of other Python rules.

This is similar to java_import.

Attributes:

  • name – (Name)

    A unique name for this target.

    mandatory

  • deps – (list[label]) (default [])

    The list of other libraries to be linked in to the binary target.

    optional

    Required providers: PyInfo

  • srcs – (list[label]) (default [])

    The list of Python package files provided to Python targets that depend on this target. Note that currently only the .egg format is accepted. For .whl files, try the whl_library rule. We accept contributions to extend py_import to handle .whl.

    optional

py_library(**attrs)

Creates an executable Python program.

This is the public macro wrapping the underlying rule. Args are forwarded on as-is unless otherwise specified. See py_library for detailed attribute documentation.

This macro affects the following args:

  • srcs_version: cannot be PY2 or PY2ONLY

  • tags: May have special marker values added, if not already present.

Args:
py_runtime(**attrs)

Creates an executable Python program.

This is the public macro wrapping the underlying rule. Args are forwarded on as-is unless otherwise specified. See py_runtime for detailed attribute documentation.

This macro affects the following args:

  • python_version: cannot be PY2

  • srcs_version: cannot be PY2 or PY2ONLY

  • tags: May have special marker values added, if not already present.

Args:
py_runtime_pair(name, py2_runtime=None, py3_runtime=None, **attrs)

A toolchain rule for Python.

This is a macro around the underlying py_runtime_pair rule.

This used to wrap up to two Python runtimes, one for Python 2 and one for Python 3. However, Python 2 is no longer supported, so it now only wraps a single Python 3 runtime.

Usually the wrapped runtimes are declared using the py_runtime rule, but any rule returning a PyRuntimeInfo provider may be used.

This rule returns a platform_common.ToolchainInfo provider with the following schema:

platform_common.ToolchainInfo(
    py2_runtime = None,
    py3_runtime = <PyRuntimeInfo or None>,
)

Example usage:

# In your BUILD file...

load("@rules_python//python:py_runtime.bzl", "py_runtime")
load("@rules_python//python:py_runtime_pair.bzl", "py_runtime_pair")

py_runtime(
    name = "my_py3_runtime",
    interpreter_path = "/system/python3",
    python_version = "PY3",
)

py_runtime_pair(
    name = "my_py_runtime_pair",
    py3_runtime = ":my_py3_runtime",
)

toolchain(
    name = "my_toolchain",
    target_compatible_with = <...>,
    toolchain = ":my_py_runtime_pair",
    toolchain_type = "@rules_python//python:toolchain_type",
)

# In your WORKSPACE...

register_toolchains("//my_pkg:my_toolchain")
Args:

  • name – str, the name of the target

  • py2_runtime – (default None)

    optional Label; must be unset or None; an error is raised otherwise.

  • py3_runtime – (default None)

    Label; a target with PyRuntimeInfo for Python 3.

  • attrs – Extra attrs passed onto the native rule

py_test(**attrs)

Creates an executable Python program.

This is the public macro wrapping the underlying rule. Args are forwarded on as-is unless otherwise specified. See py_test for detailed attribute documentation.

This macro affects the following args:

  • python_version: cannot be PY2

  • srcs_version: cannot be PY2 or PY2ONLY

  • tags: May have special marker values added, if not already present.

Args:
provider PyInfo

Encapsulates information provided by the Python rules.

Instead of creating this object directly, use PyInfoBuilder and the PyCommonApi utilities.

PyInfo.<init>(direct_original_sources, direct_pyc_files, direct_pyi_files, has_py2_only_sources, has_py3_only_sources, imports, transitive_implicit_pyc_files, transitive_implicit_pyc_source_files, transitive_original_sources, transitive_pyc_files, transitive_pyi_files, transitive_sources, uses_shared_libraries, venv_symlinks)
PyInfo.direct_original_sources: depset[File]

The .py source files (if any) that are considered directly provided by the target. This field is intended so that static analysis tools can recover the original Python source files, regardless of any build settings (e.g. precompiling), so they can analyze source code. The values are typically the .py files in the srcs attribute (or equivalent).

Added in version 1.1.0.

PyInfo.direct_pyc_files: depset[File]

Precompiled Python files that are considered directly provided by the target and must be included.

These files usually come from, e.g., a library setting precompile=enabled to forcibly enable precompiling for itself. Downstream binaries are expected to always include these files, as the originating target expects them to exist.

PyInfo.direct_pyi_files: depset[File]

Type definition files (usually .pyi files) for the Python modules provided by this target. Usually they describe the source files listed in direct_original_sources. This field is primarily for static analysis tools.

These files are usually build-time only and not included as part of a runnable program.

Note

This may contain implementation-specific file types specific to a particular type checker.

Added in version 1.1.0.

PyInfo.has_py2_only_sources: bool

Whether any of this target’s transitive sources requires a Python 2 runtime.

PyInfo.has_py3_only_sources: bool

Whether any of this target’s transitive sources requires a Python 3 runtime.

PyInfo.imports: depset[str]

A depset of import path strings to be added to the PYTHONPATH of executable Python targets. These are accumulated from the transitive deps. The order of the depset is not guaranteed and may be changed in the future. It is recommended to use default order (the default).

PyInfo.transitive_implicit_pyc_files: depset[File]

Automatically generated pyc files that downstream binaries (or equivalent) can choose to include in their output. If not included, then transitive_implicit_pyc_source_files should be included instead.

Added in version 0.37.0.

PyInfo.transitive_implicit_pyc_source_files: depset[File]

Source .py files for transitive_implicit_pyc_files that downstream binaries (or equivalent) can choose to include in their output. If not included, then transitive_implicit_pyc_files should be included instead.

Added in version 0.37.0.

PyInfo.transitive_original_sources: depset[File]

The transitive set of .py source files (if any) that are considered the original sources for this target and its transitive dependencies. This field is intended so that static analysis tools can recover the original Python source files, regardless of any build settings (e.g. precompiling), so they can analyze source code. The values are typically the .py files in the srcs attribute (or equivalent).

This is superset of direct_original_sources.

Added in version 1.1.0.

PyInfo.transitive_pyc_files: depset[File]

The transitive set of precompiled files that must be included.

These files usually come from, e.g., a library setting precompile=enabled to forcibly enable precompiling for itself. Downstream binaries are expected to always include these files, as the originating target expects them to exist.

PyInfo.transitive_pyi_files: depset[File]

The transitive set of type definition files (usually .pyi files) for the Python modules for this target and its transitive dependencies. this target. Usually they describe the source files listed in transitive_original_sources. This field is primarily for static analysis tools.

These files are usually build-time only and not included as part of a runnable program.

Note

This may contain implementation-specific file types specific to a particular type checker.

Added in version 1.1.0.

PyInfo.transitive_sources: depset[File]

A (postorder-compatible) depset of .py files that are considered required and downstream binaries (or equivalent) must include in their outputs to have a functioning program.

Normally, these are the .py files in the appearing in the target’s srcs and the srcs of the target’s transitive deps, however, precompile settings may cause .py files to be omitted. In particular, pyc-only builds may result in this depset being empty.

Changed in version 0.37.0: The files are considered necessary for downstream binaries to function; previously they were considerd informational and largely unused.

PyInfo.uses_shared_libraries: bool

Whether any of this target’s transitive deps has a shared library file (such as a .so file).

This field is currently unused in Bazel and may go away in the future.

Warning

Experimental API. This API is still under development and may change or be removed without notice.

Added in version 1.5.0.

provider PyRuntimeInfo

Contains information about a Python runtime, as returned by the py_runtime rule.

Warning

This is an unstable public API. It may change more frequently and has weaker compatibility guarantees.

A Python runtime describes either a platform runtime or an in-build runtime. A platform runtime accesses a system-installed interpreter at a known path, whereas an in-build runtime points to a File that acts as the interpreter. In both cases, an “interpreter” is really any executable binary or wrapper script that is capable of running a Python script passed on the command line, following the same conventions as the standard CPython interpreter.

PyRuntimeInfo.<init>(abi_flags, bootstrap_template, coverage_files, coverage_tool, files, implementation_name, interpreter, interpreter_path, interpreter_version_info, pyc_tag, python_version, site_init_template, stage2_bootstrap_template, stub_shebang, supports_build_time_venv, zip_main_template)
PyRuntimeInfo.abi_flags: str

The runtime’s ABI flags, i.e. sys.abiflags.

Added in version 1.0.0.

PyRuntimeInfo.bootstrap_template: File

A template of code responsible for the initial startup of a program.

This code is responsible for:

  • Locating the target interpreter. Typically it is in runfiles, but not always.

  • Setting necessary environment variables, command line flags, or other configuration that can’t be modified after the interpreter starts.

  • Invoking the appropriate entry point. This is usually a second-stage bootstrap that performs additional setup prior to running a program’s actual entry point.

The --bootstrap_impl flag affects how this stage 1 bootstrap is expected to behave and the substutitions performed.

  • --bootstrap_impl=system_python substitutions: %is_zipfile%, %python_binary%, %target%, %workspace_name, %coverage_tool%, %import_all%, %imports%, %main%, %shebang%

  • --bootstrap_impl=script substititions: %is_zipfile%, %python_binary%, %python_binary_actual%, %target%, %workspace_name, %shebang%, %stage2_bootstrap%

Substitution definitions:

  • %shebang%: The shebang to use with the bootstrap; the bootstrap template may choose to ignore this.

  • %stage2_bootstrap%: A runfiles-relative path to the stage 2 bootstrap.

  • %python_binary%: The path to the target Python interpreter. There are three types of paths:

    • An absolute path to a system interpreter (e.g. begins with /).

    • A runfiles-relative path to an interpreter (e.g. somerepo/bin/python3)

    • A program to search for on PATH, i.e. a word without spaces, e.g. python3.

    When --bootstrap_impl=script is used, this is always a runfiles-relative path to a venv-based interpreter executable.

  • %python_binary_actual%: The path to the interpreter that %python_binary% invokes. There are three types of paths:

    • An absolute path to a system interpreter (e.g. begins with /).

    • A runfiles-relative path to an interpreter (e.g. somerepo/bin/python3)

    • A program to search for on PATH, i.e. a word without spaces, e.g. python3.

    Only set for zip builds with --bootstrap_impl=script; other builds will use an empty string.

  • %workspace_name%: The name of the workspace the target belongs to.

  • %is_zipfile%: The string 1 if this template is prepended to a zipfile to create a self-executable zip file. The string 0 otherwise.

For the other substitution definitions, see the stage2_bootstrap_template docs.

Changed in version 0.33.0: The set of substitutions depends on --bootstrap_impl

PyRuntimeInfo.coverage_files: depset[File] | None

The files required at runtime for using coverage_tool. Will be None if no coverage_tool was provided.

PyRuntimeInfo.coverage_tool: File | None

If set, this field is a File representing tool used for collecting code coverage information from python tests. Otherwise, this is None.

PyRuntimeInfo.files: depset[File] | None

If this is an in-build runtime, this field is a depset of Files that need to be added to the runfiles of an executable target that uses this runtime (in particular, files needed by interpreter). The value of interpreter need not be included in this field. If this is a platform runtime then this field is None.

PyRuntimeInfo.implementation_name: str | None

The Python implementation name (sys.implementation.name)

PyRuntimeInfo.interpreter: File | None

If this is an in-build runtime, this field is a File representing the interpreter. Otherwise, this is None. Note that an in-build runtime can use either a prebuilt, checked-in interpreter or an interpreter built from source.

PyRuntimeInfo.interpreter_path: str | None

If this is a platform runtime, this field is the absolute filesystem path to the interpreter on the target platform. Otherwise, this is None.

PyRuntimeInfo.interpreter_version_info: struct

Version information about the interpreter this runtime provides. It should match the format given by sys.version_info, however for simplicity, the micro, releaselevel, and serial values are optional. A struct with the following fields:

  • major: int, the major version number

  • minor: int, the minor version number

  • micro: int | None, the micro version number

  • releaselevel: str | None, the release level

  • serial: int | None, the serial number of the release

PyRuntimeInfo.pyc_tag: str | None

The tag portion of a pyc filename, e.g. the cpython-39 infix of foo.cpython-39.pyc. See PEP 3147. If not specified, it will be computed from implementation_name and interpreter_version_info. If no pyc_tag is available, then only source-less pyc generation will function correctly.

PyRuntimeInfo.python_version: str

Indicates whether this runtime uses Python major version 2 or 3. Valid values are (only) "PY2" and "PY3".

PyRuntimeInfo.site_init_template: File

The template to use for the binary-specific site-init hook run by the interpreter at startup.

Added in version 1.0.0.

PyRuntimeInfo.stage2_bootstrap_template: File

A template of Python code that runs under the desired interpreter and is responsible for orchestrating calling the program’s actual main code. This bootstrap is responsible for affecting the current runtime’s state, such as import paths or enabling coverage, so that, when it runs the program’s actual main code, it works properly under Bazel.

The following substitutions are made during template expansion:

  • %main%: A runfiles-relative path to the program’s actual main file. This can be a .py or .pyc file, depending on precompile settings.

  • %coverage_tool%: Runfiles-relative path to the coverage library’s entry point. If coverage is not enabled or available, an empty string.

  • %import_all%: The string True if all repositories in the runfiles should be added to sys.path. The string False otherwise.

  • %imports%: A colon-delimited string of runfiles-relative paths to add to sys.path.

  • %target%: The name of the target this is for.

  • %workspace_name%: The name of the workspace the target belongs to.

Added in version 0.33.0.

PyRuntimeInfo.stub_shebang: str

“Shebang” expression prepended to the bootstrapping Python stub script used when executing py_binary targets. Does not apply to Windows.

PyRuntimeInfo.supports_build_time_venv: bool

True if this toolchain supports the build-time created virtual environment. False if not or unknown. If build-time venv creation isn’t supported, then binaries may fallback to non-venv solutions or creating a venv at runtime.

In order to use the build-time created virtual environment, a toolchain needs to meet two criteria:

  1. Specifying the underlying executable (e.g. /usr/bin/python3, as reported by sys._base_executable) for the venv executable ($venv/bin/python3, as reported by sys.executable). This typically requires relative symlinking the venv path to the underlying path at build time, or using the PYTHONEXECUTABLE environment variable (Python 3.11+) at runtime.

  2. Having the build-time created site-packages directory (<venv>/lib/python{version}/site-packages) recognized by the runtime interpreter. This typically requires the Python version to be known at build-time and match at runtime.

Added in version 1.5.0.

PyRuntimeInfo.zip_main_template: File

A template of Python code that becomes a zip file’s top-level __main__.py file. The top-level __main__.py file is used when the zip file is explicitly passed to a Python interpreter. See PEP 441 for more information about zipapp support. Note that py_binary-generated zip files are self-executing and skip calling __main__.py.

The following substitutions are made during template expansion:

  • %stage2_bootstrap%: A runfiles-relative string to the stage 2 bootstrap file.

  • %python_binary%: The path to the target Python interpreter. There are three types of paths:

    • An absolute path to a system interpreter (e.g. begins with /).

    • A runfiles-relative path to an interpreter (e.g. somerepo/bin/python3)

    • A program to search for on PATH, i.e. a word without spaces, e.g. python3.

  • %workspace_name%: The name of the workspace for the built target.

Added in version 0.33.0.