Numba Dynamic Exceptions

Numba 0.57 was recently released, and it added an important feature: dynamic exceptions. Numba now supports exceptions with runtime arguments. Since version 0.13.2, Numba had limited support for exceptions: arguments had to be compile-time constants.

Although Numba's focus is on compiling Python into fast machine code, there is still value in providing better support for exceptions. Improving support means that exception messages can now include more comprehensive content - for example, an IndexError can now include the index in the exception message.

Past, present and future

Before Numba 0.57, exceptions were limited to compile-time constants only. This means that users could only raise exceptions in the following form:


from numba import njit
@njit
def getitem(lst: list[int], idx: int):
    if idx >= len(lst):
        raise IndexError('list index out of range')
    return lst[idx]

Attempting to raise an exception with runtime values in versions prior to 0.57 would result in a compilation error:


from numba import njit
@njit
def getitem(lst: list[int], index: int):
    if index >= len(lst):
        raise IndexError(f'list index "{index}" out of range')
    return lst[index]


$ python -c 'import numba; print(numba.__version__)'
0.56.4
$ python example.py
Traceback (most recent call last):
  File "/Users/guilhermeleobas/git/blog/example.py", line 13, in <module>
    print(getitem(lst, index))
  File "/Users/guilhermeleobas/miniconda3/envs/numba056/lib/python3.10/site-packages/numba/core/dispatcher.py", line 480, in _compile_for_args
    error_rewrite(e, 'constant_inference')
  File "/Users/guilhermeleobas/miniconda3/envs/numba056/lib/python3.10/site-packages/numba/core/dispatcher.py", line 409, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.ConstantInferenceError: Failed in nopython mode pipeline (step: nopython rewrites)
Constant inference not possible for: $24build_string.6 + $const22.5
File "example.py", line 7:
def getitem(lst: list[int], index: int):
    <source elided>
    if index >= len(lst):
        raise IndexError(f'list index "{index}" out of range')
        ^

This example works just fine in the latest release.


$ python -c 'import numba; print(numba.__version__)'
0.57.0
$ python example.py
Traceback (most recent call last):
  File "/Users/guilhermeleobas/git/blog/example.py", line 13, in <module>
    print(getitem(lst, index))
  File "/Users/guilhermeleobas/git/blog/example.py", line 7, in getitem
    raise IndexError(f'list index "{index}" out of range')
IndexError: list index "4" out of range

In the future, Numba users can expect better exception messages raised from Numba overloads and compiled code.

How does it work?

Numba is a JIT compiler that translates a subset of Python into machine code. This translation step is done using LLVM. When Numba compiled code raises an exception, it must signal to the interpreter and propagate any required information back. The calling convention for CPU targets specifies how signaling is done:


retcode_t (<Python return type>*, excinfo_t **, ... <Python arguments>)

The return code is one of the RETCODE_* constants in the callconv.py file.

Control flow of execution when an exception is raised
Figure contains a high-level illustration of the control flow when a Numba function raises an exception.

Static Exceptions

When an exception is raised, the struct excinfo_t** is filled with a pointer to a struct describing the raised exception. Before Numba 0.57, this struct contained three fields:

A pointer (i8*) to a pickled string.
String size (i32).
Hash (i8*) of this same string.

Take for instance the following snippet of code:


@jit(nopython=True)
def func():
    raise ValueError('exc message')

The triple (ValueError, 'exc message', location) is pickled and serialized to the LLVM module as a constant string. When the exception is raised, this same serialized string is unpickled by the interpreter (1) and a frame is created for the exception (2).

Dynamic Exceptions

To support dynamic exceptions, we reuse all the existing fields and introduce two new ones.

A pointer (i8*) to a pickled string containing static information.
String size (i32).
The third argument (i8*), which was previously used for hashing is now used to hold a list of native values.
A pointer to a function (i8*) that knows how to convert native values back to Python values. This is called boxing.
A flag (i32) to signal whether an exception is static or dynamic. A value greater than zero not only indicates whether it is a dynamic exception, but also the number of runtime arguments.

Using Python code, dynamic exceptions work as follows:


@jit(nopython=True)
def dyn_exc_func(s: str):
    raise TypeError('error', s, len(s))

For each dynamic exception, Numba will generate a function that boxes native values into Python types. In the example above, __exc_conv will be generated automatically:


def __exc_conv(s: native_string, i: int64) -> Tuple[str, int]:
    # convert
    py_string: str = box(s)
    py_int: int = box(i)
    return (py_string, py_int)

The code mentioned earlier is used for illustrative purposes. However, in practice, __exc_conv is implemented as native code.

The excinfo struct will be filled with:

Pickled string of compile-time information: (exception type, static arguments, location).
String size.
A list of dynamic arguments: [native string, int64].
A pointer to __exc_conv.
Number of dynamic arguments: 2.

During runtime, just before the control flow is returned to the interpreter, function __exc_conv is invoked to convert native string/int values into their equivalent Python str/int types. At this stage, the interpreter also unpickles constant information, and both static and dynamic arguments are combined into a unified list (3).

I encourage anyone interested in further details to read the comments left on callconv.py::CPUCallConv (ref).

Limitations and future work

Numba has a page describing what is supported in exception handling. Some work still needs to be done to support exceptions to their full extent.

We would like to thank Bodo for sponsoring this work and the Numba core developers and community for reviewing this work and the useful insights given during code review.