uarray: Attempting to move the ecosystem forward
Published November 10, 2019
There comes a time in every project where most technological hurdles have been surpassed, and its adoption is a social problem. I believe
unumpy had reached such a state, a month ago.
I then proceeded, along with Ralf Gommers and Peter Bell to write NumPy Enhancement Proposal 31 or NEP-31. This generated a lot of excellent feedback on the structure and the nuances of the proposal, which you can read both on the pull request and on the mailing list discussion, which led to a lot of restructuring in the contents and the structure of the NEP, but very little in the actual proposal. I take full responsibility for this: I have a bad tendency to assume everyone knows what I'm thinking. Thankfully, I'm not alone in this: It's a known psychological phenomenon.
Of course, social problems can take a long time to resolve one way or another, regardless of the proponents. And I consider this a good thing: it's better not to be stuck with an API decision that may bite you a few years down the line, especially with a project with API compatibility guarantees and number of dependents as NumPy.
I must confess I felt discouraged at some points in the
uarray journey: However, realising my flaws will make me perform better in the future.
Although my main focus at this point in my career isn't
uarray (not that I don't want it to be, only that social problems don't take very much of your time), it isn't a small library at the back of my head by any means.
With that out of the way, I'd like to present some improvements the Quansight team has made to
uarray over the course of the past several months:
Progress on Integration
uarray is now the official override mechanism for
scipy.fft. This is a huge win.
NEP-31 merged with draft status
Already discussed above, but NEP-31 is available on the NumPy website in draft form.
C++ Implementation Merged
The C++ implementation of the
uarray protocol is now merged.
uarray is now lightning fast: About as fast as
Global backend as the "only" backend
set_global_backend gained an
only= kwarg. You can read what this means in the docs, but basically: In the absence of any local backends; no other backend will be tried at all.
We now have cross-platform CI for Windows, macOS and Linux.
Building wheels on CI
Backends can fail fast
New website 🥳
scipy.fft in CI
Re-entrant context managers
Then we cache for you
Rich comparison of backends
A bit of performance here and there
More coverage of the NumPy API, mainly. 🏃