A Nouveau graphics driver update

The kernel community's relationship with NVIDIA "has gone up and down" over the years, Airlie began. Recently, though, the company has rearchitected its products, adding a large RISC-V processor (the GPU system processor, or GSP) and moving much of the functionality once handled by drivers into the GSP firmware. The company allows that firmware to be used by Linux and shipped by distributors. This arrangement brings a number of advantages; for example, it is now possible for the kernel to do reclocking of NVIDIA GPUs, running them at full speed just like the proprietary drivers can. It is, he said, a big improvement over the Nouveau-only firmware that was provided previously.

There are a number of disadvantages too, though. The firmware provides no stable ABI, and a lot of the calls it provides are not documented. The firmware files themselves are large, in the range of 20-30MB, and two of them are required for any given device. That significantly bloats a system's /boot directory and initramfs image (which must provide every version of the firmware that the kernel might need), and forces the Nouveau developers to be strict and careful about picking up firmware updates.

Nouveau work has taken a bit of a setback since longtime developer Ben Skeggs left the project, but he did manage to do a lot of refactoring before he went. Nouveau now has initial GSP support for one firmware version; that code was merged in for the 6.7-rc1 release. It is only enabled for the Ada series of GPUs by default; with a command-line argument it can be made to work with Turing and Ampere devices as well. It is missing some features, including fault handling (which "shouldn't be too hard" to add) and sensor monitoring, which doesn't work at all.

NVIDIA's firmware, Airlie said, comes with a set of include files that, in turn, define structures that change over time. To deal with these changes, the driver is going to need some sort of automated ABI generation; he noted that the developers working on the Apple M1 GPU driver have run into the same problem. This problem could be made easier to tackle, he suggested, if the driver were, like the M1 driver, to be rewritten in Rust.

Next steps

Supporting the GSP firmware is just the beginning, though; at this point, Airlie took a step back and talked about the task of making a useful GPU driver in general. Years ago, a graphics card came with some video RAM and a graphics translation table (GTT). The driver would map system memory into the graphics card; user space could then submit buffer handles that would be relocated for the graphics device. This approach works, he said, but it is slow.

Current GPUs have full virtual memory, instead, which saves a lot of that overhead. The kernel has grown a number of subsystems for working with this virtual memory, including the graphics execution manager (GEM) for buffer-object management, the translation table manager (TTM) for discrete video-RAM buffer-object management, and a bunch of synchronization and fencing code. Initially, the DRM subsystem would tie the allocation of a buffer to an allocation of virtual memory at the same time; that was easy to do and sufficed to implement OpenGL. But, he said, the graphics world moved on from there.

Specifically, Vulkan came along. It brought the concept of sparse memory and, with it, virtual memory that is managed by user space. Vulkan can handle both synchronous and asynchronous virtual-area updates, but it "gets complicated". Various drivers started inventing their own virtual-area management; as a way of bringing that work back together, the VM_BIND API was developed.

This is consistent with a recurring pattern, Airlie said. The DRM developers work to share common code between graphics drivers, but the driver developers keep trying to reinvent wheels, a tendency that has to be resisted. The subsystem did well with regard to mode setting, he said, but less well on the acceleration side; there is a "common GPU scheduler" that is only used by one driver, for example. Similarly, there are a lot of drivers implementing VM_BIND by doing their own virtual-area management.

In response, Airlie came up with the "good idea" of getting somebody else to write a common virtual-area manager, called GPUVM, inspired by the amdgpu code. It is intended to be useful for all drivers; it is used by the Nouveau, Xe (Intel's new driver), and Panfrost drivers now. Hopefully the amdgpu and MSM drivers will pick it up as well. The best part is that there are multiple developers who understand it and can help to keep it from going off in the wrong direction. GPUVM has been through a lot of iterations, he said, providing "lots of learning experiences".

As an example, he talked about the problem of fence signaling. A fence indicates when a series of GPU operations has been completed; waits for these fences have to be time-bounded, or the memory-management subsystem might deadlock. In short, a GPU can easily pin down all of a system's RAM if given the opportunity. There is a shrinker that can be called when memory gets tight, but it will have to wait for fences to be signaled to know when memory can be freed. If the code that set the fence decides to allocate more memory while this is happening, a deadlock results. To avoid this outcome, developers have to strictly limit the operations that can be performed in fence-signaling critical sections; care must also be taken before acquiring any locks. It would be nice to be able to update the page tables during this code, but that ran into deadlock problems and had to be backed out.

Returning to Nouveau, Airlie said that the initial VM_BIND API, using GPUVM, synchronous objects, and integration with the scheduler, was merged for the 6.6 release. There are a lot of improvements in the works that should land in 6.8. At this point, he said, we have the core of a modern GPU driver for NVIDIA hardware — for graphics, at least. More work will be required before Nouveau can support compute applications.

On the user-space side, Faith Ekstrand has been developing the NVK Vulkan driver for Nouveau; this driver recently reached Vulkan 1.0 conformance. This work involved creating a new compiler, called NAK, that has just been merged into Mesa; this compiler yields far better performance (from 20 frames per second to over 1000) than the old "codegen" compiler did. Naturally, this compiler is written in Rust. The next step, Airlie concluded, is to move forward to Vulkan 1.3.

Video and slides from the talk are available.

[Thanks to the Linux Foundation, LWN's travel sponsor, for supporting our travel to this event.]