Why RISC-V doesn't (yet) support KVM
Linux supports a number of virtualization mechanisms, but KVM is generally seen as the native solution. It provides a standard interface across systems, but much of KVM is necessarily architecture-specific, since the mechanisms for supporting virtualization vary from one processor to the next. Thus, architectures that support KVM generally have a kvm directory nestled in with the rest of the architecture-specific code.
Given that, some eyebrows were raised when Anup Patel's patch series adding RISC-V KVM support deposited the architecture-specific code into the staging directory instead. Staging is normally used for device drivers that do not meet the kernel's standards for code quality; if all goes well they are improved and eventually "graduate" out of the staging directory. It is not usually a place for architecture support. So staging maintainer Greg Kroah-Hartman was quick to ask why things were being done that way.
The answer comes down to the patch-acceptance policy for RISC-V code, found in Documentation/riscv/patch-acceptance.rst, which reads:
We'll only accept patches for new modules or extensions if the specifications for those modules or extensions are listed as being "Frozen" or "Ratified" by the RISC-V Foundation. (Developers may, of course, maintain their own Linux kernel trees that contain code for any draft extensions that they wish.)
Virtualization for RISC-V is described by the hypervisor extension specification. As Patel explained, that extension has not yet been approved:
The KVM RISC-V patches have been sitting on the lists for almost 2 years now. The requirements for freezing RISC-V H-extension (hypervisor extension) keeps changing and we are not clear when it will be frozen. In fact, quite a few people have already implemented RISC-V H-extension in hardware as well and KVM RISC-V works on real HW as well.Rationale of moving KVM RISC-V to drivers/staging is to continue KVM RISC-V development without breaking the Linux RISC-V patch acceptance policy until RISC-V H-extension is frozen.
It is fair to say that Kroah-Hartman was not
impressed; he stated that circumventing policies in other parts of the
kernel tree is not the purpose of the staging directory,
so the RISC-V KVM code would not be accepted there. Paolo Bonzini, the
overall maintainer for KVM, added
that "the RISC-V acceptance policy as is just doesn't work
",
as demonstrated by the fact that developers are having to work around it.
This is especially unfortunate here, he said, because the RISC-V KVM
implementation "is also a very good example of how to do things
right
". He went
on to suggest that there may be some players out there who benefit from
the slowing down of patches on their way into the kernel.
Kroah-Hartman responded that
slowing down the merging of useful code is "horrible
" and that
the job of the kernel community is to make hardware work; a policy that
prevents the merging of good code that adds support for existing hardware does
not make sense. He asked the RISC-V maintainers to explain this policy;
they have yet to answer that question. Back in April, though, RISC-V
maintainer Palmer Dabbelt acknowledged
that the acceptance policy is not producing the desired results:
My goal with the RISC-V stuff has always been getting us to a place where we have real shipping products running a software stack that is as close as possible to the upstream codebases. I see that as the only way to get the software stack to a point where it can be sustainably maintained. The "only frozen extensions" policy was meant to help this by steering vendors towards a common base we could support, but in practice it's just not working out.
He added that the policy could change, but only when there is a new policy for patch acceptance in place that everybody agrees to. Co-maintainer Paul Walmsley, though, placed the blame with RISC-V International's specification process and said that any fixes should be applied there instead.
It is not that hard to understand why the RISC-V maintainers might not want to attempt to support every nonstandard CPU that is out there. The open nature of RISC-V makes it relatively easy for anybody to create their own variant, and supporting them all could become unworkable. The counterpoint is, as Kroah-Hartman said, that the kernel's job is to run on the hardware that is out there. Blocking support for shipping systems can only have the effect of pushing those systems toward vendor-supplied kernels with a lot of out-of-tree code — an unfortunate outcome for what is supposed to be an open architecture.
When the subject is support for a feature as fundamental as virtualization,
the question becomes even more urgent.
Hopefully this episode will lead to a rethinking of the patch-acceptance
policies for the RISC-V architecture. Failing that, Kroah-Hartman has signaled his
willingness to allow that support into staging if there is truly no
alternative. So Linux seems likely to gain KVM support for RISC-V in the
relatively near future, even if it's necessary to circumvent the
architecture's maintainers to do it.
Index entries for this article | |
---|---|
Kernel | Architectures/RISC-V |
Kernel | KVM |
Kernel | Virtualization/KVM |
(Log in to post comments)
Why RISC-V doesn't (yet) support KVM
Posted May 20, 2021 18:29 UTC (Thu) by iabervon (subscriber, #722) [Link]
Why RISC-V doesn't (yet) support KVM
Posted May 20, 2021 21:02 UTC (Thu) by pbonzini (subscriber, #60935) [Link]
Why RISC-V doesn't (yet) support KVM
Posted May 20, 2021 21:36 UTC (Thu) by pdp7pdp7 (subscriber, #96517) [Link]
[1] https://lore.kernel.org/linux-riscv/daa30135-8757-8d33-a9...
Why RISC-V doesn't (yet) support KVM
Posted May 20, 2021 22:25 UTC (Thu) by pbonzini (subscriber, #60935) [Link]
1. Company X don't have hardware ready that implements an extension
2. Company X slows down ratification of the extension
3. Competitors must ship out of tree code to support the extension in Linux
The conflict of interest is outside Linux, which is what Paul Welmsley alluded to with respect to speeding up the process of ratification, but it affects Linux and the maintainers are caught in the crossfire.
Looking at the history of the hypervisor extension specification[1], the last significant change was done about one year ago (May 15, 2020). There is no justification at all for the specification not to be frozen yet.
[1] https://github.com/riscv/riscv-isa-manual/commits/master/...
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 0:09 UTC (Fri) by yootis (subscriber, #4762) [Link]
So which "company X" is slowing the system down now?
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 9:11 UTC (Fri) by farnz (subscriber, #17727) [Link]
My understanding is that the slowdown is not a specific company this time - it's getting the PLIC and CLIC specifications close enough to done that the RISC-V Foundation can be confident that the Hypervisor extension will not need to be modified to support direct delivery of interrupts to guest IRQ handlers without hypervisor involvement.
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 12:08 UTC (Fri) by pbonzini (subscriber, #60935) [Link]
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 1:45 UTC (Fri) by ncm (guest, #165) [Link]
To me the worst continuing omission is the B extension, and the worst missing bit of that is the POPCOUNT instruction, which really should be in with the base instructions alongside ADD and AND. I gather some lunatics are agitating to make it optional even in B, which is insane. POPCOUNT has been in or lately added to every production architecture ever, for reasons. (And, no, trap emulation would be much worse than useless.)
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 7:43 UTC (Fri) by pbonzini (subscriber, #60935) [Link]
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 10:54 UTC (Fri) by khim (subscriber, #9252) [Link]
> Are kernel maintainers prepared to support more than one version of RISC-V virtualization?Why would that be a problem? AMD and Intel use totally different schemes and they both are supported.
And I think being implemented in ASIC which is shipping should be considered “effectively frozen”. I can see how people may implement thousands of different things on FPGA. Supporting these would be a nightmare from practical POV. And FPGA implementation can definitely be changed thus it's not “effectively frozen” anyway. But if something is already in ASIC it couldn't ever be changed (on that specific ASIC) thus I think kernel should support it.
Kernel supports lots of devices (WiFi and others) which only support draft of some spec or another… why RISC-V extensions should be different?
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 15:11 UTC (Fri) by mss (subscriber, #138799) [Link]
It's a waste of effort to implement what's effectively the same thing two different ways.
And these aren't trivial things.
Why RISC-V doesn't (yet) support KVM
Posted May 23, 2021 7:38 UTC (Sun) by ssmith32 (subscriber, #72404) [Link]
Why RISC-V doesn't (yet) support KVM
Posted May 23, 2021 11:57 UTC (Sun) by mss (subscriber, #138799) [Link]
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 4:11 UTC (Fri) by pabs (subscriber, #43278) [Link]
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 4:43 UTC (Fri) by wsy (subscriber, #121706) [Link]
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 7:59 UTC (Fri) by rwmj (subscriber, #5474) [Link]
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 21:40 UTC (Fri) by rgmoore (✭ supporter ✭, #75) [Link]
Whether the people working on something are smart or not has nothing to do with whether or not it's being done by a committee. A committee of geniuses working in their element is still a committee. Committees can be a great way of doing things. There are any number of famous, successful, and greatly loved projects that were done by committee. Almost all of the popular translations of the Bible were done by committee, for instance.
I think that points to the big strength and the big danger of committees. A well run committee can get the minds of all its members working on a problem so they can do something none of them could manage alone. A poorly run committee puts the minds of its members in opposition, so the group gets less done than any of them could manage alone. The difference is whether the members can put aside their parochial interests to advance the good of the project. If they can, the committee can be a huge success. If they can't the committee will almost surely bog down into infighting. Vendors trying to game the system are a great example of the weakness of a committee structure; the members are so wrapped up by their own interests that the project as a whole suffers.
Why RISC-V doesn't (yet) support KVM
Posted May 22, 2021 10:41 UTC (Sat) by willy (subscriber, #9762) [Link]
Committees can be bad, but they can also be very, very good.
Why RISC-V doesn't (yet) support KVM
Posted May 23, 2021 7:40 UTC (Sun) by ssmith32 (subscriber, #72404) [Link]
K210 MMU support
Posted May 21, 2021 8:26 UTC (Fri) by geert (subscriber, #98403) [Link]
So can we get Canaan Kendryte K210 MMU support in drivers/staging/, too? ;-)
Why RISC-V doesn't (yet) support KVM
Posted May 21, 2021 18:54 UTC (Fri) by pwsan (subscriber, #56604) [Link]
> The counterpoint is, as Kroah-Hartman said, that the kernel's job is to run on the hardware that is out there. Blocking support for shipping systems can only have the effect of pushing those systems toward vendor-supplied kernels with a lot of out-of-tree code — an unfortunate outcome for what is supposed to be an open architecture.
> When the subject is support for a feature as fundamental as virtualization, the question becomes even more urgent. Hopefully this episode will lead to a rethinking of the patch-acceptance policies for the RISC-V architecture.
One of the issues with the proposal to merge KVM support now for RISC-V is that no one has yet been able to identify any shipping systems that support the RISC-V hypervisor extension. What's being proposed, instead, is that we merge support for a hypervisor extension that isn't yet part of the RISC-V ISA. While we certainly understand Anup's and Paolo's desire to get their code merged upstream as soon as possible, I'm afraid these important points have been lost, both in the article and in most of the discussion.
Why RISC-V doesn't (yet) support KVM
Posted May 22, 2021 8:19 UTC (Sat) by pbonzini (subscriber, #60935) [Link]
Why RISC-V doesn't (yet) support KVM
Posted May 22, 2021 16:37 UTC (Sat) by mfuzzey (subscriber, #57966) [Link]
Presumably those developing this feature are testing it on something.
If the chips exist just no one is selling systems with them yet (presumably the developers having custom prototype boards) then I see no reason not to merge the feature as the chips aren't going to change.
OTOH if no chips exist yet (eg if prototyping is being done on a FPGA) I can see the rational for waiting if there is a real risk that the final hardware is different.
But even in this case does it really matter given that the user space won't be affected? Sure the kernel code may have to change for the final hardware but if the prototype hardware never shipped does this matter as there will be no need to support both in the kernel.
I may be missing something obvious here as I'm not well versed in RISC V
Why RISC-V doesn't (yet) support KVM
Posted May 23, 2021 7:44 UTC (Sun) by ssmith32 (subscriber, #72404) [Link]
"In fact, quite a few people have already implemented RISC-V H-extension in hardware as well and KVM RISC-V works on real HW as well. "
And which sounds like the main reason it may end up being merged into stg.
Why RISC-V doesn't (yet) support KVM
Posted May 26, 2021 0:29 UTC (Wed) by pwsan (subscriber, #56604) [Link]
Probably most people are using QEMU to experiment with this, with the instructions and code here: