Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm opposed to this, and it's not because of some idiologic kernel-should-be-pure-c thing. I'm opposed to it because the rust compiler is slow. And the rust compiler is written in rust. Compiling rust is a nightmare if you don't have a high-end PC.

I want to be able to actually compile my software if I wish so. This is becoming increasingly difficult. Rust is adding to the problem.



I'm not a fan of rust either, not by a long mile. But current kernel approach to memory safety is a complete, utter, demonstrated failure. Look at last week's CVE-2022-41674:

https://seclists.org/oss-sec/2022/q4/23

This is a catastrophic bug, that (after some work on developing an actual RCE) lets anybody within wifi range to get root on your laptop (or phone, or access point). And all it took is this one line:

https://git.kernel.org/pub/scm/linux/kernel/git/wireless/wir...

We've all been repeating the "1000 eyes - all bugs are shallow" mantra for far too long. This one was in the mainline for more than 3 years, and nobody noticed. How many more are lurking there?


Separately I feel "with enough eyes, all bugs are shallow" fails to apply in codebases with many users but most of whom consume it as a black box without looking inside. Some issues making it harder for me to read the code I use:

- The Linux kernel and issuance .so files lack a "view source" button on compiled binaries. And even checking out the matching source, building a replacement binary, and diffing your local changes from the matching source is an arduous progress to setup per program/library from a tarball/Git tag, wait for the computer to finish, install dependency .so files globally, ensure symbols are present, ensure you can breakpoint static functions...

- Dynamic dispatch and generic code might help maintainers and code extensibility but (in my experience) definitely impede external eyeballs from understanding code.


Yes! That's why I adore the way plan 9 has the whole source tree in /sys/src. Everything is there.

To a smaller extent, netbsd has pkgsrc, and most BSD friends have something like that too.


s/issuance/userspace, s/computer/compiler


Does Rust help solve overflow errors? Don't you need to explicitly used "saturated_add" or "checked_add"?


Rust doesn't directly solve overflow errors. What Rust does do is turn them into logic errors, rather than memory issues. To break it down:

* Overflow with the default operators is not UB in Rust, it will either panic or two's compliment wrap, depending on various things (including things you can set to choose this global behavior). This already prevents various issues.

* You can also explicitly choose to do various operations with whatever overflow semantics you want, as you mention with saturated_add and friends.

* Because indexing is bounds checked, where in some languages an overflowed integer would lead to incorrect indexing and therefore possible memory problems, you'll either get a panic or a logic bug, not a memory bug.

* If your integer isn't being used for indexing, you'll end up with some sort of error, but again, at worst a logical error, not a memory error.


is this really that bad of a bug though? i need to be using that driver, and someone near me has to be actively knowing and trying to inject... and it's a DoS ? i'm sincerely asking btw


Driver in question is cfg80211. Majority of popular wireless cards use it.

DoS is the only publicly known exploit, right now. There is understanding that RCE is possible with additional specialist work.

Local packet injection is an implementation detail of the first public PoC. There are other implementations of the same exploit, that deliver packets over the air instead. Including one that runs on ESP32, and attacks unmodified Linux nearby:

https://github.com/jo-m/linux-wifi-ota-crash


> and someone near me has to be actively knowing and trying to inject

Just leave a small board, a battery, and a solar panel on a roof or tree near your office. If RCE is possible with this (people think it is), this would be very valuable.

On a tree near my office, you'd gain access to our network, Microsoft's and a European central bank. Not bad at all.


Confused by that bug. Adding 2 to a U8 causing overflow makes it smaller, not larger. It will copy too little? not too much.


How would Rust fix this? Here's a Rust program that overflows a u8, with no compile error or warning. https://godbolt.org/z/ME3e9KeMe

You can turn on runtime overflow checking with the Rust compiler, but you can do that with gcc compiling C too.

I guess the answer is that you wouldn't do the memcpy like that in idiomatic Rust - you'd use some higher level construct that gives the compiler more chance to catch your errors. Could anyone comment on how this works in a case like this?

edit: Can't reply to repliers. Defined overflow doesn't help here. A checked_add() function could be used in the Linux kernel in C, just as easily as in Rust. Forcing checked_add() to be used in the Rust compiler would help, but would also have performance and readability impact, which is presumably why it is still being debated.


> How would Rust fix this? [...] I guess the answer is that you wouldn't do the memcpy like that in idiomatic Rust - you'd use some higher level construct that gives the compiler more chance to catch your errors.

Yes, that's the trick: it will be caught not in the u8 overflow, but either in the memcpy equivalent (which is "dst.copy_from_slice(src)"), or in the slice manipulation before it. What happens is that slices in Rust are represented by a "fat pointer", a pair of the starting address and the length, and both the "copy_from_slice" method and the index/index_mut operator check the bounds before doing the operation.

(You could do things using "unsafe" and raw pointers, which have only the starting address without the length, but in idiomatic Rust you'd use slices most of the time.)


In C when you use an inappropriate type (such as u8 here for a size) it's just coerced. In Rust the wrong type doesn't compile. So, I think the programmer is less likely to choose u8 in the first place.


I thought the idea was that Rust’s behavior in cases of overflow was defined, whereas c/c++ is not defined by the language.


The C language says the unsigned integers wrap. That is, C's 8-bit unsigned integer (e.g. uint8_t or in Linux source u8) behaves the same as Rust's Wrapping<u8> type.

In Rust the u8 type wraps in default release builds, but panics on overflow in debug builds, but this is kernel code so it will definitely be built in release mode with wrapping.


That is because you're using a normal add instead of `checked_add`. There are discussions about forcing the compiler to only allow `checked_x` and `wrapping_x` operations for any mathematical operation within your code.


"The subsequent memcpy" is this:

  memcpy(pos, mbssid + cpy_len, ((ie + ielen) - (mbssid + cpy_len)));


> (after some work on developing an actual RCE)

While I agree that bug is serious, that "some" is doing pretty heavy lifting here. Is there an RCE for this bug?


There are almost certainly some private ones. This is an extremely high impact bug.


I'm not sure it's realistic to expect a safety focused compiler to compete with one that doesn't offer those checks.

We should aspire to make it as fast but in the short term a slower compiler in exchange for less CVEs and random buffer overrun crashes seems like a reasonable trade off to me.

Distributions such as Fedora offer build infrastructure[1] that you can use to compile packages to use in your system for testing if you feel your local hardware isn't powerful enough.

[1]: https://copr.fedorainfracloud.org


What's the old adage? It's easier to make a correct thing fast than it is to make a fast thing correct.


Actually C/C++ compilers are shooting themselves in the foot performance-wise with the header mechanism, so writing a compiler that's faster than C/C++ is not that hard. Not sure what the reasons of the Rust compiler being (even more?) slow are, maybe the ability to easily interface with C code has something to do with it?


Last time I checked [1], the Rust part of the Rust compiler is not the slowest. LLVM spends a lot of time processing relatively poor IR code which the Rust compiler emits.

Rust compilation is not intrinsically slow; better IR and more incremental compilation can and will improve it. Incremental compilation (only changed files, not crates) is 7 years in making though.

[1]: https://prev.rust-lang.org/en-US/faq.html#why-is-rustc-slow


Thanks for that link! That sounds like the really major factor is the decision to recompile the entire crate when a file in the crate changes. I'm sure that made sense at the time ;) And, if slow compilation is such a pain point, I wonder why it takes seven years and counting to fix this bad decision - incremental compilation might have its edge cases and pitfalls, but other compilers are doing it just fine, so it shouldn't be rocket science?

Other reasons:

- suboptimal LLVM IR code, as you mentioned, and other tech debt

- the preferred strategy of monomorphising generics is fast at runtime, but slow to compile, using trait objects is faster

- the complex type system of course also plays a part

- LLVM itself is of course not optimized for Rust


> if slow compilation is such a pain point, I wonder why it takes seven years and counting to fix this bad decision

I'm not sure it was a "bad decision" if it was something that can be addressed after the fact and allowed them to ship working software more quickly.

The vast majority of the popular projects I am aware of have this kind of technical debt, likely due to survivorship bias. Project teams that refuse to take on technical debt are rarely successful enough to become popular.


> if slow compilation is such a pain point, I wonder why it takes seven years and counting to fix this bad decision

Maybe we are giving Rust developers computers that are too fast. ;-)

OTOH, it's always wise to design for the future, so it also makes sense to give Rust developers beefy server grade hardware so they can play with SIMD pipelines and all those extra cores and threads, because that's what workstations will be a couple years from now.


In this case I really have to nitpick about the use of "C/C++": This specific problem really only exists in C++, because there it is common to put implementation code into headers via inline and template functions (worst example is the C++ stdlib), while in C you only put declarations into headers, and just parsing a few hundred lines of struct and function declarations won't blow up your build times (while just including a common C++ stdlib header like <vector> pulls in tens of thousands of lines of tricky template code.


In VC++ "import std", which imports the whole C++ standard library, is faster than just doing "#include <iostreams>".

It is only a matter of time, until GCC and clang finally catch up.


According to Wikipedia, both have support for precompiled headers: https://en.m.wikipedia.org/wiki/Precompiled_header

Shouldn't this be enough? Or does it require explicitly including the precompiled header whereas VC++ has magic to make that transparent?


C++20 modules and pre-compiler headers aren't the same thing in terms of technology.


There are languages with similar type safety, which are faster, because the authors have placed focus on having several ways available to compile the code.

Ada, Delphi, OCaml, C#/F# (.NET Native / Native AOT), D, Nim,...


I will admit I never understood why Ada never took off (is it only because of the tooling being propertiary for a long time?), but c# is not as memory safe as rust.


The only memory safety that C# lacks in comparisaion with Rust is one special case, data races for in-process data.

For everything else regarding concurrent access to shared data out of process, they are on the same foot.

Ada didn't took off, because of several reasons, price of compilers, on UNIX it was an additional SKU on top of the respective developers SDK that already offered C and C++ in the box, 1980's mainstream hardware wasn't able to cope with it, most OS vendors outside UNIX decide to migrate from their toolchains into C and C++, so again additional money on top of the OS SDK.


>> I will admit I never understood why Ada never took off (is it only because of the tooling being propertiary for a long time?)

Early Ada compilers had expensive licensing and required expensive hardware to run.

By the time GNAT was added to GCC, C++ had already taken over most of the spaces that were not Ada exclusive (meaning safety critical / defense / aerospace niches where either Ada was once mandated or has thrived in despite the original high costs).


Does C# not have a GC? I thought that would immediately disqualify it for kernel work.


It does have GC but that is not the point being made.

I believe the point the author is making is that other languages provide better safety than C and have faster compile times than Rust, therefore Rust should be able to improve its compile times.


If those languages achieve their memory safety at runtime, like C# does with GC, then that becomes relevant to the point of the compile time performance. The C# compiler has to do less as that complexity has been pushed elsewhere with different tradeoffs.


Rust safety with runtime dependencies, Vec bounds checking, Arc and Rc datatypes.

Ada with SPARK 2014 formal proofs, for your other part of the remark.


GC is not a roadblock for OS kernel work. Smalltalk is itself an OS and had GC from the start. IBM's i (the descendant of OS/400) most likely has GC as part of its kernel. Lisp machines had hardware-assisted GC.

What you may want to try to avoid is complex and non-deterministic GC, which makes it harder to reason about.



For Linux yes, for those that aren't blinded by anti-GC religion, no.

https://www.wildernesslabs.co/


Anti GC isn't a religion. GC is a performance cliff in some cases.


It is certainly a religion, as most people that advocate against it hardly ever learned to use a profiler, or bothered to learn that not all GC languages are alike, and many of them offer the same capabilities like C and C++ for low level coding.


It's not, collaborating on a C# game engine and what I've seen is basically a lot of skirting around GC, because it's impact shows up in profiler. It has gotten to a point arrays and stack alloc are prefered over List and HashSet were basically forbidden in the hottest path.

Except most of C# ecosystem relies on classes and GC. A lot of these problems are caused by overuse of GC-ed classes, and their ease of use.


Don't confuse Unity runtime with C#, specially since it lacks all the nice features post C# 7.

C and C++ aren't immune to heap abuse as well.

You do on C# just like on them, think about data structures, and avoid heap during render loop


Did I mention Unity? It was a independently developed engine. The engine is in C# 10.

They still ran into limitations where basically "This would be a lot easier if we didn't have GC".


My take on this is a little different than the comments previous. I tend to side with your position over the default position the GC is workable as a languages base assumption. I think the problem really is an expressiveness issue. In a language conceptualized to have memory managed either manually or GC’d and that said choice should be easily made (i.e. it should not take much work to designate some code path as utilizing a GC’d strategy), but I would say for implementation and performance you would give up being able to easily abstract over the memory strategy.

tr;dl — I’d really prefer the option to determine when I’d like GC, as opposed to dodging the collector to avoid performance hits, hot code can default to manual. Probably not a realistic ask, but it could work.


> I’d really prefer the option to determine when I’d like GC, as opposed to dodging the collector to avoid performance hits

Fair enough. If SS14 used D maybe they wouldn't have these problems.

D and early Rust (pre 0.2, like alpha alpha) had that. Problem is you split your community in two. You get a version of "What GC-color is your function/lib?".

It's a tradeoff for some domains - allow no GC bypass you're going to run into nigh insurmountable performance cliff.

Allow GC as opt-in and you run into issue of splitting your APIs in two.


Well that is what everyone is actually using nowadays when they state they are doing a game in C#.

"This would be a lot easier if we didn't have malloc()/free()", basically.

In any case Sony, Nintendo and Microsoft have been doing graphics stuff with C# for their platforms, although they could have kept being pure C++.

Also we shouldn't silo ourselves into only C# as discussion point, Go, D, Nim, Swift, Eiffel are also possible examples.


> "This would be a lot easier if we didn't have malloc()/free()", basically.

Well, not really. It's just some libraries like YamlDotNet copy waaaay more than needed and it shows in serialization. They mostly minimized calling YamlDotNet, but true solution would be a zero copy parser.

Other issue was HashSet operation like Clear had huge impact. Think they replaced those with arrays.

Third issue was something about flecs and archetype ECS. I don't know if it was a jest, but they mentioned changing GC layout or rewriting GC.

Points is, they now face a steep performance cliff. The only way out of it is through sheer effort.

So no, it's not anti-GC religion. Some domains and GC really badly mix.


I think it's more related to the grammatical complexity of the language than to safety itself

(Same reason why C with typedefs is slower to compile than plain C, why C++ is slower, etc)

(that and cargo dependencies, etc - also C compilers have some +30yrs of optimizations)


I've seen some Rust programs that look nice, but in the kernel, the code ends up making perl'ls propensity for line noise look competitive. I'm sure some of that is the need to shovel between languages, but it does make you wonder if the entire effort is a little misguided.

A full Rust kernel might be neat. A compromised Rust driver inside of a C kernel is never going to be the default choice.


Most of the code committed to the kernel is infrastructure for C<->Rust interoperability. It's the least readable of it all, and as fun to work on as watching paint dry.

The actual driver code is pretty readable, see https://lwn.net/Articles/863459/.


>A full Rust kernel might be neat.

May I interest you in Redox? https://www.redox-os.org/


If "embedded OS" counts as "full Rust kernel", you may want to check out https://hubris.oxide.computer/

Obviously very different than Linux.


As others have pointed out most of the performance hit is in the optimization of the relatively verbose IR code the Rust compiler sends to LLVM, not the Rust compiler itself.

There's been multiple passes at making the Rust compiler faster. It'll happen. There's a GCC-based Rust implementation in the works too.


More complexe languages are faster to compile. The issue is mostly the implementation. It could, should and will be better but it takes time.


Nim is also memory safe and the compilation is really fast.


> No, Nim doesn't have stronger or weaker safety guarantees than Rust [1]. Rust's memory safety is nothing new, either. It is mostly that some older languages like C/C++ are the exceptions in not being memory-safe. There is nothing new or magical about memory safety. LISP was already memory-safe when it was invented in 1958. The only question is how much performance you need to trade away for it (the value is never zero for non-trivial programs, but can vary greatly, depending on whether the language was designed with it in mind or not).

> The main difference between Rust and other languages is that it does some more (but not all [2]) safety checks at compile time rather than at runtime. It also allows you to avoid GC, but does not provide you any memory safety over GC. Rust's borrow checker allows you to statically prove that references are live [3]; a GC simply avoids deallocating any memory that has a live reference to it (on the other hand, a GC can ensure that references remain live even where this is hard or impossible to prove statically). The end result is the same with respect to memory safety (the reason some people want to avoid GC is for performance reasons, not memory safety).

https://forum.nim-lang.org/t/1961


While it's true that memory safety has been done in a lot of languages for a long time this overstates the point. Doing memory safety at compile time is an important change that allows replacing C/C++ in a lot more situations. In the benchmarks game for example Rust has been the only language capable of breaking into the C/C++ league, even after very many years of investment into Java for example.

Rust also uses the same mechanisms to get compile time thread safety with full memory sharing between threads. Does any other language that doesn't have a global lock, throwing away most of the advantage, have that? There are actual new and interesting advantages to the compile time ownership model.


Benchmarks vary wildly but Haskell tends to perform in the same realm as C++, fwiw. It's GC'd, uses immutable data structures, and uses a green threading model that can be easily exploited by user code for extreme levels of parallelism with full memory safety. It's even possible to hold mutable references and share them across threads safely with STM (not a global lock).

If you want fully lock-free it might be possible to prove that with extensions like Liquid Haskell, via Linear types, and is definitely easier to prove with a theorem prover than it is, generally, for C++ code. Not sure about Rust though I realize quite a lot of its moving parts have already been formalized which is super cool.

There are reasons for using Rust but it's not the only game in town. And GC doesn't automatically mean pessimistic performance.


> Rust also uses the same mechanisms to get compile time thread safety with full memory sharing between threads.

A minor advantage on the age of microservices and OS IPC to shared external resources.

Sendable doesn't apply when those threads are accessing shared external resources.


That's outdated information from 2016. Now Nim uses ARC/ORC. ARC implements memory management at compile time like Rust and it's still very fast at compiling.


Cool, I wasn't aware, thanks. https://nim-lang.org/blog/2020/10/15/introduction-to-arc-orc...

> The main difference between ARC and Nim GCs is that ARC is fully deterministic - the compiler automatically injects destructors when it deems that some variable (a string, sequence, reference, or something else) is no longer needed. In this sense, it’s similar to C++ with its destructors (RAII). To illustrate, we can use Nim’s expandArc introspection (will be available in Nim 1.4).

> This shows one of the main ARC features: scope-based memory management. A scope is a separate region of code in the program. Scope-based MM means that the compiler will automatically insert destructor calls for any variables which need a destructor after the scope ends. Many Nim constructs introduce new scopes: procs, funcs, converters, methods, block statements and expressions, for and while loops, etc.

> ARC also has so-called hooks - special procedures that can be defined for types to override the default compiler behaviour when destroying/moving/copying the variable. These are particularly useful when you want to make custom semantics for your types, deal with low-level operations involving pointers, or do FFI.


Rust doesn't have ARC like Nim or Swift. (And Nim and Swift's are a bit different here but closer to each other than anything that exists in Rust.)


Really. I did not know that. so, can I use today Nim but with a Rust-like ARC?


--gc:arc/--mm:arc has been around for a couple years (as per that blog post link) and fairly stable for >1 year. It is slated to become the default automatic memory management strategy in Nim-2.0. The hope is to release 2.0 this year.

You can always make it your own personal default with older versions (e.g. nim-1.6), by editing your $HOME/.config/nim/nim.cfg to say so or doing similar on per project/file basis.


Anything you can statically prove at compile time is interest you don't have to pay at run time.

Rust is a huge, huge win for that reason. It allows for code written in a Lisp/OCaml/Nim style that's memory safe without any additional overhead.


Not on my laptop it wasn't. I wonder if someone's done a performance comparison.


Points from the flip side:

- the Linux project is very likely to stick to simple, fast elements of Rust (based on the excellent approach of the Linux/Rust devs thus far)

- the more Rust is used, the more work will be done to improve its performance

- you can still build a kernel on a low-powered device... i've built kernels that took > 12 hours on, for example, PA-RISC boxes that were once regarded as beefy :-)

- most people don't (and shouldn't) compile their kernel, and by most I mean more than 99%


Could you expand on why people shouldn't compile their kernel? I think it's fairly useful to compile their own to get a better understanding of what the kernel does and to better suit everyone's needs. For example, if I have little free space on my boot partition and I have my disk encrypted, I want my kernel to be as small as possibile, so I will deselect every driver I don't need. Or maybe the driver for my new device is not included in the kernel builds of my distribution.

Not only I woundn't say that most people shouldn't compile their kernel, I would say that most linux users* should do it at least once, so they can understand the power they have compared to closed-source operaring systems.

*with linux users I mean users that use linux as their main operating system, not people that do ssh once in a while or rarely boots their linux partition


Lots of reasons:

- everyone has better things to do than compile software they didn't write

- a good distro has probably tested it on a bunch of hardware, and hopefully signed it (or at least the packaging), so you know it was securely acquired and built

- you won't learn much at all about the Linux kernel by compiling it... you may learn a tiny introductory about about it by configuring it, but that's still not very much at all, really (it may seem like a lot when you don't know how to measure what you're learning)

- what you should learn from configuring and compiling a Linux kernel is that you don't ever want to be in a situation where you have to do it again (without a really spectacular reason, or being paid)

- if you're compiling a kernel because your boot partition is small... make it bigger, or don't have one at all. come on.


using the same binaries shares the verification for said binary.

a modular kernel with a custom inird (generated by the distro) is small enough for most.

so if you are into adventures or you are in the business of kernel development yes roll your own. anybody else is better served standing on the shoulders of a maintained binary distribution.


It will make less people compile their kernel, further concentrating decision-making into the hands of distributors.

With every year, abandoning OSS and looking for non-computing hobbies gets more attractive.


All that is fair. It's just, maybe a bit early? I don't usually say this, but I would've appreciated if it came a little later.

All I really want is to be able to compile my stuff without waiting overnight (or more).


It almost certainly won't be relevant to you, as a Linux kernel user, for a while anyway. So, it will be a little later.


I used to work on the Rust compiler itself on a Chromebook with a 1.1GHz dual core, 4G RAM and 32G of disk. That's about as far from a high-end PC you can get. Most mid-range phones nowadays have more processing power and memory than that. And the Rust compiler has been sped up considerably since then. Even with a 4 year old mid-tier PC you can get a complete Rust compile in half an hour. Roughly half of that is building LLVM.

So you can of course compile your Rust compiler. If you are used to compile clang or gcc, it's not that much of a hassle. And the benefits have already been shown. If you only want to compile Rust code, and not develop it, mrustc might also be a good choice for you (it doesn't implement borrowck, just what's needed for codegen).

Finally, if you don't want to use Rust drivers, you can simply configure them out and don't need to build Rust. It'll be quite a long while until Rust will arrive in the kernel outside of drivers (which tend to benefit most from Rust anyway).


BTW, people assume it's slow because of the safety checks, but that's not the case. `cargo check` runs just the checks, and is pretty fast.

Majority of the time is spent in LLVM, because rustc throws a ton of code at it to clean up. This is being addressed by MIR optimizations (rustc's built-in optimizer working on higher-level code) to remove costly abstractions before they become a pile of low-level code to eliminate.


I remember the times when installing Gentoo was a matter of days. Compiling the kernel was a matter of hours. Sure, rustc is slower than gcc, but if you're not constantly compiling your software over and over and over again, then the time spent in the compiler is not your primary concern. Start the compile, go to bed, let it run.


Gentoo.. the only reason I'd ever know or need to know that 'distcc' is a thing that exists.


Ha! I was a wimp and did a "stage 3" install back in the day. Or was it stage 2? (shrug). Now I'm even wimpier and just use Fedora.

Came across this page, on the motivations of Gentoo, a few years later:

> Installing a working Linux box used to require over 550 man hours, learning a Nordic language, sacrificing a goat, wading through hundreds of pages of (purposely) inscrutable help files...Old-school Linux users were desperate to find a new way to feel superior.

https://en.uncyclopedia.co/wiki/Gentoo


It honestly wasn't bad at all. Daniel Robbins documentation for Gentoo is still one of the best doc I've ever read and I'm sure he inspired many others. IMO It's actually the greatest thing about Gentoo (along with Portage).


Running Gentoo on 9900KS and 980 Pro SSD with 64GB of memory, kernel compiles in about 8 minutes, the longest project to build I've seen is Chromium and QT Web Engine at ~120m. It's OK but I'm tempted by the new Ryzen CPUs.


Now run the same thing on a first gen Celeron.


I remember spending days to compile KDE fbsd port on Pentium 4, nobody sane should be running Gentoo or any from source rolling release OS on sub-optimal hardware.


Celeron was about the affordable option when Gentoo was released. It was just that: compiling things took time. But even when you ran redhat or suse, you sometimes needed to recompile the kernel to support the features you needed. And that took hours. But it was never a major impediment to using Linux. It was a one-time cost, like a windows install (that also took hours), or a windows update.


It's a fair remark, but your requirement is very niche. There's very few people who will value compilation time over runtime speed, safety, good abstractions, expressiveness.


This ethos played out in the C++ community 5-ish years ago. It turned into a compiler with quadratic behavior, and eventually many people who didn't care at all about compiler performance cared a lot.

It turns out that compiler speed makes development faster, keeps people interested in the language, and ultimately allows more iterations before release (which can be better for speed and safety than throwing in a bunch of extra compile steps).


You misread my comment. I didn't say it was useless, I said there are things with higher priority. And well… unlike Rust, C++ sucked at some of those things too, this I think makes them different.

Rust offers a better trade-off between compilation time and other parameters, than C++.


They didn't say it was useless either; they also said it wasn't a high priority. Developer velocity and ergonomics should always be a high priority for compiler writers.


"High" as in "higher than runtime speed, safety, abstractions, plugin system"? Where would you put it if you had to rank compilation speed with other criteria?

And let's admit even though Rust compilation speed could use some improvement, it's not terrible.


I re-read it. Yes, they're not saying it's useless, but then what are they objecting to?


Rust could have been designed for faster compilation without sacrificing runtime speed, safety, good abstractions, expressiveness and whatnot in any regard. The people designing Rust just either did not care or did not have the experience to do so. And it can not be fixed anymore because that would require too much breaking changes.

And yes, a substantial time is also spent for things like the borrow checker but no all of it.

An no, it is not a niche requirement. Short compile times are absolutely critical for developer productivity. One main reason Golang exists and got popular is that people got fed up with how slow C++ is to compile. Not to mention that most people on this earth are not as privileged as to have a beefy machine.


> Rust could have been designed for faster compilation without sacrificing runtime speed, safety, good abstractions, expressiveness

Can you back this up? I'm not a programming language expert, so if there's some common knowledge you're referring to, I'm not aware of it.

> most people on this earth are not as privileged

There's very little "privilege" you need to compile Rust. In my spare time, I develop mainly in Haskell on my ThinkPad X270 and it goes just fine. Building libraries takes time, yes, but you need to do it just once. And while it's building you can think with a piece of paper more — also privilege in a way.


cargo check is pretty fast.


> I'm opposed to this. I want to be able to actually compile my software if I wish so.

So what you're saying is that you're opposed to millions of people having more secure software, and perhaps millions of dollars spared from breaches, because it makes your own occasional singular personal experience of compiling the software faster?

Or did you mean something else?


I mean, slow compile times affect everyone who compiles the kernel. And being able to compile things yourself (possibly with patches) is one of the key features of open source...


Considering only compile time is a shallow approach to the idea of using Rust more widely. I would encourage you to think about the aggregate amount of time our industry spends finding, and then fixing, and then repairing the damage done by, classes of bugs which idiomatic Rust completely prevents. It does all this without impacting runtime (like GC language often do).

Fast compute at this point is quite literally the least expensive part of the equation. Machines will get faster. Compilers will get optimized.

We've spent decades optimizing the developer experience (compile times) at the expense of the rigor, robustness, and quality of our resulting product. I've been doing this for 30 years, and I can categorically say that I've spent FAR more time chasing NPE, OBO, and race condition bugs than I would have ever added to my build time with a slightly slower compiler.

itripn&


I wonder if a solution to your use case could look something like mrustc: https://github.com/thepowersgang/mrustc

The idea here would be to compile in a way that just assumes everything is correct and either crashes catastrophically or produces invalid output otherwise. But in doing so, it should allow at least slightly faster compilation.

You could even remove the need for this "fast and loose" compiler to do any type inference by shipping pre-processed source with all types resolved. However I don't know if this addition would fit your needs if, e.g., your goal is to be able to compile from any given commit rather than only official releases.

At very least it could be an interesting experiment to discover what tradeoffs are possible.


Seeing this as the top issue for Rust on linux in this thread, I can see the future is bright for Rust and Linux.


Is this sarcasm?


No. If the most pressing issue found is that it compiles slower on old computers, then there is an absence of real issues.


While I lack personal experience with Rust and I really appreciate fast compilers (that is why I am a Go user), over all features and characteristics, Rust seems to be the best choice for safe kernel development. Other posters have described well how urgent it is, to improve the security of kernel code. So just not doing anything about this, doesn't seem to be a good option.

It seems, there is a wide group of developers which thinks that Rust is the best candidate as a kernel development language. If you see issues with that choice, now would be the time to propose an alternative and try to find momentum in the developer community supporting that alternative. While I also lack practical experience there, by all what I heard, ADA could be one. But I don't know how it exactly compares to Rust and what the trade offs are. But so far, no one has pushed for ADA as a possible kernel implementation language.


gccrs is entering the scene with GCC-13. We'll see how it turns out. It'll become part of the GCC toolchain, so everything can use the same pipe.


From a bootstrapping perspective, GCC is not perfect, either, because of the C++ addition in 5.0 (at least I think that's the version.)

So to bootstrap rust this way, you'd need to go GCC 4.2 -> GCC 13+ -> Rust


I meant directly compiling Rust with gccrs, which will be included in gcc toolchain starting gcc13.


Just as a data point, my desktop is a Xeon e3-1230 v5, which is the same silicon as the i7 from 2015. The CPU cost $250 new, the entire desktop was $1100 or so, not including the monitors.

I followed the https://rustc-dev-guide.rust-lang.org/building/how-to-build-...

Then ran the build step "time ./x.py build -j 8" ... Build completed successfully in 0:36:53

real 36m53.398s user 254m52.720s sys 12m48.289s

Seems pretty reasonable considering it's not a particularly high end desktop from 2015. Seems like a cheap price to pay for increased reliability and security.


Optimizing language choice for something as fundamental as the kernel for compile speed seems very naive and short-sighted to me.

If I had a choice between 50% CVEs and 2h per compile or 5' compiles, I'd take the former in a blink.


Absolutely, and my own experience is that it's possible to create a language with Rust's safety guarantees while making it quick to compile.

I know this because I've managed to add a form of RAII and borrow checking to portable C11, and C is known for being faster than Rust to compile. Imagine what happened if we made a language with that stuff built-in.

The funny thing is that C is also slower to compile than it could be because of headers.


Yeah we should probably optimize for the 0.5% niche of people that care. Who needs builtin in memory and thread safety on core server systems so long as I can compile the kernel on my 486. Oh never mind, they are dropping 486 support too soon. Why can't it stay 1996 forever.


Whatever happened to CraneLift?


It's coming along well, here's an update from 2 weeks ago https://bjorn3.github.io/2022/10/12/progress-report-okt-2022...


I'm using it right now for faster compile times, following instructions from Perseus repo (Perseus is a web framework) https://framesurge.sh/perseus/en-US/docs/next/reference/comp...


Rust compiler does all the Rust’s magic (where the borrow checker is the biggest part). This magic is really important and helpful, and it's better to do it during compilation, not in runtime (for huge performance benefits).


I'm aware. I know how rust works and why people want it.

I haven't checked the times, but if the borrow checker is really the slowest part, maybe making rust skip it is a valid approach for end users. Sounds like an interesting experiment.


There was a PR merged yesterday which should give us 5-10% improvements in compile times [0]. As long as there are people monitoring and working on compile times, one could optimistically hope that as more companies become willing to throw money at Rust, we'll see improvements over time.

Regardless of how fast one's machine is, I think having some compile caching infra like sscache should help improve time to compile.

[0] https://perf.rust-lang.org/compare.html?start=9be2f35a4c1ed1...


The borrow checker is only one reason for the slow compile times. Cargo provides a separate `check` command to only do code & typechecking and that can be much faster than full compilation. LLVM and the way rustc interacts with it also plays a role.

In my experience when compilation takes longer the actual delays are in the last steps, long after the compiler is done checking and printing warnings.

On an unrelated note, I also noticed a large amount of static string literals in the code can slow down compilation to a surprising degree.


Cargo check is hardly usefull when doing graphics coding.


Why not skip the whole compilation step and just download the precompiled kernel? There is really no point compiling software you didn’t modify and if you did modify it, you want the borrow checker on.


> but if the borrow checker is really the slowest part, maybe making rust skip it is a valid approach for end users.

Relevant username?...


I mean, if the people that write the code don't skip it, I don't see the issue. The code in the source tree is supposed to be safe, so you might as well not do borrow checking and whatever other safety stuff you have.


If you aren't changing the code yourself, then why not download a binary? And if you are changing it, then you probably don't want to turn off the borrow checker.


By “biggest” I meant “giving the biggest impact”. I don't know if it's the slowest part.


It is expected to compile rust with a rust written-compiler (rust fanboys will improve it, probably).

Now, I wonder what is the most reasonable option: writting a naive and simple 'c11' compiler or a naive and simple rust compiler.

I wonder if somebody has done a "syntax complexity diff" between 'c11' and rust.

I know that linux is written in "gcc C", not 'c11'... so...

On the other end of the software stack, we have servo, mozilla web engine written in rust. What's up there? Still a drop of rust in a ocean of c++? (SDK included).

Because after years, if it is still impossible to run servo without c++ code, this is bad omens for kernel rust.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: