The Go programming language is not seen as a language used in low-level Operating System development. There have been many posts on the r/golang subreddit asking if it is even possible (2015, 2017, 2020, 2021 and 2023). That question is easily proven. See eggos, gopher-os, biscuit and LetsGo-OS.
Of course, most of the comments admit it to being possible but claim the GC or runtime would be too slow for an OS or you wouldn't really be writing Go. My intention with this series is to prove that these claims don't hold. However, this post is about why, for me, Go is the only logical choice for OS development. The reason is that C is just the wrong abstraction layer.
An average programming story
You're a developer assigned to create a new feature for the product your company sells. They need you to list which processes are taking the most amount of time on the CPU and store it in a database for later analysis. Seems simple enough at first. You go googling and see that on Linux the only way to get the information about which processes are running is with ProcFS. Oh, how elegant you say. I can just use the open, read and close system calls to get the information. Your boss has you write in C because it needs "to be fast"™️ and everything seems good.
Later, the boss comes over and says they also want it running on macOS and Windows. Ok, you say. This shouldn't be too difficult just google the APIs for those platforms and use #ifdef to compile to the different platforms. A few hours go by and you've got it tested and working.
You move on to other things but eventually, a few weeks later the boss comes rushing in frantically saying the latest version of macOS has broken your code and you need to fix it. You go and investigate and see that Apple went ahead and changed the semantics of the API. No big deal right just update to use the new and improved version. However, this isn't okay since some of the customers haven't updated to the latest version yet and we can't break their systems.
The first choice you think of is to add another pragma option for compilation. But then you think what if you make this process-grabbing piece its own executable and then have another application download whichever version of the proc grabber it needs? That way we're insulated from any changes by the OS. You sketch up a version in Python because you were just curious how easy it would be and plan on replacing it later with a C version.
Well later never came and you got busy fixing other major bugs. In the months following your decision, other employees have utilized your auto-update feature for all kinds of wacky stuff. You don't seem to mind since it hasn't broken anything.
However, now you've been assigned to port the whole system to a new obscure Unix system. You get the cross-compiler built which was a challenge on its own but finally, you get the program onto the OS. You run it and it crashes because your auto-update tool used Python 3 but the OS only supports Python 2. Now you could port the program back to Python 2 but that would be very error-prone. It's not possible to require your clients to install Python 3. You then have the thought: what if there was a way to package a version of Python 3 with your program? One of your co-workers mentioned this thing called Docker which allows you to bundle a group of software together into one container.
You bang it together and ship it to the new clients with success. On reflecting on this journey you ask yourself why this was so difficult.
What is an abstraction?
Each time you pull in a library for use in your code, create a config file in JSON, YAML or XML, or even use the compiler to build the program you've abstracted. Abstractions take one concept and transform it into a more generalized and hopefully easier-to-use and understand one.
Of course, abstractions are good; a necessity in fact. However, programmers have taken some abstractions too far. We've abstracted the web so far from reality with JavaScript that we had to build a compiler to bring back types with TypeScript. Even the binaries we create can't run anymore on modern OSs. The fragile dynamic libraries issue has grown so bad that we recreated static binaries with Flatpak, Snaps and AppImages except they have none of the performance benefits.1 It's not even possible to configure our programs properly without the need for a whole virtualized OS on top of our OS with Docker.
Now of course these tools do have their uses and have been quite beneficial but does all this abstraction make more maintable software?
What makes a good abstraction?
There are three parts to a good abstraction:
Simple
Slow Changing
Widely Used
Simple: When I say simple I don't mean JavaScript simple where any sequence of random characters will somehow successfully do something. That's just confusing not simple. Nor do I mean writing every instruction the CPU executes in assembly. That's just tedious.
By simple I mean there are only a few concepts to understand and they mesh together in a congruent and useful way. C and C++ are not simple. They have so many different ways of doing things like memory allocation, or Unicode manipulation and every few years new libraries crop up. C and C++ have a whole second language that manipulates your code before it compiles.2 This is not even to mention supposed trivial things like the disagreement on how the code should be formatted. There are so many concepts to understand to even write the code that C and C++ just can't be classified as simple.
Slow Changing: Now this one C is super good at. Backward compatibility truly is the shining achievement of why C has been so successful. If people's programs started breaking they'd move on to more sustainable languages.
Although this is C's biggest strength it's also its biggest weakness. Not changing means new idioms and ideas and better ways of working don't get added. Now I'm not saying that C should break old programs. It's just that C no longer provides the level of abstraction that modern programmers expect.
Widely Used: This may seem like C and C++ have this one down. Everything uses C or it's deranged brother C++, right? Well, of course, it is used quite a lot. I cannot deny that but by widely used I mean the ecosystem as a whole.
There are tons of libraries with contradicting ways of handling HTTPS, testing code or grabbing dependencies. There's so little uniformity that it takes a super genius just to figure out how to build a GitHub C project because there are a plethora of varying different tools. Does the project use make, ninja, cmake, bazel and what about autoconf? How can no one agree on how to build programs?! Isn't that what programmers do all day?
My Regards: Now, this isn't C or C++'s fault. The languages were made in a different era where they were a better fit for the level of abstraction they provide. Times have changed and a unified testing framework, debugging, fast development time and easy cross-compiling experience are what people expect to be able to agree upon.
And none of these ideas are new. It's what Sandi Metz meant in 2016 by
duplication is far cheaper than the wrong abstraction 3
Or by Rob Pike (co-creator of Go) when he created the Go proverb
The bigger the interface the weaker the abstraction.4
C and C++ have now reached the limit of duplication that a clear abstraction can be formulated.
Why Go is the only choice
Now back to the original question: why Go is the only choice for OS development? The reason is that Go is an Island. Now when fasterthanlime wrote that he meant it as a negative. For him, the Go compiler, linker, assembler, testing framework, formatter and debugger are all different from the rest of the world and don't interop well with other languages. He is right. But as he even acknowledges that is the point. It's gophers all the way down (or at least as far as the platform will let it go). When something breaks the only language you need to know is Go.
When you take in a dependency you've created a new coupling that you are now responsible for and must make sure it continues to work. This is what Russ Cox talked about in Our Software Dependency Problem. But a dependency is not just a snippet of code from the internet. It also includes every tool required to build the compiler that builds your program and their tools to build themselves all the way backward. This is what Ken Thompson noticed in his famous paper Reflections on Trusting Trust from back in 1984.
This is why I rather have a very wide dependency graph because that means there are fewer concepts to understand. Each part of my toolchain is in one language I know well. No other language provides as complete of an ecosystem as Go. Perhaps, one day Rust and their love for "rewriting everything in Rust" would make it a good choice but even then Rust changes a lot. Go does not.5 Is Go the perfect language? No. But I'm inclined to think that no such language can exist. So when it comes to my hobby OS that no one will care about I'd rather only think in one language; the Go one.
Cover Image Created by Alfrey Davilla and used under the CCA 3.0 License. Source https://dribbble.com/shots/18090283-Gopher-with-Laptop. The only change was the removal of the Adlibris logo.
I understand they have other benefits like sandboxing. However, sandboxing should be provided at the OS layer and is not exclusive to dynamic linking. ↩
And templates are so complex that they may even be Turing complete. https://rtraba.files.wordpress.com/2015/05/cppturing.pdf ↩
https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction ↩
Maintaining backward compatibility is extremely important to the Go philosophy. See Russ Cox's talk How Go Programs Keep Working. ↩