Porting UNIX code to Dusk

Some of Dusk's goals, such as being able to visualize PDFs, imply a big chunk of code has to be written for them. The Forth ecosystem have a few chunks of logic here and there, but Forth being Forth, nothing is standardized and, more importantly, it's nowhere near the UNIX ecosystem in terms of completeness.

We don't want, however, to embark on an endless journey of reimplementing this logic. We want, instead, to be able to piggy-back on the existing code as much as possible.

Therein lies a problem: that code is written on top of a UNIX platform, a platform which we reject because of its complexity (see doc/design/limits). How to reconcile these conflicting goals?

First, by counting on the fact that most of the tricky logic we want to piggy-back on is "pure" logic and doesn't actually depend on the UNIX platform.

Second, by counting on the fact that the most important pieces of logic out there are written in C.

Third, by creating an "almost C" compiler that tries to strike a good balance between two conflicting goals: on the one side, stay true to Dusk OS design goals and on the other hand, minimize the amount of work that has to be poured into a particular piece of UNIX C code before it can be compiled and run by Dusk.

Dusk' big bet is that with that compiler, it will be possible, with a minimal effort, to piggy-back on existing UNIX code to provide the same logic in a much simpler environment.

Porting drivers

Another pile of code worth leveraging is existing device drivers written in C. The problem with those is that they're typically deeply intertwined with their host kernel abstractions and those abstractions are typically too complicated to be a good fit to Dusk, making the porting effort very difficult.

I took a stab at porting NetBSD's drivers, with modest success, but the crap I have to wade through seems too much, relative to the end result, to be worth it.

Then I took a better look at Plan 9. You know how you sometimes hear people saying that Plan 9's code is clean? It's true! Compared to NetBSD's code, to paraphrase Linus, it's a work of art. Cleaner abstractions, grokkable code.

So I began porting that code instead. So far it's a smooth sailing. But I know there's a big challenge ahead and it's...

Making drivers async

All modern OSes having a wide support for hardware (and thus being a good source of code to leverage) support pre-emptive multitasking, including Plan 9.

This means that driver logic is littered locking logic, which we can get rid of, but also with "sleep for a bit, waiting for some condition to happen on the hardware side" bits and pieces of code. This can be ported to Dusk just fine... but will make the system jittery!

Once that porting is done, an extra challenge will be to transform that driver into "async style" code where instead of waiting (except for very small waits. sometimes, it's just "security wait to make sure the hardware had the time to acknowledge that register write"), we return. This means we'll have to keep track of "execution status" to know where we need to return to once the Dusk's idle loop is ready to give its attention to that particular driver again.

So, that's a lot of work, beggin the question of whether it's worth porting driver code at all. The answer there is, as always, "it depends". Some drivers, such as a USB stack, have a really big list of "business logic" that you don't want to re-implement yourself, so it's worth pouring the "asynching" work.

Sometimes, hardware documentation is not publicly available, so the only actual source of information about that piece of shit hardware is in the form of code. Guessing hardware specs from code is not fun, one might as well reuse the code entirely, even if that means "asynching" work.

In other cases, I guess it will make more sense to write the driver from scratch. When the specs are clear, that there is little "business logic" on top of it, why bother?