Dusk OS sales pitch

Dusk OS has been around for a while and has been so far gathered only a small number of people. I believe it's for the better because a clear direction was needed and too many people hinder that.

The core of the system is now solid enough that I think it would be ready for more people to participate, but for that I think it needs a sales pitch, this sales pitch.

Preserving knowledge through tumultuous times

Let's begin with the main reason Dusk OS exists. Its stated goal is to help preserve knowledge through civilisational collapse.

Let's imagine that we lose the ability to manufacture new computers. What then? Existing computers will continue to run for quite a while, that's true, and they can run Linux or BSDs for a long time, that's also true.

However, what is also true is that these projects are maintained by a global network of highly specialized programmers. Linus himself said he doesn't understand Linux in its entirety. And that's just the kernel. A typical Linux system is much more than its kernel.

What if that global network is broken? Linux systems will still run but the ability for fragmented communities to maintain it will greatly diminish.

In the first years, it won't matter. Those systems are already pretty solid, they'll keep running for quite a while. Problems will show up much later, but unfortunately, by then those problems will have become insurmountable and with devastating consequences.

The first problem will be to gracefully adapt to degrading hardware. Let's say that a power surge fried a part of your computer. Not entirely, but something is broken and must be worked around. In today's world, nobody bothers. Buy a new computer and forget about it. Tomorrow, it might be the difference between having one half-working computer or none at all.

No Linux programmer ever spent time thinking of this eventuality, but future hackers might have to. Their chances of success is vastly diminished if the system they have to tweak is highly complex.

The second problem is one of education. Let's imagine that, as a civilization, we want to rebuild a technological world, rebuild means to produce computers. Those computers will be vastly different than our own and they might even be created by our children rather than ourselves. For a software system to be of any help in that context, it needs to be vastly simpler than typical systems like Linux.

This is what Dusk OS is all about, maximizing the chances for it to be useful in such tumultuous and uncertain times.

An elitist project in this era of racing to the bottom

By design, Dusk OS is an elitist project, that is, it isn't designed to be as easy to use by as many people as possible, but rather to be as powerful as possible for people who excel or who seek to excel.

At the core of its design goals is the requirement that a single excellent person can holistically understand it, but at the same time the features it ambitiously shoots for are in apparent contradiction with this core goal. I believe that the radical ideas at the core of Forth allows these apparently contradictory requirements to be reconciled, but only by reminding everyone that simple doesn't mean easy.

Forth is demanding for the mind. Unlike many other computer systems, it's difficult to learn because some mind muscles have to be trained first. Stack juggling in your mind and navigating immediateness levels is usually something you don't have to do with other systems. With Forths, as long as those muscles aren't trained, you'll be a mediocre Forth programmer, which might be enough to demotivate you.

What is this doing in a sales pitch? Ironically, I believe that this era of LLMs that has been unleashed upon us made this aspect of Forth into an advantage. Let me explain.

Everyone, even people who enthusiastically adopt LLMs, agree to say that regular LLM usage causes skills atrophy. Some say it's for the better, because we don't need those skills anymore, but others are worried about it. They worry that this atrophy makes them into lesser programmers.

In a sense, they're right. One could say that you trade one skill for the other. But what's that other skill exactly? Prompting? That's barely a skill and it changes all the time. Therefore, the tradeoff here isn't to exchange some skill for some other skill, but to exchange some skill in exchange for throughput. Whether that tradeoff is a good one depends on the point of view, but what stays true regardless is that this atrophy makes one a strictly lesser programmer. Going to work in a car might be faster than by bicycle, but it makes you a strictly lesser athlete.

So, there's this real dilemma going on. My goal here is not to debate that question, but to highlight an even more interesting one: what if we take the question from the other side? What should we think about systems that are more demanding for the programmer? By the same logic, shouldn't we come to the conclusion that they make one a strictly better programmer?

Personally, I believe so and after 6 years of Forth, I'm really glad I trained those muscles. I feel like I have superpowers. Even better, these superpowers stay around when using lesser programming languages.

Therefore, this is an invitation, if you are burdened by negative LLM-related emotions about being lesser, to examine the question from the other side and turn that pessimism into optimism. Dusk OS isn't the only project offering this kind of mind expansion potential, but it's certainly one of the good ones out there. You might want to consider it.

Holistic simplicity

The defining characteristic of Dusk OS compared to other non-Forth systems is holistic simplicity. To be clear, I don't mean API simplicity, like many other systems tout. I mean the simplicity of the system itself, in its entirety. Dusk OS is not easy, but it is simple. Comparatively, often by orders of magnitude.

The problem with simplicity is that it doesn't sell well. It's like virtue. Everyone wants it, but at the end of the day, it loses to shinier things.

Its benefits are tangible but hard to discern. The main benefit of a simple system is that it's easy to add new features to it without having to fight the existing complexity. The paradox of simplicity is that adding features is the main way to lose it.

However, this is not inevitable. Through careful design and choices, it's possible to add features to a system with minimal impact to its holistic simplicity. That's what Dusk OS is all about.

Compared to most traditional Forths, Dusk is significantly more complex, but it also packs significantly more power than many of them. Conversely, compared to non-Forth systems, Dusk OS is vastly simpler, but surprisingly, there's not that much of a sacrifice to make in terms of power.

I will give a few examples below and for each example, I'll mention the size of the code that powers this or that feature. That figure will seem hard to believe at first, but you can double-check it: they're true. They are also, in many cases, orders of magnitude smaller than the smallest alternative you can find, in any language or system. Don't suspend your disbelief, check the numbers, check the code, it's really that small. That's holistic simplicity.

Oh, also, don't let some of the expressiveness you see below make you think that this is an interpreted system. Dusk OS takes the subject of meta-compilation very far. This is compiled alright. Carefully crafted Dusk code is as fast as C.

What does it all mean? It means that despite the pretty impressive power it already packs, Dusk OS still has plenty of room for much more.

Examples

Iterators

Like most Forths, Dusk has loops, but it also has something more: iterators. Iterators bind data and code together to provide an interruptible way to gradually process something. Iterators are at the heart of Dusk's async I/O capabilities. Example:

needs lib/iter
: printnumbers ( iter -- ) begin dup next while . spc> repeat drop ;
42 3 niter 5 take printnumbers \ prints 42 45 48 51 54

And here is the same thing, functional-style:

needs lib/func lib/iter
[: . spc> :] 42 3 niter 5 take fold \ prints 42 45 48 51 54

The lib/iter unit is less than 110 lines of code. The lib/func unit is less than 120 lines of code.

Tasks

Note: tasks are a spanking new feature of Dusk which is available only in the git repo until v38 is released.

Dusk OS is built around cooperative -- rather than pre-emptive -- multi-tasking. This design choice is among those that have the most effect on holistic simplicity.

In Dusk, multi-tasking revolves around the concept of, drum roll... tasks. Let's give them a try. If you run make run at the root of the project, you get a prompt. Let's launch a task:

needs bench/task
pingpong

You now get "Ping!" and "Pong!" printing at roughly one second intervals. You've launched what we call an active task, you can't do anything else while this task is active. There is only ever one active task pulsing at once.

You could quit it by pressing 'q', but that wouldn't be fun. Instead, press CTRL+Q (make run has inhibited the CTRL+Q signal through stty -ixon, so it goes through). What this does is that it moves that task into what we call the Limbo, where tasks are neither dead nor alive.

You see that your prompt works again and that you can type commands. Type .bg. It will print something that looks like this:

Background
0 Snoozer   
Limbo    
0 Ping Pong!

It shows that you have one background task, Snoozer and one task in Limbo, your pingpong task. If you type 0 resume, it will move the pingpong task (the 0th task of the limbo list) from the Limbo to the Background list.

You'll then see the "Ping!" and "Pong!" message spout again. You'll be in a weird state because typing still goes to your prompt, you can still type commands (that tells you that tasks that print shouldn't be ran as background tasks...). If you type .bg in that mess, you'll see that the pingpong task is at index 1 of the background tasks. You can bring it back to limbo with 1 pause.

You could then forcibly stop that task with 0 kill, but you can also bring it back to the foreground with 0 foreground. You're now back to where you started, with an active pingpong task that shadows the prompt. You can press 'q' to stop the task and go back to prompt. bye quits Dusk.

The lib/task unit is less than 100 lines of code.

Assemblers

Let's say you are on a ARM system and that you want to compile a tight loop that sums a list of numbers in assembler. Let's go:

needs asm/arm
create numbers map< , 42 54 123 12 23 34
code addlist ( a u -- sum )
    ldr) r0 rd) rPSP rn) 4 +i) post) ,) \ Pop "a" in r0
    mov) r1 rd) rW rm) ,) \ move "u" to r1
    mov) rW rd) 0 imm) ,) \ initialize sum
    begin
        ldr) r2 rd) r0 rn) 4 +i) post) ,) \ read number and postinc
        add) rW rdn) r2 rm) ,) \ add to sum
        sub) r1 rdn) 1 imm) f) ,) \ decrease counter and set flag
        abs>rel b) ne) ,) \ loop as long as Z flag not set
    exit, \ our sum is ready
numbers 6 addlist . \ prints 288

This word that is compiled here, "addlist", is native code, it's as fast as it gets.

Yes, this syntax doesn't look as familiar as any other assemblers you know, but a great advantage of Dusk's assemblers is that you can drop to it at any time, inline whatever assembler code you want, then resume whatever you were previously doing. In other systems, assembler integration is rarely as harmonious as in Dusk.

Dusk's ARM assembler's source code is less than 130 lines of code.

It also has assemblers for x86, m68k and RISC-V.

HAL

As shown above, Dusk allows you to assemble binary code anywhere you want. But you know what's better than this? Doing so in a portable way. At the core of Dusk is the HAL, the Harmonized Assembly Layer which is an API that looks like assemblers because it targets registers and memory addresses and its operations map directly to CPU ones. However, it wraps this API around cross-CPU guarantees that allows one to write portable low level code. Here's that "addlist" above in HAL:

create numbers map< , 42 54 123 12 23 34
code addlist ( a u -- sum )
    PSP) A>) @+, \ pop "a" in A register
    S) &) !, \ move "u" to S register
    0 i) @, \ initialize sum in W register
    begin
        A) +, \ read number and add to W
        4 i) A>) +, \ increase A by 4
        1 i) S>) -, \ decrease counter by 1
        ?brnz, \ loop as long as Z flag is unset
    exit, \ our sum is ready
numbers 6 addlist . \ prints 288

The smallest HAL implementation, the i386 one, is less than 280 lines of code. The largest HAL implementation, the RISC-V one, is less than 310 lines of code.

C compiler

Dusk also has a C compiler written in Forth. As you've guessed, the HAL is used to generate binary code. Here's the "addlist" word written in C:

needs comp/c
create numbers map< , 42 54 123 12 23 34
:c uint addlist(uint u, uint *a) {
    uint sum = 0;
    while (u--) sum += *a++;
    return sum;
}
numbers 6 addlist . \ prints 288

The C compiler as well as compiler-oriented libraries in Dusk contain less than 1300 lines of code.

Security through clarity

Dusk OS hasn't been designed with security in mind because this system is designed for a world where the internet doesn't exist in its current form. Security threats are wildly different.

Why even mention security then? Isn't Dusk OS practically swiss cheese with its shared memory and cooperative multi-tasking model?

Superficially, yes, it's true. Dusk OS doesn't have any of those bells and whistles that could possibly make you feel safe in that system.

There is, however, a sweet irony about the LLM age, which also ushered an age of discovering security holes in "serious" software projects at a continuous and alarming pace.

Those systems all have the required bells and whistles, but you know what? A significant part of the security holes that pop out all the time are downright embarrassing. These are goofy holes. Not serious at all.

What does this tell us? I believe that it tells us that elaborate security measures doesn't buy us security. The only thing that can possibly bring us security (and thus peace of mind) is perfect code.

Most people will say that perfect code is not possible, but I believe that those people haven't experienced holistic simplicity enough. Maybe that perfect code cannot be reached, but I'm pretty sure that we can get much closer to it in simpler systems.

I am not a security expert, but I have this intuition that if we tried building secure systems in Dusk, we might discover that it holds quite better than one would naively have guessed. Maybe it's worth digging this subject further.

Participation welcomed

I hope that this sales pitch made you consider participating to the Dusk OS project. Although the project itself aims to be holistically understood by a single person, writing that system is too much for a single person. To be successful, this project needs to be built by a group.

If you don't know Forth yet, you have a lot to learn first, but believe me, it's worth it. Tumble Forth is a good first step.

Then, when you feel you're ready, you need to find a way to insert yourself in the development process. Finding that insertion point might not be easy because the needs of the project change over time. Ask around, we don't bite.