Usermode Dusk is a wrapper around Dusk OS allowing it to be used as an applicative platform on modern OSes. This wrapper allows the code to be ran at native speed (that's crazy fast) on the host OS. This wrapper is known to be able to compile on:
Because we're talking about native speed, this means that this can only be ran on CPUs for which there's a support in Dusk OS and for which there is a usermode kernel. This means:
i386 (works on x86_64)arm. Theoretically, Dusk's ARM code runs on armv5+, but it has only been
tested on v6+ so far. No aarch64 for now, but theoretically, it works. We
probably have to fiddle with GCC flags.Usermode Dusk can run on 64-bit variants of supported CPUs (for example
x86_64), but only if the host OS has the capability to run 32-bit executables.
To compile this wrapper on a 64-bit host OS, it often implies additional
tooling. For example, on Debian, you need to install gcc-multilib as well as
the 32-bit variants of libraries used by the different wrapper "flavors".
Theoretically, all combinations of hosts/CPU listed above work, but they haven't been all tested. Moreover, the Makefile is likely broken for some of those combinations. If you stumble on such combinations, please send patches to fix the Makefile.
There are 3 "flavors" of Usermode Dusk: stream, grid and graphic.
The "stream" flavor is the lowest common denominator and has a serial interface
which is plugged in the host OS' standard input and output for programs. The
name of its target is dusk-stream.
The "grid" flavor allows a Text User Interface (TUI) through Dusks' Grid and
piggy backs on the curses library on POSIX hosts and conio on Windows. The
names of its targets are dusk-curses and dusk-conio.
The "graphic" flavor allows a graphic interface through Dusk's Screen and uses
the SDL2 library. The name of its target is dusk-sdl.
While Usermode Dusk can be used in "regular" interactive mode, one exciting possibility that it allows is to package application in compact, fast and standalone executables for the host OS. These applications are called "Dusk packages".
To create a package, what you need is a "payload", that is, a stream of Forth
source that will intepreted by the kernel. A Dusk kernel starts from nothing but
the HAL, so that payload is likely to begin with the contents of
/xcomp/boot.fs. But afterwards, you place what you want.
In interactive Usermode Dusk, both the kernel and payload live in regular files
named kernel and payload_<flavor> in the same directory as the executable.
This allows you to fiddle with them for maximum debuggability.
Dusk packages will typically embed both their kernel and payload in the
executable themselves. The most straightforward and portable way to do so is to
generate ".h" file with hardcoded contents in it. You'll need to generate a
kernel.h and a payload.h from usermode/kernel and from your generated
payload. For this, you can use embedh.sh supplied by Dusk.
Then, you wrap all this in a main() function that will call common_setup()
and common_exec(). Your C source might look like this:
#include "duskos/usermode/common.h"
#include "kernel.h"
#include "payload.h"
int main(int argc, char *argv[]) {
int res = common_setup(
argc, argv, 4*1024*1024 /* 4MB of memory */,
kernel, sizeof(kernel), /* supplied by kernel.h */
payload); /* supplied by payload.h */
if (res) return res;
common_exec();
return 0;
}
Usermode Dusk does some host system probing to figure out proper flags to send
to the C compiler, you're likely to want to use the same. For this, you might
want to use the special machineflags target in usermode/common.mk which
spits out the machine specific flags for your host.
This will result in a standalone executable that runs your payload!
Confused? The best way to figure out how to build your package is to look at examples. Take a look at existing Dusk packages:
To build Usermode Dusk, you need a compiler that can compile 32-bit code for
your host CPU, which might mean gcc-multilib depending on your OS.
Then, it's only a matter of doing cd usermode && make. This builds targets
for all flavors. If you're missing a dependency for a flavor, this might result
in an error. In that case, you might be more specific. For example,
make dusk-stream will avoid such errors.
Then, you can invoke the executable without argument to begin an interactive
Usermode Dusk session. The "stream" flavor has the same double-echo problem as
the POSIX VM and should be ran with a terminal in "raw" mode. You can run
make run-stream as a shortcut.
The ability of the Usermode wrapper to run Dusk natively rests on Dusk kernels' ability to auto-relocate themselves. They aren't quite position independent because links in the system dictionary are absolute addresses, but the kernels are built in a way that it's possible (trivial even) for it, at boot time, to examine itself, know where it's ran from, then modify itself to run properly from this location.
From that point on, it reads the "boot arguments" supplied by the Usermode wrapper which gives it enough information to bootstrap itself into whatever its final purpose is.
That's why the wrapper job at boot time is relatively simple:
The Usermode wrapper and Dusk communicate through a structure defined in
common.h called interopzone. It's through this struct that boot arguments
are passed, but it's also through there that API functions receive their
arguments and yield their results.
This structure lives in Dusk memory at a pre-defined address, a constant we call
BOOTZONESZ, which has a value of 8KB. That zone represents the maximum size
that a Dusk kernel (which is quite small) can have. This doesn't include the
payload, which lives outside Dusk memory and is read-only.
The interopzone struct lives at the very end of BOOTZONESZ, which means that
the actual maximum size of a kernel is rather BOOTZONESZ-sizeof(interopzone).
With such a predefined constant, Dusk Usermode kernels know where to look for boot arguments, which allows them to bootstrap themselves properly.
The Usermode wrapper does more than merely booting Dusk, it also provides it
with an API to the Host system. As previously mentioned, this is done through
the interopzone struct. common.c exposes a global pointer to it as the iz
variable.
iz->funcs is a pointer to an array of function pointers, which all have a
void (*)(void) signature. Its those pointers that the wrapper API words
described below call. iz->arg{1,3} are for arguments passed to and from those
functions.
We don't pass arguments directly to the function because different host OSes have different calling conventions, so passing arguments reliably is murky, hence this struct.
Like regular Dusk, Usermode Dusk bootstraps itself from a tiny kernel. It does so quickly, so boot time is not a concern for executables designed to run more than a few milliseconds, but for executables that are designed to be short-lived and repeatedly called, this warm up time becomes a problem.
To mitigate this, Usemode Dusk has "freezing": When Dusk is finished bootstrapping itself, right before it's ready to rumble, it halts itself and we dump its memory. Then, we compile a new executable with that contents as a "kernel" instead. All this kernel does is jump to its compiled "payload word" (we could call it "main()") and thus fulfill its purpose.
Simple, right? Yeah, but there's a problem: Dusk is the opposite of Position Independent. It's Very Position Dependent. That the kernel is able to auto- relocate itself already requires careful threading, but relocating memory of a bootstrapped Dusk system is way outside the realm of the reasonable.
Therefore, we need something specific from the host OS: the ability to create a RWX mmap at a fixed address. Not all OSes allow this, which means not all host OSes can use Usermode freezing. If we can manage to have a consistent mmap address, then we can run that frozen kernel without having to relocate anything.
On the Dusk side, there's only one thing we must do: call freeze ( "name" --
) at the freezing point. This finds name in the dictionary and writes a
call to that word at address 0. Then, it calls the host's freeze API (see
below) with proper arguments.
This call will be the last call of your payload.
On the wrapper side, it's more involved. A Usermode wrapper first checks if it
was called with the _dusk_freeze_ as its last argument. If it was, the
argument is removed from the list before passing them on to Dusk. Then, the
wrapper knows it's in "wants to freeze mode". The global wantstofreeze
variable exposed in common.h indicates whether we're in this mode.
When we're not in this mode, the freeze API function is a noop, which means
that the freeze ( "name" -- ) word described above will simply call that
word and then shutdown.
When in this mode, when it's time to create the mmap, it does so with the
MAP_FIXED option and asks for the FREEZEADDR address, which is a constant.
If it manages to get it, good, it continues. Otherwise, it exits with a failure.
Then, it runs the payload normally until its freeze API function is called.
When it is, it dumps the frozen kernel to frozen_kernel and then exits with
success.
Then, that frozen kernel can be reused to compile a new Usermode package. This
package can behave identically to a regular package, with the peculiarity that
the "kernel" size will be bigger than BOOTZONESZ, but that doesn't cause
actual problems because the interopzone memory area has been reserved by Dusk
when it bootstrapped itself. All good.
And that's it! Drastically improved speed of short-lived packages! See the Dusk Examples repo for... examples.
Each usermode "flavor" exposes an API to the Dusk code so that it can interact with the host machine.
The API that is described below is the "user visible" API, as wrapped in Forth
code in api.fs, glue.fs, grid.fs and graphic.fs.
The actual API as implemented by C wrapper code is a bit different, but it's not worth documenting unless you're writing API wrapping code or developing a usermode kernel. At that point, you're better off looking at the code.
Usermode Dusk has the exact same fd* API as the one described in
posix/README.md.
Regular (as in: not Dusk Packages) Usermode builds use this API to open "disk.img" at startup and thus have full filesystem functonality.
These words are present in all flavors and implemented in api.fs and
glue.fs.
bye ( -- )
Shutdown the program. Initially, it is to that word that ABORTPTR is
hooked.
freeze ( here -- )
If in "wants to freeze" mode, freeze the program and shutdown.
Otherwise, this is a noop.
breathe ( -- )
Let the system breathe a little bit (sleep 1ms). This should be called
in idle loops. Other appropriate "breathe" actions can be taken by the
wrapper.
putchar ( c -- )
Print character "c" to the console/screen at current position and
advance that position.
?getchar ( -- ?c f )
Check if a key has been pressed by the user. If yes, f=1 and "c" is
present. Otherwise, f=0 and "c" is absent.
dbgPrint ( n1 n2 -- )
Print numbers "n1" and "n2" to the console/screen in a debuggable way.
sleep ( n -- )
Sleep for "n" microseconds.
ticks ( -- n )
Yield a "tick number" appropriate for sys/timer.
transferin ( srcname dstpath dstname -- )
Open *host* file path "srcname" and read its contents into a file
"dstname" that will be created at Path "dstpath". Example:
S" /my/host/path" p" /lib" S" foo.fs" transferin
transferout ( dstname -- )
Write the contents of current "work file" [doc/sys/file] into *host*
path "dstname". Example:
f" /lib/foo.fs" S" /my/host/path" transferout
In addition to the words above, the common API also defines a StdIO struct
with its companion stdio structbind that works very much like the Console
(see [doc/sys/io]) except that it wraps ?getchar and putchar directly
instead of wrapping in< and emit. This is useful because in< is plugged
to the payload stream rather than to ?getchar. Therefore, stdio is the only
straightforward way to access stdin and stdout through Dusk's I/O semantics.
Example usage:
S" Hello" stdio :puts
stdio :readline stype
In the "Grid" flavor implemented in grid.fs, we have a UMGrid struct that is
bound to the grid structbind [doc/sys/grid]. That grid supplies us with a
console.
In the "Graphic" flavor implemented in graphic.fs, we have a UMScreen struct
that is bound to the screen structbind [doc/sys/screen]. On top of that
screen, we have a Framebuffer Grid [drv/fbgrid] giving us a Grid and thus a
console.
On top of that, we also have a UMMouse struct that is bound to the mouse
structbind [doc/sys/mouse].
It's sometimes useful to have your own API calls, for example if you want to
wrap a library on the host OS. You can do so by creating a new C function with
a void (*)(void) signature and assign it to a cbfuncs[] slot. There are
APIFUNCCNT (256) available slots and the first APIRESERVEDCNT (32) ones are
reserved for Dusk itself, the rest is yours.
Arguments are passed through the global iz (interopzone, see above) variable
This structure has arg1, arg2 and arg3 members which are used both for
input arguments and output results. Thus, for an API word with the signature (
a1 a2 a3 -- r2 r1 ), a1 goes comes from arg1, a2 comes from arg2, a3
comes from arg3. The return values are reversed and r2 goes in arg2 and
r1 goes in arg1. Here's a simple example:
void myadder() { // ( a b -- n )
iz->arg1 = iz->arg1 + iz->arg2;
}
// ... Later in setup code
cbfuncs[42] = myadder;
On the Forth side, those APIs have to be wrapped with the syscallback,
(function with no result) or syscallbackr, (function with result). You supply
it with the number of arguments, the number of return values and a cbfuncs slot
ID and it generates a word for you. For example, let's wrap myadder:
2 1 42 syscallbackr, myadder
12 23 myadder . \ Prints "35"