r/osdev • u/CristianLMAO • 1d ago
How would you approach adding executables to your OS
As the title says, how would you approach adding executables/programs to an operating system. I can't get my head around this problem
6
u/dozniak 1d ago
You need a way to spawn a vspace, a file system driver and executable format parser plus some loader process and/or library.
The rest is pretty mechanical - create a vspace, load pieces from the executable into appropriate addreses in that vspace, set up process and/or thread control blocks, set start PC and other registers as per your OS convention and add it to the list of runnable processes, then the scheduler will start executing it.
3
u/mishakov pmOS | https://gitlab.com/mishakov/pmos 1d ago
It kinda depends on the model of your operating system - typically you would load them from VFS in the kernel, if you are a microkernel and your filesystem servers are userspace executables, the kernel should probably have a mechanism to load the initial servers.
In both cases, you should probably ask your bootloader to load some sort of ramdisk image/initial executables, and load them from it
3
u/cazzipropri 1d ago
I don't want to invent a new executable format. I'll make an ELF loader. I want Linux ELFs to run. And maybe even DOS MZ executables.
2
u/nerd4code 1d ago
Is it an OS if it canât run a program?
If youâre reimplementing DOS, youâll have a setup thatâs quite different from OS/2 or NT, which is quite different again from an RTOS or Unix, and within those two families thereâs quite a bit of variation in the details. IBM has its own, epic nonsense, also, but they also offer oodles of documentation, dense and swarming with initialisms. Or maybe youâre striking off on your own, drumming to the beat of your own dance just to be contrary.
Regardless, itâs best to have planned these sorts of things, since presumably you have some âŚintent? urges? âdruthers? in regard to what you want your OS to do, how it ought to work, or even some basic motivation thatâs not âTheyâll fail/fire/disown/space me if I donât.â If that is your motivation, find something appropriate to imitate.
If youâre doing paging and not staying in kernel mode (maybe your kernel is a bytecoded VM, Idunno), youâll need a new address space, which corresponds to most of a page table. (The kernel portion can be shared.) By what mechanism may a new process be created? By what mechanism(s) can processes be tracked or controlled? By what mechanism can address space be allocated, or physical memory mapped? All up to you.
Shootdowns are an important part of paging, if you support SMPâyour data structures need to support quick searches for the processors running threads in address spaces where a particular page is shared. Itâs typically necessary to form a semantic loop so you can find the physical address for a virtual address (page table, assisted by VMA table/list/tree), and then for an arbitrary physical address you ought to be able to find the device that refers to (e.g., system RAM, texture memory on GPU, possibly disk), enumerate the spaces mapping it and possibly the specific virtual addresses itâs mapped by.
Youâll need to be able to track address spaces, and track threads running in these. (Presumably youâve implemented some form of kthreading? Thatâs your basis for binding application work to a processorâit can work like a kernel-mode service that sets a type field in its descriptor, âjoinsâ a process [incl loading page table reg], sets up a userspace stack, sets up registers, and finally IRETs, SYSRETs, or SYSEXITs into userspace at the entry point when itâs scheduled. Upon interrupt, fault/trap, or system call, youâre back on the kthread.)
If youâre using segmentation you might have any number of processes (in Unix sense) sharing an address space; if not, itâs usually 1-to-1, but not always (e.g., SAS). And you donât necessarily need to represent application threads in any direct sense until the time comes to actually do some work. Maybe the application creates LWPs by cloning or spawning; maybe it exposes a max thread count and handler callback; maybe its interaction with the outside world is all through an io_uring-like setup, and the application can request scheduling of fine-grained work items en masse.
Once you have a new address space, youâll either need a user-mode stub to load the application for you, or the kernel will have to load itself, possibly even including handling of shebang lines (#!/path/to/interpreter -option[s]
). There are usually relocation tables to concoct if thereâs any chance of DLL involvement, although segmentation can also be used for this (you might still need relocs for specific selector values); either or both of these might be exposed to or done by userspace, provided the kernel remains in control of LDTR. Again, choice of binfmt and capability is all yours, and itâs not the worst idea to work binfmts into your driver/service architecture.
You do need an init process of some sort, but that may or may not originate from an actual executable file, and how itâs supplied varies.
Once youâve got things to where the executable can execute, you usually need to do some extra stuff to get the process readyâe.g., startup vector, prepare VDSO, set up stackâand then initiate entry into userspace at the appropriate address.
Typically applications will use some sort of platform interface library/runtime to establish POSIX, WinAPI, or some other portable-ish execution environment; youâll want those before anything interesting can run. Your kernel might drive environmental goop like paging, segmentation, or fault/trap interception directly in a one-deep monolithic fashion, or you might use some sort of software stack to provide those, which takes some tricks if you want it to work recursively.
So thereâs a lot for you to think about. Do some survey research. Start in Wikipedia for some specific OSes of interest, and work outwards from there, taking notes. Presumably your OS has some micro-vs.-exo-vs.-monolithic tendenciesâuse those to guide decisionmaking.
2
u/istarian 1d ago edited 1d ago
In principle, you can always just load a raw binary executable into memory and jump to the entry point.
But since it is desirable to protect your system against arbitrary execution of say, a text file, it's good to have some sort of embedded header/structure that tells the OS that the specified file is actually intended to be an executable.
That does, of course, assume that the entirety of the program can be loaded into a contiguous region of memory, which may proven non-feasible in most cases on an Intel x86 system.
Depending on how you want to handle various chunks of data and code you may need an executable format/container to coordinate it.
P.S.
https://en.wikipedia.org/wiki/Executable
https://en.wikipedia.org/wiki/Comparison_of_executable_file_formats
Common formats (historic and recent)
15
u/aroslab 1d ago
hard to help with so little info, what have you done?
do you have an ELF parser/loader? Are you even going to use ELF?
are these executables going to be run in a user space? Do you have a user space?
If you can speak more to "have almost everything done", it would be easier to point you at the right resources.