Privileges
Hardware
We mentioned how one of the major tasks of the operating system is to implement security; that is to not allow one application or user to interfere with any other that is running in the system. This means applications should not be able to overwrite each others memory or files, and only access system resources as dictated by system policy.
However, when an application is running it has exclusive use of the processor. We see how this works when we examine processes in the next chapter. Ensuring the application only accesses memory it owns is implemented by the virtual memory system, which we examine in the chapter after next. The essential point is that the hardware is responsible for enforcing these rules.
The system call interface we have examined is the gateway to the application getting to system resources. By forcing the application to request resources through a system call into the kernel, the kernel can enforce rules about what sort of access can be provided. For example, when an application makes an open() system call to open a file on disk, it will check the permissions of the user against the file permissions and allow or deny access.
Privilege Levels
Hardware protection can usually be seen as a set of concentric rings around a core set of operations.
In the inner most ring are the most protected instructions; those that only the kernel should be allowed to call. For example, the HLT instruction to halt the processor should not be allowed to be run by a user application, since it would stop the entire computer from working. However, the kernel needs to be able to call this instruction when the computer is legitimately shut down.[1]
Each inner ring can access any instructions protected by a further out ring, but not any protected by a further in ring. Not all architectures have multiple levels of rings as above, but most will either provide for at least a "kernel" and "user" level.
386 protection model
The 386 protection model has four rings, though most operating systems (such as Linux and Windows) only use two of the rings to maintain compatibility with other architectures that do now allow as many discrete protection levels.
386 maintains privileges by making each piece of application code running in the system have a small descriptor, called a code descriptor, which describes, amongst other things, its privilege level. When running application code makes a jump into some other code outside the region described by its code descriptor, the privilege level of the target is checked. If it is higher than the currently running code, the jump is disallowed by the hardware (and the application will crash).
Raising Privilege
Applications may only raise their privilege level by specific calls that allow it, such as the instruction to implement a system call. These are usually referred to as a call gate because they function just as a physical gate; a small entry through an otherwise impenetrable wall. When that instruction is called we have seen how the hardware completely stops the running application and hands control over to the kernel. The kernel must act as a gatekeeper; ensuring that nothing nasty is coming through the gate. This means it must check system call arguments carefully to make sure it will not be fooled into doing anything it shouldn't (if it can be, that is a security bug). As the kernel runs in the innermost ring, it has permissions to do any operation it wants; when it is finished it will return control back to the application which will again be running with it's lower privilege level.
Fast System Calls
One problem with traps is that they are very expensive for the processor to implement. The process of looking up the tables
Modern processors have realised this overhead and strive to reduce it.
Other ways of communicating with the kernel
ioctl
about ioctls
File Systems
about proc, sysfs, debugfs, etc