Operating Systems Security I
Tom Kelliher, CS 325
Oct. 3, 2011
- Protected objects and general protection mechanisms.
- Memory protection
- CPU execution modes and kernel virtualization of CPU.
- Server virtualization.
Operating Systems Security II.
With today's operating systems, multiple processes/threads run together
``simultaneously,'' necessitating that protection mechanisms exist for the
- Shared I/O devices (disk drives, etc.).
- Serially reusable I/O devices (printers, etc.).
- Sharable programs and libraries.
- Sharable data.
Separation is the basis of protection:
- Physical separation: Run sensitive processes on separate systems.
- Temporal separation: Run sensitive processes at different times.
- Logical separation: OS/hardware provide a logical separation of sensitive
- Cryptographic separation: Sensitive processes conceal their data and
How can processes share data and code?
- No protection: acceptable when temporal separation is in use.
- Isolate: one process is completely unaware of the existence of another
process. (Logical separation)
- Share all or share nothing: objects declared public (access by all) or
private (access by no one else).
- Share via access limitation: Maintain a list of (user, access rights) pairs for
each object in the system.
- Share by capabilities: Each user (process) holds a list of access rights
(capabilities) for objects in the system. (Similar to previous mechanism.)
- Limit use of an object: Extend the usual read, write, execute notion to
such actions as print and copy.
Multiple memory locations can be accessed per each instruction executed, so
access limitation mechanisms must be extremely efficient.
- Variable and fixed fence registers. Basic idea:
Access ``under'' the fence results in an exception.
- Base/Bounds registers: Fence register grants access all the way to the end
of memory. The bounds register specifies the size of the memory segment.
Multiple base/bounds pairs can be used to provide separate areas for the
four memory segments associated with a program: code, heap, stack, and
- Tagged architectures: associate tag bits with each memory word.
Burroughs used three tag bits. Tag meanings: data, double precision data,
loop index, uninitialized data, pointer, code, data block descriptor, program
control word. The first four tags are for R/W words. The second four are
- Paging: Further partitions memory into frames. Maps a logical (page,
offset) address to a physical (frame, offset) address by use of a page
Virtual memory takes this one step further by removing the restriction that
all logical addresses be mapped to physical addresses.
- Some machine instructions are ``dangerous.'' Examples: halt, I/O access.
User programs should not be able to execute these instructions.
- Solution: CPU runs in one of two modes: supervisor or user.
- Privileged instructions generate traps when executed in user mode.
Generally, the trap handler terminates the process.
- How do we get from user mode to supervisor mode to perform I/O?
System call instruction.
- The kernel emulates the CPU for user processes.
The only exception are the privileged instructions.
- Processes are isolated from one another.
- A host OS, or hypervisor, runs multiple guest OSs.
Each guest OS runs in complete isolation from other guests.
Each guest runs as if it has its own resources (disk, network, etc.).
- In a sense, a hypervisor is an OS that runs other OSs.
The hypervisor emulates the CPU for guest OSs.
In contrast, now the privileged instructions must also be emulated.
- Virtualization requirements for an architecture:
Any instruction which changes certain ``sensitive'' machine state must
generate a trap if executed in user mode.
The trap is used so as to return control to the hypervisor and it can
provide the behavior required by the guest, rather than the hardware
providing the behavior.
x86 does not meet this requirement -- paravirtualization requires some
changes to the guest OS.
- Both Intel and AMD have added virtualization support.
- When done right, virtualization can be extremely efficient, as opposed to
the earlier technique of emulation. For example, running Windows on a
PowerPC by emulating an X86.
Thomas P. Kelliher