Final paper for OS II

OSU Coursework/CS 444 - Operating Systems II/Writing Assignments/Final Paper/latexmkrc

@@ -0,0 +1,31 @@
+# This config is copied from overleaf so output there will match compiled output here
+
+# support for the glossaries package:
+add_cus_dep('glo', 'gls', 0, 'makeglossaries');
+add_cus_dep('acn', 'acr', 0, 'makeglossaries');
+sub makeglossaries {
+  system("makeglossaries \"$_[0]\"");
+}
+
+# support for the nomencl package:
+add_cus_dep('nlo', 'nls', 0, 'makenlo2nls');
+sub makenlo2nls {
+  system("makeindex -s nomencl.ist -o \"$_[0].nls\" \"$_[0].nlo\"");
+}
+
+# from the documentation for V. 2.03 of asymptote:
+sub asy {return system("asy \"$_[0]\"");}
+add_cus_dep("asy","eps",0,"asy");
+add_cus_dep("asy","pdf",0,"asy");
+add_cus_dep("asy","tex",0,"asy");
+
+# metapost rule from http://tex.stackexchange.com/questions/37134
+add_cus_dep('mp', '1', 0, 'mpost');
+sub mpost {
+  my $file = $_[0];
+  my ($name, $path) = fileparse($file);
+  pushd($path);
+  my $return = system "mpost $name";
+  popd();
+  return $return;
+}

OSU Coursework/CS 444 - Operating Systems II/Writing Assignments/Final Paper/makefile

@@ -0,0 +1,18 @@
+# Makefile created by Corwin Perren
+# Generic makefile for all LaTeX projects downloaded from overleaf
+# 
+# All this makefile does is call perl against latexmkrc, which is
+# the latex equivalent of make
+
+LATEXMK_COMPILE_FLAGS = -pdf
+LATEXMK_CLEAN_FLAGS = -c
+
+.DEFAULT_GOAL := all
+
+all: latexmk_output clean
+
+latexmk_output: 
+	perl latexmk.pl $(LATEXMK_COMPILE_FLAGS)
+	mv output.pdf CS444_final_paper_perrenc.pdf
+clean:
+	perl latexmk.pl $(LATEXMK_CLEAN_FLAGS)

OSU Coursework/CS 444 - Operating Systems II/Writing Assignments/Final Paper/output.aux

@@ -0,0 +1 @@
+\relax 

OSU Coursework/CS 444 - Operating Systems II/Writing Assignments/Final Paper/output.fdb_latexmk

@@ -0,0 +1,7 @@
+# Fdb version 3
+["pdflatex"] 1528865467 "output.tex" "output.pdf" "output" 1528865467
+  "output.aux" 1528865467 8 a94a2480d3289e625eea47cd1b285758 ""
+  "output.tex" 1528889868 2009 e496266a8eadcc090902d4bf3568571d ""
+  (generated)
+  "output.log"
+  "output.pdf"

OSU Coursework/CS 444 - Operating Systems II/Writing Assignments/Final Paper/output.tex

@@ -0,0 +1,103 @@
+\documentclass[onecolumn, draftclsnofoot, 10pt, compsoc]{IEEEtran}
+\usepackage{graphicx}
+\graphicspath{{./figures/}}
+
+\usepackage{url}
+\usepackage{setspace}
+\usepackage{multicol}
+\usepackage{pdflscape}
+\usepackage{pdfpages}
+\usepackage[british]{babel}
+\usepackage{listings}
+\usepackage{xcolor}
+\usepackage{listings}
+\usepackage{hyperref}
+\usepackage{subfig}
+
+\usepackage{geometry}
+\geometry{textheight=9.5in, textwidth=7in}
+
+% \overfullrule=2in
+\hypersetup{
+    colorlinks=true,
+    linkcolor=blue,
+    filecolor=magenta,      
+    urlcolor=cyan,
+}
+
+%Personal \newcommands
+
+\definecolor{backcolor}{rgb}{0.95,0.95,0.92}
+\lstset{basicstyle=\ttfamily,
+  backgroundcolor=\color{backcolor},  
+  showstringspaces=false,
+  commentstyle=\color{red},
+  keywordstyle=\color{blue},
+  columns=fullflexible,
+  breaklines=true,
+  postbreak=\mbox{\textcolor{red}{$\hookrightarrow$}\space},
+}
+			
+\newcommand{\NameSigPair}[1]{
+  \par
+  \makebox[2.75in][r]{#1} 
+  \hfill
+  \makebox[3.25in]{
+      \makebox[2.25in]{\hrulefill} 
+      \hfill
+      \makebox[.75in]{\hrulefill}
+  }
+  \par\vspace{-12pt} 
+  \textit{
+      \tiny\noindent
+      \makebox[2.75in]{} 
+      \hfill
+      \makebox[3.25in]{
+          \makebox[2.25in][r]{Signature} 
+          \hfill
+          \makebox[.75in][r]{Date}
+      }
+  }
+}
+% 3. If the document is not to be signed, uncomment the command below
+\renewcommand{\NameSigPair}[1]{#1}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\begin{document}
+\begin{titlepage}
+	\pagenumbering{gobble}
+	\begin{singlespace}
+		\par\vspace{2in}
+		\centering
+		\scshape{
+			\huge Operating Systems II Final Paper \par
+			{\large\today}\par
+			\vspace{.5in}
+			\vfill
+			\vspace{5pt}
+			{\large Prepared by }\par
+
+			{\Large
+				\NameSigPair{Corwin Perren}\par
+			}
+			\vspace{20pt}
+		}
+	\end{singlespace}
+\end{titlepage}
+\newpage
+\pagenumbering{arabic}
+\tableofcontents
+\clearpage
+
+\input{writing/intro}
+\input{writing/interrupts}
+\input{writing/io}
+\input{writing/processes}
+\input{writing/conclusion}
+
+\end{document}
+
+
+
+
+

OSU Coursework/CS 444 - Operating Systems II/Writing Assignments/Final Paper/writing/conclusion.tex

@@ -0,0 +1,2 @@
+\section{Conclusion}
+After reviewing the core implementations and features of how Windows, Linux, and FreeBSD handle process, I/O, and interrupts it is easy to see that at their cores every operating system has the same goals in mind. Every system needs certain core features that all have to work in a standardized way (even if it is only to one family of operating systems) so that developers and hardware manufacturers all know what and how they should be developing their applications and how those applications will run. Certain features, like I/O, are vastly different between Windows and Unix based operating systems but still offer similar lower level features such as a hardware abstraction layer, cryptography API, and kernel level queuing of I/O operations. Trade-offs made between these different operating systems are largely subjective and ultimately they all work well, otherwise they wouldn't be as popular as they are. Many programmers will skip out on learning about these low-level features, and for a large percentage of these people, it may never matter. For those wishing to get the best performance, efficiency, and security out of their applications however, a working knowledge of operating systems is a must and the topics covered here are a great start of the many kernel sub-systems that should be studied. This becomes astronomically more important for any computer scientists who end up in a field developing drivers or embedded firmware. 

OSU Coursework/CS 444 - Operating Systems II/Writing Assignments/Final Paper/writing/interrupts.tex

@@ -0,0 +1,14 @@
+\section{Interrupts}
+\subsection{Overview}
+Interrupts are a useful hardware feature of processors that are often overlooked by computer scientists and software developers. Interrupts do exactly what the name implies and interrupts whatever the processor is doing to instead do something else. Many kinds of interrupts exists such as timers that cause an interrupt after a certain period of time has elapsed, flag based interrupts that cause the processor to do something else if execution fails, or external hardware interrupts such as from a PS2 port that tells the processor it needs to stop to process some input from a device like a keyboard or mouse. Without interrupts, much of what makes a modern computer as powerful and versatile would not be possible as it would require a massive singular sequential polling loop to process every potential operation on the computer. It's not hard to imagine how difficult generating a screen full of graphics if the processor could be processing a math operation for seconds at a time, and all the while, graphical updates would stop. These interrupts are used on any modern processor from every major manufacturer 
+
+\subsection{Individual OS Implementations and Features}
+\subsubsection{Linux and FreeBSD}
+For both of these operating systems, the underlying implementation is the same as they're both Unix based and the core design was built off the same foundation. The system sets up interrupt service routines (ISRs) as handlers for an interrupt occurring. Every individual interrupt has its own unique service routine that is connected to a c function in the driver for the piece of hardware. When the interrupt actually triggers, there are two portions to consider that they call the top half and bottom half. The top half is what is run during the duration of the hardware interrupt. This part must be kept as short as possible so that it doesn't break the timing of other aspects of the system. This section could do something as simple as setting a flag, queuing data, or moving a chunk of data from the hardware to ram. The lower section is used if the top half doesn't have enough time to process whatever needs to happen in the interrupt. This also is only used when the interrupt isn't highly timing specific. This work that has to be performed will be scheduled whenever there is free time, but is allowed to be interrupted by other top half interrupts if one occurs during its run. Internally, these Unix kernels set up some hardware timers on the CPU that run on a regular and fast basis to trigger frequent interrupts. These are used to perform tasks such as context switching of processes, or handling higher level software interrupts for hardware of software created interrupts that don't need true hardware interruption
+
+\subsubsection{Windows}
+In Windows, interrupts are handled by what they call trap handlers with the traps being their name for when the CPU "captures" the running thread so it can transfer execution to the handler. The trap handlers in turn trigger and interrupt service routine (ISR) that handles the low level processing of whatever has triggered it. In most cases, the interrupt is caused by hardware, such as a PCI express device telling the processor that it has data that it needs to send to it. The trap handler in this case would be handed off to an ISR in the driver for the particular piece of hardware. Software interrupts on the other hand could be used to signal to a higher level application that a time as passed, or some other useful even has occurred, but not necessarily triggered by hardware. To determine which interrupt should run, it is looked up in the interrupt dispatch table (IDT) and then hands off to the driver's ISR. Windows interrupt handlers also have what are called deferred procedure calls (DPCs), that are a way to process an interrupt where the processing does not have to be real-time. The interrupt creates a DPC that is scheduled whenever there is free time on the processor. This allows the kernel to keep the ISR short for tasks that don't require precision timing, but still generate an interrupt to let the system know something is ready to process.
+
+\subsection{Comparisons}
+\subsubsection{Windows vs. Linux/FreeBSD}
+In this particular part of the operating system, the way that the Unix based and Windows kernels handle interrupts is actually pretty much the same. It's not hard to see the reason for this being that it's based more around how the hardware has to function than some arbitrary software decision made at a high level. Computers have strict timing requirements when it comes to interacting with hardware, and not obeying those requirements will lead to the OS crashing at best. In extreme circumstances, it could even cause hardware failure. In fact, I would not be surprised if the developers of the interrupt kernel systems were basing in on and Intel design document describing the best ways to implement the lowest level portion of the OS to be most efficient and be least likely to cause issues. It also makes sense when you consider that in older days of computing, interrupts were the only made to get seemingly concurrent things to happen, but the end user actually had access to these hardware systems directly. As computers got more and more complex, a framework was needed to keep all but driver developers from accidentally interacting with hardware systems they shouldn't be. 

OSU Coursework/CS 444 - Operating Systems II/Writing Assignments/Final Paper/writing/intro.tex

@@ -0,0 +1,3 @@
+\section{Introduction}
+Understanding the core constructs of a single operating system is incredibly useful to computer scientists and software developers. Knowing how the system kernel will internally handle a call to open a file, allocate memory, or handle system interrupts allows for more intelligent choices to be made by the programmer so that applications will be more efficient and less likely to crash. It is also incredibly important for security, which is an ever increasing threat in today's world where everything is connected to the Internet. Making a simple choice to use a safer alternative to a particular system call, when you as the developer know that it would leave the computer extra vulnerable in the case where someone bypassed the security of your program, will at minimum minimize potential fallout or in best cases negate the threat completely. Knowing this information about a single operating system is incredibly helpful, but when the world is running on many operating systems it's important to know the similarities and differences between how common features work across a multitude of them. While most programmers in the current age know very little about the underlying hardware they're writing code for, the ones that do have a significant head start in their quest for writing more stable, more secure, and more efficient applications and drivers. This will also be useful to anyone interested in developing kernel modules for Windows or Unix based OS's, or even those interested in doing microcontroller development. With this in mind, this document will cover the basics of process management, I/O, and interrupts in Windows, Linux, and FreeBSD as well as some simple comparisons between their implementations.
+

OSU Coursework/CS 444 - Operating Systems II/Writing Assignments/Final Paper/writing/io.tex

@@ -0,0 +1,14 @@
+\section{I/O}
+\subsection{Overview}
+For every operating system a way is needed to abstract complex physical hardware into easier to process logical items. All of these pieces of hardware perform some kind of I/O operation, whether it be memory, a graphics card, networking card, hard drive, or USB device. Abstraction layers are needed so that different providers for different hardware can conform to a standard such that developers don't need to learn a new API for every individual piece of hardware. This means that a keyboard made my Logitech will show up the same as a keyboard made by Microsoft. All the developer has to think about it interacting with a generic keyboard device and the driver abstracts away any of the hardware differences. In the following sections we'll take a look at how Windows, Linux, and FreeBSD handle their I/O as well as similarities and differences between the different operating systems. 
+
+\subsection{Individual OS Implementations and Features}
+\subsubsection{Linux and FreeBSD}
+Linux and FreeBSD both run kernels that share historical roots in Unix, meaning that the core of how I/O is handled is the same minus different naming conventions in the raw code itself. The most interesting feature of I/O on Unix based kernels is that everything is treated like a "file", or more accurately, accessed via file descriptors. This means that you can access hardware under an absolute path on the filesystem, but instead of reading and writing to a place on disk, the bytes read and written are actually massaged through kernel drivers to actually interact with the hardware. Reading and writing on Unix kernels in done in either structured (block) or unstructured (character) device modes. For block read/writes, an entire "block" (4kb for example) is sent or received at any given time. Character mode is just that, reading or writing a single character at a time. When it comes to cryptography, all the most common ones are available via the kernel such as RSA or AES. This largely is the case as hardware on CPUs is leveraged to do faster crypto calculations than if that processing were done without the extra hardware.
+
+\subsubsection{Windows}
+In Windows the most common data structure for interacting with drivers and hardware devices are IRPs or I/O request packets. These IRPs are created by the I/O manager built into windows that takes in a request made by a call to a DLL, such as request to send a network packet. The IRP stores useful header information about the I/O operation such as whether the request is synchronous or not, as well as the interface to the low level driver. Once this IRP is made, it exists until access to the resource is no longer needed. Due to the complexity of some I/O devices, it's possible for the IRPs to point to another device driver rather than a hardware device. For example, if using a pen drive that is encrypted, the first would be the abstracted IRP that sees the resource as a file, but that IRP would point to the cryptography driver which would transparently provide block level encryption / decryption from a driver level. For options when it comes to encryption, the most common and most used ones are available such as RSA, AES. This largely is the case as hardware on CPUs is leveraged to do faster crypto calculations than if that processing were done without the extra hardware.
+
+\subsection{Comparisons}
+\subsubsection{Windows vs FreeBSD/Linux}
+Due to the intense similarities between Linux and Unix, I'll only be comparing Unix-based vs. Windows here. The biggest similarity between the two operating systems is that there is abstraction from the hardware. In older computers, there was very little abstraction which made writing code, or an operating system for that matter very difficult. Everything had to be custom written to fit the exact configuration of the computer as it was on a user's table. By putting a layer of abstraction above this hardware, a single operating system or single source file could be easily ported from one computer (or even variant in the case of Linux vs Unix) to another, at the expense of processor time to manage a driver subsystem. Without this layer, computers and operating systems as we know them would be very different than what they actually are today. Beyond the simple fact that abstraction exists, there's very little that's similar between the way Windows handles IO vs Unix-based systems. Unix treats everything like a file, to the point that you can directly read/write to memory using /dev/mem if you have elevated privileges. Trying to do something like that on Windows would be incredibly difficult as Windows tries to hide that driver manager layer and doesn't provide easy access to hardware directly. One thing this means is that writing drivers for Linux is significantly easier and less time consuming than on Windows, as there is less overhead management of the drivers. Also, the open system source code also helps.

OSU Coursework/CS 444 - Operating Systems II/Writing Assignments/Final Paper/writing/processes.tex

@@ -0,0 +1,35 @@
+\section{Processes}
+\subsection{Overview}
+\subsection{Individual OS Implementations and Features}
+\subsubsection{FreeBSD}
+\paragraph{Processes}
+Processes in FreeBSD have a process ID, group, owner, and permissions that determine what the process can interact with. The process also knows what its parent ID is. If the process does not take any direct user input on a command line or GUI, the process is known as a daemon. When child processes are made, these child processes will die if the parent process dies. 
+\paragraph{Threads}
+FreeBSD actually implements two different implementations of threading. The first is M:N threading (hybrid threading) where the threads get placed onto their own kernel entities. This complicates the underlying kernel code for handling threads but allows for fast context switching as the threading library itself schedules the threads on the CPU. However, this also means that if the library isn't well or efficiently coordinated, thread efficiency can suffer. Because it is a complex threading method, it also makes code harder to maintain.
+The other threading library uses the 1:1 threading model where each thread has its own kernel threads. This model has the benefit of being able to split across multiple CPU cores, but can be slow to setup the threads themselves. The kernel also has a low limit on the number of threads that can be created in total, creating a bottleneck for systems such as servers that need to spawn many many processes with many threads. 
+
+\paragraph{Scheduling}
+The FreeBSD scheduler is called the ULE scheduler after the word schedULEr itself. This is a more recent scheduler in the total lifespan of FreeBSD but is now default on all the mainline distros. The scheduler is a queue based and cuts up the CPU into time slices that it shares among the queue using load balancing algorithms. Overall, it tries to be fair about how it schedules time on the CPU, but unlike older scheduling systems for the OS, it does have some prioritization built in so that important processes such as those that the user is currently interacting with can be given a little extra CPU time.
+
+\subsubsection{Windows}
+\paragraph{Processes}
+Processes in Windows are items with a unique process ID and at least one thread. They contain information about open handles to file system objects, virtual addresses for memory, and access token to ensure that the process is being run using the correct user. In Windows, child processes know very little about their parent process, literally just their parent's PID. If the parent dies, the child processes will continue running until they are terminated another way. 
+\paragraph{Threads}
+In Windows, there are multiple kinds of threads available. These are single, apartment, free, and a combination of both apartment and free. Single threaded threads run on the main process and cause code to block as other parts run. Apartment threading gives each thread its own "apartment" inside of the the larger process "building". This allows each thread to run independently and to share resources. Free threads are run as a multi-threaded apartment object and will incur a large communications penalty if threads needed to intercommunicate as the threads do not have direction access to each other. The final options which is a combination of both of the last two allows for direct access while multi-threading but without the performance issues of either.
+\paragraph{Scheduling}
+The Windows scheduled uses a multilevel queue to handle its processes. Processes are assigned a priority value from 0-31 with higher numbers being higher priority. When the scheduler tries to determine which thread to give time on the CPU, it first starts by checking the threads with the highest priority and giving any that are ready their time up to a maximum that's set at a system level. Once that priority level has no more processes that are ready, it goes to the next lower priority value. However, if a higher priority process becomes ready while that lower priority process is running, the lower priority process will be slept so the more important one can run. 
+
+\subsubsection{Linux}
+\paragraph{Processes}
+So far as I can tell, the core information that processes store is the same on linux as it is on FreeBSD. This includes the process ID, group, owner, and runtime permissions. Same as FreeBSD, child processes of a parent will die if the parent dies.
+\paragraph{Threads}
+Threads on linux are implemented using pthread. Essentially, pthread creates a clone of the process is it called from, allowing it to share the same memory space, but running a different section of code. This implementation is a 1:1 model where a new process is created for each thread, but can be changed depending on the exact version of linux being used to be part of the same parent thread instead. 
+\paragraph{Scheduling}
+The scheduler on linux is called the completely fair scheduler, or CFS for short. Unlike other operating systems, the CFS does not use traditional queues but instead uses a red/black tree where the indexes are the amount of time the process has spent on the CPU. Because of this implementation, the code for this scheduler is much more complex than other systems such as a queue. However, what this tree does allow for is very easy picking of which process needs to run next. Items on the left side of the tree are the ones that haven't had as much processor time, where the ones on the right have had enough time that they should continue waiting.
+
+\subsection{Comparisons}
+\subsubsection{FreeBSD vs. Linux}
+Ultimately, FreeBSD and Linux are both derived from a common background of Unix and it shows in that they're both quite similar. Process-wise, the two operating systems are nearly identical, and while they're not identical when it comes to threading, they both have overlap in that they either run or can run in 1:1 threading models where a unique kernel thread is made for each. The biggest difference between the two operating systems seems to be that the schedulers are extremely different. While FreeBSD uses a queue based system, linux's red/black fair scheduler is a much more complex scheduler. Ultimately, to me it looks like FreeBSD is more focused around maintaining backwards compatibility while providing the most flexibility and options to the user. Linux on the other hand very much seems to be more pointed towards working out over time what the BEST solution to each problem is, whether it be threading or scheduling, and making it the primary way the OS functions in the expectation that by designing the OS right the first time, the user won't have to worry about it as much. 
+
+\subsubsection{Windows vs. Linux}
+Windows and Linux are surprisingly different in terms of how each system handles their processes, threads, and scheduling. At their root, there are definitely similar features such as process IDs, child processes, and runtime permissions but ultimately they're implemented in very different ways. The biggest two differences are in how threading is handled and the way their schedulers work. Windows has more in common with FreeBSD in this context, where the threads can be adapted to whichever mode is most useful and efficient to the user at any given time, and is a stark contrast to linux's 1:1 mode. The schedulers couldn't really be more different. Linux focuses on maintaining fair CPU usage time on the processor where Windows is all about making sure processes that are deemed more important get more time, quite explicitly. Ultimately, I can understand why both OSes would choose to take the routes they did. For Windows, they wanted to make sure lower level important system processes could never be overrun by less important user processes. And in Linux, this essentially was handled by making sure that both user and system processes couldn't take up too much time because they'd always have to share. It's really two different valid solutions to the same problem.