Skip to content

Commit

Permalink
documentation page linking complete
Browse files Browse the repository at this point in the history
  • Loading branch information
briansalehi-zz committed Jun 11, 2018
1 parent e5708cf commit a5d5c40
Show file tree
Hide file tree
Showing 25 changed files with 179 additions and 1 deletion.
6 changes: 5 additions & 1 deletion docs/Introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@ The OS is not compatible with the older versions of the Raspberry Pi, because ne

One major drawback of any technical book is that very soon after release each book becomes obsolete. Technology nowadays is evolving so fast that it is almost impossible for book writers to keep up with it. That's why I like the idea of an "open source book" - a book that is freely available on the internet and encourages its readers to participate in content creation and validation. If the book content is available on Github, it is very easy for any reader to fix and develop new code samples, update the book content, and participate in writing new chapters. I understand that right now the project is not perfect, and at the time of writing it is even not finished. But I still want to publish it now, because I hope that with the help of the community I will be able to not only complete the project faster but also to make it much better and much more useful than it was in the beginning.

##### Previous Page

[Main Page](https://github.com/s-matyukevich/raspberry-pi-os#learning-operating-system-development-using-linux-kernel-and-raspberry-pi)

##### Next Page

[Contribution guide](../docs/Contributions.md)
[Contributing to the Raspberry PI OS](../docs/Contributions.md)
7 changes: 7 additions & 0 deletions docs/Prerequisites.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,10 @@ Strictly speaking, Docker is not a required dependency. It is just convenient to

If for some reasons you want to avoid using Docker, you can install the [make utility](http://www.math.tau.ac.il/~danha/courses/software1/make-intro.html) as well as `aarch64-linux-gnu` toolchain. If you are using Ubuntu you just need to install `gcc-aarch64-linux-gnu` and `build-essential` packages.

##### Previous Page

[Contribution guide](../docs/Contributions.md)

##### Next Page

1.1 [Kernel Initialization: Introducing RPi OS, or bare metal "Hello, world!"](../docs/lesson01/rpi-os.md)
7 changes: 7 additions & 0 deletions docs/lesson01/exercises.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,10 @@ Exercises are optional, though I strongly recommend you to experiment with the s
1. Try to use all 4 processor cores. The OS should print `Hello, from processor <processor index>` for all of the cores. Don't forget to set up a separate stack for each core and make sure that Mini UART is initialized only once. You can use a combination of global variables and `delay` function for synchronization.
1. Adapt lesson 01 to run on qemu. Check [this](https://github.com/s-matyukevich/raspberry-pi-os/issues/8) issue for reference.

##### Previous Page

1.4 [Kernel Initialization: Linux startup sequence](../../docs/lesson01/linux/kernel-startup.md)

##### Next Page

2.1 [Processor initialization: RPi OS](../../docs/lesson02/rpi-os.md)
8 changes: 8 additions & 0 deletions docs/lesson01/linux/build-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,3 +210,11 @@ Wow, it was a long journey inside kernel build system internals! Still, we skipp
1. How `vmlinux` is linked from all top-level `build-in.o` files.

My main goal was that after reading this chapter you will gain a general understanding of all above points.

##### Previous Page

1.2 [Kernel Initialization: Linux project structure](../../../docs/lesson01/linux/project-structure.md)

##### Next Page

1.4 [Kernel Initialization: Linux startup sequence](../../../docs/lesson01/linux/kernel-startup.md)
7 changes: 7 additions & 0 deletions docs/lesson01/linux/kernel-startup.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,10 @@ And indeed, if you search for `bcm2835-aux-uart` in the Linux source code, you c

You can think about this chapter as a preparation for reading `arm64` boot code - without understanding the concepts that we've just explored you would have a hard time learning it. In the next lesson, we will go back to the `stext` function and examine in details how it works.

##### Previous Page

1.3 [Kernel Initialization: Kernel build system](../../../docs/lesson01/linux/build-system.md)

##### Next Page

1.5 [Kernel Initialization: Exercises](../../../docs/lesson01/exercises.md)
8 changes: 8 additions & 0 deletions docs/lesson01/linux/project-structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,11 @@ Next, let's take a look at the folders that we can find inside the Linux reposit
* [fs](https://github.com/torvalds/linux/tree/v4.14/fs) You can look here to find different filesystem implementations.

This explanation is very high-level, but this is enough for now.In the next chapter, we will try to examine Linux build system in some details.

##### Previous Page

1.1 [Kernel Initialization: Introducing RPi OS, or bare metal "Hello, world!"](../../../docs/lesson01/rpi-os.md)

##### Next Page

1.3 [Kernel Initialization: Kernel build system](../../../docs/lesson01/linux/build-system.md)
5 changes: 5 additions & 0 deletions docs/lesson01/rpi-os.md
Original file line number Diff line number Diff line change
Expand Up @@ -481,5 +481,10 @@ Now, as we have gone through all source code, it is time to see how it works. To
1. Power on your Raspberry PI (This can be done using the same USB to TTL serial cable)
1. Open your terminal emulator. You should be able to see `Hello, world!` message there.

##### Previous Page

[Prerequisites](../../docs/Prerequisites.md)

##### Next Page

1.2 [Kernel Initialization: Linux project structure](../../docs/lesson01/linux/project-structure.md)
8 changes: 8 additions & 0 deletions docs/lesson02/exercises.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,11 @@
1. Instead of jumping directly from EL3 to EL1, try to first get to EL2 and only them switch to EL1.
1. One issue that I ran into when working on this lesson was that if FP/SIMD registers are used then everything works well at EL3, but as soon as you get to EL1 print function stops working. This was the reason why I've added [-mgeneral-regs-only](https://github.com/s-matyukevich/raspberry-pi-os/blob/master/src/lesson02/Makefile#L3) parameter to the compiler options. Now I want you to remove this parameter and reproduce this behavior. Next, you can use `objdump` tool to see how exactly gcc make use of FP/SIMD registers in the absence of `-mgeneral-regs-only` flag. Finally, I want you to use 'cpacr_el1' to allow ussing FP/SIMD registers.
1. Adapt lesson 02 to run on qemu. Check [this](https://github.com/s-matyukevich/raspberry-pi-os/issues/8) issue for reference.

##### Previous Page

2.2 [Processor initialization: Linux](../../docs/lesson02/linux.md)

##### Next Page

3.1 [Interrupt handling: RPi OS](../../docs/lesson03/rpi-os.md)
7 changes: 7 additions & 0 deletions docs/lesson02/linux.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,3 +300,10 @@ The last 3 functions are very important, but they all are related to virtual mem

In this chapter, we briefly discussed how a processor is initialized when the Linux kernel is booted. In the next lesson, we will continue to closely work with the ARM processor and investigate a vital topic for any OS: interrupt handling.

##### Previous Page

2.1 [Processor initialization: RPi OS](../../docs/lesson02/rpi-os.md)

##### Next Page

2.3 [Processor initialization: Exercises](../../docs/lesson02/exercises.md)
8 changes: 8 additions & 0 deletions docs/lesson02/rpi-os.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,3 +153,11 @@ Usually `spsr_el3` is saved automatically when an exception is taken to EL3. How
### Conclusion

That is pretty much it: when we enter `el1_entry` function the execution should be already at EL1 mode. Go ahead and try it out!

##### Previous Page

1.5 [Kernel Initialization: Exercises](../../docs/lesson01/exercises.md)

##### Next Page

2.2 [Processor initialization: Linux](../../docs/lesson02/linux.md)
8 changes: 8 additions & 0 deletions docs/lesson03/exercises.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,11 @@
1. Use local timer instead of the system timer to generate processor interrupts.
1. Handle MiniUART interrupts. Replace the final loop in the `kernel_main` function with a loop that does nothing. Setup MiniUART device to generate an interrupt as soon as the user types a new character. Implement an interrupt handler that will be responsible for printing each newly arrived character on the screen.
1. Adapt lesson 03 to run on qemu. Check [this](https://github.com/s-matyukevich/raspberry-pi-os/issues/8) issue for reference.

##### Previous Page

3.4 [Interrupt handling: Timers](../../docs/lesson03/linux/timer.md)

##### Next Page

4.1 [Process scheduler: RPi OS Scheduler](../../docs/lesson04/rpi-os.md)
7 changes: 7 additions & 0 deletions docs/lesson03/linux/interrupt_controllers.md
Original file line number Diff line number Diff line change
Expand Up @@ -304,3 +304,10 @@ static void bcm2836_chained_handle_irq(struct irq_desc *desc)

You can think about this code as an advanced version of what we did [here](https://github.com/s-matyukevich/raspberry-pi-os/blob/master/src/lesson03/src/irq.c#L39) for the RPi OS. [get_next_armctrl_hwirq](https://github.com/torvalds/linux/blob/v4.14/drivers/irqchip/irq-bcm2835.c#L217) uses all 3 pending registers to figure out which interrupt was fired. [irq_linear_revmap](https://github.com/torvalds/linux/blob/v4.14/include/linux/irqdomain.h#L377) uses irq domain to translate hardware irq number into Linux irq number and [generic_handle_irq](https://github.com/torvalds/linux/blob/v4.14/include/linux/irqdesc.h#L156) just executes irq handler. Irq handler was set in the initialization function and it points to [handle_level_irq](https://github.com/torvalds/linux/blob/v4.14/kernel/irq/chip.c#L603) that eventually executes all irq actions associated with the interrupt (this is actually done [here](https://github.com/torvalds/linux/blob/v4.14/kernel/irq/handle.c#L135)). For now, the list of irq actions is empty for all supported interrupts - a driver that is interested in handling some interrupt should add an action to the appropriate list. In the next chapter, we are going to see how this is done using system timer as an example.

##### Previous Page

3.2 [Interrupt handling: Low-level exception handling in Linux](../../../docs/lesson03/linux/low_level-exception_handling.md)

##### Next Page

3.4 [Interrupt handling: Timers](../../../docs/lesson03/linux/timer.md)
7 changes: 7 additions & 0 deletions docs/lesson03/linux/low_level-exception_handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -369,3 +369,10 @@ Next, we need to look at [handle_arch_irq](https://github.com/torvalds/linux/blo

As a conclusion, I can say that we've already explored the low-level interrupt handling code and trace the path of an interrupt from the vector table all the way to the `handle_arch_irq`. This is the point were an interrupt leaves architecture specific code and started to be handled by a driver code. Our goal in the next chapter will be to trace the path of a timer interrupt through the driver source code.

##### Previous Page

3.1 [Interrupt handling: RPi OS](../../../docs/lesson03/rpi-os.md)

##### Next Page

3.3 [Interrupt handling: Interrupt controllers](../../../docs/lesson03/linux/interrupt_controllers.md)
7 changes: 7 additions & 0 deletions docs/lesson03/linux/timer.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,3 +258,10 @@ A few important things are done in this function:

Now you see how long is the way of an ordinary timer interrupt, but we followed it from the beginning to the very end. One of the things that are the most important, is that we finally reached the place were the shceduler is called. The scheduler is one of the most critical parts of any operating system and it relies heavily on timer interrupts. So now, when we've seen where the scheduler functionality is triggered, its time to discuss its implementation - that is something we are going to do in the next lesson.

##### Previous Page

3.3 [Interrupt handling: Interrupt controllers](../../../docs/lesson03/linux/interrupt_controllers.md)

##### Next Page

3.5 [Interrupt handling: Exercises](../../../docs/lesson03/exercises.md)
7 changes: 7 additions & 0 deletions docs/lesson03/rpi-os.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,3 +176,10 @@ Here we first update compare register so that that next interrupt will be genera

The last thing that you might want to take a look at is the [kernel_main](https://github.com/s-matyukevich/raspberry-pi-os/blob/master/src/lesson03/src/kernel.c#L7) function where all previously discussed functionality is orchestrated. After you compile and run the sample it should print "Timer interrupt received" message after an interrupt is taken. Please, try to do it by yourself and don't forget to carefully examine the code and experiment with it.

##### Previous Page

2.3 [Processor initialization: Exercises](../../docs/lesson02/exercises.md)

##### Next Page

3.2 [Interrupt handling: Low-level exception handling in Linux](../../docs/lesson03/linux/low_level-exception_handling.md)
7 changes: 7 additions & 0 deletions docs/lesson04/exercises.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,10 @@
1. Allow the kernel to have an unlimited number of tasks (right now the limit is 64).
1. Adopt lesson 04 to run on qemu. Check [this](https://github.com/s-matyukevich/raspberry-pi-os/issues/8) issue for reference.

##### Previous Page

4.4 [Process scheduler: Scheduler](../../docs/lesson04/linux/scheduler.md)

##### Next Page

5.1 [User processes and system calls: RPi OS](../../docs/lesson05/rpi-os.md)
7 changes: 7 additions & 0 deletions docs/lesson04/linux/basic_structures.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,10 @@ I have already shown you how all general purpose registers are saved on the stac

There are far more important structures, algorithms and concepts related to scheduling, but what we've just explored is a minimal set, required to understand the next two chapters. Now it's time to see how all of this actually works.

##### Previous Page

4.1 [Process scheduler: RPi OS Scheduler](../../../docs/lesson04/rpi-os.md)

##### Next Page

4.3 [Process scheduler: Forking a task](../../../docs/lesson04/linux/fork.md)
7 changes: 7 additions & 0 deletions docs/lesson04/linux/fork.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,3 +148,10 @@ That's it about `copy_thread` function. Now let's return to the place in the for

Now, when you understand how new tasks are created and added to the scheduler, it is time to take a look on how the scheduler itself works and how context switch is implemented - that is something we are going to explore in the next chapter.

##### Previous Page

4.2 [Process scheduler: Scheduler basic structures](../../../docs/lesson04/linux/basic_structures.md)

##### Next Page

4.4 [Process scheduler: Scheduler](../../../docs/lesson04/linux/scheduler.md)
7 changes: 7 additions & 0 deletions docs/lesson04/linux/scheduler.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,10 @@ We have already seen so many examples were `schedule` is used, so now you are pr

Now we are done with the Linux scheduler. The good thing is that it appears to be not soo difficult if you focus only on the very basic workflow. After you understand the basic workflow you probably might want to to make another path through the schedule code and pay more attention to the details, because there are so many of them. But for now, we are happy with our current understanding and ready to move to the following lesson, which describes user processes and system calls.

##### Previous Page

4.3 [Process scheduler: Forking a task](../../../docs/lesson04/linux/fork.md)

##### Next Page

4.5 [Process scheduler: Exercises](../../../docs/lesson04/exercises.md)
7 changes: 7 additions & 0 deletions docs/lesson04/rpi-os.md
Original file line number Diff line number Diff line change
Expand Up @@ -663,3 +663,10 @@ The described above sequence of steps is very important - I personally consider
We are done with scheduling, but right now our kernel can manage only kernel threads: they are executed at EL1 and can directly access any kernel functions or data. In the next 2 lessons we are going fix this and introduce system calls and virtual memory.
##### Previous Page
3.5 [Interrupt handling: Exercises](../../docs/lesson03/exercises.md)
##### Next Page
4.2 [Process scheduler: Scheduler basic structures](../../docs/lesson04/linux/basic_structures.md)
8 changes: 8 additions & 0 deletions docs/lesson05/exercises.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,11 @@
1. When a task is executed in a user mode, try to access some of the system registers. Make sure that a synchronous exception is generated in this case. Handle this exception, use `esr_el1` register to distinguish it from a system call.
1. Implement a new system call that can be used to set current task priority. Demonstrate how priority changes are dynamically applied while the task is running.
1. Adapt lesson 05 to run on qemu. Check [this](https://github.com/s-matyukevich/raspberry-pi-os/issues/8) issue for reference.

##### Previous Page

5.2 [User processes and system calls: Linux](../../docs/lesson05/linux.md)

##### Next Page

6.1 [Virtual memory management: RPi OS](../../docs/lesson06/rpi-os.md)
7 changes: 7 additions & 0 deletions docs/lesson05/linux.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,3 +199,10 @@ The important things here are the first line, were interrupts are disabled, and

We've now gone through the process of generating and processing a system call. This process is relatively simple, but it is vital for the OS, because it allows the kernel to set up an API and make sure that this API is the only mean of communication between a user program and the kernel.

##### Previous Page

5.1 [User processes and system calls: RPi OS](../../docs/lesson05/rpi-os.md)

##### Next Page

5.3 [User processes and system calls: Exercises](../../docs/lesson05/exercises.md)
7 changes: 7 additions & 0 deletions docs/lesson05/rpi-os.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,3 +299,10 @@ Following Linux convention, we are not deleting the task at once but set its sta

Now, when RPi OS can manage user tasks, we become much closer to the full process isolation. But one important step is still missing: all user tasks share the same physical memory and can easily read one another's data. In the next lesson, we are going to introduce virtual memory and fix this issue.

##### Previous Page

4.5 [Process scheduler: Exercises](../../docs/lesson04/exercises.md)

##### Next Page

5.2 [User processes and system calls: Linux](../../docs/lesson05/linux.md)
9 changes: 9 additions & 0 deletions docs/lesson06/exercises.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
## 6.3 Exercises

1. Adapt lesson 06 to run on qemu. You have to implement [identity mapping](https://wiki.osdev.org/Identity_Paging) in order to do this. Check [this](https://github.com/s-matyukevich/raspberry-pi-os/issues/8) issue for reference.

##### Previous Page

6.2 Virtual memory management: Linux (in progress)
6.1 jump backward to [Virtual memory management: RPi OS](../../docs/lesson06/rpi-os.md)

##### Next Page

7.1 Signals and interrupt waiting: RPi OS (to be done)
7 changes: 7 additions & 0 deletions docs/lesson06/rpi-os.md
Original file line number Diff line number Diff line change
Expand Up @@ -705,4 +705,11 @@ So in the first 2 lines of the `do_mem_abort` function, we check whether the cur

This was a long and difficult chapter, but I hope it was useful as well. Virtual memory is really one of the most fundamental pieces of any operating system and I am glad we've passe thought this chapter and, hopefully, start to understand how it works at the lowest level. With the introduction of virtual memory now we have full process isolation, but the RPi OS is still far from completion. It still doesn't support file systems, drivers, signals and interrupt waitlists, networking and a lot of other useful concepts, and we will continue to uncover them in the upcoming lessons.

##### Previous Page

5.3 [User processes and system calls: Exercises](../../docs/lesson05/exercises.md)

##### Next Page

6.2 Virtual memory management: Linux (in progress)
6.3 jump forward to [Virtual memory management: Exercises](../../docs/lesson06/exercises.md)

0 comments on commit a5d5c40

Please sign in to comment.