# Lab 8: Systemd

## Task 1: Create a shell script

- Create a custom web server with bash. The web server displays the processes running on the server by executing the `top` command.

      $ sudo vim /usr/bin/script.sh

- Add the following to the file.

      #!/bin/bash

      while true;
        do dd if=/dev/zero of=/dev/null
      done &

      while true;
        do echo -e "HTTP/1.1 200 OK\n\n$(top -bn1)" \
        | nc -l -k -p 8080 -q 1;
      done

  > `dd` has been added for a purpose which you will see in further sections.
  > Consider the web server as a means of remotely viewing the resource usage of `dd`.

- Save the script and set execute permission.

      $ sudo chmod +x /usr/bin/script.sh

You have to manually run this script whenever the system is restarted. We can solve this by creating a systemd service for it. This will give us more options for managing the script’s execution.

## Task 2: Create a systemd file

Next, create a systemd service file for the script on your system. This file must have `.service` extension and saved under the `/lib/systemd/system/` directory

    $ sudo vim /lib/systemd/system/shellscript.service

Now, add the following content and update the script filename and location. You can also change the description of the service. After that, save the file and close it.

    [Unit]
    Description=My custom web service to show system processes

    [Service]
    ExecStart=/usr/bin/script.sh

    [Install]
    WantedBy=multi-user.target

> The [Unit] section describes the service, specifies the ordering dependencies, as well as conflicting units. In [Service], a sequence of custom scripts is specified to be executed during unit activation, on stop, and on reload. Finally, the [Install] section lists units that depend on the service.

## Task 3: Enable the service

- Your systemd service has been added to your system. Let’s reload the systemctl daemon to read new files. You need to reload the configuration file of a unit each time after making changes in any `*.service` file.

      $ sudo systemctl daemon-reload

- Enable the service to start on system boot and also start the service using the following commands.

      $ sudo systemctl enable shellscript.service
      $ sudo systemctl start shellscript.service

  > When you enable a service, a symbolic link of that service is created in the `/etc/systemd/system/multi-user.target.wants` directory.

- Finally, verify that the script is up and running as a systemd service.

      $ sudo systemctl status shellscript.service

- Example output when we access the web service via port 8080 is shown below.
- The `dd` process shown in the output was executed by the service we just created. This process is using 100% of the CPU. We can fix this problem by making use of systemd control groups.

## Task 4: Systemd control groups (cgroups)

Systemd control group is a mechanism that allows you to control the use of system resources by a group of processes.

- You can view the cgroup of a service with the `systemctl status <service_name>` command.
  Let’s view the status of our custom system service `shellscript.service` again.

      $ systemctl status shellscript.service

  You get an output similar to the following

      ● shellscript.service - My Shell Script
         Loaded: loaded (/lib/systemd/system/shellscript.service; enabled; vendor preset: enabled)
         Active: active (running) since Sun 2022-10-23 19:39:02 +04; 1s ago
       Main PID: 6821 (script.sh)
          Tasks: 4 (limit: 9415)
         Memory: 1.0M
            CPU: 1.945s
         CGroup: /system.slice/shellscript.service
                 ├─6821 /bin/bash /usr/bin/script.sh
                 ├─6822 /bin/bash /usr/bin/script.sh
                 ├─6824 nc -l -k -p 8080 -q 1
                 └─6825 dd if=/dev/zero of=/dev/null

  The output shows that `shellscript.service` is under the `system.slice` control group.
  The second line `6821 /bin/bash /usr/bin/script.sh` shows the process ID and the command used to start `shellscript.service`.
  Subsequent lines under the `CGroup` are the other commands executed in the service.

- View your system’s cgroup hierarchy

      $ systemctl status

- You can view the system resource usage by each cgroup

      $ systemd-cgtop

- Slices allow one to create a hierarchical structure in which relative shares of resources are defined for the entities that belong to those slices.
  View a list of all systemd slices

      $ systemctl -t slice --all

- Create a systemd slice at `/etc/systemd/system/testslice.slice`. Add the following to the file.

      [Unit]
      Description=Custom systemd slice for SNA lab on systemd.
      Before=slices.target

      [Slice]
      MemoryAccounting=true
      CPUAccounting=true
      MemoryMax=10%
      CPUQuota=10%

  This slice will set CPU and memory usage limit to all processes running under it. As seen in the configuration, a maximum of 10% of memory and CPU resources can be used by the processes running under this control group `testslice.slice`.

- Let’s add our custom service `shellscript.service` to this new control group.
  Modify the service file `/lib/systemd/system/shellscript.service` to use this systemd slice by adding the line `Slice=testslice.slice` to the `[Service]` section as shown below:

      [Unit]
      Description=My custom web service to show system processes

      [Service]
      ExecStart=/usr/bin/script.sh
      Slice=testslice.slice

      [Install]
      WantedBy=multi-user.target

- Reload the daemon and restart the service to apply the changes

      $ sudo systemctl daemon-reload
      $ sudo systemctl restart shellscript.service

- Refresh the web page. The output shows that `dd` now uses less than 10% of the CPU.

- Run `$ systemd-cgtop` to view resource usage by cgroups.

      Control Group                            Tasks   %CPU   Memory  Input/s Output/s
      /                                          774   13,9     1.7G        -        -
      testslice.slice                              2    9,9   956.0K        -        -
      testslice.slice/shellscript.service          2    9,9   576.0K        -        -
      user.slice                                 489    3,1     1.5G        -        -
      user.slice/user-1000.slice                 489    3,4     1.3G        -        -

- View the hierarchy and other artifacts about the cgroup.

      $ systemctl -t slice --all
      $ systemctl status

- View systemd log for your service

      $ journalctl -u shellscript.service

## Questions to answer

> Upload the scripts you create to moodle.
> Show test results of your implementation in your report.

1. Show the following boot-up performance statistics on your system:

   - Time spent in the kernel space before the user space was reached.
   - Show an SVG image that contains services that have been started, and how long it took for them to initialize.

2. Take the systemd unit `graphical.target` as your starting point, start tracing backwards using only the `Requires` variable. At what systemd unit do you reach a dead end where there is no more `Requires` variable?

   - Provide brief explanation for each of the systemd units you encounter while performing this trace.
   - The unit at this dead end `Wants` some systemd units. Why does it want these units?

   > Show screenshots of every step as you trace.

3. Create a simple web server in bash that shows the following: system uptime, inode usage, current memory, disk space usage statistics, and the last 15 lines of `/var/log/syslog`.

   - The required information should be queried from the server everytime a user opens or refreshes the page.
   - You do not need to save the results anywhere. Users only need live updates when the server is visited.
   - The results should be displayed on a single page in an orderly manner that is easy to read.
   - Create a systemd service on your system to run this script (web server). Show how you can start your new service, and configure it to run after system reboot.
   - Your systemd service should restart the web server if the web server crashes or is killed.
   - This service is allowed to use a maximum of 15% of the CPU and 256MB memory.

   > Show all steps taken, and all unit files created in your report.
   > At the end of this task, you must have at least one bash script, one service file, and one slice file all working together to achieve the objectives.

4. Create a systemd service that will update your package sources list from the repository.

   - The service should update the package source list five minutes after booting, and then every day after that.
   - The schedule of the execution should be done with only systemd.

### Bonus

5. Create a custom target in `/etc/systemd/system/<your_target>.target`.
   - Add a description of the target file.
   - Create a directory `/etc/systemd/system/<your_target>.wants/`
   - Create sylinks to additional services you wish to enable in this new directory. It should be a symlink to services from `/usr/lib/systemd/system/` that you wish to enable.