Lab 8: Systemd

Task 1: Create a shell script

• Create a custom web server with bash. The web server displays the processes running on the server by executing the top command.

  $ sudo vim /usr/bin/script.sh 
  

• Add the following to the file.

  
  
  
  
  
  
  
  
  
  #!/bin/bash
  
  while true;
    do dd if=/dev/zero of=/dev/null
  done &
  
  while true;
    do echo -e "HTTP/1.1 200 OK\n\n$(top -bn1)" \
    | nc -l -k -p 8080 -q 1;
  done
  

  dd has been added for a purpose which you will see in further sections.
  Consider the web server as a means of remotely viewing the resource usage of dd.

• Save the script and set execute permission.

  $ sudo chmod +x /usr/bin/script.sh 
  

You have to manually run this script whenever the system is restarted. We can solve this by creating a systemd service for it. This will give us more options for managing the script’s execution.

Task 2: Create a systemd file

Next, create a systemd service file for the script on your system. This file must have .service extension and saved under the /lib/systemd/system/ directory

$ sudo vim /lib/systemd/system/shellscript.service 

Now, add the following content and update the script filename and location. You can also change the description of the service. After that, save the file and close it.

[Unit]
Description=My custom web service to show system processes

[Service]
ExecStart=/usr/bin/script.sh

[Install]
WantedBy=multi-user.target

The [Unit] section describes the service, specifies the ordering dependencies, as well as conflicting units. In [Service], a sequence of custom scripts is specified to be executed during unit activation, on stop, and on reload. Finally, the [Install] section lists units that depend on the service.

Task 3: Enable the service

• Your systemd service has been added to your system. Let’s reload the systemctl daemon to read new files. You need to reload the configuration file of a unit each time after making changes in any *.service file.

  $ sudo systemctl daemon-reload 
  

• Enable the service to start on system boot and also start the service using the following commands.

  $ sudo systemctl enable shellscript.service 
  $ sudo systemctl start shellscript.service 
  

  When you enable a service, a symbolic link of that service is created in the /etc/systemd/system/multi-user.target.wants directory.

• Finally, verify that the script is up and running as a systemd service.

  $ sudo systemctl status shellscript.service 
  

• Example output when we access the web service via port 8080 is shown below.
  
• The dd process shown in the output was executed by the service we just created. This process is using 100% of the CPU. We can fix this problem by making use of systemd control groups.

Task 4: Systemd control groups (cgroups)

Systemd control group is a mechanism that allows you to control the use of system resources by a group of processes.

• You can view the cgroup of a service with the systemctl status <service_name> command.
  Let’s view the status of our custom system service shellscript.service again.

  $ systemctl status shellscript.service
  

  You get an output similar to the following

  ● shellscript.service - My Shell Script
     Loaded: loaded (/lib/systemd/system/shellscript.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2022-10-23 19:39:02 +04; 1s ago
   Main PID: 6821 (script.sh)
      Tasks: 4 (limit: 9415)
     Memory: 1.0M
        CPU: 1.945s
     CGroup: /system.slice/shellscript.service
             ├─6821 /bin/bash /usr/bin/script.sh
             ├─6822 /bin/bash /usr/bin/script.sh
             ├─6824 nc -l -k -p 8080 -q 1
             └─6825 dd if=/dev/zero of=/dev/null
  

  The output shows that shellscript.service is under the system.slice control group.
  The second line 6821 /bin/bash /usr/bin/script.sh shows the process ID and the command used to start shellscript.service.
  Subsequent lines under the CGroup are the other commands executed in the service.

• View your system’s cgroup hierarchy

  $ systemctl status
  

• You can view the system resource usage by each cgroup

  $ systemd-cgtop
  

• Slices allow one to create a hierarchical structure in which relative shares of resources are defined for the entities that belong to those slices.
  View a list of all systemd slices

  $ systemctl -t slice --all
  

• Create a systemd slice at /etc/systemd/system/testslice.slice. Add the following to the file.

  
  
  
  
  
  
  
  
  [Unit]
  Description=Custom systemd slice for SNA lab on systemd.
  Before=slices.target
  
  [Slice]
  MemoryAccounting=true
  CPUAccounting=true
  MemoryMax=10%
  CPUQuota=10%
  

  This slice will set CPU and memory usage limit to all processes running under it. As seen in the configuration, a maximum of 10% of memory and CPU resources can be used by the processes running under this control group testslice.slice.

• Let’s add our custom service shellscript.serviceto this new control group.
  Modify the service file /lib/systemd/system/shellscript.service to use this systemd slice by adding the line Slice=testslice.slice to the [Service] section as shown below:

  
  
  
  
  
  
  
  
  [Unit]
  Description=My custom web service to show system processes
  
  [Service]
  ExecStart=/usr/bin/script.sh
  Slice=testslice.slice
  
  [Install]
  WantedBy=multi-user.target
  

• Reload the daemon and restart the service to apply the changes

  $ sudo systemctl daemon-reload
  $ sudo systemctl restart shellscript.service
  

• Refresh the web page. The output shows that dd now uses less than 10% of the CPU.
  

• Run $ systemd-cgtop to view resource usage by cgroups.

  Control Group                            Tasks   %CPU   Memory  Input/s Output/s
  /                                          774   13,9     1.7G        -        -
  testslice.slice                              2    9,9   956.0K        -        -
  testslice.slice/shellscript.service          2    9,9   576.0K        -        -
  user.slice                                 489    3,1     1.5G        -        -
  user.slice/user-1000.slice                 489    3,4     1.3G        -        -
  

• View the hierarchy and other artifacts about the cgroup.

  $ systemctl -t slice --all
  $ systemctl status
  

• View systemd log for your service

  $ journalctl -u shellscript.service
  

Questions to answer

Upload the scripts you create to moodle.
Show test results of your implementation in your report.

1. Show the following boot-up performance statistics on your system:

   • Time spent in the kernel space before the user space was reached.
   • Show an SVG image that contains services that have been started, and how long it took for them to initialize.

2. Take the systemd unit graphical.target as your starting point, start tracing backwards using only the Requires variable. At what systemd unit do you reach a dead end where there is no more Requires variable?

   • Provide brief explanation for each of the systemd units you encounter while performing this trace.
   • The unit at this dead end Wants some systemd units. Why does it want these units?

   Show screenshots of every step as you trace.

3. Create a simple web server in bash that shows the following: system uptime, inode usage, current memory, disk space usage statistics, and the last 15 lines of /var/log/syslog.

   • The required information should be queried from the server everytime a user opens or refreshes the page.
   • You do not need to save the results anywhere. Users only need live updates when the server is visited.
   • The results should be displayed on a single page in an orderly manner that is easy to read.
   • Create a systemd service on your system to run this script (web server). Show how you can start your new service, and configure it to run after system reboot.
   • Your systemd service should restart the web server if the web server crashes or is killed.
   • This service is allowed to use a maximum of 15% of the CPU and 256MB memory.

   Show all steps taken, and all unit files created in your report.
   At the end of this task, you must have at least one bash script, one service file, and one slice file all working together to achieve the objectives.

4. Create a systemd service that will update your package sources list from the repository.

   • The service should update the package source list five minutes after booting, and then every day after that.
   • The schedule of the execution should be done with only systemd.

Bonus

5. Create a custom target in /etc/systemd/system/<your_target>.target.
   • Add a description of the target file.
   • Create a directory /etc/systemd/system/<your_target>.wants/
   • Create sylinks to additional services you wish to enable in this new directory. It should be a symlink to services from /usr/lib/systemd/system/ that you wish to enable.