System configuration

How do I get an out-of-the-box working Linux audio workstation?

How do I build a real-time audio workstation on Linux?

The Ardour manual has a interesting introduction about getting the right computer system for digital audio. Below that article there is also a (somewhat Windows oriented) video by Richard Ames, which could be helpful to get a idea which things are involved for this topic. Summary: lots of things are involved for a well performing real-time audio workstation. It's not simply a matter of buying a computer with massive CPU and memory power, as the coming sections on this page shows.

A good starting point for optimizing your system for real-time audio on GNU/Linux, is the rtcqs script below.

After adjusting your system using the recommendations of rtcqs and with JACK (QjackCtl) and a2jmidid installed, the basics should be there for a well performing real-time audio system (depending on your hardware too of course).

Preliminary

The modifications discussed in the following sections have been tested thoroughly and a number of these modifications have been implemented in specialized distros like KXStudio and AVLinux. This doesn't guarantee though that these modifications work for all set-ups and these modifications shouldn't be used as is but should be adapted if possible to match your own set-up. It is also advised to try the modifications step by step, so do not implement them all at once. This way you can verify if each modification does what it claims to do. By implementing the modifications a step at a time it is easier to track down which modification is causing any unwanted behavior.

A good habit is to always make backups of any files you're about to modify. Take /etc/fstab for instance. Before adding the noatime mount parameter you might want to make a backup of your existing and working fstab file:

sudo cp /etc/fstab /etc/fstab.orig

This way you can restore the original file in the case you made such a typo that the system has become unbootable. The procedure of recovering an unbootable system itself is beyond the scope of this Wiki but there are plenty of resources that explain this in detail.

The kernel

Note about kernels

  • Modern kernels work pretty well without the RT patch, also for real-time pro audio usage. It's not strictly necessary anymore to install a real-time ('rt') kernel to get good results. Although the best results are still expected when using a real-time kernel. Try it, test it and decide for yourself.
  • With the rt kernel and the rtirq script you could prioritize IRQ threads but with a modern kernel it is possible to use the rtirq script with a generic kernel and the threadirqs kernel option.
  • Many distro kernels, and many third-party-provided real-time kernels, are configured without the 1000 Hz setting. As modern audio software relies on high resolution timers this is not a problem anymore, you really don't need to set the kernel timer frequency other than the default.

Status of your kernel

Run uname -a to find out which kernel you're running:

Linux bto 5.17.1-rt17 #1~focal1 SMP PREEMPT_RT Sat Apr 2 10:32:50 CEST 2022 x86_64 x86_64 x86_64 GNU/Linux

5.17.1-rt17 is the version number of the currently running kernel. The configuration file with which this kernel was created can be found in /boot/config-5.17.1-rt17. Open it, and check if it has the following characteristics:

CONFIG_HIGH_RES_TIMERS=y
CONFIG_NO_HZ_IDLE=y
CONFIG_PREEMPT=y # low-latency kernel
CONFIG_PREEMPT_RT=y # real-time kernel

CONFIG_HIGH_RES_TIMERS=y indicates your kernel has high resolution timer support, the CONFIG_NO_HZ_IDLE means your kernel is tickless and the CONFIG_PREEMPT options indicate you're either running a low-latency kernel (CONFIG_PREEMPT) or a real-time kernel (CONFIG_PREEMPT_RT_FULL).

If you can't find the config file for your kernel there is also the possibility to extract the config from /proc/config.gz if your kernel supports it (CONFIG_IKCONFIG_PROC=y):

gunzip -c /proc/config.gz > config-`uname -r`

Do I really need a real-time kernel?

With the kernel boot option threadirqs standard kernels create threaded IRQs which basically nullifies the need to use a real-time kernel in most of the cases. But a real-time kernel can still be very useful on a Linux audio platform. If you need the lowest latencies possible a real-time kernel is still an option. But if you're mainly recording, a standard kernel or an optimized kernel (like the low-latency kernel Ubuntu offers) will probably suffice.

See also: Do I need a realtime kernel to use realtime scheduling?

Using the threadirqs kernel option

This is only needed for so-called generic kernels, ie. standard kernels that are not tweaked for lowlatency performance. You can check if your kernel already includes this option with the following command:

$ grep -e "CONFIG_IRQ_FORCED_THREADING=y" -e "CONFIG_PREEMPT_RT=y" /boot/config-`uname -r`

If it returns CONFIG_IRQ_FORCED_THREADING=y and CONFIG_PREEMPT_RT=y then your kernel is using IRQ threading and you don't have to proceed with the following steps. If it only returns CONFIG_IRQ_FORCED_THREADING=y you can add the threadirqs boot option as described below. If the command returns nothing you have a kernel that cannot use threaded IRQs.

Open /etc/default/grub with your favorite editor as root. Look for the line that starts with GRUB_CMDLINE_LINUX_DEFAULT and add threadirqs to the list of options:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash threadirqs"

Save the file and update the grub configuration:

sudo update-grub

Now reboot and you should have threaded IRQs.

Disabling Spectre and Meltdown mitigations

:!: Warning: disabling these mitigations will make your machine less secure! Use with caution!

To work around the Spectre and Meltdown vulnerabilities several mitigations were built into the kernel. These mitigations can have a negative impact on the performance of your machine. To disable those mitigations and get the most out of your CPU's again you can add the following kernel parameter to your Grub configuration, add it to the value of either GRUB_CMDLINE_LINUX_DEFAULT or GRUB_CMDLINE_LINUX:

mitigations=off

Installing a real-time kernel

This section describes installing a real-time kernel on an existing Linux distribution. Some distributions contain packages you can install to get a real-time kernel, for others you'll need to compile it yourself. You can recognize real-time kernel versions by the added '-rt' or '-realtime' parts of the kernel version number.

Ubuntu

Ubuntu Studio comes with a non-real-time low-latency kernel. Unfortunately there are no packaged real-time kernels available but using the kernel build method described in this Wiki it should be fairly easy to build your own real-time kernel1).

Debian

Since Debian Wheezy an RT variant is also available in the stock repositories. These kernels incorporate Pengutronix patches (see DebianMultimedia Wiki).

Arch

Arch has rt kernels available in AUR.

Gentoo

An audio overlay can be found here https://github.com/gentoo-audio/audio-overlay.

Build your own real-time kernel

For rolling your own kernel, see Build your own real-time kernel.

Disabling resource-intensive daemons, services and processes

You should check if the following daemons, services or processes are running, and if so, consider stopping them and/or resort to alternatives:

  • CPU frequency scaling daemons that scale the frequency of the CPU depending on the CPU load could cause xruns in some cases. More recent versions of Jack1 (>= 0.118.0) are suffering less from xruns or run xrun free with a CPU frequency scaling daemon enabled that's set to a scaling governor like ondemand. A specialized CPU scaling daemon is in the works that depends on the DSP-load instead of the CPU load: jackfreqd.
  • Desktop Environments (DE): especially on less powerful machines it is recommended to use a lighter DE than Gnome, KDE or Unity. Possible alternatives include LXDE, XFCE or IceWM. Another option is to only use a lightweight Window Manager (WM) like Openbox or Fluxbox.
  • Compositing: same recommendation as for the DE you use. Try to avoid using a compositing manager like Compiz to lower the chance of running into xruns.
  • Gnome NetworkManager and WiFi: NetworkManager keeps scanning for new wireless networks in the background and this might cause xruns. The best option is to not use a wireless network manager in a low-latency real-time audio environment. If you really can't do without WiFi you probably won't have another option than to use wpa_supplicant.

CPU frequency scaling

If your CPU supports frequency scaling and the CPU frequency scaling governor is set to ondemand (which is the default on a lot of distros) you could run into xruns. The ondemand governor scales the frequency according to the CPU load, the more the load, the higher the frequency. But this is happening independently from the DSP load on your system so it could happen that the DSP load suddenly rises for instance, demanding more CPU power, and that the scaling daemon kicks in too late, resulting in xruns because the DSP load maxes out. A solution would be to use a CPU frequency scaling daemon that scales the frequency according to the DSP load on your system like jackfreqd or to simply disable CPU frequency scaling altogether. The latter can be achieved by setting the scaling governor to performance.

To check which governor is active:

cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

Setting the governor to performance:

echo -n performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

You could create a Systemd Unit that does this for you. Example of an rt-audio-setup Unit file, create as /etc/systemd/system/rt-audio-setup.service:

[Unit]
Description=Prepare system for real-time audio

[Service]
Type=oneshot
ExecStart=/bin/bash /usr/local/bin/rt-audio-setup
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

This will start /usr/local/bin/rt-audio-setup at boot so you can add the aformentioned line regarding setting the CPU's to the performance scaling governor there, make the file executable, enable the Systemd service and start it.

echo 'echo -n performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor' | sudo tee /usr/local/bin/rt-audio-setup
sudo chmod +x /usr/local/bin/rt-audio-setup
sudo systemctl enable rt-audio-setup
sudo systemctl start rt-audio-setup

The rt-audio-setup script could then also be used to set up other things.

Ubuntu

On Ubuntu systems the command in /usr/local/bin/rt-audio-setup file only works if you disable the ondemand service:

sudo systemctl mask ondemand.service
Debian

On Debian you can control the scaling governor with the cpufreq-set utility which is part of the cpufrequtils package:

sudo cpufreq-set -r -g performance

On installation the package also installs an init script in /etc/init.d/cpufrequtils and a configuration file in /etc/default/cpufrequtils. To have the governor always set to performance make sure the configuration file looks as follows:

ENABLE="true"
GOVERNOR="performance"
MAX_SPEED="0"
MIN_SPEED="0" 
BIOS

Most BIOSes allow to disable CPU frequency scaling altogether and as a result your CPU will run at its maximum clock rate. For Intel CPUs you should look for a SpeedStep option. Its AMD equivalents are either Cool'n'Quiet or PowerNow!.

If you really need to prevent the CPU from running hot (to prevent shutdowns or overheating) or if you prefer your PC's to consume less power then you can also adjust some BIOS settings (if your BIOS has them):

  • Disable PCI and/or PCIe power-management in the BIOS
  • Disable EIST and C1E halt-states

With these settings disabled the ondemand governor should work without causing any xruns.

Hardware support modules

These are modules in the Linux kernel which support various hardware. All of them will reserve resources. If, for instance, you have an HDMI audio port which you don't need, you can blacklist it to disable it. However, you should check the IRQ usage first (see related section of this document), because this might be the more vital change to make, if you do in fact have a conflict.

Example script to disable resource-intensive daemons, services and processes

There are quite some processes and drivers running that you might not need when making music. Below an extremely simple example of a Bash script to disable all unneeded things. Note that it includes a solution for IRQ conflict, documented further down.

:!: The script below is an example, do not use as is but first adapt it to your own system

#!/bin/bash

# Remove unneeded kernel modules
modules="module1 module2 module3"

for module in $modules; do
  modprobe -r $module
done

# Unbind Bluetooth
echo -n "2-1.6" > /sys/bus/usb/drivers/usb/unbind

# Disable CPU frequency scaling
systemctl mask ondemand

# Set scaling governor to performance
echo -n performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

You could run this script once you're logged in (it needs root privileges though) or run it at boot, see the rt-audio-setup suggestion above.

The modules and processes differ of course on a per machine basis so first figure out which modules are loaded with lsmod for instance and then decide which ones you really need. Same for any processes running on your machine, ps -ef or even pstree -p could be helpful when looking for unneeded processes.

Solve IRQ conflicts by unbinding devices

The above script also contains a line to unbind a rogue device that sits on the same IRQ as the sound card. Unbinding a device is basically a last resort if a device is interfering in such a way that all other possibilities, like using rtirq, fail to improve the performance of the sound card. In this particular case the thread created for the USB port started consuming so much CPU that at a certain moment xruns were all over the place. In order to prevent this from happening unbinding the USB driver from the physical USB port worked out well.

This is the situation on the affected machine:

$ cat /proc/interrupts | grep 16
16:        923   IO-APIC-fasteoi   ohci_hcd:usb2, hda_intel

So usb2 shares IRQ 16 with the HDA Intel soundcard. In order to be able to unbind the USB driver from usb2 we need to find the bus ID of usb2:

$ tree /sys/bus/usb/drivers/usb/
/sys/bus/usb/drivers/usb/
|-- 1-4 -> ../../../../devices/pci0000:00/0000:00:13.5/usb1/1-4
|-- bind
|-- uevent
|-- unbind
|-- usb1 -> ../../../../devices/pci0000:00/0000:00:13.5/usb1
|-- usb2 -> ../../../../devices/pci0000:00/0000:00:13.0/usb2
|-- usb3 -> ../../../../devices/pci0000:00/0000:00:13.1/usb3
|-- usb4 -> ../../../../devices/pci0000:00/0000:00:13.3/usb4
`-- usb5 -> ../../../../devices/pci0000:00/0000:00:13.4/usb5

6 directories, 3 files

So the ID of usb2 is 0000:00:13.0. If we write that to the unbind file in the /sys/bus/pci/drivers/ohci_hcd/ directory usb2 should get unbound from the USB driver:

# echo -n "0000:00:13.0" > /sys/bus/pci/drivers/ohci_hcd/unbind

Finding the right unbind file might be a bit trial and error, but /sys/bus/pci/drivers/ohci_hcd/ contains the right unbind file for USB1.1 root hubs such as usb2. Now we can check with the first tree command (or with lsusb) what the current status is:

$ tree /sys/bus/usb/drivers/usb/
/sys/bus/usb/drivers/usb/
|-- 1-4 -> ../../../../devices/pci0000:00/0000:00:13.5/usb1/1-4
|-- bind
|-- uevent
|-- unbind
|-- usb1 -> ../../../../devices/pci0000:00/0000:00:13.5/usb1
|-- usb3 -> ../../../../devices/pci0000:00/0000:00:13.1/usb3
|-- usb4 -> ../../../../devices/pci0000:00/0000:00:13.3/usb4
`-- usb5 -> ../../../../devices/pci0000:00/0000:00:13.4/usb5

5 directories, 3 files

As you can see, usb2 is gone, it doesn't show up with lsusb anymore either:

$ lsusb
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 002: ID 04f2:b175 Chicony Electronics Co., Ltd 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

usb2 used to correspond with Bus 002. That one seems to be gone. A final check:

$ cat /proc/interrupts | grep 16
 16:        924   IO-APIC-fasteoi   hda_intel

Great, the sound card is now sitting on its own on IRQ 16 and runs in a way more stable manner than before when it still shared an IRQ with the USB port. Major drawback of this method of course is that you loose an USB port.

More information: http://lwn.net/Articles/143397/

Power management

Quality of Service interface

The Linux kernel provides a user mode interface for registering performance expectations2). One of the parameters it could work on is CPU latency. A modern CPU can have different so-called sleep states and waking up from those sleep states can take some time (CPU latency). By opening the device node /dev/cpu_dma_latency and registering a value with it you can control these CPU latencies. I.e. when registering the value 0 you can tell the CPU to never go to sleep and to always be avialable, without any latency. In some cases this can decrease DSP load and in its turn decrease the chance of running into xruns. In order for this to work the device node has to be kept open with the registered value, simply writing a value to it will not work. DAW's like Ardour and Reaper offer the possibility to control the CPU latency and if you want to take advantage of this you will want your user to be able to write to this device node. To configure this add the udev rule from https://github.com/Ardour/ardour/blob/master/tools/udev/99-cpu-dma-latency.rules to /etc/udev/rules.d/ and reload udev:

sudo udevadm control --reload-rules
sudo udevadm trigger

To check if udev picked up your new rule:

udevadm info -a -n /dev/cpu_dma_latency

And /dev/cpu_dma_latency should now have read-write permissions for root and the audio group (mode 0660). Now you can set CPU latency as low as you like. Bear in mind that setting it to 0 will put your CPU's in an always-on state which wil run them hotter. So be prepared for some extra fan noise.

Filesystems

For audio use, it is desirable to use a filesystem that favours few big files over many small files and low-latency over long-term thoughput. It is said that ReiserFS and fuseblk are bad choices from this perspective, while ext3 and ext4 are good.

Ext filesystems are mounted with the relatime option by default. The noatime option could be used but the performance gain will be minimal.

SSD

Most distributions currently do not auto-detect and optimize your filesystem settings for SSD drives. Here is a guide on how to optimize your SSD for use under Ubuntu-based distros. There is also a PDF document available on elinux.org on how to tune your filesystem for use with a SSD drive: http://elinux.org/images/b/b6/EMMC-SSD_File_System_Tuning_Methodology_v1.0.pdf

Encryption

It won't affect the quality of the audio, but it will change how many tracks the disk can support and because encryption requires CPU, it will affect how much signal processing you can do all of which is a long-winded way of saying “don't do it

tmpfs

Both Jack1 and Jack2 now use /dev/shm/ so mounting /tmp/ to tmpfs is not necessary anymore.

ext3/ext4 tuning options

noatime

Setting the noatime parameter in your /etc/fstab file reduces the amount of disk IO (the inode access times are not updated each time a file is read) which could improve the overall performance of your system.

/dev/sdax / ext4 noatime,errors=remount-ro 0 1

Most modern distributions use either atime or relatime. See the mount manpage for more information on these parameters.

The noatime parameter implies nodiratime so when using noatime there's no need to set nodiratime.3)

nobarrier

:!: This is a potentially unsafe parameter that could lead to data loss. Use with caution!

According to this thread High latency with ext4 / jbd2 kernel thread on the ext4 mailing list, the nobarrier parameter, with an ext4 filesystem, can reduce the amount of work made by the background kernel thread jbd2 and increase write speeds. However, this is an unsafe parameter, making your filesystem less resistant to power failures.

/dev/sdax / ext4 nobarrier,errors=remount-ro 0 1

Ext3 filesystem doesn't enable barrier by default, ext4 does. See the mount manpage for more information on these parameters.

data=writeback

:!: This is a potentially unsafe parameter that could lead to data loss. Use with caution!

This option specifies the following journaling mode for file data: data may be written into the main file system after its metadata has been committed to the journal. This is rumored to be the highest-throughput option. It guarantees internal file system integrity, however it can allow old data to appear in files after a crash and journal recovery.4)

You can't use this parameter in /etc/fstab for the root partition, you will have to set it as a kernel parameter. So when using Grub you can add the following parameter to the GRUB_CMDLINE_LINUX line in /etc/default/grub:

rootflags=data=writeback

limits.conf/audio.conf

It is advisable to set up your /etc/security/limits.conf or /etc/security/limits.d/audio.conf/ file properly, for example:

 @audio - rtprio 90       # maximum realtime priority
 @audio - memlock unlimited  # maximum locked-in-memory address space (KB)

Setting memlock to unlimited shouldn't be strictly necessary since most apps would work fine with lower values (such as 500000). However, certain applications have been reported to complain or even to crash with lower values than unlimited. On the other hand, giving unlimited capability of locking memory can cause buggy applications to temporarily freeze the whole system. See http://www.linuxmusicians.com/viewtopic.php?f=10&t=2193 for more details.

You could also allow the audio group to renice processes with the help of the limits.conf file, but since nice uses SCHED_OTHER it basically does nothing to increase the performance of a low-latency real-time audio environment that relies on SCHED_FIFO/SCHED_RR. Consult the sched_setscheduler manpage for more info on this subject.

See also: http://linuxmusicians.com/viewtopic.php?f=27&t=392

If you do not use PAM (which is rare), you might want to use set_rlimits instead

sysctl.conf

Suggested settings for /etc/sysctl.conf:

 vm.swappiness = 10

This setting changes the so-called swappiness of your system, or in other words, the moment when your system starts to use its swap partition. You can check the current value with cat /proc/sys/vm/swappiness, in most cases it is set to 60. For most modern systems with enough memory this is no issue. For smaller systems (embedded systems, Raspberry Pi's) this could be too high, it could make your system start using swap too fast which might influence the overall performance.

There are references on the net on adjusting the fs.inotify.max_user_watches value also for enhanced performance. The max_user_watches parameter sets the maximum number of files your system can monitor with inotify (which is part of the kernel) for changes. Setting this parameter too low results in inotify failing. Setting it too high can make inotify needlessly consume memory. The advise of increasing this value might originate from for example DAW's or samplers that watch a lot of files and if the system runs out of watchers these DAW's or samplers fall back to polling the file system which increases disk IO which could in turn cause jitter5). And since modern systems have mostly enough memory these days increasing this value should cause no issues.

audio group

It is generally good practice to have an 'audio' group, and add any users that should be allowed to perform audio tasks to this group. This prevents some interference from non-audio-processes with audio tasks. To verify you're in the 'audio' group, run the groups command. Remember, after adding yourself to new groups, you need to log out and back in again. Be careful when adding an 'audio' group to your system: most systems come with a pre-configured 'audio' group, and do not warn you when you add another group with the same name, leading to much confusion.

Timers

:!: Setting the kernel timer frequency to 1000Hz (CONFIG_HZ_1000) and setting max_user_freq or max-user-freq to 3072 does NOT improve the performance of your system.

JACK MIDI, JACK's MIDI backend drivers (like the alsarawmidi driver) and tools like a2jmidid do not care about the kernel clock frequency. When using applications that rely on ALSA MIDI (like Qtractor, SEQ24 or Rosegarden) a high resolution timer kernel module called snd-hrtimer is used that reduces MIDI jitter and improves MIDI performance. On most systems the snd-hrtimer kernel module is readily available and gets loaded when needed.

Priorities

For rtprio, higher values mean higher priority. You can list threads and their priorities with one of the following commands

$ ps axHo user,lwp,pid,rtprio,ni,command
$ ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i irq

Or if you have rtirq installed:

$ /etc/init.d/rtirq status

Most applications take care of running some threads with higher rtprio when possible themselves, like JACK with the real-time option set. Jack1's watchdog has a higher prio than jackd itself (+10) so choose a sensible priority value for Jack1 (70 for example, this will assign prio 80 to Jack1's watchdog thread), bearing in mind also that sound devices might get a priority in the 80-90 range if you use rtirq. Jack2 doesn't have a watchdog thread (neither does the current git version of Jack1) so you can give Jack2 an even higher prio.

You can also set rtprio and scheduling settings manually using the chrt command-line tool, which should come with most distributions nowadays, or use the rtirq init script.

See also: http://subversion.ffado.org/wiki/IrqPriorities

rtirq

rtirq is a script written by Rui Nuno Capela, the author of QjackCtl and Qtractor amongst others. This script allows to make use of the threaded IRQs as used by real-time kernels or kernels with the threadirqs kernel option enabled. The term threaded IRQ is quite an abstract one but what it boils down to is that every peripheral gets an IRQ and the Linux kernel splits this IRQ in different parts of which the so-called bottom-halve is of concern in the context of the rtirq script. Bottom-halves are also known in some cases as softirqs which might make this matter a bit less abstract, 'hardirqs' (the top-halves) are the IRQs assigned by the system which can't be managed from within the OS as opposed to 'softirqs' (the bottom-halves) which can be managed.

These bottom-halves are managed through so-called tasklets that are part of the tasklet API. This functionality can be verified with the aforementioned ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i “irq” command. The process called sirq-tasklet (<= 2.6.38, kernels >= 2.6.39 do not have such a process simply because that specific part from the RT patchset has been merged into the main kernel) is the actual tasklet management tool that takes care of assigning the priorities to different bottom-halves, or actually the threads in which these bottom-halves run (hence the term threaded IRQs). The distinction between these three terms (softirq, bottom-halve and tasklet) is a bit vague but basically tasklets are a top and bottom halve implementation based on softirqs. If you'd like to know more on this issue the following links are a good place to start:

The rtirq tarball as available on the site of the author consists of two files, the script itself (rtirq.sh) and a configuration file (rtirq.conf). In most cases the rtirq.sh file is used as an init script (and hence stripped from the .sh extension) and rtirq.conf mostly ends up in /etc/default/ or /etc/sysconfig/, also stripped from its extension. Let's look at the contents of the configuration file:

#
# Copyright (c) 2004-2011 rncbc aka Rui Nuno Capela.
#
#   This program is free software; you can redistribute it and/or
#   modify it under the terms of the GNU General Public License
#   as published by the Free Software Foundation; either version 2
#   of the License, or (at your option) any later version.
#
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#
#   You should have received a copy of the GNU General Public License along
#   with this program; if not, write to the Free Software Foundation, Inc.,
#   51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
#
# /etc/sysconfig/rtirq
# /etc/default/rtirq
#
# Configuration for IRQ thread tunning,
# for realtime-preempt enabled kernels.
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License version 2 or later.
#

# IRQ thread service names
# (space separated list, from higher to lower priority).
RTIRQ_NAME_LIST="rtc snd usb i8042"

# Highest priority.
RTIRQ_PRIO_HIGH=90

# Priority decrease step.
RTIRQ_PRIO_DECR=5

# Whether to reset all IRQ threads to SCHED_OTHER.
RTIRQ_RESET_ALL=0

# On kernel configurations that support it,
# which services should be NOT threaded 
# (space separated list).
RTIRQ_NON_THREADED="rtc snd"

# Process names which will be forced to the
# highest realtime priority range (99-91)
# (space separated list, from highest to lower priority).
# RTIRQ_HIGH_LIST="timer"

The RTIRQ_NAME_LIST variable contains a list of space separated service names of which the first entry gets the highest priority. The term service seems to refer to module names and sound device designations (so the output of lsmod and aplay -l respectively) and doesn't have to correspond to the full output, part of the output may suffice as the rtirq script does the matching itself.

The RTIRQ_PRIO_HIGH variable sets the highest priority a service can get assigned.

The RTIRQ_PRIO_DECR speaks for itself, here you set the number with which the priorities for each consequent service mentioned in the RTIRQ_NAME_LIST variable should be decreased.

The RTIRQ_RESET_ALL is a legacy variable and can best be left to default.

The RTIRQ_NON_THREADED variable is another legacy option, your kernel configuration has to support it and in almost all cases it doesn't because the specific option, which was part of the CONFIG_PREEMPT_VOLUNTARY kernel config option and that allowed for setting non-threaded IRQs, simply doesn't exist anymore. So basically this variable doesn't do anything.

The RTIRQ_HIGH_LIST variable contains a list of space separated service names that get priorities in the 99-91 range, so above the value as set in the RTIRQ_PRIO_HIGH variable. Use this variable only for services of which you want to be 100% sure they don't get interrupted by anything else. You will mostly want to put timers in there like rtc or the ALSA high resolution timer (snd-hrtimer).

When to use rtirq

A common use case for rtirq is when a FireWire controller shares its IRQ with one or even more devices. With rtirq you can then give a higher priority to the FireWire bottom-halve which should lower the chance of running into xruns drastically or even enable the device to initialize properly. The grep firewire /proc/interrupts command outputs the IRQ that is in use by your FireWire controller (when using the current JuJu stack):

$ grep firewire /proc/interrupts 
16:      88504        215   IO-APIC-fasteoi   uhci_hcd:usb3, mmc0, firewire_ohci, jmb38x_ms:slot0, eth1, nvidia

As you can see IRQ 16 is really crowded in this case and a FireWire device connected to this controller simply won't initialize. By adding the firewire_ohci module to the list of separated values of the RTIRQ_NAME_LIST variable you can assign a higher priority to the softirq or bottom-halve of the FireWire controller:

RTIRQ_NAME_LIST="rtc firewire usb"

If RTIRQ_PRIO_HIGH is set to 90 and RTIRQ_PRIO_DECR to 5 then the real-time clock (rtc) will get prio 90 and the FireWire controller prio 85. All other bottom-halves of IRQ 16 will get a default prio of 50 so the FireWire controller will always take precedence. In this particular case (a HP DV7-1040) using rtirq this way enables the FireWire device (a Focusrite Saffire Pro 10 IO) to initialize properly and run in a stable manner at lower latencies (128 frames/48Khz samplerate * 3 periods = 8ms). And given the fact that the controller (an onboard JMicron chipset) sits on a very crowded IRQ shared by a NIC, a cardreader, an USB port and a GPU rtirq definitely shows off its strength here.

PCI bus latency

Conventional PCI has a shared bus topology and also suffers from only being able to address a limited number of IRQ's. So besides PCI devices sharing interrupts they could also get into each other's way because of

lengthy transactions by some devices, due to slow acknowledgments, long data bursts, or some combination, which could cause buffer underrun or overrun in other devices”6). It's successor, PCI-X, suffers less from PCI bus latency because it uses higher bandwidths, but it's still using a shared bus topology unlike the latest PCI standard, PCIe. PCIe is “based on point-to-point topology, with separate serial links connecting every device to the host”7). > >So if you're using conventional PCI sound cards, you might want to consider maximizing the PCI latency timer of the sound card and raising the latency timer of all other PCI peripherals a bit (default is 64). This may sound counter-intuitive but “the PCI latency timer determines how long a device may own the PCI bus for … Therefore, if your system allowed the video interface to hog the PCI bus for a relatively long time, but limited the audio interface to only short periods of bus ownership, then audio performance will suffer. > >The script … resets *all* devices to an acceptable middle-of-the-road timer setting, then specifically allows the audio interfaces to own the bus for even longer.8)
#!/bin/sh

case $1 in
     start)
           # "open up" the PCI bus by allowing fairly long bursts for all devices, increasing performance
           setpci -v -s "*:*.*" latency_timer=b0

           # maximise latency for RME Hammerfall, thus allowing
           # more data per PCI transfer and minimizing xruns
           setpci -v -s 01:04.0 latency_timer=ff
           # ditto for the onboard AC97 audio interface
           setpci -v -s 00:07.5 latency_timer=ff
esac

exit 0

Source of the script: http://lists.linuxaudio.org/pipermail/linux-audio-user/2007-April/044371.html

Question remains in how far adjusting the PCI latency timer affects performance for PCI-X devices. PCIe doesn't suffer from PCI bus latency because of its different topology. Besides, PCIe devices have their PCI latency timer set to 0 and the setpci command simply doesn't change anything, setting latency_timer=ff on PCIe devices still yields a setting of 0.

Screensavers

When doing audio work screensavers tend to get in the way. So you might want to consider uninstalling them. You will find then that X is still trying to blank your screen after a while. To completely disable this you could add the following lines to your .profile file:

xset -dpms
xset s off
setterm -powersave off -blank 0

This should completely disable any unwanted screensavers or screen blanking.

PulseAudio

Basically all modern distros come with PulseAudio installed. PulseAudio (PA) is a sound server (like JACK) that takes care of mixing the output of all applications that can output sound. Since PulseAudio is designed to work in as much situations and with as much hardware as possible it is not the best choice for a real-time, low-latency Linux audio set-up. In other words PulseAudio is more suited for consumer audio and JACK is more suited for pro audio.

There are distros that offer scripts or configurations to allow using both sound servers at once. But when setting up a Linux audio workstation you could ask yourself if you really need two sound daemons when the software you're going to use is probably all JACK aware. Also, think about the extra overhead and possible issues you could run into when using two sound servers. Hence it's not uncommon that people disable or uninstall PulseAudio on their Linux audio machines. On a Linux audio workstation that is used for nothing else you could uninstall PulseAudio, on a workstation that is also used for other tasks it might be a better option to disable PulseAudio for the specific session in which you want to do your audio and music production. Since PulseAudio is the default sound server on a lot of distros it is not part of the startup applications and it is also not called through init systems. So it will always run or start to run as soon as an application wants to output sound. Add to this that on most distros PA is set to respawn automatically after being killed or when it crashes and you'll understand that PA doesn't allow to be disabled easily.

There is a relatively easy way to disable PulseAudio though for a specific user session. First, modify or create the file ~/.pulse/client.conf and make sure it contains the following line:

autospawn = no

Now you can kill the pulseaudio daemon:

killall -9 pulseaudio

Or with:

pulseaudio -k

And it shouldn't come up again, providing a PA free user session.

Using FireWire

When using FireWire audio interfaces it is generally advised to use a real-time kernel. Of course this is not necessary, there are also users reporting success with generic kernels, but when you have trouble getting your system stable regarding xruns one of the first things besides installing rtirq you might want to try is to install or compile a real-time kernel.

JuJu FireWire stack

As of kernel 2.6.37 the legacy FireWire stack has been removed from the kernel sources in favor of the JuJu stack. If you have FFADO installed it should take care of properly setting up your device(s). Instead of creating a /dev/raw1394 node, the JuJu stack creates nodes per device in /dev/, first device is always /dev/fw0. Take note that the first device is always your controller, not your soundcard.

To use JACK with your FireWire device(s), set the driver to 'firewire' in QjackCtl or use the -dfirewire option on the commandline.

Troubleshooting

After setting up, configuring and tweaking your system it is still possible that it doesn't perform the way you want it to. To help analyzing why the overal performance is not as expected there are some helpful tools that might reveal the source of the bottlenecks on your system.

rtcqs

The rtcqs script automatically analyzes your current configuration. This script will make configuration suggestions based on your current setup and link to the relevant sections on this page for background information.

rtcqs - version 0.4.2

Root User
=========
[ OK ] Not running as root.

Audio Group
===========
[ OK ] User music is in the audio group.

Background Processes
====================
[ OK ] No resource intensive background processes found.

CPU Frequency Scaling
=====================
[ OK ] The scaling governor of all CPU's is set at performance.

Kernel Configuration
====================
[ OK ] Valid kernel configuration found.

High Resolution Timers
======================
[ OK ] High resolution timers are enabled.

System Timer
============
[ OK ] System timer is set at 1000 Hz and high resolution timers are enabled.

Tickless Kernel
===============
[ OK ] System is using a tickless kernel.

Preempt RT
==========
[ OK ] Kernel 5.17.1-rt17 is a real-time kernel.

Spectre/Meltdown Mitigations
============================
[ OK ] Spectre/Meltdown mitigations are disabled. Be warned that this makes your system more vulnerable to Spectre/Meltdown attacks.

RT Priorities
=============
[ OK ] Realtime priorities can be set.

Swappiness
==========
[ OK ] Swappiness is set at 10.

Maximum User Watches
====================
[ OK ] max_user_watches has been set to 524288 which is sufficient.

Filesystems
===========
[ OK ] The following mounts can be used for audio purposes: /, /mnt/data
[ WARNING ] The following mounts should be avoided for audio purposes: /run/user/1000/gvfs. See also https://wiki.linuxaudio.org/wiki/system_configuration#filesystems

IRQs
=====
[ OK ] USB port ehci_hcd:usb1 with IRQ 16 does not share its IRQ.
USB port ehci_hcd:usb2 with IRQ 23 does not share its IRQ.
USB port xhci_hcd with IRQ 26 does not share its IRQ.
Soundcard snd_hda_intel:card10 with IRQ 31 does not share its IRQ.

Power Management
================
[ OK ] Power management can be controlled from user space. This enables DAW's like Ardour and Reaper to set CPU DMA latency which could help prevent xruns.

To get the script install Git first and then pull in the script:

git clone https://codeberg.org/rtcqs/rtcqs.git
cd rtcqs
./rtcqs.py

Discuss this script at the LinuxMusicians Forum

top

top is probably the most used tool to display system information like CPU or memory usage. It is installed by default on most distros.

Terminal running top

With the help of top you can find out which process is using up all your CPU or memory. By pressing h you get a help screen. To sort on memory usage instead of CPU usage you can press F + n + Enter.

htop

htop is the somewhat more sophisticated big brother of top.

Terminal running htop

latencytop

latencytop is a very useful tool to find out what process is causing system latency to happen.

latencytop

cyclictest

Just as latencytop measures system latencies, cyclictest measures kernel latencies. cyclictest has quite some options but one of the most used ways to run the command is:

#  cyclictest -t1 -p 80 -n -i 10000 -l 10000 -m

One single thread (-t1), priority of 80 (-p 80), use clock_nanosleep (-n), use 10000 us base interval of thread (-i 1000), use 1000 loops and then exit (-l 10000), lock current and future memory allocations (-m). More on clock_nanosleep can be found in its manpage (man clock_nanosleep).

Other examples of the usage of cyclictest:

TODO

  • More on MIDI timing
  • Harddrive tuning
  • Troubleshooting tools (ffado-diag, alsa latency tool, alsa midi latency tool, jack_delay, tuna, xruncounters)
  • Disk schedulers (BFQ, noop, mq-deadline)
  • Power management
  • Rewrite rtirq section or move to separate page
wiki/system_configuration.txt · Last modified: 2022/05/15 21:21 by autostatic