Why Real-Time?

Why bother with these real-time kernels and APIs at all? Isn't timing on a modern PC good enough? Look at this:

This histogram shows latency-numbers from the same 1ms thread test run compiled without (red) and with (green) real-time enabled. All the green real-time data clusters around zero +/- 20us. Without real-time enabled the event we are expecting to happen every 1 ms might happen almost 1 ms too early, or up to 3 ms late. With real-time the timing is mostly consistent to better than 1% (10 us) with a worst-case jitter of 2% (20 us).

Latency Histogram

This shows a latency-histogram for a 1 ms thread running on Xenomai on my recently acquired ITX-board. Note how badly the histogram is approximated by a normal distribution (Gaussians look like parabolas with logarithmic y-scale!). See also Michael's recent RPi data and  Kent's Athlon/P4 data.

The usual latency-test numbers people report is the maximum latency, a measure of how far out to the left or right the most distant single data point lies. The histrogram can probably be used to extract many more numbers, but for real-time critical applications like cnc-machine control the maximum latency is probably an OK figure of merit.

The latency numbers are recorded with a simple HAL component:lhisto.comp

The instantaneous latency-number is then put in a FIFO by the real-time component sampler and written to a text-file using halsampler. I'm setting this up with the following HAL commands (put this in a file myfile.halrun and run with "halrun -f myfile.halrun")

loadrt threads name1=servo period1=1000000
loadrt sampler depth=1000 cfg=S
loadrt lhisto names=shisto
addf shisto servo
addf sampler.0 servo
net latency shisto.latency sampler.0.pin.0
start
loadusr halsampler -c 0  latencysamples.txt

The numbers can now be plotted with matplotlib. I'm using the following script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.mlab as mlab
# load data from file
x = np.loadtxt('latencysamples.txt' )
x=x/1e3 # convert to microseconds
 
fig = plt.figure()
ax = fig.add_subplot(111)
nbins = len(x)/1000
n, bins, patches = ax.hist(x, nbins,  facecolor='green', alpha=0.5, log=True)
bincenters = 0.5*(bins[1:]+bins[:-1]) # from matlplotlib example code
mu = np.mean(x)
sigma = np.std(x)
area = np.trapz(n,bincenters) # scale normpdf to have the same area as the dataset
y = area * mlab.normpdf( bincenters, mu, sigma)
l =  ax.plot(bincenters, y, 'r--', linewidth=1)# add a 'best fit' line for the normal PDF
 
ax.set_xlabel('Latency ( $ \mathrm{ \mu s } $ ) ')
ax.set_ylabel('Counts')
ax.set_title('Latency Histogram\n 12.04LTS + 3.2.21-xenomai+')
ax.set_ylim(1e-1, 10*max(n))
ax.grid(True)
plt.show()

LinuxCNC on Ubuntu 12.04LTS

Recent developments has made it possible to run LinuxCNC on the latest LTS release of Ubuntu. This is experimental work, so not recommended for controlling a real machine just yet. The main obstacle for moving LinuxCNC from 10.04LTS to a more recent distribution has been the RTAI real-time kernel, which has not been kept up-to-date with development of the normal Linux kernel. Fortunately there are alternatives such as Xenomai or RT_PREEMPT.

Here is a step-by-step description of the install/build process, if you want to experiment with this.

  1. Download and install a normal 32-bit 12.04LTS Ubuntu (ubuntu-12.04.1-desktop-i386.iso). Note that the 64-bit version is not supported for the steps that follow further down. I could not get Ubuntu's startup-disk-creator to work, so I used unetbootin to write the ISO-file to a USB-stick.
  2. It's possible to compile the xenomai-kernel from scratch, along with the runtime etc., but I used pre-compiled deb-packages by Michael Haberler from here: http://static.mah.priv.at/public/xenomai-debs/
  3. Install the xenomai kernel:
    sudo dpkg -i linux-headers-3.2.21-xenomai+_0.1_i386.deb
    sudo dpkg -i linux-image-3.2.21-xenomai+_0.1_i386.deb
  4. make sure it will show up as a GRUB-entry when booting:
    sudo update-initramfs -c -k 3.2.21-xenomai+
    sudo update-grub
  5. reboot. uname -r should now show: 3.2.21-xenomai+
  6. now install the xenomai runtime:
    sudo dpkg -i libxenomai1_2.6.1_i386.deb
    sudo dpkg -i libxenomai-dev_2.6.1_i386.deb
    sudo dpkg -i xenomai-runtime_2.6.1_i386.deb

This installs the xenomai system on top of which a recently available version of LinuxCNC can be built. There are probably many ways to now obtain the tools/dependencies that are required. I used the following:

  1. sudo apt-get install synaptic
    sudo apt-get install git
  2. Now using synaptic, install the following packages (I found these are required for a minimal linuxcnc build):
    build-essential
    autoconf
    libpth-dev
    libglib2.0-dev
    libgtk2.0-dev
    tcl-dev
    tk-dev
    bwidget
    libreadline-dev
    python-tk
    python-dev
    libgl1-mesa-dev
    libglu1-mesa-dev
    libxmu-dev
  3. Get Michael's version of LinuxCNC that can be compiled for Xenomai:
    git clone git://git.mah.priv.at/emc2-dev emc2-dev
    cd emc2-dev
    git branch --track rtos origin/rtos-integration-preview1
    git checkout rtos
  4. Configure and build for Xenomai:
    cd src
    ./configure --with-threads=xenomai-user --enable-run-in-place
    make
    sudo make setuid
  5. Test:
    . ./scripts/rip-environment
    latency-test

This new version of LinuxCNC can be built without a real-time kernel (previously called "simulator" or "sim") or with any of the real-time kernel alternatives: RTAI, Xenomai, RT_PREEMPT. It should be possible to compare real-time performance in the form of latency-numbers with different hardware and kernels.