subogero linux blog

The support period of the Lucid Lynx on the Asus UL20A is about to end in May, so I was looking for a new OS. I gave a try to Debian Testing, or Linux Mint Debian Edition, to be precise. But not Pangolin. Sorry.

It was my smoothest Linux install ever. All hardware worked out of the box. Volume and brightness buttons? Check. Wifi? Check. Two-finger scrolling? Check. I also applied the usual MATE fixes for mutt and the power-button. Debian comes with a 486-kernel by default, so it handles only one processor core. One can upgrade manually to a multi-core 686-kernel with apt-get.

aptitude search linux-image

And what do I find?

linux-image-rt-686-pae

A PREEMPT_RT patched kernel, kindly compiled by Debian. Time to plunge into real-time again!

I put together a Linux syscall tutorial project a year ago (see github) which contained a small program performing a periodic real-time task and printing statistics like average/min/max dt and its standard deviation. It’s called rt and uses the setitimer() system call to generate periodic SIGALRM signals. Time to measure latencies of a 1ms periodic task on a RT kernel!

To my utter disappointment, performance was fine, until the tapeta daemon changed my wallpaper, that is. At which point a 300% latency appeared. That’s right, period time 4ms instead of 1ms. Same as earlier, without an RT kernel. What’s wrong?

I was not using real-time scheduling, that’s what. It turned out Linux processes run at static priority zero by default, where the actual scheduling priorities are dynamic, depending on interactivity, nice level, sleep/runtime, etc. This is called the SCHED_OTHER policy. In other words, one is at the mercy of the Completely Fair Scheduler.

On the other hand, a process’ static priority can be increased, and the scheduling policy can be changed to SCHED_FIFO with the sched_setscheduler() system call. In this case the only one whose mercy we are at, is another SCHED_FIFO or SCHED_RR (Round Robin) scheduled process with equal or higher static priority, lurking in the background somewhere. Important to note, though, a process needs to run as root to be able to change its own or another process’ scheduling. There is also a command line utility called chrt, to change the scheduling policy of processes.

By the way. I’m not using threads. Threads are evil. People who use them mostly condemn themselves to a thousand hells of corrupted data, deadlocks and, at the end of the day, slower performance, thanks to cache misses and dirty cache lines. And the uClibc library in many embedded systems does not support POSIX threads. Thank goodness. I’m particularly angry with threads, as every damn RT-Linux tutorial you look at, spends 75% with setting up the bloody things. And you even have to worry about which thread your signals will be delivered to. What about learning fork() and pipe() instead? KEEP IT SIMPLE STUPID!

But enough of my rants and dodgy theories, let’s see the practice. First let’s see, how the skeleton of a periodic task looks like.

#include <unistd.h>
#include <signal.h>
#include <sys/time.h>

/* Periodic SIGALRM handler routine: everything happens here */
void periodic(int signal)
{
    ... /* do your periodic stuff */
    if (continue_running)
        signal(SIGALRM, periodic);
    else
        exit(0);
}

int main(int argc, char *argv[])
{
    ... /* initialize your stuff */
    struct itimerval period = {
        { 0, 1000, }, /* 1st signal in [s], [us] */
        { 0, 1000, }, /* period time   [s], [us] */
    };
    signal(SIGALRM, periodic);       /* install periodic() to handle SIGALRM */
    setitimer(ITIMER_REAL, &period, NULL); /* start periodic SIGALRM signals */
    /* Main idle loop: everything done by the signal handler */
    while (1)
        pause();
    return 0;
}

Next, how to set up real-time scheduling. Add this code to main(), before the setup of the periodic stuff. Remember to run your program as root for the below code to take effect.

#include <sched.h>
...
int main(int argc, char *argv[])
{
    ...
    struct sched_param schedp;
    schedp.sched_priority = 1;
    sched_setscheduler(0, SCHED_FIFO, &schedp);
    ...
}

And now let’s see what happens when a process becomes real-time. I’m measuring latencies of a 1ms task on Raspberry Pi, actually without a PREEMPT_RT kernel. I start rt as a normal process, then change its scheduling to SCHED_FIFO with chrt. See the code of rt on github.

szg@og314 ~/syscalls $ ./rt 1000
Real time child process PID 3368
------------------------------------------------------------------------
     n  Mean [us]       [us] SD  [%]      [us] Min [%]      [us] Max [%]
------------------------------------------------------------------------
   982   1017.502    246.148  24.615        67  93.300      4449 344.900
   985   1015.249    247.086  24.709        71  92.900      4865 386.500
   975   1025.625    300.905  30.091        63  93.700      4669 366.900
   989   1011.116    162.667  16.267       181  81.900      2682 168.200
   986   1014.260    155.867  15.587       259  74.100      2506 150.600
   989   1011.115    163.982  16.398        51  94.900      2674 167.400
   983   1017.492    189.123  18.912        59  94.100      2803 180.300
   983   1017.102    199.678  19.968        74  92.600      3985 298.500
   985   1015.169    214.607  21.461        62  93.800      3938 293.800
   978   1022.632    267.480  26.748        83  91.700      4355 335.500
   988   1012.008    165.071  16.507        84  91.600      2707 170.700
   991   1009.079    222.581  22.258        72  92.800      6762 576.200
  1000   1000.001     18.526   1.853       769  23.100      1218  21.800
  1000   1000.040     16.464   1.646       834  16.600      1163  16.300
  1000    999.958     12.861   1.286       924   7.600      1086   8.600
  1000   1000.002     17.503   1.750       809  19.100      1223  22.300
  1000   1000.014     20.984   2.098       793  20.700      1212  21.200
  1000   1000.035     24.159   2.416       827  17.300      1175  17.500
  1000    999.998     26.596   2.660       795  20.500      1246  24.600
  1000    999.962     18.433   1.843       844  15.600      1152  15.200
  1000    999.989     34.235   3.424       716  28.400      1314  31.400
  1000   1000.003     18.630   1.863       849  15.100      1214  21.400
  1000   1000.001     22.666   2.267       717  28.300      1268  26.800
  1000    999.996     20.419   2.042       799  20.100      1196  19.600
  1000   1000.051     15.729   1.573       831  16.900      1159  15.900
  1000    999.951     13.385   1.339       920   8.000      1109  10.900
  1000   1000.000     18.562   1.856       830  17.000      1210  21.000
  1000   1000.002     28.782   2.878       812  18.800      1222  22.200
  1000   1000.039     19.713   1.971       763  23.700      1239  23.900
  1000    999.964     25.291   2.529       907   9.300      1118  11.800
     1   1035.000     35.000   3.500      1035  -3.500      1035   3.500
------------------------------------------------------------------------
     n  Mean [us]       [us] SD  [%]      [us] Min [%]      [us] Max [%]
------------------------------------------------------------------------
 29815   1006.212    136.875  13.687        51  94.900      6762 576.200

A line of statistics is printed after every second with the following columns: n (number of events), Mean dt, SD (standard deviation), and Min/Max dt. Whilst the program was running, after the 12th line of data above, I changed rt’s scheduling with the command below.

root@og314 ~ # chrt -f -p 1 3368  # -f : SCHED_FIFO, 1 : static priority 1

The effect was truly dramatic. With normal scheduling I was missing about 2% of the events, with an average latency of 20% (0.2ms), in the worst case up to 600% (6ms). After changing to real-time scheduling, not a single event went lost, average latency dropped to 2% (0.02ms!), and the worst case delay dropped to 31% (0.31ms).

All this is causing 7-8 % processor load on a Raspberry Pi running two ssh sessions and omxplayer playing an internet radio. Single core, no PREEMPT_RT kernel. You probably need the latter in a production system which must not miss a beat for weeks.

So Linux is definitely able to run an ECU. That’s where my new ARIA25 board will excel. After someone designs an ECU for me…

Recently, I’ve upgraded the HP 874W power station to Linux Mint Maya, 64-bit, MATE edition.

It’s pretty stable, the performance is brutal, and it has very new packages out of the box, like VIM 7.3 (I’m looking at you Debian Squeeze). But the Gnome2-MATE conversion is still a work in progress. A few things are still broken, like the power button brings up the shut-down dialogue for exactly zero seconds and, unfortunately, Compiz does not work out of the box.

It took me two evenings to play with the Configuration Editor (gconf-editor), the Configuration Editor (mateconf-editor) and CCSM to reach a state, where Compiz places, decorates, wobbles and peeps through windows in the desired fashion, and the virtual desktops are spinning and exposing as in the old times. Except, there is no way to start the old hierarchical menu with Alt-F1. So I’m forced to use the chaotic MintMenu. By now my readers have figured out, correctly, that I’m a keyboard-guy.

So Ubuntu Lucid Lynx will stay on the ultra-portable ASUS UL20A for at least another year. This OS is the symbol of the golden era of the Linux desktop, 2010. Dark clouds were already gathering on the horizon, but here, down on Earth, everything looked rosy and happy. The Linux desktop was so good that Apple and Microsoft could not help copying it at an astonishing rate. Full screen for ALL apps for the first time in Mac OS X Lion, anyone?

But while we, proud users, were showing off semi-transparent rotating desktop cubes to the Apple fanboys in the neighbourhood, William Jon McCann had a feeling of terrible insecurity. He, as the lead designer of Gnome3, felt that the Linux desktop was crap and something new, more apple-ish was required. At the same time Miguel de Icaza, the founder of Gnome, had already been a closeted Mac OS X user, waiting for the great coming-out moment. Allegedly, his sound was not working and he could not play videos on Linux. I’m at a complete loss here…

But anyway, Gnome3 was born, Gnome2 was discontinued, Ubuntu defaulted to Unity, and everybody else got real angry. There is a silver lining in the clouds though, thanks to Linux Mint, but it’s still a long way. But what caused this mayhem?

It’s called inferiority complex.

It’s harmful.

No, the desktop is not going away, and no, OS X and Windows were not better than the Linux desktop, and no, goodness is not measured by market share. These gentlemen were hankering after the wrong things from the Apple world. They wanted to copy looks and features and gardens and walls. They should have noticed something else:

Self-confidence.

It’s the ability to notice that you’re good at something. The first Mac OS from 1984 and the present OS X basically look the same. They stuck with their interface even in the valley of the shadow of death. Their user interface is still the same, while they have changed the internals to Unix. The same is true for the Linux kernel. The external interfaces never change. Linus’ bash binary from 1991 still runs.

Miguel de Icaza thinks Linus Torvalds is an arrogant low-level kernel guy. But the fact that you’ve been a guilt-afflicted closeted Mac-fag, does not mean that someone else with a healthy dose of self-confidence is arrogant. In fact Linus Torvalds is an extremely humble person. He goes to great lengths to protect his users. Yes, on the surface you see the “shoot-yourself-before-you-reproduce” and the “you-are-full-of-BS”, but the comforting self-mockery is always there, and it’s all for the users’ sake.

On the Gnome side there is McCann’s polished and politically correct corporate lingo, which in fact is full of ignorance, insecurity and bullshit. And they could not care less for their existing users. But it’s Linus’ fault, of course.

Go to therapy, get rid of your inferiority complex and start using mplayer.

I’ve just reached a very severe stage of the disease called Unix. I’m learning vi.

I still vividly remember the shock when a command-line Linux expected me to edit something and kindly opened up vi for me. I did not know how to exit, so I had to shut down the computer.

After using pagers (more or less) and manual pages for some time, I slowly learned a few important hotkeys like “/pattern” to search forward for something, “n” to find the next match and “q” to quit. Playing the command-line version of robots, one quickly memorizes “h”, “j”, “k” and “l” for navigation. It turns out they all come from vi. Or vice versa.

But before going on, I’d like to enlighten my readers about quitting vi:

:q<Enter>   Quit
:q!<Enter>  Quit and discard changes
:wq<Enter>  Save and quit
ZZ          Save and quit

Now that quitting is out of the way, let me tell you about a few highlights of my steepish vi learning curve. It seems the usefulness of Unix tools is proportional with their scariness.  Same as with git.

Those 1-2-letter commands are quite logical, actually, usually consisting of a command and a move, eg “dw” stands for delete word.

Let’s take moving around in the window. After scrolling down a few pages, you find the interesting line, but it’s in the bottom of the screen along with the cursor. Command “z.” will put it in the middle.

When you try to select something with a Windows-style editor, you press Shift at the beginning, keep it pressed and move around with the arrows. Then you realize the beginning is at the wrong position. You can start over.

In vi, on the other hand, you press “v” for visual selection, move around to find the end of the highlighted block. Then you notice the beginning is wrong. Press “o” for opposite, your block’s end will get fixed and the cursor jumps to the beginning, which you can now move around. Astonishing!

And now to something completely different: the vi Cheat Sheet!

vi Cheat Sheet

Bonus: I was always wondering what text editor I should install onto my Android phone. I’ve just realized that busybox-vi has been there all the time. It works perfectly with the crude virtual keyboard.

Bonus 2: my ~/.vimrc file with syntax highlight, line numbers, autoindent, highlight search results and, most importantly, brighter foreground colours.

syntax on
set number
set autoindent
set hlsearch
set background=dark

I’m always very proud how incredibly easily my Linux systems cooperate with the external world, including Windows machines.

For instance, just now, after one day of searching, sweating and swearing, I was able to mount some Windows shares onto a Debian virtual server.

For those of you who boast that one can do this with two clicks in Windows, let me tell you that

1. it’s actually 6 clicks
2. I also could have used Nautilus and just connect to a server

But that would have been too easy and, at the same time, too difficult to use. Have you ever seen such a Gnome gvfs mount point?

~/.gvfs/My bloody space-separated sharename on Foo

Brrrr. One wants something nice like

~/Foo

One also wants to be able to mount and unmount it as a normal user. And to have write access. And not having to enter a password, but also not to reveal it to others. One is very picky.

Before all, don’t forget to create the mount point directory manually. It shall be world-executable as well.

$ mkdir ~/Foo
$ chmod 775 ~/Foo

Now let’s see the /etc/fstab entry. We’ll explain every option later.

//srv/share /home/foo/Foo cifs rw,user,credentials=/home/foo/.foo,uid=1000,gid=1000 0 0

The entries are share UNC path, mount point, file system type, options, etc.

If you miss anything from the dreaded options, you’ll get the following highly informative error message when trying the mount:

mount error(13): Permission denied

Option “rw” means read-write. Easy.

Option “user” means not only root is allowed to mount or unmount the share.

Server “srv” is in domain “bar” and shares “share” for user “baz” whose password is “spam”. We could add these options directly to fstab (username=bar,password=spam,domain=baz), but that would make it world readable. Instead the info is stored in a credentials file in Linux user foo’s home. Make sure this file is readable by foo only. Let’s see its contents too:

$ chmod 600 ~/.foo
$ ls -l ~/.foo
-rw------- 1 foo foo size date .foo
$ cat ~/.foo
username=bar
domain=baz
password=spam

Don’t put any whitespace around the “=” signs. There is a school of thought that there should not be any character after the end of the password, not even a newline.

And last but not least, the Linux user’s user and group IDs should be specified as well. In our case “foo” is the first normal user on this machine, that’s why it’s 1000.

Now user foo should be able to mount and unmount the above share:

$ mount ~/Foo
$ umount ~/Foo

I happened to activate the “Source” repos in Synaptic today and then downloaded the Linux 2.6.28 source code. Just for fun.

I also happened to have a look into CREDITS. It turned out to be an interesting read. It seems Linux comes from the happier parts of the world. Let’s see:

++ I'm proud to announce four Hungarian kernel hackers
++ But the happiest part of the Eastern Block seems to be the Czech Republic
++ The North rocks, or rather hacks (Finland, Sweden, Denmark, Canada)
++ The bulk comes from the West (USA, UK, Germany, Netherlands, France, Italy)
++ Eric S. Raymond is there (a.k.a. Master Foo)
++ Japan in da house
++ The Republic of China (Taiwan) beats ...
-- ... the People's Republic of China (only one hacker for 1 billion people)
-- India, just one? Come on! (Even he is Malaysian)
-- Same for Israel. Although I always knew Haifa is the best place there
-- Absolutely zero from Islamic countries.

I have a theory about the effects the “Religion of Peace” has on the human brain. It seems to be confirmed.

Finally. I’ve picked up my US keyboard for the ASUS UL20A today. Installation was surprisingly easy, with no warranty-void-if-removed stickers anywhere. Time to play again with keyboard settings. Default layout became US International (with dead keys). I kept Hungarian (qwerty) as well for the occasional őŐűŰ. And now to the most interesting part, the Layout Options.

Ctrl Key Position, Make CapsLock an additional Ctrl.
Hurraaaaah!!!! The bloody CapsLock is eliminated at last!

Key(s) to Change Layout, Both Shift keys together.
A combination of keys far away from each other, never normally used together.

Use keyboard LED to show alternative layout. CapsLock.
Of course. As the bastard is dead, I can use its LED to indicate if the Hungarian layout is active.

This incredible abundance of keyboard settings is amazing. A good reason alone to choose Linux over Windows.

Apple’s products are simply beautiful. Mac mini, iPod Nano, MacBook or iPhone: anything they do, they do it with style.

Except, they don’t always work, apparently. During the weekend’s skiing/snowboarding trip, one of us wanted to transfer some important photos (showing just-about-to-fall snowboarders) from her iPhone (gorgeous pink cover) to her MacBook (gorgeous flat chassis). No way.

So we connected the iPhone to my Ubuntu-driven notebook, the “Apple Inc. iPhone” icon promptly appeared on the desktop, and in a few minutes time the compromising pictures were transferred to the MacBook travelling on board of a carefree pendrive.

I’ve just purchased an ASUS UL20A-2X022V notebook. Or netbook. I don’t know. It’s actually the best of both worlds. On one hand, it has a small 12″ display, weighs in at 1.5 kg, has no CD/DVD drive, and runs 5 to 7 hours on battery. On the other hand, its small display’s resolution is 1366×768, it has a Core 2 Duo ULV SU7300 processor, 3 GB RAM, a 320 GB hard disk and a fully functional keyboard with all the special keys on the right side.

A short summary of how the different hardware components work with ubuntu:

Component                    Status  Notes
------------------------------------------
Intel Core2 Duo ULV SU7300   OK
12.1" WXGA LED display       OK      resolution autodetected
Intel GMA 4500MHD graphics   OK
3 GB RAM DDR2 800 MHz        OK
HD 320 GB 5400rpm SATA       OK      install with manual partitioning
battery Li-ion 5600 mAh      OK
power management             OK      battery life 5-6 hours
ethernet Atheros 8131        OK      see below
WLAN Atheros 9285 802.11bgn  OK      see below
Bluetooth                    OK
Sound AC'97 16bit            OK      Audacious mp3, speaker or headphone
Synaptics touchpad           OK      scrolling OK too, see below
webcam                       OK      see below
card reader                  OK
Linux kernel                 2.6.28-17

It came with Windows 7 Home Premium, which is a joke. A huge monster of an OS with the functionality of Google Chrome OS: it has a web browser.

First thing was to install ubuntu 9.04 Jaunty Jackalope. As the laptop has no CD drive I had to create a bootable Ubuntu 9.04 LiveUSB on my other ubuntu-box. The tool to use is usb-creator, which uses the iso image of the ubuntu LiveCD.

I installed ubuntu with manual partitioning. I shrank the Win7 partition from 80GB to 50GB (automounted to /windows), added a 2GB ext3 swap partition, a 50GB ext3 root partition and the rest as a /home partition. The boot menu was added, and even Win7 ran fine after a chkdisk.

The network cards were not detected upon installation. Neither LAN, nor WLAN. So there I was with no connectivity, reading with a sad irony all those posts about fixing this with apt-get xxx-backports and the likes.

I ended up downloading compat-wireless-2.6.30.tar.bz2 on another box to a USB stick, and then unpacking and compiling it on the laptop. I realised too late these kernel modules were for a newer kernel. Nevertheless I did “make install”, I did “make unload” and I did “insmod ath9k.ko”. It did not work due to incompatible kernel versions. I sadly rebooted to Windows, but later gave it another try. Miraculously, all network cards worked like a breeze! Don’t ask me why…

Touchpad: everything works, but it’s hard to feel where is the scrolling area. And the left button is too hard.

Keyboard (Hungarian): OK, but it will be swapped for a US layout, whose Hungarian is worse, but speaks better Code.

Webcam: I installed the UCview package to record videos with it. Skype works too.

I downloaded all the updates, then SynapticPackageManaged Rhytmbox/Evolution out, Audacious/Thunderbird in, I installed basic development stuff, and last but not least git-cloned and compiled the newest Midnight-Commander master with utf8-support.

One more thing: it is very very quiet. Summary? Bloody marvelleous!

This is perverse. I’ve just compiled a native Win32 application on Linux. What’s even more perverse it runs on Linux too with Wine.

To cut a long story short I needed a command line tool to send emails using the MAPI protocol, the only language spoken by Outlook and MS Exchange. Don’t ask me why. It’s by the way my first ever Win32 application using the Win32 API. Or `mapi.h’, to be precise.

I started developing it on a Win32-cygwin platform with gcc, but I wanted to control the sources with git. Which prefers Linux. So I ended up rebooting the box a few times to change between the two. But I’m extremely lazy, so I soon googled for a way to cross-compile Win32 programs on a Linux host. It did not take long. I summarize my findings in a few lines:

What to install:

sudo apt-get install mingw32

How to compile:

i586-mingw32msvc-gcc -L/usr/i586-mingw32msvc/lib \
  -I/usr/i586-mingw32msvc/include -o foo.exe foo.c

The result? See the mapis page.

G’s goal

“g” is a general-purpose command-line launcher for Gnome/Linux, an extended “gnome-open” which can open URLs, files or even programs in a new window. A bit like Windows “start”, just better.

“g” will open a new terminal window.

“g <CLI program>” will open it in a new fully independent terminal window.

“g <GTK+ program>” will open it in a fully independent window.

“g <URL>” will open it with the preferred application.

By an “independent window” I mean two things:

The starting terminal is not blocked while the started app is running;

Closing the starting terminal does not kill the started app.

Bonus: start a Google search for “foo bar” in your preferred browser with

g -g foo bar

Installation:

Download g.tar.gz and extract files in a new directory, preferably “g”.
Type “sudo make” for installation.
Enjoy.

Lessons learned:

How to decide if an argument of a shell-script is a program?

which $1
#prints the location of the command, or nothing if not a command

How to check if a shell-script has no arguments at all?

if [ $# -eq 0 ]

How to check if the last command returned no error in its exit status?

if [ $? -eq 0 ]

The trickiest: how to tell if a command is a terminal- or a GTK+ application?

Search for “libgtk-x11″ in it if it’s a compiled program.
Search for the respective gtk dependency if it’s an interpreter-script:

import gtk # Python
use Gtk2   # Perl

“g” knows these two at the moment, but the list should be extended to all known interpreted languages and graphical toolkits. Actually it should work the other way around, detecting terminal-only applications. But for instance Gimp contains the strings “stdin”, “stdout”, “stderr” and “printf” so I gave up.

I’ve created my first ever very officially looking manual page for “g”, credit goes to Jens Schweikhardt’s very helpful LINUX MAN PAGE HOWTO.

Tao in g

There is more Tao in “g” than in Windows “start”:

Try to start a native Windows program from command line which DOES NOT become fully detached, but blocks your command-line until it closes and returns an exit-status. Hmmm? On Linux, just omit “g”.

On Windows, the commands are scattered around in thousand folders, not listed in the PATH variable. On Linux, on the other hand, all commands are on the PATH, as there are only a few standard places to store them: /bin:/usr/bin:/usr/local/bin. So “g” can access them all.

View the manual page?

man g

Finally, how do you edit the “g” script itself?

g gedit g