Archive

Posts Tagged ‘raspberry pi’

RemotePi REST

December 3rd, 2014 subogero Comments off

I’m constantly bombarded with RESTful stuff at work. So much so, that I’ve started rewriting RemotePi as a REST API. Lots of learning and reading, then.

As usual most of the literature was written by people who studied computer science. They just love abstractions. Solving problems is OK, as long as they get their daily hit of abstractions. So there are resources, verbs and routers. Very nice.

You’re supposed to refer to a resource without ugly queries in the URL. Instead, it should be just http://pi/remotepi/home/ for the media root directory. But how the hell does such an URL generate a query that can be processed by a CGI or FastCGI app?

The computer scientists are silent on the issue.

So I had to dig deeper and discover mod_rewrite of Apache2. There is a lot in a name. It rewrites incoming URL requests and is able to turn them into proper queries. Thank you Ludovico Fischer.

So here is my “resource router”.

First enabled mod_rewrite:

a2enmod rewrite
service apache2 restart

And the router config in /etc/apache/sites-available/default

RewriteEngine On
RewriteRule /remotepi/(.+) /remotepi/api.pl?$1

Very Perly, ain’t it? It’s because it uses the Perl regex engine.

RemotePi FastCGI

December 3rd, 2014 subogero Comments off

RemotePi, the remote-control webb-app of my Raspberry Pi media center, felt sluggish. Until I measured the response time with Firefox WebDeveloper/Network. It’s now official: it was sluggish: it took 800 ms to respond. That’s nearly one second.

So it was time to turn the original CGI solution into a FastCGI one. Instead of firing up a new Perl process upon each request, the app keeps running in the background and replies requests in a main loop. I’m using Apache2 mod_fcgid.

The improvement is shocking: File browsing is about 25 ms, requests involving an omxd call take 80 ms. And that’s basically the pure runtime of the omxd command.

FastCGI is fast indeed.

YouTube on the Raspberry Pi At Last

July 12th, 2014 subogero Comments off

To cut a long story short: YouTube finally works in HD on my TV with RemotePi. I can’t stop watching LaFerrari and McLaren P1 videos in HD with great sound!

Why was it so difficult?

In the old times, youtube-dl -g spat out YouTube stream URLs, and omxplayer could play them straight away. The excellent ncurses based yt worked this way. But things have changed. YouTube now only streams the video if a session cookie is presented. But omxplayer can’t use cookie jars.

But curl can! We tell youtube-dl to save the cookie, and let curl save/stream the video using the cookie.

curl -b jar `youtube-dl -g --cookies jar --write-thumbnail`

will spit out the video stream.

1st Attempt – Save to File

Let curl save to a file, and start omxplayer a few seconds later to play it back. Keep fingers crossed that curl saves faster than omxplayer reads.

It does not. This solution is so disk/sdcard IO intensive, that omxplayer will reach the EOF too soon and exit.

2nd Attempt – Stream via FIFO

Let’s create a FIFO file, and let curl write the video stream into that. And start omxplayer immediately to read the video stream from the FIFO. It works beautifully:

  • there is no disk IO at all, so curl writes faster than omxplayer reads
  • even if it’s not the case due to a slow network, the player does not exit, just blocks until curl catches up. Bacause the FIFO is a character device!
  • if omxplayer exits for any reason, curl gets a SIGPIPE and exits too

The Result – rpyt

It’s all wrapped up in a new script packaged with omxd: rpyt.

And the RemotePi remote-control web-app also uses that from now. See the project homepage and the code on Github.

My Raspberry Pi Media Player Suite

September 26th, 2013 subogero Comments off
RemotePi screenshot

RemotePi browsing music

My Raspberry Pi media player suite has reached a generally usable level. At the bottom of the stack there is omxplayer, then the omxd playlist daemon, the rpi.fm command-line internet radio player and, finally, RemotePi the remote control web-app, optimized for smart-phone screens.

Check out its home page.

Why Not XBMC?

I admit I suffer from severe NIH (Not Invented Here) symptoms. But this time, I actually tried to install XBMC from Michael Gorven’s Debian package. But what do I see?

I need to get 44 MB of archives. 120 MB additional disk space will be used. RemotePi, rpi.fm and omxd are 3 MB including all sources, all binaries and all git repositories.

And lo and behold, who are among the dependencies?

  • qt3
  • mysql-common
  • samba-common-bin

Come on! I never use my Raspberry’s GUI. All the GUI RemotePi has is a web-app.

MySQL? Probably a sign of a nice tag-based media library which, in real life, is an absolute pain in the neck to use. You have a DJ-set album, from all sorts of different artists, and a tag-based player just refuses to play it in the order intended by the DJ. I’ll stick to my files and directories, thank you very much.

Samba? I’m one of those smug Penguinistas whose home is Windows-free. So no thanks.

RemotePi rpi.fm omxd

September 26th, 2013 subogero Comments off
Controls and playlist

Controls and playlist

My Raspberry Pi media player suite.

RemotePi

Browse music/videos, internet radio stations and even YouTube on your Raspberry Pi and play them on your audio system or non-smart HDMI TV.

Control everything from your smartphone via the RemotePi web-app. You don’t need an extra remote control, you don’t need SSH, you don’t need to connect physically to the Raspberry Pi, and you definitely don’t need to start the GUI.

Features

RemotePi uses the omxd/omxplayer backend.

Show what omxd is playing now, including album art or YouTube thumbnail and actual/overall clip time.

Limit file browsing within the Raspberry Pi’s file system to a root directory specified in a config file /etc/remotepi.cfg.

Insert, add and append files, directories or internet radio streams to the omxd playlist.

You can even interrupt the playlist to play a single file or a YouTube clip. The playlist resumes after the end of the movie.

Control buttons: Play/pause/stop playback, or navigate in the playlist with repeat, previous/next, Rewind/FastForward 30/600s, or previous or next album buttons.

Delete actual track or entire playlist.

Cycle through subtitles and soundtracks, switch audio output to HDMI of Jack.

RemotePi is strictly file and directory based and could not care less for MP3 tags in files. It does not use 3rd party databases for metadata, except for internet radio. Best used with a nicely arranged folder structure.

Browsing files

Browsing files

Goa internet radio stations

Goa stations

Watch YouTube on your TV

Watch YouTube on your TV

Installation

Source code and installation instructions on Github.

Tecnology

RemotePi is a single HTML5 webpage, you load it once and then it communicates with the Raspberry Pi with AJAX calls using JavaScript. It won’t work it JavaScript is disabled.

youtube-dl

From rg3, RemotePi uses this program to extract direct stream links from YouTube page URLs. You can install it from Raspbian, as well.

rpi.fm

Who does not love internet radio stations? Who does not like to listen to Liquid Funk or Detroit Techno or Verdi all day long? Now you can do it easily on your Raspberry Pi. rpi.fm is an interactive command line tool that allows you to browse hundreds of genres and thousands of stations from the database of internet-radio.com.

Check out the details on Github.

rpi.fm also uses the omxd media player daemon for playback.

omxd

But what the hell is omxd, you may ask. Well, you probably know the brilliant omxplayer that plays full HD videos using the GPU of the Raspberry Pi. It has a few limitations, though. It can play only one file at a time, unless you are a scary shell-wizard.

Would not it be great, if you could add your entire music collection to its playlist in one step?

Would not it be great if you could log out of your SSH/GUI session and it would keep cranking out Psytekk?

Would not it be great to interrupt your music playlist, watch a movie, and continue your playlist where you left off, after Vincent Vega and Jules leave the restaurant?

Exactly. That’s what omxd, the omxplayer daemon does. Maintain, manipulate a playlist, and play it in the background. It’s the heart and soul of RemotePi and rpi.fm.

Check it out on Github.

Future Features

Debian packages for all three tools. So all you have to do for installation is:

sudo apt-get install remotepi

HP Laserjet P1102 on Raspberry Pi

May 22nd, 2013 subogero 3 comments

Printing is evil. Back in the day this evilness sparked even Richard Stallman’s ire and so the Free Software Movement was born. It was 1980 and the new Xerox 9700 laser printer’s proprietary software could not be modified to add an email notification feature.

Fast forward 33 years. The HP Laserjet P1102 has a proprietary driver plugin. It’s freely available for the hp-setup program on Linux. Except, that HP does not compile it for the armv6 architecture. So my home print server, a Raspberry Pi, did not work with my new shiny P1102.

Until I found Rick Richardson’s foo2jzs driver, that is. Instructions and download on http://foo2zjs.rkkda.com.

Morale of the story: free software has nothing to do with communism. On the contrary: it promotes private property rights. The communist, a.k.a. thief, is HP. I bought their printer with my own money, but the tricky bastards did not allow me to use it. A classic case of “redistribution of wealth”. It was Rick’s opens source driver that saved my day.

rpi.fm My Stations

May 20th, 2013 subogero Comments off

The Raspberry Pi kicks ass as a media player. Thanks to, among others, rpi.fm, my and my friends’ favourite internet radio player.

Ladies and Gentlemen!

Upon popular user request, I hereby present its new “My Stations” function. rpi.fm now remembers whenever you choose a station. It builds a list from them, ranked by how often you’ve chosen each, and offers a new command “m” to access them.

Enjoy. But only after you’ve updated rpi.fm form Github.

For those in the know:

cd rpi.fm
git pull origin master
sudo make install
rpi.fm

For rpi.fm Noobs:

git clone https://github.com/subogero/rpi.fm
cd rpi.fm
sudo make install
rpi.fm

Lured into Linux Real Time

April 13th, 2013 subogero Comments off

The support period of the Lucid Lynx on the Asus UL20A is about to end in May, so I was looking for a new OS. I gave a try to Debian Testing, or Linux Mint Debian Edition, to be precise. But not Pangolin. Sorry.

It was my smoothest Linux install ever. All hardware worked out of the box. Volume and brightness buttons? Check. Wifi? Check. Two-finger scrolling? Check. I also applied the usual MATE fixes for mutt and the power-button. Debian comes with a 486-kernel by default, so it handles only one processor core. One can upgrade manually to a multi-core 686-kernel with apt-get.

aptitude search linux-image

And what do I find?

linux-image-rt-686-pae

A PREEMPT_RT patched kernel, kindly compiled by Debian. Time to plunge into real-time again!

I put together a Linux syscall tutorial project a year ago (see github) which contained a small program performing a periodic real-time task and printing statistics like average/min/max dt and its standard deviation. It’s called rt and uses the setitimer() system call to generate periodic SIGALRM signals. Time to measure latencies of a 1ms periodic task on a RT kernel!

To my utter disappointment, performance was fine, until the tapeta daemon changed my wallpaper, that is. At which point a 300% latency appeared. That’s right, period time 4ms instead of 1ms. Same as earlier, without an RT kernel. What’s wrong?

I was not using real-time scheduling, that’s what. It turned out Linux processes run at static priority zero by default, where the actual scheduling priorities are dynamic, depending on interactivity, nice level, sleep/runtime, etc. This is called the SCHED_OTHER policy. In other words, one is at the mercy of the Completely Fair Scheduler.

On the other hand, a process’ static priority can be increased, and the scheduling policy can be changed to SCHED_FIFO with the sched_setscheduler() system call. In this case the only one whose mercy we are at, is another SCHED_FIFO or SCHED_RR (Round Robin) scheduled process with equal or higher static priority, lurking in the background somewhere. Important to note, though, a process needs to run as root to be able to change its own or another process’ scheduling. There is also a command line utility called chrt, to change the scheduling policy of processes.

By the way. I’m not using threads. Threads are evil. People who use them mostly condemn themselves to a thousand hells of corrupted data, deadlocks and, at the end of the day, slower performance, thanks to cache misses and dirty cache lines. And the uClibc library in many embedded systems does not support POSIX threads. Thank goodness. I’m particularly angry with threads, as every damn RT-Linux tutorial you look at, spends 75% with setting up the bloody things. And you even have to worry about which thread your signals will be delivered to. What about learning fork() and pipe() instead? KEEP IT SIMPLE STUPID!

But enough of my rants and dodgy theories, let’s see the practice. First let’s see, how the skeleton of a periodic task looks like.

#include <unistd.h>
#include <signal.h>
#include <sys/time.h>

/* Periodic SIGALRM handler routine: everything happens here */
void periodic(int signal)
{
    ... /* do your periodic stuff */
    if (continue_running)
        signal(SIGALRM, periodic);
    else
        exit(0);
}

int main(int argc, char *argv[])
{
    ... /* initialize your stuff */
    struct itimerval period = {
        { 0, 1000, }, /* 1st signal in [s], [us] */
        { 0, 1000, }, /* period time   [s], [us] */
    };
    signal(SIGALRM, periodic);       /* install periodic() to handle SIGALRM */
    setitimer(ITIMER_REAL, &period, NULL); /* start periodic SIGALRM signals */
    /* Main idle loop: everything done by the signal handler */
    while (1)
        pause();
    return 0;
}

Next, how to set up real-time scheduling. Add this code to main(), before the setup of the periodic stuff. Remember to run your program as root for the below code to take effect.

#include <sched.h>
...
int main(int argc, char *argv[])
{
    ...
    struct sched_param schedp;
    schedp.sched_priority = 1;
    sched_setscheduler(0, SCHED_FIFO, &schedp);
    ...
}

And now let’s see what happens when a process becomes real-time. I’m measuring latencies of a 1ms task on Raspberry Pi, actually without a PREEMPT_RT kernel. I start rt as a normal process, then change its scheduling to SCHED_FIFO with chrt. See the code of rt on github.

szg@og314 ~/syscalls $ ./rt 1000
Real time child process PID 3368
------------------------------------------------------------------------
     n  Mean [us]       [us] SD  [%]      [us] Min [%]      [us] Max [%]
------------------------------------------------------------------------
   982   1017.502    246.148  24.615        67  93.300      4449 344.900
   985   1015.249    247.086  24.709        71  92.900      4865 386.500
   975   1025.625    300.905  30.091        63  93.700      4669 366.900
   989   1011.116    162.667  16.267       181  81.900      2682 168.200
   986   1014.260    155.867  15.587       259  74.100      2506 150.600
   989   1011.115    163.982  16.398        51  94.900      2674 167.400
   983   1017.492    189.123  18.912        59  94.100      2803 180.300
   983   1017.102    199.678  19.968        74  92.600      3985 298.500
   985   1015.169    214.607  21.461        62  93.800      3938 293.800
   978   1022.632    267.480  26.748        83  91.700      4355 335.500
   988   1012.008    165.071  16.507        84  91.600      2707 170.700
   991   1009.079    222.581  22.258        72  92.800      6762 576.200
  1000   1000.001     18.526   1.853       769  23.100      1218  21.800
  1000   1000.040     16.464   1.646       834  16.600      1163  16.300
  1000    999.958     12.861   1.286       924   7.600      1086   8.600
  1000   1000.002     17.503   1.750       809  19.100      1223  22.300
  1000   1000.014     20.984   2.098       793  20.700      1212  21.200
  1000   1000.035     24.159   2.416       827  17.300      1175  17.500
  1000    999.998     26.596   2.660       795  20.500      1246  24.600
  1000    999.962     18.433   1.843       844  15.600      1152  15.200
  1000    999.989     34.235   3.424       716  28.400      1314  31.400
  1000   1000.003     18.630   1.863       849  15.100      1214  21.400
  1000   1000.001     22.666   2.267       717  28.300      1268  26.800
  1000    999.996     20.419   2.042       799  20.100      1196  19.600
  1000   1000.051     15.729   1.573       831  16.900      1159  15.900
  1000    999.951     13.385   1.339       920   8.000      1109  10.900
  1000   1000.000     18.562   1.856       830  17.000      1210  21.000
  1000   1000.002     28.782   2.878       812  18.800      1222  22.200
  1000   1000.039     19.713   1.971       763  23.700      1239  23.900
  1000    999.964     25.291   2.529       907   9.300      1118  11.800
     1   1035.000     35.000   3.500      1035  -3.500      1035   3.500
------------------------------------------------------------------------
     n  Mean [us]       [us] SD  [%]      [us] Min [%]      [us] Max [%]
------------------------------------------------------------------------
 29815   1006.212    136.875  13.687        51  94.900      6762 576.200

A line of statistics is printed after every second with the following columns: n (number of events), Mean dt, SD (standard deviation), and Min/Max dt. Whilst the program was running, after the 12th line of data above, I changed rt’s scheduling with the command below.

root@og314 ~ # chrt -f -p 1 3368  # -f : SCHED_FIFO, 1 : static priority 1

The effect was truly dramatic. With normal scheduling I was missing about 2% of the events, with an average latency of 20% (0.2ms), in the worst case up to 600% (6ms). After changing to real-time scheduling, not a single event went lost, average latency dropped to 2% (0.02ms!), and the worst case delay dropped to 31% (0.31ms).

All this is causing 7-8 % processor load on a Raspberry Pi running two ssh sessions and omxplayer playing an internet radio. Single core, no PREEMPT_RT kernel. You probably need the latter in a production system which must not miss a beat for weeks.

So Linux is definitely able to run an ECU. That’s where my new ARIA25 board will excel. After someone designs an ECU for me…

Raspberry Pi Internet Radio

December 22nd, 2012 subogero 2 comments

I’ve mentioned already that the Raspberry has become a rather integral part of my flat. And now I mention that I’m a great consumer of ska, reggae, soul, house, samba, sambass, funk and liquid funk.

And that’s where internet radios come into the picture. But it’s been so awkward so far. Hook up the phone to the stereo’s AUX cable? Start an entire desktop session on a laptop to play them in a browser and hook it up to the stereo’s AUX cable?

But now the Raspberry is in place, and it’s permanently connected to both the TV and the stereo. So I had to find a way to play internet radio stations from the command line. The internet did not reveal definitive answers, so I wrote a little Perl program.

It does some data-mining on www.internet-radio.com and presents an extremely simple interactive command line interface to choose genres and stations, start playback on the audio-jack or on HDMI, to play or pause, to quit or quit but continue playback in the background.

It’s very easy to use. I ssh sometimes into the Raspberry to change the station, and then it plays LiquidBass.net or something else all day long.

It’s called rpi.fm. Please find it on GitHub.

I tested it on Raspbian Wheezy. It uses omxplayer for playback. Important: the user must be in the “audio” group. The default user “pi” is, for new users edit /etc/group as root:

audio:x:29:pulse,pi,youruser

Next project: web-app for the same thing. And then the phone can be used as a remote control.

Raspberry Ping

December 10th, 2012 subogero Comments off

My long awaited Raspberry Pi has arrived about 2 weeks ago. I’ve ordered some accessories along with it, like an SD card, a HDMI cable and a power supply. What was missing though, was any kind of input device. I flashed the Raspbian image to the SD, hooked it to the TV and go.

It booted! And then waited hopelessly for me to log in.

Fortunately my flat has built-in Ethernet everywhere, so I connected the Pi, and tried to ssh into it from an Ubuntu laptop. Success! I typed “startx” and an orphaned LXDE session popped up on the TV. Rather useless without a mouse.

But then I remembered the so called X-server, which has to be the largest program that does nothing for you according to Ken Thompson. Except that it does. Unix GUI programs don’t talk to the screen/keyboard/mouse directly, but through an X-server that handles these. And my Ubuntu had one already running. And the GUI-X communication can run over a network as well. So…

foo@bar$ ssh -X pi@raspberry  # ssh into the RPi allowing X communication
pi@raspberry$ lxsession       # start an LXDE desktop session on the RPi

and what popped up was the proof for the superiority of this weird, unfriendly, minimalistic thing called Unix:

Ubuntu & Raspberry on same screen

Ubuntu & Raspberry on same screen

Ladies and gentlemen: 2 full desktop environments running on 1 screen! And that was it for the first weekend with the Pi.

For the second weekend, I bought some more accessories: Logitech K400 wireless keyboard with built-in touchpad, 1 TB USB hard-disk and a powered USB-hub. Time to build my low-power home server. Built into the hidden recesses of my desk.

Home server

Home server

Then I turned my USB printer into a network-printer installing CUPS on the Raspberry. The last adventure so far was setting up an FTP-server on said fruit. Vsftpd. VS probably means very secure, which usually stands for totally unusable. And yes, it was a bloody nightmare to set up. At the end it turned out you have to comment out nearly all lines in the bloody /etc/pam.d/vsftpd file. Oh well.

Linux desktops on a desktop

Linux desktops on a desktop

Categories: Uncategorized Tags: , , ,