subogero linux blog

redupload, redupdate, redshow, and now redlist. The new one prints the list of issues in plain text, project-wise or globally.

$ redlist -p test localhost/redmine
# Tracker Status      Subject      Assignee      Due date   
2 Feature In Progress Test Feature Szabó Gergely            
1 Bug     In Progress Test Issue   Szabó Gergely 2013-05-26

Uploading files? Check.
Creating or updating issues? Check.
Showing issue details? Check.
Listing issues? Check.

Redlist uses the CSV export feature of Redmine. As Text::CSV is not part of the standard Perl library, I decided to write my own CSV parser. It seemed easy until I tried it on an old mine, 0.9.3 to be precise. In which case it fell apart as the bastard puts each and every existing attribute into the CSV, including the multiline Description. What started with two simple regular expressions, ended up with five, plus a state-machine.

There was some trouble with encoding too. For best results, set your terminal’s locale to utf-8, as well as Redmine’s csv-encoding. On Debian, the yml files are in /usr/share/redmine/config/locales/ for each language. Look for

   general_csv_encoding: UTF-8

Bot the old mine’s output is still too wide for a terminal. Until you define your personal filters, that is. Create ~/.redgit and add

[redlist]
Created
Updated
Description
Parent task
Estimated time

Whatever. It’s still on Github: http://github.com/subogero/redgit

We love Redmine. It’s everyone’s favourite integrated bug-tracker-wiki-gantt-roadmap-web-app running on Rails. It has everything. Except a decent command line interface.

There are some half-baked attempts on the internet, usually having suffered a few years’ time of bitrot and featuring loads of undocumented Ruby-related dependencies, so-called gems in Ruby-speak. I tried to install a few, but not being a hard-core Rubyist, I had to give up.

Long ages of desperation followed. Until I looked into Redmine’s email interface. Which, it turns out, is a command-line interface in disguise. I decided to leave Exim4 well alone, and write a user friendly command line wrapper around the obvious

/usr/share/redmine/extra/mail_handler/rdm-mailhandler.rb

My first attempt was a server-side only shell-script. But then I realized that any self-respecting person who uses the command-line has some sort of shell access to the Redmine server anyway, which also accommodates the central-ish git repos. So I added an option to run via SSH.

Ladies and Gentlemen: redupdate

Please find it on Github along with a few other scripts:
https://github.com/subogero/redgit

The other scripts include:

• redupload: upload files to Redmine
• redshow: view issues in plain text format
• gitcreate: create bare repo, add HTTP access, optional email notification upon pushes

I’ve mentioned already that the Raspberry has become a rather integral part of my flat. And now I mention that I’m a great consumer of ska, reggae, soul, house, samba, sambass, funk and liquid funk.

And that’s where internet radios come into the picture. But it’s been so awkward so far. Hook up the phone to the stereo’s AUX cable? Start an entire desktop session on a laptop to play them in a browser and hook it up to the stereo’s AUX cable?

But now the Raspberry is in place, and it’s permanently connected to both the TV and the stereo. So I had to find a way to play internet radio stations from the command line. The internet did not reveal definitive answers, so I wrote a little Perl program.

It does some data-mining on www.internet-radio.com and presents an extremely simple interactive command line interface to choose genres and stations, start playback on the audio-jack or on HDMI, to play or pause, to quit or quit but continue playback in the background.

It’s very easy to use. I ssh sometimes into the Raspberry to change the station, and then it plays LiquidBass.net or something else all day long.

It’s called rpi.fm. Please find it on GitHub.

I tested it on Raspbian Wheezy. It uses omxplayer for playback. Important: the user must be in the “audio” group. The default user “pi” is, for new users edit /etc/group as root:

audio:x:29:pulse,pi,youruser

Next project: web-app for the same thing. And then the phone can be used as a remote control.

My new favourite distro is Debian Squeeze. I use some installations as servers and virtual machines. They all share a surprising feature for a 3 year-old Linux Noob:

No GUI

No desktops, no menus, no windows and no mice, just sshd and the Bourne Again Shell. I’ve been struggling with git fetch/push on such machines for a while, having to add the passphrase of my private key every time. Until now, that is.

Basic command-line installations of Debian Squeeze don’t run ssh-agent automatically like the GUI versions of popular Linux distros. The setup is up to the user or sysadmin.

My solution provides the following features:

• ssh-agent runs on a per-user basis as a daemon (not per-system and not per-session)
• user enters pass-phrases once per power-cycle of the machine (not once per logging in)

The solution is implemented in /etc/profile:

# SSH Agent
export SSH_AUTH_SOCK=~/.ssh/.ssh-socket
echo --- LIST ADDED KEYS ---
ssh-add -l
if [ $? = 2 ]; then
 echo --- ADD KEYS ---
 rm -f ~/.ssh/.ssh-{socket,agent-pid,script}
 ssh-agent -a $SSH_AUTH_SOCK 2>/dev/null >~/.ssh/.ssh-script
 . ~/.ssh/.ssh-script >/dev/null
 echo $SSH_AGENT_PID >~/.ssh/.ssh-agent-pid
 ssh-add .ssh*/id_rsa
fi

Adding keys to ssh-agent forever on a server might be a security risk, but defining a timeout is also possible, the example below shows one enough for a workday (8 hours):

ssh-add -t 28800 .ssh*/id_rsa

szg is the Lazy Man’s command line calculator, crunching the maximum number of numbers with the minimum number of keystrokes.

szg stands for SZámolóGép, which is Hungarian for calculator.
It also happens to be the initials of the author, a particularly Lazy Man.

If you you’re lazy too;

If you like binary, octal and hex formats;

If you hate typing too much;

If you hate mouse-clicking even more;

If you hate calculators going floating-point after the first division;

If you still need floating-point occasionally;

If you typo a lot and prefer to undo them all;

If you like to comment your calculations (well, you don’t, but it’s possible):

Accept no substitute! Use szg.

Features

• 32-bit integer or floating-point arithmetic
• decimal, binary, octal or hexadecimal format (commands D, B, O, X)
• input of expressions, variable assignments and commands on stdin
• input of expressions directly from the command line
• output of calculations on stdout
• error messages (syntax error, division by zero) or stderr
• unlimited undo
• user defined variables like $foo
• operators ()~ ^ */%& +-| =
• symbol _ means last result, like Perl/Python
• missing identifier means _ (+5 means last result + 5) like Perl (sorry Python)
• math functions @s @c @a @l @e @r = sin cos atan log exp sqrt
• optional unsigned arithmetic (commands N, S)
• floating-point not upon division, only by user request, like Python (but not Python3)
• combination if expressions/commands on same line
• comments starting with #
• interpreter mode when input file specified
• manual page

Installation

Debian Packages

The easiest way to install szg is using apt-get on Debian-based systems. Add my Debian repo to /etc/apt/sources.list:

deb http://linux.subogero.com/deb/ /
deb-src http://linux.subogero.com/deb/ /

And then

# apt-get update
# apt-get install szg

Source Tarball

From http://linux.subogero.com/deb/ get szg_X.Y.orig.tar.gz.
szg can be compiled and installed on Linux or Windows/Cygwin. On Windows, it runs both in Cygwin and Windows command line windows. Download and extract the newest one, then

$ cd szg-X.Y
$ ./configure
$ make
# make install # as root

Windows

From http://linux.subogero.com/deb/ get szg_X.Y.zip.
Download and extract the newest one, then copy szg.exe to C:\WINDOWS or another folder which is on PATH.

Git for the Brave

$ git clone http://subogero.dyndns.org/git/szg.git || \
  git clone https://github.com/subogero/szg.git
$ cd szg
$ ./configure
$ make
$ make install # as root

Examples

Desktop calculator workflow with a twist

d 5000    # one steak
5000
d +3*2000 # three whiskies neat
11000
d +1000   # tip
12000
d 20000-  # Let's see how much money I've got left from my 20000 note
8000
d

Command/statement combination for hex/dec conversion, signed/unsigned modes. Watch the prompt!

d XffffD       # change to hex, enter FFFF, change back to dec
65535
d X ffffffff D # change to hex, enter FFFFFFFF, change back to dec
-1
d N            # unsigned (natural number) mode
4294967295
D 240X         # stay in dec, enter 240, change to hex
f0
X D239X        # change to dec, enter 239, change to hex
ef
X

Undo, difference between subtraction and unary minus.

84
d ~42        # Let's subtract 42 ...
-42
d U-42       # Oops, I hit tilde instead of minus
42

szg works as an interpreter as well, an input file can be specified on the command line. This also allows to create executable szg interpreter scripts. For instance let’s create an executable file called LifeUniverseAndEverything:

#!/usr/bin/szg
33*2 X

szg supports floats and binary formats. You can even peek into the insides of floats.

D 1+2
3
d /2    # no implicit float-conversion upon division
1
d /2.0  # Pythonic float-conversion if float-format entered
0.5
f B     # let's peek inside
00111111 00000000 00000000 00000000
f

You can specify an expression on the command line, to maximize laziness:

$ szg -e XffD
255

Background

This is the first remake of my first shot at a unix-style program. In the first shot you had to enter too many capital letters for hex numbers.
It’s a filter, as in unix nearly everything should be. I tried to follow the Sacred Rules of the Art from esr’s book:

Command line option conventions

-h –help for help
-V –version for version info

Data driven development

The source of the usage and version screens are text files which look exactly like those screens. During make, they are sedded into header files with lists of string initializers. The headers are then included in the middle of const table definitions, to make the text available as a list of strings, line by line.

Undo is possible thanks to a stack data-structure and the user-defined variables’ heart and soul is a hash-table, with buckets and all that.

Write programs to write programs

I used Lex and Yacc to parse the input and to write the syntax. I can still hardly believe what these nearly 40 year old tools can do.

The command line has been outdated for 20 years.

This is the claim I heard yesterday from a colleague, along with a statement that he needs useable interfaces. The revelation came during a discussion about git vs MKS Source Integrity*.

Why, one might ask, comes Windows 8 Server, for the first time in history, without a GUI?  Why do retarded Linux freaks still claim that the command line is way more effective than any GUI?

The shocking answer is that, surprisingly, the command line is our natural way of communication since the dawn of the human race. We have an organ to form and send text streams. It’s called the mouth. We have another pair of organs that receive text streams. The ears, ladies and gentlemen. We also have a way of batch-processing these messages. Some would call it reading and writing. Others call it literacy.

That’s the reason that, against all odds, the only remaining paradigm of the last half century of computing is Unix, which embraced TEXT as its core value. Everything is a file, in other words a stream of characters, text.

What is the very essence of the C language, the ultimate superclass of all superclasses, which is compatible with everything by definition? It’s this, the pointer to the universal byte stream:

void *

And we’ve just arrived to the most crucial question. Why do so many people still despise the command line? Exactly. Because it involves learning languages.

But come on! By the age of two, everybody has learned one. It’s not that difficult. Of course, we all start with GUIs. We click our toys. We play with the mother of all GUIs, our Mom’s b… erm… buttons.

But as time passes, our parents watch with ecstatic joy as we form our first text streams. They tell their friends about it. Then we go to school and what do we learn first? Scripting.

I’ll go further. The command line is older than the human race. Text is more universal than the universe.

In the beginning was the Word, and the Word was with God, and the Word was God.

Chew on that, you serial clickers.

* Some would propose a better name: MKS Source Disintegration.

ogc 6.0 rc2 is out for the general public to enjoy. It now supports any number of user defined variables in the form of $FOO or $SPAM. Dollar sign plus all-caps. I know, it’s somewhat LarryWallesque. The dollar-sign, not the all-caps, you understand.

Just to show the severity of my mental state, I’ve implemented “hash-buckets” in C. With vi.

The new version of the Lazy Man’s calculator maximizes laziness by taking your expressions directly from the command line. No pipes, no interactivity, no interpreter-scripts. Just what the customer wants.

$ ogc -e 2*21
42

The clever customer can actually take ogc’s laziness to even greater extremes. Just add the following line to ~/.bashrc

alias og='ogc -e'

And voilá:

$ og xFFd
255

You can download sources and Linux/Win32 binaries from the ogc page.

Geeks can clone the git repo as well:

git clone https://github.com/subogero/ogc.git
git clone http://subogero.dyndns.org/git/ogc.git

The introduction of the lex (flex) tokenizer into the ogc development allows an incredible amount of new bloatware to be implemented. The first menacing omen was floating-point support.

And now it’s math functions. Our friends from <math.h> sin, cos, atan, log, exp, sqrt are available as @s @c @a @l @e @r. The little bastards, besides looking very ugly, also perform a sneaky implicit floating-point conversion.

Check it out. But where will this all end?

Erm… ogc 4.0 had some certain erm… bugs in its syntax.

But ogc 4.1 is now downloadable from the ogc page. Including native Linux and Windows binaries.