subogero linux blog

Being a bloody webmaster has taught me a few hard lessons about git submodules.

The reason is that my CGI scripts managing the mailing lists and adding the navigation to the web pages proved to be quite reusable and I’ve moved them into their own git project. They are now used at at least 3 websites at the office as git submodules.

When Mr Noob first starts playing with them, usually this happens:

You have a project in ~/foo and another in ~/bar. Foo wants to reuse Bar as a subproject. So you call

$ cd foo
$ git submodule add ~/bar

It’s all very nice. Until, as usual, your Foo project becomes very popular and the whole Galaxy starts clonig it. At this moment great panic of cosmic proportions breaks out, as they cannot obtain the Bar submodule’s files from

/home/Noob/bar

Of course not. You get it? The Tao of git submodules is this:

Think Big.

In other words specify a path to your submodule in a universal way. With a Universal Resource Locator.

The used protocol is also of extreme importance. It should allow unlimited read access for your entire target audience. For write-access they can always add another remote to the subproject later. If they get write-access at all.

/somewhere/in/your/filesystem

We have already discussed this.
Bad.

smb://

Some will have access, but most not. Samba is a LAN thingy. And it’s bloody slow.
Bad.

ssh://

It’s a URL, but it’s not universal enough, as your users won’t have access.
Bad.

http://

Put your subproject on a webserver, allow directory listing: universal read access obtained.
Good. If a bit slow with git.

git://

Guarantees universal read access too, and it’s fast as well.
Good. The only problem is some corporate proxies are watching port 9418 with raised eyebrows.

Bottom line: use the http:// or git:// protocols.

To cut a long story short, the number of Linux kernels running at home has increased. Again.

This time it’s a TP-Link WR1043ND router managing my new internet connection. And check this: the package came with a copy of the GNU General Public License. We live in a beautiful world!

The first thing I did was to overwrite the factory firmware with OpenWrt. It’s a fantastic thing. At the moment of writing, I’m running the following services on it:

• USB hard disk with some ext3 partitions attached (big)
• My own git-hub
• A home file-server using Samba
• Transmission bit-torrent client
• It even has its own domain name

Git was a bit tricky, as it’s big, so it’s installed into /opt, mounted from the USB HDD.

I have a theory, that the real solution to increasing the capacity of such routers is not extroot, not /opt, but mounting a partition to /usr.

But I have not tried it. Yet.

It happened that I had to put together a few html pages containing important info about a certain project at the company. Originally I stored them on a Samba share on the official Windows file-server. I turned out to be terrible. For instance Firefox needs FIVE slashes after the protocol-id in the URL:

file://///server/share/page.html

IE needs two, Chrome needs four. First step in the downward spiral: I need the http protocol, in other words a webserver. So I installed Cygwin’s lighttpd on our constantly running desktop box. Then moved the html files to “/srv/www/htdocs/site/”. First step done.

Next step: it’s very cumbersome to change the navigation links, whenever a new page is added. Time to automatize. Tools: SSI (server side includes) and CGI scripts in Perl.

Then, what about setting up a few mailing lists. No archiving, just free subscribing and sending mail via “mailto:” links. One day job in Perl.

Later SSI turned out to be a bad choice as you need an absolute path to your CGI script. Difficult if you want a test-site. So I rewrote the entire site that all links actually refer to the same CGI script which loads the required page, adding dynamic navigation.

The trickiest and nicest part is managing the website with git. On the server I set up a git repo in my home folder, with a detached worktree in “/srv/www/htdocs/site”:

server ~$ mkdir site.git && cd site.git
server ~/site.git$ git --bare init
server ~/site.git$ git config core.worktree '/srv/www/htdocs/site'
server ~/site.git$ git config receive.denycurrentbranch false

Additionally, we need a hook routine which checks out anything to the website whenever you push it into this repo:

server ~/site.git$ mv hooks/post-receive.sample hooks/post-receive
server ~/site.git$ mcedit hooks/post-receive
# checkout the received branch to the website
read OLD NEW REF
git checkout -f $RE

On my own machine, I also have a git repo to manage the site. To allow easy server-updates, I did this:

mybox ~/site$ git remote add web ssh://server/~/site.git
mybox ~/site$ git push web master

Finally, one wants a test-site. Lighty endeavours to give satisfaction, so simply enable module mod_userdir in /etc/lighttpd/lighttpd.conf, and each user’s public_html folder becomes his own website. The two lines below now refer to the same folder:

/home/foo/public_html/
http://server/~foo/

Let’s create a non-bare repo here and configure it to receive and automatically check out pushed commits:

server ~/public_html$ mkdir site && cd site
server ~/public_html/site$ git init
server ~/public_html/site$ git config receive.denycurrentbranch false

I applied the same post-receive hook as with the official website’s repo. But it refused to check out the pushed commits. After a considerable amount of cursing it turns out the bloody git runs the hooks in the “.git” folder. I discovered this when I found all my html files there. So the post-receive hook should look like this in a non bare repo:

#!/bin/sh
unset GIT_DIR
unset GIT_WORK_TREE
cd ..
read OLD NEW REF
REF=`echo $REF | sed 's:refs/..*/::'` # branch instead of detached head
git checkout -f $REF

On my box I added an other remote to allow pushing to the test site:

mybox ~/site$ git remote add test ssh://server/~/public_html/site
mybox ~/site$ git push test master

I even applied a post-commit hook on the local machine which pushes master-commits to the official site and branch-commits to the test site.

I’ve measured the reboot time after the Lucid install on the ASUS UL20A. That’s what I call progress.

                         Jaunty  Lucid
--------------------------------------
from power-on until login  19 s   20 s
from login until ready     15 s    5 s
power off                  12 s    5 s
--------------------------------------
overall reboot time        46 s   35 s

The introduction of the lex (flex) tokenizer into the ogc development allows an incredible amount of new bloatware to be implemented. The first menacing omen was floating-point support.

And now it’s math functions. Our friends from <math.h> sin, cos, atan, log, exp, sqrt are available as @s @c @a @l @e @r. The little bastards, besides looking very ugly, also perform a sneaky implicit floating-point conversion.

Check it out. But where will this all end?

Well, Jaunty was not able to use the full (you know what I mean) resolution of my new TV, so I thought it was time to upgrade. I started Lucid Lynx from a live USB, attached the TV and, to my utter amazement, I was immediately presented with 2 073 600 deep purple pixels.

Upgrade time! Or even more than that. Time to reorganize my partitions. All I left was the original 200 GB home partition, now mounted as “/public”, mainly for my strictly legal (in Hungary) music and movies. If you live in the US of A, do not do this! The monopolists will confiscate your possessions, kill your family and jail you for 2000 years.

Anyway, 65 GB of unused Windows 7 (which I had no other choice than to pay for) was permanently removed, replaced by a new “/home” partition. The remaining 50 GB became “/” (root for starters).

Installation went like a breeze, as usual with Ubuntu. WLAN connected immediately. The telly as an external full high definition monitor? You betcha. Import stuff from old user profile, apt-get all the important packages (gimp, development stuff, openarena), compile and install the freshest hypest midnight commander, and there you go.

One thing. The bloody LCD brightness buttons. They did not work at all. Nor could I set brightness any other way, including the GUI and my Jaunty hack. Nice. This is the point where most people start thinking about suicide.

Others, however, use Google. Which reveals the solution immediately in the form of an Ubuntu wiki page:

acpi-backlight=vendor

has to be added to the GRUB kernel command line. Since then it works beautifully, even the flickering of my Jaunty hack is a thing of the past.

While setting up the ssh-agent on Cygwin, I run into a small problem. After reboot, the ssh-agent is not running yet, but the /tmp/.ssh* files are still there from the previous session. When starting a Cygwin shell, the start of the ssh-agent fails, if these files have not been manually removed before.

Task: delete these files automatically at boot time. I googled it, and the Windows way of doing it seemed extremely complicated. As usual, as it springs to the lips of smug Linux geeks. Not me. Then I ran across the “cron” scheduling daemon somehow, and there it was: insert the line below into the “/etc/crontab” file.

@reboot SYSTEM rm -f /tmp/.ssh*

The rest of this post is a distilled version of my adventures with setting up cron on Cygwin 1.7.5.

cron is the Unix-equivalent of “Scheduled Tasks” in Windows, just better. For instance, you can schedule something to run at boot time (I may have mentioned that before) and, the scheduled commands don’t throw up a scary cmd-window on your desktop. So in the meantime I’ve moved all my scheduled tasks to cron.

Start the cygwin-setup and add the “cron” and “cygrunsrv” packages from the “Admin” category.

We’ll run cron as a service by user SYSTEM. Poor SYSTEM therefore needs a home directory and a shell. The “/etc/passwd” file will define them.

$ mkdir /root
$ chown SYSTEM:root /root
$ mcedit /etc/passwd
SYSTEM:*:......:/root:/bin/bash

The start the service:

$ cron-config
Do you want to remove or reinstall it (yes/no) yes
Do you want to install the cron daemon as a service? (yes/no) yes
Enter the value of CYGWIN for the daemon: [ ] ntsec
Do you want the cron daemon to run as yourself? (yes/no) no
Do you want to start the cron daemon as a service now? (yes/no) yes

Local users can now define their scheduled tasks like this (crontab will start your favourite editor):

$ crontab -e  # edit your user specific cron-table
HOME=/home/foo
PATH=/usr/local/bin:/usr/bin:/bin:$PATH
# testing
* * * * *   touch ~/cron
@reboot     ~/foo.sh
45 11 * * * ~/lunch_message_to_mates.sh

Domain users: it does not work. Poor cron is unable to run scheduled tasks on behalf of domain users on the machine. But there is another way: cron also runs stuff found in the system level cron table in “/etc/crontab”. So insert your suff there, so that SYSTEM does it on its own behalf:

$ touch /etc/crontab
$ chown SYSTEM /etc/crontab
$ mcedit /etc/crontab
HOME=/root
PATH=/usr/local/bin:/usr/bin:/bin:$PATH
* * * * *   SYSTEM touch ~/cron
@reboot     SYSTEM rm -f /tmp/.ssh*

Finally a few words about crontab entries. They are either environment settings or scheduled commands. As seen above, on Cygwin it’s best to  create a usable PATH. Home dir and shell are normally taken from “/etc/passwd”.

As to the columns of scheduled commands see the manual page.

If certain crontab entries do not run, the best diagnostic tool is this:

$ cronevents

It will print out details on all successful and failed commands.

Erm… ogc 4.0 had some certain erm… bugs in its syntax.

But ogc 4.1 is now downloadable from the ogc page. Including native Linux and Windows binaries.

The new version of the Lazy Man’s calculator is out.

It’s no more baffled by floating-point numbers. It became fluent in binary format as well, if you want to deal only with the usual suspects (zero and one).

See the ogc page.

See the new Corporate Git pages!

It’s about setting up git repos in Windows XP boxes, shared among each other via Cygwin/OpenSSH.