Archive

Archive for November, 2010

Curling

November 29th, 2010 subogero Comments off

I’m not referring to the most boring sport EVER. Instead, it’s a command line tool I’ve discovered recently: curl. You specify a URL on the command line (e.g. curl http://napirajz.hu), and it prints the contents to standard output.

Time to write my most popular website ever:

The business case:
Bloody multinational corporations filter the internet. One cannot access inspiring content like http://napirajz.hu. Let’s make it accessible.

The tools:
My OpenWrt router hanging on the internet. I’ll create a mirror site on it, refreshed every day at 5:00 (before breakfast), 9:00 (arriving at the office), 13:00 (after lunch) and 18:00 (before I get home).

The beloved cron daemon will do the timing. And the mirroring? On OpenWrt, one lacks the luxuries or Perl and even the cosy comfort of Bash. One has to make do with the BusyBox versions of the Almquist Shell (ash), curl, sed and wget.

#!/bin/ash
PICS=`curl 'http://napirajz.hu' 2>/dev/null \
| sed -nr 's,^[\t ]*<p><img src=\"([a-z0-9/_.]+)\".+$,http://napirajz.hu\1,p'`
rm -f *.jp*
echo '<html><head><title>subogero napi</title></head><body>'
echo '<h1 align="center">subogero napi</h1>'
echo '<hr>'
for i in $PICS; do
 wget $i
 echo $i \
 | sed -r 's,^.+/([a-z0-9_]+)(\.[jpeg]+)$,<p><a href=\"napi/\1\2\">\1</a>,'
done
echo '<hr>'
echo '(c) CC - Mer&eacute;nyi D&aacute;niel'
echo '<br>Eredeti: h t t p : / / n a p i r a j z . h u'
echo '<p>Friss&iacute;tve:'
date "+%Y.%m.%d %H:%M"
echo '</body></html>'

See the result at http://subogero.dyndns.org/git/napi/

Categories: Uncategorized Tags: , , , , ,

Extend OpenWrt

November 9th, 2010 subogero 4 comments

An OpenWrt router needs to be extended a bit to run big stuff. For instance to build a git-hub. The installed size of git is 91MB. One has an external USB HDD, of course, but what is the right way to do it? The forums are full of extroots and opts and similar stuff. I think they are all evil.

Extroot seems to involve compiling custom firmware images, while /opt brakes the nice Unixy directory structure (standard locations of programs, libs and daemons). There must be a better way:

Mount a partition of an external USB HDD to /usr

Just think about it. The really big files go to /usr/bin /usr/sbin and /usr/lib almost exclusively. Only small stuff goes to /etc, like config files and daemon startup scripts.

Today my theory was confirmed by practice. It’s very simple. These are the important steps:

1. Create an ext3 partition on your HDD just for this purpose

I actually did the partitioning before, on my normal Ubuntu laptop. It’s 2 GB and it’s called /dev/sda2 on the router.

2. Mount it to /opt temporarily

Create the /opt directory (or whatever else) temporarily and mount your partition into it using LUCI/Administration/System/Mount Points.

3. Copy the entire contents of /usr to /opt

The goal is that, initially, /usr looks the same with or without mounting something into it.

$ cp -dpr /usr/* /opt # preserve symlinks
4. Mount partition to /usr

Unmount /dev/sda2 from /opt, mount it into /usr using LUCI. You can also delete the temporary /opt directory.

5. Update /etc/opkg.conf

This involves OpenWrt’s most mysterious concept, overlay. Changing the line below has no effect except to calm down the worrying router about available storage-size for new packages:

option overlay_root /usr

You’re ready to install anything. As for me, I’ve since reinstalled mc and git. No dirty hacks this time with inserting /opt into PATH and manually creating some symlinks from /usr to /opt/usr. Just installed them like a breeze. My github is up and running again.

Categories: Uncategorized Tags: , , ,

Tao of Git Submodules 2

November 3rd, 2010 subogero Comments off

During my exploits to spread the Gospel of git-managed websites, once a great crisis broke out. A user threatened me he would return to MKS Source Integrity.

Now, such an event would shake the foundations of the entire Open Source Free Software movement. Richard Stallman would put sackcloth upon his loins. Linus Torvalds would rend his clothes. I would mourn for many days and my soul would refuse to be comforted.

Time to do something. Or rather, time to think.

I had actually been wondering, why Git’s submodule interface is so inconsistent. There is a –recursive clone, but there is no recursive option for any other remote operation. And then I was enlightened. Of course

There Is No Recursive Push

How could there be? The entire concept of submodules is based on universally read-accessible subprojects inserted into your superproject. You reuse one from gnu.org, another from the Moon, and some from Alpha Centauri. Write access for these is not guaranteed, it’s the privilege of a chosen few.

So what to do with your git-managed remote website? There are two options.

1. The remote repo has a detached worktree (repo in ~, website in /var/www)

Set up remotes and server-side post-receive hooks to check out for the subprojects.
You may want to add a pre-commit hook to the superproject pushing the submodules first.

2. The remote repo is in the website worktree

Add “git submodule update” to the post-update hook on the server side.
This assumes the server is able to access the submodule URLs.

Categories: Uncategorized Tags: , ,