Archive

Posts Tagged ‘git’

Redmine Command Line Interface – redupdate

May 19th, 2013 subogero Comments off

We love Redmine. It’s everyone’s favourite integrated bug-tracker-wiki-gantt-roadmap-web-app running on Rails. It has everything. Except a decent command line interface.

There are some half-baked attempts on the internet, usually having suffered a few years’ time of bitrot and featuring loads of undocumented Ruby-related dependencies, so-called gems in Ruby-speak. I tried to install a few, but not being a hard-core Rubyist, I had to give up.

Long ages of desperation followed. Until I looked into Redmine’s email interface. Which, it turns out, is a command-line interface in disguise. I decided to leave Exim4 well alone, and write a user friendly command line wrapper around the obvious

/usr/share/redmine/extra/mail_handler/rdm-mailhandler.rb

My first attempt was a server-side only shell-script. But then I realized that any self-respecting person who uses the command-line has some sort of shell access to the Redmine server anyway, which also accommodates the central-ish git repos. So I added an option to run via SSH.

Ladies and Gentlemen: redupdate

Please find it on Github along with a few other scripts:
https://github.com/subogero/redgit

The other scripts include:

  • redupload: upload files to Redmine
  • redshow: view issues in plain text format
  • gitcreate: create bare repo, add HTTP access, optional email notification upon pushes

The Command Line Is Outdated

November 23rd, 2011 subogero Comments off

The command line has been outdated for 20 years.

This is the claim I heard yesterday from a colleague, along with a statement that he needs useable interfaces. The revelation came during a discussion about git vs MKS Source Integrity*.

Why, one might ask, comes Windows 8 Server, for the first time in history, without a GUI?  Why do retarded Linux freaks still claim that the command line is way more effective than any GUI?

The shocking answer is that, surprisingly, the command line is our natural way of communication since the dawn of the human race. We have an organ to form and send text streams. It’s called the mouth. We have another pair of organs that receive text streams. The ears, ladies and gentlemen. We also have a way of batch-processing these messages. Some would call it reading and writing. Others call it literacy.

That’s the reason that, against all odds, the only remaining paradigm of the last half century of computing is Unix, which embraced TEXT as its core value. Everything is a file, in other words a stream of characters, text.

What is the very essence of the C language, the ultimate superclass of all superclasses, which is compatible with everything by definition? It’s this, the pointer to the universal byte stream:

void *

And we’ve just arrived to the most crucial question. Why do so many people still despise the command line? Exactly. Because it involves learning languages.

But come on! By the age of two, everybody has learned one. It’s not that difficult. Of course, we all start with GUIs. We click our toys. We play with the mother of all GUIs, our Mom’s b… erm… buttons.

But as time passes, our parents watch with ecstatic joy as we form our first text streams. They tell their friends about it. Then we go to school and what do we learn first? Scripting.

I’ll go further. The command line is older than the human race. Text is more universal than the universe.

In the beginning was the Word, and the Word was with God, and the Word was God.

Chew on that, you serial clickers.

* Some would propose a better name: MKS Source Disintegration.

Categories: Uncategorized Tags: , , , ,

Gits and Bugs

October 24th, 2011 subogero 2 comments

One may wonder what was happening in the last few months. Everyone knows about the Euro-crisis. A certain Mr Erdogan has been trying to revive the Ottoman Empire, sharpening his Unix-skills, especially bashing, on Israel.  Mahmoud failed to turn Judea and Samaria into a Judenrein state at the UN. It also turned out that a single Israeli soldier is worth 1027 heroic Palestinian freedom fighters. What’s more, Muammar Kadhafi, in a markedly Perl-like fashion, said:

keep $power or die $!

Yes, but what about the real important stuff, you may ask. Oh, I see what you mean! Here it is.

I needed a recursive git-grep function to scan submodules as well. So I wrote one. Along with a vim-plugin, which is a beautiful piece of plagiarism, but that’s open source for you!

http://github.com/subogero/git-grep-recursive-vim

I got acquainted with gdb, the GNU Debugger, courtesy of Richard Stallman. Typical of him, it prints 2 pages of licensing info and political agenda when started up. But after you get rid of that,

alias gdb='gdb -q'

what a brilliant piece of software it is! Easy, comfortable, fast, debugging on the command line. It’s just incredible. Gdb is a bit better explained here:

http://betterexplained.com/articles/debugging-with-gdb/

I rewrote my calculator, because one had to type too many uppercase letters for hex numbers. It’s all lowercase now. And it’s called “szg”, and no, not after my initials. It means SZámolóGép, which is Hungarian for calculator. Nothing to do with Hungarian notation, though. Which one of my colleagues turned out to actually like. Strange taste. It must give a deep satisfaction, when after 2 years of maintenance “u8Foo” has become a signed long.

Anyway the new calculator is available on github:

http://github.com/subogero/szg

At the office, I’m now co-developing a rather large project in git with a colleague. Lots of fun…

Having mentioned Muammar Kadhafi, there was a lot of hype about the death of Steve Jobs recently. A much more important person has passed away too: Dennis Ritchie, creator of the C programming language, co-creator of Unix with Ken Thompson. As a tribute, some quotes from the two heroes:

Obviously, the person who had most influence on my career was Ken Thompson.
Dennis Ritchie
That brings me to Dennis Ritchie. Our collaboration has been a thing of beauty.
Ken Thompson 
Categories: Uncategorized Tags: , , , ,

Cheat Sheets

July 15th, 2011 subogero 1 comment

This is the collection of cheat sheets I’ve uploaded to other posts and pages. I’ve been thinking about the format a good deal and ended up going for something I otherwise frown upon: the Open Document Format. Why?

  • PDF? I hate it even more, and users can’t edit it
  • HTML? Can’t control the printed format, not dense enough
  • Plain text? Can’t format chapter titles to stand out, thus non-navigable with the eye
  • DOC? Come on…
git Cheat Sheet

git Cheat Sheet

vi Chet Sheet

vi Cheat Sheet

Copyright notice: All rights reserved. The cheat sheets can be copied, distributed, printed and modified only with the explicit written permission of the copyright holder.

Not.

Categories: Uncategorized Tags: , , ,

git Cheat Sheet

July 5th, 2011 subogero Comments off

Attention! This sheet is not for learning git. It does not explain most commands, it merely lists the syntax of their most frequently used forms.

git Cheat Sheet

git Cheat Sheet

Categories: Uncategorized Tags: ,

Extend OpenWrt

November 9th, 2010 subogero 4 comments

An OpenWrt router needs to be extended a bit to run big stuff. For instance to build a git-hub. The installed size of git is 91MB. One has an external USB HDD, of course, but what is the right way to do it? The forums are full of extroots and opts and similar stuff. I think they are all evil.

Extroot seems to involve compiling custom firmware images, while /opt brakes the nice Unixy directory structure (standard locations of programs, libs and daemons). There must be a better way:

Mount a partition of an external USB HDD to /usr

Just think about it. The really big files go to /usr/bin /usr/sbin and /usr/lib almost exclusively. Only small stuff goes to /etc, like config files and daemon startup scripts.

Today my theory was confirmed by practice. It’s very simple. These are the important steps:

1. Create an ext3 partition on your HDD just for this purpose

I actually did the partitioning before, on my normal Ubuntu laptop. It’s 2 GB and it’s called /dev/sda2 on the router.

2. Mount it to /opt temporarily

Create the /opt directory (or whatever else) temporarily and mount your partition into it using LUCI/Administration/System/Mount Points.

3. Copy the entire contents of /usr to /opt

The goal is that, initially, /usr looks the same with or without mounting something into it.

$ cp -dpr /usr/* /opt # preserve symlinks
4. Mount partition to /usr

Unmount /dev/sda2 from /opt, mount it into /usr using LUCI. You can also delete the temporary /opt directory.

5. Update /etc/opkg.conf

This involves OpenWrt’s most mysterious concept, overlay. Changing the line below has no effect except to calm down the worrying router about available storage-size for new packages:

option overlay_root /usr

You’re ready to install anything. As for me, I’ve since reinstalled mc and git. No dirty hacks this time with inserting /opt into PATH and manually creating some symlinks from /usr to /opt/usr. Just installed them like a breeze. My github is up and running again.

Categories: Uncategorized Tags: , , ,

Tao of Git Submodules 2

November 3rd, 2010 subogero Comments off

During my exploits to spread the Gospel of git-managed websites, once a great crisis broke out. A user threatened me he would return to MKS Source Integrity.

Now, such an event would shake the foundations of the entire Open Source Free Software movement. Richard Stallman would put sackcloth upon his loins. Linus Torvalds would rend his clothes. I would mourn for many days and my soul would refuse to be comforted.

Time to do something. Or rather, time to think.

I had actually been wondering, why Git’s submodule interface is so inconsistent. There is a –recursive clone, but there is no recursive option for any other remote operation. And then I was enlightened. Of course

There Is No Recursive Push

How could there be? The entire concept of submodules is based on universally read-accessible subprojects inserted into your superproject. You reuse one from gnu.org, another from the Moon, and some from Alpha Centauri. Write access for these is not guaranteed, it’s the privilege of a chosen few.

So what to do with your git-managed remote website? There are two options.

1. The remote repo has a detached worktree (repo in ~, website in /var/www)

Set up remotes and server-side post-receive hooks to check out for the subprojects.
You may want to add a pre-commit hook to the superproject pushing the submodules first.

2. The remote repo is in the website worktree

Add “git submodule update” to the post-update hook on the server side.
This assumes the server is able to access the submodule URLs.

Categories: Uncategorized Tags: , ,

Tao of Git Submodules

October 31st, 2010 subogero Comments off

Being a bloody webmaster has taught me a few hard lessons about git submodules.

The reason is that my CGI scripts managing the mailing lists and adding the navigation to the web pages proved to be quite reusable and I’ve moved them into their own git project. They are now used at at least 3 websites at the office as git submodules.

When Mr Noob first starts playing with them, usually this happens:

You have a project in ~/foo and another in ~/bar. Foo wants to reuse Bar as a subproject. So you call

$ cd foo
$ git submodule add ~/bar

It’s all very nice. Until, as usual, your Foo project becomes very popular and the whole Galaxy starts clonig it. At this moment great panic of cosmic proportions breaks out, as they cannot obtain the Bar submodule’s files from

/home/Noob/bar

Of course not. You get it? The Tao of git submodules is this:

Think Big.

In other words specify a path to your submodule in a universal way. With a Universal Resource Locator.

The used protocol is also of extreme importance. It should allow unlimited read access for your entire target audience. For write-access they can always add another remote to the subproject later. If they get write-access at all.

/somewhere/in/your/filesystem

We have already discussed this.
Bad.

smb://

Some will have access, but most not. Samba is a LAN thingy. And it’s bloody slow.
Bad.

ssh://

It’s a URL, but it’s not universal enough, as your users won’t have access.
Bad.

http://

Put your subproject on a webserver, allow directory listing: universal read access obtained.
Good. If a bit slow with git.

git://

Guarantees universal read access too, and it’s fast as well.
Good. The only problem is some corporate proxies are watching port 9418 with raised eyebrows.

Bottom line: use the http:// or git:// protocols.

Categories: Uncategorized Tags: , ,

OpenWrt

October 31st, 2010 subogero 2 comments

To cut a long story short, the number of Linux kernels running at home has increased. Again.

This time it’s a TP-Link WR1043ND router managing my new internet connection. And check this: the package came with a copy of the GNU General Public License. We live in a beautiful world!

The first thing I did was to overwrite the factory firmware with OpenWrt. It’s a fantastic thing. At the moment of writing, I’m running the following services on it:

  • USB hard disk with some ext3 partitions attached (big)
  • My own git-hub
  • A home file-server using Samba
  • Transmission bit-torrent client
  • It even has its own domain name

Git was a bit tricky, as it’s big, so it’s installed into /opt, mounted from the USB HDD.

I have a theory, that the real solution to increasing the capacity of such routers is not extroot, not /opt, but mounting a partition to /usr.

But I have not tried it. Yet.

I am a Webmaster

September 27th, 2010 subogero Comments off

It happened that I had to put together a few html pages containing important info about a certain project at the company. Originally I stored them on a Samba share on the official Windows file-server. I turned out to be terrible. For instance Firefox needs FIVE slashes after the protocol-id in the URL:

file://///server/share/page.html

IE needs two, Chrome needs four. First step in the downward spiral: I need the http protocol, in other words a webserver. So I installed Cygwin’s lighttpd on our constantly running desktop box. Then moved the html files to “/srv/www/htdocs/site/”. First step done.

Next step: it’s very cumbersome to change the navigation links, whenever a new page is added. Time to automatize. Tools: SSI (server side includes) and CGI scripts in Perl.

Then, what about setting up a few mailing lists. No archiving, just free subscribing and sending mail via “mailto:” links. One day job in Perl.

Later SSI turned out to be a bad choice as you need an absolute path to your CGI script. Difficult if you want a test-site. So I rewrote the entire site that all links actually refer to the same CGI script which loads the required page, adding dynamic navigation.

The trickiest and nicest part is managing the website with git. On the server I set up a git repo in my home folder, with a detached worktree in “/srv/www/htdocs/site”:

server ~$ mkdir site.git && cd site.git
server ~/site.git$ git --bare init
server ~/site.git$ git config core.worktree '/srv/www/htdocs/site'
server ~/site.git$ git config receive.denycurrentbranch false

Additionally, we need a hook routine which checks out anything to the website whenever you push it into this repo:

server ~/site.git$ mv hooks/post-receive.sample hooks/post-receive
server ~/site.git$ mcedit hooks/post-receive
# checkout the received branch to the website
read OLD NEW REF
git checkout -f $RE

On my own machine, I also have a git repo to manage the site. To allow easy server-updates, I did this:

mybox ~/site$ git remote add web ssh://server/~/site.git
mybox ~/site$ git push web master

Finally, one wants a test-site. Lighty endeavours to give satisfaction, so simply enable module mod_userdir in /etc/lighttpd/lighttpd.conf, and each user’s public_html folder becomes his own website. The two lines below now refer to the same folder:

/home/foo/public_html/
http://server/~foo/

Let’s create a non-bare repo here and configure it to receive and automatically check out pushed commits:

server ~/public_html$ mkdir site && cd site
server ~/public_html/site$ git init
server ~/public_html/site$ git config receive.denycurrentbranch false

I applied the same post-receive hook as with the official website’s repo. But it refused to check out the pushed commits. After a considerable amount of cursing it turns out the bloody git runs the hooks in the “.git” folder. I discovered this when I found all my html files there. So the post-receive hook should look like this in a non bare repo:

#!/bin/sh
unset GIT_DIR
unset GIT_WORK_TREE
cd ..
read OLD NEW REF
REF=`echo $REF | sed 's:refs/..*/::'` # branch instead of detached head
git checkout -f $REF

On my box I added an other remote to allow pushing to the test site:

mybox ~/site$ git remote add test ssh://server/~/public_html/site
mybox ~/site$ git push test master

I even applied a post-commit hook on the local machine which pushes master-commits to the official site and branch-commits to the test site.