During my exploits to spread the Gospel of git-managed websites, once a great crisis broke out. A user threatened me he would return to MKS Source Integrity.

Now, such an event would shake the foundations of the entire Open Source Free Software movement. Richard Stallman would put sackcloth upon his loins. Linus Torvalds would rend his clothes. I would mourn for many days and my soul would refuse to be comforted.

Time to do something. Or rather, time to think.

I had actually been wondering, why Git’s submodule interface is so inconsistent. There is a –recursive clone, but there is no recursive option for any other remote operation. And then I was enlightened. Of course

There Is No Recursive Push

How could there be? The entire concept of submodules is based on universally read-accessible subprojects inserted into your superproject. You reuse one from gnu.org, another from the Moon, and some from Alpha Centauri. Write access for these is not guaranteed, it’s the privilege of a chosen few.

So what to do with your git-managed remote website? There are two options.

1. The remote repo has a detached worktree (repo in ~, website in /var/www)

Set up remotes and server-side post-receive hooks to check out for the subprojects.
You may want to add a pre-commit hook to the superproject pushing the submodules first.

2. The remote repo is in the website worktree

Add “git submodule update” to the post-update hook on the server side.
This assumes the server is able to access the submodule URLs.

Being a bloody webmaster has taught me a few hard lessons about git submodules.

The reason is that my CGI scripts managing the mailing lists and adding the navigation to the web pages proved to be quite reusable and I’ve moved them into their own git project. They are now used at at least 3 websites at the office as git submodules.

When Mr Noob first starts playing with them, usually this happens:

You have a project in ~/foo and another in ~/bar. Foo wants to reuse Bar as a subproject. So you call

$ cd foo
$ git submodule add ~/bar

It’s all very nice. Until, as usual, your Foo project becomes very popular and the whole Galaxy starts clonig it. At this moment great panic of cosmic proportions breaks out, as they cannot obtain the Bar submodule’s files from

/home/Noob/bar

Of course not. You get it? The Tao of git submodules is this:

Think Big.

In other words specify a path to your submodule in a universal way. With a Universal Resource Locator.

The used protocol is also of extreme importance. It should allow unlimited read access for your entire target audience. For write-access they can always add another remote to the subproject later. If they get write-access at all.

/somewhere/in/your/filesystem

We have already discussed this.
Bad.

smb://

Some will have access, but most not. Samba is a LAN thingy. And it’s bloody slow.
Bad.

ssh://

It’s a URL, but it’s not universal enough, as your users won’t have access.
Bad.

http://

Put your subproject on a webserver, allow directory listing: universal read access obtained.
Good. If a bit slow with git.

git://

Guarantees universal read access too, and it’s fast as well.
Good. The only problem is some corporate proxies are watching port 9418 with raised eyebrows.

Bottom line: use the http:// or git:// protocols.

This page should serve as a summary of my experience with these two version control systems and a comparison of their possible workflows. There are four main areas where I shall compare them:

• Basics
• Development in a team
• Code reuse
• Integration with task-tracking

I have a lot of experience with MKS in all areas but, as I’ve been working with git on my private hobby projects so far, I’m very much limited to Basics and Code reuse here. Actually my goal with this page is to help me remember those awkward git commands.

1 Basics

1.1 Central vs Distributed Repositories

MKS is a client-server program. There is a central server that stores the repositories of each Project. People work with Sandboxes, private copies of a Project, that allow status tracking and all sorts of operations on the Project as well. A Sandbox is represented by a foo.pj file and the Working Files.
The server is usually under the control of a very busy and important admin, so don’t even ask me how to create a project. Just write him an email and in a matter of days you’ll have your new project. Another few hours/days will be needed to sort out access rights. To create a Sandbox, select the Sandbox/Create… menu item in the GUI. Or use the command line:

~$ mkdir foo
~$ si createsandbox --project=foo.pj foo # sandbox file foo/foo.pj is created

In git, you always have your own repository, represented  by the .git directory in the root of your project. To create a repo and a copy of the Working Tree, simply type

~$ mkdir foo && cd foo
~/foo$ git init

Or you can copy an existing repo anywhere (any URL is possible), similar to a Sandbox (but it’s more than that):

~$ git clone //bar/Projects/foo
~$ cd foo

The client-server concept of MKS seems superior at first sight, but I’m not sure. If the server’s backup strategy is not just-so, disaster is threatening, while in git every developer has a full copy of the entire repo. With MKS there is absolutely no way to track your work offline. Nor can you do so if you’ve received Read-only access only. Client server architectures are also very slow, as most operations go through the network. So on second thoughts, I’m absolutely sure: distributed is far superior.

1.2 Simple Workflow

I’ll cover adding and updating a file to the project. In both cases it starts with creating the file.

In MKS you can use the GUI: Member/Add, Member/Lock, Member/Edit Working File, Member/Check in, Project/Checkpoint. Or using the command line:

~/foo$ echo '#!bin/sh' > foo
~/foo$ si add foo
~/foo$ si lock foo
~/foo$ echo 'echo foo' >> foo
~/foo$ si ci --update foo
~/foo$ si checkpoint --sandbox=foo.pj

In git it’s far simpler:

~/foo$ echo '#!bin/sh' > foo
~/foo$ git add foo
~/foo$ echo 'echo foo' >> foo
~/foo$ git commit -a

1.3 Files vs Content

The reason for git’s simplicity is that it’s tracking content, not files separately. A “commit -a” will check in everything you’ve changed in your Working Tree as one atomic operation. Hence the lack of the checkpoint operation as well. An explicit “add” is only necessary for new files.

Note: If you use commit without -a, you need “git add” for each changed file as well before commit.

2 Development in a Team

MKS handles co-operation with a repo on the central server, private sandboxes and file Locks. A Lock means that only the locking developer is allowed to Check In the locked file. It’s very controlled, but may result in deadlocks if multiple people want to edit the same file. The second wannabe locker has to wait for the other’s checkin, merge the new revision into his Working File, lock and check in. The problem can be mitigated with good software-design based on nicely separated modules where people seldom queue up for a single file.

Git, on the other hand takes all this in its stride, considering multiple people working on the same file at the same time as a fact of life. The key is easy branching and merging. I’m not practiced but is seems the key operations are:

~/foo$ git pull # update you cloned repo with newest commits
~/foo$ git checkout -b newfoonction # create branch for development
~/foo$ echo 'echo newfoonction' >> foo
~/foo$ git commit -a
~/foo$ git push # publish your branch to original repo

Then it’s up to the maintainer of the main repo to merge the new branch to master. More of that later when I get some practice.

3 Code Reuse

MKS enables code reuse with the Shared Subproject concept. Project “foo.pj” that resides on the central server can be inserted into Project “bar.pj” into an arbitrary subfolder under an arbitrary name. You can also specify to reuse a certain Checkpoint of Project “foo.pj” (Build), or develop on it’s mainstream (Normal) or DevelopmentPath (Variant). In the latter two cases you’re able to Lock, Check In files and Checkpoint your Shared Subproject foo.pj from within the reusing bar.pj. You can also change between a fixed checkpoint (Build), mainstream (Normal) or DevPath (Variant) development, using the Configure Subproject menu item.

Git does this with submodules. Development from within a main project is easier, as remote clones of your main repo always see a specific commit of your submodule, never the index (like a Normal or Variant Subproject in MKS). But simple operations like updating your submodule to different versions or pulling from a repo containing submodules is tricky.

Below, I’ll list a few MKS Subproject related operations and give the closest git equivalent of them.

3.1 Add Shared Subproject

This MKS operation inserts another Project into a reusing Project’s folder structure.

~/bar$ git submodule add ~/foo foo
~/bar$ git submodule update --init

3.2 Resync Sandbox Recursively

Let’s assume bar is a cloned repo, containing submodules.

~/bar$ git pull
~/bar$ git submodule update

3.3 Configure Subproject to latest Build

~/bar$ cd foo
~/bar/foo$ git checkout HEAD
~/bar/foo$ cd ..
~/bar$ git commit -a

3.4 Configure Subproject to ANY Build

~/bar$ cd foo
~/bar/foo$ git checkout HEAD^^ # 2 versions before newest
~/bar/foo$ cd ..
~/bar$ git commit -a

3.5 Develop Subproject from within Main Project

Coming soon…

3.6 Drop Subproject

Coming soon…

4 Integration with Task-Tracking

MKS contains a full bug/task tracking tool called Integrity Manager, fully integrated with its version control called Source Integrity. The task/bug unit is called “issue” and its linked with version control operations via “Change Packages”.

As far as git is concerned, task-tracking is not built in, but you can find stunning web based solutions in free software projects. Midnight Commander’s Trac-git combo springs to the lips.

I always wanted to try out something like that. My bro mentioned they use Redmine at the office. I gave it a try, and it integrates with git so easily and beautifully, that I did not want to believe my own eyes. It’s become a semi-official tool at our place as well.

One needs sort of a central git-repo on the server where the Redmine web-app runs. This repo should be the one where everyone pushes. Redmine offers issue-tracking, which is slightly better than that of MKS, and to connect a git-commit to an issue, one has to mention the issue number in the first line of the commit message. Yes, it’s that simple:

1cd87ac Fix foo to avoid a bar-baz interference #42

The above line is from a oneline git-log, showing part of the SHA1 id of the commit and the commit-message. When such a commit is pushed to the central repo, it automatically connects it to issue 42, which in turn lists all related commits. No hassle with check-ins and Change Packages. Just commit and push.

Not to mention Redmine’s excellent web-based repository browsing.

The scariest free open-source tool of all – Git – proved to be one of the best. Again.

It is scary. I grew up on MKS, with a nice GUI, a central repository and files locked for editing. With git all that is blown out of the window. Instead, you get cryptic commands (git bisect bad, anyone?), a separate repo for everybody, branches and merges.

After the initial shock, which may have lasted a few months I must admit, I begin to see MKS Source Integrity as a broken tool. Nice GUI? It crashes after a few hours, anyway. Central repo? Why can’t I work off-line? And why shall I wait ages until someone breaks his lock for me? And it takes only one “git commit -a” to realise that those Check-ins, Update-Members and Checkpoints may be a bit over the top.

My most recent discovery in Git was submodules. It’s for reusing code. I realised that I had implemented the -h/-V switches (for help and version display) in multiple projects separately. That’s a cardinal sin, while constructive laziness is the cardinal virtue.

So I created a project called arg1 with a single module with a single function, which takes your program’s argv[1] as an argument, checks whether it’s -h or -V and prints the usage or version text respectively if it is, and exits.

void arg1Eval(const char* arg1);

The texts shall be stored as simple text-files (usage.txt and version.txt) in the reusing superproject’s up-dir. The arg1 project contains a makefile as well, which turns these txt files into .h files with a string initializer-list. It’s made by a tricky sed command:

%.h: ../%.txt
 @sed -e 's/[ \t]*$$//g' -e 's/^/"/g' -e 's/$$/\\n",/g' <$< >$@

To reuse this little thing in your project simply type

git submodule add ~/Projects/arg1 arg1

Now comes the best part: Instead of reusing something from your filesystem (~/Projects/arg1) you can use any URL. You can reuse the whole internet. You can even develop that submodule from within your main project.

My new MKS vs Git page explains a lot of submodule related functionality.