See the new Corporate Git pages!

It’s about setting up git repos in Windows XP boxes, shared among each other via Cygwin/OpenSSH.

---

previous – home – next

---

We’ve arrived to the most exciting part: You may not be aware, but by now the “bar.baz” machine is a minor git-hub!

You’re still Master Foo, you already have an exciting project stored on your machine in git in “~/eggs/” (you’re a Cygwin user, obviously). The eggs are becoming popular within the corporation and you want to involve more developers. Here is how it goes:

MasterFoo@foo ~$ ssh git@bar.baz
Last login: ...
git@bar ~$ mkdir eggs.git && cd eggs.git
git@bar ~/eggs$ git --bare init
Initialized empty Git repository in /home/git/eggs/
git@bar ~/eggs$ exit
logout
connection to bar closed.
MasterFoo@foo ~$ cd eggs
MasterFoo@foo ~/eggs$ git remote add origin ssh://git@bar.baz/~/eggs.git
MasterFoo@foo ~/eggs$ git push origin master

A bare repo has no working tree. So the magic stuff is not in “~/eggs/.git/” but directly in “~/eggs.git/”. It’s a convention that bare repositories’ names should end with the “.git” extension.

Older versions of git allowed pushing into a non-bare repo. But push just updates the history and the HEAD, leaving a) the index and the working files intact b) the local repo-user in a severe shock. So it’s no more allowed from git 1.7.0. Bare repos are just a push-pull hub.

Now even the local MCSE can collaborate with you if he uses Cygwin and gives his public key to the owner of “bar.baz”.

MCSE@mcse ~$ git clone ssh://git@bar.baz/~/eggs.git
Initialized empty Git repo...
...
MCSE@mcse ~$ cd eggs
MCSE@mcse ~/eggs$ echo '# Even MCSEs love Unix' >> eggs
MCSE@mcse ~/eggs$ git commit -a -m "MCSEs love Unix"
MCSE@mcse ~/eggs$ git push origin master

Now you probably see how, with a bit more sophisticated handling of users, you could build your own Guerrilla GitHub. All you need is Cygwin. Have fun!

---

previous – home – next

---

Generating Keys for the ssh Client

Let’s say you’re Master Foo, and want to connect as user “git” to the machine described in the previous chapters, which we’ll call “bar.baz”.

You need a private/public key pair. You can give your public key to the whole world. They can encrypt stuff with it for you which, in turn, can be decoded only with your private key. So guard your private key with your life and, preferably, with a passphrase.

$ ssh-keygen

Accept the default location for the private key: ~/.ssh/id_rsa
When prompted for the passphrase, enter it twice. The passphrase actually encrypts your private key, so even if someone steals the file, he cannot use it.

When finished, your keys are saved in

~/.ssh/id_rsa
~/.ssh/id_rsa.pub

Logging in

Give your public key “id_rsa.pub” to the owner of the “bar.baz” box. If he trusts you, he’ll append it to “/home/git/.ssh/authorized_keys”. Once that done you can

$ ssh git@bar.baz

and after entering your private key’s passphrase, you’re logged in, and you can do whatever you want or, rather, whatever you’re allowed to. Done!

Using ssh-agent

Nearly. After a while, you’ll find find it cumbersome having to enter your passphrase every time you log in to “bar.baz”. Or every time you push or pull in git terms. But fear not, only believe!

The wise elders of OpenBSD have developed the ssh-agent to avoid this. You run your Cygwin session under its protection, and you’ll have to enter your passphrase just once at the start of your first bash after power-on. Add this to the system-wide /etc/profile

export SSH_AUTH_SOCK=/tmp/.ssh-socket

and the code below to your profile “~/.profile”. How it works is explained on Ovidiu Predescu’s excellent page. I’ve applied a minor fix. Find it!

# Start or connect to the ssh-agent
ssh-add -l >/dev/null 2>&1
if [ $? = 2 ]; then
  ssh-agent -a $SSH_AUTH_SOCK 2>/dev/null >/tmp/.ssh-script
  . /tmp/.ssh-script >/dev/null
  echo $SSH_AGENT_PID >/tmp/.ssh-agent-pid
  echo
  ssh-add
fi
# Command to stop the ssh-agent
function killssh {
  kill `cat /tmp/.ssh-agent-pid`
  rm -f /tmp/.ssh*
}

One more small gotcha: the outdated .ssh* files should be removed from /tmp during or after reboot, before starting the first Cygwin session. Preferably automatically.

---

previous – home – next

---

Start the Secure Shell Daemon (sshd) service on Windows

This will allow users on remote computers to log in to your box, get an interactive shell and do whatever they want. Or rather whatever you allow them to do. Remember? User “git” is not an admin…

There is a nice guide with a lot of troubleshooting info here.

In a nutshell. Start a Cygwin Shell window.

$ ssh-host-config -y

When prompted for “CYGWIN=”, type “tty ntsec”.

I do not allow login by password, only the public-key method. See below. So change the related line in “/etc/sshd_config” like this:

# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication no

Finally start the service:

cygrunsrv --start sshd

Administering users with public-key authentication

Master Foo wants to log in remotely to your box’s “git” account.
He sends you his public RSA key in any way, email, pen-drive, whatever YOU trust.
If you find Master Foo worthy, you authorize his key, and from then he will be able to log in to your “git” account via ssh without typing git’s password:

$ cat MasterFoo_rsa.pub >> /home/git/.ssh/authorized_keys

---

previous – home – next

---

Install cygwin. See cygwin.com. Follow the installation instructions. I recommend the following options:

• Install from Internet
• All Users
• Use IE5 Settings
• Select a mirror near you (if you can find out where they are!)
• Unix style linefeeds (LF)

Make sure to Select Packages below (among others) besides the Base packages:

• git (git-gui, gitk)
• mintty
• cygrunsrv
• openssh

After installation let’s sort out the home folders. Cygwin 1.7.x uses it’s own user profiles in /home (C:\WhereeverYouInstalledCygwin\home). I don’t like that. I prefer Cygwin to use the original Windows user profiles. I sorted this by mounting “D:\Documents and Settings” to “/home”. Just a simple entry to “/etc/fstab”:

D:/Documents\040and\040Settings /home ntfs binary 0 0

Now let’s get a proper semi-transparent terminal window. The icon added by the Cygwin installer starts “C:\WhereeverYouInstalledCygwin\Cygwin.bat”. This opens an interactive login shell (bash) in an ugly Windows Command prompt. To fix this, right-click the “Cygwin Bash Shell” icon, Properties. Note the minus sign at the end!

Target:    C:\WhereeverYouInstalledCygwin\bin\mintty.exe -
Start in:  C:\WhereeverYouInstalledCygwin\bin

Last but not least, let’s set up a cosy and comfortable home environment.

$ cd ~
$ echo "export EDITOR=mcedit" > .profile  # for everybody else, it's vi
$ echo 'export HISTCONTROL="ignoredups"' >> .profile

If you’ve already installed Cygwin before creating user “git” (see previous chapter), you can export the Windows user list to Cygwin.

$ cp /etc/passwd /etc/passwd.old    # backup for safety
$ mkpasswd > /etc/passwd

---

previous – home – next

---

You need to have administrator privileges on your Windows XP. If you do not, contact your IT department. If they deny it from you, look for another job immediately.

Create a user to represent the git service on your box. Right-click “My Computer”, Manage, Local Users and Groups, Users. Right-click the right pane, New User…

Let’s call it “git”. Password never expires, user does not have to (or cannot) change it. Password does not really matter, we’ll see later why.

Member of “Users”. Not an admin. We won’t let other people screw up our box completely through ssh.

It’s user profile and home folder should be the standard “D:\Documents and Settings\git” or similar (I store the user profiles on “D:\”).

Contents

Intro
Credits
Prepare Windows
Set up Cygwin
Set up the OpenSSH server – sshd
Set up the OpenSSH client – ssh and ssh-agent
Git over ssh
The big picture
Politics
git Cheat Sheet
MKS vs git

Intro

In the original spirit of the subogero linux blog, this page is aimed to be a comprehensive guide for relative Newbies to set up git servers in a (corporate) network on Windows XP boxes. After reading this, you will be pushing and pulling among each other at a rate that will baffle both your managers and your IT department.

Large corporations tend to force developers to use a) Windows, b) a centralized version control system. Life under these circumstances is miserable. To make it bearable, one needs a Unix and a cool distributed version control tool. Fortunately you can turn your Windows machine into a Unix by using Cygwin (GNU/Windows), which comes with git, the Daddy of Distribution from His Majesty Linus Torvalds himself.

Listening to the Grand Master’s speech, pushing and pulling between each other seems to be the most trivial thing on earth. Once I tried this with a slightly remote server (pun intended) using the Samba protocol. It was incredibly slow. Git only shines on the local machine. With the SMB protocol, it’s the local git that accesses the remote server ten thousand times. That’s why git’s preferred protocol is ssh: behind the scenes, you run git on the remote box as well, and the two old gits reduce network traffic to the necessary minimum.

That’s what these pages are all about: setting up ssh and git on Windows XP.

Credits

Richard Stallman – the creator of GNU
Cygwin – the GNU/Windows people
Linus Torvalds – the creator of Git
OpenBSD – the creators of OpenSSH
Nicholas Fong – a compehensive Cygwin and OpenSSH setup page
Ovidiu Predescu – ssh-agent setup on Cygwin
Tim Lucas – how to setup a new remote git repo

This page should serve as a summary of my experience with these two version control systems and a comparison of their possible workflows. There are four main areas where I shall compare them:

• Basics
• Development in a team
• Code reuse
• Integration with task-tracking

I have a lot of experience with MKS in all areas but, as I’ve been working with git on my private hobby projects so far, I’m very much limited to Basics and Code reuse here. Actually my goal with this page is to help me remember those awkward git commands.

1 Basics

1.1 Central vs Distributed Repositories

MKS is a client-server program. There is a central server that stores the repositories of each Project. People work with Sandboxes, private copies of a Project, that allow status tracking and all sorts of operations on the Project as well. A Sandbox is represented by a foo.pj file and the Working Files.
The server is usually under the control of a very busy and important admin, so don’t even ask me how to create a project. Just write him an email and in a matter of days you’ll have your new project. Another few hours/days will be needed to sort out access rights. To create a Sandbox, select the Sandbox/Create… menu item in the GUI. Or use the command line:

~$ mkdir foo
~$ si createsandbox --project=foo.pj foo # sandbox file foo/foo.pj is created

In git, you always have your own repository, represented  by the .git directory in the root of your project. To create a repo and a copy of the Working Tree, simply type

~$ mkdir foo && cd foo
~/foo$ git init

Or you can copy an existing repo anywhere (any URL is possible), similar to a Sandbox (but it’s more than that):

~$ git clone //bar/Projects/foo
~$ cd foo

The client-server concept of MKS seems superior at first sight, but I’m not sure. If the server’s backup strategy is not just-so, disaster is threatening, while in git every developer has a full copy of the entire repo. With MKS there is absolutely no way to track your work offline. Nor can you do so if you’ve received Read-only access only. Client server architectures are also very slow, as most operations go through the network. So on second thoughts, I’m absolutely sure: distributed is far superior.

1.2 Simple Workflow

I’ll cover adding and updating a file to the project. In both cases it starts with creating the file.

In MKS you can use the GUI: Member/Add, Member/Lock, Member/Edit Working File, Member/Check in, Project/Checkpoint. Or using the command line:

~/foo$ echo '#!bin/sh' > foo
~/foo$ si add foo
~/foo$ si lock foo
~/foo$ echo 'echo foo' >> foo
~/foo$ si ci --update foo
~/foo$ si checkpoint --sandbox=foo.pj

In git it’s far simpler:

~/foo$ echo '#!bin/sh' > foo
~/foo$ git add foo
~/foo$ echo 'echo foo' >> foo
~/foo$ git commit -a

1.3 Files vs Content

The reason for git’s simplicity is that it’s tracking content, not files separately. A “commit -a” will check in everything you’ve changed in your Working Tree as one atomic operation. Hence the lack of the checkpoint operation as well. An explicit “add” is only necessary for new files.

Note: If you use commit without -a, you need “git add” for each changed file as well before commit.

2 Development in a Team

MKS handles co-operation with a repo on the central server, private sandboxes and file Locks. A Lock means that only the locking developer is allowed to Check In the locked file. It’s very controlled, but may result in deadlocks if multiple people want to edit the same file. The second wannabe locker has to wait for the other’s checkin, merge the new revision into his Working File, lock and check in. The problem can be mitigated with good software-design based on nicely separated modules where people seldom queue up for a single file.

Git, on the other hand takes all this in its stride, considering multiple people working on the same file at the same time as a fact of life. The key is easy branching and merging. I’m not practiced but is seems the key operations are:

~/foo$ git pull # update you cloned repo with newest commits
~/foo$ git checkout -b newfoonction # create branch for development
~/foo$ echo 'echo newfoonction' >> foo
~/foo$ git commit -a
~/foo$ git push # publish your branch to original repo

Then it’s up to the maintainer of the main repo to merge the new branch to master. More of that later when I get some practice.

3 Code Reuse

MKS enables code reuse with the Shared Subproject concept. Project “foo.pj” that resides on the central server can be inserted into Project “bar.pj” into an arbitrary subfolder under an arbitrary name. You can also specify to reuse a certain Checkpoint of Project “foo.pj” (Build), or develop on it’s mainstream (Normal) or DevelopmentPath (Variant). In the latter two cases you’re able to Lock, Check In files and Checkpoint your Shared Subproject foo.pj from within the reusing bar.pj. You can also change between a fixed checkpoint (Build), mainstream (Normal) or DevPath (Variant) development, using the Configure Subproject menu item.

Git does this with submodules. Development from within a main project is easier, as remote clones of your main repo always see a specific commit of your submodule, never the index (like a Normal or Variant Subproject in MKS). But simple operations like updating your submodule to different versions or pulling from a repo containing submodules is tricky.

Below, I’ll list a few MKS Subproject related operations and give the closest git equivalent of them.

3.1 Add Shared Subproject

This MKS operation inserts another Project into a reusing Project’s folder structure.

~/bar$ git submodule add ~/foo foo
~/bar$ git submodule update --init

3.2 Resync Sandbox Recursively

Let’s assume bar is a cloned repo, containing submodules.

~/bar$ git pull
~/bar$ git submodule update

3.3 Configure Subproject to latest Build

~/bar$ cd foo
~/bar/foo$ git checkout HEAD
~/bar/foo$ cd ..
~/bar$ git commit -a

3.4 Configure Subproject to ANY Build

~/bar$ cd foo
~/bar/foo$ git checkout HEAD^^ # 2 versions before newest
~/bar/foo$ cd ..
~/bar$ git commit -a

3.5 Develop Subproject from within Main Project

Coming soon…

3.6 Drop Subproject

Coming soon…

4 Integration with Task-Tracking

MKS contains a full bug/task tracking tool called Integrity Manager, fully integrated with its version control called Source Integrity. The task/bug unit is called “issue” and its linked with version control operations via “Change Packages”.

As far as git is concerned, task-tracking is not built in, but you can find stunning web based solutions in free software projects. Midnight Commander’s Trac-git combo springs to the lips.

I always wanted to try out something like that. My bro mentioned they use Redmine at the office. I gave it a try, and it integrates with git so easily and beautifully, that I did not want to believe my own eyes. It’s become a semi-official tool at our place as well.

One needs sort of a central git-repo on the server where the Redmine web-app runs. This repo should be the one where everyone pushes. Redmine offers issue-tracking, which is slightly better than that of MKS, and to connect a git-commit to an issue, one has to mention the issue number in the first line of the commit message. Yes, it’s that simple:

1cd87ac Fix foo to avoid a bar-baz interference #42

The above line is from a oneline git-log, showing part of the SHA1 id of the commit and the commit-message. When such a commit is pushed to the central repo, it automatically connects it to issue 42, which in turn lists all related commits. No hassle with check-ins and Change Packages. Just commit and push.

Not to mention Redmine’s excellent web-based repository browsing.

The scariest free open-source tool of all – Git – proved to be one of the best. Again.

It is scary. I grew up on MKS, with a nice GUI, a central repository and files locked for editing. With git all that is blown out of the window. Instead, you get cryptic commands (git bisect bad, anyone?), a separate repo for everybody, branches and merges.

After the initial shock, which may have lasted a few months I must admit, I begin to see MKS Source Integrity as a broken tool. Nice GUI? It crashes after a few hours, anyway. Central repo? Why can’t I work off-line? And why shall I wait ages until someone breaks his lock for me? And it takes only one “git commit -a” to realise that those Check-ins, Update-Members and Checkpoints may be a bit over the top.

My most recent discovery in Git was submodules. It’s for reusing code. I realised that I had implemented the -h/-V switches (for help and version display) in multiple projects separately. That’s a cardinal sin, while constructive laziness is the cardinal virtue.

So I created a project called arg1 with a single module with a single function, which takes your program’s argv[1] as an argument, checks whether it’s -h or -V and prints the usage or version text respectively if it is, and exits.

void arg1Eval(const char* arg1);

The texts shall be stored as simple text-files (usage.txt and version.txt) in the reusing superproject’s up-dir. The arg1 project contains a makefile as well, which turns these txt files into .h files with a string initializer-list. It’s made by a tricky sed command:

%.h: ../%.txt
 @sed -e 's/[ \t]*$$//g' -e 's/^/"/g' -e 's/$$/\\n",/g' <$< >$@

To reuse this little thing in your project simply type

git submodule add ~/Projects/arg1 arg1

Now comes the best part: Instead of reusing something from your filesystem (~/Projects/arg1) you can use any URL. You can reuse the whole internet. You can even develop that submodule from within your main project.

My new MKS vs Git page explains a lot of submodule related functionality.

After switching to Linux, one feels the need to switch to an equally cool version control system as well. And none is cooler than the one invented by Linus Torvalds himself: Git.

It’s called the Fast Version Control System. Being an MKS user professionally, git was a bit on the scary side. No GUI, strange commands and a completely different workflow. Instead of the central repo, there is cloning, pulling and pushing. Instead of the rigid but readable version numbering, there are unreadable SHA1 hashes but, at the same time, user-defined tags as well. Instead of locking, there is branching and merging. Instead of dealing with separate files, there is the universal commit. Main advantage: FAST! And I feel the workflow more and more natural.

I could not get used to one thing, however. The “git diff” command displaying source differences. I’m a huge fan of command lines, but there are limits. So I googled for a visual difference tool. I found meld.

sudo apt-get install meld

Then I looked for a way to integrate git and meld, so when I type git diff, it would open meld, instead of displaying that unreadable mess on the terminal. I found Nathan Hoad’s website.

First step is to tell git it should use meld. While the Bash Guru types

git config --global diff.external meld

the Config Wizard adds a few lines to ~/.gitconfig:

[diff]
  external = /usr/bin/meld

It does not work, however. Git sends 5 parameters to the diff tool, while meld expects only the 2 filenames to compare. So a little filter is needed. Nathan Hoad proposes a Python script:

#!/usr/bin/python
import sys
import os
os.system('meld "%s" "%s"' % (sys.argv[2], sys.argv[5]))

I know that Python is the language of the day, but in this case I felt it was a bit like Shooting a Sparrow with a Cannon, if you allow me this Hungarism. If one is lazy, one needs a one-liner. And the language of choice is non other than the Bourne Again Shell. The filter is called ~/gitvdiff:

git config --global diff.external ~/gitvdiff

And it contents:

#!/bin/sh
meld $2 $5

Let’s see the results.

On the right side you may notice the entire source file displayed. With syntax highlighting. With a map of the diffs. And with the diffs nicely highlighted.