subogero linux blog

It happened that I had to put together a few html pages containing important info about a certain project at the company. Originally I stored them on a Samba share on the official Windows file-server. I turned out to be terrible. For instance Firefox needs FIVE slashes after the protocol-id in the URL:

file://///server/share/page.html

IE needs two, Chrome needs four. First step in the downward spiral: I need the http protocol, in other words a webserver. So I installed Cygwin’s lighttpd on our constantly running desktop box. Then moved the html files to “/srv/www/htdocs/site/”. First step done.

Next step: it’s very cumbersome to change the navigation links, whenever a new page is added. Time to automatize. Tools: SSI (server side includes) and CGI scripts in Perl.

Then, what about setting up a few mailing lists. No archiving, just free subscribing and sending mail via “mailto:” links. One day job in Perl.

Later SSI turned out to be a bad choice as you need an absolute path to your CGI script. Difficult if you want a test-site. So I rewrote the entire site that all links actually refer to the same CGI script which loads the required page, adding dynamic navigation.

The trickiest and nicest part is managing the website with git. On the server I set up a git repo in my home folder, with a detached worktree in “/srv/www/htdocs/site”:

server ~$ mkdir site.git && cd site.git
server ~/site.git$ git --bare init
server ~/site.git$ git config core.worktree '/srv/www/htdocs/site'
server ~/site.git$ git config receive.denycurrentbranch false

Additionally, we need a hook routine which checks out anything to the website whenever you push it into this repo:

server ~/site.git$ mv hooks/post-receive.sample hooks/post-receive
server ~/site.git$ mcedit hooks/post-receive
# checkout the received branch to the website
read OLD NEW REF
git checkout -f $RE

On my own machine, I also have a git repo to manage the site. To allow easy server-updates, I did this:

mybox ~/site$ git remote add web ssh://server/~/site.git
mybox ~/site$ git push web master

Finally, one wants a test-site. Lighty endeavours to give satisfaction, so simply enable module mod_userdir in /etc/lighttpd/lighttpd.conf, and each user’s public_html folder becomes his own website. The two lines below now refer to the same folder:

/home/foo/public_html/
http://server/~foo/

Let’s create a non-bare repo here and configure it to receive and automatically check out pushed commits:

server ~/public_html$ mkdir site && cd site
server ~/public_html/site$ git init
server ~/public_html/site$ git config receive.denycurrentbranch false

I applied the same post-receive hook as with the official website’s repo. But it refused to check out the pushed commits. After a considerable amount of cursing it turns out the bloody git runs the hooks in the “.git” folder. I discovered this when I found all my html files there. So the post-receive hook should look like this in a non bare repo:

#!/bin/sh
unset GIT_DIR
unset GIT_WORK_TREE
cd ..
read OLD NEW REF
REF=`echo $REF | sed 's:refs/..*/::'` # branch instead of detached head
git checkout -f $REF

On my box I added an other remote to allow pushing to the test site:

mybox ~/site$ git remote add test ssh://server/~/public_html/site
mybox ~/site$ git push test master

I even applied a post-commit hook on the local machine which pushes master-commits to the official site and branch-commits to the test site.

While setting up the ssh-agent on Cygwin, I run into a small problem. After reboot, the ssh-agent is not running yet, but the /tmp/.ssh* files are still there from the previous session. When starting a Cygwin shell, the start of the ssh-agent fails, if these files have not been manually removed before.

Task: delete these files automatically at boot time. I googled it, and the Windows way of doing it seemed extremely complicated. As usual, as it springs to the lips of smug Linux geeks. Not me. Then I ran across the “cron” scheduling daemon somehow, and there it was: insert the line below into the “/etc/crontab” file.

@reboot SYSTEM rm -f /tmp/.ssh*

The rest of this post is a distilled version of my adventures with setting up cron on Cygwin 1.7.5.

cron is the Unix-equivalent of “Scheduled Tasks” in Windows, just better. For instance, you can schedule something to run at boot time (I may have mentioned that before) and, the scheduled commands don’t throw up a scary cmd-window on your desktop. So in the meantime I’ve moved all my scheduled tasks to cron.

Start the cygwin-setup and add the “cron” and “cygrunsrv” packages from the “Admin” category.

We’ll run cron as a service by user SYSTEM. Poor SYSTEM therefore needs a home directory and a shell. The “/etc/passwd” file will define them.

$ mkdir /root
$ chown SYSTEM:root /root
$ mcedit /etc/passwd
SYSTEM:*:......:/root:/bin/bash

The start the service:

$ cron-config
Do you want to remove or reinstall it (yes/no) yes
Do you want to install the cron daemon as a service? (yes/no) yes
Enter the value of CYGWIN for the daemon: [ ] ntsec
Do you want the cron daemon to run as yourself? (yes/no) no
Do you want to start the cron daemon as a service now? (yes/no) yes

Local users can now define their scheduled tasks like this (crontab will start your favourite editor):

$ crontab -e  # edit your user specific cron-table
HOME=/home/foo
PATH=/usr/local/bin:/usr/bin:/bin:$PATH
# testing
* * * * *   touch ~/cron
@reboot     ~/foo.sh
45 11 * * * ~/lunch_message_to_mates.sh

Domain users: it does not work. Poor cron is unable to run scheduled tasks on behalf of domain users on the machine. But there is another way: cron also runs stuff found in the system level cron table in “/etc/crontab”. So insert your suff there, so that SYSTEM does it on its own behalf:

$ touch /etc/crontab
$ chown SYSTEM /etc/crontab
$ mcedit /etc/crontab
HOME=/root
PATH=/usr/local/bin:/usr/bin:/bin:$PATH
* * * * *   SYSTEM touch ~/cron
@reboot     SYSTEM rm -f /tmp/.ssh*

Finally a few words about crontab entries. They are either environment settings or scheduled commands. As seen above, on Cygwin it’s best to  create a usable PATH. Home dir and shell are normally taken from “/etc/passwd”.

As to the columns of scheduled commands see the manual page.

If certain crontab entries do not run, the best diagnostic tool is this:

$ cronevents

It will print out details on all successful and failed commands.

Erm… ogc 4.0 had some certain erm… bugs in its syntax.

But ogc 4.1 is now downloadable from the ogc page. Including native Linux and Windows binaries.

See the new Corporate Git pages!

It’s about setting up git repos in Windows XP boxes, shared among each other via Cygwin/OpenSSH.

---

previous – home – next

---

We’ve arrived to the most exciting part: You may not be aware, but by now the “bar.baz” machine is a minor git-hub!

You’re still Master Foo, you already have an exciting project stored on your machine in git in “~/eggs/” (you’re a Cygwin user, obviously). The eggs are becoming popular within the corporation and you want to involve more developers. Here is how it goes:

MasterFoo@foo ~$ ssh git@bar.baz
Last login: ...
git@bar ~$ mkdir eggs.git && cd eggs.git
git@bar ~/eggs$ git --bare init
Initialized empty Git repository in /home/git/eggs/
git@bar ~/eggs$ exit
logout
connection to bar closed.
MasterFoo@foo ~$ cd eggs
MasterFoo@foo ~/eggs$ git remote add origin ssh://git@bar.baz/~/eggs.git
MasterFoo@foo ~/eggs$ git push origin master

A bare repo has no working tree. So the magic stuff is not in “~/eggs/.git/” but directly in “~/eggs.git/”. It’s a convention that bare repositories’ names should end with the “.git” extension.

Older versions of git allowed pushing into a non-bare repo. But push just updates the history and the HEAD, leaving a) the index and the working files intact b) the local repo-user in a severe shock. So it’s no more allowed from git 1.7.0. Bare repos are just a push-pull hub.

Now even the local MCSE can collaborate with you if he uses Cygwin and gives his public key to the owner of “bar.baz”.

MCSE@mcse ~$ git clone ssh://git@bar.baz/~/eggs.git
Initialized empty Git repo...
...
MCSE@mcse ~$ cd eggs
MCSE@mcse ~/eggs$ echo '# Even MCSEs love Unix' >> eggs
MCSE@mcse ~/eggs$ git commit -a -m "MCSEs love Unix"
MCSE@mcse ~/eggs$ git push origin master

Now you probably see how, with a bit more sophisticated handling of users, you could build your own Guerrilla GitHub. All you need is Cygwin. Have fun!

---

previous – home – next

---

Generating Keys for the ssh Client

Let’s say you’re Master Foo, and want to connect as user “git” to the machine described in the previous chapters, which we’ll call “bar.baz”.

You need a private/public key pair. You can give your public key to the whole world. They can encrypt stuff with it for you which, in turn, can be decoded only with your private key. So guard your private key with your life and, preferably, with a passphrase.

$ ssh-keygen

Accept the default location for the private key: ~/.ssh/id_rsa
When prompted for the passphrase, enter it twice. The passphrase actually encrypts your private key, so even if someone steals the file, he cannot use it.

When finished, your keys are saved in

~/.ssh/id_rsa
~/.ssh/id_rsa.pub

Logging in

Give your public key “id_rsa.pub” to the owner of the “bar.baz” box. If he trusts you, he’ll append it to “/home/git/.ssh/authorized_keys”. Once that done you can

$ ssh git@bar.baz

and after entering your private key’s passphrase, you’re logged in, and you can do whatever you want or, rather, whatever you’re allowed to. Done!

Using ssh-agent

Nearly. After a while, you’ll find find it cumbersome having to enter your passphrase every time you log in to “bar.baz”. Or every time you push or pull in git terms. But fear not, only believe!

The wise elders of OpenBSD have developed the ssh-agent to avoid this. You run your Cygwin session under its protection, and you’ll have to enter your passphrase just once at the start of your first bash after power-on. Add this to the system-wide /etc/profile

export SSH_AUTH_SOCK=/tmp/.ssh-socket

and the code below to your profile “~/.profile”. How it works is explained on Ovidiu Predescu’s excellent page. I’ve applied a minor fix. Find it!

# Start or connect to the ssh-agent
ssh-add -l >/dev/null 2>&1
if [ $? = 2 ]; then
  ssh-agent -a $SSH_AUTH_SOCK 2>/dev/null >/tmp/.ssh-script
  . /tmp/.ssh-script >/dev/null
  echo $SSH_AGENT_PID >/tmp/.ssh-agent-pid
  echo
  ssh-add
fi
# Command to stop the ssh-agent
function killssh {
  kill `cat /tmp/.ssh-agent-pid`
  rm -f /tmp/.ssh*
}

One more small gotcha: the outdated .ssh* files should be removed from /tmp during or after reboot, before starting the first Cygwin session. Preferably automatically.

---

previous – home – next

---

Start the Secure Shell Daemon (sshd) service on Windows

This will allow users on remote computers to log in to your box, get an interactive shell and do whatever they want. Or rather whatever you allow them to do. Remember? User “git” is not an admin…

There is a nice guide with a lot of troubleshooting info here.

In a nutshell. Start a Cygwin Shell window.

$ ssh-host-config -y

When prompted for “CYGWIN=”, type “tty ntsec”.

I do not allow login by password, only the public-key method. See below. So change the related line in “/etc/sshd_config” like this:

# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication no

Finally start the service:

cygrunsrv --start sshd

Administering users with public-key authentication

Master Foo wants to log in remotely to your box’s “git” account.
He sends you his public RSA key in any way, email, pen-drive, whatever YOU trust.
If you find Master Foo worthy, you authorize his key, and from then he will be able to log in to your “git” account via ssh without typing git’s password:

$ cat MasterFoo_rsa.pub >> /home/git/.ssh/authorized_keys

---

previous – home – next

---

Install cygwin. See cygwin.com. Follow the installation instructions. I recommend the following options:

• Install from Internet
• All Users
• Use IE5 Settings
• Select a mirror near you (if you can find out where they are!)
• Unix style linefeeds (LF)

Make sure to Select Packages below (among others) besides the Base packages:

• git (git-gui, gitk)
• mintty
• cygrunsrv
• openssh

After installation let’s sort out the home folders. Cygwin 1.7.x uses it’s own user profiles in /home (C:\WhereeverYouInstalledCygwin\home). I don’t like that. I prefer Cygwin to use the original Windows user profiles. I sorted this by mounting “D:\Documents and Settings” to “/home”. Just a simple entry to “/etc/fstab”:

D:/Documents\040and\040Settings /home ntfs binary 0 0

Now let’s get a proper semi-transparent terminal window. The icon added by the Cygwin installer starts “C:\WhereeverYouInstalledCygwin\Cygwin.bat”. This opens an interactive login shell (bash) in an ugly Windows Command prompt. To fix this, right-click the “Cygwin Bash Shell” icon, Properties. Note the minus sign at the end!

Target:    C:\WhereeverYouInstalledCygwin\bin\mintty.exe -
Start in:  C:\WhereeverYouInstalledCygwin\bin

Last but not least, let’s set up a cosy and comfortable home environment.

$ cd ~
$ echo "export EDITOR=mcedit" > .profile  # for everybody else, it's vi
$ echo 'export HISTCONTROL="ignoredups"' >> .profile

If you’ve already installed Cygwin before creating user “git” (see previous chapter), you can export the Windows user list to Cygwin.

$ cp /etc/passwd /etc/passwd.old    # backup for safety
$ mkpasswd > /etc/passwd

---

previous – home – next

---

You need to have administrator privileges on your Windows XP. If you do not, contact your IT department. If they deny it from you, look for another job immediately.

Create a user to represent the git service on your box. Right-click “My Computer”, Manage, Local Users and Groups, Users. Right-click the right pane, New User…

Let’s call it “git”. Password never expires, user does not have to (or cannot) change it. Password does not really matter, we’ll see later why.

Member of “Users”. Not an admin. We won’t let other people screw up our box completely through ssh.

It’s user profile and home folder should be the standard “D:\Documents and Settings\git” or similar (I store the user profiles on “D:\”).

Contents

Intro
Credits
Prepare Windows
Set up Cygwin
Set up the OpenSSH server – sshd
Set up the OpenSSH client – ssh and ssh-agent
Git over ssh
The big picture
Politics
git Cheat Sheet
MKS vs git

Intro

In the original spirit of the subogero linux blog, this page is aimed to be a comprehensive guide for relative Newbies to set up git servers in a (corporate) network on Windows XP boxes. After reading this, you will be pushing and pulling among each other at a rate that will baffle both your managers and your IT department.

Large corporations tend to force developers to use a) Windows, b) a centralized version control system. Life under these circumstances is miserable. To make it bearable, one needs a Unix and a cool distributed version control tool. Fortunately you can turn your Windows machine into a Unix by using Cygwin (GNU/Windows), which comes with git, the Daddy of Distribution from His Majesty Linus Torvalds himself.

Listening to the Grand Master’s speech, pushing and pulling between each other seems to be the most trivial thing on earth. Once I tried this with a slightly remote server (pun intended) using the Samba protocol. It was incredibly slow. Git only shines on the local machine. With the SMB protocol, it’s the local git that accesses the remote server ten thousand times. That’s why git’s preferred protocol is ssh: behind the scenes, you run git on the remote box as well, and the two old gits reduce network traffic to the necessary minimum.

That’s what these pages are all about: setting up ssh and git on Windows XP.

Credits

Richard Stallman – the creator of GNU
Cygwin – the GNU/Windows people
Linus Torvalds – the creator of Git
OpenBSD – the creators of OpenSSH
Nicholas Fong – a compehensive Cygwin and OpenSSH setup page
Ovidiu Predescu – ssh-agent setup on Cygwin
Tim Lucas – how to setup a new remote git repo