Using the Git versioning system

Everything you need to get started with Git, plus some tips for advanced users!

Introduction

Git is a distributed version control system. It was created in 2005 but has already met with great success, especially in open-source projects (Nokia's Qt library and the Linux kernel, for example).

It is currently used for most programming projects at Aldebaran.

Concepts

The main feature of Git, after great efficiency, is that anyone who has a copy of a repository also has the complete history of the repository.

A contributor (also called committer) to a git project is then able to work locally on their own git history tree, on their own, local, branches.

They can save their changes locally, and synchronize them with the main, remote, repository, branch by branch, separately. The contributor may then have to solve conflicts if the changes to the local branch cannot be automatically merged with the changes which have been made by other contributors on the remote branch.

Here are a few (very) basic concepts that you need to know when working with Git:

Tree: equivalent to a directory of files on the hard drive (similar to an Apache Subversion working copy).

Staged change: set of changes to be made by the next commit.

Commit: a coherent set of changes made to the project files. Commits always come with a name, an e-mail address and a message.

Branch: branches allow parts of software to be developed in parallel. Unlike in SVN, each developer automatically gets their own branch (we'll come back to this in a minute)

Push: applies local commits to a remote branch.

Pull: fetches any changes made to a remote branch, and attempt to merge them with the files on the local branch.

Commands

In practice, how does it work? (Command-line commands are in bold, and similar commands are generally available on graphical user interfaces). What follows is only an example of a possible workflow.

git clone: clones a repository into a newly created directory, creates remote-tracking branches for each branch in the cloned repository (visible using git branch -r), and creates and checks out an initial branch that is forked from the cloned repository’s currently active branch. By default, this is called the Master branch.

We start working, which means that there are now some differences between our local version of the files and the remote version.

git add: choose a set of changes to add to the index.

git commit: stores the current contents of the index in a new commit along with a log message from the user describing the changes. So far, we're still working locally, without any network communication. We can make several succesive commits, amend them etc.

git pull: fetch any changes that have have been made on the remote branch in the meantime. If there are any conflicts, now is the time to solve them (but for now we'll assume there aren't).

git push: once we've checked that any merges have gone successfully and that everything's still working, we can push our work to the remote branch so everyone else can use it.

Revisions

One thing that may come as quite a surprise to new Git users is that commits aren't identified by a globally incremented revision number. Instead, they look something like: ceed0089e985040aa5f7c4582cf84b40fb9e5c35. It's a kind of hash, i.e. a unique identifier for the commit.

In practice, these identifiers rarely come in useful (thank goodness!). If you need them for any reason, the first n characters - 8 is plenty most of the time - are generally enough when the situation is unambiguous.

Getting started

Installation: Linux

The main package to install is git-core. You probably want to install the git-gui and gitk packages too, which will provide you with graphic interfaces.

Installation: Windows

The recommended installation options are as follows:

Step "Adjusting your PATH environment"

 * use git bash only

Step "Choosing CR/LF behavior"

 * Checkout Windows style, commit Unix-style line endings

Then read the configuration instructions below to configure your access to Developer Program repositories.

Installation: Mac OS

A Mac OS installer is available here for Leopard and later versions: http://code.google.com/p/git-osx-installer/

If you need to use Git on the command line, then (as indicated in the README file) you need to launch the script which you'll find next to the package. A classic Git-GUI is also available on this page, under the name OpenInGitGui.

Another, better, GUI is GitX: http://gitx.frim.nl/

Don't forget to restart your session in order for the changes to take effect.

General configuration

If using a GUI, it's a good idea to go into the options menu, and set it to use your username and email address instead of the default options.

 

To use Git for Developer Program projects, you will first need to configure your SSH identity to work with Git repositories:

  • First, find your SSH public key. On Windows and Mac, you can do this directly from Git Gui, in the menu Help > Show SSH Key. Your key should be displayed, and key generation should be available if not. On Linux, if you already have a public key then you should find it in $HOME/.ssh/id_dsa.pub. If not, you can generate a pair of SSH keys using the ssh-keygen command.
  • Once you have obtained your public key, copy it to the clipboard.
  • Then go to your profile, and paste the key into the "Public SSH key" field.

Retrieving a project's source code

Go to the "Developers' corner" page of the project you want to work on, where you will find a command something like:

git clone git@src.aldebaran-robotics.com:yourproject.git

You can either type this command directly into a terminal, or use the Git Gui method described just below.

If you cannot clone a repository, and you get this kind of error:

 The remote end hung up unexpectedly
  • You might be using the wrong URL, double-check it.
  • You may have misconfigured your SSH identity, please regenerate an SSH key pair and put the new public key in your profile. If the problem is not solved, on Windows, check for your environment variable GIT_SSH. From Git Bash, type:
 env | grep GIT_SSH

If found, ensure it is not set in the Windows environment variables, or in any Git Bash startup script.

Using the Git Gui

This tutorial is valid for Windows and Linux

As you almost certainly know, a GUI is a Graphical User Interface.

Of course, there are other GUIs for Git (like TortoiseGIT on Windows - there are zillions on Linux)

Git-Gui is the most standard interface, so that's the one we're going to present (it's ugly, but it's functional)

It's best friend is gitk, which we'll be discussing very shortly.

On Mac, you can use GitX: http://gitx.frim.nl/

Setup

General setup

Let's use the project "YourProject" as an example.

Launch Git Gui.

  • Choose "Clone existing repository" in the dialog box.
  • Then, in "URL", type "git@src.aldebaran-robotics.com:yourproject.git"
  • In "Directory", put a the address of the directory where you want the repository to be cloned (this directory mustn't already exist)
  • Click "Clone"

Essential: In Edit/Options, in the "Global" tab, enter your full name and e-mail address.

Setup

A good tool for solving conflicts is kdiff3.

You can install kdiff3 from here on Windows or Mac, or as a package on Linux.

Copy the following lines into:

  • C:/Documents and Settings/<User>/.gitconfig on Windows XP
  • C:/Users/<User>/.gitconfig on Windows Vista or Windows Seven
  • ~/.gitconfig on Linux
[gui]
  encoding = utf-8
 
[guitool "clean"]
  cmd = git clean -f
  confirm = yes
 
[guitool "pull"]
  cmd = git pull
  confirm = no
 
[guitool "pull-rebase"]
	cmd = git pull --rebase
 
[merge]
  tool = kdiff3
 
[mergetool "kdiff3"]
  path = c:/Program Files/KDiff3/kdiff3.exe
 
[ui]
  color = true
 
[push]
  default = current
 
[user]
  name = A. Developer
  email = adeveloper@aldebaran-robotics.com
 
[core]
  excludesfile = ~/.gitexcludes

NB: the [mergetool "kdiff3"] section is not necessary on Windows.

You can then edit the ~/.gitexcludes file to ignore any files generated by your IDEs. For example: .pydevproject (for Eclipse), CMakeLists.txt.user (for QtCreator)

Git Gui, one day at a time

Add files to commit

When you launch Git Gui, there are two frames in the left column:

  • unstaged changes - the names of the files that have been changed
  • staged changes - the files that have already been added to the index

You can see the changes that have been made by clicking on the filenames, and change a file's status from unstaged to staged or vice versa by clicking on the icon just to the left of its name.

Committing

When all the files are ready, you can enter a commit message in the frame near the bottom, then press commit.

When you're ready to push all the commits you've made locally to the remote repository, you can press the push button.

Pulling

When you want to fetch all changes from the repository, click "Tools/pull". (this will only work if you've set up the configuration file described above)

If you're a Linux user (and you haven't touched the ~/.gitconfig file), you can just type "git pull" in a terminal, somewhere in your local project directory.

If everything goes as planned, you should get a pop-up summary of what's been changed, and commit called "merge branch <blabla> in <balbal>" will automatically be created. (NB: you need to use the configuration file described above for this to work)

Troubleshooting

You may encounter the following kind of errors:

 The remote end hung up unexpectedly
  • Perhaps you are using the wrong repository URL address. Check that it is correct.
  • Check that you have installed Git correctly.
 git.aldebaran.lan[0: 10.0.2.18]: errno=Invalid argument
 fatal: unable to connect a socket (Invalid argument)
  • Perhaps you are using the wrong repository URL address. Check that it is correct.

Resolving a conflict

If there are conflicts, you will see that Git has still prepared a commit for merging, which contains all files on which there were no conflicts and are ready to be added.

Files with conflicts appear with a "requires merge resolution" line.

You can right-click on the differing text, then select "run merge tool" to resolve the conflict.

On Linux, type "git mergetool" in a terminal. This command will launch kdiff3 if you have configured Git correctly.

You may also notice files with strange names like "thing.cpp.back.cpp" appearing. These are created automatically when there's a conflict, but they aren't destroyed automatically. If you want, you can select "Tools/clean" to get rid of them.

Once you have resolved the conflict, simply add the offending files to the index and commit.

Pushing

Just click on "push", and then "push" again in the subsequent pop-up (the default values are fine).

Forget pointless local changes

Sometimes you want to forget some changes you've made that turned out not to be useful, but do have some useful commits on the same files. To forget these "change sets" in relation to your local Master branch, you can do:

git checkout file_name

in a Git bash or a terminal. All unneccessary changes will be forgotten, but your local commits will be protected!

Important: don't forget the file name, it makes a huge difference to the meaning of the command!

 

Working on branches

Rebase versus merge

TODO: insérer schema ala progit.

Using git pull --rebase instead of git pull

Avantages :

* History is linear

Disadvantages:

 * Command-line only!

Coding on a branch, one day at a time

  • Creating a new branch

git checkout -b my_feature (create the branch locally)

git push origin my_feature:my_feature (copy the local branch to the remote repository)

 

  • Other developments on my_feature:

git fetch git checkout -b my_feature original/my_feature

 

  • During development:

[on branch my_feature]

git pull --rebase

git push

 

  • Recover the Master branch in a stable way:

git rebase origin/master

/!\: my_feature's history is totally *rewritten* so that there are no conflicts

If you resolve any conflicts, they will be lost!

DO NOT use if the conflicts are of any important (do a merge instead)

 

  • Merge on master once you're happy

git checkout master git pull git merge origin/my_feature

(Often, it feels like you've nearly finished at this point. You haven't.) Don't forget to test!

git push origin master:master

-> automatically creates a review on Review Board

Cloning your repository to identify differences between two branches

git checkout master cd .. git clone myproject myproject-master cd myproject git checkout my_feature

kdiff3 myproject myproject-master

Using gitk

Gitk is a repository browser that shows all commits in tree form. You can launche Git Gui from gitk (File/Start git gui), or launch gitk from the Git Gui (Repository/Visualize All branch history)

On launching gitk, you will see a list of all commits in the repository (labelled with the text from the first line of each commit message)

One line will be tagged with a coloured, outlined rectangle (labelled "master", for example, if you're working on the master branch): this shows the last commit made on a given branch. In our example, "master" is the local branch you're currently working on, and "remotes/origin/master" is the corresponding remote branch. The label for your current position on the tree is shown in bold.

 

Tips and Tricks

gitk and Git Gui don't refresh automatically

Feel free to use and abuse the F5 button.

(on Windows) Avoid leaving your IDEs open when calling Git commands

Classic example: you try to pull the latest version of your project, but a file that has been deleted from the remote branch is still open in your IDE. Git will be unable to delete the file from your computer as it is in use, and your file set will end up in a really weird state. If this happens, close the IDE, do a hard reset on the last commit, and then start again from the beginning.

NB: it's generally ok to leave your IDE open, as long as you're not debugging, and you don't have any files open that will cease to exist after the Git command.

Create a patch between 2 revisions

git format-patch rev1..rev2

ou aussi

git format-patch rev1..HEAD

ou alors

git format-patch "HEAD~2..HEAD"

takes the last two commits.

Git gc

This repository currently has approximately 250 loose objects. To maintain optimal [...] Compress the database now? [yes] [no]

You can edit your preference file (%USER_DIR%/.gitconfig) and add the following: (replace <value> by a number, the default being 250)

[gc]
  auto = <value>

Loose objects will now only be compressed when there are <value> of them. Performance will probably suffer if you set the mark too high!

For the diehards out there, you can also do this through the command prompt:

git config --global gc.auto <valeur>

To find out more: http://www.kernel.org/pub/software/scm/git/docs/git-gc.html

Error messages

Here are a few of the error messages you're likely to come across, along with a possible solution.

subprocess aborted anormally

This error message appears in gitk when you've done something strange or something's gone wrong, but isn't very helpful.

There's only one way to get to the bottom of it: open a terminal window and see what you get from "git status".

If the error is "currently not on any branch", you need to do "git checkout <the branch that you were on>"

Append the command you were attempting to run ("git status" + cherry-pick, pull,...) for a more explicit error message.

If the error mentions "permission denied" and you're running Windows, check that the files in your local copy are closed, then try again.

foobar.cpp is not uptodate: cannot merge

You just tried to pull from the remote branch, but you have changes to your local copy that haven't yet been committed. Commit these changes, then try again.

git push: failed to push some refs (non fast-forward)

You just tried to push your changes without pulling first. Pull all changes from the remote repository and then try again.

You asked me to pull without telling me which branch you want to merge with, and 'branch.foobar.merge' in your configuration file does not tell me either

You were working on a branch other than master and did a git pull.

Edit your .git/config file (to be found at the root of your working project) to obtain something like this:

[remote "origin"]
	url = git@src.aldebaran-robotics.com:yourproject.git
	fetch = fetch = +refs/heads/*:refs/remotes/origin/*

[branch "master"]
	remote = origin
	merge = refs/heads/master
[branch "foobar"]
        remote = origin
        merge = refs/heads/foobar

This way, your local foobar branch will now track the remote foobar branch. (It's generally a good idea to give your local branches the same names as the remote ones)

warning: You did not specify any refspecs to push, and the current remote ....

You did a git push through the command line. For once, the error message is explicit (though very very long). One solution is to edit ~/.gitconfig so as to have:

[push]
    default = current

Re-read the message and do whatever feels right to you.

This problem never occurs when using the Git Gui, because you always choose from where to where you wish to push.

Managing remote branches

Creating a branch in the repository

All you need to do is push your local branch to the repository On the command line:

 git push remote_name myBranch:myBranch

Generally, remote_name is origin, which makes that:

 git push origin myBranch:myBranch

With the GUI, you just need to click "push" whilst on the "myBranch" branch.

Tracking a new remote branch

Next time you do a git pull (or fetch), you'll be informed that a new remote branch has been created:

  new branch -> remotes/origin/newBranch

You may then wish to create a local branch to follow what's happening on this new branch:

 git checkout -b newBranch origin/newBranch

Or, you can search in gitk for the branch which corresponds to origin/newBranch, right-click then select "create new branch", edit the .git/config file.

Deleting a remote branch

Let's imagine that you've got a local branch called oldBranch which is tracking a remote branch called origin/oldBranch.

You can't delete the branch unless it has been totally merged with another branch (probably master). Let's start by checking that:

 git branch -d oldBranch

This shouldn't send back any error messages. Check that you're up to date, and then do:

 git push origin :oldBranch

Cross-compiling NaoQi with Git

Procedure

  • We'll assume here that your local copy is accessible from a Linux OS (if your local copy is already on Linux then great; if you're a Windows user, then you need to set up a virtual Linux machine, and mount a shared directory containing your local copy)
  • Clone the local copy to another copy, which you'll use ONLY for cross-compilation (and must be on a Linux filesystem, i.e. not in the shared directory), which is the whole point of all this.
 git clone /path/to/original/local/copy /path/to/cross-compilation/local/copy
  • Create a branch called 'head':
 git branch head
  • Recover the cross-toolchain, which you can extract to ~/ctc/ctc-<version> for example.
 cd /path/to/cross-compilation/local/copy
 CTC_PATH = /home/<user>/ctc/ctc-<version> make init

You'll need to do this every time the cross-toolchain is updated. This command will launch crosscompile.sh with all the right arguments, and then launches ccmake. Now is the time to choose between DEBUG and RELEASE.

  • then, to test an ongoing development, commit changes to your original local copy. Then, go back to your cross-compilation copy and type:
 make

This will update your cross-compilation copy up to the current commit on your original copy, then launches compilation.

  • To send it all onto a robot:
 IP=<robot IP> make deploy

This will copy libmotion.so and libalcommon.so onto the robot, in a location where they will be loaded by NAOqi at the next startup.

Advantages of this procedure

- Compared to sharing a local copy directly, and then compiling and cross-compiling in this local copy:

1. Your cross-compilation environment is kept clean

2. Your development environment is kept clean

3. You no longer need to fiddle around with buildconfig.cmake all the time, and you can just have a module in the cross-compilation copy of buildconfig.cmake.

4. You no longer need to recompile everything because a config.h has been changed.

- Compared to sharing two versioned repositories:

5. Synchronisation of sources between both directories is done automatically by a dedicated tool, without the need to go through a shared server.

(This is the end of commit messages like: "does this work on Linux?" followed by "fixed compilation on Linux")

6. All commands (../crosscompile.sh, cmake, ccmake, scp ... ) are lined up in just one file, which is also versioned.

Advanced use

If you've read this far, then you deserve to learn more.

These commands aren't accessible via gitk/Git Gui, which is why we haven't mentioned them so far.

git stash

You're working on a really great feature, so on the "greatFeature" branch. You've got a few non-committed changes on your local copy, and anyway nothing's compiling any more: you're mid-development.

Suddenly, someone walks in shouting: "there's a major bug that needs fixing on the master branch, now!" What should you do?

Here's a practical solution: type "git stash" into the command line, and your working copy is now free of local changes, any uncommitted changes are saved elsewhere. You can now checkout the master branch, and fix the bug.

When you've finished, you can come back to the "greatFeature" branch and type "git stash apply". You'll now be back to where you started on that branch.

Important: if you do "git stash apply" on the wrong branch, who knows what might happen...

git revert

This is nothing like an "svn revert". "svn revert" allows you to cancel local, non-committed changes (equivalent to git reset --hard) "git revert" allows you to cancel changes made by a specific commit.

Example:

 7cf95: improve doc
 84c35: bug fix
 75d66: foolishness
 54ccc: small optimisation

Do "git revert 75d66", and you'll create a commit called "revert foolishness, and reverses the effects of the commit in question.

In the end, it's as if the history was:

 7cf95: improve doc
 84c35: bug fix
 54ccc : small optimisation
トップに戻る