qgis

My QGIS git workflow


I thought it might be handy to post the git workflow that I use when working on QGIS, or any project for that matter.

In the following examples upstream = https://github.com/qgis/Quantum-GIS.git. If you have cloned from your github fork of QGIS you can add upstream using:

$ git remote add upstream https://github.com/qgis/Quantum-GIS.git

The first thing we need to do is pull down the latest changes from the main QGIS repo aka upstream

$ git fetch upstream
remote: Counting objects: 13, done.
remote: Compressing objects: 100% (1/1), done.
remote: Total 7 (delta 6), reused 7 (delta 6)
Unpacking objects: 100% (7/7), done.
From github.com:qgis/Quantum-GIS
   18cd145..89bdb10  master     -> upstream/master

Now that we have the changes in our local repo we need to bring our master branch up to date with the latest changes from upstream. I use rebase here because I don’t want to see merge master into master etc etc each time I want to bring my master branch up to date. In the end I want my local master branch to reflect upstream/master exactly

$ git rebase upstream/master
First, rewinding head to replay your work on top of it...
Fast-forwarded master to upstream/master.

Note: You can combine the two into one call using: git pull upstream master --rebase

In order to do any work in git you should really be using branches. We can check a new one out using:

$ git checkout -b working
Switched to a new branch 'working'

This will checkout a new working branch off my local master branch and switch to it.

Lets do some work.

$ git commit -a -m "Add some feature"
[working 8cd2f4b] Add some feature

$ git commit -a -m "More feature stuff"
[working 72d30ad] More feature stuff

$ git commit -a -m "bug fix"
[working 25b10e5] bug fix

$ git commit -a -m "bug fix"
[working 211e387] bug fix

Note: The -a means add any changed files to the commit. You can also use git add. I’m trusting you already understand how to add files to a commit.

Now at this point I could merge my changes into the master branch and push it up, or if you don’t have commit rights you can issue a pull request. However having heaps of “fix this”, “fix that” commits is pretty ugly. This is where git rebase can come in handy.

We can check which commits we have added that are not in master by doing:

$ git log --oneline master..
211e387 bug fix
25b10e5 bug fix
72d30ad More feature stuff
8cd2f4b Add some feature

Alt Text

There we can see we have four commits that differ and that 8cd2f4b is the first commit we made. I really want to merge all the commits into one to make this a little cleaner.

$ git rebase -i 8cd2f4b^

Note: ^ means go back one commit from the one listed. git rebase doesn’t include the commit that you list so you have to go back one before it.

pick 8cd2f4b Add some feature
f 72d30ad More feature stuff
f 25b10e5 bug fix
f 211e387 bug fix

# Rebase 89bdb10..7d02daf onto 89bdb10
#
# Commands:
#  p, pick = use commit
#  r, reword = use commit, but edit the commit message
#  e, edit = use commit, but stop for amending
#  s, squash = use commit, but meld into previous commit
#  f, fixup = like "squash", but discard this commit's log message
#  x, exec = run command (the rest of the line) using shell
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.

I have changed all but the first commit to f this will merge all the commits into the first one. The latest commit is at the bottom so you should read the rebase screen from bottom up.

[detached HEAD d5620a5] Add some feature
 1 file changed, 3 insertions(+)
 create mode 100644 test.txt
Successfully rebased and updated refs/heads/working.

Alt Text

At this point I normally merge it into master and push it upstream, but if you don’t have commit rights then you can push it up to your github repo and open a pull request.

# Push them up for review
$ git push myrepo working

Important Note:

git rebase -i will change the commit hash for anything that is included in the range of commits. Make sure you only rebase commits that are not public yet. Only rebase commits that in your local repo.

Open Source, qgis

Visualizing QGIS code activity between 1.6 and 1.7 – Video


Here is a quick video I did this morning using the very cool tool Gource. The video shows all the commits (1265) that were made between release 1.6 and 1.7. QGIS 1.7 was released on 19th June 2011 but there was still some clean up in the 1.7 branch paste that date into September.

It’s interesting to note the large burst of activity around November.

Open Source, qgis

Fresh off the press – QGIS 1.7 is released!


Tonight Tim Sutton officially made the release announcement for QGIS 1.7, so I’m guess I’m free to blog about the newest version now and its cool features.

 

 

 

 

What are you still doing here? Go and get it! http://www.qgis.org/wiki/Download

I am, as a heavy QGIS user and a guy-who-tries-to-write-features-and-patches-for-the-code, very happy with this release. I know a lot of people have put a lot of hard work and free time into working on features and bug fixes that keep making this free GIS system even better.

Some of the more notable new features in this release are…well there are just way to many for me to list here so go and check out the official list at http://qgis.org/component/content/article/127-qgis-1-7-release.html

The QGIS team has shifted their source control system to using GIT, which I am very happy about as a lot of the guys on the #qgis IRC channel will know :). The bug tracer has also been moved tohttp://hub.qgis.org/projects/quantum-gis.

Since the release of QGIS 1.6 there have been 1199 commits (using git to count: git log –pretty=oneline upstream/release-1_6_0..upstream/release-1_7_0 | wc -l). Not a bad effort if I may so myself.

 

If you are still reading this, I really hope it’s because you are waiting for QGIS 1.7 to install.

Open Source, qgis

Delete all branches on a github repo


I forked the QGIS repo on GitHub a little while ago although one thing that bothered me was that it gave me copies of all the branches that the main QGIS git repo had.  This is understandable as it’s the way it works however I don’t really need all these branches in my forked copy of the repo as I don’t care about them.   I only care the ones I am working on, and I don’t want to see a big list of branches in my git fork that have nothing to do with me.

So the next question was how do I delete all the branches on the remote repo at GitHub. Well you would normally do:

git push origin :branch_name

although doing that by hand for each branch is, well, a pain in the butt! After a chat with a guy (strk) on the #qgis IRC channel who is more skilled at bash then me (I’m still a Linux noob) he came up with this:

git branch -r | grep origin/ | grep -v master | grep -v HEAD| cut -d/ -f2 | while read line; do git push origin :$line; done;

Very handy.

Open Source, qgis

Generating a Gource source commit history visualization for QGIS (Quantum GIS)


A couple of days I found a pretty cool open source project for visualizing the history of version controlled code.  The project is called Gource and can be found here: http://code.google.com/p/gource/ On the videos wiki page there are a few videos of other projects that have used gource to generate cool videos of their commit history so I thought I should make one for QGIS.

After downloading and building the latest source for Gource and fetching the current trunk of QGIS following these instructions: http://spatialgalaxy.net/2010/12/27/contributing-to-qgis-using-git/ I ran the following in my terminal on my Ubuntu machine, with my current directory being the download git repo from the above instructions:

  gource --title "Quantum GIS" --logo images\icons\qgis-icon.png \
  --hide filenames --date-format "%d, %B %Y" --seconds-per-day 0.15 \
  --highlight-all-users --auto-skip-seconds 0.5 --file-idle-time 0 --max-files 999999999 \
  --multi-sampling --stop-at-end --elasticity 0.1 -b 000000 \
  --disable-progress --user-friction .2 --output-ppm-stream - | \
  ffmpeg -an -threads 4 -y -b 3000K -vb 8000000 -r 60 -f image2pipe \
  -vcodec ppm -i - -vcodec libx264 -vpre libx264-medium qgis.mp4

Don’t worry I know it looks crazy but it’s really not that bad.  I’ll break it down.

  • –title “Quantum GIS” Well, yeah, adds a title to the project.
  • –logo images/icons/qgis-icon.png Adds a icon watermark
  • –hide filenames This hides the filenames of the files being committed . I hide these because it makes it pretty hard to see.
  • –date-format “%d, %B %Y” Formats the date at the top of the video.
  • –seconds-per-day 0.15 How many seconds represent a day. The lower this is the fast a day goes by, meaning me commits in less time.
  • –highlight-all-users Highlights all the users all the time.
  • –auto-skip-seconds 0.5 If there are no commits for this time it will skip to the next commit.
  • –file-idle-time 0 How long before the file disappears from the video, 0 means never good for seeing the full file tree.
  • –stop-at-end Stops the video at the end of all the commits.
  • -b 000000 The background colour, in this case black.
  • –output-ppm-stream – Tells gource to output the result ppm stream to STDOUT, which is then piped ( | ) into the ffmpeg

That is a quick overview of some of the gource arguments, running grouce -H in your terminal will print out the full list.  I’m not going to go into the ffmpeg arguments because frankly I don’t understand them very well and video isn’t really my thing.  I’m sure there are ffmpeg experts that would be able to do it better then what I have.

After running the above commands in my terminal window and letting it do its thing, I had a resulting mp4 file which I then uploaded to YouTube.

Below is the video that I uploaded to YouTube, which took about 5 hours due to my very very slow (read 138kb/s) upload speed.  The video is about 8 years of QGIS development in just over 9 minutes.

As it’s a bit hard to see in the video due to the quality, each cluster of files is a directory and the branches show the folder hierarchy.

Direct link to video: http://www.youtube.com/watch?v=-NILKRiMtcU

Happy new year and happy coding :)