Speeding up QGIS build times with Ninja


As a developer, feedback is important when you are working.  The quicker you have the feedback the quicker you can fix the issues.  This doesn’t just apply to feedback from users/clients but also from your tooling.

Finding a new tool that increases my productivity is one of the best feelings, and this is one of those cases.   I was told about using Ninja for building instead Make, Visual Studio, Jom (Qt Build Tool).

If you are not a developer and don’t know what those tools are, they are what we use to build and compile all the code in QGIS.  If this step is slow the feedback loop is slow and it becomes annoying.  Improving this feedback loop greatly increases your workflow and happiness, and by happiness I really do mean that.

Ninja is one of these tools that did this. It’s optimized to be fast.  It does very little work in order to be faster any time it can.  Ninja was built by a developer on the Google Chrome team in order to improve their build times (read the history here)

Building QGIS with Ninja is super easy:

Install Ninja from package manager or using the ninja.exe (the whole tool is a single exe) if you are on windows

cd qgis-src
mkdir ninja-build
cd ninja-build
ccmake -GNinja ..
ninja

Done

You can build just the targets you need using

ninja qgis
ninja pycore

etc

The ccmake setup generates the ninja.build file that ninja uses. Myself and Matthias Kuhn have already patched our QGIS cmake files to handle any odd things that got generated – only a handful of things which was nice

The best thing I find about Ninja is how smart it is on knowing if it needs to build something or not, and this is the point that I find other tools fail on. They spend ages wasting time looking for what to do. Ninja knows it in a instant.

When running Ninja with no code changes I get this (on Windows):

21:18:54: Running steps for project qgis2.15.0...
21:18:54: Starting: "C:\QtCreator\bin\ninja.exe" qgis
ninja: no work to do.
21:18:54: The process "C:\QtCreator\bin\ninja.exe" exited normally.
21:18:54: Elapsed time: 00:00. 

Not even a second. If I did the same with VS or JOM I could have written this post before it finished working out what to do.

Here is what happens changing a single file:

21:19:48: Running steps for project qgis2.15.0...
21:19:48: Starting: "C:\QtCreator\bin\ninja.exe" qgis
[1/6] Building CXX object src\core\CMakeFiles\qgis_core.dir\raster\qgshillshaderenderer.cpp.obj
[2/6] Linking CXX shared library output\bin\qgis_core.dll
21:19:51: The process "C:\QtCreator\bin\ninja.exe" exited normally.
21:19:51: Elapsed time: 00:03.

It’s super impressive. Even a cold build on Windows is shorter now. On Linux it’s even faster due to faster disk access in Linux vs Windows

If you build QGIS form source I would highly recommend giving it a crack because I know you will love it.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s