Using Hy, a dialect of Lisp for Python, with QGIS


So tonight I rediscovered Hy. I had seen Hy before a while ago but never really sat down and tried it. Tonight just must have been one of those days to try something new.

So Hy is a dialect of Lisp but embedded in Python which means you can use any Python library will using a Lisp dialect. Pretty nifty.

My next thought was, how would this look using the QGIS libraries. So lets give it a try.

First we need to install Hy:

pip install Hy

Now just create a .hy file and add some code

(import qgis)
(import [qgis.core [QgsVectorLayer]])
(import [qgis.core.contextmanagers [qgisapp]])

(setv layers [])

(defn load-layer [file name]
    (setv layer (QgsVectorLayer file name "ogr"))
    (.append layers layer))

(defn print-layer [layer]
    (print "Layer Name:" (.name layer))
    (print "Valid:" (.isValid layer))
    (print "Extents:" (.toString (.extent layer))))

(defn main [app]
    (load-layer r"F:gis_datatest.shp" "test")
    (for [layer layers] (print-layer layer)))


(with [[app (apply qgisapp [] {"guienabled" False})]]
    (print "Loading QGIS")
    (main app))

run it in our shell and bingo.

F:devhy-qgis>hy qgistest.hy
Loading QGIS
Layer Name: test
Valid: True
Extents: 392515.3457026787800714,6461581.2076761415228248 : 392683.3794420150225051,6461705.1012571481987834

Sweet.

Just for reference the Python version of the above would be:

import qgis
from qgis.core import QgsVectorLayer
from qgis.core.contextmanagers import qgisapp

layers = []

def load_layer(file, name):
    layer = QgsVectorLayer(file, name, "ogr")
    layers.append(layer)

def print_layer(layer):
    print "Layer Name:", layer.name()
    print "Valid:", layer.isValid()
    print "Extents:", layer.extent().toString()

def main(app):
    load_layer(r"F:gis_datatest.shp", "test")
    for layer in layers:
        print_layer(layer)

with qgisappl(guienabled=False) as app:
    main(app)

More readable? No doubt, that is why I love Python, however the strange thing is the first time I looked at Lisp, including Hy, I thought “whoa all those parentheses back it up!1!” but strangely after using it for a while (read: not even a few hours) they don’t seem to be an issue, or not much of one anyway. YMMV.

The cool thing with using Hy is you can still use all the libraries you are used to as the example above shows, PyQt, QGIS, anything.

The other interesting, and pretty funky, thing is that you are able to import .hy files like normal Python files. If you create a file winning.hy you can just import winning into any Python application and it works.

Why bother? Mainly because learning something new is never a bad thing, and you never know what you might pick up.

Check out the Hy for more info on what you can do and how it works

I have also created a hy-qgis GitHub repo for some experiments.

Enjoy!

Serving live tiles from a QGIS project via TileStache


I’m more then likly way behind the 8 ball here, aren’t all the cool kids doing tiles these days, but regardless I thought it was pretty cool to share. The other day I found TileStache a neat little Python app that can generate, cache, and serve tiles from a list of providers. The normal way is via Mapnik (and others) to render a image, there is also a vector provider which can render vector tiles. Nifty.

A while ago I wrote qgis2img which can generate an image for project, or layers, and export it for you. It serves two roles, one is to benchmark a project and layer render times, the other as a simple export tool. I thought it would be pretty cool to be able to export tiles from it but was I never really up for working on the math and all the logic so I left it. Then I found TileStache.

The best part about TileStache, apart from that it’s Python, is that you can make your own providers for it, and the API is dead easy

class Provider:
    def __init__(self, layer):
        self.layer = layer

    def renderArea(self, width, height, srs, xmin, ymin, xmax, ymax, zoom):
        pass

How easy is that! Just implement one method and you are good to go. So that’s what I did. I created a custom provider that will load a QGIS project and render out images. Thanks to the work done by Martin from Lutra Consulting for the multithreaded rendering in QGIS 2.4 this is a hell of a lot easier then it used to be.

Ignoring some of the setup code to create and load the session the whole export logic is in these few lines

   extents = QgsRectangle(xmin, ymin, xmax, ymax)
   settings.setExtent(extents)
   settings.setOutputSize(QSize(width, height))
   layers = [layer.id() for layer in project.visiblelayers()]
   image, rendertime = qgis2img.render.render_layers(settings, layers, RenderType=QgsMapRendererSequentialJob)

with render_layers defined as

def render_layers(settings, layers, RenderType):
    settings.setLayers(layers)
    job = RenderType(settings)
    job.start()
    job.waitForFinished()
    image = job.renderedImage()
    return image, job.renderingTime()

As this is build on top of my qgis2img tool you can see the full code here

Running it is as simple as installing TileStache, cloneing qgis2img, updating tilestache.cfg, and running the server.

$ pip install TileStache
$ git clone https://github.com/DMS-Aus/qgis2img.git
$ cd qgis2img

In tilestache.cfg you can just change the path to the project to render:

{
  "cache": {
    "name": "Test",
    "path": "/tmp/stache"
    },
  "layers": {
    "qgis":
    {
      "provider": {"class": "qgis2img.tilestache:Provider",
                   "kwargs": {"projectfile": "data/test.qgs"}
                  }
    }
  }
}

Then run the server

$ tilestache-server /path/to/tilestache.cfg

Note: The path to the .cfg file seems to have to be the full path. I had issues with relative paths working.

To view the tiles you can load the preview URL that TileStache provides or you can use it in something like LeafLet

<!DOCTYPE html>
<html>
<head>
    <title>QGIS Tiles WOOT!</title>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="http://cdn.leafletjs.com/leaflet-0.7.3/leaflet.css" />
</head>
<body>
    <div id="map" style="position: absolute; top: 0; right: 0; bottom: 0; left: 0;"></div>
    <script src="http://cdn.leafletjs.com/leaflet-0.7.3/leaflet.js"></script>
    <script>

        var map = L.map('map').setView([51.505, -0.09], 5);

        L.tileLayer('http://10.0.0.11:8080/{id}/{z}/{x}/{y}.png', {
            maxZoom: 18,
            id: 'qgis'
        }).addTo(map);

    </script>
</body>
</html>

And the result is live tiles from a QGIS project.

tiles

Winning!

Some Caveats

  • The provider currently doesn’t use metatiles so labels and points will get chopped at tile edge. I have working code for this but haven’t pushed it yet.

  • I don’t kill the QGIS session that I create. Creating a session for each request was really expensive so I just keep it around.

  • I only load the project once so any changes mean starting and stopping the server. Wouldn’t be hard to add a file watcher for this.

  • It’s just me using this at home for fun, I have no idea on how it would scale, or even if it would, but I would be keen to hear feedback on that theory.

Animated QGIS map canvas item


Have you ever wanted to animate a QGIS map canvas item. No? Well soon you will.

First we need to create a custom QgsMapCanvasItem

from PyQt4.QtCore import QPointF, QRectF, QTimer, QObject, pyqtProperty, QPropertyAnimation, Qt
from PyQt4.QtGui import QPainter, QBrush, QColor
from qgis.gui import QgsMapCanvasItem
from qgis.core import QgsPoint

class PingLocationMarker(QgsMapCanvasItem):
    """
    Position marker for the current location in the viewer.
    """
    class AniObject(QObject):
        def __init__(self):
            super(PingLocationMarker.AniObject, self).__init_<a href="https://woostuff.files.wordpress.com/2014/10/marker.gif"><img src="https://woostuff.files.wordpress.com/2014/10/marker.gif?w=300" alt="marker" width="300" height="193" class="alignnone size-medium wp-image-61493" /></a>_()
            self._size = 0
            self.startsize = 0
            self.maxsize = 32

        @pyqtProperty(int)
        def size(self):
            return self._size

        @size.setter
        def size(self, value):
            self._size = value

    def __init__(self, canvas):
        self.canvas = canvas
        self.map_pos = QgsPoint(0.0, 0.0)
        self.aniobject = PingLocationMarker.AniObject()
        QgsMapCanvasItem.__init__(self, canvas)
        self.anim = QPropertyAnimation(self.aniobject, "size")
        self.anim.setDuration(1000)
        self.anim.setStartValue(self.aniobject.startsize)
        self.anim.setEndValue(self.aniobject.maxsize)
        self.anim.setLoopCount(-1)
        self.anim.valueChanged.connect(self.value_changed)
        self.anim.start()

    @property
    def size(self):
        return self.aniobject.size

    @property
    def halfsize(self):
        return self.aniobject.maxsize / 2.0

    @property
    def maxsize(self):
        return self.aniobject.maxsize

    def value_changed(self, value):
        self.update()

    def paint(self, painter, xxx, xxx2):
        self.setCenter(self.map_pos)

        rect = QRectF(0 - self.halfsize, 0 - self.halfsize, self.size, self.size)
        painter.setRenderHint(QPainter.Antialiasing)
        painter.setBrush(Qt.green)
        painter.setPen(Qt.green)
        painter.drawEllipse(QPointF(0,0), self.size, self.size)

    def boundingRect(self):
        return QRectF(-self.halfsize * 2.0, -self.halfsize * 2.0, 2.0 * self.maxsize, 2.0 * self.maxsize)

    def setCenter(self, map_pos):
        self.map_pos = map_pos
        self.setPos(self.toCanvasCoordinates(self.map_pos))

    def updatePosition(self):
        self.setCenter(self.map_pos)

marker = PingLocationMarker(iface.mapCanvas())

and this is the result

marker

wweeeee animated canvas item.

Run the above code in a QGIS Python console editor window and you should get the same effect.

So what is this magic? Well it turns out to be pretty easy all thanks to the handy class QPropertyAnimation. QPropertyAnimation takes a QObject and sets a property value until the end value is hit over the duration, the cool thing with this class is that it can take any QVariant type, which is pretty much anything, and it will go from start value to end value. You can also use other easing curves to change how the values change over time.

Super nifty!

The main important part of this is:

self.anim = QPropertyAnimation(self.aniobject, "size")
self.anim.setDuration(1000)
self.anim.setStartValue(self.aniobject.startsize)
self.anim.setEndValue(self.aniobject.maxsize)
self.anim.setLoopCount(-1)
self.anim.valueChanged.connect(self.value_changed)
self.anim.start()

which calls the value_changed and self.update() methods, when update is called paint will be called and we grab the current animation value. Note: -1 loop count means run forever.

self.aniobject is a custom QObject to hold our current animation value. QgsMapCanvasItem is not a QObject so we have to make another object to hold that value for us. I tried double inheritance here and it didn’t like it so I went with a nested class which is nice anyway.

And that is all you need to make a animated QgsMapCanvasItem, remember that QPropertyAnimation can be used on any QObject so you could do some pretty cool stuff with this if you have the need.

Be interested to hear any ideas on if we can use this in QGIS.

Note: A canvas item like this isn’t part of any layer. Canvas items live on on the canvas itself, above or below the map image. The markers that you see when you enable editing on a layer are canvas items, as are the lines when drawing a measure line, even the image you see in the canvas is a canvas item, etc.

Thank you from Stacey and I.


Though the kindness of everyone we made it to over $2000 to donate two, not one like we had planned at the start, camera packs to hospitals that need them. One will be going to the new Gold Coast hospital where Ellie was born and the other to another hospital who needs it. Both cameras will have Ellie’s name on them in her memory. They will go a long way to help preserve the memories of those last minutes just like it did for us with Ellie.

The money is now with Heartfelt and hopefully the cameras will be done soon. I will update this post with photos once they are done.

It was really amazing to see the number of people who I have never meet in person from all over the internet throwing in what they could to help us reach the goal. We are extremely grateful for everything everyone put in.

A massive thanks to everyone who donated:

  • Lyn Noble
  • Kylie and Nathan
  • Luke Bassett
  • Joanne Smith
  • Darryl & Angela Browning
  • Digital Mapping Solutions
  • David Baxter
  • Carl Wezel
  • Grandad and Grandma Woodrow
  • Bill Williamson
  • Helen Gillman
  • Karlie Jones
  • Lisa Gill
  • Andrew & Peta
  • Sally Drews
  • Matt Travis
  • Mummy, Daddy, Harry & Little Sis..
  • Terry Stigers
  • James McKeown
  • Jill Pask
  • Kym Zevenbergen
  • Jessica Nayler
  • Amelia Woodrow
  • Judy Burt
  • Russell and Suzann Woodrow
  • Jenny & Mark Gill
  • Emeley Sands
  • Rebecca Penny
  • Larissa Collins
  • Ross McDonald
  • Shantelle Sweedman
  • Rebbecca Ben izzy Erica
  • Aidan Woodrow & Andrew Smith
  • simbamangu
  • Sarah Rayner
  • Sassá
  • Matt Travis
  • Marco Giana
  • Heikki Vesanto
  • Jorge Sanz
  • Pure K.
  • Toby Bellwood
  • Andy Tice
  • Ujaval Gandhi
  • Matt Robinson
  • Geraldine Hollyman
  • Anonymous
  • Teresa Baldwin
  • Alexandre Neto
  • Chelsea Fell
  • Stephane Bullier
  • Nathan Saylor
  • Adrien ANDRÉ
  • Steven Feldman
  • Anita Graser
  • Chris Scott
  • Vicky Gallardo
  • Anonymous
  • Anonymous
  • Stevie Little

I will also add a massive thank you to DMS who has been super supportive though this whole year since Elly died, and they know very well the effect it has had on Stace and I over the past year.

In a perfect world we would never had to run a fund raiser for this but I’m glad Heartfelt exist to help those of us in need at the time.

Thank you from Stacey and I.

QGIS atlas on non geometry tables


This is proof that no matter how close you are to a project you can still miss some really cool stuff that you never knew or considered was possible.

The problem to solve:

You have a CSV with a row of colours. Each row should be a new map and each column is the colour for that feature.

This is example of that kind of input

A       B
#93b2f3    #FF0000 
#dfbdbb    #FF0000
#f9d230    #FF0000

This questhion was asked on GIS.SE this morning. When I first saw it I had no idea it was even possible, I was thinking along the same lines as the person asking, that it would have to be done with Python. Not hard, but a lot harder then something built in and I put it in the too hard basket. I thought the atlas can almost do that, almost but not really.

Well almost was wrong. It can.

Note: You will need QGIS 2.5 (2.6 when released) for this to work

Lets make some cool maps! (and go to GIS.SE and upvote Nyalls answer)

First open your vector layer and the CSV. Don’t worry about style just yet, we will do it later.

Create a composer and add your map.

Here comes the first part of the trick.

Enable Atlas and set the coverage layer to the CSV layer. Wait? What? That doesn’t make any sense. If you think about it for a while it does. We need a map for each row (or “feature”) in the CSV and atlas does just that.

atlas_colours

How do we style the features? Well here is the other part of the trick. In 2.6 there is a magic expression function that returns a field value from another feature. And it’s as simple as attribute( $atlasfeature , 'A' ) – give me the attribute from the current atlas feature for field ‘A’. Simple.

First we categorize our features so we have a symbol for each feature. I’m using a sample layer I have but you can understand how this works. The first feature is A and the other is B, etc, etc

render

Now to use another awesome feature of QGIS. The data defined symbol properties (and labels too). Change each symbol and define the colour data defined property. Using attribute( $atlasfeature , 'A' ) for the first one and attribute( $atlasfeature , 'B' ) for the second.

atlas_feature

That is it. Now jump back over to your composer and enable Atlas preview.

atlas1

atlas2

Bam! Magic! How awesome is that!

Now my other thought was. “Ok cool, but the legend won’t update”. I should learn by now not to assume anything. The legend will also update based on the colours from the feature.

atlas1_legend

atlas2_legend

How far can we take this. What if you need the label to match the colour. Simple just make the label text look like this:

<h1 style='color:[% "A" %]'>This is the colour of A</h1>

atlas1_label

Heaps of credit to Nyall and the others who have added all this great stuff to the composer, atlas, and the data defined properties. It’s not something that you will do every day but it’s great to see the flexibility of QGIS in these situations.

You can even make the background colour of the page match the atlas feature

atlas1_back

but don’t do that because people might think you are mad ;)

Exporting QGIS symbols as images


Ever wanted to export your QGIS symbols as images? Yes. Well here is some Python code that will let you do just that:

from PyQt4.QtCore import QSize
from PyQt4.QtGui import QImage, QPainter

style = QgsStyleV2.defaultStyle()
names = style.symbolNames()
size = QSize(64, 64)

for name in names:
    symbol = style.symbol(name)
    if not symbol.type() == QgsSymbolV2.Marker:
        continue

    image = QImage(size, QImage.Format_ARGB32_Premultiplied)
    image.fill(0) 
    painter = QPainter(image)
    symbol.drawPreviewIcon(painter, size)
    painter.end()
    image.save(r"C:temp{}.png".format(name), "PNG")

Or in 2.6 it’s even easier:

from PyQt4.QtCore import QSize
from PyQt4.QtGui import QImage, QPainter

style = QgsStyleV2.defaultStyle()
names = style.symbolNames()
size = QSize(64, 64)

for name in names:
    symbol = style.symbol(name)
    if not symbol.type() == QgsSymbolV2.Marker:
        continue

    image = symbol.asImage(size)
    image.save(r"C:temp{}.png".format(name), "PNG")

Bam!

symbols

Why? Because we can.

Fundraising for Eloise and Heartfelt


The death of our daughter was one of the hardest things my wife and I have ever had to deal with. It’s not something that I wish anyone ever have to experience and feel really sorry for those who have had to do it many times. There is something really raw about losing your own flesh and blood. It cuts deep, really really deep. There are really no words to describe the emptiness that you feel, or the feelings that follow after the event. Even with all the pain of loosing a child there is a great Australian service that helps to capture some of the final monents. The service is called Heartfelt and we used them for Ellie.

Heartfelt is a great free service that provides a photo session, including editing and prints (hard and digital) after, in the last and final days. This is the quote from their site:

Heartfelt is a volunteer organisation of professional photographers from all over Australia dedicated to giving the gift of photographic memories to families that have experienced stillbirths, premature births, or have children with serious and terminal illnesses.

Heartfelt is dedicated to providing this gift to families in a caring, compassionate manner.

All services are provided free of charge

Pretty impressive stuff. The last thing you want to have to do in a time like that is shell out for photos when you have other pressing issues.

As Ellie’s 1st Birthday is coming in up October Stace and I would love to raise enough money to donate a camera pack to a hospital though Heartfelt in Elly’s name. We have created a fundraiser page in her name at: http://www.mycause.com.au/page/79669/eloises1stbirthdayheartfelt in order collect dontations for anyone who would like to help.

Camera packs can be donated to a hospital to allow staff at the hospital to capture photos if Heartfelt can’t make it. The bonus is that Heartfelt will still edit and print the photos. How bloody awesome is that! More info on the camera packs is at: http://www.mycause.com.au/page/79669/eloises1stbirthdayheartfelt

We would be greatful for any donations, big or small, so we can donate a camera pack in Elly’s name.

We love and miss you a lot Eloise.