Update: added mapdeck, leafgl, and tmap to the Tour d’Horizon

TL;DR: If you’re just here for the code / hacks, feel free to skip this intro and jump directly to the hands-on part.

1 A Bit of a Context: Spatial Tour d’Horizon

To be precise, my R “career” only started to gain traction when I was asked to create a low-budget CC0 print map by a colleague almost one and a half years ago. This particular use case gave me enough of an impression of the awesomeness of the Rstats / FOSS-minded community, the holistic potential of R re: all things data, and in particular the #GIStribe’s continuous efforts to make spatial processing with R accessible to everyone. What I did not know back then (among maaaaany other things) was that there had already existed a dual system of Base R and Tidy / Tidy-fied R - even for spatial…

If you consider that this post by @mdsumner was written already two years ago, and R users potentially have been able to do stuff such as to “colour a WebVR mesh by shading using a raster layer” (cf. screenshot above; McBain 2018) with the r2vr package in R for more than half a year now, I think this quote by Michael D. Sumner just perfectly sums up the status quo and the potential of “spatial / GIS with R” - just that this was two years ago…:

“GIS itself needs what we can already do in R, it’s not a target we are aspiring to[,] it’s the other way around.” (Sumner 2017)

I mean, this rayshader output (pictured below; click on the image to see the full GIF in action) by the package’s developer Tyler Morgan-Wall is not just a proof-of-concept. It’s out there on CRAN, it’s one technique already available for everyone with just a basic understanding of R and/or GIS.

Screenshot of Tyler Morgan-Wall’s rayshader demo. Click this link to see the GIF in action.

Screenshot of Tyler Morgan-Wall’s rayshader demo. Click this link to see the GIF in action.

Or have a look at what Michael D. Sumner has been working on with the quadmesh package, which will allow us - among other things - to take raster to another level. Same for Edzer Pebesma’sstars package with regards to its potential for tidy spatiotemporal data.

Update: @timsalabim3 and @ozjimbob reminded me of other prominent spatial packages I should have had mentioned here, but didn’t do so as I don’t have worked with them yet:

Ever heard of Uber’s {WebGL + Mapbox} based framework deck.gl for rendering really large datasets, XYZ data, and so on? @symbolixAU has you covered with the mapdeck package for R:

Mapdeck madness! Courtesy of SymbolixAU/mapdeck.

Prefer Leafletover Mapbox but still need to display 100K+ points on a map? @timsalabim3 / Team r-spatial have ported something for ya’:

And if you need to make thematic maps and/or don’t want to plot your spatial objects with ggplot, have a look at Martijn Tennekes’s tmap, which is also very well covered by Lovelace/Nowosad/Muenchow 2018, and gives you a lot with very few lines:

# from tmap vignette
# https://cran.r-project.org/web/packages/tmap/vignettes/tmap-changes-v2.html#tm_tiles
data(World, metro)

tm_basemap(leaflet::providers$CartoDB.PositronNoLabels, group = "CartoDB basemap") +
tm_shape(World) +
    tm_polygons("HPI", group = "Countries") +
tm_tiles(leaflet::providers$CartoDB.PositronOnlyLabels, group = "CartoDB labels") +
tm_shape(metro) +
    tm_dots(col = "red", group = "Metropolitan areas") +
tm_view(set.view = 1)  

In short, GIS with R is efficient, effective, affordable, Shiny shiny, and complex enough to serve as the domain for showcasing R and as an exhaustive use case for new R users (or their supervisors). Let us look into that claim now.

2 Use Case

Recently, I did another but far more extensive freelance gig involving custom mapping of real-world geodata, and thereby discovered so many really helpful / effective / efficient / cool packages, methods, tools and workflows that I decided to write another GIS/mapping post. It’s simply time to give something back to the community (and to create a useful reference entry for my future self).

Update: The study on “The Logic of Chemical Weapons Use in Syriahas now been published during the Munich Security Conference 2019. It’s an acribic piece of research by Tobias Schneider and Theresa Lütkefend from the Berlin-based Global Public Policy Institute (GPPi). The study examines and explains the logic behind 336 chemical weapons attacks (98% by regime) during the Syrian Civil War. I had the challenging honour to help visualise the overall extent and the attack patterns in a few selected case studies:

Selected visuals from the “Nowhere to Hide” study on the use of Chemical Weapons in Syria. Read the full report on GPPi’s website. Almost all maps are CC-licensed.

All maps / visuals (including the spatial event data) where preprocessed and rendered with R and are based on vector data from Natural Earth and OpenStreetMap. Final polishing for print/pdf/web was done with Affinity Designer.

In this post, we’re going to take a tour through several versatile spatial packages for R. As a use case, we’re going to visualize cycling data from Berlin-based Tagesspiegel’s DDJ #radmesser-project for my hometown Berlin, Germany.

This post is not about basic geocomputation / GIS / cartography concepts. Regarding this, there already are plenty of excellent up-to-date resources such as Lovelace/Nowosad/Muenchow 2018, Jesse Adler’s solid series on R for Digital Humanities, or Sébastien Rochette’s Introduction to mapping with {sf} & Co..

Rather, this post is more of my own glossary for all the magical things / hacks.

If there’s a single really helpful thing to keep in mind for now, it’s this common basic representation of GIS’ data layers:

Source: National Coastal Data Development Centre (NCDDC), National Oceanic and Atmospheric Administration (NOAA), USA

Source: National Coastal Data Development Centre (NCDDC), National Oceanic and Atmospheric Administration (NOAA), USA

3 Packages

These are the packages we’re going to use:

# library(raster) # I prefer to use raster::fun() since raster::select() masks dplyr::select()

4 Basic Workflows - Vector

Let’s assume that we want to make some kind of a map, be it because we actually need a map (i.e. for a great book on European foreign policy), or maybe just to give some spatial data some kind of a cognitive canvas. As geodata tends to take up non-trivial amounts of resources (bandwidth, memory, CPU), it could be useful to focus on only a particular geographic extent. This is where I learned to love the concept of a bounding box (BBOX). Instead of literally downloading the whole world (thereby straining someone else’s bandwidth / server capacity) and then running costly query/filter operations, we can easily define bounds first.

If you’re spatially literate and therefore easily can spot whether you need Lat/Lon or Lon/Lat, or what projection / CRS your favourite BBOX-providing workflow offers, you probably can skip this and just define your BBOX manually with something like bbox <- c(xmin, ymin, xmax, ymax). I tend to struggle with this (even when using web tools such as http://bboxfinder.com), because I either mix up Lat/Lon or underestimate the spatial extent of my geodata or whatever. This is why I think that these three approaches below might be helpful to others, too.

4.1 Basemap Bounding Box: Three Approaches

So first of all we want to decide on the extent of the basemap. Since this is a recurrent task but with varying parameters, I’m offering three different approaches, based on

  • the bounding box (BBOX) of a spatial object (i.e. Admin 1 level Federal state|s)

  • the bounding box of spatial data (i.e. the geocoded cycling data)

  • a hand-drawn rectangle turned into a BBOX

4.1.1 Bounding Box: Object-based

Here, we start with our spatial object of reference. I prefer to work with Open Access / Public Domain data (and not Google Maps / Bing et al.), so let’s fetch some vector data from the Natural Earth project - but as class = Simple Feature instead of Esri’s proprietary (and less tidyverse-friendly) Shapefile format.

For your use case, consult the vignette for rnaturaleath::ne_download() and the Natural Earth website. For the sake of resource efficiency I also recommend to download the data once and then to store it locally with save/saveRDS for future use. As I work a lot with Natural Earth data, I have mounted a network folder D:/GIS/ with all the raster and vector data I’ve downloaded so far. However, I do not recommend downloading the “Download all X themes” files, since this gives you a) shapefiles instead of Simple Features, and b) UTF-8 encoding issues for the feature labels, depending on when the particular theme was compiled the last time.

Download a certain Natural Earth feature set (“theme”): world-wide substates / Admin-1 level

substates10 <- rnaturalearth::ne_download(scale = 10, type = "admin_1_states_provinces", category = "cultural", returnclass = "sf")

# saveRDS(substates10, file = str_c(here::here("data", "GIS_workflows", "/"), "substates10.rds"))
# substates10 <- readRDS(file = str_c(here::here("data", "GIS_workflows"), "/", "substates10.rds"))

Subset the Federal State of Berlin

berlin_sf <- substates10 %>% filter(name == "Berlin")

Next, we’ll use the amazing sf package to fetch Berlin’s bounding box (BBOX).

berlin_bbox <- sf::st_bbox(berlin_sf)

So… wanna have a quick glimpse at Berlin’s bounding box? Here’s all that it takes with the game-changing mapview package by Tim Appelhans et al.:

berlin_bbox %>% mapview::mapview()

Caveat: To keep memory and CPU load low for everyone browsing this post, I mostly will display the output of interactive widgets as .jpg.

If you want to set the “CartoDB Dark Matter” basemap as your default (or any other of the themes supported by Leaflet ), you might want to add this setting to your .Rprofile:

mapview::mapviewOptions(basemaps = c("CartoDB.DarkMatter", "CartoDB.Positron", "Esri.WorldImagery", "OpenStreetMap", "OpenTopoMap"))

And since mapview() is just cool, we clan plot and inspect our Berlin object and the BBOX at the same time. For this we’re going to convert the BBOX object into a regular simple feature object with sf::st_as_sfc on the fly:

mapview::mapview(list(sf::st_as_sfc(berlin_bbox), berlin_sf))

This box would probably be too narrow for further static editing (i.e. for a print map), so it would be cool if we could simply increase the BBOX’s extent. Since the coordinates in the bbox are stored as a numeric vector c(xmin, ymin, xmax, ymax), we can easily expand the bbox by providing another vector of length = 4 and see if the extent is better on the fly:

(berlin_bbox + c(-0.1, -0.1, 0.1, 0.1)) %>%
  # sf::st_as_sfc() %>% 

That seems generous enough. So let’s preserve this box for later.

berlin_bbox <- berlin_bbox + c(-0.1, -0.1, 0.1, 0.1)

4.1.2 BBOX from data

Another approach to get a reasonable bounding box is to calculate the BBOX based on your geodata.

As mentioned above, I’m going to use the freshly released cycling data from the Berlin-based Tagespiegel DDJ / Innovation Lab for the final use case. The data has been collected as part of the sensor-based #radmesser project, where Lab leader Hendrik Lehmann’s team of journalists and/or developers equipped 100 (!) cyclists with close-range sensors to measure the distance of passing-by cars and trucks on Berlin’s roads. Goal: Demonstrate the at-risk status of cyclists in Germany’s capital.

Make sure to check out the award-winning project’s website and the project’s repo.

It’s probably worth the remark that getting the data into R is as simple as sf::st_read(URL)

# License: ODC-By v1.0/Tagesspiegel Radmesser/https://radmesser.de
# cf. https://github.com/tagesspiegel/radmesser/blob/master/opendata/LICENSE.md
berlin_bike <- sf::st_read("https://github.com/tagesspiegel/radmesser/blob/master/opendata/detailnetz_ueberholvorgaenge.geo.json?raw=true")

# saveRDS(berlin_bike, file = str_c(here::here("data", "GIS_workflows", "/"), "berlin_bike.rds"))
# berlin_bike <- readRDS(file = str_c(here::here("data", "GIS_workflows"), "/", "berlin_bike.rds"))

However, st_read silently drops the “stats” field (which is legit, since a GeoJSON feature is defined as geometry + properties only) which contains the single measurements. Or rather: I have no 1-liner idea how to preserve it, be it in QGIS, with jsonlite, or by transforming with http://geojson.io. Fortunately, the Radmesser-team also provides a CSV with the measurements and a column with the respective streets key. So we can address this later (in Pt. 2 of this series). FYI, this is what the structure of this GeoJSON file looks like, so feel free to hit me up on Twitter if you know a solution:

Help! How to preserve the stats field!?

Help! How to preserve the stats field!?

Nonetheless, we can quickly have a look at the traced roads where the 15K+ individual measurements where taken, and also visualize the road class with zcol = "variable".

mapview::mapview(berlin_bike, zcol ="STRKLASSE1")

And now the data-based BBOX:

berlin_bike_bbox <- sf::st_bbox(berlin_bike)
mapview::mapview(list(berlin_bike, st_as_sfc(berlin_bike_bbox)))