Datasets visualization suitable roofs for installation of solar panels
Michael TuijpGeospatial Data Scientist
In my previous blog post, I wrote about our project at the municipality of Haarlem.They asked Tensing to come up with a solution to help their residents determine whether their roofs are suitable for installing solar panels. We built a viewer that allows residents to quickly see if their houses are suitable for solar panels. The viewer is made up of three datasets. In this article we zoom in on the three datasets used.
FME as data integration platform
In addition to the datasets, we also used FME. FME (Feature Manipulation Engine) is a data integration platform from Safe Software. The platform allows you to easily connect a large number of applications, transform data and automate data processes. With the FME processes created, the municipality can reproduce the results as new data becomes available over the years.
tHE THREE DATASETS
The three datasets are delivered at multiple levels:
- Dataset #1 contains a 2D map of all roofs from each BAG address within the inner city, containing a classification on whether it’s a flat or slanted roof and whether the building has a monumental status or not (this is important in the municipality’s policy)
- Dataset #2 contains a 2D map of all roof parts from each BAG address within the inner city containing a classification on whether it’s a flat or slanted roof and whether the building has a monumental status or not.
- Dataset #3 contains a 2D map of the viewsheds of the roof parts from each BAG address within the inner city which are visible from the public road.
Interested in the full project description of our work for the Municipality of Haarlem? Download the case study here.
Dataset #1: Roof Types
To make the solution possible, both the municipality and Tensing wanted to use as much open data as possible. This ensures that the municipality can work with data that is easy to obtain and transform. As a result, they do not need data licenses and do not face risks or liabilities from other parties. For Tensing, the use of open data makes it easier to replicate and potentially implement the solution at other municipalities to support them with similar problems.
The municipality of Haarlem is an active FME user. They therefore also wanted the entire process to be built in FME. That was, of course, no problem for us!
In building the first dataset, we used 4 sources:
- The boundaries of Haarlem’s inner city (as the only area of interest), available at https://kaart.haarlem.nl/app/map/62.
- BAG data for all of the buildings within the municipality (building-specific data), available at https://www.pdok.nl/geo-services/-/article/basisregistratie-adressen-en-gebouwen-ba-1.
- Haarlem’s KER data (Kernregistratie data, containing details of buildings having a monumental status or being above/below ground, etc), available at https://kaart.haarlem.nl/app/map/62.
- AHN4 height data in raster format of 0.5m by 0.5m resolution, available at https://www.ahn.nl/ahn-4.
With this data in FME, we only used the downtown buildings, added all the BAG & KER details in a single dataset, and clipped the elevation data for each individual building. Using these small roof grids, we calculated the average slope for each roof and classified it as sloped or flat, based on a benchmark value. In addition, we also classified whether the buildings had monumental status or not. Finally, we transformed the grids and classifications into polygons that could be written into a final vector layer.
A snippet of this final dataset looks something like the view below, with the different classifications based on roof type and monumental status on the left:
Dataset #2: Partial Roof Types
The second dataset was built on the FME process of the first, the only difference being that this dataset contains all the individual roof sections of each building, rather than the entire roof.
We had two options for creating the individual roof sections:
- Reinvent the wheel and create a custom model or process that indicates the edges of roof sections, chimneys, dormers.
- Using already available open data and easily integrating it into our process.
Since we are focused on building an accurate product in a timely manner, we chose to use the data that is already available to everyone, which is the 3D BAG dataset published by TU Delft at https://3dbag.nl/en/viewer. This data looks like the image below, and can be converted into the exact roof sections we need when read in 2D.
Since much of the original process of dataset #1 remained the same, we created a similar process for the roof sections following the same transformation steps. This final dataset looks similar to the first one, but is much more nuanced because it is now classified by roof section. For example, the buildings in the middle of the image below contain a number of flat areas in the middle of the roofs (which is not evident in the more straightforward first dataset). These areas are ideal for installing solar panels!
Dataset #3: Roof Viewsheds in Sight
The third dataset we built is somewhat different from the first two. Additionally, it was also more challenging to build, especially in finding the right tools which could be incorporated into the municipality’s processes.
First, the objective of this dataset was to create viewsheds from a multitude of points along the public road. What that exactly means is best illustrated in the animation below: what can be seen from each point along a particular path?
We’ve thought about using something like the 3D Viewshed Analysis in ArcGIS Earth, however this would require adding subscriptions from both the client and our side and would also be outside of the scope of FME, which was undesirable. After some research we found the tool GRASS GIS, which is already available as an add-on in QGIS, but can also be imported and used as a Python module. For me as a Data Scientist, this seemed like an exciting challenge in familiar territory!
First, we added the public road data, which is available from the open data viewer from the municipality at: https://kaart.haarlem.nl/app/map/57. We transformed this data, so we only have the edges of the road from which we could pinpoint coordinates along these lines each 15 meters. Aa rough estimation of distance would enable enough points to view the whole inner city from, but not too many so that it might significantly lengthen processing time).
After some trial & error on how to handle the viewshed analyses best, we’ve created a Python loop where a viewshed analysis is done for each coordinate along the public road lines, each viewshed is patched onto the previous viewsheds, and the final viewshed raster is transformed into a vector layer.
To give an idea of what these viewsheds look like in practice, below are some examples of output viewsheds which are visible from a particular coordinate in the center of each image.
On their own, these viewsheds don’t seem very informative. However, when we patch each viewshed onto the previous ones, the result suddenly becomes a lot clearer. In the image below we see part of the inner city where anything grey can be seen from the public road.
The most interesting part is when the full viewshed raster is transformed into a vector layer and clipped on only the registered buildings within the inner city. Then we see the final result of the viewsheds on all roof parts which are visible from the public road!
These three datasets are really useful information products in their own right, but their full potential comes when they are combined. Below you can see the classifications of dataset #2 (colored) combined with the viewsheds of dataset #3 (shaded black on top of the roof sections).
From this view you can see exactly which roof sections are flat, sloped, historic and in view or not.
Based on these computerized results, the municipality was able to create a policy map. The map can be viewed here. With this policy map on their public website, the municipality of Haarlem can now clearly indicate for each roof area which areas are eligible for the placement of solar panels.
Are you curious about the entire case? Download the full case study here.