Object recognition with Deep Learning for more efficient inspections
Ivo de Liefde
Esri Certified Professional
Many organizations are busy optimizing their (work) processes. The reasons why they are doing so differs from one company to the next. Some must continue to deliver goods and services within the framework of a tight labor market; others must focus on reducing CO2 emissions to meet their climate objectives. The latest technologies to help them optimize their operations represent an important toolbox — containing, among other things, the Internet of Things and the Cloud, but also machine learning and artificial intelligence.
In this blog post, we explain how to recognize cars in aerial photos with TensorFlow and Keras, by feeding a statistical model with examples. For the detection of cars in ortho aerial photos with Keras, we carried out a Proof of Concept (POC) in collaboration with Stedin to demonstrate the possibilities.
OBJECT RECOGNITION IN AERIAL PHOTOS
Artificial Intelligence (AI) is becoming increasingly important in the Geo and GIS domain. Due to the large amount of available data, high-quality data analysis is becoming increasingly important. In addition, we see that almost all information has a location component. The technology enables us to use Machine Learning (ML) and AI to identify objects in aerial photographs based on statistical models
Above-ground infrastructure is placed in such a way that the chance of collisions is minimal. This means that this infrastructure is protected against collisions by installing posts or a guardrail. In many cases the infrastructure can also be placed out of reach of the traffic, behind bushes or in a green area. However, the situation at the time of construction does not always remain the same — for example, bushes can be removed and in their place a parking structure can be built. This increases the chance of collisions, but these situations are not always announced. The automatic search for these changes is therefore preferred
Dataset en Data processing
The dataset is comprised of aerial photos from the country-wide PDOK Web Map Service. The starting point of machine learning is that a training dataset with examples is available. ArcGIS Pro makes it easy to make the data from the aerial photos available with minimal manual actions, with the help of the "Export Training Data for Deep Learning" tool. The resulting labeled photos (with a size of 28 * 28 * 3 — height * length * RGB), are recognized by the computer, which knows whether a car can be seen or not. With this we can get started with Deep Learning.
Deep Learning photo Classification with Keras CNN
With a simple Convolutional Neural Network (CNN) it is possible to recognize objects in the aerial photos. Keras, a Deep Learning library in Python (based on Tensorflow), is the framework with which we developed this CNN.
The result of the model is that it recognizes cars with an accuracy of around 96 percent on the test set. The following plot shows tiles from the test data, where our model recognized the car:
With high accuracy, there are still situations where the incorrect predictions are made. In the two images below from the test set, cars are labeled, but the model does not recognize them. By further improving the model, this type of error (false positives and false negatives) can be prevented more often:
The good results of this POC show how powerful these types of models are. The bounding box that is currently used depends on the segmented photos. It does not yet detect exactly which pixels represent a car, but only whether a car can be seen in the photo. With the help of the YOLO algorithm it is possible to get a more targeted outcome. Furthermore, there are many other practical applications for this technology, such as the detection of solar panels or other types of objects.
Geospatial domain knowledge is increasingly required on data science teams. The power of Geo in combination with the latest developments in ML and AI creates many opportunities to get even more insights from data.
Do you want to know more about how this model is built? Do you need support with geospatial data in your team of data scientists? Contact us and let us know how we can help your company thrive in this new environment.