Platforms for Making Deep Learning Easier for GIS

Mark Altaweel


While so-called ‘deep learning’ methods, that is machine learning techniques that use layered, artificial neural networks, to perform learning of unstructured data, has gained great popularity in research in the last few years, its integration within GIS is not always easy without specialist knowledge. New tools, however, are being and have been developed that potentially will allow a wider field of researchers to better take advantage of machine learning and deep learning specifically. So far, two clear leaders have emerged.

Deep Learning with ArcGIS

ArcGIS has spent significantly in recent years to develop better tools for deep learning. They have used a variety of platforms that have been developed for imagery analysis, such as Keras and Tensorflow which are popular deep learning platforms, and have integrated them with ArcGIS Pro tools, such as ArcGIS Image Server. Most work has focused on image classification. ArcGIS has built some ready to apply modules, such as Regularize Building Footprints, that allow mapping of buildings to be done quickly. Tools allow automated, deep learning driven classification capabilities for mapping features in the built environment, although some projects have included counting individuals shown in imagery to estimate crowd sizes and other landscape features.

Some interesting applications have included automatically detecting and counting swimming pools in given urban environments. Swimming pools could be breeding grounds for disease carrying insects if housing becomes abandoned. Estimates of swimming pools could be used to estimate disease potential in a given area.[1] Other application focus on broader segmentation and classification of imagery. Semantic segmentation, which classifies pixels in images, are also part of the package, with users given a Python interface to shape or create their own deep learning technique and mapping. Nevertheless, most of the efforts have been focused on computer vision techniques that allow image analysis and classification, particularly for aerial imagery.[2]

ArcGIS Pro tool for labeling and exporting training data for deep learning.  Image: Jha & Singh, 2018.
ArcGIS Pro tool for labeling and exporting training data for deep learning. Image: Jha & Singh, 2018.

Integrating Open Source Deep Learning Tools With QGIS

One popular open source tool is Orfeo, which is a tool that analyses high resolution optical, multi-spectral and radar images up to terabyte scales. The project was initially developed by the French space agency, CNES, but has since incorporated many developers in an open source format. The tool can be used along with and called from popular GIS applications such as QGIS, making it one of the better open source options out there for deep learning that requires relatively little programming experience. Once again, the focus is on imagery, although techniques could potentially be modified for analyzing other data forms. The tool is also closely associated with Monteverdi, which applies Orfeo’s deep learning methods in visualization. Monteverdi can be used to visualize deep learning output such as classification of features in remote sensing data. The tool also has a QGIS interface, enabling its integration directly with this popular open source option.[3]

Free weekly newsletter

Fill out your e-mail address to receive our newsletter!

Using the Orfeo toolbox within QGIS for deep learning.  Image: GeoFranker
Using the Orfeo toolbox within QGIS for deep learning. Image: GeoFranker

Deep Learning Tools for GIS

Other tools that have been used to support open source projects include GRASS and the R statistical package. Both tools not only allow general and varied forms of spatial analysis, but they also facilitate different tool integration.[4] However, tools such as Orfeo, on the open source side, and ArcGIS Pro’s suit of AI tools, have gained more traction due to the scalable architecture deployed that allows them also to take advantage of cloud computing in handling large image data. While many tools can potentially be integrated into GIS tools, particularly open source ones, the limitation is often in scalability, where researchers also need to have access to powerful GPU processors or large computing clusters, which are not always available.

While Esri and open source tools such as Orfeo as increasingly making deep learning more tangible for users, areas of application still remain relatively limited. In particular, integrating text data, such as natural language, along with imagery and other spatial data have not been applied to a great extent. Furthermore, there is also a realization that the Internet of Things (IoT) will mean that multi-sensor data will increasingly need to be integrated into analytical platforms such as ArcGIS Pro. This will perhaps be the next great challenge for both proprietary GIS as well as open source tools.


[1]    For more on the example of detecting swimming pools as possible disease breeding ground using deep learning, see:

[2]    For more on deep learning integrated with ArcGIS, see:

[3]    For more on the Orfeo toolbox, see:

[4]    For more on GRASS and R integration with deep learning, see:  Grippa, T., Lennert, M., Beaumont, B., Vanhuysse, S., Stephenne, N., Wolff, E., 2017. An Open-Source Semi-Automated Processing Chain for Urban Object-Based Classification. Remote Sensing 9, 358.


Photo of author
About the author
Mark Altaweel
Mark Altaweel is a Reader in Near Eastern Archaeology at the Institute of Archaeology, University College London, having held previous appointments and joint appointments at the University of Chicago, University of Alaska, and Argonne National Laboratory. Mark has an undergraduate degree in Anthropology and Masters and PhD degrees from the University of Chicago’s Department of Near Eastern Languages and Civilizations.