Best of Deep Learning in ArcGIS

September 21, 2023

Genevieve George

Follow this guide for a compilation of the best platforms for deep learning across the ArcGIS ecosystem

Deep learning is becoming easier to integrate into geospatial and imagery workflows than ever before and Esri offers a plethora of tools and applications to help users leverage that GeoAI technology.

Deep learning functionality can be accessed via the ArcGIS API for Python, geoprocessing tools in ArcGIS Pro, analysis tools in ArcGIS Online Map Viewer and Deep Learning Studio. With so many options, it can be difficult to determine when to use what and what advantages each platform offers. This blog is a guide to help you choose the best deep learning platform for your use case!

First off, let's take a look at the different options and their varying system requirements.

ArcGIS Pro - Deep Learning Geoprocessing Tools and Wizards

ArcGIS Pro is a desktop application that includes deep learning tools with the Image Analyst, Spatial Analyst, and 3D Analyst extensions. To perform deep learning in ArcGIS Pro you will need:

ArcGIS Pro Desktop license
Image Analyst, Spatial Analyst, OR 3D Analyst extensions (depending on which tools you want to use)
deep learning libraries (free to download)

For more information on the system requirements, including how to install the deep learning frameworks, check out this guide.

ArcGIS Online Map Viewer Analysis Tools

ArcGIS Online is a cloud-based software-as-a-service (SAAS) online platform that includes some deep learning tools in the Map Viewer. To perform deep learning in ArcGIS Online you will need:

An ArcGIS Online Organizational Account with a Creator or GIS Professional user type
ArcGIS Image for ArcGIS Online
Publisher, Facilitator or Administrator role, or an equivalent custom role
Modern desktop web browser

For more information on enabling access to the deep learning tools in ArcGIS Online, check out this blog post.

ArcGIS Enterprise – Deep Learning Studio

Deep Learning Studio is a web application available with ArcGIS Image Server for ArcGIS Enterprise. It is a comprehensive and collaborative application for model training and inferencing. To perform deep learning with Deep Learning Studio, you will need:

ArcGIS Enterprise 11.x
ArcGIS Image Server
ArcGIS Enterprise Publisher or Administrator role or equivalent custom role OR ArcGIS Enterprise Editor user type or equivalent custom role
Modern desktop web browser

For more information on the system requirements for ArcGIS Deep Learning Studio, check out this documentation.

ArcGIS API for Python

The ArcGIS API for Python is a Python library for scripting workflows across the ArcGIS suite including for GIS organization administration, content management, and spatial analysis/data science. Among its functionality is the arcgis.learn module, with various functions for manipulating data and training a deep learning model. The ArcGIS API for Python is free to install but most of its functionality requires it to be connected to an ArcGIS account (ArcGIS Developer/Platform account, ArcGIS Online account or ArcGIS Enterprise account). The type of account, user type and role you need depends on the resource you need to access and the operations you wish to undertake.

For more information on getting started with the ArcGIS API for Python, check out this documentation.

So which one should you use?
The best product for your deep learning workflow will depend entirely on what you want to accomplish and your expertise with various components of the ArcGIS product suite. To determine which product is best for your needs, we have compiled a list of the best deep learning products by their primary capability below.

Best Overall

ArcGIS Pro

ArcGIS Pro packages deep learning capabilities within a suite of geoprocessing tools and wizards. This familiar environment provides an intuitive workflow for existing ArcGIS Pro users while also providing ample user-friendly wizards for collecting training data and configurable parameters for tuning your models.

ArcGIS Pro has wizards and geoprocessing tools for all steps of the deep learning workflow, including the Label Objects for Deep Learning pane and Training Samples Manager for training sample collection, Train Deep Learning Model tool to train a model, and model-specific geoprocessing tools for model inferencing, including Detect Objects, Classify Objects, and Classify Pixels.

Screen capture animation of the Geoprocessing toolbox in ArcGIS Pro to find the Detect Objects Using Deep Learning tool, open the tool and input the tool parameters and run the tool.
The deep learning capabilities in ArcGIS Pro are available as geoprocessing tools.

There is plenty of documentation and tutorials available to walk through the whole process of using the deep learning tools in ArcGIS Pro, which makes it accessible for users that have never dabbled in deep learning before. Check out these ArcGIS Pro videos on installing the deep learning libraries, using a pre-trained deep learning model and creating and using a deep learning model in ArcGIS Pro.

Best for New Users

ArcGIS Online Map Viewer

Are you just looking to try out a pre-trained deep learning model from a colleague or the ArcGIS Living Atlas? The deep learning analysis tools in ArcGIS Online may be the best place for you to get started with deep learning in ArcGIS. The three analysis tools, Detect Objects Using Deep Learning, Classify Objects Using Deep Learning and Classify Pixels Using Deep Learning are easy to use and allow you to use input imagery and models from your organization or shared through ArcGIS Online and the ArcGIS Living Atlas.

The analysis tools in the ArcGIS Online Map Viewer allow you to use a pre-trained model to inference data but they do not provide the same model training capabilities as the other platforms. This makes ArcGIS Online a less powerful platform to leverage deep learning but also significantly more user friendly with a lower barrier to entry than the other options. If your aim is to use an existing model on data available in ArcGIS Online, then the deep learning analysis tools in the ArcGIS Online Map Viewer are the simplest and most intuitive way of doing that.

Screen capture of the ArcGIS Online Map Viewer with a satellite image of Guelph and the analysis toolbar open to a search of deep learning tools which yields three results: Classify Objects Using Deep Learning, Classify Pixels Using Deep Learning, and Detect Objects Using Deep Learning.
The ArcGIS Online Map Viewer has three analysis tools for using a deep learning model: Classify Objects Using Deep Learning, Classify Pixels Using Deep Learning, and Detect Objects Using Deep Learning.

Best for Collaboration

ArcGIS Deep Learning Studio

Deep Learning Studio includes multiple tools to manage the workload of building and using a deep learning model across multiple users. The first step in using DL Studio is to create a project and then you can set the data source, set the scheme for training samples, invite project members, and set up work units. The primary advantage of Deep Learning Studio is that it enables collaboration by creating project members and dividing the project into discrete work units.

Project Members

Project members can be chosen from groups in your ArcGIS Enterprise Organization. Contributor's capabilities are divided into three roles (project owner, analyst, sample collector) and are related to their user privileges in the ArcGIS Enterprise Organization e.g. anyone with editing privileges can have the sample collector role. See the table below for the tasks that can be performed by each role.

Screen capture of table with description of capabilities for Project Owner, Analyst, and Sample Collector role types in an ArcGIS Deep Learning Studio project from the Deep Learning Studio documentation. From https://doc.arcgis.com/en/deep-learning-studio/latest/essentials/workload-management.htm
Overview of the Deep Learning Studio capabilities for roles within a Deep Learning Studio project.

Work Units

Work units are sections of the imagery that are split up so that individual users can collect training data in an organized fashion, without overlap. There are three options for setting up work units:

Grid system: You can define the size of the grid that overlays the imagery so that each member works on one grid cell at a time.
Custom work units: You can use an existing polygon feature layer to define polygonal areas for each member to work on
Individual images: For imagery with multiple individual images, each member can work on one image at a time.

Screen capture of Deep Learning studio work unit configuration page with options for Grid System, Custom work units and Individual images. You can choose between three options to split training sample collection into multiple work units.

These two features allow project members to collaborate on collecting and reviewing training samples. Once an area is marked complete, project analysts and owners can go in and review the work.

Can you still collaborate with other platforms? Sure! As with other analysis workflows, data can always be shared at intermediate steps. For example, if you wanted to have multiple users collect training data, then you could do that on separate instances of Pro and merge the different feature classes together to create one large training dataset. However, there is no built-in way of performing this collaboration except in Deep Learning Studio, which makes it an ideal platform to perform deep learning when there are multiple people involved.

For a detailed overview of how to collaborate with organization members on a deep learning project in ArcGIS Deep Learning Studio, check out this video.

Best for Model Training/Python Users

ArcGIS API for Python

Understanding python unlocks new layers in any GIS workflow and deep learning is no exception.

The ArcGIS API for Python currently offers flexibility in model training that is unique in the ArcGIS ecosystem. These tools are only available in Python, so with no user interface you should be pretty comfortable in Python to use them successfully. The ArcGIS API for Python arcgis.learn module offers options to incorporate external models into the ArcGIS workflow, as well as more options to tune hyperparameters and augment your data. Using the ArcGIS API for Python library allows users to leverage some of the best open-source deep learning libraries like TensorFlow and Keras seamlessly integrated into the ArcGIS ecosystem.

Another benefit of using the ArcGIS API for Python is that it is designed for web hosted features so you can work with hosted imagery and feature classes.

However, with no user interface, collecting training data must be done outside of the python workflow. You can use the tools in ArcGIS Pro or other tools to collect training data as a feature class, and then use the export_training_data function to convert the training data to the right format and continue the work in Python.

Are there other options for programmers without using the ArcGIS API for Python? Yes! All of the ArcGIS Pro geoprocessing tools, including the deep learning tools, are available with the ArcPy library. You can still train your model with ArcPy, but it offers less flexibility and fewer advanced options than the ArcGIS API for Python. Python users can also combine the ArcPy and ArcGIS API for Python libraries with other open-source libraries to maximize flexibility and leverage the benefits of each library.

Summary

Deep learning is available in different formats across the ArcGIS ecosystem, making it increasingly accessible for users of varying skillsets. Whether you're interested in using ArcGIS Online to test out a pre-trained model or using the ArcGIS API for Python to create a custom model there is an ArcGIS option for you! Whichever platform you choose, ArcGIS has the tools to help you become a deep learning whiz.