Added, Updated or Deleted? Track changes to your data in ArcGIS Online

ArcGIS Online provides a "Keep track of changes to the data" functionality, allowing users to monitor and manage changes made to their spatial and nonspatial attributes hosted on the platform. This feature is particularly useful in scenarios where multiple users or teams are collaborating on a dataset. In this blog article, I will showcase how you can access the recent transactions (up to 180 days) that have occurred on your hosted feature service.

Have you or a colleague ever accidentally deleted a feature or modified an attribute by mistake? As a Quality Assurance/Quality Control specialist, have you ever noticed any inconsistencies in your data model and wondered why a certain feature disappeared or why contingent attribute values aren’t lining up? While user-managed Enterprise Geodatabases allow for versioning, thereby enabling multiple users to edit data simultaneously without causing conflicting copies prior to all transactions being accepted by a database administrator, in the ArcGIS Online datastore, it’s a case of "last edit wins." Even with extremely vigorous data design rules, user error can still occur. Having spoken with numerous clients, very few are aware of the Keep track of changes to the data (add, update, delete features) or if they are aware, few know how to access the information related to those changes.  Firstly, the enable editing and the keep track of changes to the data parameter needs to be activated in the hosted feature layers properties. 

In the next sections, I will showcase how you can access the recent (180 days maximum) transactions that have occurred on your hosted feature service.

Please note: Applying data design principles in a file geodatabase and publishing to ArcGIS Online from ArcGIS Pro remains the optimal way to reducing user input error and maintaining the integrity of your data model. ArcGIS Online is becoming increasingly compatible with data design rules published from ArcGIS Pro and can greatly benefit your organizations editing workflows. For more information on data design principles in ArcGIS Pro, see this storymap.

Once track changes to the data has been enabled, go to your hosted feature service’s ArcGIS REST Services Directory. 

Once in the ArcGIS Rest Services Directory, click on the JSON hyperlink in the top left corner:

After clicking the JSON hyperlink, use “command f” to find changeTrackingInfo element in the JSON. You will need to copy the following information for each of the layers within your hosted feature service that you would want to gather track change information. In my example, my service only contains one layer ([0]):

[{"id":0, "minServerGen":3524436, "ServerGen" :352446}]

Return to the main ArcGIS Rest Services Directory page and scroll down to where you see "ExtractChanges":

Once in the Extract Change page of the ArcGIS Rest Services directory you need to enter the layers you wish to track edits (0,1,2,3…etc.) and paste the layerServerGens information into the proper box. For more details on the significance of minServerGen and ServerGen, see this Esri Technical Support article. Feel free to test the various parameters below to only retain the information you wish to keep.

Once you have completed filling out the necessary inputs, click on the Extract Changes button at the bottom of your page. This will lead you to a new page where the request is being processed. Once the process is complete, you will receive a hyperlink towards a JSON file with all the transactions which have occurred on your requested hosted feature layers.

Note:  The JSON file returned is not ESRI JSON format and therefore will require some form of Extract, transform, and load (ETL) if you wish to convert this into a feature class or hosted feature.  

Processing may take some time but once you’ve downloaded your JSON file, open it in a code editor such as notepad++ or Visual Studio Code (Hint: In Visual Studio Code, use Command+Shift+L and type "Format JSON" to format your JSON file). Depending on the parameters you selected when extracting the changes, you will be able to see the Adds, Updates and Deletes that have occurred within the hosted feature layers requested. 

Closing remarks

As previously mentioned, a robust data model incorporating data design principles such as subtypes, domains, contingent values and attribute rules is your greatest chance of limiting user-input error. Data governance, as explained by my colleague Matthew Lewin’s blog article further enhances an organization’s ability to trust their data by limiting each user’s ability to wrongfully edit and or interact with sensitive data by limiting their access to the essentials needed to complete their tasks at hand. Still, mistakes do happen and maintaining a system of record for each transaction that has occurred within your dataset can be one of the tools your organization uses to ensure data integrity. Please note that track changes does consume storage credits and trimming the log can help reduce credit consumption. With that in mind, who did what where when? Now you know!

This post was written by Tristan Gingras-Hill and can be viewed in French here.