Imperial Oil deploys drones for more efficient bitumen recovery

Based at their Kearl Oil Sands Site in Alberta, Imperial Oil’s unmanned aerial vehicle (UAV) pilot project demonstrates how to get daily mine face snapshots, in 3D, that are accurate down to the centimetre.

Imperial Oil is a large, integrated Canadian petroleum company affiliated with ExxonMobil. It owns assets across Canada, including the Kearl Oil Sands Site, which is one of Canada’s largest open-pit mining operations and which is situated in the Athabasca Oil Sands Region. Sherry Qing, a geomatics engineer on the Surface Geomatics and GIS Support Team at Imperial Oil, shares how her team used unmanned aerial vehicles (UAVs) and Esri software to digitally transform how site geologists perform geological interpretations  to ensure that mining happens in the most efficient way possible.

Imperial’s team transformed their existing, highly labour-intensive process, in which site geologists manually stitched photos together. This process could introduce up to 100 m of error. Using drones, they are now able to create easily shareable, high-resolution 2D imageries and 3D models of the mine face with centimetre-level accuracy, allowing geologists to better guide engineering and operations to maximize production.

Challenge

In the mine pit at the Kearl Oil Sands Site (also known as “Kearl”), the oil sands are excavated by large shovels and transported by trucks for further extraction and treatment in Imperial Oil’s plants. The amount of bitumen that can be recovered depends on the work of site geologists, who evaluate how ore grade and embedded waste seams vary within the ore body from the exposed surface of the mine—known as the facies—for priority areas every day. The more accurate the site geologists’ interpretations are, the higher operational efficiency can be.

Developing these geological interpretations and models involves a lot of data gathering and processing effort. Site geologists drive out to the mine pit every day with handheld cameras. They then go into the pit and take photos of the mine face from at least 50 m away as a safety precaution. Once they’ve taken the photos, the geologists upload the images to their computers, then manually stitch the photos together using image editing software to get a composite image of the mine face. They crop out operations vehicles and visible sky, for example, and try to minimize the distortions inevitably introduced by taking photos from a single point perspective and from a low angle.

Although the photos are taken by professionals, there’s always a hazard of the mine face being obscured in the photos because they need to be taken from inside an active operations area. Equipment, vehicles, shadow and dust are always present in an environment like this. Not only that, but site geologists can only take photos within a short timeframe due to safety concerns associated with them being in the mine pit.

After editing the photos, the site geologists manually georeference the composites based on weekly as-built data and the approximate locations at which they took the photos. Because the photos can be obscured or distorted in so many ways, inaccuracies in georeferencing are inevitably introduced. Normally these errors are in the tens of metres, but sometimes up to 100 m of error can creep in.

An example of the type of panoramic composite image developed by site geologists at Imperial. These images can take many hours to produce, factoring in travel to the site to take photos from ground level, photo editing and manually georeferencing the photos.

This highly manual workflow—taking, editing and georeferencing the photos—takes many hours, with some batches of photos taking more time than others to process, depending on the quality of the data captured and the experience of the geologist.

Once processed, the site geologists use the composite photos to make geological interpretations, build models and make day-to-day decisions about mining operations. They’re also used to support longer-term planning and prediction. The more accurate the composites, the more efficient the mining—and the less environmentally impactful, given that efficient mining results in fewer tailings. Of course, the other side of the coin is that inaccurate photos negatively affect efficiency.

Having seen an opportunity to transform the way that site geologists capture the mine face and make interpretations, Sherry worked with key individuals from Imperial Oil’s Kearl geoscience team, its Heavy Oil Mining Research group, contacts on site and third party contractors to test the use of UAVs for taking photos of the mine face. This had never been done before at Imperial.

Solution

Sherry first joined the UAV initiative in the fall of 2018 and supported multiple trials in 2019. The goal was to develop an efficient, robust workflow that used new technology to reduce post-processing effort. The team also wanted to be able to visualize the mine face in both 2D and 3D.

To provide a basis for the pilot project, a contractor team performed test flights with UAVs. Sherry then processed the collected data using the ortho mapping tool in ArcGIS Pro. She immediately saw a huge improvement in data quality compared to the previous workflow. Not only that, but also, having better access to areas that were otherwise difficult to enter on foot due to safety concerns meant potentially huge safety improvements for site geologists. The pilot proceeded.

Of course, magic doesn’t happen overnight. To get the team and contractors prepared for further test flights, safety training and site orientations were required. This ensured that Imperial’s six test flights, conducted from July to September 2019, met all safety requirements. Internal employees were also trained on drone operation so that the work could be brought in-house.

The pilot workflow always begins with a drone flight. Imperial plans to direct the UAV to acquire mine face data and process it to generate high-resolution, georeferenced outputs in both 2D and 3D. Thanks to its ability to shoot the same mine face from multiple camera angles, the UAV can “see around” active mining equipment, unlike a site geologist taking photos at ground level from a single fixed point.

Originally, Sherry used the ArcGIS Pro ortho mapping tool to process the drone data. She then used the ArcGIS Pro tasks tool to automate the repeatable elements of that workflow. The streamlined data processing and refinement procedures ensured that no steps were missed or performed inconsistently. The processed data could then be shared to ArcGIS Online in the form of web scenes, allowing Sherry to show the data to a larger number of users. They could then visualize and analyze the data without the need to install ArcGIS Pro on their computers.

At first, Sherry’s revised workflow used a combination of ArcGIS Pro ortho mapping and ArcGIS Pro tasks. Here, the ortho mapping workspace maps out the flight path and image locations.

Sherry created a palette step to store a variety of tools, like Compute Tie Points, Manage GCPs and so on. “The palette step gives me freedom to pick and choose which tools to run in any order I want, allowing me to further refine and process the data until I’m satisfied with the results,” she says.

This was already a huge improvement over the previous process, which had offered countless opportunities for human error to be introduced. But Imperial’s team wanted to go further. With the ortho mapping workflow, they were creating 3D models by essentially draping 2D imagery on top of elevation data, which sometimes caused the mine face imagery to lose clarity. But what they really wanted was to directly generate 3D products like point clouds and 3D mesh of terrains.

That’s how Sherry came to ArcGIS Drone2Map, which the pilot project is now testing to process and visualize mine face data. Drone2Map better accommodates 3D workflows and allows the team to generate 3D products directly. It also has a user-friendly interface that’s approachable for non-experts. And its tight integration with the ArcGIS system allows Sherry to easily share data and content. The team is able to take Drone2Map data into ArcGIS Pro to perform analysis and geological interpretations, then publish their high-resolution 3D GIS models directly to ArcGIS Online.

“I was really excited to be part of this project. We moved to Drone2Map because it offers a huge improvement to our accuracy and results. It’s tightly integrated to the rest of the ArcGIS system, and it’s genius because of all the stuff we can do: publishing 3D models as web scenes, performing 3D interpolations to pick up all the mine face features… overall, I find the system is really convenient and practical.” —Sherry Qing, Geomatics Engineer, Imperial Oil

Using Drone2Map, Imperial’s team is able to pan, rotate and zoom in to the details of the 3D mesh that represents the mine face. This can then be shared to ArcGIS Online, allowing for faster sharing and collaboration. Whereas before they could run into errors of up to 100 metres, Imperial’s pilot workflow is able to achieve up to centimetre-level accuracy.

Although there’s still time spent in data processing, the software does the work, freeing Imperial’s staff to do other tasks while they wait. The several hours that site geologists used to need to work to stitch together and process photos have been reallocated. Not only that, but the site geologists no longer need to drive into the mine pit to take photos: instead, they can direct the UAVs into position while keeping a safe distance from active operations areas. UAVs can even get to places that vehicles can’t, meaning they have great potential for exploring areas that would be hard to access for safety reasons. And web scenes can be adjusted to show different times of day, too, so drone missions can be planned for times of optimal lighting, allowing UAVs to capture better imagery. So not only is Imperial saving time, but their workers are also benefiting from an added measure of safety, and the quality of the data they’re getting is much improved.

Using ArcGIS Online’s web scene feature, workers across Imperial’s organization can change the daylight levels at the Kearl mine to plan for the best time for their next drone flight.

Using this new workflow, decision makers at Imperial will be able to plan operations faster, easier and using better data than ever before—and those operations will be much more accurate, with a higher return on investment for every hour that the mining equipment is working. Imperial’s digital twin of the mine face, and therefore decision makers’ understanding of what’s happening in the field, will be easily and quickly updatable. They’ll be able to turn around decisions faster than ever before and base them on current data, making collaboration and sharing across Imperial’s groups easier, too. Before, the mine face composites that the site geologists were producing were sometimes a week out of date and dozens of metres off. With UAVs, they’ve been able to achieve up to centimetre-level accuracy.

The 3D meshes Imperial’s team produces in Drone2Map can be taken into ArcGIS Pro and shared to ArcGIS Online with the help of Scene Viewer. This way, people in other parts of the organization can interact with the visualizations, take measurements or analyze them without needing to have ArcGIS Pro installed.

Essentially, Drone2Map is being tested to develop the Kearl Oil Sands in the most efficient way possible. And why stop at mine face interpretations? The scope of the UAV pilot is already being expanded to evaluate the potential for UAVs to assist with streamlining other aspects of operations at Kearl, including site surveillance and vegetation monitoring.

“Together with the integration of Esri software, our workflow has been digitally transformed. These innovations demonstrate a huge uplift in efficiency for the bitumen recovery process,” says Sherry.