SIMULATORS & TRAINING
Remote control operation of Komatsu mining shovel at its Arizona Proving Grounds from MINExpo 2021 in Las Vegas , over 400 miles away . Immersive Technologies ' development operator assistance tools relating to areas like depth perception for simulator training are also proving important in making productive teleoperated mining equipment a reality
A new training and operator reality
Simulator-based training gets ever more advanced and is playing an important role in helping operators and controllers prepare to manage and operate increasingly digitised and autonomous equipment in mining , reports Paul Moore
Global player in advanced simulator-based training solutions , Immersive Technologies , occupies a unique position in advanced simulator training technology . While it is now a part of Komatsu Ltd , it continues to be OEMagnostic and says its main focus continues to be serving its customers and addressing their needs to meet market challenges . It is also true that many customers also operate mixed fleets , so that is another reason for Immersive to continue to provide solutions for a range of OEMs . Ravitha Sukumaran , Group Product Manager ( Autonomous Solutions and Data & Analytics ) told IM that Komatsu is very supportive of it continuing to do that and puts no restrictions on the company in any way .
But of course being part of Komatsu also comes with great opportunities , especially those helping to take the industry forward . At MINExpo 2021 , the teleoperation of a Komatsu PC7000-11 shovel was carried out live , with the operator in Las Vegas and the shovel at Komatsu ’ s Arizona Proving Grounds over 400 miles way . This was possible through leveraging an Immersive Technologies console that provides the user information and views of what is seen from the machine ’ s actual cab onsite . Multiple cameras and sensors around the machine and bucket , as well as a 360-degree monitor and machine display , support visibility and situational awareness for the remote operator . Sukumaran adds : “ This was a great collaborative synergy project between us and Komatsu – who were able to utilise some of our strengths and technologies to make it possible . We were able to apply the vast learnings we have built up on the human machine interface side of things over more than 29 years of simulator technology ; and we were able to put that to good use as part of that collaboration .”
Immersive had already applied camera views as part of simulator projects so had a lot of knowledge in that area which was used in designing the console , to the associated software and adding assistance . “ When you are operating remotely from the machine , looking at the screens , some things are missing , and in particular depth perception . So how do you compensate for that ? We had already solved these problems in simulation so this experience was put to good use . We also added things not usually possible in real operation to assist the operator – virtual assistance tools to help them be more productive .”
Examples given by Sukumaran include augmented reality to indicate to the operator when the next truck is coming and to visualise that in an easy way . Immersive is also looking at colour coding the material so they can see if they are digging waste or ore according to the block model . “ These , plus other things we are developing give Komatsu ’ s teleoperation system a real competitive advantage when you compare it to those other OEMs are developing .”
Of course , this has moved Immersive from a world of simulated training into real teleoperation – and this may yet be an evolving part of its business going forward as shovel teleoperation becomes commercialised in mining . However , for now the teleoperation project is still firmly in Komatsu ’ s stable but “ powered by Immersive Technologies .”
Teleoperation aside , guided spotting is another operator assistance technology where Immersive has had important involvement . Modular Mining , also a Komatsu brand , offers the ProVision Guided Spotting system , which is a high-precision machine guidance tool based on high precision GPS , which does what it says – guides haul truck operators to the correct loading position while the shovel continues productive work . Working with Modular Mining , Immersive developed support for ProVision Guided Spotting for Immersive Technologies ’ range of haul truck simulator Conversion Kits . This simulation was developed to accelerate the deployment of Guided Spotting system in mining without impacting production . Immersive Technologies ’ full range of Haul Truck simulator Conversion Kits can also be configured to support Guided Spotting via a software update .
“ The major customers for Guided Spotting wanted a training solution in place and together with Modular we were able to deliver this in another synergy project . With Modular being a sister company , we were able to get a lot of the technical information we needed from them these very easily .”
Fleet management perhaps unexpectedly is another market for simulator training . Interestingly , Immersive since 2021 has been able to include Modular Mining ’ s DISPATCH fleet management system elements into its simulator environment . Sukumaran states : “ Here the need came directly from the customer . We are not talking about the back-end elements of DISPATCH – it relates more to a panel that is on the trucks . To some extent the DISPATCH system still relies on some data supplied by the operators – and good data makes for better analysis . As soon as the operator stops their shift and before they get out of the truck , they are meant to log out of the DISPATCH system . Sometimes operators forget to do it , then the next operator doesn ’ t log in . So the shift is recorded under the wrong operator . This also means any issues that might require further training are attributed to the wrong person . The DISPATCH system works differently at different sites , some of the latest are more automated . With shovels for example , these automatically record the first bucket loaded from a shovel and when the truck leaves fully loaded . But some sites still require the truck operator to press a button confirming the first bucket has been loaded . If this isn ’ t done then the spotting times
24 International Mining | NOVEMBER / DECEMBER 2022