INGENIEUR
INGENIEUR
Harvesting robots require 3D spatial position data of the selected fruit to guide the end effectors accurately during the harvesting procedure . However , the camera only records the 2D co-ordinates of the target in the image space . This requires establishing a mapping association between the target ’ s position in the 2D image space and its corresponding position in the 3D space [ 10 ]. For these purposes , 2D cameras equipped with either charge-coupled device ( CCD ) sensors or complementary metal oxide semiconductor ( CMOS ) sensors are often utilised [ 11 ]. To obtain 3D spatial position data from a single 2D camera , it is necessary to incorporate either an additional calculation model or an additional sensor . Parish and Goksel [ 3 ] used a single camera to determine the centre of mass of the fruit in the image . Subsequently , a mathematical transformation was performed to ascertain the association between the 2D coordinates of objects and a 3D co-ordinate system .
Colour is an essential feature used in machine vision systems to distinguish fruit from leaves , branches , and other objects in the background of an orchard environment during target detection . Multiple research [ 12 ] [ 13 ] [ 14 ] [ 15 ] [ 16 ] have utilised colour-based segmentation to detect fruits that have easily recognisable hues , such as tomatoes , red apples , peaches , mangoes , pineapples , and citrus fruits .
Various 3D acquisition and range-measuring techniques have been created . However , stereo vision , structure light , and time-of-flight ( TOF ) are the main principles used in industrial applications for 3D imaging [ 17 ]. Consequently , an examination is conducted on these specific methods of 3D data collection and measurement .
The tomato harvesting robot ( see Figure 4 ) was successfully developed by using a detection algorithm that utilises colour and geometric cues for item detection , as well as a localisation method that utilises three-dimensional spatial data from a 3D camera coupled with COBOT and a soft gripper . COBOT is a cutting-edge and contemporary phenomenon in the realm of robot advancement , specifically engineered to collaborate and operate in conjunction with humans . Since the harvesting procedure took place within the building , the researchers conducted tests on the structural light 3D imaging approach .
B
Figure 4 : Harvesting process with COBOT with 3D camera : ( A ) Joints and angles inverse kinematics calculation , ( B ) Tomato picking
Firstly , it achieves high accuracy when measuring target objects in close proximity . Secondly , it is capable of rapidly acquiring data . Thirdly , it has the capacity to measure a significant volume of data . Additionally , it involves less complicated computation compared to stereo vision and consumes less computational power compared to TOF . Lastly , the performance of this technique is dependent on ambient light , which can be effectively controlled in the close area of the greenhouse ( see Figure 4 ). Furthermore , a structured light-type 3D camera called Pickit 3D camera is available for purchase on the market . The harvesting procedure is fully automated , requiring no input or decision-making from the operator . As depicted in Figure 4 , the system possesses the capability to autonomously scan the tomato plant . The image processing is effectively executed to facilitate the recognition of the tomato by the object detection algorithm , which relies on colour and geometric characteristics . Subsequently , the structure light 3D imaging approach effectively localises the designated tomato . The 3D spatial data can be transmitted to the COBOT controller
A
46 VOL 99 JULY - SEPTEMBER 2024