Fruit size estimation using Multitask Deep Neural Networks

Author: Mar Ferrer Ferrer

Virtual Room:

Date & time: 22/09/2021 – 9:30 h

Session name: Agricultural Applications

Supervisors: Javier Ruiz Hidalgo, Jordi Gené Mola


The measurement of fruit size is of great interest to estimate the ripeness of the crop and predict the harvest resources in advance. Non-invasive estimation of fruit size remains a challenging task that has to deal with occlusions, which may be caused by the foliage or shadows. This work proposes a novel technique for in-field apple measurement based on Deep Neural Networks (DNNs). The proposed framework has been trained with RGB-D data and consists of an end-to-end multitask architecture that performs the following tasks: 1) detect and segment every fruit from its surroundings; 2) estimate the diameter of each of the detected fruits. The network has been trained to perform instance segmentation and amodal segmentation, which when combined, allow us to see the relation between the occlusion percentage of the apple and the error of the diameter estimation. Our presented model is based on the Mask R-CNN architecture, which was extended in order to achieve all of the required tasks. This methodology was tested with a total of 2491 apples at different stages of growth, with diameters varying from 27 mm to 95 mm. We obtained a F1-score for instance segmentation of 0.78 and a mean absolute error of the diameter estimation of 6.8 mm. These state-of-the-art results show the potential of Deep Learning for fruit size estimation tasks.


– President: Angel Sappa(UAB)
– Secretary: Daniel Ponsa(UAB)
– Vocal: Ramon Baldrich Caselles(UAB)