FOSS4G-Asia 2023 Seoul

Automatic Cocoa fruit count using a mobile app and computer vision
11-30, 17:20–17:40 (Asia/Seoul), Seoul Archive

Cocoa is a major global commodity, and its production involves millions of mostly smallholder farmers in developing countries. Efficient and accurate counting of cocoa fruits during harvest is essential for effective resource management, yield estimation, and overall productivity improvement. Traditional manual counting methods are time-consuming, labor-intensive, and susceptible to errors, necessitating the exploration of innovative technological solutions.

This research introduces a novel approach to automatic cocoa fruit counting using a mobile application integrated with computer vision techniques. The proposed system leverages four services: 1) acquisition of farmers monitoring data using the mobile devices, 2) develop explainable AI model for automatic cocoa fruit recognition, 3) presentation of Cocoa pod detection and counting services through a web & mobile based application and 4) collect the farmers feedback on cocoa pod count to improve the services. To do this, Mobile app for data collection was co-designed and tested with local farmers in Cameroon. This app allows farmers to take geo-tagged photographs and transfer them to the server when connectivity permits. In total, 2132 Cocoa tree photos were manually annotated and different artificial intelligence (AI) models (e.g., faster R-CNN, darknet, ResNet-50, mobilenetv3) were developed and tested for automatic counting of Cocoa pods. The Faster R-CNN model with a ResNet-50 performed better and provides accuracy above 70%. The model was deployed on a web service and provides real-time prediction through a web- & Mobile based application

The insights gleaned from this research will spawn a new generation of tools for Cocoa farmers to use. It is an efficient, non-destructive, and low-cost method which offers useful information for them to plan agricultural work and obtain economic benefits from the correct administration of resources. Nonetheless, further research is needed to address potential limitations data collection strategies (specifically variable environmental conditions, complex Backgrounds for image collection), image Annotation challenges and ensure seamless integration into the existing agricultural workflow.

See also: presentation pdf format (4.1 MB)

Dr. Arun Pratihast is a Senior Data Scientist at Wageningen Environmental Research in the Netherlands. He specializes in utilizing data and technology for forest, biodiversity, and agriculture monitoring. His expertise includes citizen science, geoinformation technologies, mobile app development, open data, data standardization, and software engineering. Arun holds a PhD from Wageningen University for his research on "Interactive Community-Based Tropical Forest Monitoring Using Emerging Technologies."

Dr. Eliya Buyukkaya received her PhD in Computer Science from University Paris 6 in 2011. She worked as a R&D engineer in several EU research projects at Telecom Bretagne (Brest), ENSSAT (Lannion) and University Rennes 1 in France. From 2014 to 2020, she worked as an Assistant Professor at Kadir Has University (Istanbul). Since September 2020, she has been working as a Software Engineer in Earth Informatics team at Wageningen Environmental Research.

Jappe Franke, MSc. is a Researcher & Software Engineer at Wageningen Environmental Research in the Netherlands. He specialises in utilising data and technology for environmental and agriculture monitoring. His expertise includes unmanned aerial vehicles, geoinformation technologies, HPC development, open data, remote sensing imagery, and software engineering.

Scientific software engineer, specialized in geospatial software