Eider Duck counting

Counting Eider ducks from aerial imagery

Highlights

  • Finetuned the previously developed Openwildlife model on the Eider duck partial annotations
  • Developed an end-to-end deep learning-assisted operational pipeline to streamline manual annotation
  • Built a Flask-based machine learning backend service to support model prediction and training within the annotation interface
  • Containerized the complete pipeline (frontend + backend) using Docker and deployed on Google Cloud Platform for scalability
  • Reduced annotation time per image from 60 minutes to 15 minutes, significantly lowering operational costs

Data

The Eider duck dataset consists of aerial images captured via drones over the arctic region. The image depicts a large flock of Eider ducks over polynyas, which are areas of open water surrounded by sea ice. Detecting and counting these birds is crucial for understanding their population dynamics as innuits rely on them for food. The dataset is challenging due to the very close proximity of the birds, which makes it difficult to distinguish between them. Moreover, all the annotations are partial, meaning that only a subset of the birds in the image are labeled, which makes it difficult to train a model.

Image 1
Image 2
Image 3
Image 4

These images show the complexity of annotating and counting these ducks. Click on each image to zoom in and out. Partial annotations can be seen by clicking the blue button above.

End-to-End Annotation Pipeline

The annotation process for the Eider duck dataset was labor-intensive, requiring approximately 60 minutes per image. To fastrack this process, we developed an end-to-end deep learning-assisted operational pipeline.

The frontend of the pipeline is using Labelstudio, an open-source annotation tool that provides a user-friendly interface for annotators.

The backend is powered by a Flask-based machine learning service that integrates with the Labelstudio API. The backend service is responsible for model prediction and training, allowing the annotators to receive real-time feedback on their annotations with a click of a button.

Other features of the pipeline include:

  1. train region annotation - For images with partial annotations, the model is trained on the annotated regions only. This allows the model to learn from the available data without requiring complete annotations for every image.
  2. real-time model updates - The model is continuously updated with new annotations, allowing it to learn from the data as it is being annotated.
Example of Partial annotation

Example of Customised Labelstudio inteface with train region annotation. The annotator can select the region of interest and the model will be trained on that region only. This allows for faster annotation and better model performance.

Result

Original Image
Image 1
Model Predictions on Entire Image
Image 2
Partial Annotations on a region
Image 2

The first images shows the original image. The second image shows the partial annotation. The third image shows the model prediction on the entire image. Click on each image to zoom. The image in total contains 7000 ducks. Wow so many ducks!

Demo

Demo video of OpenWildlife detector in action

Deployment

Poetry is used to manage the dependencies of the project. The project is containerized using Docker, which allows for easy deployment and scalability. The complete pipeline (frontend + backend) is deployed on Google Cloud Platform (GCP) and is used by Arctic Eider Society to annotate images. This allows for easy scaling of the pipeline to handle large datasets and multiple annotators. The project will be soon open-sourced and made available to the public.