Robust Detection of Marine Life with Label-free Image Feature Learning and Probability Calibration

Schanz, Tobias¹ , Möller, K. O.¹ , Rühl, S.¹ , Greenberg, D. S.¹

Advances in imaging technology for in situ observation of marine life has significantly increased the size and quality of available datasets, but methods for automatic image analysis have not kept pace with these advances. On the other hand, knowing about distributions of different species of plankton for example would help us to better understand their lifecycles, interactions with each other or the influence of environmental changes on different species. While machine learning methods have proven useful in solving and automating many image processing tasks, three major challenges currently limit their effectiveness in practice. First, expert-labeled training data is difficult to obtain in practice, requiring high time investment whenever the marine species, imaging technology or environmental conditions change. Second, overconfidence in learned models often prevents efficient allocation of human time. Third, human experts can exhibit considerable disagreement in categorizing images, resulting in noisy labels for training. To overcome these obstacles, we combine recent developments in self-supervised feature learning based with temperature scaling and divergence-based loss functions. We show how these techniques can reduce the required amount of labeled data by ~100-fold, reduce overconfidence, cope with disagreement among experts and improve the efficiency of human-machine interactions. Compared to existing methods, these techniques result in an overall 2 % to 5 % accuracy increase, or a more than 100-fold decrease in the human-hours required to guarantee semiautomated outputs at the same accuracy level as fully supervised approaches. We demonstrate our results by using two different plankton image datasets collected from underwater imaging systems at the coast of Helgoland and from a research vessel cruise in front of Kap Verde.