- Deutsches Klimarechenzentrum (DKRZ)
- Helmholtz Zentrum München
The European Space Agency (ESA) launched the AI4EO initiative to bridge the gap between the artificial intelligence (AI) and the Earth observation (EO) communities [1]. In the AI4FoodSecurity challenge [2], the goal is to identify crops in agricultural fields using time series remote sensing data from different satellites. In the first challenge track, predictions were made for a region in South Africa, including a spatial domain shift. In the second challenge track, predictions were made for a region in Germany (Brandenburg), including a spatio-temporal domain shift.
We here present our contribution to the AI4FoodSecurity challenge. As data sources we selected both radar wavelength images from the Sentinel-1 satellite, as well as visual and infrared images from the Planet Fusion Monitoring satellites. We implemented a Pixel-Set Encoder with Lightweight Temporal Attention (PseLTae) [3]. Samples are constructed by randomly selecting pixels from a given agricultural field. We train separate encoders for each data source. Attention heads are used to extract characteristic changes of the distinct crop types throughout the growing season. At the decoder stage, both sources are combined to yield a prediction for the crop type. We used data augmentation by oversampling the agricultural fields, as well as cross validation.
The quality of the predictions is evaluated by a multi-class classification score. We finished the challenge in second place in both tracks. We found the model performs better in the South African region, where only the spatial domain changed, compared to the German region that was evaluated in the next vegetation period. Our code is available on Github [4].
Figure: Colored map of crop predictions for the region in Germany, including nine crop types.
References:
[1] A.-K. Debien et al, 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, 251-253 (2021).
[2] AI4FoodSecurity Challenge: https://ai4eo.eu/ai4eo-ai4foodsecurity-challenge
[3] V. Sainte Fare Garnot et al, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 12325-12334 (2020)
[4] https://github.com/crlna16/ai4foodsecurity