Application-Aware Image Compression and Sensing Platform for Plant Phenotyping

Plant phenotyping investigates how a plant’s genome, interacting with the environment, affects the phenome (i.e. the observable traits of a plant). Quantitative assessment of phenotypes is central to our quest towards efficient and sustainable agriculture. Image-based approaches to plant phenotyping are gaining momentum and, on par with growing scientific and commercial interest, exciting computer vision problems arise. Currently available solutions for image-based plant phenotyping are either destructive and low-throughput or high-throughput and costly. We propose an affordable solution based on a distributed sensing and analysis framework. Time-lapse sequences of the scene are acquired by affordable sensors (as such, they will have limited computational power and knowledge access). The images are transmitted to the cloud, where high computational resources permit the extraction of fine-grained phenotypic information. For the automated analysis of such images, we develop a multi-channel active contour segmentation with probabilistic priors on plant appearance. To validate our approach we collect two image datasets of growing Arabidopsis plants, portions of which are manually annotated and publicly released. However, the transmission of large volumes of image data necessitates compression to meet bandwidth constraints. After demonstrating that lossy image compression does affect vision-based measurement of plant traits and can jeopardize phenotypic analyses, we investigate application-aware compression strategies on resource-constrained devices to reduce transmission and storage cost of the acquired images without compromising analysis accuracy. The possibility of sharing information between sensor and receiver is exploited: the receiver feeds back to the sensor information to optimize image compression. We inject application knowledge at different levels of the lossy encoding process. The sensor estimates regions of interest within an image and applies different levels of compression to foreground (plants) and background. We also save bits in color representation, using an orthogonal transform with class separation capabilities obtained with supervised learning. Finally, we investigate application-aware distortion metrics for pixel-level classification accuracy, and their implementation in the rate control algorithm of the High Efficiency Video Coding (HEVC) standard. We hope with such an affordable solution to increase adoption of image-based approaches to plant phenotyping by small labs and breeders, and also in developing countries, in pursuance of the democratization of science and technology.