plant phenotyping software

2021/05/21

In other words, the Otsu method identifies the value between two peaks where the variances of both classes are minimized. We encourage contribution to the project by posting bug reports and issues, developing or revising analysis methods, adding or updating unit tests, writing documentation, and posting ideas for new features. "Following" is like subscribing to any updates related to a publication. The get_nir function identifies the path of the NIR image that matches VIS image. Image processing pipelines, which process single images (possibly containing multiple plants), can be deployed over large image sets using PlantCV parallelization, which outputs an SQLite database of both measurements and image/experimental metadata. There does not need to be an object in each of the grid cells. Such semi/pseudo-landmarking strategies have been utilized in cases where traditional homologous landmark points are difficult to assign or poorly represent the features of object shape (Bookstein, 1997; Gunz, Mitteroecker & Bookstein, 2005; Gunz & Mitteroecker, 2013). In cases where the auto-threshold value does not adequately separate the target object from background, the threshold can be adjusted by modifying the stepwise input. Additions or revisions to the PlantCV code or documentation are submitted for review using pull requests via GitHub. Images of wheat (Triticum aestivum L.) infected with wheat stem rust (Puccinia graminis f. sp. Furthermore, to decentralize the computational resources needed for parallel processing and prepare for future integration with high-throughput computing resources that use file-in-file-out operations, results from PlantCV pipeline scripts (one per image) are now written out to temporary files that are aggregated by the parallelization tool after all image processing is complete. The following are details on improvements to the structure, usability, and functionality of PlantCV since the v1.0 release. Landmarks are generally geometric points located along the contours of a shape that correspond to homologous biological features that can be compared between subjects (Bookstein, 1991). For Type III landmarks, the x_axis_pseudolandmarks and y_axis_pseudolandmarks functions identify homologous points along a single dimension of an object (x-axis or y-axis) based on equidistant point locations within an object contour. Developments in primatology: progress and prospects, IEEE Transactions on Systems, Man, and Cybernetics, the R Foundation for Statistical Computing, Journal of Histochemistry and Cytochemistry, The PeerJ Bioinformatics Software Tools Collection, https://github.com/danforthcenter/plantcv-v2-paper, http://plantcv.danforthcenter.org/pages/data.html, http://plantcv.readthedocs.io/en/latest/analysis_approach/, https://www.python.org/dev/peps/pep-0008/, http://plantcv.readthedocs.io/en/latest/jupyter/, http://plantcv.readthedocs.io/en/latest/multi-plant_tutorial/, http://plantcv.readthedocs.io/en/latest/vis_nir_tutorial/, http://plantcv.readthedocs.io/en/latest/machine_learning_tutorial/, https://github.com/danforthcenter/plantcv, Naive Bayes pixel-level plant segmentation, 2016 IEEE western New York image and signal processing workshop (WNYISPW), Moderate to severe water limitation differentially affects the phenome and ionome of Arabidopsis, Combining semi-automated image analysis techniques with machine learning algorithms to accelerate large scale genetic studies, Morphometric tools for landmark data: geometry and biology, Measures for interoperability of phenotypic data: minimum information requirements and formatting, Statistical shape analysis: with applications in R, Notes on scientific computing for biomechanics and motor control, A quick guide for developing effective bioinformatics programming skills, A versatile phenotyping system and analytics platform reveals diverse temporal responses to water availability in Setaria, Lights, camera, action: high-throughput plant phenotyping is ready for a close-up, Time dependent genetic analysis links field and controlled environment phenotypes in the model C4 grass Setaria, On the encoding of arbitrary geometric configurations, Phenomicstechnologies to relieve the phenotyping bottleneck, Semilandmarks: a method for quantifying curves and surfaces, Modern morphometrics in physical anthropology, SciPy: open source scientific tools for Python, Learning OpenCV 3: computer vision in C++ with the OpenCV library, Jupyter Notebooksa publishing format for reproducible computational workflows, Positioning and power in academic publishing: players, agents and agendas: proceedings of the 20th international conference on electronic publishing, Euclidean distance geometry and applications, Image analysis in plant sciences: publish then perish, An online database for plant image analysis software tools, Data structures for statistical computing in python, Proceedings of the 9th Python in Science Conference, Image-based plant phenotyping with incremental learning and active contours, A threshold selection method from gray-level histograms, The quest for understanding phenotypic variation via integrated approaches in the field environment, Ten simple rules for taking advantage of Git and GitHub, Deep machine learning provides state-of-the-art performance in image-based plant phenotyping, Leaf segmentation in plant phenotyping: a collation study, Statistical shape and deformation analysis, NIH Image to ImageJ: 25 years of image analysis, Machine learning for high-throughput stress phenotyping in plants, R: a language and environment for statistical computing, RStudio: integrated development environment for R, Raspberry Pi powered imaging for plant phenotyping, Deep plant phenomics: a deep learning platform for complex plant phenotyping tasks, The NumPy array: a structure for efficient numerical computation, ggplot2: elegant graphics for data analysis, Automatic measurement of sister chromatid exchange frequency. A pipeline can be as long or as short as it needs to be, allowing for maximum flexibility for users using different imaging systems and analyzing features of seed, shoot, root, or other plant systems. As noted above for the two-class approach, it is important to adequately capture the variation in the image dataset for each class when generating the training text file to improve pixel classification. Here we document the structure of PlantCV v2 along with examples that demonstrate new functionality. When the angle score is calculated for each position along the length of a contour, clusters of acute points can be identified, which can be segmented out by applying an angle threshold. These sixty points located along each axis possess the properties of semi/pseudo-landmark points (an equal number of reference points that are approximately geometrically homologous between subjects to be compared) that approximate the contour and shape of the object (Fig. When specified a priori, landmarks should be assigned to provide adequate coverage of the shape morphology across a single dimensional plane (Bookstein, 1991). For PlantCV v2, the parallelization framework was completely rewritten in Python using a multiprocessing framework, and the use of Matplotlib was updated to mitigate the issues and processor constraints in v1.0. The full database schema is available on GitHub (see Materials and Methods) and in PlantCV documentation. For example, two classes of features in an image may be visually distinct but similar enough in color that simple thresholding is not sufficient to separate the two groups. Several updates to PlantCV v2 addressed the need to increase the flexibility of PlantCV to analyze data from other plant phenotyping systems. An example VIS/NIR dual pipeline to follow can be accessed online (http://plantcv.readthedocs.io/en/latest/vis_nir_tutorial/). If a specific area is not selected then the whole image is used. Jeffrey C. Berry, Leonardo Chavez, Andy Lin, Csar Lizrraga, Michael Miller, Eric Platon, Monica Tessman and Tony Sax contributed reagents/materials/analysis tools, reviewed drafts of the paper. 1B). (B) Geometrically homologous semi/pseudo-landmarks across both the, Correlation between plant area in pixels (px) detected using thresholding pipelines (. For this module to function correctly we assume that the size marker stays in frame, is unobstructed, and is relatively consistent in position throughout a dataset, though some movement is allowed as long as the marker remains within the defined marker region of interest. For example, if there are large fluctuations in light intensity throughout the day or plant color throughout the experiment, the training dataset should try to cover the range of variation. The latest version or a specific release of PlantCV can be cloned from GitHub. Once a satisfactory pipeline script is developed, the PlantCV parallelization script (plantcv-pipeline.py) can be used to deploy the pipeline across a large set of image data (Fig. If images are captured in a greenhouse, growth chamber, or other situation where light intensity is variable, image segmentation based on global thresholding of image intensity values can become variable. A tutorial of how to implement naive Bayes plant detection into an image processing pipeline is online (http://plantcv.readthedocs.io/en/latest/machine_learning_tutorial/). Future releases of PlantCV may incorporate additional strategies for detection and identification of plants, such as arrangement-independent K-means clustering approaches (Minervini, Abdelsamea & Tsaftaris, 2014). While creating multiple regions of interest (ROI) to demarcate each area containing an individual plant/target is an option, we developed two modules, cluster_contours and cluster_contours_split_img, that allow contours to be clustered and then parsed into multiple images without having to manually create multiple ROIs (Fig. The watershed segmentation function can be used to segment and estimate the number of objects in an image. The Otsu, mean, and Gaussian threshold functions in PlantCV are implemented using the OpenCV library (Bradski, 2000). PlantCV contributors are asked to follow the PEP8 Python style guide (https://www.python.org/dev/peps/pep-0008/). Pixel-level segmentation of images into two or more classes is not always straightforward using traditional image processing techniques. Because ofthe web-based interface and useful export options, Jupyter notebooks are also a convenient method of sharing pipelines with collaborators, or in publications, and teaching others to use PlantCV. Image blurring, while reducing detail, can help remove or reduce signal from background noise (e.g., edges in imaging cabinets), generally with minimal impact on larger structures of interest. Despite the abundance of software packages, long-term sustainability of individual projects may become an issue due to the lack of incentives for maintaining bioinformatics software developed in academia (Lobet, 2017). The multiclass naive Bayes approach requires a tab-delimited table for training where each column is a class (minimum two) and each cell is a comma-separated list of RGB pixel values from the column class. Additionally, the identification of landmark points should be repeatable and reliable across subjects while not altering their topological positions relative to other landmark positions (Bookstein, 1991). The pull request mechanism is essential to protect against merge conflicts, which are sections of code that have been edited by multiple users in potentially incompatible ways. The naive Bayes classifier can be trained using two different approaches for two-class or multiclass (two or more) segmentation problems. All approaches for improving crops eventually require measurement of traits (phenotyping) (Fahlgren, Gehan & Baxter, 2015). The Plant Image Analysis database currently lists over 150 tools that can be used for plant phenotyping (http://www.plant-image-analysis.org/; Lobet, Draye & Prilleux, 2013). The cluster_contours function takes as input: an image, the contours that need to be clustered, a number of rows, and a number of columns. Finally, the use of a permissive, open-source license (MIT) allows PlantCV to be used, reused, or repurposed with limited restrictions, for both academic and proprietary applications.

Scripts used for image and statistical analysis are available on GitHub at https://github.com/danforthcenter/plantcv-v2-paper. 3). Each function has a debugging option to allow users to view and evaluate the output of a single step and adjust parameters as necessary. For the three example images, the watershed segmentation function was used to estimate the number of leaves for, (A) Automatic identification of leaf tip landmarks using the acute and acute_vertex functions (blue dots). Multiple methods for leaf segmentation have been proposed (Scharr et al., 2016), and in PlantCV v2 we have implemented a watershed segmentation approach. (C) Example of a merged pseudocolored image with pixels classified by the naive_bayes_classifier as background (black), unaffected leaf tissue (green), chlorotic leaf tissue (blue), and pustules (red). Methods that utilize machine learning techniques are a promising approach to tackle these and other phenotyping challenges (Minervini, Abdelsamea & Tsaftaris, 2014; Singh et al., 2016; Ubbens & Stavness, 2017; Atkinson et al., 2017; Pound et al., 2017). Leaves or other plant parts can sometimes be detected as distinct contours from the rest of the plant and need to be grouped with other contours from the same plant to correctly form a single plant/target object. Here we define high-throughput as thousands or hundreds of thousands of images per dataset. The rotate_img and shift_img functions allow the image to be adjusted so objects are better aligned to a grid pattern. PlantCV v2: Image analysis software for high-throughput plant phenotyping.

The crop_position_mask function is then used to adjust the placement of the VIS mask over the NIR image and to crop/adjust the VIS mask so it is the same size as the NIR image. Steven T. Callen analyzed the data, contributed reagents/materials/analysis tools, wrote the paper, reviewed drafts of the paper. 6). John G. Hodge and Andrew N. Doust contributed to the research described while working at the University of Oklahoma. Note: You are now also subscribed to the subject areas of this publication The pixel area of the marker is returned as a value that can be used to normalize measurements to the same scale. This function estimates the vertical, horizontal, Euclidean distance, and angle of landmark points from two landmarks (centroid of the plant object and centroid localized to the base of the plant). PlantCV v2.1 is archived on Zenodo at https://doi.org/10.5281/zenodo.1035894. Machine learning: Our goal is to develop additional tools for machine learning and collection of training data. The following information was supplied regarding data availability: PlantCV is available on GitHub at https://github.com/danforthcenter/plantcv. The cluster_contours function clusters contour objects using a flexible grid arrangement (approximate rows and columns defined by a user). For example, identification of petals can be used to measure flowering time, but petal color can vary by species. The cluster_contours_split_img function creates a new image for each cluster group.

Systems for collecting image data in conjunction with computer vision techniques are a powerful tool for increasing the temporal resolution at which plant phenotypes can be measured non-destructively. Otsus binarization (otsu_auto_threshold; (Otsu, 1979)) is best implemented when a grayscale image histogram has two peaks since the Otsu method selects a threshold value that minimizes the weighted within-class variance. This method can likely be used for a variety of applications, such as identifying a plant under variable lighting conditions or quantifying specific areas of stress on a plant. If there is no clustered object in a grid cell, no image is outputted. Here, we focus on the software tools required to nondestructively measure plant traits through images. It is assumed that the pot position changes consistently between VIS and NIR image datasets. We previously described the Plant Computer Vision (PlantCV) software package, which is an image processing toolkit for plant phenotyping analysis. The inputs required are an image, an object mask, and a minimum distance to separate object peaks. Because standards for data collection and management for plant phenotyping data are still being developed (Pauli et al., 2016), image metadata is often stored in a variety of formats on different systems. Utilizing a rectangular neighborhood around a center pixel, median_blur replaces each pixel in the neighborhood with the median value. Computational tools that are flexible and extendable are needed to address the diversity of plant phenotyping problems. If a white color standard is visible within the image, the user can specify a region of interest. See the online documentation for an example multi-plant imaging pipeline (http://plantcv.readthedocs.io/en/latest/multi-plant_tutorial/). Second, PlantCV was written in Python, a high-level language widely used for both teaching and bioinformatics (Mangalam, 2002; Dudley & Butte, 2009), to facilitate contribution from both biologists and computer scientists. As noted throughout, we see great potential for modular tools such as PlantCV and we welcome community feedback. After objects are clustered, the cluster_contour_split_img function splits images into the individual grid cells and outputs each as a new image so that there is a single clustered object per image. Total image size is detected, and the rows and columns create a grid to serve as approximate ROIs to cluster the contours (Fig. User-provided templates are built using a restricted vocabulary so that metadata can be collected in a standardized way. PlantCV v2 has added new functions for image white balancing, auto-thresholding, size marker normalization, multi-plant detection, combined image processing, watershed segmentation, landmarking, and a trainable naive Bayes classifier for image segmentation (machine learning). Images of Setaria viridis (A10) and Setaria italica (B100) are from publicly available datasets that are available at http://plantcv.danforthcenter.org/pages/data.html (Fahlgren et al., 2015; Feldman et al., 2017). The current method for multi-plant identification in PlantCV is flexible but relies on a grid arrangement of plants, which is common for controlled-environment-grown plants. 3).

The triangle threshold method uses the histogram of pixel intensities to differentiate the target object (plant) from background by generating a line from the peak pixel intensity (Duarte, 2015) to the last pixel value and then finding the point (i.e., the threshold value) on the histogram that maximizes distance to that line. PeerJ promises to address all issues as quickly and professionally as possible. In addition to producing the thresholded image in debug mode, the triangle_auto_threshold function outputs the calculated threshold value and the histogram of pixel intensities that was used to calculate the threshold. Kernel density estimation (KDE) is used to calculate a probability density function (PDF) from a vector of values for each HSV channel from each class. The field of digital plant phenotyping is at an exciting stage of development where it is beginning to shift from a bottleneck to one that will have a positive impact on plant research, especially in agriculture. The get_nir function requires that the image naming scheme is consistent and that the matching image is in the same image directory. For a growing plant, potential landmarks include the tips of leaves and pedicel and branch angles. Alternatively, gaussian_blur determines the value of the central pixel by multiplying its and neighboring pixel values by a normalized kernel and then averaging these weighted values (i.e., image convolution) (Kaehler & Bradski, 2016). A recent example of this latter approach built on PlantCV, using its image preprocessing and segmentation functions alongside a modular framework for building convolutional neural networks (Ubbens & Stavness, 2017). The release for this paper is v2.1. The number of rows and columns approximate the desired size of the grid cells. An example of one such application is the landmark_reference_pt_dist function. PlantCV v1.0 required pipeline development to be done using the command line, where debug mode is used to write intermediate image files to disk for each step. As an example, we used images of wheat leaves infected with wheat rust to collect pixel samples from four classes: non-plant background, unaffected leaf tissue, rust pustule, and chlorotic leaf tissue, and then used the naive Bayes classifier to segment the images into each class simultaneously (Fig. The ability to subjectively adjust the window size used for generating angle scores also helps to tailor analyses for identifying points of interest that may differ in resolution. The function uses the input mask to calculate a Euclidean distance map (Liberti et al., 2014). Targeted plant phenotypes can range from measurement of gene expression, to flowering time, to grain yield; therefore, the software and hardware tools used are often diverse. In terms of speed, the user is only limited by the complexity of the pipeline and the number of available processors. Contributors to PlantCV submit bug reports, develop new functions and unit tests, or extend existing functionality or documentation. PlantCV is an open-source, open-development suite of analysis tools capable of analyzing high-throughput image-based phenotyping data (Fahlgren et al., 2015). If you are following multiple publications then we will send you Eric Platon contributed to the research described while working as a founder and employee of Cosmos X. Tony Sax contributed to the research described while a full-time student at the Missouri University of Science and Technology. A random sample of 10% of the foreground pixels and the same number background pixels are used to build the PDFs. 1A). An example of how the watershed segmentation method was used to assess the effect of water deficit stress on the number of leaves of A. thaliana plants can be found in Acosta-Gamboa et al. We currently use the Pixel Inspection Tool in ImageJ (Schneider, Rasband & Eliceiri, 2012) to collect samples of pixel RGB values used to generate the training text file. The location of landmark points can be used to examine multidimensional growth curves for a broad variety of study systems and tissue types and can be used to compare properties of plant shape throughout development or in response to differences in plant growth environment. Triangle, Otsu, mean, and Gaussian auto-thresholding functions were added to PlantCV to further improve object detection when image light sources are variable. The output of image files mainly used to assess image segmentation quality is now optional, which should generally increase computing performance. In PlantCV v1.0, image analysis parallelization was achieved using a Perl-based multi-threading system that was not thread-safe, which occasionally resulted in issues with data output that had to be manually corrected. Once the training table is generated, it is input into the plantcv-train.py script to generate PDFs for each class. New data sources: Handling and analysis of data from specialized cameras that measure three-dimensional structure or hyperspectral reflectance will require development or integration of additional methods into PlantCV. To help mitigate image inconsistencies that might impair the ability to use a single global threshold and thus a single pipeline over a set of images, a white balance function was developed. The Bellwether Phenotyping Facility has both RGB visible light (VIS) and near-infrared (NIR) cameras, and images are captured 1min apart (Fahlgren et al., 2015). The modular structure of the PlantCV package makes it easier for members of the community to become contributors. New functions have been added to PlantCV v2 that enable individual plants from images containing multiple plants to be analyzed. A PlantCV pipeline is written by the user as a Python script. PlantCV can be used to generate binary masks for the training set using the standard image processing methods and the new output_mask function. Preliminary evidence from a water limitation experiment performed using a Setaria recombinant inbred population indicates that vertical distance from rescaled leaf tip points identified by the acute_vertex function to the centroid is decreased in response to water limitation and thus may provide a proximity measurement of plant turgor pressure (Figs. Kerrigan B. Gilbert prepared figures and/or tables, reviewed drafts of the paper. Mean and Gaussian thresholding are executed by indicating the desired threshold type in the function adaptive_threshold. The mean and Gaussian methods will produce a variable local threshold where the threshold value of a pixel location depends on the intensities of neighboring pixels. Malia A. Gehan, Noah Fahlgren and Max J. Feldman conceived and designed the experiments, performed the experiments, analyzed the data, contributed reagents/materials/analysis tools, wrote the paper, prepared figures and/or tables, reviewed drafts of the paper. The average pixel value output allows for concave landmarks (e.g., leaf axils and grass ligules) and convex landmarks (e.g., leaf tips and apices) on a contour to be differentiated in downstream analyses. We used 99 training images (14 top view and 85 side view images) from a total of 6,473 images. To extend PlantCV beyond quantification of size-based morphometric features, we developed several landmarking functions. Modifying the stepwise input shifts the distance calculation along the x-axis, which subsequently calculates a new threshold value to use. These updates will appear in your home dashboard each time you visit PeerJ. The scale that might be considered high-throughput for root phenotyping might not be the same for shoot phenotyping, which can be technically easier to collect depending on the trait and species. 5). 4C and 4D). The PlantCV metadata processing system is part of the parallelization tool and works by using a user-provided template to process filenames. To address this, PlantCV v2 contains functions to identify anatomical landmarks based upon the mathematical properties of object contours (Type II) and non-anatomical pseudo-landmarks/semilandmarks (Type III), as well as functions to rescale and analyze biologically relevant shape properties (Bookstein, 1991; Bookstein, 1997; Gunz, Mitteroecker & Bookstein, 2005; Gunz & Mitteroecker, 2013). Several functions were also added to aid the clustering function. Statistical analysis and data visualization was done using R v3.3 (R Core Team, 2017) and RStudio v1.0 (RStudio Team, 2016). This is a challenging area of research because the visual definition of phenotypes vary depending on the target species. Typos, corrections needed, missing information, abuse, etc. (B) The cluster_contours_split_img function was used to split the full image into individual plants. You can also choose to receive updates via daily or weekly email digests. Project-specific GitHub repositories are kept separate from the PlantCV software repository because their purpose is to make project-specific analyses available for reproducibility, while the main PlantCV software repository contains general purpose image analysis modules, utilities, and documentation. Photo credit: Katie Liberatore and Shahryar Kianian. There is growing interest among the PlantCV user community to process images with multiple plants grown in flats or trays, but PlantCV v1.0 was built to processes images containing single plants. In PlantCV v2, several service integrations were added to automate common tasks during pull requests and updates to the code repository. There are several areas where we envision future PlantCV development. Malia A. Gehan, Noah Fahlgren, Arash Abbasi, Jeffrey C. Berry, Steven T. Callen, Leonardo Chavez, Max J. Feldman, Kerrigan B. Gilbert, Steen Hoyer, Andy Lin, Csar Lizrraga, Michael Miller and Monica Tessman contributed to the research described while working at the Donald Danforth Plant Science Center, a 501(c)(3) nonprofit research institute. Therefore, several functions were added to allow the plant binary mask that results from VIS image processing pipelines to be resized and used as a mask for NIR images. For mean adaptive thresholding, the threshold of a pixel location is calculated by the mean of surrounding pixel values; for Gaussian adaptive thresholding, the threshold value of a pixel is the weighted sum of neighborhood values using a Gaussian window (Gonzalez & Woods, 2002; Kaehler & Bradski, 2016).