Vineyard pruning weight assessment by machine vision: towards an on-the-go measurement system This article is published in cooperation with the 21th GIESCO International Meeting, June 23-28 2019, Thessaloniki, Greece. Guests editors : Stefanos Koundouras and Laurent Torregrosa
Abstract
Aim: Pruning weight is an indicator of vegetative growth and vigour in grapevine. Traditionally, it is manually determined, which is time-consuming and labour-demanding. This study aims at providing a new, non-invasive and low-cost method for pruning weight estimation in commercial vineyards based on computer vision.
Methods and results: The methodology relies on computer-based analysis of RGB images captured manually and on-the-go in a VSP Tempranillo vineyard. Firstly, the pruning weight estimation was evaluated using manually taken photographs using a controlled background. These images were analysed to generate a model of wood pruning weight estimation, resulting in a coefficient of determination (R2) of 0.91 (p<0.001) and a root-mean-square error (RMSE) of 87.7 g. After this, a mobile sensor platform (modified ATV) was used to take vine images automatically and on-the-go without background. These RGB images were analysed using a fully automated computer vision algorithm, resulting in R2 = 0.75 (p<0.001) and RMSE = 147.9 g. Finally, the mobile sensor platform was also used to sample a commercial VSP vineyard to map the spatial variability of wood pruning weight, and hereafter vine vigour.
Conclusions: The results showed that the developed computer vision methodology was able to estimate the vine pruning weight in commercial vineyards and to map the spatial variation of the pruning weight across a vineyard.
Significance and impact of the study: The presented methodology may become a valuable tool for the wine industry for rapid assessment and mapping of vine vigour. This information can be used to support decision making on pruning, fertilization and canopy management.
Introduction
Pruning weight is an important indicator used to appraise biomass production, carbon storage cycle, vigour and vine balance (Smart and Robinson, 1991; Keller, 2015). It is particularly sensitive to multiple factors such as soil fertility and depth, as well as water availability (White, 2015), and shows a high spatial variability in vineyards (Dobrowski et al., 2003). The ratio between vine yield and pruning weight was defined as the Ravaz Index (Champagnol, 1984), and it is an indicator of vine balance and grape quality (Smart and Robinson, 1991). Manual pruning weight assessment is a laborious and time-consuming process, which interferes with the usual pruning flow, because of the need to collect and weigh the corresponding pruned shoots (Taylor and Bates, 2012).
Advanced sensing technologies have been widely applied in precision viticulture (Zarco-Tejada et al., 2014; Matese and di Gennaro, 2015). Both remote and proximal platforms can be used in precision viticulture. Airborne sensors can be used to monitor entire vineyards but at the cost of expensive technologies, weather constraints and costly revisiting operations. Another limitation of the utilisation of aerial sensors to monitor non-continuous crops (like vineyards) from zenithal view is the influence of soil reflectance in the calculation of indices (Stamatiadis et al., 2006). Conversely, proximal sensing using ground sensors does not have this limitation and can greatly reduce the cost of multi-temporal data acquisition as data acquisition can be performed by non-specialist workers and with lower weather constraints than airborne operations. Manually operated proximal sensor devices have been used for several applications in viticulture (De Bei et al., 2011; Baluja et al., 2012; Pou et al., 2014). However, using portable sensors for manually vineyard monitoring is slow and labour demanding. To overcome this pitfall, on-the-go sensors have been recently used for vineyard monitoring (Sepúlveda-Reyes et al., 2016; Palleja and Landers, 2017; Fernández-Novales et al., 2018), reducing the cost, especially when conducted simultaneously to another vineyard operation, like tillage.
Computer vision as a non-invasive, fast, and low-cost technology has been utilised as a proximal sensing tool to assess several features of grapevines. The analysis of indoor acquired images has enabled the precise and fast evaluation of berry (Kicherer et al., 2013) and cluster characteristics (Cubero et al., 2015; Diago et al., 2015). Alternatively, in-field imaging does not require destructive sample collection, thus increasing its industrial applicability, with use cases including the estimation of the number of flowers per inflorescence (Millan et al., 2017; Liu et al., 2018), the assessment of canopy architecture (Diago et al., 2016a) or grapevine phenotyping (Kicherer et al., 2015; Klodt et al., 2015). Image sensors have been mounted on agricultural vehicles for yield prediction (Nuske et al., 2011; Liu et al., 2017; Aquino et al., 2018; Millan et al., 2018).
Computer vision has also been proposed for vine pruning weight assessment. McFarlane et al. (1997) tried to develop an image analysis algorithm to process manually acquired vine images with uncontrolled background. Though promising, a more accurate classification of vine organs was required. Botterill et al. (2013, 2017) attempted to overcome the background and illumination pitfalls by employing a wheeled platform with controlled lighting, designed to completely cover the vines. The large platform was pulled along the rows at very low speed, resulting in a complex and slow application for viticulture. Another approach to avoid uncontrolled lighting conditions and the interference of the vines in the background was addressed by Kicherer et al. (2017) through two different methods: manual segmentation on vine images using a white screen as background; and the use of a multi-camera system for depth reconstruction. Controlled scene using a white curtain as background was also proposed (Gao and Lu, 2006; Gao, 2011) in conjunction with stereoscopic images to develop a robot that can perform automated pruning in grapevines. Pruning weight could also be assessed using other technologies mounted on different platforms. Different strategies, involving the use of multispectral sensing from airborne platforms (Dobrowski et al., 2003) or ground-based LIDAR monitoring have also been applied for pruning weight estimation (Grocholsky et al., 2011; Tagarakis et al., 2018). Despite all this range of successful methodologies, their industrial applications in viticulture are very limited due to the complexity of platforms and technologies, the Physiocap® sensor, developed in France, being, to the best of our knowledge, the only commercially available. This sensor is an on-the-go system featuring laser micrometers and capable of providing detailed measurement of vine vigour expressed as cane number and diameter, and total pruning weight per vine using an over-the-row frame side-mounted to an agricultural vehicle (Debuisson et al., 2012; Demestihas et al., 2018). Although interesting, the Physiocap® sensor only samples a small part of the woods to provide an estimation. This measuring principle is interesting when the shoots are vertical and the training system keeps the shoots in a vertical plane. On the contrary, it can be problematic when the shoots are inclined like it may happen in some regions or with some varieties.
The main aim of this work was to develop a new machine vision-based methodology for assessing wood pruning weight in a commercial vineyard using on-the-go imaging. Spatial variability of grapevine pruning weight was also mapped as a proof of concept of the methodology capabilities.
Materials and methods
The experiment was split into three related objectives: i) to develop an image analysis-based methodology for pruning weight assessment using manually acquired images; ii) to use a mobile sensing platform for automated and on-the-go image acquisition; and iii) to map the spatial variability of the pruning weight of a commercial vineyard as a proof of concept of the applicability of the presented methodology.
1. Experimental design
Image acquisition for model development and validation was carried out in December 2015 before winter pruning in a Tempranillo (Vitis vinifera L.) commercial vineyard plot located in Logroño (Latitude = 42.434853º, Longitude = 2.513719º, Altitude = 477m asl; WGS 84; La Rioja, Spain). Grapevines were pruned to double cordon Royat and trained to a vertically shoot-positioned (VSP) trellis system with northwest-southeast row orientation at 3.0 × 1.2 meters inter and intra row distances. The vines were planted in 2010 and grafted on Richter 110 rootstock.
The vines were selected to cover the widest possible range of pruning weight, including grapevines showing different self-occlusion conditions. Likewise, some vines were subjected to partial manual pruning to decrease their shoot number. The adjacent vines within the row to the one under study were also pruned to prevent their canes from interfering with the scene of the vine under analysis, thus affecting the precision of the ground truthing process. RGB images were acquired manually and on-the-go in the commercial vineyard. After image capturing, the vines were pruned and weighted using a hanging scale (Kern and Sohn GmbH, Balingen-Frommern, Germany) to be used as reference data.
2. Image acquisition
2.1 Manual image acquisition
A total of 44 vines were selected and manually photographed using a Nikon D5300 digital reflex camera (Nikon corp., Tokyo, Japan) equipped with an AF-S DX NIKKOR 18-55mm f/3.5-5.6G VR lens. Images were taken at daytime using a white screen as background and saved at a resolution of 24 Mpx (6000 × 4000 pixels) in the RGB colour space with 8 bits per channel. The camera was fixed to a tripod positioned at a distance around 1.2 m. No illumination was used apart from natural sunlight. The camera was configured with a sensitivity of ISO 640, aperture of f/4.5 and the exposure time was automatically selected.
2.2 On-the-go image acquisition
The same 44 vines that were photographed manually were then automatically imaged at nighttime and on-the-go using a mobile sensing platform, consisting of a modified all-terrain vehicle (ATV) (Trail Boss 330, Polaris Industries, Minnesota, USA) moving at 7 km/h. Taking the images at night and with a careful illumination and camera parametrization allowed to avoid the interference of the vines in the back without utilising a controlled background. The images were acquired using a Sony alpha 7-II digital mirrorless camera (Sony Corp., Tokyo, Japan) mounted at 100 cm from the ground and 250 cm from the canopy with a Zeiss Vario-Tessar FE 24-70 mm lens with optical stabilization (Figure 1). The images were saved at a resolution of 24 Mpx (6000 × 3376 pixels) in the RGB colour space with 8 bits per channel. A 900 LED Bestlight panel and two Travor spash IS-L8 LED lights were used for scene illumination. The ATV was fitted with an adjustable mechanical structure (Figure 1), which could be fixed to different heights and depths to adapt to the vines’ configuration. The structure also provided protection against canopy impact and allowed the attachment of the illumination equipment. The camera was triggered by a custom-built controller based on Arduino Mega (Arduino LLC, Ivrea, Italy). The controller generated the shooting signal based on the information received from an inductive sensor attached to the rear axle, which was activated three times per wheel revolution. A Leica Zeno 10 GNSS receiver (Heerbrugg, St. Gallen, Switzerland) was used to geo-position the images. The triggering signal along with the actual position was processed by the controller and displayed in a 4.3-inch screen for debugging purposes. Georeferencing data was also stored in a secure digital card.
Figure 1. Mobile sensing platform: modified all-terrain vehicle (ATV) equipped with a digital RGB camera, automated triggering system, illumination on an adjustable structure and GNSS receiver for data georeferencing.
It was used for the on-the-go image acquisition to estimate pruning weight.
3. Development of the image-based algorithm for pruning weight assessment
Manual and on-the-go acquired images were analysed using two algorithms developed in Matlab (R2018b, Mathworks, Natick, MA, USA), the main differences between the image sets being the background segmentation procedure (the manual images had a white background while the on-the-go images had a dark one) and the automated ROI selection procedure developed for the automatically process of the on-the-go acquired images. The image analysis procedure consisted of three main steps (image segmentation, trunk/cane identification, and ROI definition) as follows (Figure 2):
Image segmentation: Images were segmented using a Mahalanobis distance classifier (Mahalanobis, 1936; McLachlan, 1999) similar to the approach used in Diago et al. (2016a) together with mathematical morphology. The classifier was trained using supervised learning by manually selecting 30 pixels per set with the objective of covering as much variability as possible. All the pixels in the original images were assigned to three different classes (wood, wires/posts and background) covering the expected objects. From this, a pre-processing procedure based on mathematical morphology was performed in the wood set (trunk and cane) in order to minimize the misclassification with the wires using the orientation and characteristic form of the wires when compared to the vegetal material. A multi-phase filtering was executed: the first step consisted of removing the VSP system wires using a structuring element of a line with different angle variation from -45º to 45º on the cable class. After this, a procedure to fill the discontinuities on the cane that laid under the cables was implemented using a morphological reconstruction (Soille, 2004) using a small disk as structuring element (SE).
Trunk/cane identification: Since the colour of the canes and trunk/cordon was very similar, there was no possibility of separating them using only RGB information. This was overcome by dividing wood pixel set into several pixel blocks of arbitrary sizes and shapes using the watershed transformation (Meyer, 1994) and classifying them in trunk/cordon or cane classes using a Support Vector Machine or SVM (Vapnik, 2000). The watershed was calculated over the Euclidean distance of every pixel to the nearest one assigned to the wood class. Previously, the number of regional minima of the Euclidian distances was reduced by computing the extended-minima transform and then applying morphological reconstruction using the “imimposemin” function of MATLAB in order to minimize the further “oversegmentation” (i.e. divide the pixel set into a very high number of blocks) produced by the watershed transformation. Different features based on statistical moments, texture and relative position were extracted from the pixel groups identified by the watershed transformation and they were used as predictors in an SVM classifier to label the groups in two sets: trunk/cordon and canes. Finally, all neighbour regions divided by lines formed by the watershed transformation were merged to obtain both the cane and trunk/cordon binary images removing all the small pixel groups.
Region of Interest (ROI) definition: The last step consisted of defining a ROI that included the canopy of the vine. This was automatically performed using the wood binary image, and the procedure required two steps:
3.1. Cordon identification
A “vertical accumulator” was defined. This was obtained from the sum of the number of pixels per row of the wood binary image. The higher the horizontal concentration of pixels corresponding to the wood class, the bigger the value of the accumulator for this position. A vertical accumulator is represented in Figure 3A. Analysing the accumulator, the cordon position (in the vertical axis) can be identified as the maximum value of the smoothed (using a mobile mean) accumulator vector. This is represented by a light blue line in Figure 3B.
3.2. Trunk identification
To locate the trunk position all the pixels corresponding to the wood class below the cordon position were used to generate a “horizontal accumulator” for each column. This is represented in Figure 3B superimposed to the original image in light green colour. Since more than one vine can be present in an image (and thus more than one trunk) the analysis was performed for the two main peaks (selected by its amplitude and magnitude). In Figure 3B, two main peaks (P1 and P2) can be identified. Should more than two peaks appear in the scene (i.e. P3 in Figure 3B), the extra peaks would be ignored. The peak located nearest to the scene centre (usually the biggest peak of the accumulator due to the better illumination in this part of the image) was selected as the trunk of the vine to be analysed. To avoid errors associated to pixel misclassified as trunk, the maximum of the peak must be at least 75% of the value of the other candidate to be chosen. In the example illustrated in Figure 3B, P1 was selected as the trunk of the vine under consideration. By doing this, the ROI would fit the most centred vine in the image, which usually corresponded to a fully present and well-illuminated vine image.
From this, the ROI limits could be defined as follows:
- Lateral limits: These were fixed using some images for training. Half of the number of pixels of the cordon width was assigned to each “x” direction from the trunk position.
- Vertical limits: The inferior limit was set as the cordon position, and the superior corresponded to the upper classified cane pixel value in the vertical zone delimited between the lateral limits.
Figure 2. Flow-chart of the multi-step automated image processing algorithm used in on-the-go captured images.
The first step consisted of image segmentation using a Mahalanobis classifier and morphological mathematical-based processing to improve the results and filter the VSP trellis. The second step was based on the watershed transformation and SVM classification to differentiate canes and cordon/trunk of the vine. Finally, a ROI was automatically defined to encompass the vine under study.
After this procedure, the ROI was selected and represented in Figure 3B as a red rectangle. The number of pixels corresponding to the cane clusters that were within the limits of the ROI was used to estimate the pruning wood weight per vine.
Figure 3. Automated ROI selection process representation.
(A) A vertical accumulator corresponding to the sum of the pixels classified as wood over the horizontal axis is represented. This value is used to identify the vine cordon position. (B) Original vine image captured automatically on-the-go with the cordon position in light blue and the horizontal accumulator corresponding to the sum of the pixels classified as wood over the vertical axis from the bottom of the image to the cordon position represented in light green. Three peaks can be identified, the most centred one (P1) being the selected one to define the ROI (represented in red). As can be seen in the image, the peaks located closer to the centre of the image were usually bigger due to better illumination and thus easier segmentation.
4. Mapping of pruning weight spatial variability in a commercial vineyard
Once the image analysis and the models for pruning weight estimation using either manually or on-the-go acquired images were developed and validated, they were applied to a practical use-case. Concretely, mobile sensor platform for automated image acquisition was used to assess the spatial variability of vine pruning weight in another commercial vineyard located in Ábalos (Latitude = 42.579158º, Longitude = -2,707921º, Altitude = 636m asl; WGS 84; La Rioja, Spain) with a total area of 0.5 ha. Tempranillo (clone ISV-F-V6 planted on rootstock SO4) vines were planted in N-S orientation in 2006, pruned to double cordon Royat and trained to a VSP trellis system with 2.5 m row spacing and 0.8 m vine spacing.
Following the same on-the-go image acquisition set up and procedure explained above, canopy grapevine images were captured automatically and on-the-go. From this dataset, 80 data points were selected and analysed to generate the pruning weight estimation map. These vines remained unaltered, that is, they were not partially pruned before image acquisition. After image capture and analysis, the previously developed models and calibrations were applied to obtain georeferenced pruning weight estimation. The dataset was finally mapped using kernel interpolation with ArcGIS 10.4 (ESRI, Redlands, CA, USA).
5. Statistical analysis
The data generated using the described image analysis algorithm and their relationship to the reference values were analysed using Sigma Plot 12.0 (Systat Software Inc., San José, CA, USA). The regression lines with their coefficient of determination (R2), 95% confidence intervals of the slope coefficients and p-values were also calculated. Model validation was performed using leave-one-out cross-validation (LOOCV) in Weka 3.8.0 (University of Waikato, Hamilton, New Zealand). To run the LOOCV data were separated into two groups: n-1 samples were used for training and the remaining data point was used for validation. The process was repeated for the n possible combinations of the training and validation sets, and the error obtained for every model was averaged to generate the LOOCV error indices. The models were benchmarked using the mean absolute error (MAE) and the root-mean-square error (RMSE). MAE is the average of the absolute errors in the prediction of the model, whereas RMSE offers an absolute value of the prediction error. The use of LOOCV in conjunction with RMSE and MAE ensures the prediction capabilities for pruning weight of the models.
Results
1. Pruning weight assessment from RGB images manually acquired
Figure 4A shows an example of a manually acquired image of a grapevine. The ROI was manually selected to include all the shoots of the vine while avoiding the trunk. After the photographing process, the images were segmented, and the results can be observed in Figure 4B, where the pixels corresponding to the canes are shown in blue, the trellis wires in yellow and the background in black. The regression plot for the number of pixels segmented as pruning wood and the pruning weight yielded a R2 = 0.92 (Figure 5). The model was externally validated using LOOCV, which resulted in R2 = 0.91, RMSE = 87.7 g and MAE = 61.7 g.
Figure 4. Manually acquired vine images.
(A) Example of the vine canopy image captured manually with a white screen as background. (B) Segmented image of the pruning wood using image analysis. The cane pixels are represented in blue, the trellis wires in yellow, the background in black, and the boundaries of the ROI (manually selected) used for image analysis in red.
Figure 5. Relationship between the wood pruning weight and the number of pixels segmented as pruned wood on manually captured vine images.
(A) 44 grapevine images were manually captured in the field with uncontrolled light conditions using a white screen as background. Solid line corresponds to the linear regression y = 109048x + 1E+06 (R2 = 0.92***). Dashed lines represent the 95% prediction band.
2. Pruning weight assessment from RGB images acquired on-the-go
The ROI for the on-the-go acquired vine images was selected automatically as previously described (Figure 6A). It must be noted that, in contrast to the manual dataset in which a white screen was used as background, nothing was employed to this end in this case. This approximation greatly simplifies the acquisition process allowing for on-the-go imaging at 7 km/h. Differentiation of the vines under evaluation from those in the adjacent row was successfully achieved by means of controlled illumination and camera parametrization. The results of the segmentation of the images are shown in Figure 6B, where the pixels segmented as pruning wood are represented in light blue, the trunk and cordon in dark brown, the ROI (automatically calculated) is represented in red and the remaining pixels in black. The regression plot for the number of pixels segmented as shoots and the pruning weight yielded a R2 = 0.77 and is displayed in Figure 7, along with the 95% prediction band.
Figure 6. On-the-go acquired vine RGB image.
(A) Example of vine canopy image obtained using a mobile sensing platform at night. (B) Segmented image of the pruning wood using image analysis. Blue pixels correspond to the pruning wood, brown is assigned to trunk/cordon class and the red rectangle corresponds to the boundaries of the automatically generated ROI.
The LOOCV for external validation of the linear model resulted in R2 = 0.75, RMSE = 148 g and MAE = 147.9 g. If only the vines with pruning weight over 150 g were selected (35 vines), the LOOCV linear model resulted in R2 = 0.53, MAE = 137 g and RMSE = 158.6 g.
Figure 7. Relationship between the wood pruning weight and the number of pixels segmented as pruned wood on on-the-go captured vine images.
A mobile sensing platform was used for the automated image capturing process of 44 vines. Solid line corresponds to the linear regression y = 280841x + 29311 (R2 = 0.77***). Dashed lines represent the 95% prediction band.
3. Mapping of pruning weight in a commercial vineyard
Figure 8 displays the obtained map for pruning weight estimation of a Tempranillo commercial vineyard expressed in kg/vine assigning the cane pixels to each vine as the ones included in the automatically defined ROI. Three different classes were represented. The first corresponded to low vigour (pruning weight < 0.75 kg/vine) and was mainly distributed on the sides of the central part of the vineyard. The zone with high vigour values (pruning weight > 1.25 kg/vine) was located in the middle of the vineyard, especially in the northeast and southwest of the plot. These aggregations were surrounded by a large area of vines in the range between 0.75 and 1.25 kg/vine (medium vigour).
Figure 8. Map of the pruning wood weight estimation of a 0.5-ha commercial VSP Tempranillo vineyard (La Rioja, Spain).
Three classes were defined: less than 0.75 kg/vine; 0.75–1.25 kg/vine; and over 1.25 kg of pruning weight estimation per vine. The map was generated from the analysis of images captured on-the-go using a mobile sensing platform.
Discussion
The results obtained in the present study confirm the capability of the developed computer vision method from in-field captured vine RGB images on-the-go to estimate the pruning weight values and variability in commercial vineyards. The methodology was evaluated on VSP vineyards as it is the most widely used training system worldwide. The possibility to use a relatively simple and low-cost technology, such as RGB imaging, and to work on-the-go at 7 km/h speed for efficient mapping greatly improves the applicability of this new method for grapevine pruning weight estimation as an indicator of the spatial variability of vegetative growth and vine vigour.
Technically speaking, the two main pitfalls inherent to image analysis applications outdoors, like the variation of the lighting conditions and the influence of opposite vines in the background, were successfully overcome both in the manual and on-the-go imaging modes. In the manual acquisition mode, a white background was used to avoid the presence of the opposite vines in the scene, while this was also achieved in the on-the-go mode, taking the images at nighttime without any colour background. The manually captured images were taken under uncontrolled, natural illumination generating variations in the background (white screen) and foreground (shoots and other objects). Nevertheless, the coefficient of determination for these images revealed good prediction capabilities as demonstrated by the RMSE and MAE obtained using LOOCV. It must be noted that the coefficient of determination was decreased when the lower pruning weight vine values (introduced to increase the variability of the samples) were removed, but MAE and RMSE were not affected. These were very similar or even superior to those obtained in other studies (Kicherer et al., 2017) in which the pruning weight was estimated from a depth map obtained with a manually operated image-based device composed of three cameras using a white screen to avoid the influence of background vines. The RGB computer-vision approach presented in this work is simpler than previously reported methods yielding similar pruning weight estimation accuracy (Kicherer et al., 2017).
The lack of a controlled background and the motion during on-the-go image acquisition was a challenge in the image-taking process. The use of external illumination, together with a proper parametrization of the camera settings, that is, by finding the optimum balance between shutter speed, ISO sensitivity and aperture, led to the acquisition of sharp images in which the vines in the back rows of the scene were obscured. The necessity of night operation is a drawback of this technique; future work will concentrate on adapting the image acquisition process for daytime conditions. Likewise, the use of shorter exposure times and brighter external lights will be explored. The estimation performance values obtained for the on-the-go imaging mode were also very satisfactory and agreed with those obtained in previous works (Kicherer et al., 2017) for manual and automated analysis.
Our results show that the new computer vision-based method, using a simple RGB camera, is rapid and reliable for grapevine pruning weight assessment under field conditions with performance values similar to those reported in previous studies using more sophisticated sensors installed in ground-moving vehicles. Likewise, Tagarakis et al. (2018) conducted an experiment consisting on winter shoot scanning using a laser sensor that obtained high precision measurements (0.65 2 < 0.70) for the two campaign seasons. In a similar experiment, Grocholsky et al. (2011) measured the canopy shape and volume using a laser scanner and camera during the pre-harvest period obtaining an indirect estimation of the pruning weight within 10% of the actual value. Additionally, a new optical proximal sensor was developed in Italy for assessing and mapping the vigour and yield parameters in vineyards (Gatti et al., 2016). In Champagne (France), the Physiocap® sensor has appeared to be an interesting tool to assess vine vigour and to improve fertilization and pruning practices in precision viticulture (Debuisson et al., 2012; Demestihas et al., 2018).
Other indirect approaches based on the use of airborne multispectral imagery have also shown adequate correlations (r ~ 0.80) between spectral reflectance indices such as NDVI and PCD determined at veraison, and pruning weight (Baluja et al., 2012). It must be noted that these indirect measurements did not directly quantify the vine wood but made use of the size of the canopy to assess pruning weight. Among the drawbacks of these indirect methods for pruning weight assessment is that estimations are subjected to variations from season to season and to other factors that affect the canopy such as hail or strong winds removing leaves, but do not appear as variations in the pruning wood.
The capability of the developed on-the-go RGB imaging method to map the variability of the vine pruning weight, hence of vine vigour, is of great interest and usability. For this reason, a proof of concept is included in this work, consisting of a map generated from the images captured on-the-go and automatically processed. No validation is provided for this dataset and future work will concentrate on proper evaluation of this technique. Nevertheless, the possibility to generate pruning weight maps (which are expressed in kg/vine) offers a tool for the viticulturist that can be used to adapt the pruning severity or the number of buds per vine as well as the fertilization rate applied to grapevines to optimize vine balance and grape quality. From an automated perspective, these pruning weight maps could be installed in the new generation of variable rate machinery to adapt the severity of the pre-pruning, the fertilization rate or even the hedging or defoliation intensity according to this vigour variability information.
The presented RGB image-based methodology for pruning weight estimation was validated with images acquired under different illumination conditions and capturing circumstances (manual static capture vs automated on-the-go acquisition) on vines trained to the widely used VSP system. Future work will be carried out to test the estimation capabilities on other vine training systems and in extensive map generation. For the on-the-go mode, images were taken at 7 km/h confirming that the described methodology was adaptable to other agriculture vehicles such as tractors. Moreover, the flexibility and relative simplicity of the system makes it also feasible to be adapted to agricultural robots (Diago et al., 2016b; Rose et al., 2016) allowing for continuous and effortless vineyard monitoring.
Conclusions
A new methodology based on simple, cost-efficient, RGB computer vision was developed and validated for pruning weight estimation in commercial vineyards. The methodology can be applied to images acquired both manually using a cheap camera and automatically on-the-go. Our results support that the grapevine pruning weight can be accurately estimated from images captured directly under field conditions.
The possibility for automated and on-the-go image acquisition greatly increases the commercial application of the developed computer vision methodology. Moreover, high-density sampling and georeferencing in conjunction with fully automated computer vision algorithms enable the generation of pruning weight maps, which represent the grapevine vigour spatial variability within the vineyard. The inexpensive, non-destructive and time-saving presented procedure will support informed decisions of cultural practices in viticulture (fertilization, bud pruning number, etc.) and to improve yield and grape quality.
Acknowledgements
We would like to thank Ignacio Barrio and Saúl Río for their help collecting and analysing field data. Dr Maria P. Diago is funded by the Spanish Ministry of Science, Innovation and Universities with a Ramon y Cajal grant RYC-2015-18429. Borja Millán is recipient of a Juan de la Cierva-Formación research contract (FJCI-2017-31824) funded by the Spanish Ministry of Science, Innovation and Universities. This work received funding from the European Community’s Seventh Framework Program (FP7/2007–2013) under Grant Agreement FP7-311775, Project Innovine.
References
- Aquino A., Millan B., Diago M.P. and Tardaguila J., 2018. Automated early yield prediction in vineyards from on-the-go image acquisition. Computers and Electronics in Agriculture, 144, 26–36. doi:10.1016/j.compag.2017.11.026
- Baluja J., Diago M.P., Goovaerts P. and Tardaguila J., 2012. Assessment of the spatial variability of anthocyanins in grapes using a fluorescence sensor: relationships with vine vigour and yield. Precision Agriculture, 13(4), 457–472. doi:10.1007/s11119-012-9261-x
- Botterill T., Green R. and Mills S., 2013. Finding a vine’s structure by bottom-up parsing of cane edges. In 28th International Conference on Image and Vision Computing, New Zealand, pp. 112–117. IEEE. doi:10.1109/IVCNZ.2013.6727001
- Botterill T., Paulin S., Green R., Williams S., Lin J., Saxton V., Mills S., Chen X.-Q. and Corbett-Davies S., 2017. A robot system for pruning grape vines. Journal of Field Robotics, 34(6), 1100–1122. doi:10.1002/rob.21680
- Champagnol F., 1984. Eléments de physiologie de la vigne et viticulture générale. Saint- Gely-du-Fesc.
- Cubero S., Diago M.P., Blasco J., Tardaguila J., Prats-Montalbán J.M., Ibáñez J., Tello J. and Aleixos N., 2015. A new method for assessment of bunch compactness using automated image analysis. Australian Journal of Grape and Wine Research, 21(1), 101–109. doi:10.1111/ajgw.12118
- De Bei R., Cozzolino D., Sullivan W., Cynkar W., Fuentes S., Dambergs R., Pech J. and Tyerman S., 2011. Non-destructive measurement of grapevine water potential using near infrared spectroscopy. Australian Journal of Grape and Wine Research, 17(1), 62–71. doi:10.1111/j.1755-0238.2010.00117.x
- Debuisson S., Morlet M., Germain C., Garcia O., Panigai L. and Moncomble D., 2012. Precision viticulture: using on-board sensors to map vine variability and characterize vine trajectories. In IXth International Terroir Congress, pp. 17–19.
- Demestihas C., Debuisson S. and Descotes A., 2018. Decomposing the notion of vine vigour with a proxydetection shoot sensor: Physiocap®. In XII International Terroir Congress. doi:10.1051/e3sconf/20185003003
- Diago M.P., Krasnow M., Bubola M., Millan B. and Tardaguila J., 2016a. Assessment of vineyard canopy porosity using machine vision. American Journal of Enology and Viticulture, 67(2), 229–238. doi:10.5344/ajev.2015.15037
- Diago M.P., Rovira-Mas F., Blasco J., Saiz-Rubio V., Faenzi E., Évain S., Labails S., Stoll M., Scheidweiler M., Millot C., Campos-Gomez E. and Tardaguila J., 2016b. Vinerobot: on-the-go vineyard monitoring with non-invasive sensors. In International Cool Climate Wine Symposium, Brighton.
- Diago M.P., Tardaguila J., Aleixos N., Millan B., Prats-Montalban J.M., Cubero S. and Blasco J., 2015. Assessment of cluster yield components by image analysis. Journal of the Science of Food and Agriculture, 95(6), 1274–1282. doi:10.1002/jsfa.6819
- Dobrowski S.Z., Ustin S.L. and Wolpert J.A., 2003. Grapevine dormant pruning weight prediction using remotely sensed data. Australian Journal of Grape and Wine Research, 9(3), 177–182. doi:10.1111/j.1755-0238.2003.tb00267.x
- Fernández-Novales J., Tardaguila J., Gutiérrez S., Marañón M. and Diago M.P., 2018. In field quantification and discrimination of different vineyard water regimes by on-the-go NIR spectroscopy. Biosystems Engineering, 165, 47–58. doi:10.1016/j.biosystemseng.2017.08.018
- Gao M., 2011. Image processing and analysis for autonomous grapevine pruning. University of Adelaide, School of Mechanical Engineering. https://digital.library.adelaide.edu.au/dspace/bitstream/2440/72721/8/02whole.pdf
- Gao M. and Lu T.-F., 2006. Image processing and analysis for autonomous grapevine pruning. In 2006 International Conference on Mechatronics and Automation, pp. 922–927. doi:10.1109/ICMA.2006.257748
- Gatti M., Dosso P., Maurino M., Merli M. C., Bernizzoni F., Pirez F. J., Platè B., Bertuzi G. C. and Poni S., 2016. MECS-VINE®: a new proximal sensor for segmented mapping of vigor and yield parameters on vineyard rows. Sensors, 16(12), 2009. doi:10.3390/s16122009
- Grocholsky B., Nuske S., Aasted M., Achar S. and Bates T., 2011. A camera and laser system for automatic vine balance assessment. In 2011 ASABE Annual International Meeting, 7004(11), Paper number 1111651. doi:10.13031/2013.38146
- Keller M., 2015. The science of grapevines: anatomy and physiology (2nd ed.). San Diego: Academic Press.
- Kicherer A. Herzog K., Pflanz M., Wieland M., Rüger P., Kecke S., Kuhlmann H. and Töpfer R., 2015. An automated field phenotyping pipeline for application in grapevine research. Sensors, 15(3), 4823–4836. doi:10.3390/s150304823
- Kicherer A., Klodt M., Sharifzadeh S., Cremers D., Töpfer R. and Herzog K., 2017. Automatic image-based determination of pruning mass as a determinant for yield potential in grapevine management and breeding. Australian Journal of Grape and Wine Research, 23(1), 120–124. doi:10.1111/ajgw.12243
- Kicherer A., Roscher R., Herzog K., Šimon S., Förstner W. and Töpfer R., 2013. BAT (berry analysis tool): a high-throughput image interpretation tool to acquire the number, diameter, and volume of grapevine berries. Vitis, 52(3), 129–135. https://ojs.openagrar.de/index.php/VITIS/article/view/4001
- Klodt M., Herzog K., Töpfer R. and Cremers D., 2015. Field phenotyping of grapevine growth using dense stereo reconstruction. BMC Bioinformatics, 16(1), 143. doi:10.1186/s12859-015-0560-x
- Liu S., Cossell S., Tang J., Dunn G. and Whitty M., 2017. A computer vision system for early stage grape yield estimation based on shoot detection. Computers and Electronics in Agriculture, 137, 88–101. doi:10.1016/j.compag.2017.03.013
- Liu S., Li X., Wu H., Xin B., Tang J., Petrie P. R. and Whitty M., 2018. A robust automated flower estimation system for grape vines. Biosystems Engineering, 172, 110–123. doi:10.1016/j.biosystemseng.2018.05.009
- Mahalanobis P.C., 1936. On the generalised distance in statistics. Proceedings of the National Institute of Sciences of India, 2(1), 49–55.
- Matese A. and Di Gennaro S.F., 2015. Technology in precision viticulture: a state of the art review. International Journal of Wine Research, 7, 69–81. doi:10.2147/IJWR.S69405
- McFarlane N.J.B., Tisseyre B., Sinfort C., Tillett R.D. and Sevila F., 1997. Image analysis for pruning of long wood grape vines. Journal of Agricultural Engineering Research, 66(2), 111–119. doi:10.1006/jaer.1996.0125
- McLachlan G.J., 1999. Mahalanobis distance. Resonance, 4(6), 20–26. doi:10.1007/BF02834632
- Meyer F., 1994. Topographic distance and watershed lines. Signal Processing, 38(1), 113–125. doi:10.1016/0165-1684(94)90060-4
- Millan B., Aquino A., Diago M.P. and Tardaguila J., 2017. Image analysis-based modelling for flower number estimation in grapevine. Journal of the Science of Food and Agriculture, 97(3), 784–792. doi:10.1002/jsfa.7797
- Millan B., Velasco-Forero S., Aquino A. and Tardaguila J., 2018. On-the-go grapevine yield estimation using image analysis and Boolean model. Journal of Sensors, 2018, 9634752. doi:10.1155/2018/9634752
- Nuske S., Achar S., Gupta K., Narasimhan S. and Singh S., 2011. Visual yield estimation in vineyards: experiments with different varietals and calibration procedures. Technical Report CMU-RI-TR-11-39. Pittsburgh.
- Palleja T. and Landers A. J., 2017. Real time canopy density validation using ultrasonic envelope signals and point quadrat analysis. Computers and Electronics in Agriculture, 134, 43–50. doi:10.1016/j.compag.2017.01.012
- Pou A., Diago M.P., Medrano H., Baluja J. and Tardaguila J., 2014. Validation of thermal indices for water status identification in grapevine. Agricultural Water Management, 134, 60–72. doi:10.1016/j.agwat.2013.11.010
- Rose J., Kicherer A., Wieland M., Klingbeil L., Töpfer R. and Kuhlmann H., 2016. Towards automated large-scale 3D phenotyping of vineyards under field conditions. Sensors, 16(12), 2136. doi:10.3390/s16122136
- Sepúlveda-Reyes D., Ingram B., Bardeen M., Zúñiga M., Ortega-Farías S. and Poblete-Echeverría C., 2016. Selecting canopy zones and thresholding approaches to assess grapevine water status by using aerial and ground-based thermal imaging. Remote Sensing, 8(10), 822. doi:10.3390/rs8100822
- Smart R. and Robinson M., 1991. Sunlight into wine: a handbook for winegrape canopy management. Adelaide: Winetitles.
- Soille P., 2004. Morphological image analysis: principles and applications (2nd. Ed). Berlin: Springer Verlag. doi:10.1007/978-3-662-05088-0
- Stamatiadis S., Taskos D., Tsadilas C., Christofides C., Tsadila E. and Schepers J.S., 2006. Relation of ground-sensor canopy reflectance to biomass production and grape color in two merlot vineyards. American Journal of Enology and Viticulture, 57(4), 415–422. https://www.ajevonline.org/content/57/4/415
- Tagarakis A.C., Koundouras S., Fountas S. and Gemtos T., 2018. Evaluation of the use of LIDAR laser scanner to map pruning wood in vineyards and its potential for management zones delineation. Precision Agriculture, 19(2), 334–347. doi:10.1007/s11119-017-9519-4
- Taylor J.A. and Bates T.R., 2012. Sampling and estimating average pruning weights in Concord grapes. American Journal of Enology and Viticulture, 63(4), 559–563. doi:10.5344/ajev.2012.12069
- Vapnik V.N., 2000. The nature of statistical learning theory. New York: Springer Verlag. doi:10.1007/978-1-4757-3264-1
- White R.E., 2015. Understanding vineyard soils (2nd ed.). New York: Oxford University Press.
- Zarco-Tejada P.J., Hubbard N. and Loudjani P., 2014. Precision agriculture: an opportunity for EU farmers - potential support with the CAP 2014-2020. Brussels, Belgium. doi:10.2861/58758