Assessment of canopy size using UAV-based point cloud analysis to detect the severity and spatial distribution of canopy decline

This study aimed to validate the use of UAV-based point cloud analysis to detect canopy decline severity and its spatial distribution in vineyards. A new approach to assess canopy decline, caused by Eutypa dieback-like symptoms, using unmanned aerial vehicle (UAV) remote sensing was compared with ground visual assessment in the vineyard. Canopy point cloud captured by UAV-based imagery during the growing season was analysed by a customized program to determine canopy decline severity and spatial distribution in a symptomatic Shiraz vineyard in Eden Valley, South Australia. Experienced assessors performed a ground visual assessment in the vineyard at E-L stage 15. k-means clustering was used to detect reduced canopy volume due to Eutypa dieback-like symptoms. Results from point cloud analysis showed that 12.5 % of the total canopy length in the vineyard had Eutypa dieback symptoms while the ground visual assessment detected 11.4 %. Confusion matrix results showed an accuracy of 87.4 % and a kappa coefficient of 0.43 compared with ground visual assessments. Additionally, automatic analysis of the point cloud was quicker than the ground visual assessment and generated precise geographic coordinates of the symptomatic canopy sections. Point cloud analysis can detect Eutypa dieback-like symptoms and its spatial distribution with 87.4 % accuracy, compared with the ground assessment. Similar to ground visual assessment, E-L stage 15 appears to be a suitable stage to apply point cloud analysis to make Eutypa dieback-like symptom assessments. Grapevine canopy decline, caused by various factors such as Eutypa dieback and inadequate management, can cause yield reduction and threaten vineyard longevity. Compared with tedious ground visual assessments, point cloud analysis can accelerate the assessment of canopy decline in vineyards and help with the planning of remedial practices using precise geographic coordinates of the affected sections.


INTRODUCTION
In the vineyard, canopy decline can lead to yield losses and quality reductions (Magarey et al., 2000). Canopy decline can be caused by factors such as grapevine trunk diseases, virus and inappropriate canopy management practices (Bowman, 2018). Grapevine trunk diseases as an example, which include diseases such as Eutypa dieback and Botryosphaeria dieback, contribute to grapevine decline and threaten the longevity of vineyards globally (Sipiora and Cuellar, 2015). In Australia, Eutypa dieback is widespread with all major grapevine varieties grown being susceptible (Wine Australia, 2019). It has also been found that, if trunk disease is detected early and preventive practices are applied, the lifespan of an infected vineyard can be extended by between 26 % -47 % (Kaplan et al., 2016).
As a major cause for canopy decline, Eutypa dieback infections can occur when fungal spores enter through pruning wounds on the grapevine (Gramaje et al., 2018). As the infection develops along the cordon, the foliar symptoms of dieback will appear on shoots and gradually grow towards the trunk at the speed of around 50 mm per year (Sosnowski et al., 2008). Due to its slow development, the canopy structure between infected and healthy vines and even between cordons of the same infected grapevine can be highly variable. Remedial surgery on infected cordons has been shown to improve productivity (Sosnowski et al., 2011). Remedial surgery practices include cordon removal, retraining and replanting and the application of each differs depending on the severity of Eutypa dieback (Creaser and Wicks, 2004;Sosnowski et al., 2011). Therefore, mapping of the severity and spatial distribution of canopy decline in a vineyard is crucial to plan preventative and remedial activities. Knowing the proportion of canopy decline relative to total canopy length can also indicate the reduction in potential yield (Munkvold et al., 1994).
To assess canopy decline symptoms such as Eutypa dieback, visual assessments of foliar symptoms in the field are commonly carried out in spring, when shoot length is around 30-70 cm, between E-L stages 12-17 (Bertsch et al., 2013). During these developmental stages, Eutypa dieback incidence, severity and spatial distribution can be estimated. Currently, this practice is performed by experienced viticulturists and can be tedious and costly for large vineyards. Another potential option to assess canopy decline is the use of digital applications on portable devices. Recently, several smartphone applications such as VitiCanopy have been developed to calculate plant area index (PAI) from canopy cover imagery (De Bei et al., 2016;Fuentes et al., 2014;Poblete-Echeverria et al., 2015). PAI is a measurement of the canopy structure and corresponds to the total area of leaf and cordon per unit ground area (De Bei et al., 2016). The use of PAI to quantify canopy size and structure may also be useful in assessing canopy decline.
Remote sensing has been applied to measure the severity and distribution of grapevine canopy decline in the vineyard with the advantages of lowering labour requirements and costs (Andújar et al., 2019;Delenne et al., 2010;Hall, 2018). Red-green-blue (RGB) and multispectral imagery captured by aircraft and satellites have been used to detect missing grapevines and spectral reflectance changes in the canopy due to disease infection (Albetis et al., 2018;Chanussot et al., 2005;MacDonald et al., 2016). However, for Eutypa dieback-derived canopy decline assessment, analysis of aerial imagery captured in spring can be challenging due to interference from ground vegetation (Matese et al., 2015;Poblete-Echeverría et al., 2017). The inclusion of ground vegetation in imagery analysis has been found to reduce the accuracy of detecting missing grapevines and can potentially reduce the accuracy of canopy decline detection and decline severity assessments (Delenne et al., 2010). Therefore, it remains unclear whether the canopy development stages (E-L 12-17), suitable for ground visual assessment, still apply to UAV remote sensing.
Alternatively, point clouds reconstructed from high-resolution RGB imagery captured by an unmanned aerial vehicle (UAV) can potentially be used. Point cloud data contains three-dimensional canopy information that can be analysed to calculate canopy height, surface area and volume values (de Castro et al., 2018;Pádua et al., 2018). As a result, severity and distribution of canopy decline may be detected from low volume canopy sections, potentially representing stunted shoots and bare cordons with no canopy growth. The analysis of point cloud can also effectively filter out floor vegetation, by only extracting points above the cordon height, to reduce the interference of ground vegetation in two-dimensional imagery (Weiss and Baret, 2017). Therefore, point cloud analysis may have greater potential for measuring canopy decline in spring when the symptoms are most obvious and ground vegetation is more likely to be present.
This study aims to investigate the potential of point cloud analysis in the estimation of canopy decline severity and spatial distribution. To achieve this, UAV flights were performed and compared with ground visual assessments that are typically used to assess canopy decline, in a vineyard infected by Eutypa dieback. Subsequent UAV flights were also performed through the whole growing season (spring to autumn) to compare the performance of point cloud analysis at different developmental growth stages.

Experimental site details
A 1.2 hectare Shiraz vineyard block in Eden Valley, South Australia, Australia (Lat 34°37'57.0"S, Lon 139°06'56.8"E, elevation 386.8 m) was selected for this study during the 2018-19 growing season. In the vineyard, grapevines displayed canopy decline, likely caused by the symptoms of Eutypa dieback ( Figure 1). However, the distribution and severity of canopy decline were unclear. The block contained 20 rows of grapevines at a row spacing of 3.2 m and rows orientated east-west. Each row consisted of panels of three vines, planted at a spacing of 2 m. The grapevines were trained to a vertical shoot position (VSP) trellis system and the cordon height was 1.3 m above ground. Standard commercial vineyard management and irrigation practices were applied in the vineyard during the study period.

UAV imagery collection and process
From budburst to harvest, UAV flights were performed approximately every two weeks, with a total of 11 flights performed. Among the data collected, the focus of analysis in this paper was concentrated on the second flight at E-L stage 15, considered the optimal stage for ground assessments (Bertsch et al., 2013).
Using a Phantom 4 Pro UAV (DJI, Shenzhen, China), the on-board RGB sensor was used to capture nadir imagery at 20 megapixels resolution. The UAV flight plan was set and controlled by a remote controller connected to a smartphone installed with the flight planning software Pix4DCapture (v.4.2; Pix4D SA, Lausanne, Switzerland). A double-grid flight route covering the whole block and an imagery overlap ratio of 85 % were set for every flight and the aircraft was maintained at a constant travel speed of 2 m/s and height of 30 m above ground. Each flight took around 1hr to capture the entire block. During each UAV flight, around 1000 images were collected and the geographic coordinates of the imagery were recorded by the onboard Global Positioning System (GPS) receiver.
Using the commercial photogrammetry software PIX4Dmapper (v.4.4.12;Pix4D SA,Lausanne,Switzerland), the orthomosaic images and the point clouds of the vineyard block were reconstructed from the RGB images and stored as .ply files. The orthomosaic images had an average ground sampling distance (GSD) of 0.69 cm and point clouds covering the study block contained around 40 million points on average. Using the MATLAB computer vision toolbox and statistics and machine learning toolbox (v.2019b; Natick, Massachusetts, United States), a customized process was developed to analyse the vineyard point cloud and extract canopy points.
In the process, the vineyard densified point cloud (DPC) was first rotated to horizontal view. In this view, the coordinates of the first and last row centroids were manually selected, and middle row centroid coordinates were calculated automatically using the fixed row spacing (Figure 2, step 1). With row coordinates determined, region of interest (ROI) polygons (2 m wide), wider than the canopy width but narrower than row spacing, were  (Torr and Zisserman, 2000). This was achieved by setting the maximum absolute angular distance among neighbouring points and maximum Euclidean distance of points from the plane. With the ground plane determined, all points were categorized as either canopy points (green) or non-canopy points (purple, Figure 2, step 4). After categorization, canopy points were appended into a separate canopy point cloud which included around 6-10 million, depending on the canopy size at different growing stages. An alpha shape (polyhedra) was created from the canopy point cloud (Figure 2, step 5) (Edelsbrunner et al., 1983;Milella et al., 2019). The volume (m 3 ) of the alpha shape was calculated as the corresponding grapevine canopy volume.
To detect canopy-decline-induced canopy volume reduction, canopy point clouds were divided into subsections (0.05 m wide) along the row orientation. The volume of all the subsections followed a normal distribution. Any stunted or no growth canopy subsections were detected when their volume was below the set thresholds. Fixed thresholds for defining low canopy volume were unsuitable for the rapidly expanding canopy in the spring. To create thresholds for detecting low canopy volume between different growing stages, the k-means clustering was used to categorize the subsection volume values at each stage into five classes. The number of classes was adopted from the classification approach described in Sosnowski et al. (2016). Subsections with volume above the first (smallest) class threshold were labelled as "healthy". Sections with volume below that threshold were labelled as either "stunted" or "no growth". The first class was then further divided into another five classes with the first two smallest classes labelled as 'no growth' and the larger three classes as "stunted". The threshold between "no growth" and "stunted" category within the class was determined empirically according to their ratio in the ground assessment. With all subsections labelled, consecutive subsections labelled as "stunted" or "no growth" longer than 0.2 m (four subsections) were recorded. The length threshold of 0.2 m was used to filter out gaps from the natural spacing between shoots and could be variable for different vineyards according to the shoot density.

Ground assessments of the canopy decline
To provide the ground truth measurements, the grapevines in the study block were assessed by a combination of visual assessment, canopy cover imagery analysis and canopy FIGURE 2. Point cloud analysis procedure for canopy volume calculation. development measurements. The visual ground assessment was performed at E-L stage 15 to better capture stunted canopy sections and no growth sections before they were filled or covered by rapidly developing healthy shoots. During the assessment, experienced assessors walked through all rows in the block and visually evaluated canopy health and searched for foliar symptoms on every vine. Plants displaying canopy decline had (1) the severity (%) estimated; (2) the recorded section location as either left cordon, trunk or right cordon of the vine recorded ( Figure 4) and (3) length of the sections measured. The smartphone app, VitiCanopy was used to capture canopy cover imagery and calculate PAI, in parallel with ground visual assessments. PAI measurement was used to measure the difference in canopy structure. Briefly, one image was collected for each of the two cordons per vine (sampling distance = 1 m; n = 2160) and detailed procedures can be found in De Bei et al. (2016). The severity of the canopy decline was also assessed by counting the total shoot number and measuring mean shoot length from three randomly selected shoots. Sections found without any shoot development were labelled as no growth. For comparison with the canopy structure of declined canopy, the shoot number and shoot length of 20 randomly selected grapevines with healthy canopy were also recorded.

Canopy decline mapping and accuracy analysis
To enable the mapping of the multiple measurements taken, the geographic coordinates of individual grapevines are necessary. Theoretically, with fixed plant spacing, the geographic coordinates of all grapevines in any given vineyard can be calculated from the point cloud. The precision (centimetre level) of the coordinates calculated from the point cloud was also considered to be higher than what portable Global Navigation Satellite System (GNSS) terminal can achieve (2-3 metres) unless more complicated and expensive professional systems are used (e.g. differential GNSS or real-time kinematic). However, due to variable plant spacing in row-end panels in the vineyard, additional in-field measurements were taken to obtain the vine number and spacing in the end panels. With the vine coordinates, the spatial distribution of Eutypa dieback-like symptoms and PAI can be interpolated from field measurement points using kriging in QGIS (v.3.12.0, QGIS Project) with the spherical model used for the empirical semivariogram. . For all canopy sections, k-means clustering was used to generate labels of "healthy", "stunted" and "no growth" (colour coded). In this study, any stunted canopy or no growth sections longer than 0.2 m on a target grapevine were recorded. Point coordinates are for illustration only, not the actual geographic coordinates. To map the distribution of canopy decline, centroid coordinates of low canopy volume sections detected by point cloud were calculated. For ground measurements, the centroid coordinates were calculated from the combination of relative positions of sections on the vine and vine coordinates. Using the length and centroid coordinates of the stunted canopy section and no growth sections, polygon vector geometries were created and stored as shapefiles (.shp) to represent sections displaying canopy decline. To determine the accuracy of the point cloud analysis, compared with ground visual assessments, a confusion matrix was constructed using QGIS and the kappa coefficient was calculated (Landis and Koch, 1977).
To assess the variance in shoot length, shoot number and canopy volume measurements, mean value and standard error of mean were calculated using MATLAB statistics and the machine learning toolbox (v.2019b; Natick, Massachusetts, United States).

Eutypa dieback-like symptom detection: ground vs. point cloud
The severity of the Eutypa dieback infection, representing the level of canopy decline, measured by visual assessment, can be visualised in Figure  5. The severity of Eutypa dieback (Figure 5a), was found to be higher in the northern part of the block and lower towards the south which clearly demonstrated the progression of the Eutypa dieback-induced canopy decline. In comparison, the PAI map (Figure 5b) shows variable patches in the vineyard with lower canopy growth, which do not fully align with Eutypa dieback-like symptom distribution. As will be discussed later, this can be explained by the spatial variability in canopy development caused by a range of factors, such as soil profile, water availability, management variability, etc. It is important to note that since canopy variability can be influenced by multiple factors and weak canopy growth without obvious foliage symptoms can still be healthy. Furthermore, PAI at this development stage is relatively low and demonstrates the challenges with assessment at early developmental stages and vineyard variation when using imagery.
Average shoot number, shoot length and canopy volume, were also found to be significantly impacted by Eutypa dieback (Table 1). At E-L stage 15, the mean shoot number in stunted canopy sections was significantly less than healthy sections and stunted shoots were less than half of the length of healthy shoots. For no growth sections, no shoots were present and the mean shoot length was zero. Mean canopy volume, derived from point cloud analysis, of the stunted canopy and no growth sections were significantly smaller than healthy canopy sections. Although cordons with no growth contained no shoots, the canopy volume of the canopy gaps was greater than 0. The residual canopy volume reflects the volume of the bare cordons, as shown in Figure 3. Only in the case of cordon removal, was a canopy volume of zero recorded. Using the no growth section volume, point cloud analysis also provides the potential to distinguish between dead cordons and missing cordon/ grapevines which can be otherwise challenging for two-dimensional imagery analysis of RGB  and multispectral imagery (Delenne et al., 2010;Poblete-Echeverría et al., 2017;Puletti et al., 2014).
From point cloud analysis, the study block's total canopy length was 2398.1 m. Ground visual assessment found 181.1 m stunted canopy (7.6 % of total length) and 91.8 m no growth cordon (3.8 %). With the k-means clustering categorizing the canopy volume, point cloud analysis detected 207.8 m stunted canopy (8.7 %) and 91.7 m canopy gaps (3.8 %). The total length of the stunted canopy and no growth cordon was found to be 272.9 m (11.4 %) and 299.5 (12.5 %), as measured by ground assessment and point cloud analysis, respectively.
Ground visual assessment revealed that both stunted canopy and no growth sections were present in every single row ( Figure 6). Point cloud analysis also identified widespread stunted and no growth canopy sections. Both approaches detected large sections of canopy decline in the first three rows in the north of the block, indicating the severity of the Eutypa dieback problem. However, as can be observed in Figure 6, the ground visual assessment identified more short canopy sections with dieback symptoms than the point cloud analysis. This can be explained by the multiple factors used to grade the Eutypa dieback-induced canopy decline in ground assessment, including foliage symptoms and the overall canopy growth. As a result, ground measurement provided a more detailed but subjective assessment of the canopy decline.
Using the confusion matrix of the classification accuracy, the performance of the point cloud analysis for canopy decline detection was quantified (Tables 3 and 4). When thresholds were set for point cloud analysis to detect the similar total length of symptomatic sections with ground assessments, 87.4 % (cells in green) of the categorization by point cloud analysis matched with sections detected by ground measurement (Table 4). However, the chances of false positives (healthy canopy classified as stunted or no growth in point cloud analysis) and false negatives (stunted or no growth canopy detected as healthy in point cloud analysis) were 4.3 % (cells in red of the first column) and 5.1 % (cells in red of the first row), respectively (Table 3). In addition, 3.3 % (cells in yellow) of the detection were misclassified between stunted or no growth classes (Table 3). The kappa coefficient of 0.43 also showed moderate agreement between the point cloud analysis and ground visual assessment (Table 4). The percentage agreement observed is also higher than the percentage agreement expected by chance. The analysis of point cloud demonstrates interesting and promising results that with further development may become a method for canopy decline assessment in vineyards.
Several potential reasons for Eutypa dieback-like symptom detection variations between ground and point cloud analysis approaches have been found and summarised: Potential reasons leading to the location variations include: 1. The locations of the ground measurements were less precise than results from point cloud analysis. The ground measurement only recorded relative locations (left cordon, trunk or right cordon) of the infected sections on the grapevine. In comparison, point cloud analysis provided more precise geographic coordinates of the infected sections with sub-centimetre  Detection accuracies are colour-coded to show the correct and incorrect classifications. Green stands for the correct classifications between point cloud analysis and ground visual assessment. Red stands for incorrect classifications of the healthy canopy as declined categories (stunted or no growth) and vice versa. Orange stands for incorrect classifications between the declined categories. 2. Real grapevine cordon length could be variable. This was caused by the existing remedial surgery practices that cut back some severely infected or dead cordons and extended healthy cordon from another grapevine to fill in the canopy gap (Sosnowski et al., 2011). It can also be caused by the differences that occurred during the initial vineyard set up. Therefore, the actual grapevine boundary could be different to the boundary from point cloud analysis, calculated according to fixed grapevine spacing. The boundary detection in continuous canopy point cloud remains difficult and the accuracy of Eutypa diebacklike symptoms detection at the grapevine level can be reduced as a result.
Potential reasons leading to the classification variations include: 1. Inaccuracies from the point cloud analysis program. As the infected canopy has a smaller size and fewer points in the point cloud than a healthy canopy point cloud, the canopy alpha shape created from a reduced number of points can be less accurate (Park et al., 2005) and the accuracy of the canopy volume may be influenced, as a result. Points representing posts between panels can also be counted as canopy points due to similar height. Although these points have a different colour profile to the canopy points, colour thresholds for more precisely selecting canopy points were not applied in the current study. Increasing the point density during point cloud reconstruction and the additional analysis of point colour may help improve classification accuracy but potentially at the cost of longer processing time and larger point cloud file size.
2. Human error from ground visual assessment as it depends on the experience and judgement of the assessor.
3. Canopy sections with low volume detected by point cloud analysis may also be influenced by other environmental factors (Iland et al., 2011;White, 2015). Ground assessors identified canopy sections with Eutypa dieback-like symptoms according to foliar symptoms and asymptomatic grapevines, even with relatively smaller canopies but healthy, were not recorded. For point cloud analysis, the detection of the infected canopy was solely dependent on the canopy volume and its accuracy can be reduced when canopy growth variations are large across the vineyard. This is considered to be a limiting factor for the current photogrammetry for point cloud reconstruction. Unless the RGB imagery resolution is much higher to be able to reveal the foliage symptom of the Eutypa diebackinduced canopy decline, it remains challenging for point cloud to distinguish between weak growth canopy and Eutypa dieback-induced canopy decline. This range of error for point cloud analysis will also increase with the canopy structure variability in the vineyard.

Implications of mapping Eutypa diebacklike symptoms for vineyard management
Areas with high levels of Eutypa dieback-like symptoms may require remedial surgery to control the further spread of Eutypa dieback and the application of pruning wound protection in nearby rows may become necessary (Sosnowski and Mundy, 2019). The mitigation of Eutypa dieback using remedial surgery practices (e.g., top work to re-establish the cordon or replanting) can be expensive and result in a loss in productivity for several years (Sosnowski et al., 2011). Preventive measures on early symptoms can improve the economics and effectiveness of application (Kaplan et al., 2016). Using point cloud analysis to combine the spatial distribution of Eutypa dieback-like infected canopy, infection severity and their percentages of the total productive canopy may provide important information when making decisions on preventative measures, reworking and whether or not to replant the vineyard when considering overall vineyard performance (Baumgartner et al., 2019;Munkvold et al., 1994).
For the 1.2 ha study block, the ground visual assessment took four experienced assessors eight hours, equivalent to 32 man-hours, and the capture of canopy cover imagery for LAI calculation took three man-hours. In comparison, the UAV flight and the data preparation took around four man-hours and the automatic analysis workflow can complete the analysis of vineyard point cloud in around two hours. However, it is important to note that the development and implementation of the codes needed for the point cloud analysis program took a considerable amount of time to complete and test.
The Eutypa dieback-like symptom detection provided by point cloud analysis offers a convenient and labour-saving alternative to visual assessment. It also suggests that point cloud analysis may be especially useful when assessing large vineyards where performing ground visual assessment can be very costly and not practically feasible. The automatic point cloud analysis process also limits potential human errors when using ground assessments which are related to the experience and judgement of the assessor (Kaplan et al., 2016). The highly precise geographic coordinates of the infected canopy extracted from the point cloud also offer other integration potentials, e.g., variable chemical spray rates for healthy and stunted canopy and the option to stop spraying where the cordon is bare (Wandkar et al., 2018).

Eutypa dieback-like symptom detection through the growing season using point cloud analysis
Across the whole growing season, 11 UAV flights were performed and the percentages of stunted and no growth canopy sections at each flight were calculated from point clouds (Figure 7). The highest levels of stunted and no growth canopy sections were detected at E-L stage 11 in the first flight. This was considered to reflect the natural shoot spacing and variation in shoot development at early development stages (before E-L stage 10) more than Eutypa dieback infection. After the first measurement, the percentages of stunted canopy and no growth sections declined substantially with canopy development. At E-L stage 15, when the ground visual assessment was performed, an infection level of 12.5 % was found and it dropped to around 8 % at E-L stage 17 and 25. The estimated infection levels reduced further to around 5 % at E-L stage 27 and remained stable until harvest (E-L stage 38).
The ideal stage for performing ground visual assessments for Eutypa dieback has been suggested to be between E-L stages 12-17 when the symptoms are most obvious (Bertsch et al., 2013). The point cloud analysis at stage 15 in this study generated similar results to ground measurements. In Figure 7, the percentage of the stunted canopy and no growth sections found after FIGURE 7. The percentages of total canopy length in the study block detected with stunted canopy (yellow) and no growth (red) sections during the 2018-19 growing season, calculated by point cloud analysis.
At E-L stage 15 (highlighted), both the UAV flight and the ground visual assessment were performed. Percentage of symptomatic canopy length (11.4 %) determined by ground visual assessment were shown as the horizontal red dash line. this stage were lower suggest a limitation to the application of point cloud analysis. Therefore, these findings support previous studies, which indicated that a suitable time to apply point cloud analysis is around E-L stage 15.

CONCLUSION
The analysis of the vineyard point cloud at E-L stage 15 can be used to assess Eutypa diebackinduced canopy decline in a vineyard. Canopy health classifications by the point cloud were accurate during this stage when compared with ground visual assessment (87.4 %). Point cloud analysis also supplied the precise geographic coordinates of the infected canopy to map the spatial distribution of Eutypa dieback-like symptoms and facilitate the application of remedial practices. The automatic analysis of the point cloud is more convenient and less labour-intensive than ground visual assessment. However, factors including existing canopy remedial surgery, limited point density and the vineyard environmental conditions can influence the accuracy of point cloud analysis.
Results from this study suggest that E-L stage 15 is suitable for Eutypa dieback detection using point cloud analysis as it aligns well with visual assessments. The number of infected canopy sections this method can detect declines later in the season as the canopy develops.