UAV and ground-based imagery analysis detects canopy structure changes after canopy management

Aim: To analyse unmanned aerial vehicle (UAV)-based imagery to assess canopy structural changes after the application of different canopy management practices in the vineyard. Methods and results: Four different canopy management practices: i–ii) leaf removal within the bunch zone (eastern side/both eastern and western sides), iii) bunch thinning and iv) shoot trimming were applied to grapevines at veraison, in a commercial Cabernet-Sauvignon vineyard in McLaren Vale, South Australia. UAV-based imagery captures were taken: i) before the canopy treatments, ii) after the treatments and iii) at harvest to assess the treatment outcomes. Canopy volume, projected canopy area and normalized difference vegetation index (NDVI) were derived from the analysis of RGB and multispectral imagery collected using the UAV. Plant area index (PAI) was calculated using the smartphone app VitiCanopy as a ground-based measurement for comparison with UAV-derived measurements. Results showed that all three types of UAV-based measurements detected changes in the canopy structure after the application of canopy management practices, except for the bunch thinning treatment. As expected, ground-based PAI was the only technique to effectively detect internal canopy structure changes caused by bunch thinning. Canopy volume and PAI were found to better detect variations in canopy structure compared to NDVI and projected canopy area. The latter were negatively affected by the interference of the trimmed shoots left on the ground. Conclusions: UAV-based tools can provide accurate assessments to some canopy management outcomes at the vineyard scale. Among different UAV-based measurements, canopy volume was more sensitive to changes in canopy structure, compared to NDVI and projected canopy area, and demonstrated a greater potential to assess the outcomes of a range of canopy management practices. Significance and impact of the study: Canopy management practices are widely applied to regulate canopy growth, improve grape quality and reduce disease pressure in the bunch zone. Being able to detect major changes in canopy structure, with some limitations when the practice affects the internal structure (i.e., bunch thinning), UAV-based imagery analysis can be used to measure the outcome of common canopy management practices and it can improve the efficiency of vineyard management.


INTRODUCTION
Among the vineyard management practices, canopy management is widely applied to regulate canopy growth, reduce disease pressure, improve bud fertility and improve berry quality (Dry, 2008;Mirás-Avalos et al., 2017;Trought et al., 2017;Wolf et al., 2003). Commonly applied canopy management practices such as leaf removal, shoot trimming and bunch thinning aim to modify the source-sink relationship by reducing leaf density and/or crop load (Smart & Robinson, 1991). By selecting different practices and their levels/ intensity of application, canopy treatments can have various outcomes. Low levels of input often have a limited impact on the canopy structure and are inefficient. In contrast, excessive application can negatively impact yield and quality including increasing the risk of exposing the crop to extreme weather conditions, such as heat waves (Caravia et al., 2016;Reynolds et al., 2005;Vasconcelos & Castagnoli, 2000).
To effectively apply canopy management practices, it is crucial to have convenient and accurate assessments of their outcomes. One approach could be to directly compare the differences in canopy structure before and after the application of canopy management practices, through the assessment of parameters such as plant area index (PAI: total leaf and cordon area per unit ground area) (Bréda, 2003;De Bei et al., 2016) and canopy volume. However, the most accurate estimations for these parameters involve destructive and labour-intensive sampling practices in the field (Gower et al., 1999;Jonckheere et al., 2004). To overcome these disadvantages, there have been recent developments in smartphone apps that analyse upward-looking canopy cover imagery and offer an objective and accurate solution to the measurement of PAI (De Bei et al., 2016;Fuentes et al., 2014;Poblete-Echeverria et al., 2015). These tools, that estimate PAI, have been effectively applied to assess changes in canopy structure during the growing season Wang et al., 2019).
Alternatively, optical remote sensing using unmanned aerial vehicles (UAV), aircraft and satellite platforms can be applied to estimate canopy structure (Hall, 2018). Amongst these platforms, recent advancements in UAV related research have led to a wide range of UAV applications for monitoring vineyard performance such as the rate of canopy development, canopy structure spatial variability, disease incidence and canopy water status (Albetis et al., 2018;de Castro et al., 2018;Mathews & Jensen, 2013;Pádua et al., 2018;Romero et al., 2018;Su et al., 2016). Through the collection of highresolution red/green/blue (RGB), multispectral or hyperspectral imagery, UAV-mounted sensors are capable of providing data to create highresolution RGB and spectral indices maps at the vineyard scale, e.g., normalized difference vegetation index (NDVI) and plant cell density (PCD) maps (Xue & Su, 2017). In addition, threedimensional digital models, including vineyard point cloud and digital canopy model (DCM), can be created from overlapping images captured by the UAV (Comba et al., 2018). From vineyard digital models, parameters such as canopy height, projected area and volume can be calculated and provide detailed information regarding the canopy structure (Matese & Di Gennaro, 2018;Weiss & Baret, 2017). Compared with manned airc raft and satellite-based remote sensing, UAV also offers convenience in simple flight preparation, flexible operation options (Khaliq et al., 2019) and is more cost-effective for small and mediumsize vineyards (Andújar et al., 2019;Matese et al., 2015). These advantages can help obtain a prompt evaluation of canopy management outcomes during critical developmental stages. Despite the demonstrated potential of these techniques, applying UAV remote sensing for measuring canopy management outcomes is currently limited.
This study aimed to assess whether UAV remote sensing can detect canopy structure changes after the application of different canopy management practices. These assessments were compared to ground-based PAI measured at the same time as the UAV flights. The advantages and limitations of using UAV as a monitoring platform in evaluating canopy structure changes are discussed.

Study vineyard and experimental design
A commercial Cabernet-Sauvignon vineyard in the wine region McLaren Vale, South ) was used for the study during the 2017-18 growing season. The climate of the region is classified as Mediterranean with low summer rainfall and the soil type is red/brown loamy sand (Wine Australia, 2020). In the study vineyard, vine and row spacings were 2 m and 3 m, respectively, and row orientation was north to south. Vines were trained with spur pruning and develops a sprawling canopy at the approximate width of 1.3 m and the cordon height of 1.1 m. The commercial vineyard was managed with standardised management practices, drip irrigated and no cover crop was grown in the mid-row or under-vine.
At veraison (E-L stage 35), four canopy management treatments were applied: i) leaf removal on the eastern side of the canopy, ii) leaf removal on both sides of the canopy, iii) bunch thinning and iv) shoot trimming on the eastern side of the canopy (Table 1). Removing leaves and/or shoot trimming on the eastern side only is often performed to reduce canopy density while protecting the bunches on the western side from the intense afternoon sunlight (Reynolds & Vanden Heuvel, 2009). A randomised replicate design was used to apply the treatments in the vineyard. As shown in Figure 1, each replicate consisted of six vines (two panels) and the different coloured sections in the figure correspond to different treatments. Each treatment was replicated six times; 180 grapevines in total were monitored and measured in the study.

Ground and UAV-based Imagery Acquisition and Analysis
To measure the impact of treatments on canopy structure, a combination of ground and aerial imagery analysis was used. Measurements were taken: i) before treatment application (E-L stage 35), ii) one week after treatment application and iii) at harvest (E-L stage 38). For the ground measurement, two images were acquired from the middle vine in each panel, one on the left and one on the right side of the trunk. A total of 24 images were acquired per treatment at each measurement. From these images, PAI was calculated using the smartphone app VitiCanopy (University of Adelaide, South Australia, Australia). The ground imagery was captured by the frontal RGB camera of an Apple iPhone 7 at a resolution of 7.2 megapixels (Apple Inc., Cupertino, CA, USA), detailed procedures for image capture and PAI calculation can be found in De Bei et al. (2016). The PAI was also calibrated against real LAI, measured by destructive leave removal measurements in the same study site, and their correlations can be found in the supplementary information.
UAV flights were performed to acquire RGB and multispectral imagery of the study vineyard. RGB imagery was captured by the RGB sensor of a Phantom 4 Pro quadcopter (DJI, Shenzhen, China) at a resolution of 20 megapixels. Multispectral imagery was acquired by the Parrot Sequoia multispectral sensor (Parrot SA, Paris, France) recording spectral bands at green (550 nm), red (660 nm), red edge (735 nm) and near-infrared (790 nm) bands, at a resolution of 1.2 megapixels. The Sequoia camera was integrated into a Phantom 3 Adv (DJI, Shenzhen, China) quadcopter using a customized integration package (MicaSense, v1.3, Seattle, USA). The multispectral images were radiometrically calibrated by the onboard downwelling sunlight sensor during the flight and a calibrated reflectance panel on the ground (MicaSense, Seattle, USA). On a clear day, the flights were conducted at solar noon to minimise any shadow effect. The single-grid routes covering all replicate panels at an overlap ratio of 80 % were set. The aircraft was maintained at a height of 30 m above ground and a constant speed of 2 m/s during the flight. The geographic coordinates of images captured were recorded by the onboard Global Positioning System (GPS) receiver.
The commercial photogrammetry software PIX4Dmapper (v.4.4.12; Pix4D SA, Lausanne, Switzerland) was used for UAV imagery  Figure 2). Using multispectral images, single-band orthomosaic images were created from images of each spectral band. The average ground sampling distances (GSD) of the RGB and single-band orthomosaic images were 0.8 cm and 3.2 cm, respectively. Using a customized processing procedure in ArcGIS (v.10.5.1; ESRI, Redlands, CA, USA), the DSM was normalized to generate the digital canopy model (DCM) rasters.
Using the image processing toolbox in Matlab (v.2019b, Natick, Massachusetts, United States), green canopy pixels in the RGB orthomosaic imagery were extracted according to the Lab colour space profile to create the canopy layer for projected canopy area calculation.
To create the normalized difference vegetation index (NDVI) raster for the study vineyard, a calculation was performed using singleband orthomosaic images as follows: (Rouse et al., 1974) where: NIR = Near-infrared band orthomosaic image; Red = Red band orthomosaic image.
With the NDVI raster for the whole vineyard, unsupervised classification was performed to extract the canopy-related pixels using the Iso Cluster Unsupervised Classification function in ArcGIS. As a result, the raster was classified into two classes: background and grapevine canopy.
The canopy class was extracted as the classified NDVI raster for the calculation of mean NDVI per vine.
Vineyard rasters of DCM, RGB imagery of green canopy layer and classified NDVI containing only canopy-related pixels were stored in the tagged image file format (.tiff) image files. Rasters were geo-referenced using ground control points and experimental replicates were marked using the high-resolution RGB imagery by identifying posts separating individual panels. Polygon vectors for replicates were then created and used as masks to extract replicate rasters. According to the fixed planting distance, single grapevine rasters were further separated from the individual replicate rasters. Using the single vine rasters, canopy volume per vine (m 3 ) were calculated from the DCM using volume above the cordon height plane (1.1 m above ground). Projected canopy area per vine (m 2 ) was calculated using the sum of the single canopy pixel area in the RGB green pixel layer. The mean NDVI per vine was calculated from the mean NDVI values of all extracted canopy class pixels.

Statistical Analysis
Analysis of variance (

RESULTS AND DISCUSSION
With the application of canopy treatments, changes in the canopy structure were observed (Figure 3). Significant reductions in the mean canopy volume per vine were observed after leaf removal (LR-E and LR-B) and shoot trimming (ST-E). Leaf removal and shoot trimming applied to one side of the canopy (LR-E and ST-E) had also lowered canopy volume (from 1.29 m 3 to 1.06 m 3 and from 1.37 m 3 to 1.12 m 3 , respectively). Doublesided leaf removal reduced canopy volume (from 1.42 m 3 to 0.82 m 3 ) more than the single-sided application. No difference in canopy volume was found when bunch thinning was applied and compared to the control group. This was expected as bunch thinning only removed bunches at the bunch zone inside the canopy which was covered by leaves and shoots, meaning the overall dimensions of the canopy were not altered. At harvest, the canopy volume of all treatment groups declined to significantly lower levels than after treatment application. The general decline in the canopy volume can be explained by the seasonal reduction in canopy development due to leaf senescence leading to leaf fall and the reduction in irrigation application (Pádua et al., 2018).
Similar to canopy volume, projected canopy area and NDVI values in the treatment groups where leaf removal and shoot trimming were applied (LR-E, LR-B and ST-E) were reduced while the control and bunch thinning groups (C and BT) remained relatively unchanged.
Ground-based PAI measurements recorded significant differences in all treatment groups, including the bunch thinning treatment which was not detected by the UAV based remote sensing approaches. PAI was calculated from upwardlooking canopy cover imagery (De Bei et al., 2016), and was shown to be capable of detecting the changes of internal canopy structure under foliage cover, such as the removal of bunches ( Figure 4). In addition, canopy porosity, which is an important indicator of the light conditions inside the canopy and closely related to PAI, can also be generated from the canopy cover imagery (De Bei et al., 2016). Thus, PAI was found to be more advantageous in detecting variations in both internal and external dimensions in the canopy structure than remote sensing approaches which only detected external dimension and spectral value variations.
Comparing different measurements for the same treatment group, canopy volume and PAI were found to detect greater differences in canopy structure after canopy management practices were applied. When comparing measurements taken before and after treatment application ( Figure 5), canopy volume and PAI demonstrate the largest percentage changes for leaf removal and shoot trimming treatments. In contrast, very little change was observed in NDVI values with the largest decline of 13 % when double-sided leaf removal was applied. The same treatment recorded reductions of 43 % and 30 % in canopy volume and PAI measures, respectively. At harvest ( Figure 5), canopy volume recorded the biggest percentage change across all measurements, followed by PAI. This may have been accentuated by leaf fall. Canopy volume and PAI measures detected greater differences in canopy structure and may be more useful measures for making informed decisions and determining the effectiveness of canopy management practices.   The treatments used in this study showed the limitations of NDVI and canopy area measurements in detecting changes in canopy structure. As discussed previously, bunch thinning cannot be detected by remote sensing approaches due to foliage cover. In addition, it was also found, in the shoot trimming group (ST-E), that the leaves and portion of shoots that were trimmed and left on the ground were detected by the NDVI and canopy area measurements using unsupervised classification (Figure 6a and b). The portion of the canopy that was removed at trimming likely had similar spectral properties to the canopy that remained on the vine. When unsupervised classification was applied, both pixels on the grapevine and on the ground (residual pixels) were extracted and these residual pixels reduced the accuracies of NDVI and canopy area. For the canopy area, residual pixels increased the total canopy pixel number for the measured grapevine and, as a result, increased the canopy area. By setting higher thresholds in the Lab colour space profile when extracting canopy pixels, some residual pixels in the RGB orthomosaic were filtered out (Figure 6c). Nonetheless, strict thresholds also filtered out part of the canopy pixels and resulted in the underestimation of the projected canopy area. With these measurements already taken one week after the shoots were trimmed, they are unlikely to be capable of providing accurate real-time assessments of the canopy structure immediately after shoot trimming. Therefore, it is suggested that NDVI and canopy area measurements using UAV-based two-dimensional rasters (e.g., spectral indices and RGB rasters) for the assessment of shoot trimming should be avoided to minimise any potential errors.
In Figure 6d, DCM was shown to contain only the grapevine canopy after filtering out shoots left on the ground during DSM normalization. Without the interference from ground residual pixels, the canopy volume calculated from the DCM reflected more accurately the volume measurements of the grapevine canopy. Canopy volume measurements can also be obtained immediately after canopy management practices are applied without the interference of the trimmed shoots or leaves on the ground, unlike NDVI and projected canopy volume measures. In addition, although no ground vegetation (weed or cover crop) grew in the study vineyard, it has been found that DCM can also filter out the interference from ground vegetation as only the partial volume of the DCM that was above the cordon height was included (Vanegas et al., 2018;Weiss & Baret, 2017). With these advantages, canopy volume calculated from DCM displayed potential as a suitable tool for monitoring canopy management outcomes.
Compared with UAV, the ground-based PAI measurement has a simpler process and offers the convenience of collecting and analysing the data quickly. It can assess the outcomes of the canopy management practice and report the results in the field which allows growers to promptly adjust their management practices. Nonetheless, as the canopy cover imagery for the PAI calculation was collected discretely between vines, the sampling distances between images should be considered in conjunction with target map resolution when using this method for mapping the spatial variability. By integrating the ground imagery acquisition system with vineyard vehicles can potentially enable the on-the-go canopy structure assessment, as explored in previous studies (Bramley et al., 2007;Liu et al., 2017;Rose et al., 2016). However, these systems often require regular calibrations during the operation for capturing suitable imagery under the dynamic light conditions in the field and more research is required to improve the robustness of the integrated system.
Between UAV-based remote sensing approaches, the processes for data collection and primary analysis were similar, both in terms of the UAV flight and the vineyard raster reconstruction. For canopy volume, DSM needs to be normalized to create the DCM which was more complicated than the canopy pixel classification and extraction for the NDVI and projected canopy area calculation. However, the collection of multispectral imagery for NDVI calculation required integrating extra multispectral sensors while the DCM was created from the RGB imagery captured by the originally on-board RGB sensor. Therefore, the cost and labour required for the acquisition and integration of multispectral sensors can be avoided by using DCM. In addition, DCM can also be used to provide the surface area of the canopy and knowing the volume and surface area of the canopy can potentially be useful for guiding other canopy management operations. For example, it can be used to advise the chemical spray volume required and the spray application rate can be adjusted during the operation according to the canopy size (Llorens et al., 2010;Llorens et al., 2011). The latter authors proposed the use of ultrasonic and LIDAR sensors for the proximal sensing of canopies in vineyards and concluded that these sensors could provide valuable information on canopy volume and leaf are index. However, they also concluded that the post-processing of LIDARacquired information can be a limiting factor to its usability. Similarly, the work from Siebers et al. (2018) has demonstrated that proximal sensing in viticulture could provide useful information on canopy architecture; the authors developed a proximal sensing platform, the Grover, equipped with LiDAR but with the capability to host and test multiple sensors. Diago et al. (2019) proposed the use of an on-the-go system to collect RGB images for the assessment of grapevine canopy FIGURE 6. A comparison of different remote sensing approaches for extracting the grapevine canopy.
Rasters of the canopy that received shoot trimming, captured one week after the application at veraison (E-L stage 35), are shown: (a) NDVI raster (overlaying RGB orthomosaic image), (b) canopy pixel raster for calculating projected canopy area, (c) canopy pixel raster with the higher extraction threshold for green pixels and (d) digital canopy model (DCM) for calculating canopy volume. Note the trimmed shoots and leaves left on the ground (residual pixels) were captured by NDVI (a) and canopy area measurements (b). Canopy pixel extraction with the higher threshold (c) filtered out most ground residual pixels but also reduced the actual projected canopy area in the canopy zone. In comparison, DCM (d) can effectively filter out ground interference.
parameters with successful results. Similar to our findings an on-the-go system can be used as a tool to assess the efficiency of canopy management operations.
UAV-based remote sensing can be used to provide assessments of the whole area before or after the application of canopy management practices to advise the optimal input level and assess outcomes. Various flight control software also helps maintain the course and speed of the aircraft automatically during the flight which greatly reduces the difficulties and complexities in capturing overlapped imagery suitable for the reconstruction of orthomosaic imagery using photogrammetry. The operational flexibility of UAV also allows the timely assessment of the canopy management outcomes, compared with manned aircraft and satellite remote sensing.

CONCLUSION
The UAV-based canopy volume assessment was able to account for differences obtained through canopy management, specifically the canopy structure. Canopy volume is more sensitive to changes in the canopy structure compared to NDVI and projected canopy area. It is also a more cost-effective measure than the multispectral index NDVI, where a multispectral sensor is required, increasing costs of hardware and analysis and interpretation requirements. The accuracies of NDVI and projected canopy area measurements were negatively impacted when practices such as shoot trimming, which leave plant material on the ground were applied. Due to the cover of foliage, UAV-based measurements cannot be used to measure the impact of bunch thinning on canopy structure. However, UAV-based approaches can be used to provide vineyard scale measurements that cover all the grapevines in the vineyard and have the potential to be integrated with other vineyard management practices, such as targeted chemical spraying. PAI calculated from ground-based canopy cover imagery can be used to measure all three types of canopy management practices (leaf removal, shoot trimming and bunch thinning) assessed in this study and was a convenient approach for in-field assessments.