Introduction

Quantitative Descriptive Analysis (QDA) by a trained panel is often considered the reference method for assessing the sensory characteristics associated with a product (Sidel, 2004). However, this method is both time-consuming and expensive due to the amount of training it requires. There is thus an ongoing search for alternative methods of sensory analysis that are easy, fast, and cost-effective.

In recent years, sensory characterisation has become increasingly oriented toward consumers (Ares and Varela, 2017), and several methods have been developed for use with non-specialist panels, such as Check-All-That-Apply (CATA) or Rate-All-That-Apply (RATA) (Ares et al., 2014; Danner et al., 2017; Meyners et al., 2016; Vidal et al., 2018; Copper et al., 2019; Mezei et al., 2021; Pineau et al., 2022). While these methods are not as accurate as the descriptive analysis, Danner et al. (2017) reported that when the RATA method was applied to wines with a panel of untrained consumers, it generated results that were, to a large extent, similar to those obtained from a descriptive analysis carried out with a trained panel.

With regard to the wine sector, sensory methods have also been adapted for use with panels composed of professionals without collective training (Campo et al., 2010; Coulon-Leroy et al., 2017; Lawrence et al., 2013; Longo et al., 2020; Perrin et al., 2008). Other proposed methods of sensory characterisation of wine are based on the free description, such as free profile methods (Perrin et al., 2008; Varela and Ares, 2014) or mixed profile methods (Coulon-Leroy et al., 2017). In these, each subject freely generates their own attributes to describe wine, in addition (or not) to a predefined list. Free comment methodology has been successfully used for the description of wine by both consumers and professionals (Lawrence et al., 2013; Mahieu et al., 2020). The advantage of these methods is that they are more flexible, offering greater freedom to the subjects. However, they also generate a high number of individual attributes, which makes data aggregation difficult and can hinder the interpretation of the results.

Wine presents an aromatic complexity (Spence and Wang, 2018) and to simplify the wide spectrum of aroma terms, they are often presented following a hierarchical arrangement or grouped in a manner that structures and manages a large number of attributes. Categories and aromatic terms can be represented as an aroma wheel, as proposed by Noble et al. (1984) and Noble et al. (1987), Sáenz-Navajas et al. (2021), or arranged in a table, as in Caillé et al. (2017), Campo et al. (2008), Campo et al. (2010), or Bindon et al. (2014). With these approaches, terms are structured in such a way as to create nested groups of general and specific attributes. For example, the group ‘Fruity' includes the sub-group ‘Citrus', which, in turn, includes the attributes ‘Lemon' and ‘Orange'.

Initially, the purpose of Noble's wheel was to align the various terminologies proposed to describe the aromatic characteristics of wines, to facilitate communication between winemakers, researchers, marketers and consumers (Lawless and Civille, 2013; Noble et al., 1984). Some authors have used it as a basis for training panellists, as in the work of Caillé et al. (2017) or Bindon et al. (2014). The main benefits of nesting are that it reduces the number of attributes to be considered and can facilitate the standardisation of terms among studies or panels. For this reason, the nesting of attributes is a common step in the training session of a panel. Nowadays, using hierarchical links between attributes to simplify sensory description is common practice in sensory analysis (Bindon et al., 2014; Caillé et al., 2017; Larssen et al., 2018; Villière et al., 2018). However, to the best of our knowledge, information on the hierarchical structure of attributes (general and specific) is lost in the process of data acquisition or data analysis. In Campo et al. (2008), the authors proposed a sensory descriptive analysis based on citation frequencies, which involved a large number of terms structured in categories. However, the evaluation was mainly focused on specific attributes considered as independent. Even if they considered the hierarchical nature of the terms when they built their contingency table, no strategy was proposed to organise the information from specific attributes to categories they belonged to and considered the proximity between the attributes.

This study aimed to go further by incorporating the hierarchical structure of a wine-odour lexicon into the analysis of a sensory descriptive task and not just by using the structure during the characterisation by the panel. Indeed, it is not easy to assess all attributes quantitatively (as in quantitative descriptive analysis) due to the high aromatic complexity of wines. Nevertheless, we thought that the hierarchical method could allow assessing the aromatic intensity and finally increases the discrimination power because only the selection of the most salient attributes would be expected (Vidal et al., 2018); we chose to base our approach on RATA methodology. In comparison to a classical RATA analysis, which includes between 15 and a maximum of 50 attributes (e.g., Ares et al., 2018), we used a total of 118 attributes. Therefore, we propose here a Hierarchical-Rate-All-That-Apply method (HRATA), including techniques for data acquisition and data treatment, for use in generating reliable aromatic characterisations of wine in a professional context (wine competitions, sensory controls as part of the regulated quality schemes, sensory benchmarking by companies or Geographical Indications unions).

The study is divided into two parts:

Firstly, we will present the implementation of the HRATA approach. Particular attention will be paid to the statistical treatments used for HRATA data and the adaptations of the RATA method of data treatment (Meyners et al., 2016) to the specificities of the HRATA dataset.

Secondly, we will present sensory descriptions obtained using the HRATA method for characterisation of the aromatic profiles of rosé wines. Finally, the results obtained using the HRATA approach will be compared to those of a descriptive analysis performed on the same set of products.

Materials and methods

3. Hierarchical-Rate-All-That-Apply task (HRATA) methodology

The main goal of HRATA is to propose a hierarchical structure of attributes that depicts the set of samples involved in the evaluation. Subjects can freely select as many attributes (family, category, and/or terms) as they wish, depending on their individual appreciation (Figure 1).

Figure 1. A screenshot showing 118 attributes could be checked off and evaluated on a 3-point intensity scale (FIZZ software, Biosystèmes®, 1990).

Save View full size Expand inline Collapse inline

The family name and category names were also available for selection if judges could not select specific attributes under each. Each subject is then asked to rate (three-point scale) the perceived intensity of each selected attribute.

As the objective was firstly to develop the method and not to characterise the wines too thouroughly and secondly not to tire out the panel too much, we have chosen not to have a complete description of the wines but only to focus on the lexicon of odours. Judges were asked to focus only on orthonasal aromatic description.

3.1. Subjects

Sixty-six subjects (41 wine consumers, 13 students in viticulture and oenology, and 12 professionals from the wine sector) were recruited for the HRATA experiment. All subjects possessed a minimum level of wine knowledge, which was evaluated using a questionnaire similar to that presented by Koenig et al. (2020). Subjects with an objective knowledge score greater than or equal to 8/14 were retained for the experiment. The type level of expertise was not considered during the data analysis. As the objective is to use HRATA in a professional context and without consuming too much time, subjects did not receive any collective or specific training in the description of rosé wine before the HRATA task.

Table 1 depicts the socio-demographic characteristics of the subjects involved in the experiment.

Table 1. Frequency table of gender, age, and occupation within the panel (%, n = 66).

View popup Expand inline Collapse inline

Characteristic	Modalities	Frequency (%) in the sample
Gender	Female	48
	Male	52
Age	Less than 40 years old	50
	Between 40 and 60	38
	More than 60 years old	12
Type of Subjects	Wine consumers	62
	Students in viticulture and oenology	20
	Professionals from the wine industry	18

3.2. Hierarchical structuration

Attributes used for the description of wines were arranged according to the hierarchical structure presented on the left-hand side of Table 2. These attributes of HRATA were selected and hierarchically structured according to the results of Koenig et al. (2021). The hierarchical structuration was aimed to be a general structure for wine description and not specific to the wines from the study. Here, the term ‘families' is used for more general attributes (e.g., ‘fruit'), ‘categories' refers to intermediate attributes (e.g., ‘red fruit' or ‘white/yellow fruit'), and ‘terms' are the more-specific attributes (e.g., ‘blackcurrant', ‘cherry', or ‘apricot'). Thus, if a subject specifically recognised the odour of ‘lemon', s/he could select it on the software interface. If subjects only perceived a fruity odour, without a specific identity, they could select the attribute ‘fruit' or ‘citrus'.

Table 2. On the left are the attributes used for HRATA analysis; on the right are the attributes used for descriptive analysis (DA). To better compare the two methods, the rows indicate the likely equivalence between DA and HRATA attributes; the DA attributes were fitted into the same hierarchical system for interpretation but not for data capturing or analysis (in italics, attributes fitting several categories).

View popup Expand inline Collapse inline

HRATA attributes			DA attributes
Family f	Category c	Terms t
Fruit_f	Red_fruit_c	blackcurrant–cherry–strawberry–raspberry–black fruit–currant–blackberry–blueberry–fig–ripe_fruit	red_ black_fruit ripe_compote_confit_fruit
	White_yellow _fruit_c	apricot–stone_fruit–white_fruit–yellow fruit–melon–peach–pear–apple–plum–grape	white_yellow_ fruit ripe_compote_confit_fruit
	Citrus_c	lemon–lime–orange–grapefruit	citrus ripe_compote_confit_fruit
	Tropical_fruit_c	pineapple–passion_fruit–lychee–mango–coconut	exotic_fruit ripe_compote_confit_fruit
	Amylic_c	english candy–banana	amylic
Floral_f	Floral_c	acacia–orange_blossom–jasmine–lilac - rose - violet	flowers dried_fruit_flower_vegetal
Spicy_f	Mentholated_c	licorice–anise–mint	spice_aromatics
	Spice_c	cinnamon–clove–pepper–black_olive
Vegetal_f	Aromatic_plant_c	laurel–mint–tea–thyme	vegetal dried_fruit_flower_vegetal
	Vegetal_c	grass–cut grass–green pepper–green
Roasted_f	Lactic_c	butter–yeast–bread	empyreumatic_lactic_woody
	Dried_grass_c	hay–tobacco–tea
	Burned_c	coffee–smoke–roasted–toasted_bread
Mineral_f	Mineral_c	flint	mineral
Chemical_f	Chemical_c	alcohol–rubber–nail polish remover–oil–sulfur–vinegar	faults
Undergrowth_f	Animal_c	sweat–leather	animal
	Woody_c	oak–cork–pine	empyreumatic_lactic_woody
	Earthy_c	mushroom–cork–musty–undergrowth–truffle–dust	mushroom faults
Sweet_f	Sweet_c	caramel–chocolate–honey –vanilla	dried_fruit_flower_vegetal sweets
	Dried_fruit_c	dried apricot–almond–hazelnut–nut

3.3. Application of HRATA to wine characterisation

The six samples (five wines plus a duplicated sample for one) were characterised according to the HRATA methodology. From the table of 118 attributes, the 66 subjects were instructed to select all attributes that corresponded to odours perceived in the wine. As our aim was not to characterise products exhaustively but to test the HRATA approach, we focused on orthonasal aromatic description, which is a less-demanding sensory task.

A full hierarchical structure of the attributes was presented to the subjects on a paper sheet to have an overall view of the attributes and their structuration. The main objective of HRATA is to propose a hierarchical structure of attributes representing all the samples concerned by the evaluation. At this stage, this structure is materialised using an exploration interface which consists in identifying each family by a tab grouping all the attributes belonging to it. In the same way, the subject can interactively choose an attribute to visualise the nested terms associated with it. During the evaluation phase, the subject can thus select, without constraint, as many attributes (family, category and/or terms) as they wish according to their individual sensitivity (Figure 1). For each attribute selected (family name, category name, or at the most specific level a term), subjects were next asked to assign an intensity score on a three-point scale (‘low’, ‘medium’, and ‘intense’). This prototype interface was designed in Fizz software (Biosystèmes®, 1990) to respond specifically to the desired structuring of HRATA. Families and categories were presented as tabs, and the attributes belonging to each family (or category) were listed in a corresponding tab. It was easy to switch from one table to another to find an attribute.

Figure 2. Sheets available for subjects to help them visualise all the attributes and their hierarchical organisation, during the HRATA experiment, as the software interface was still at a prototype stage.

Save View full size Expand inline Collapse inline

3.4. Data coding of the raw HRATA evaluations

As the HRATA subjects could rate the set of products with different attributes and, most importantly, at different levels of generality, we designed a dedicated statistical processing strategy for HRATA evaluation. Compared to conventional RATA data, our dataset was composed of a larger number of attributes (118, comprising 89 terms, 20 categories, and 9 families), of which only a few were likely to be selected. Consequently, our HRATA dataset contained a much larger proportion of non-evaluated data than what is usually encountered with RATA. To address this, an initial coding step was performed and the statistical techniques usually applied to RATA data (see Meyners et al., 2016) were adapted to this specific data configuration.

The HRATA data analysis process, including the coding pre-treatment, is illustrated in Figure 2 and detailed below.

Figure 3. Schematic representation of the HRATA data analysis, including a coding step designed to (b) integrate the hierarchical structure and (c) apply a rule for non-evaluated data. (d) Data were then subjected to logistic regression for each attribute, involving subsets of subjects and computation of the Dravnieks scores for PCA. For illustrative purposes, the evaluation of six attributes for three wines is depicted. Family-level attributes are underlined and in bold; category-level attributes are in bold.

Save View full size Expand inline Collapse inline

3.4.1. Step 1: Data coding with the integration of the hierarchy

HRATA data were collected in a product x attribute data table for each subject with intensity scores ranging from 1 (low) to 3 (high) (Figure 3(a)). If an attribute was not selected by a subject for a given product, then the corresponding cell was left empty (as for the ‘Fruit’ attribute for Wine A in the example of Figure 3(a)). At this stage, the data array included all the attributes selected, regardless of their nature, i.e., without taking into account the hierarchical structure.

To incorporate the hierarchical structure of the attributes, a first coding step was used to aggregate intensity ratings from the lower hierarchical levels to the higher ones, as explained below (Figure 3(b)). A second coding step was then used to address non-evaluated data (Figure 3(c)).

The hierarchical structure of the data was accounted for using the following rules:

(1) If a subject rates a category, then the rating provided to the category is imputed to the family to which it belongs.

(2) If a subject rates a term, then the rating provided to the term is imputed to both the category and the family to which it belongs (following rule (1)).

In practice, if several terms were evaluated within a category (or family), an aggregation rule was applied to combine the various scores at the lower level into a single value at the higher one. More specifically, the maximum aggregation rule was adopted:

(3) If several terms are evaluated within a single category, then the category is assigned the highest value given to any term nested within it. The same reasoning applies to the family level.

Furthermore,

(4) If a subject scores both a category and at least one of the terms belonging to it (or a family and at least one category within it), then the same rule is applied: the highest value of the different scores is assigned to the category (or family).

In this way, a product x attribute data table was computed for each subject that represented the hierarchical structure of the odour terms used in the evaluation.

3.4.2. Step 2: Rules for non-evaluated data and attributes

At this stage, each table contained numerous non-evaluated attributes. In a typical RATA data treatment, the non-evaluated attributes are commonly imputed with a value of 0 (Ares et al., 2018; Danner et al., 2017; Meyners et al., 2016; Oppermann et al., 2017; Vidal et al., 2018). However, this can have a significant impact on the results when the number of non-evaluated attributes is large. Moreover, one can argue that if an attribute was not evaluated by a subject, this does not necessarily mean that the subject considered this attribute to have an intensity of zero; instead, it could simply mean that they did not take it into account, given the large number of attributes available.

A high frequency of zero values can bias usual indicators of central tendency and dispersion. This potential for bias was amplified here because of the larger number of attributes considered; typical RATA datasets generally comprise between 15 and 50 attributes (e.g., Ares et al., 2018), while our dataset contained 118 attributes (9 families, 20 categories, and 89 terms). The number of attributes available for selection was, therefore, much greater than the number of attributes evaluated by each subject for each wine.

To limit this bias, an additional coding rule was defined as follows:

(1) If an attribute was never selected by a subject for any product, then the attribute qualifies as non-considered (NC) (as for ‘mango’ in Figure 3(c)).

(2) If an attribute was selected by the subject for at least one product, then the non-evaluated data for the other products are considered to be of zero intensity and imputed as 0.

This imputation rule aimed to differentiate among attributes that, on the one hand, were simply not perceived by a subject in a given product and, therefore, should have an intensity of zero and, on the other hand, those that were never taken into consideration for any product and should therefore correspond to NC.

This rule made it possible to filter the initial table by only retrieving the attributes selected by a subject, i.e., those with at least one assigned value for one product. As a result, the number of attributes considered varied among subjects. Finally, the retained attributes were transformed from three-point intensity variables to four-point intensity variables {0,1,2,3} to take into account the imputation rule.

3.5. Statistical data analysis of recorded HRATA data

The data obtained after these coding steps were then analysed to identify the attributes that could significantly differentiate among the various products and to obtain a product map.

3.5.1. Determination of discriminant terms, categories, and families

When dealing with RATA data, in most cases, the failure to meet assumptions of data normality does not prevent the use of analysis of variance in testing whether or not an attribute is discriminant. Indeed, Meyners et al. (2016) showed, based on the comparative analysis of two consumer studies using the RATA question, that this type of test is robust for analysing RATA data. Tests for product difference usually gave the same results when RATA data were analysed with ANOVA and Cochran's Q test. However, due to a large number of attributes proposed in our HRATA approach, a very pronounced asymmetry was found in the distribution of the scores for a large majority of low-level attributes (terms), with a high frequency of zeros; this made ANOVA an inappropriate choice for our analysis.

As an alternative, then, we decided to assess the frequency of use. For a given attribute, the question was whether its frequency of selection differed significantly among products and among subjects. Attributes were, therefore, transformed from four-point intensity variables to binary variables (0 for zero-intensity and 1 otherwise, i.e., with an intensity value of 1, 2, or 3). A logistic regression was then performed for each attribute, with subject and product as independent factors (Dobson and Barnett, 2008). As mentioned above, subjects who did not consider an attribute were excluded from the analysis of that attribute so that the number of subjects for each attribute varied. From the logistic regression outputs, discriminant attributes were identified using a level of risk, α, of 5 %.

3.5.2. Multidimensional descriptive analysis

A multidimensional descriptive analysis allows products to be described based on a set of attributes. To this end, data were aggregated for each attribute across all subjects who scored that attribute. Dravnieks' score was used to take into account both the frequency of selection of the attribute and the intensity values assigned (Dravnieks, 1982; Vidal et al., 2018). For the calculation of this score, the frequency of attribute selection and the sum of intensity values were expressed as a percentage of the maximum possible values for each of the respective criteria (frequency and intensity). The geometric mean of these two percentages was then used to obtain the percentage of applicability of the attribute.

Equation 1:        Dik=pikpixxikxi

D_ik: Dravnieks score for the attribute i and the product k

p_ik: number of subjects who rated the attribute i for the product k

p_i: number of subjects who rated the attribute i for at least one product

x_ik: sum of intensity scores for the attribute i and the product k

x_i: sum of maximal values for the attribute i and at least one product

A Principal Component Analysis (PCA) was performed on the Dravnieks score matrix (Figure 2(d)), with the 20 categories as active variables. The variables were centred but not standardised (cov-PCA) as they were derived from the same Dravnieks scoring procedure. The terms and families were introduced as supplementary variables. This emphasis on category attributes derives from the fact that, in the hierarchical wine lexicon, categories correspond to the attributes commonly involved in the sensory description of wines (e.g., King et al., 2013, Green et al., 2011, Danner et al., 2017). Most of the commonly used attributes (at a ‘category level') were present in the list of DA attributes used by the ‘Rosé Wine Experimentation and Research Centre' (Table 1). Therefore, we considered the category level to be the baseline of our hierarchy. Using supplementary variables then made it possible to be more precise (with the terms) or more global (with the families).

Results

1. Sensory characterisation by HRATA

1.1. Integration of the hierarchy

Data were aggregated from lower levels to higher ones, as illustrated in Figure 3 (b). Figure 4 depicts the number of attributes (categories and families) with non-zero intensity values before and after hierarchical integration, demonstrating how this step reduces the number of missing scores. For example, hierarchical integration increased the frequency of scores for the ‘fruit’ family attribute from 27 to 302. Indeed, the ‘fruit’ family itself originally received very few scores since the majority of subjects described their sensory perceptions using finer-scale (category- or term-level) attributes within this family; the difference was particularly notable in this case since the family ‘fruit’ contains 5 categories and 31 terms. Another interesting example was the ‘chemical’ attribute, which was only rarely selected as a family or category, while the terms ‘alcohol’ and ‘rubber’ within those groups were chosen more often.

Figure 4. Number of scores per attribute (families and categories belonging to each family), with non-zero intensity values, before and after integration of the hierarchy. Family levels are underlined.

Save View full size Expand inline Collapse inline

1.2. Determination of discriminant terms, categories, and families

Logistic regression was performed to determine the discriminant attributes, to identify those for which the probability of selection differed significantly among products. Table 3 shows the 5 family-level attributes, 10 category-level attributes, and 23 terms for which a significant product effect was identified (p-value < 0.05) (from a total of 9 families, 20 categories, and 89 terms).

Half of the categories were used to discriminate among products, in the sense that they were frequently cited for certain products than for others. Within these categories, some terms stood out. For example, the ‘lactic’ category had a very low p-value (Table 3), and within this category, the term ‘butter’ significantly distinguished certain products from others, while the terms ‘yeast’ and ‘bread’ did not. In the case of the ‘earthy’ category (p-value for the product effect of less than 5 %), though, none of the terms within the category were found to be discriminant; it seems that only the combination of terms in the category made it possible to differentiate among products. On the other hand, some categories were not associated with any significant discriminating ability (e.g., ‘chemical_c’), while certain terms belonging to them were (e.g., ‘alcohol’ and ‘rubber’).

Table 3. Dravnieks’ score and logistic regression results for wines based on the frequency of use for each family, category, and term.

View popup Expand inline Collapse inline

		p-value (< 0.05)	Dravnieks’ scores						Number of subjects who rated the attribute
			Wine A	Wine B	Wine C	Wine D	Wine E	Wine E*
Family	Roasted_f	0.000	50.39	18.25	24.49	32.40	29.50	36.24	49
	Undergrowth_f	0.000	63.09	29.23	23.11	26.95	34.88	26.86	53
	Fruit_f	0.012	54.42	76.90	63.27	72.33	61.23	67.70	66
	Spicy_f	0.034	20.67	31.78	32.90	28.09	41.89	30.91	53
	Floral_f	0.050	24.15	39.20	43.09	43.09	34.35	32.02	54
Category	Spice_c	0.000	20.77	23.79	30.09	16.48	45.03	26.80	47
	Animal_c	0.000	68.68	20.39	10.65	9.52	11.66	4.76	21
	Red_fruit_c	0.000	12.94	43.29	37.08	24.48	41.57	50.84	55
	Earthy_c	0.001	52.04	21.79	17.08	23.98	21.21	20.87	40
	White_yellow_fruit_c	0.001	30.08	46.85	46.55	47.18	22.56	35.09	57
	Lactic_c	0.004	44.18	8.61	25.09	20.00	25.82	23.80	30
	Tropical_fruit_c	0.004	23.98	32.60	23.98	46.90	18.82	15.61	40
	Burned_c	0.020	36.56	11.02	13.61	19.54	25.46	28.05	24
	Mentholated_c	0.021	5.59	25.47	28.85	27.62	16.45	16.66	31
	Floral_c	0.050	24.15	39.20	43.09	43.09	34.35	32.02	54
Term	pepper	0.000	7.86	20.29	30.09	9.62	44.44	16.97	18
	sweat	0.000	72.84	37.80	0.00	0.00	14.29	0.00	7
	pineapple	0.000	9.07	54.43	27.22	31.43	0.00	0.00	9
	blackcurrant	0.001	0.00	32.87	15.71	21.52	22.22	37.41	18
	stone_fruit	0.002	0.00	8.16	0.00	22.36	8.16	52.92	10
	lychee	0.006	20.20	8.25	4.12	37.80	10.10	0.00	14
	blackberry	0.007	0.00	40.82	5.77	18.26	42.82	26.46	10
	cherry	0.009	0.00	27.66	23.33	15.97	37.80	23.33	14
	leather	0.010	44.72	8.16	8.16	20.00	0.00	0.00	10
	strawberry	0.013	0.00	21.82	17.50	11.66	24.74	42.26	14
	cinnamon	0.015	0.00	0.00	20.20	0.00	8.25	40.41	7
	rubber	0.016	47.14	0.00	9.62	0.00	0.00	9.62	6
	alcohol	0.017	22.63	22.63	44.60	29.24	47.62	29.47	27
	white_fruit	0.018	23.54	18.23	36.46	47.27	10.53	24.69	19
	apple	0.018	27.77	20.33	14.85	29.69	0.00	0.00	11
	butter	0.022	37.27	0.00	18.63	21.52	28.87	22.05	12
	licorice	0.023	0.00	17.68	42.70	7.22	30.62	17.68	8
	toasted_bread	0.024	31.94	0.00	8.25	0.00	0.00	37.80	7
	peach	0.033	21.81	23.22	32.35	33.01	8.45	11.95	29
	violet	0.040	3.21	7.86	29.40	11.11	23.57	26.06	18
	musty	0.045	51.03	12.50	17.68	20.41	0.00	17.68	8
	raspberry	0.046	0.00	16.33	10.00	32.66	34.64	26.46	10
	currant	0.046	0.00	30.55	22.36	8.16	14.14	32.66	10

1.3. Multidimensional data analysis

Data were analysed using a non-standardised PCA on the matrix of Dravnieks scores. All category-level attributes were used as active variables, while family-level attributes and terms were included as supplementary variables. The results of the PCA with only the category-level attributes are displayed in Figure 5. For the sake of clarity, the discriminant categories (listed in Table 3) are shown in black in the panel, with all other categories in grey (Figure 5 (b)).

Wine A stands out, particularly, in the first dimension, which accounts for more than 60 % of the inertia. This wine was characterised by ‘animal’ and ‘earthy’ odour notes. The other wines are separated along the second dimension of the PCA based on notes of ‘tropical fruits’ and ‘white and yellow fruits’; wines B, C, and D had ‘white and yellow fruit’ aromas and wine D was characterised by ‘tropical fruit’. Wines E and E* (corresponding to the replicated wine) were characterised by notes of ‘red fruits’. The proximity of the two replicates (E and E*) with respect to the first two principal components demonstrates the consistency of the orthonasal aromatic evaluation of the subjects.

Figure 5. PCA plots based on Dravnieks’ scores for category-level attributes. (a) Configuration of products and (b) configuration of variables (attributes in grey were found to be not discriminant, at a 5 % level of risk).

Save View full size Expand inline Collapse inline

Figure 5 shows the discriminant category-level attributes (in black) together with all discriminant terms and family-level attributes (Table 3) projected as supplementary variables onto the two first principal component axes.

The projection of specific terms onto the PCA plot in Figure 4 made it possible to obtain more detailed information about the specific odour notes of the wines. We can illustrate this by focusing on two examples, namely the ‘chemical’ category and the ‘floral’ category.

The ‘chemical_c’ category was composed of six attributes (‘alcohol’, ‘rubber’, ‘nail polish remover’, ‘oil’, ‘sulfur’, and ‘vinegar’), of which only two were significant: ‘rubber’ and ‘alcohol’. The term ‘rubber’ was selected by six subjects and had a Dravnieks score of 47.14 for wine A (Table 3). In the PCA plot (Figure 6), this attribute was highly correlated with the first dimension, while the term ‘alcohol’ was negatively correlated with both the first and the second dimensions. Thus, wine A was alone in being characterised by a ‘rubber’ note.

For the category ‘floral_c’, the Dravnieks scores for the different wines were between 24.15 and 43.09 (Table 3) based on evaluations by 54 subjects. This category, which is quite often used according to these scores, was composed of six attributes (‘acacia’, ‘orange blossom’, ‘jasmine’, ‘lilac’, ‘rose’, and ‘violet’). Interestingly, within this category, the term ‘violet’ (evaluated by 18 subjects) was more closely associated with wines C, E, and E*.

Figure 6. PCA plots based on Dravnieks’ scores for category-level attributes. Only significant attributes (p-value < 0.05) are represented (families, categories, and terms).

Save View full size Expand inline Collapse inline

From a sensory point of view, these two examples show how the decomposition of categories into their component attributes can provide additional information on wine characterisation. In classical sensory evaluation, such as DA description, wines are more likely to be described using general attributes, for example, ‘chemical’ and ‘floral’. Here, however, HRATA evaluation enabled a deeper assessment of more specific characteristics of the wine, for example, ‘rubber’ and ‘violet’, even those the use of those terms differed from their corresponding categories.

2. Comparison of HRATA data to DA data

2.1. Determination of discriminant terms, categories, and families

A two-way analysis of variance was performed on DA data to identify the attributes that discriminate among products; we considered odours and aromas. The expert panel used 29 attributes (14 odours and corresponding aromas and the attribute ‘faults’) to define the sensory properties of the studied rosé wines. Values of the Fisher statistics of the analysis of variance (ANOVA) and p-values are given in Table 4. The effect of the wine factor was found to be significant (p-value < 0.05) for 4 odours, 4 aromas, and the attribute ‘faults’.

Table 4. F ratios and P-values of the variance analysis (ANOVA) based on the 6 wines analysed, 13 judges (random factor), for each sensory attribute (O: odour, orthonasal olfaction: A: aroma, retronasal olfaction).

View popup Expand inline Collapse inline

	Wine		Judge		Wine * Judge
	F	p-value	F	P-value	F	p-value
O_citrus	21.26	< 0.000	2.60	< 0.000	5.61	0.107
O_animal	35.77	0.001	11.17	< 0.000	4.73	< 0.000
O_flowers	33.07	0.012	2.13	< 0.000	3.23	0.145
O_sweets	35.31	0.019	1.96	< 0.000	2.96	0.162
O_empyreumatic	25.33	0.054	< 0.00	< 0.000	2.24	1.000
O_ripe_compote-confit-fruit	16.69	0.088	11.57	< 0.000	2.03	< 0.000
O_exotic_fruit	21.41	0.260	10.04	< 0.000	1.34	0.002
O_dried_fruit_flower_vegetal	31.53	0.274	11.88	< 0.000	1.31	< 0.000
O_mushroom	142.66	0.396	< 0.00	< 0.000	1.04	1.000
O_vegetal	30.90	0.511	3.08	< 0.000	0.86	0.079
O_red_black_fruit	15.58	0.592	8.64	< 0.000	0.75	0.003
O_mineral	68.82	0.611	3.29	< 0.000	0.72	0.070
O_spice_aromatics	57.64	0.907	1.00	< 0.000	0.31	0.317
O_white_yellow_fruit	31.29	0.954	1.38	< 0.000	0.22	0.241
A_ citrus	9.69	0.002	0.60	0.002	4.49	0.438
A_ ripe_compote-confit-fruit	21.17	0.008	5.99	< 0.000	3.51	0.014
A_ red_black_fruit	20.16	0.008	10.74	< 0.000	3.44	0.001
A_empyreumatic	8.24	0.012	2.18	0.004	3.24	0.140
A_vegetal	34.91	0.069	5.22	< 0.000	2.17	0.022
A_ mushroom	154.09	0.082	3.36	< 0.000	2.07	0.067
A_animal	28.98	0.096	< 0.00	< 0.000	1.91	1.000
A_ exotic_fruit	22.51	0.110	14.07	< 0.000	1.89	< 0.000
A_sweets	34.95	0.169	0.52	< 0.000	1.62	0.469
A_flowers	40.74	0.365	7.83	< 0.000	1.11	0.005
A_mineral	58.04	0.537	3.60	< 0.000	0.82	0.058
A_ dried_fruit_flower_vegetal	45.10	0.633	0.66	< 0.000	0.69	0.417
A_ white_yellow_fruit	30.70	0.753	2.85	< 0.000	0.53	0.091
A_ spice_aromatics	46.90	0.874	1.43	< 0.000	0.36	0.232
faults	44.14	0.008	7.29	< 0.000	3.46	0.007

2.2. Multidimensional data analysis

The data obtained from the Descriptive Analysis (DA) conducted by the trained panel were submitted to a PCA. We used only the discriminant attributes with a p-value < 0.20 and odour attributes in coherence with the HRATA experiment, aroma attributes were used as illustrative variables (Figure 7). Wine A was mainly characterised by an ‘animal’ odour. Wines B, C, and E were characterised by ‘citrus’, ‘flowers’, and ‘sweets’ odours. Finally, wines E and E* were characterised by ‘ripe, compote, confit fruit’ odours.

These results were largely consistent with the aromatic characterisation produced using the HRATA method, especially with respect to the ‘red fruit’ note of wines E and E* if we also considered the aroma attribute and the ‘animal’ and ‘faults’ notes of wine A (Figure 7).

Moreover, when we compared the configuration of the wines obtained from the PCA (first two dimensions) of the HRATA data with that of the DA results, we found an acceptable degree of similarity, with an RV coefficient of 84 %.

Figure 7. PCA plots based on DA data from trained subjects. (a) Configuration of the products and (b) configuration of the variables (in black: odour attributes evaluated by orthonasal olfaction, in grey: aromatic aromas evaluated by retronasal olfaction).

Save View full size Expand inline Collapse inline

We noted differences in the use of the attributes ‘dried fruit/flowers/vegetables’ and ‘ripe fruit/compote’, which did not stand out as much in the description obtained from HRATA. Generally speaking, though, the description generated by HRATA was similar to that obtained by descriptive analysis.

Discussion

Here, the HRATA approach enabled us to obtain results for wine aromatic characterisation that was comparable to the results of a DA performed by a trained panel. Similar comparisons have been performed between the RATA method and DA (Ares et al., 2018; Danner et al., 2017; Oppermann et al., 2017), and while the maps produced by the two methods are generally similar, Ares et al. (2018) reported that RATA is less discriminating than DA when describing complex products (such as wine). This was consistent with the differences noted here in the distances between wines B, C, and D on the PCA plots (Dim 1 x Dim 2) of HRATA data (Figure 5(a)) and DA data (Figure 7(a)).

One of the main reasons for this discrepancy was likely the fact that all aromatic notes did not necessarily have the same weight and meaning in the two lexicons. Some aromatic notes were not clearly represented in the HRATA lexicon compared to that used for DA: the DA attribute ‘ripe/compote/confit fruits’ was present in the HRATA lexicon only as the attribute ‘ripe fruit’ in the ‘red fruits’ category. Instead, the attribute ‘dried fruits/flowers/vegetables’ was divided into three different categories in HRATA. These differences were due to the fact that the DA was based on the typical lexicon used by the trained panel of the ‘Rosé Wine Experimentation and Research Centre’ instead of on the hierarchical lexicon generated and used for the HRATA analysis.

One more, judges performing the DA may be focused less on the odours as they knew that they would also evaluate the aromas. To investigate this further, it would be interesting to perform a DA by selecting only the category-level attributes of HRATA and comparing characterisations based on odour and aroma evaluations.

Regarding the HRATA approach, we made a choice to use subjects who possessed at least a minimal knowledge of wine but who were not specifically trained for the characterisation of rosé wine. The subjects, therefore, represented a wide range of wine expertise, from consumers, students in viticulture and oenology, and professionals from the wine sector. Consumers were overrepresented in the HRATA evaluation. We compared, on one side, a classical DA evaluation with a trained panel and, on another side, the HRATA evaluation with a mixed panel without common training, as the aim was to propose an alternative approach. We also preferred to use two independent panels rather than compare the method with the same group. We have shown that the characterisation provided by the HRATA method was in agreement with that of a classical DA.

From the taster’s point of view, this approach permits greater freedom in evaluation, as the hierarchical structure makes it possible to assess general and/or more specific attributes depending on the sensitivity of the individual without difficulty. A new friendly and ergonomic interface has been created since the end of this study (Figure 8).

From the experimenter’s point of view, incorporating the lexicon structure should increase the accuracy of product characterisation via the introduction of a greater number of specific attributes.

Furthermore, the proposed strategy for data analysis is easy to implement and provides results that fully take into account the hierarchical structure for the purpose of interpretation.

In designing this study, we made several choices regarding the methodology and statistical approach to be used. First of all, to aggregate information from one hierarchical level to the next, we chose to use a maximum rating aggregation rule, which seemed to us to be the most logical choice compared to summation or averaging of intensity ratings, for example. Nevertheless, other aggregation rules are possible, for example, those proposed within the framework of multi-criteria aggregation in fuzzy logic (Grabisch and Perny, 2003). Alternatively, a weighting aggregation rule could have been investigated instead of our practice of weighting all attributes equally regardless of the rating selected.

This first coding step allowed information to be imputed from the lowest to the highest level in the hierarchy. However, the number of non-evaluated attributes remained very large. For this reason, our analysis distinguished between two situations: attributes that were not considered at all by the subject and attributes that were given a score of zero intensity for some products. In classical RATA data processing, this distinction is not made; an attribute that is not evaluated has a score of 0. However, we found it preferable to limit the number of cases of automatic imputation to 0 as much as possible.

By using logistic regression, we could directly address whether an attribute was chosen to characterise one product more often than others. Although this approach has the disadvantage of losing quantitative information about intensity scores, it was the method best adapted for dealing with the high number of non-selected attributes. Indeed, the very high frequency of 0 in the dataset strongly skewed the distribution of the response. This further demonstrated that ANOVA would be an inappropriate statistical analysis for this type of dataset. For reasons of robustness, the probability of citation, as calculated in this study, was a more appropriate statistical treatment given the data distribution.

As explained above, only attributes at the category level were used as active variables in the PCA. This choice enabled us to carry out a standard statistical treatment that is easy to implement and simple to interpret in the context of sensory description. By using the other significant attributes as supplementary variables, we could consider the lexicon's hierarchical structure and provide clarification in terms of interpretation.

Within the hierarchical presentation of the attributes used in this study, we further incorporated an intensity rating scale, with which the subjects were asked to record the intensity of each attribute. Without this rating step, though, this method easily becomes a Hierarchical-Check-All-That-Apply (HCATA) approach. On this point, Meyners et al. (2016) pointed out that RATA may be more prone to bias than CATA with respect to response strategies (Sudman and Bradburn, 1986). RATA requires more cognitive input; people need to think more about the attributes scoring the intensity of each selected attribute, and subjects may decide to simplify the task by not taking the time to select the same number of attributes they otherwise would. Pineau et al. (2022) investigate faster alternatives to sensory profiling (including CATA and RATA) and highlight that RATA characterisation was closer to sensory profiling. Further investigation should be considered to better evaluate the differences between different rating scales (or no rating scale for HCATA) in terms of both the aromatic characterisation of the products and the user experience. A large number of attributes is considered in our HRATA methodology compared to RATA experiments (e.g., Copper et al., 2019; Mezei et al., 2021); having a large number of attributes can allow one to characterise precisely the wines as in free profiling, it avoids to select words by consensus as classically performed to evaluate and describe the sensory space of wines (Barbe et al., 2021).

HRATA needs more data pre-treatment before univariate and multivariate analyses; this may cost more time in the data analysis compared to RATA; however, these steps can be easily automated. This disadvantage is compensated by the superiority of HRATA to access more informative data compared to RATA or DA.

Figure 8. New graphical interface of Fizz software using the HRATA method with an aroma wheel (Biosystèmes®, 1990).

Save View full size Expand inline Collapse inline

Conclusion

The HRATA approach presented in this study represents an interesting alternative for obtaining aromatic characterisation using a panel of subjects without collective training, generating and selecting descriptors and also with diverse skill sets (in our case: consumers, professionals, and students). Our objective was to propose an easy-to-implement approach as well as a simple and relevant strategy for data analysis. Compared to classical descriptive sensory analysis, HRATA provides an effective solution for describing complex products like wine. We considered a hierarchical structuration of sensory attributes with three levels (families, categories and terms). The category level, classically used in a descriptive sensory analysis with a pre-established list of descriptors, can be the baseline of the hierarchy during data analysis. Terms allow for precise characterisation, and families allow one to summarise the characterisation.

More generally, the guidelines provided in this paper are general enough to be applied to the aromatic evaluation of any complex product whose attributes can be represented hierarchically. Our data processing approach can be used for other sensory or hierarchically designed data sets. Of course, the hierarchical structure and the number of levels may vary depending on the product under consideration.

Acknowledgements

This work was carried out with the financial support of the regional program "Food for Tomorrow / Cap Aliment, Research, Education, and Innovation in Pays de la Loire”, which is supported by the French Region Pays de la Loire and the European Regional Development Fund (FEDER), as well as the Ecole Supérieure des Agricultures in Angers. The study was carried out using the facilities of the Senso’Veg platform. We would like to thank the technical team of ESA for their help in carrying out the HRATA evaluation and Clémence Salou and the technical team from the Rosé Wine Experimentation and Research Centre for the DA evaluation. Finally, we are deeply grateful to all the subjects who took part in the study.

References

• Ares, G., Bruzzone, F., Vidal, L., Cadena, R. S., Giménez, A., Pineau, B., Hunter, D. C., Paisley, A. G., & Jaeger, S. R. (2014). Evaluation of a rating-based variant of check-all-that-apply questions: Rate-all-that-apply (RATA). Food Quality and Preference, 36, 87–95. https://doi.org/10.1016/j.foodqual.2014.03.006
• Ares, G., Picallo, A., Coste, B., Antúnez, L., Vidal, L., Giménez, A., & Jaeger, S. R. (2018). A comparison of RATA questions with descriptive analysis: Insights from three studies with complex/similar products. Journal of Sensory Studies, 33(5), e12458. https://doi.org/10.1111/joss.12458
• Ares, G., & Varela, P. (2017). Trained vs. consumer panels for analytical testing: Fueling a long lasting debate in the field. Food Quality and Preference, 61, 79–86. https://doi.org/10.1016/j.foodqual.2016.10.006
• Barbe, J.-C., Garbay, J., Tempere, S. (2021). The sensory space of wines: From concept to evaluation and description. a review. Foods, MDPI, 10(6), 1424, https://doi.org/10.3390/foods10061424
• Bindon, K., Holt, H., Williamson, P. O., Varela, C., Herderich, M., & Francis, I. L. (2014). Relationships between harvest time and wine composition in Vitis vinifera L. cv. Cabernet Sauvignon 2. Wine sensory properties and consumer preference. Food Chemistry, 154, 90–101. https://doi.org/10.1016/j.foodchem.2013.12.099
• Biosystèmes® (1990). FIZZ: Logiciel de gestion de l’analyse sensorielle et des tests consommateurs.
• Caillé, S., Salmon, J.-M., Bouvier, N., Roland, A., & Samson, A. (2017). Modification of the olfactory sensory characteristics of Chardonnay wine through the increase in sotolon concentration. Food Quality and Preference, 56(Part A), 225–230. https://doi.org/10.1016/j.foodqual.2016.08.001
• Campo, E., Do, B., Ferreira, V. & Valentin, D. (2008), Aroma properties of young Spanish monovarietal white wines: a study using sorting task, list of terms and frequency of citation. Australian Journal of Grape and Wine Research, 14: 104-115. https://doi.org/10.1111/j.1755-0238.2008.00010.x
• Campo, E., Ballester, J., Langlois, J., Dacremont, C., & Valentin, D. (2010). Comparison of conventional descriptive analysis and a citation frequency-based descriptive method for odor profiling: An application to Burgundy Pinot noir wines. Food Quality and Preference, 21(1), 44–55. https://doi.org/10.1016/j.foodqual.2009.08.001
• Copper, A.W., Johson, T.E., Danner, L., Bastian, S.E.P., Collins C. (2019). Preliminary sensory and chemical profiling of Cypriot wines made from indigenous grape varieties Xynisteri, Maratheftiko and Giannoudhi and acceptability to Australian consumers. OenoOne, 53(2). https://doi.org/10.20870/oeno-one.2019.53.2.2423
• Coulon-Leroy, C., Symoneaux, R., Lawrence, G., Mehinagic, E., & Maitre, I. (2017). Mixed Profiling: A new tool of sensory analysis in a professional context. Application to wines. Food Quality and Preference, 57, 8–16. https://doi.org/10.1016/j.foodqual.2016.11.005
• Coulon-Leroy, C., Pouzalgues, N., Cayla, L., Symoneaux, R., Masson, G. (2018). Is the typicality of “Provence Rosé wines” only a matter of color? OenoOne, 52-4, https://doi.org/10.20870/oeno-one.2018.52.4.2125
• Danner, L., Crump, A. M., Croker, A., Gambetta, J. M., Johnson, T. E., & Bastian, S. E. P. (2017). Comparison of Rate-All-That-Apply (RATA) and Descriptive Analysis (DA) for the Sensory Profiling of Wine. American Journal of Enology and Viticulture. https://doi.org/10.5344/ajev.2017.17052
• Dobson, A. J., & Barnett, A. (2008). An Introduction to Generalized Linear Models, Third Edition (3rd ed.). Chapman and Hall/CRC.
• Dravnieks, A. (1982). Odor quality: Semantically generated multidimensional profiles are stable. Science (New York, N.Y.), 218(4574), 799–801. https://doi.org/10.1126/science.7134974
• Grabisch, M., & Perny, P. (2003). Agrégation multicritère. In Logique floue, principes, aide à la décision: Vol. Traité IC2, série Informatique et Systèmes d’Information (Hermès-Lavoisier, p. 81 120).
• Green, J. A., Parr, W. V., Breitmeyer, J., Valentin, D., & Sherlock, R. (2011). Sensory and chemical characterisation of Sauvignon blanc wine: Influence of source of origin. Food Research International, 44(9), 2788–2797. https://doi.org/10.1016/j.foodres.2011.06.005
• Husson, F., Josse, J., Le, S., & Mazet, J. (2018). FactoMineR: Multivariate Exploratory Data Analysis and Data Mining (1.41) [Computer software].
• ISO (2007). Sensory analysis: General guidance for the design of test rooms, ISO standard 8589. Geneva, Switzerland: International Organization for Standardization.
• King, E. S., Dunn, R. L., & Heymann, H. (2013). The influence of alcohol on the sensory perception of red wines. Food Quality and Preference, 28(1), 235 243.
• https://doi.org/10.1016/j.foodqual.2012.08.013
• Koenig, L., Cariou, V., Symoneaux, R., Coulon-Leroy, C., & Vigneau, E. (2021). Additive trees for the categorization of a large number of objects, with bootstrapping strategy for stability assessment. Application to the free sorting of wine odor terms. Food Quality and Preference, 89, 104137. https://doi.org/10.1016/j.foodqual.2020.104137
• Koenig, L., Coulon-Leroy, C., Symoneaux, R., Cariou, V., & Vigneau, E. (2020). Influence of expertise on semantic categorization of wine odors. Food Quality and Preference, 83. https://doi.org/10.1016/j.foodqual.2020.103923
• Larssen, W. E., Monteleone, E., & Hersleth, M. (2018). Sensory description of marine oils through development of a sensory wheel and vocabulary. Food Research International, 106, 45–53. https://doi.org/10.1016/j.foodres.2017.12.045
• Lawless, L. J., & Civille, G. V. (2013). Developing lexicons: A review. Journal of Sensory Studies, 28(4), 270–281.
• Lawrence, G., Symoneaux, R., Maitre, I., Brossaud, F., Maestrojuan, M., & Mehinagic, E. (2013). Using the free comments method for sensory characterisation of Cabernet Franc wines: Comparison with classical profiling in a professional context. Food Quality and Preference, 30(2), 145–155. https://doi.org/10.1016/j.foodqual.2013.04.005
• Longo, R., Pearson, W., Merry, A., Solomon, M., Nicolotti, L., Westmore, H., Dambergs, R., & Kerslake, F. (2020). Preliminary Study of Australian Pinot Noir Wines by Colour and Volatile Analyses, and the Pivot© Profile Method Using Wine Professionals. Foods, 9(9), 1142. https://doi.org/10.3390/foods9091142
• Mahieu, B., Visalli, M., Thomas, A., & Schlich, P. (2020). Free-comment outperformed check-all-that-apply in the sensory characterisation of wines with consumers at home. Food Quality and Preference, 84, 103937. https://doi.org/10.1016/j.foodqual.2020.103937
• Mezei, L.V., Johnson, T.E., Goodman, S., Collins, C., Bastian, S.E.P. (2021). Meeting the demands of climate change: Australian consumer acceptance and sensory profiling of red wines produced from non-traditional red grape varieties. OenoOne, 55(2). https://doi.org/10.20870/oeno-one.2021.55.2.4571
• Meyners, M., Jaeger, S. R., & Ares, G. (2016). On the analysis of Rate-All-That-Apply (RATA) data. Food Quality and Preference, 49, 1–10. https://doi.org/10.1016/j.foodqual.2015.11.003
• Noble, A. C., Arnold, R. A., Buechsenstein, J., Leach, E. J., Schmidt, J. O., & Stern, P. M. (1987). Modification of a Standardized System of Wine Aroma Terminology. American Journal of Enology and Viticulture, 38(2), 143–146.
• Noble, A. C., Arnold, R. A., Masuda, B. M., Pecore, S. D., Schmidt, J., & Stern, P. M. (1984). Progress towards a standardized system of wine aroma terminology. American Journal of Enology and Viticulture, 35(2), 107–109.
• Oppermann, A. K. L., de Graaf, C., Scholten, E., Stieger, M., & Piqueras-Fiszman, B. (2017). Comparison of Rate-All-That-Apply (RATA) and Descriptive sensory Analysis (DA) of model double emulsions with subtle perceptual differences. Food Quality and Preference, 56, 55–68. https://doi.org/10.1016/j.foodqual.2016.09.010
• Perrin, L., Symoneaux, R., Maître, I., Asselin, C., Jourjon, F., & Pagès, J. (2008). Comparison of three sensory methods for use with the Napping® procedure: Case of ten wines from Loire valley. Food Quality and Preference, 19(1), 1–11. https://doi.org/10.1016/j.foodqual.2007.06.005
• Pineau, N., Girardi, A., Lacoste Gregorutti, C., Filion, L., Labbe, D. (2022). Comparison of RATA, CATA, sorting and Napping® as rapid alternatives to sensory profiling in a food industry environment. Food Research international, 158-111467. https://doi.org/10.1016/j.foodres.2022.111467
• Robert, P., & Escoufier, Y. (1976). A Unifying Tool for Linear Multivariate Statistical Methods: The RV- Coefficient. Journal of the Royal Statistical Society. Series C (Applied Statistics), 25(3), 257–265. https://doi.org/10.2307/2347233
• Sáenz-Navajas, M., Arias-Pérez, I., Ferrero-del-Teso, S., Escudero, A., Ferreira, V., Fernández-Zurbano, P., Valentin, D. (2021). Access to wine experts' long-term memory to decipher an ill-defined sensory concept: the case of green red wine? OenoOne, 55(1), https://doi.org/10.20870/oeno-one.2021.55.1.4475.
• Sidel, H. S. L. (2004). Sensory Evaluation Practices, Third Edition. Academic Press.
• Spence, C., & Wang, Q. J. (2018). What does the term ‘complexity’ mean in the world of wine? International Journal of Gastronomy and Food Science, 14, 45–54. https://doi.org/10.1016/j.ijgfs.2018.10.002
• Sudman, S., & Bradburn, N. M. (1986). Asking Questions: A Practical Guide to Questionnaire Design (Jossey-Bass).
• Varela, P., & Ares, G. (2014). Novel Techniques in Sensory Characterization and Consumer Profiling. CRC Press.
• Vidal, L., Ares, G., Hedderley, D. I., Meyners, M., & Jaeger, S. R. (2018). Comparison of rate-all-that-apply (RATA) and check-all-that-apply (CATA) questions across seven consumer studies. Food Quality and Preference, 67, 49–58. https://doi.org/10.1016/j.foodqual.2016.12.013
• Villière, A., Le Roy, S., Fillonneau, C., & Prost, C. (2018). InnOscent system: Advancing flavor analysis using an original gas chromatographic analytical device. Journal of Chromatography A, 1535, 129–140. https://doi.org/10.1016/j.chroma.2017.12.053