Sage Journals: Discover world-class research

Abstract

Objective

Nonhuman primates (NHPs) are suitable for being model animals in the study of consciousness and loss of consciousness (LoC) with a similar brain structure and function to humans. However, there is no effective consciousness assessment scale for them. This study aimed to develop a behavioral assessment scale of consciousness for NHPs.

Methods

We constructed an initial indicator framework based on the clinical consciousness disorder assessment scales and the physiological characteristics, consciousness, and arousal behavior of NHPs. A two-round online Delphi method was conducted by a multidisciplinary expert panel to construct a behavioral assessment scale of consciousness for NHPs. The indicators and descriptions were revised according to the experts’ feedback and then sent out for repeated consultations along with a summary of the results of the previous round of consultations. The accepted competencies of indicators were established with mean scores in two scoring criteria (importance and feasibility) ≥4.0, agreement rate with a rating of importance or essential ≥70.0%, and a coefficient of variation ≤0.25, as well as discussions of the research group.

Results

Consensus was achieved after the second round of consultations, which was completed by 28 experts who specialized in rehabilitation, neuroscience, psychology, neurosurgery, and neurology. A new behavioral assessment scale of consciousness for NHPs, including 37 items organized hierarchically within seven dimensions including visual function, auditory function, motor function, orofacial movements, arousal, brainstem reflexes, and respiration, was developed in this study.

Conclusions

This study has successfully developed a behavioral assessment scale for measuring the conscious state of NHPs or NHP models with LoC. This tool is expected to facilitate future research into the underlying mechanisms of consciousness by providing a detailed and comprehensive means of measurement.

Keywords

Scale Delphi consciousness nonhuman primates behavior evaluation

Introduction

Consciousness is thought not to be directly observed, but to be identified primarily through reasoning from the perspective of a third person.¹ This mainly relies on subjective reports of the participants to evaluate their consciousness, which poses considerable challenges to the reasoning when their subjective reports are not available. Currently, the understanding of consciousness and loss of consciousness (LoC) is largely dependent on individual case studies or clinical trials in different patients or healthy controls,² which have limited ability to export data and implement controlled experimental designs. For example, long-term clinical trials are subject to selection bias or cohort effects which reduce the reproducibility of results, and many outcomes cannot be directly measured in humans for ethical or moral reasons. This may lead to the mechanisms responsible for consciousness and LoC being incompletely understood. Additionally, the lack of such information may limit physicians in the accurate determination of levels of consciousness ultimately,³ which has a negative influence on making treatment plans and prognosis assessments for patients with LoC.

The use of preclinical animal models is already widespread for elucidating unknown mechanisms underlying consciousness and LoC because they offer precise control with reduced variability, rich data output, and limited ethical complexity.⁴ Among these, rodents have been studied more and play an irreplaceable role, however, their neural network connectivity and circuit function differ greatly from that of humans, and some types of LoC model cannot be realized, such as diffuse axonal injury.⁵ Nonhuman primates (NHPs) have evolutionary similarities to humans in terms of anatomy, physiology, genetics, and neurological function as well as their cognitive, emotional, and social behavior⁶ enabling them to play a vital role in medicine and biomedical research. Especially, neuroscience research using NHPs has led to a deeper understanding of the mechanisms of advanced brain function and diseases.⁷ Furthermore, numerous experiments have shown that NHPs have consciousness.^8,9 Therefore, we believe that it is feasible and valuable to use NHPs to study the mechanism of consciousness and LoC.

Although electroencephalography (EEG),¹⁰ positron emission tomography-computed tomography (PET-CT),¹¹ and functional magnetic resonance imagery (fMRI)¹² can be used as tools to assess consciousness, behavior evaluation is still the gold standard currently because of their expensive, time-consuming and inconvenient operation. Scales such as the Glasgow Coma Scale (GCS),¹³ Full Outline of Un-responsiveness (FOUR),¹⁴ and Coma Recovery Scale-revised (CRS-R)¹⁵ are widely used in clinical diagnosis and treatment. There is still a lack of systematic research on the assessment tools for the state of consciousness of NHPs. Gennarelli et al.¹⁶ found it difficult to describe the consciousness state of the NHP model with diffuse axonal injury. Then, they selected several behavioral indicators based on clinical scales used in patients with LoC, such as corneal reflexes, eye-opening, and visual following.¹⁶ Similarly, Tasserie et al.¹⁷ and Redinbaugh et al.¹⁸ have developed evaluation indexes for the anesthetic awakening behavior of NHPs based on clinical assessment scales, including limb face movements, oral signs, and body movements. Although there were fewer items in the above scales, the previous studies indicated that we can extract assessment indexes suitable for NHPs from human clinical behavioral consciousness assessment scales. However, it is necessary to develop a scientific, effective, and operable tool for NHPs.

The Delphi method is a research technique used for collecting and transforming experts’ opinions and information into a group consensus.¹⁹ The investigators draw up a draft scale and consult with the members of the expert group, respectively, in the form of letters according to a certain procedure, while the experts submit feedback anonymously. After several rounds of survey and feedback, the opinions of the members of the expert group gradually became consensus, and finally, obtain a collective judgment result with high accuracy.²⁰ Implementing an online Delphi study may overcome some limitations found with decision-making processes in face-to-face committee meetings, with the advantage of less cost and feedback being provided in a controlled form.²¹ It is commonly chosen as a way to achieve a consensus for clinical standards and the construction of scales.²²

This study utilized a two-round online Delphi method to formulate a behavioral assessment scale of consciousness for NHPs, incorporating the consciousness and behavior characteristics of NHPs and the clinical conscious behavior assessment scales used for patients with disorders of consciousness (DOC). This work provided a potential tool for further research on the neural mechanisms of consciousness and LoC.

Materials and methods

Delphi procedure

In the present study, we first formulated an original draft of the behavior assessment scale for NHPs’ consciousness state through a literature review, theoretical analysis, and discussion within the research group. Then, the experts’ opinions were collected using an e-Delphi method with two-round online questionnaire surveys. If a consensus was reached in round two, the decision could be made to stop the process, conversely, another round might be set up. After the first round of consultation, the suggestions and opinions of the experts were sorted and analyzed. The researchers appropriately modified the questionnaire for the second round of consultation, based on the outcomes of the preliminary consultation and group discussion. After two rounds of online consultation, expert opinion on the evaluation of each item tended to be consistent and the consultation was stopped, and the final version of the behavioral assessment scale of consciousness in NHPs was formed finally.

Formulating an original draft of the evaluation scale

We organized a research group responsible for literature review, theoretical analysis, selection of the expert panel, compilation and distribution of questionnaires, data process, and analysis, including three professors, two associate professors, and five postgraduates.

The literature on consciousness assessment tools for NHPs and humans in PubMed, Elsevier Science Direct, OVID, Web of Science, Google Scholar, China National Knowledge Infrastructure (CNKI, in Chinese), China Science and Technology Journal Database (VIP, in Chinese), Wanfang Data (in Chinese) and the Chinese Biomedical Database (CBM, in Chinese) were searched and reviewed comprehensively before the Delphi survey to gather evaluation indicators, evaluation manual, and descriptions. Researchers also identified additional relevant articles using experiential knowledge of the LoC scale. The relative studies on rodents, dogs, horses, and other animals were also included. As a result, 27 behavioral scales on the assessment of consciousness and LoC were screened out, which are detailed in Supplemental Table S1.²³ Further, the grading standards and operating procedures of each evaluation indicator in those scales were classified and summarized. From this, the evaluation indicators suitable for NHPs were selected according to their physiological characteristics, and conscious and arousal behavior. Finally, the original draft of the consciousness behavioral evaluation scale on NHPs was generated, including 38 items organized hierarchically within seven dimensions, as well as detailed assessment methods and response scores, which are detailed in Supplemental Table S2.

Construction of correspondence questionnaire

Round one. The Delphi expert questionnaire consisted of three parts. (a) The research introduction and filling instructions, including background, purpose, methodology, questionnaire completion, timing, and contact information. (b) Original behavioral assessment scale of consciousness in NHPs. The experts evaluated their level of agreement for scale using a Likert 5-level scoring method, 5 = very important, 4 = important, 3 = general, 2 = unimportant, and 1 = completely unimportant depending on the importance and feasibility of each item, respectively. Moreover, a free text box for each item was provided to allow the experts to revise, delete, or comment on the indicator and to raise additional components that they felt were missing. (c) Characteristics of experts, including gender, age, education, working experience, education, specialty, professional title, etc., as well as familiarity with the content of the survey and judgment bias.

Round two. The Delphi questionnaire consisted of two parts. (a) The results of round one, including the participant's previous score presented as the mean, standard deviation, coefficient of variation (CV), interquartile range (IQR), and percentage of agreement, and a summary of the experts’ comments. (b) A revised behavioral assessment scale of consciousness in NHPs. The experts re-evaluated their level of agreement with each item.

Formation of the expert panel

In the present Delphi study, experts were selected based on their academic credentials, clinical and/or research experience in consciousness, LoC, NHPs, and constructing a scale. The inclusion criteria were as follows: (a) at least 10 years of work and academic experience, (b) an assistant professor title or above, (c) a bachelor's degree or above, and (d) being willing to answer the expert consultation questionnaire in every round. There were no rigorous standards for the sample size of the Delphi survey.²⁴ Thirty potential experts were invited by purposive sampling, and they were anonymous to each other and did not receive any service fee for participation.

Data collection and management

We conducted the online Delphi method with two iterations of questionnaires from 1 July 2022 to 30 October 2022. The self-designed questionnaires were distributed via an online system https://www.wjx.cn/ (Questionnaire Star, Changsha Ranxing Information Technology Co., Ltd, Changsha, China). Next, we shared the URL with experts via the social media WeChat and requested them to complete the questionnaires within two weeks. Experts remained anonymous to one another throughout the study, and they could contact the researcher at any time if the questionnaire was not clear or items were missing. Every response to the questionnaire was automatically collected into an Excel spreadsheet by the Questionnaire Star for further data analyses.

Data process and statistical analysis

The scientific soundness and rationality of the Delphi method are reflected by four indicators: positive coefficient, concentration degree, authority coefficient, and coordination coefficient of experts.

The experts’ positive coefficient

The positive coefficient reflects the importance that experts attach to this research and determines the credibility and scientific basis of the results, which is usually calculated by the effective response rate of the consultation questionnaire. Generally, the more active the experts are, the higher the experts attach importance to this questionnaire, and then the more they pay attention to this research project. Response rate = [(Number of returned responses)/(Number of invitations)×100%]. Relevant authoritative studies show that an effective response rate of 50% is the minimum acceptable value and over 70% is a very good standard for the Delphi method.²⁵

The experts’ concentration degree

The concentration of expert opinion is expressed by the mean value of importance, standard deviation, the CV, and the agreement rate of participants with a rating of 4 or above. In this study, the Likert 5-level scoring method was used to score the importance of each item. The values from 1 to 5 were "not important" to "very important," and the mean value was from 1 to 5. The higher the mean value, the more important the item accordingly. The agreement rate of participants refers to the ratio of the number of experts who think that an item is very important (5) or relatively important (4). The range of agreement rate value is 0–1. An agreement value ≤0.5 was regarded as an indication of small importance, while, an agreement value >0.6 means this item can be included in the next round of the Delphi study. The CV reflects the fluctuation degree of experts’ opinions on each item, also an important basis for item deletion. The lower value of CV means a lower divergence of views among experts and better consistency of views on the importance or feasibility of this item.²⁶

The experts’ authority coefficient

There is a close relationship between the authority of experts and the consultation results. The authority coefficient (Cr) is determined jointly by two dimensions: the familiarity coefficient (Cs), which represents the expert's familiarity with the investigation, and the judgment coefficient (Ca), which reflects the evidence for the expert to make a judgment. As shown in Table 1, we used a scoring matrix that has been widely used and validated for Delphi studies in China.²⁷ Cr can be measured by the formula Cr = (Cs + Ca)/2. Generally, a Cr value >0.7 is regarded as an acceptable indicator of reliability.²⁸

Table 1.

Judgment basis and familiarity for consultation from experts.

Judgment basis (Cs)	Degree			Familiarity (Ca)	Value
Judgment basis (Cs)	Great deal	Moderate	Little	Familiarity (Ca)	Value
Theoretical analysis	0.30	0.20	0.10	Very familiar	0.90
Practical experience	0.50	0.40	0.30	More familiar	0.70
Referring to literature	0.10	0.10	0.10	Average	0.50
Intuition	0.10	0.10	0.10	Less familiar	0.30
				Unfamiliar	0.10

The experts’ coordination coefficient

Kendall's coefficient of concordance (Kendall's W) is used to assess the degree of coordination between expert opinions on the evaluation item. Its value ranges from 0 to 1 and mostly fluctuates around 0.50. A higher Kendall's W value indicates a better consistency of coordination among experts. Generally, the statistical significance (P < 0.05) of Kendall's W indicates consensus among experts, and that the results are acceptable, otherwise, it indicates that the results are not acceptable.

The inclusion and exclusion criteria

The accepted item inclusion criterion in all two scoring criteria (importance and feasibility): mean score ≥4.0, agreement rate ≥70.0%, and CV ≤0.25. If a CV >0.25, the research team further discussed and decided whether to retain or delete the corresponding item by combining the agreement ratio and mean value comprehensively.

Statistical analysis

Two researchers entered and checked the consultation results from each round independently. All quantitative data analysis was undertaken using the Statistical Package for the Social Sciences (SPSS software; Version 26, IBM Inc., Chicago, IL, USA).²⁹ The level of statistical significance was predetermined as the P value of <0.05.

Results

Experts’ socio-demographic characteristics and enthusiasm

In the round one consultation of the Delphi study, 30 experts were invited to participate, and 28 completed responses were received, resulting in a response rate of 93.33%. In the round two consultations, all 28 experts who participated in the first round responded, resulting in a 100.00% response rate. These experts were selected from diverse regions and disciplinary backgrounds and were highly authoritative and representative, contributing significantly to the validity and reliability of the research. Specifically, the expert panel consisted of 20 males (71.43%) and 8 females (28.57%). Twenty-three experts (82.14%) held a PhD degree, and all experts held a mid-level or higher professional title. Twenty-seven experts (96.43%) held either a deputy senior or senior professional title. Of the panel, two experts (7.14%) were employed in universities and 26 experts (92.86%) in hospitals. The panel's professional expertise spanned several disciplines, including neuroscience, neurosurgery, rehabilitation, and related fields. The experts were from 13 different provinces, including Beijing (5), Shanghai (5), Guangdong Province (5), Zhejiang Province (2), Jiangsu Province (2), Tianjin (1), Hebei Province (1), Hubei Province (1), Hunan Province (1), Anhui Province (1), Shaanxi Province (1), Guangxi Province (1), and Jiangxi Province (1). See Figure 1 for details.

Figure 1.

Characteristics of experts in the Delphi study.

Experts’ authority coefficient

The authority coefficients of Ca, Cs, and Cr were 0.882, 0.736, and 0.809 in round one consultation (Table 2). After the round two consultations, the coefficients increased to 0.896, 0.743, and 0.820, respectively. These results show that the authority coefficient was satisfactory, as it exceeded 0.70. The high authority coefficients indicate that the results of this study are more reliable.

Table 2.

Authority coefficient of the Delphi survey experts.

Item	Ca	Cs	Cr
Round one	0.882	0.736	0.809
Round two	0.896	0.743	0.820

Experts’ opinion coordination coefficient

In round one, Kendall's W on the importance of the first, and second level and all items were 0.493, 0.561, and 0.493 respectively, meanwhile, Kendall's W on feasibility was 0.591 (all P < 0.01). In round two, Kendall's W on the importance of the first and second level, and all items were 0.448, 0.588, and 0.502, respectively, meanwhile, Kendall's W on feasibility was 0.505 (all P < 0.01). The results show that experts have good coordination and consensus on the evaluation statistically (Table 3).

Table 3.

Opinions’ coordination coefficient of the Delphi survey experts.

	Round one				Round two
	Index (n)	Kendall’s W	χ²	P	Index (n)	Kendall's W	χ²	P
Importance	45	0.493	599.526	＜0.01	45	0.448	544.475	＜0.01
First level	7	0.574	108.521	＜0.01	7	0.588	111.223	＜0.01
Second level	38	0.561	575.599	＜0.01	37	0.502	514.886	＜0.01
Feasibility	38	0.591	606.424	＜0.01	37	0.505	518.525	＜0.01

The experts’ concentration degree

The grading on the original scale in round one

The original questionnaire consisted of 38 items organized hierarchically within seven dimensions, as well as detailed assessment methods and response scores. The concentration of expert opinion was assessed by the mean value of importance and feasibility, standard deviation, CV, and agreement rate of participants with a score of 4 or 5.

All mean values on the importance of the items were >3.5, and only 1 item was <4.0 (Ⅳ-5, 3.964 ± 0.999). The IQR of all these items was ≤1.00. All agreement rates of the items were >60.00% with seven items achieving 100.00% agreement, and only one item had <80.00% (Ⅳ, 78.57%). CV of seven primary indicators and 34 secondary items were <0.25, and the items Ⅳ-1, Ⅳ-4, Ⅳ-5, and Ⅳ-5 were valued at 0.260–0.265.

All mean values on the feasibility of the secondary items were >4.0. The IQR of all these items was ≤1.00. All agreement rates of the items were >60.00% with 10 items achieving 100.00% agreement, and only 1 item had <80.00% (Ⅶ-5, 78.57%). CV of 36 secondary items was <0.25, and the items Ⅳ-4 and Ⅳ-6 valued at 0.260 and 0.267. These data are reported in Table 4.

Table 4.

The grading on the original scale in round one.

Item	Importance					Feasibility
Item	Mean ± SD	Median	IQR	CV	Agreement (%)	Mean ± SD	Median	IQR	CV	Agreement (%)
I	4.286 ± 0.535	4	[4,5]	0.124	84.38
I-1	4.679 ± 0.476	5	[4,5]	0.102	84.85	4.571 ± 0.504	5	[4,5]	0.110	100.00
I-2	4.464 ± 0.637	5	[4,5]	0.143	83.87	4.464 ± 0.576	4.5	[4,5]	0.129	84.38
I-3	4.464 ± 0.508	4	[4,5]	0.114	84.85	4.500 ± 0.509	4.5	[4,5]	0.113	100.00
I-4	4.357 ± 0.731	4	[4,5]	0.168	83.87	4.250 ± 0.752	4	[4,5]	0.177	83.33
I-5	4.214 ± 0.833	4	[4,5]	0.198	83.87	4.286 ± 0.810	4	[4,5]	0.189	84.38
I-6	4.286 ± 0.810	4	[4,5]	0.189	84.38	4.286 ± 0.810	4	[4,5]	0.189	84.38
II	4.321 ± 0.612	4	[4,5]	0.141	83.87
II-1	4.571 ± 0.504	5	[4,5]	0.110	100.00	4.536 ± 0.508	5	[4,5]	0.112	100.00
II-2	4.571 ± 0.504	5	[4,5]	0.110	100.00	4.500 ± 0.509	4.5	[4,5]	0.113	100.00
II-3	4.464 ± 0.576	4.5	[4,5]	0.129	84.38	4.464 ± 0.508	4	[4,5]	0.114	100.00
II-4	4.429 ± 0.504	4	[4,5]	0.114	100.00	4.393 ± 0.497	4	[4,5]	0.113	100.00
III	4.393 ± 0.685	4.5	[4,5]	0.155	83.33
III-1	4.429 ± 0.879	5	[4,5]	0.198	83.87	4.571 ± 0.573	5	[4,5]	0.125	84.38
III-2	4.429 ± 0.634	4.5	[4,5]	0.143	83.87	4.429 ± 0.634	4.5	[4,5]	0.143	83.87
III-3	4.429 ± 0.573	4	[4,5]	0.129	84.38	4.464 ± 0.637	5	[4,5]	0.143	83.87
III-4	4.500 ± 0.509	4.5	[4,5]	0.113	100.00	4.536 ± 0.576	5	[4,5]	0.127	84.38
III-5	4.500 ± 0.509	4.5	[4,5]	0.113	100.00	4.536 ± 0.576	5	[4,5]	0.127	84.38
III-6	4.500 ± 0.509	4.5	[4,5]	0.113	100.00	4.536 ± 0.576	5	[4,5]	0.127	84.38
III-7	4.250 ± 0.752	4	[4,5]	0.177	83.33	4.286 ± 0.810	4	[4,5]	0.189	82.76
Ⅳ	4.107 ± 0.994	4	[4,5]	0.242	78.57
Ⅳ-1	4.107 ± 1.066	4	[4,5]	0.260*	82.76	4.286 ± 0.854	4	[4,5]	0.199	83.87
Ⅳ-2	4.429 ± 0.573	4	[4,5]	0.129	84.38	4.429 ± 0.836	5	[4,5]	0.189	84.38
Ⅳ-3	4.321 ± 0.819	4	[4,5]	0.190	84.38	4.357 ± 0.826	4	[4,5]	0.190	84.38
Ⅳ-4	4.036 ± 1.071	4	[4,5]	0.265*	82.14	4.143 ± 1.079	4	[4,5]	0.260*	82.76
Ⅳ-5	3.964 ± 0.999#	4	[4,4.75]	0.252*	82.76	4.179 ± 0.905	4	[4,5]	0.217	82.76
Ⅳ-6	4.000 ± 1.054	4	[4,5]	0.264*	82.14	4.071 ± 1.086	4	[4,5]	0.267*	82.14
Ⅴ	4.750 ± 0.441	5	[4.25,5]	0.092	84.85
Ⅴ-1	4.464 ± 0.962	5	[4,5]	0.215	83.87	4.429 ± 0.959	5	[4,5]	0.217	83.87
Ⅴ-2	4.679 ± 0.476	5	[4,5]	0.102	100.00	4.571 ± 0.504	5	[4,5]	0.110	100.00
Ⅴ-3	4.607 ± 0.567	5	[4,5]	0.123	84.38	4.607 ± 0.497	5	[4,5]	0.108	100.00
Ⅴ-4	4.643 ± 0.488	5	[4,5]	0.105	100.00	4.571 ± 0.504	5	[4,5]	0.110	100.00
Ⅴ-5	4.464 ± 0.508	4	[4,5]	0.114	100.00	4.500 ± 0.509	4.5	[4,5]	0.113	100.00
Ⅴ-6	4.250 ± 0.887	4	[4,5]	0.209	83.33	4.321 ± 0.905	4.5	[4,5]	0.209	83.33
Ⅵ	4.464 ± 0.744	5	[4,5]	0.166	83.87
Ⅵ-1	4.286 ± 0.854	4	[4,5]	0.199	83.87	4.321 ± 0.905	4.5	[4,5]	0.209	83.33
Ⅵ-2	4.357 ± 0.870	4.5	[4,5]	0.200	83.87	4.321 ± 0.905	4.5	[4,5]	0.209	83.33
Ⅵ-3	4.321 ± 0.863	4	[4,5]	0.200	83.87	4.321 ± 0.905	4.5	[4,5]	0.209	83.33
Ⅵ-4	4.357 ± 0.870	4.5	[4,5]	0.200	83.87	4.357 ± 0.911	5	[4,5]	0.209	83.33
Ⅶ	4.286 ± 0.810	4	[4,5]	0.189	82.76
Ⅶ-1	4.214 ± 0.957	4	[4,5]	0.227	82.14	4.321 ± 0.983	5	[4,5]	0.228	82.14
Ⅶ-2	4.214 ± 0.917	4	[4,5]	0.218	82.76	4.286 ± 0.976	5	[4,5]	0.228	82.14
Ⅶ-3	4.179 ± 0.905	4	[4,5]	0.217	82.76	4.286 ± 0.976	5	[4,5]	0.228	82.14
Ⅶ-4	4.214 ± 0.917	4	[4,5]	0.217	82.76	4.286 ± 0.976	5	[4,5]	0.228	82.14
Ⅶ-5	4.250 ± 0.928	4	[4,5]	0.218	82.76	4.250 ± 1.005	5	[4,5]	0.236	78.57

SD: standard deviation; IQR: interquartile range; CV: coefficient of variance.

* represents the coefficient of variance value greater than 0.25.

Experts’ opinion and revision

It was evident that during the first round Delphi study, further discussion was required for potential items that had statistical indicators at the critical level. Specifically, attention was focused on the secondary items of IV (orofacial movements), as shown in Table 5. After considering the mean score, agreement rate, and CV, as well as holding discussions within the research group, we deleted item V-6 and made appropriate modifications and retentions to dimension IV and its items. They were subsequently modified and adjusted before being proposed for a new rating in the second round of the study.

Table 5.

The potential items need further discussion after round one.

Code	Item	Score	Expert Opinion
Ⅳ	Orofacial movements
Ⅳ-1	Normal	5	One expert believes that orofacial movements are insufficiently relevant to consciousness so he gave items in this subscale low scores.
Ⅳ-2	Vocalize	4
Ⅳ-3	Moan	3
Ⅳ-4	Spontaneous oral movement	2	The two experts worried that observing the NHP's orofacial movements carefully could put researchers at risk of injury.
Ⅳ-5	Oral reflexive movement	1
Ⅳ-6	No	0	One expert suggests the complete statement: absent of oral reflexive movement
Ⅴ-6	Inability to open the eyes	C	One expert suggested deleting this item.

The grading on the original scale in round two

All mean values on the importance of the items were >4.0. The IQR of all these items was ≤ 1.00. All agreement rates of the items were >80.00% with 15 items achieving 100% agreement (33.33%), and only 3 items had <85.00% (Ⅳ, Ⅳ-6, and Ⅶ-1). All CVs of the items were <0.25.

All mean values on the feasibility of the secondary items were >4.0. The IQR of all these items was ≤ 1.00. All agreement rates of the items were >80.00% with 12 items achieving 100.00% agreement (31.58%), and only 3 items were also <85.00% (Ⅳ-6, Ⅶ-1, and Ⅶ-5). All CVs of the items were <0.25. These data are reported in Table 6.

Table 6.

The grading on the original scale in round two.

Item	Importance					Feasibility
Item	Mean ± SD	Median	IQR	CV	Agreement (%)	Mean ± SD	Median	IQR	CV	Agreement (%)
I	4.321 ± 0.476	4	[4,5]	0.110	100.00
I-1	4.679 0.476	5	[4,5]	0.102	100.00	4.571 ± 0.504	5	[4,5]	0.110	100.00
I-2	4.536 ± 0.637	5	[4,5]	0.141	92.86	4.536 ± 0.576	5	[4,5]	0.127	96.43
I-3	4.536 ± 0.508	5	[4,5]	0.112	100.00	4.536 ± 0.508	5	[4,5]	0.112	100.00
I-4	4.429 ± 0.742	5	[4,5]	0.168	92.86	4.321 ± 0.772	4	[4,5]	0.179	89.29
I-5	4.357 ± 0.559	4	[4,5]	0.128	96.43	4.500 ± 0.577	5	[4,5]	0.128	96.43
I-6	4.357 ± 0.559	4	[4,5]	0.128	96.43	4.464 ± 0.576	4.5	[4,5]	0.129	96.43
II	4.321 ± 0.612	4	[4,5]	0.142	92.86
II-1	4.571 ± 0.504	5	[4,5]	0.110	100.00	4.571 ± 0.504	5	[4,5]	0.110	100.00
II-2	4.643 ± 0.488	5	[4,5]	0.105	100.00	4.536 ± 0.508	5	[4,5]	0.112	100.00
II-3	4.571 ± 0.573	5	[4,5]	0.125	96.43	4.500 ± 0.509	4.5	[4,5]	0.113	100.00
II-4	4.500 ± 0.509	4.5	[4,5]	0.113	100.00	4.464 ± 0.508	4	[4,5]	0.114	100.00
III	4.536 ± 0.508	5	[4,5]	0.112	100.00
III-1	4.429 ± 0.879	5	[4,5]	0.198	92.86	4.607 ± 0.497	5	[4,5]	0.108	100.00
III-2	4.464 ± 0.637	5	[4,5]	0.143	92.86	4.464 ± 0.637	5	[4,5]	0.143	96.43
III-3	4.429 ± 0.573	4	[4,5]	0.129	96.43	4.500 ± 0.638	5	[4,5]	0.142	92.86
III-4	4.536 ± 0.508	5	[4,5]	0.112	100.00	4.607 ± 0.497	5	[4,5]	0.108	100.00
III-5	4.500 ± 0.509	4.5	[4,5]	0.113	100.00	4.607 ± 0.567	5	[4,5]	0.123	96.43
III-6	4.536 ± 0.508	5	[4,5]	0.112	100.00	4.571 ± 0.573	5	[4,5]	0.125	96.43
III-7	4.321 ± 0.670	4	[4,5]	0.155	89.29	4.357 ± 0.678	4	[4,5]	0.156	89.29
Ⅳ	4.214 ± 0.833	4	[4,5]	0.198	82.14
Ⅳ-1	4.250 ± 0.928	4	[4,5]	0.218	85.71	4.393 ± 0.629	4	[4,5]	0.143	92.86
Ⅳ-2	4.429 ± 0.573	4	[4,5]	0.129	96.43	4.571 ± 0.573	5	[4,5]	0.125	96.43
Ⅳ-3	4.393 ± 0.567	4	[4,5]	0.129	96.43	4.500 ± 0.577	5	[4,5]	0.128	96.43
Ⅳ-4	4.143 ± 0.891	4	[4,5]	0.215	85.71	4.286 ± 0.897	4	[4,5]	0.209	89.29
Ⅳ-5	4.143 ± 0.651	4	[4,5]	0.157	85.71	4.357 ± 0.621	4	[4,5]	0.143	92.86
Ⅳ-6	4.179 ± 0.723	4	[4,5]	0.173	82.14	4.214 ± 0.738	4	[4,5]	0.175	82.14
Ⅴ	4.750 ± 0.441	5	[4.25,5]	0.093	100.00
Ⅴ-1	4.571 ± 0.742	5	[4,5]	0.162	92.86	4.500 ± 0.745	5	[4,5]	0.166	92.86
Ⅴ-2	4.679 ± 0.476	5	[4,5]	0.102	100.00	4.571 ± 0.504	5	[4,5]	0.110	100.00
Ⅴ-3	4.643 ± 0.559	5	[4,5]	0.120	96.43	4.643 ± 0.488	5	[4,5]	0.105	100.00
Ⅴ-4	4.643 ± 0.488	5	[4,5]	0.105	100.00	4.571 ± 0.504	5	[4,5]	0.110	100.00
Ⅴ-5	4.536 ± 0.508	5	[4,5]	0.112	100.00	4.500 ± 0.509	4.5	[4,5]	0.113	100.00
Ⅵ	4.536 ± 0.576	5	[4,5]	0.127	96.43
Ⅵ-1	4.429 ± 0.504	4	[4,5]	0.114	100.00	4.393 ± 0.685	4.5	[4,5]	0.156	89.29
Ⅵ-2	4.464 ± 0.637	5	[4,5]	0.143	92.86	4.464 ± 0.637	5	[4,5]	0.143	92.86
Ⅵ-3	4.393 ± 0.629	4	[4,5]	0.143	92.86	4.571 ± 0.573	5	[4,5]	0.125	96.43
Ⅵ-4	4.429 ± 0.634	4.5	[4,5]	0.143	92.86	4.536 ± 0.576	5	[4,5]	0.127	96.43
Ⅶ	4.357 ± 0.678	4	[4,5]	0.156	89.29
Ⅶ-1	4.286 ± 0.763	4	[4,5]	0.178	82.14	4.429 ± 0.790	5	[4,5]	0.178	82.14
Ⅶ-2	4.393 ± 0.629	4	[4,5]	0.143	92.86	4.429 ± 0.742	5	[4,5]	0.168	85.71
Ⅶ-3	4.393 ± 0.685	4.5	[4,5]	0.156	89.29	4.464 ± 0.637	5	[4,5]	0.143	92.86
Ⅶ-4	4.357 ± 0.678	4	[4,5]	0.156	89.29	4.429 ± 0.836	5	[4,5]	0.189	85.71
Ⅶ-5	4.321 ± 0.723	4	[4,5]	0.167	85.71	4.357 ± 0.780	5	[4,5]	0.179	82.14

SD: standard deviation; IQR: interquartile range; CV: coefficient of variance.

The comparison of the two-round Delphi survey is shown in Figure 2. Figure 2A and D indicates that the mean scores of importance values for dimension in the second round (4.433 ± 0.183) were significantly higher than those in the first round (4.372 ± 0.199) with a statistically significant difference (t = 3.032, P = 0.023). Figure 2B and D indicates that the overall mean scores of importance values for items in the second round (4.365 ± 0.179) were significantly lower than those in the first round (4.440 ± 0.138), with a statistically significant difference (t = 8.130, P < 0.01). Furthermore, Figure 2C and D show that the mean score of feasibility in the second round (4.390 ± 0.135) was significantly higher than that in the first round (4.482 ± 0.098), with a statistically significant difference (t = 8.782, P < 0.01).

Figure 2.

Comparison of importance and feasibility values in the two-round consultations: (A) comparison of dimension importance scores between the two rounds; (B) comparison of item importance scores between the two rounds; (C) comparison of feasibility scores between the two rounds; (D) comparison of total mean scores between the two rounds. The green vertical line in (A)–(D) represents the score of X = 4.

Final items after the Delphi study

Finally, a behavioral assessment scale of consciousness in NHPs was established, including 37 items organized hierarchically within seven dimensions (Table 7), as well as detailed assessment methods and response scores, based on the sufficient synthesis of the mean score, and agreement rate and CV, as well as discussions of the research group in this Delphi study.

Table 7.

Framework of behavioral assessment scale of consciousness for nonhuman primates (NHPs).

Dimension	Code	Item	Score
I. Visual function	I-1	Normal	6
	I-2	Object localization: reaching	5
	I-3	Visual pursuit	4
	I-4	Fixation	3
	I-5	Visual startle	2
	I-6	No response of visual startle	1
II. Auditory function	II-1	Normal	4
	II-2	Localization to sound	3
	II-3	Auditory startle	2
	II-4	No response	1
III. Motor function	III-1	Normal	7
	III-2	Automatic motor response	6
	III-3	Localization to noxious stimulation	5
	III-4	Withdrawal to pain	4
	III-5	Flexion to pain	3
	III-6	Extension to pain	2
	III-7	No response	1
Ⅳ. Orofacial movements	Ⅳ-1	Normal	6
	Ⅳ-2	Vocalize	5
	Ⅳ-3	Moan	4
	Ⅳ-4	Spontaneous oral movement	3
	Ⅳ-5	Oral reflexive movement	2
	Ⅳ-6	No oral reflexive movement	1
Ⅴ. Arousal	Ⅴ-1	Normal	5
	Ⅴ-2	Spontaneous eye-opening	4
	Ⅴ-3	Eyes open to speech	3
	Ⅴ-4	Eyes open to pain	2
	Ⅴ-5	No response	1
Ⅵ. Brainstem reflexes	Ⅵ-1	Corneal reflex and pupil reflex are present	4
	Ⅵ-2	One pupil is dilated and fixed	3
	Ⅵ-3	Corneal reflex or pupil reflex is absent	2
	Ⅵ-4	Corneal reflex and pupil reflex are absent	1
Ⅶ. Respiration	Ⅶ-1	Normal	5
	Ⅶ-2	Periodic	4
	Ⅶ-3	Hyperventilation	3
	Ⅶ-4	Irregular hypoventilation	2
	Ⅶ-5	None (apnea)	1

Discussion

It is always a complex problem to study the mechanism of consciousness and LoC.³⁰ Preclinical animal models provide precise controls, reduced variability, rich data output, and reduced ethical complexity.³¹ The differences in neural network connectivity and loop function between humans and rodents are enormous. While NHPs have evolutionary similarities and highly conserved gene sequences with humans,³² the animal models developed with NHPs have the most significant similarity to humans. Findings based on these models best explain the mechanisms of human brain function and diseases, making them ideal animal models for neuroscience. However, no proper assessment tool is available for the state of consciousness in NHPs.

Consciousness is mainly evaluated through subjective evaluation in clinical diagnosis and treatment. The behavior evaluation on noticeable and operable behavioral signs of patients quantitatively by clinical scales is still the gold standard.³³ Consciousness is the awareness of the external environment and/or the self. To evaluate the conscious behavioral response, we can assess the behavioral response caused by external stimuli and the spontaneous conscious behavioral response under non-external environmental stimuli. In other words, the assessment scales mainly reflect the possible state of consciousness by evaluating the behavior response to a stimulus and identifying what actions cause the related consciousness behavior.³⁴

In recent years, the use of anesthesia-induced loss and recovery of consciousness paradigm in NHPs to study the neurobiological mechanisms of consciousness, DOC, and rehabilitation techniques of DOC has attracted increasing attention.³⁵ Schroeder's team used five indicators to assess the state of consciousness in rhesus monkeys: limb/facial movements, mouth movements, body movements, eye movements, and vital signs.³⁶ Redinbaugh et al.¹⁸ used no response to external stimuli as a sign that rhesus monkeys were in a state of LoC and selected five indicators such as spontaneous movement, pedal reflex, blink reflex, limb manipulation, and vertical nystagmus to evaluate the level of consciousness of rhesus monkeys. Bastos et al.³⁷ defined LoC as the moment when a rhesus monkey's eyes closed for the last time. Recovery of consciousness was defined as the moment when the eyes were open and kept continuously open. Yanagawa et al.³⁸ believe that when rhesus monkeys are unable to control their hands and do not respond to a cotton swab touching their nostrils during the experiment, it is considered to be a state of loss of consciousness caused by anesthesia induction, while rhesus monkeys were judged to be in a state of consciousness when their hands responded to a cotton swab to touch their nostrils and were able to sit calmly for 20 min. It can be seen that behavioral assessment is also the primary way to evaluate the level of consciousness and behavior of NHP animal models, and the eye, mouth, limb movements, and spontaneous movements are the focus. By manipulating the conscious state of animals, the relationship between behaviors, neural activities, and circuits in different conscious states can be established to infer and explain the neural mechanism of consciousness.³⁹ There is still a lack of systematic research on the assessment tools for the state of consciousness of NHP animals.

In a previous study, we proposed that behavioral indicators suitable for assessing consciousness in NHPs could be extracted from clinical DOC assessment scales used for human patients with DOC based on the behavioral characteristics and physiological features of NHPs.²³ The Delphi method is commonly chosen to achieve a consensus for clinical standards and the construction of scales.⁴⁰ This present study was the first attempt to develop a comprehensive and systematic behavioral assessment scale of consciousness for NHPs based on this hypothesis using a Delphi method. Specifically, 46 dimensions were extracted from 27 human clinical consciousness and behavior assessment scales that may suit the NHPs. Then, we formed an original entry pool for the Delphi study according to their physiological characteristics, conscious and arousal behavior, etc. The experts’ opinions were collected using an e-Delphi method. A consensus was reached after two-round online questionnaire surveys. Patients with consciousness impairment may have dysfunction of one or more sensory channels.⁴¹ Multiple sensory channels are often examined to ensure that more reliable consciousness behaviors are triggered.⁴² The scales widely used in clinical practice, such as GCS, FOUR, and CRS-R, include several subscales. Each item assesses the presence or absence of a specific physical sign that represents the integrity of brain function.⁴³ In addition, the compilation of clinical consciousness assessment scales is mainly based on the clinical experience and theoretical knowledge of experts and a reference to other similar scales. The three aspects of behavioral response, motor response, verbal response, and eye-opening, which GCS chose to examine, were selected from many previous reports on coma.⁴⁴ The seven subscales of CRS-R were selected from an original pool of 41 indicators designed by a team of multidisciplinary professionals specializing in acute traumatic brain injury rehabilitation.⁴⁵ The Nociception Coma Scale (NCS) was developed based on pain scales for non-communicative patients with dementia and newborns to evaluate behaviors linked to nociceptive pain in the vegetative and minimally conscious state.⁴⁶ Similarly, this present study finally developed a behavioral assessment scale for NHPs comprising 37 items organized hierarchically within seven dimensions, including visual function, auditory function, motor function, orofacial movements, arousal, brainstem reflexes, and respiration after a two-round Delphi study. Each dimension comprises items arranged in a hierarchical order, signifying a progression in neurological functioning. The lowest items indicate a lack of function, while the highest represent cortical-based capabilities. The visual function dimension (I) items were extracted from clinical evaluation scales such as the CRS-R, western neurosensory stimulation profile, and the coma near coma scale, among others. Visual function is crucial for maintaining and regulating consciousness and is responsible for processing most sensory information reaching the brain.⁴⁷ Therefore, visual function evaluation is important for assessing patients with LoC. The auditory function dimension (II) items were extracted from clinical evaluation scales such as the Loewenstein Communication Scale, DOC Scale, and CRS-R. The auditory cortex is a significant brain area involved in processing acoustic stimuli and is closely connected to other brain areas affected by consciousness. Studies have demonstrated that activation of the auditory cortex correlates with the level of consciousness,⁴⁸ making auditory function assessments a valuable part of clinical consciousness assessment scales. The motor function dimension (III) items were extracted from clinical evaluation scales such as the GCS, FOUR, and CRS-R. Motor function plays a vital role in maintaining consciousness and is often evaluated in the clinical assessment of patients with altered mental status.⁴⁹ Assessment of motor function can aid in the differential diagnosis of underlying etiologies of altered mental status and provide valuable information about a patient's neurological status and potential for recovery. The arousal dimension (V) items were extracted from clinical evaluation scales such as the CRS-R, comprehensive level of consciousness scale, and the Steward awakening score. The relationship between the arousal dimension and consciousness is complex and multifaceted. It is closely linked to attention, perception, and cognition and plays a crucial role in regulating the brain's information processing flow. Research has shown that changes in arousal can significantly impact an individual's level of consciousness, ranging from drowsiness to deep coma.⁵⁰ The brainstem reflex dimension (VI) items were extracted from multiple clinical assessment scales, including the Glasgow-Liege scale, Innsbruck coma scale, FOUR, etc. The brainstem reflexes are automatic responses mediated by the brainstem that are critical for maintaining vital physiological functions such as respiration, blood pressure, and heart rate.⁵¹ These reflexes can also provide important clinical information about the integrity of the brainstem and its connections to higher brain centers. Assessing brainstem reflexes is particularly important for evaluating the level of consciousness in patients with neurological injury or illness. Research has shown a correlation between the presence and strength of brainstem reflexes and the level of consciousness, where their absence or significant reduction is often associated with deeper levels of coma and worse prognoses.⁵² Therefore, including brainstem reflex testing in the neurological examination of patients with altered levels of consciousness is essential. The respiratory function dimension (VII) items were extracted from clinical evaluation scales such as the Loewenstein communication scale, FOUR, modified Adrete scale, etc. The stability and orderliness of the respiratory function are crucial for maintaining a normal consciousness state.⁵³ Abnormal respiratory function can affect the brain's oxygen supply and carbon dioxide elimination, resulting in impaired brain function and altered levels of consciousness. Conversely, changes in consciousness can also affect respiratory function. Hence, monitoring and evaluating the respiratory function status is essential for assessing and managing the level of consciousness in clinical practice. Notably, the statistical indicators of orofacial movements dimension (IV) were found to be at a critical level and prompted serious discussion after the first round of survey. The orofacial movement dimension and its secondary items were primarily extracted and incorporated from clinical scales such as the facial expression subscale of the NCS,⁵⁴ the coma exit chart,⁵⁵ the oromotor/verbal function subscale of CRS-R,⁵⁶ and the eating and emotional response subscales of the Chinese vegetative state scale,⁵⁷ among others. These indicators were also derived from similar studies on NHPs, such as the limb/face movements and oral signs indicators developed by Redinbaugh et al.¹⁸ based on clinical arousal scales. Since NHP animals cannot express their mental states through language, orofacial movements may be a valuable indicator for assessing consciousness. Therefore, after discussion among the research group, it was modified and retained in the evaluation scale.

The identification and selection of experts are crucial in a Delphi study.⁵⁸ The experts involved in this study had high levels of professional representativeness and authority. Their opinions were well-coordinated and focused. Additionally, they displayed remarkable enthusiasm and engagement in participating in this study. Twenty-eight experts with senior professional titles or doctoral degrees from multiple relative academic backgrounds, including rehabilitation medicine, neurosurgery, neurology, neuroscience, and psychology, participated in this study. They were clinical scientists, professors, or researchers from 13 provinces in China. The response rate to this Delphi survey was high (93.33% and 100.00% in the two rounds, respectively), and several constructive suggestions and comments were received. The sufficient positive coefficient also indicated that experts cared about this research and were enthusiastic about participation.⁵⁹ Usually, the expert authority coefficient ≥0.70 is considered acceptable.⁶⁰ Meanwhile, it is 0.809 and 0.820 in the two rounds of this work, respectively, indicating that the experts consulted had high authority. All items’ scores on the importance and feasibility met the pre-established criteria, meaning that expert opinions were concentrated and the items were of high importance and feasibility. The coefficients of variation of all items were <0.25, indicating a high degree of coordination among them. Kendall's coordination coefficient W, used to evaluate the agreement among 28 experts, was 0.493, 0.561, and 0.493 in the first round and 0.448, 0.588, and 0.502 in the second round, with all P < 0.001. This indicated that all experts agreed with the consensus results. In summary, these statistical indicators meet the standards of Delphi studies, which suggests that the behavioral assessment scale of consciousness for NHPs developed in this study is scientifically valid.

This study presents several limitations. While the Delphi method was utilized to compile the behavioral assessment scale of consciousness for NHPs, which benefits from the intelligence and expertise of the participating experts, it still has subjective limitations. Therefore, further evaluation of the self-designed scale is required to establish its reliability, validity, and other psychometric indicators, making it a more reliable and effective tool for subsequent studies. For instance, an NHP model with anesthesia-induced LoC can be developed to combine the behavior analysis obtained from the scale with objective techniques such as EEG, PET-CT, fMRI, and others to validate the scale. Moreover, limiting the expert sample for the Delphi study to only those from China may introduce bias and potential limitations. By excluding experts from other regions, the results may not fully reflect the global perspectives and experiences relevant to the topic of interest. Future studies could aim to include experts from diverse regions and backgrounds. This could be achieved through collaborations with international research institutions or organizations to recruit participants from different countries and cultures.

Conclusions

In this study, we developed a behavioral assessment scale of consciousness for NHPs using a two-round online Delphi method. The consensus achieved by a multidisciplinary expert panel provides a comprehensive and detailed tool for measuring conscious states in NHPs or NHP models with LoC. This tool will facilitate the study of the underlying mechanisms of consciousness and LoC in NHPs and have significant implications for translational research into human consciousness disorders. Additionally, this study highlights the importance of developing assessment tools tailored to NHPs to bridge the gap between animal and human studies, and further advance our understanding of consciousness.

Supplemental Material

sj-doc-1-sci-10.1177_00368504231200995 - Supplemental material for Behavioral assessment scale of consciousness for nonhuman primates: A Delphi study

Supplemental material, sj-doc-1-sci-10.1177_00368504231200995 for Behavioral assessment scale of consciousness for nonhuman primates: A Delphi study by Weiming Sun, Xiangli Dong, Guohua Yu, Yang Yang, Binjun He, Yingming Wei, Shijin Li, Zhen Feng and Chaolin Ma in Science Progress

Footnotes

Acknowledgments

We wish to express our sincere gratitude to all the experts who participated in the Delphi process, which provided invaluable insights and expertise that significantly contributed to the research presented in this publication. Furthermore, we would like to extend our heartfelt thanks to Professor Haibo Di and his research team at the International Unresponsive Wakefulness Syndrome and Consciousness Science Institute, School of Basic Medical Sciences, Hangzhou Normal University, for their guidance and support.

Author contributions

Weiming Sun: Conceptualization; methodology; investigation; data curation; data analysis; formal analysis; writing-original draft; project administration; writing—review and editing. Xiangli Dong: Data analysis; methodology; formal analysis; writing—review and editing. Guohua Yu: Investigation; writing—review and editing. Yang Yang: investigation; formal analysis; writing—review and editing. Binjun He: Investigation; data analysis; writing—review and editing. Yingming Wei: Investigation; data curation; data analysis; writing—review and editing. Shijin Li: Investigation; writing—review and editing. Zhen Feng: Conceptualization; methodology; investigation; project administration; writing—review and editing. Chaolin Ma: Conceptualization; methodology; project administration; supervision; resources; writing—review and editing.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding author.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics approval

This study was approved by the Medical Research Ethics Committee of the First Affiliated Hospital of Nanchang University on 7 March 2022, with the code number 2022–60–2.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work was supported in the Natural Science Foundation of Jiangxi Province (20232BAB216090), and the Special Fund for Postgraduate Innovation of Jiangxi Province (YC2021-B021).

Informed consent statement

Informed consent was obtained from all subjects involved in the study.

ORCID iD

Weiming Sun

Supplemental material

Supplemental material for this article is available online.

Author biographies

Weiming Sun obtained his master’s degree in the Department of Psychology, Nanchang University in 2015 and worked in the Department of Rehabilitation Medicine, the First Affiliated Hospital of Nanchang University since 2016. He obtained his PhD in 2023 under the supervision of Prof. Chaolin Ma and Prof. Zhen Feng in the Institute of Life Science, Nanchang University. His research focuses on neurobiology and neurological rehabilitation.

Xiangli Dong obtained her master’s degree in the Department of Psychology, Nanchang University in 2017 and worked in the Department of Psychosomatic Medicine, the Second Affiliated Hospital of Nanchang University since 2019. Her research focuses on psychological assessment and psychological therapy.

Guohua Yu is a distinguished associate professor with Department of Rehabilitation Medicine, Hospital of Nanchang University. Her research focuses on rehabilitation of neurological diseases.

Yang Yang is a master student in Neurobiology from the School of Life Sciences, Nanchang University since 2021 under the supervision of Prof. Chaolin Ma.

Binjun He obtained his master’s degree in Neurobiology from the School of Life Sciences, Nanchang University in 2023 under the supervision of Prof. Chaolin Ma.

Yingming Wei obtained her master’s degree in Neurobiology from the School of Life Sciences, Nanchang University in 2023 under the supervision of Prof. Chaolin Ma.

Shijin Li is a master student in Neurobiology from the School of Life Sciences, Nanchang University since 2021 under the supervision of Prof. Chaolin Ma.

Zhen Feng is a distinguished professor with Department of Rehabilitation Medicine, Hospital of Nanchang University. She obtained her master’s degree from Wuhan University, China. Her research focuses on rehabilitation of neurological diseases, especially the clinical research and basic research of consciousness disorders.

Chaolin Ma is a distinguished professor with Institute of Life Science, Nanchang University and the principal investigator of Cognitive Function and Disease Lab. He obtained his PhD in Neurobiology from Fudan University, China. His research focuses on the functions of the prefrontal cortex in working memory, decision, making and attention-deficit hyperactivity disorder, etc.

References

Weaver

Cogan

O'Brien

, et al. Determining the hierarchy of coma recovery scale-revised rating scale categories and alignment with aspen consensus criteria for patients with brain injury: A Rasch analysis. J Neurotrauma 2022; 39: 1417–1428.

Monti

Schnakers

. Flowchart for implementing advanced imaging and electrophysiology in patients with disorders of consciousness: to fMRI or not to fMRI? Neurology 2022; 98: 452–459.

Jain

Ramakrishnan

. Electrophysiological and neuroimaging studies – during resting state and sensory stimulation in disorders of consciousness: A review. Front Neurosci 2020; 14: 555093.

O'Donnell

Browne

Kilbaugh

, et al. Challenges and demand for modeling disorders of consciousness following traumatic brain injury. Neurosci Biobehav Rev 2019; 98: 336–346.

Campos-Luna

Miller

Beard

, et al. Validation of mouse welfare indicators: A Delphi consultation survey. Sci Rep 2019; 9: 10249.

Sturgeon

Ramirez

JSB

, et al. Interindividual variability of functional connectivity in awake and anesthetized rhesus macaque monkeys. Biol Psychiatry Cogn Neurosci Neuroimaging 2019; 4: 543–553.

Roelfsema

Treue

. Basic neuroscience research with nonhuman primates: A small but indispensable component of biomedical research. Neuron 2014; 82: 1200–1204.

Chang

Zhang

Poo

, et al. Spontaneous expression of mirror self-recognition in monkeys after learning precise visual-proprioceptive association for mirror images. Proc Natl Acad Sci USA 2017; 114: 3258–3263.

Lyn

Greenfield

Savage-Rumbaugh

, et al. Nonhuman primates do declare! A comparison of declarative symbol and gesture use in two children, two bonobos, and a chimpanzee. Lang Commun 2011; 31: 63–74.

10.

Sondag

Ruijter

Tjepkema-Cloostermans

, et al. Early EEG for outcome prediction of postanoxic coma: Prospective cohort study with cost-minimization analysis. Crit Care 2017; 21: 111.

11.

Qin

Duncan

, et al. GABAA receptor deficits predict recovery in patients with disorders of consciousness: a preliminary multimodal [(11)C]flumazenil PET and fMRI study. Hum Brain Mapp 2015; 36: 3867–3877.

12.

Fan

Zhuo

, et al. The human brainnetome atlas: A new brain atlas based on connectional architecture. Cereb Cortex 2016; 26: 3508–3526.

13.

Reith

Van den Brande

Synnot

, et al. The reliability of the Glasgow coma scale: A systematic review. Intensive Care Med 2016; 42: 3–15.

14.

Foo

Loan

JJM

Brennan

. The relationship of the FOUR score to patient outcome: A systematic review. J Neurotrauma 2019; 36: 2469–2483.

15.

Frigerio

Molteni

Colombo

, et al. Neuropsychological assessment through coma recovery scale-revised and coma/near coma scale in a sample of pediatric patients with disorder of consciousness. J Neurol 2022; 270: 1019–1029.

16.

Gennarelli

Thibault

Adams

, et al. Diffuse axonal injury and traumatic coma in the primate. Ann Neurol 1982; 12: 564–574.

17.

Tasserie

Uhrig

Sitt

, et al. Deep brain stimulation of the thalamus restores signatures of consciousness in a nonhuman primate model. Sci Adv 2022; 8: eabl5547.

18.

Redinbaugh

Phillips

Kambi

, et al. Thalamus modulates consciousness via layer-specific control of cortex. Neuron 2020; 106: 66–75.e12.

19.

Powell

. The Delphi technique: Myths and realities. J Adv Nurs 2003; 41: 376–382.

20.

Knaapen

Hall

Moulin

, et al. International core outcome set for acute simple appendicitis in children: Results of a systematic review, Delphi study, and focus groups with young people. Ann Surg 2022; 276: 1047–1055.

21.

Langbroek

Wolkerstorfer

Horbach

SER

, et al. A core outcome domain set for clinical research on capillary malformations (the COSCAM project): An e-Delphi process and consensus meeting. Br J Dermatol 2022; 187: 730–742.

22.

Pari

. Collaboration for research implementation, training in critical care, Asia Africa 'CCAA'. Development of a quality indicator set to measure and improve quality of ICU care in low- and middle-income countries. Intensive Care Med 2022; 48: 1551–1562.

23.

Sun

Liu

, et al. The evaluation indexes suitable for nonhuman primates can be extracted from clinical consciousness disorder assessment scales: A hypothesis. J Integr Neurosci 2022; 21: 159.

24.

McPherson

Reese

Wendler

. Methodology update: Delphi studies. Nurs Res 2018; 67: 404–410.

25.

Babbie

. The practice of social research. 10th Chinese Language Edition. Beijing: Huaxia Publisher, 2005.

26.

Makhmutov

. The Delphi method at a glance. Pflege 2021; 34: 221.

27.

Liu

, et al. Conditions required to ensure successful detection and management of mild cognitive impairment in primary care: A Delphi consultation study in China. Front Public Health 2022; 10: 943964.

28.

Liu

Chen

. Identifying the service capability of long-term care facilities in China: An e-Delphi study. Front Public Health 2022; 10: 884514.

29.

Wiener

Chacko

Brown

, et al. Delphi survey of research priorities. J Nurs Manag 2009; 17: 532–538.

30.

Kang

Zhong

Chen

, et al. Development and validation of a website to guide decision-making for disorders of consciousness. Front Aging Neurosci 2022; 14: 934283.

31.

Calderon

Schiff

. Objective and graded calibration of recovery of consciousness in experimental models. Curr Opin Neurol 2021; 34: 142–149.

32.

Hayashi

Hou

Glasser

, et al. The nonhuman primate neuroimaging and neuroanatomy project. Neuroimage 2021; 229: 117726.

33.

Giacino

Schnakers

Rodriguez-Moreno

, et al.

Behavioral assessment in patients with disorders of consciousness: Gold standard or fool's gold?

Prog Brain Res 2009; 177: 33–48.

34.

Kondziella

Bender

Diserens

, et al. European Academy of Neurology guideline on the diagnosis of coma and other disorders of consciousness. Eur J Neurol 2020; 27: 741–756.

35.

Huang

Green

, et al. Dynamic changes in rhythmic and arrhythmic neural signatures in the subthalamic nucleus induced by anaesthesia and tracheal intubation. Br J Anaesth 2020; 125: 67–76.

36.

Schroeder

Irwin

Gaidica

, et al. Disruption of corticocortical information transfer during ketamine anesthesia in the primate brain. Neuroimage 2016; 134: 459–465.

37.

Bastos

Donoghue

Brincat

, et al. Neural effects of propofol-induced unconsciousness and its reversal using thalamic stimulation. Elife 2021; 10: e60824.

38.

Yanagawa

Chao

Hasegawa

, et al. Large-scale information flow in conscious and unconscious states: An ECoG study in monkeys. PLoS One 2013; 8: 8e80845.

39.

Koch

Massimini

Boly

, et al. Neural correlates of consciousness: Progress and problems. Nat Rev Neurosci 2016; 17: 307–321.

40.

Huang

Mao

, et al. Consensus on the health education of home-based negative pressure wound therapy for patients with chronic wounds: A modified Delphi study. Burns Trauma 2021; 9: tkab046.

41.

Blond

Andersen

Wæhrens

, et al. Perspectives on consciousness in patients with disorders of consciousness from brain injury: Group concept mapping study across clinic, research, and families. BMC Health Serv Res 2023; 23: 471.

42.

Birch

Schnell

Clayton

. Dimensions of animal consciousness. Trends Cogn Sci 2020; 24: 789–801.

43.

Sattin

Minati

Rossi

, et al. The coma recovery scale modified score: A new scoring system for the coma recovery scale-revised for assessment of patients with disorders of consciousness. Int J Rehabil Res 2015; 38: 350–356.

44.

Teasdale

Jennett

. Assessment of coma and impaired consciousness. A practical scale. Lancet 1974; 2: 81–84.

45.

Giacino

Kalmar

Whyte

. The JFK coma recovery scale-revised: Measurement characteristics and diagnostic utility. Arch Phys Med Rehabil 2004; 85: 2020–2029.

46.

Chatelle

Majerus

Whyte

, et al. A sensitive scale to assess nociceptive pain in patients with disorders of consciousness. J Neurol Neurosurg Psychiatry 2012; 83: 1233–1237.

47.

Babiloni

Marzano

Soricelli

, et al. Cortical neural synchronization underlies primary visual consciousness of qualia: evidence from event-related potentials. Front Hum Neurosci 2016; 10: 310.

48.

Rohaut

Faugeras

Chausson

, et al. Probing ERP correlates of verbal semantic processing in patients with impaired consciousness. Neuropsychologia 2015; 66: 279–292.

49.

Edlow

Claassen

Schiff

, et al. Recovery from disorders of consciousness: Mechanisms, prognosis and emerging therapies. Nat Rev Neurol 2021; 17: 135–156.

50.

Albrechtsen

Riis

RGC

Amiri

, et al. Impact of MRI on decision-making in ICU patients with disorders of consciousness. Behav Brain Res 2022; 421: 113729.

51.

Yurchenko

. From the origins to the stream of consciousness and its neural correlates. Front Integr Neurosci 2022; 16: 928978.

52.

Benghanem

Mazeraud

Azabou

, et al. Brainstem dysfunction in critically ill patients. Crit Care 2020; 24: 5.

53.

Grund

Pabst

, et al. Respiration, heartbeat, and conscious tactile perception. J Neurosci 2022; 42: 643–656.

54.

Wang

Hua

, et al. A study of the reliability and validity of the Chinese version of the nociception coma scale-revised. Clin Rehabil 2020; 34: 1112–1121.

55.

Freeman

. The coma exit chart: Assessing the patient in prolonged coma and the vegetative state. Brain Inj 1996; 10: 615–624.

56.

Zhang

Wang

Schnakers

, et al. Validation of the Chinese version of the coma recovery scale-revised (CRS-R). Brain Inj 2019; 33: 529–533.

57.

Wang

Gao

, et al. Clinical application of Chinese Nanjing persistent vegetative state scale. Chin Med J (Engl) 2020; 133: 1404–1408.

58.

Bogetz

Johnston

Thienprayoon

, et al. Development of primary palliative care end-of-life quality measures: A modified Delphi process. Pediatrics 2022; 150: e2022058241.

59.

Diamond

Grant

Feldman

, et al. Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 2014; 67: 401–409.

60.

Hong

Pluye

Fàbregues

, et al. Improving the content validity of the mixed methods appraisal tool: A modified e-Delphi study. J Clin Epidemiol 2019; 111: 49–59.e1.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.10 MB