Pain measurement: Visual Analogue Scale (VAS) and Verbal Rating Scale (VRS) in clinical trials with OTC analgesics in headache

Abstract

Aim: The aim was to assess the performance of the Visual Analogue Scale (VAS) in patients recruited in a clinical trial with over the counter analgesics in headache.

Methods: The Thomapyrin Study showed the significant superiority of the fixed combination of acetylsalicylic acid + paracetamol + caffeine over the combination without caffeine, the single preparations, and placebo in the treatment of headache. Patients enrolled into the study were trained in the handling of the VAS by naming categories of a 6-point Verbal Rating Scale (VRS). These data were used to evaluate the level of order consistency between the VAS and VRS, to deduce cut-off points for rescaling the continuous VAS into a discrete ordinal scale using the receiver operating characteristic methodology, and to assess the test-retest performance.

Results: Approximately 75% of the patients recorded the pain intensity on the VAS in the same order as given on the VRS. However, in 12.6% of patients, the German terms ‘leicht’ (mild) and ‘mäßig’ (moderate) were mixed up regarding their order on the VAS. Substantial overlapping of the frequency distributions of the VAS assessment were found for the VRS categories mild and moderate pain as well as severe and very severe pain. Grouping of the VAS assessments into a discrete ordinal scale necessitated a non-equidistant rescaling based on the categories of the VRS. By means of analysis of the receiver operating characteristic curves, the following cut-off points were determined on a 100 mm VAS: no pain 0–2 mm, mild pain 2–17 mm, moderate pain 17–47 mm, severe pain 47–77 mm, very severe pain 77–96 mm, most severe pain imaginable 96–100 mm. Repeated assessment up to several months after the first assessment demonstrated a test-retest agreement on the VAS in 61.0–91.4% of the patients, depending on the VRS category.

Conclusions: This study shows that the VRS categories cannot be presented in an equidistant manner on the VAS, and that contrary to previous assumptions, the pain intensity descriptors are less clear and can have different meanings in different languages. Therefore, both in the 3rd edition of the International Headache Classification (ICHD-III) and in the guidelines for clinical trials of patients with headache illnesses, rather than a 4-grade VRS, a 6-grade or higher level VRS or a VAS should be recommended, with correspondingly broadly defined anchor points.

Keywords

Clinical endpoint cut-off point headache migraine OTC analgesics ROC curve analysis tension-type headache visual analogue scale (VAS)verbal rating scale (VRS)

Introduction

Pain is a personal, subjective experience that comprises sensory-discriminative, motivational-affective and cognitive-evaluative dimensions (1). Reliable and accurate pain assessment is a prerequisite for successful pain management (2). Reliable and valid measures of pain are also essential for conducting clinical trials of pain treatments. Perhaps the most important aspect of a pain measure’s validity is its sensitivity or ability to detect changes in pain over time and due to treatment. Without adequate sensitivity, the real effects of pain interventions cannot be estimated, and real effects of treatment can go undetected (3). Fortunately, in most situations, the most commonly used measures of pain intensity, including visual analogue scales (VASs), numerical rating scales (NRSs) and verbal rating scales (VRSs, also known as categorical scales, verbal descriptor scales, fixed descriptor scales or Likert scales) have been shown to have adequate sensitivity to changes in pain associated with treatment across many populations and settings (4).

Accurate evaluation of actual headache intensity is crucial in both clinical and research settings (5). Although pain is a multidimensional phenomenon, the severity of migraine headache is usually characterized by pain intensity, frequency, duration and disability (6). The evolution of verbal and non-verbal scales can be traced from the traditional ordinal scales which ask patients to rate their pain as none, mild, moderate or severe, or from 1 to 4, l to 10, etc., to the verbal descriptor tests and numerical scaling procedures (5). One of the most widely used non-verbal measurement techniques is the VAS. This measure is a form of cross-modality matching in which line length is the response continuum (5).

The VRS (4 grade scale) and the VAS are both recommended for assessing headache severity in the 2nd edition of the International Headache Society guidelines for controlled trials of drugs in migraine (7) and tension-type headache (8). It is reported that the VAS allows a greater sensitivity for determining treatment effects than a categorical scale such as the VRS (4 grade scale), because there is a greater range of possible scores (9), but it was discussed that the interpretation of statistically significant but small changes in VAS scores could be difficult (10). On the other hand, for analysis purposes, the VRS (4 grade scale), widely used during the development of triptans, is usually reduced to a binary scale (patients are categorized as responders or non-responders), which might be expected to further reduce its sensitivity compared with a continuous scale (9).

As part of the Thomapyrin Study that showed the significant superiority of the fixed combination of acetylsalicylic acid + paracetamol + caffeine over the combination without caffeine, the single preparations, and placebo in the treatment of headache (11), patients enrolled into the study were trained in the handling of the VAS by naming categories of a 6-point VRS. The same procedure was repeated in all patients at study conclusion.

These data were used:

to assess both scales with respect to consistency of rank order of the categories;

to determine cut-off points on the VAS related to the categories of the VRS;

to evaluate how the categories of the VRS are represented on the VAS (equidistant, non-equidistant, specific pattern); and

to assess test-retest reliability after repetition of the complete training procedure at study conclusion.

Methods

Patients

The data for this analysis were collected as part of a randomized, placebo-controlled, double-blind, multi-centre parallel group trial with six treatment arms, conducted between September 1998 and January 2003 (11). The primary objective of the study was to investigate the efficacy, safety and tolerability of the fixed combination of acetylsalicylic acid + paracetamol + caffeine in comparison with the combination without caffeine, the single preparations, and placebo in patients who were used to treating their episodic tension type headache or migraine attacks with non-prescription analgesics.

The patients were enrolled by practitioners and specialists in general and internal medicine throughout Germany. Male or female patients (18–65 years) who were not consulting for headache were asked whether they had headaches that they treated with non-prescription analgesics. Usual headaches had to meet International Headache Society (2) criteria for episodic tension-type headache (2.1) and/or migraine with or without aura (1.1, 1.2.1) (12). Patients must have experienced these headaches for at least 12 months with a minimum of two headache episodes within the previous 3 months.

Patients were excluded if previous or concomitant diseases or medication could interfere with one of the study drugs or influence headache symptoms. Drug overuse connected with the headache and alcohol or drug abuse was also an exclusion criterion, as was pregnancy, lactation or participation in another clinical trial within 4 weeks of entering this study.

Before enrolment the patients gave their written informed consent according to paragraphs 40 and 41 of the German Drug Law (AMG) and ICH GCP (International Conference on Harmonization, Guidance for Good Clinical Practice, E6) standards. Patients were allowed to terminate participation in the trial at any time, without giving reasons. The study was conducted in accordance with the Declaration of Helsinki, the German Drug Law (AMG), and ICH GCP standards and did not start before Independent Ethics Committee approval was obtained.

Training of the VAS

Patients recorded pain intensity on a 100 mm VAS by drawing a vertical line on the horizontally positioned VAS before and at several time points after drug intake during the course of the study in a diary. The left end of the VAS represented ‘no pain’ and the right end ‘most severe pain imaginable’ with no intermediate divisions or descriptive terms. During the screening visit (visit 1), the investigator gave a standardized explanation of the VAS using a written explanatory text. He handed a VAS to the patient and named one of the categories of a 6-point VRS used to describe pain intensity (the original German wording is given in brackets): 0 = no pain (kein Schmerz), 1 = mild pain (leichter Schmerz), 2 = moderate pain (mäßiger Schmerz), 3 = severe pain (starker Schmerz), 4 = very severe pain (überaus starker Schmerz), 5 = most severe pain imaginable (stärkster vorstellbarer Schmerz). Within 30 seconds, the patient was to draw a vertical line on the horizontal scale, where, according to their assessment, the mentioned pain intensity would be represented, based on the experience of their own headache. This procedure was repeated until all six categories had been named randomly by the investigator according to a randomization list provided (PROC PLAN, SAS Institute Inc., Cary, NC, USA). Before naming the next category, the patient returned the page to the investigator so that the next evaluation was made without reference to the previous one. The patient was not informed on the number or the wordings of the VRS categories. The procedure was repeated at the patient’s final conclusion visit (visit 4). The data from patients who concluded at least one of the two trainings as planned for were included in the analyses.

Statistical analysis

Descriptive statistics and frequency distributions of the assessments on the VAS recorded in the training sessions were calculated, sorted by the scores of the VRS. The level of order consistency between the VAS and the VRS was analysed by evaluating within patients the permutations of the assessment on the VAS in comparison to the order of the scores of the VRS.

For rescaling the continuous VAS into a discrete ordinal scale, cut-off points were determined on the VAS at which the categories of the VRS could best be represented. These cut-off points were deduced from receiver operating characteristic (ROC) curves (13). A cut-off point on the VAS was to be defined where patients irrespective of the actual VRS were assigned to the higher pain category if the mark on the VAS was above the cut-off point or to the lower pain category if the mark was below. If the patient named the higher category on the VRS and their mark on the VAS was above the cut-off point, the outcome was called true higher. The ROC curve displays the true higher (sensitivity) versus the false higher (one minus specificity) rates for the range of possible cut-off points. As the sensitivity is a decreasing and the specificity an increasing function of the cut-off values, it is not possible to select a cut-off maximizing both. Instead we chose the cut-off point at which the sensitivity and specificity are closest assuming equal importance of sensitivity and specificity. The area under the ROC curve was calculated as a summary measure of its discriminatory ability. It ranges from 0.5 (no discriminatory accuracy, distribution of VAS for both categories identical) to 1.0 (perfect discriminatory accuracy, no overlapping of distribution of VAS for both categories). The AUC provides an estimation of the probability of correctly ranking a pair of randomly selected categories from the VRS on the VAS. The cut-off points were determined for each pair of the VRS scores separately using logistic regression analysis.

The repetition of the training procedure at the patient’s final conclusion visit enabled a test-retest evaluation several weeks or sometimes even months after the first training. Based on the cut-off points determined, the number of concordant and discordant pairs of classifications was summarized to evaluate a possible shift in the assessments on the VAS.

Results

Patient characteristics

Of the 2100 patients planned for the VAS training, 1871 patients concluded the training with complete data at visit 1 and 1457 patients at the final visit 4 (Table 1). These patients were included in the analyses. Patients were excluded if they did not take part in the training, the random list was not returned or the training ended with invalid or incomplete data. Non-compliance or use of a random list not according to the allocated randomization scheme was no reason for exclusion of the patients from the subsequent analyses. In total, 75% of the patients were women and 25% men with a median age of 38 years; range 16–72 years.

Table 1.

Patient disposition

	Visit 1	Visit 4
Total number of patients	2100	2100
Training not done	22	434
No random list	164	170
Invalid data	35	31
VAS incomplete	8	8
Patients with complete training	1871	1457
No compliance	58	33
Patients compliant	1813	1424
False random list used	19	5
Patients with correct training	1794	1419

Differences between treatment groups

The superior efficacy of the triple combination acetylsalicylic acid + paracetamol + caffeine could be shown for all efficacy endpoints such as the time to 50% pain relief (primary endpoint), time until reduction of pain intensity to 10 mm, pain intensity difference, percentage sum of pain intensity difference weighted, extent of impairment of daily activities, and patient’s global efficacy assessment. Further details on the efficacy, tolerability and safety results were reported previously (11).

Frequency distribution of the assessment on VAS by VRS categories

The two extreme categories of the VRS (no pain and most severe pain imaginable) resulted in assessments very close to the anchor points of the VAS with very low inter-individual variability (Table 2). The frequency distribution of the VAS assessments for the other four scores of the VRS showed somewhat higher variability. The shape of the distribution was nearly symmetrical and did not markedly deviate from the normal distribution fitted to the data (Figure 1). Substantial overlapping of the VAS assessments could be seen for the categories mild and moderate pain as well as the categories severe and very severe pain. Except for a trend to slightly higher average values (less than 1 mm on the 100 mm VAS), inter-individual variability and overall frequency distributions of the VAS assessments were very similar for all categories of the VRS at the first training at the study entry and at the repetition at the study conclusion (Table 2, Figure 1).

Figure 1.

Frequency distribution of the assessment on the visual analogue scale (VAS) sorted by the scores of the verbal rating scale (VRS) and the two visits. The observed frequencies and the fitted normal distribution are presented. Dotted lines mark the cut-off points on the VAS in the calibration on the VRS at visit 1.

Table 2.

Descriptive statistics of the assessment on the visual analogue scale (VAS) sorted by the scores of the verbal rating scale (VRS)

	VRS score	Mean	SD	Median	Quartile
	VRS score	Mean	SD	Median	Q1	Q3
Visit 1	None	1.3	1.6	1	1	1
(n = 1871)	Mild	13.4	8.5	12	8	17
	Moderate	28.6	13.8	28	19	37
	Severe	67.3	13.7	68	59	77
	Very severe	83.7	11.2	85	78	91
Most severe imaginable	98.3	5.4	100	99	100
Visit 4	None	1.4	3.8	1	1	1
(n = 1457)	Mild	13.9	8.0	13	9	18
	Moderate	29.2	13.8	28	19	39
	Severe	67.5	13.5	68	59	77
	Very severe	84.3	9.9	85	79	91
Most severe imaginable	98.7	4.9	100	100	100

Order consistency between VRS and VAS

Overall, 74.5% of the patients at visit 1 and 78.0% at visit 4 rated all six categories of the VRS in the same order on the VAS as given on the VRS (Table 3). Except for some rare cases of probably complete misunderstanding or mix-up of the anchor points of the scale, noteworthy order inconsistencies between VRS and VAS were observed for the categories with overlapping frequency distributions: 12.6% and 13.2% between mild and moderate pain and 9.1% and 6.7% between severe and very severe pain at visit 1 and visit 4, respectively. In addition, some patients rated very severe pain and most severe pain imaginable in reversed order on the VAS (5.1% and 3.0% at visit 1 and visit 4, respectively).

Table 3.

Number and percentage of patients with permutations of the assessment on the visual analogue scale (VAS) in comparison to the order of the scores of the verbal rating scale (VRS)

Permutation on VAS	Visit 1 (n = 1871)		Visit 4 (n = 1457)
Permutation on VAS	n	%	n	%
Without	1393	74.5	1137	78.0
Mild/none	8	0.4	7	0.5
Moderate/none	3	0.2	4	0.3
Moderate/mild	236	12.6	193	13.2
Severe/none	0	0.0	2	0.1
Severe/mild	2	0.1	3	0.2
Severe/moderate	17	0.9	18	1.2
Very severe/none	0	0.0	2	0.1
Very severe/mild	1	0.1	2	0.1
Very severe/moderate	4	0.2	2	0.1
Very severe/severe	171	9.1	97	6.7
Most severe imaginable/none	0	0.0	1	0.1
Most severe imaginable/mild	1	0.1	2	0.1
Most severe imaginable/moderate	2	0.1	2	0.1
Most severe imaginable/severe	21	1.1	12	0.8
Most severe imaginable/very severe	95	5.1	44	3.0

Number of patients and percentages do not add up to total and 100%, respectively, as some patients may have several permutations.

Calibration of VAS on VRS

Grouping of the VAS assessments into a discrete scale resulted in non-equidistant rescaling based on the named categories of the VRS. The best fit with the VRS categories was found with the following cut-off points based on the ROC curves: no pain 0–2 mm, mild pain 2–17 mm, moderate pain 17–47 mm, severe pain 47–77 mm, very severe pain 77–96 mm, most severe pain imaginable 96–100 mm (Table 4, Figure 1). The cut-off points were shifted by 1 mm to higher VAS values at visit 4. The sensitivity was between 87.7% and 98.0% to correctly decide between no and mild pain, between moderate and severe pain and between very severe pain and most severe pain imaginable. The cut-off points for the other two adjacent VRS categories (mild/moderate pain, severe/very severe pain) could be defined with somewhat lower sensitivity (76.6 to 78.9%).

Table 4.

Cut-off points on the visual analogue scale (VAS) in the calibration on the verbal rating scale (VRS)

	VRS score	Cut-off points on VAS		ROC curve
	VRS score	Lower	Upper	Sensitivity (%)	AUC
Visit 1	None	0	2	95.1	0.985
(n = 1871)	Mild	2	17	76.6	0.832
	Moderate	17	47	90.9	0.971
	Severe	47	77	77.5	0.833
	Very severe	77	96	87.7	0.917
Most severe imaginable	96	100	–
Visit 4	None	0	3	98.0	0.991
(n = 1457)	Mild	3	18	78.7	0.835
	Moderate	18	48	91.4	0.973
	Severe	48	78	78.9	0.851
	Very severe	78	96	91.8	0.944
Most severe imaginable	96	100	–

The corresponding ROC curves are displayed in Figures 2 and 3 and the calculated AUC summarized in Table 4. The ROC curves corresponding to cut-off points separating none and mild, moderate and severe, and very severe and most severe pain imaginable approached very closely the upper left corner of the coordinate system, indicating excellent discrimination between the respective categories. This was confirmed by the calculated AUCs very close to 1 (Table 4). The ROC curves corresponding to the other two cut-off points were also far from the diagonal of the coordinate system, with all AUCs above 0.832 also indicating very good discrimination between the VRS categories.

Figure 2.

ROC curve for cut-off points of data from training at visit 1.

Figure 3.

ROC curve for cut-off points based on assessment at visit 4.

Test-retest assessment

The repetition of the complete procedure of VAS training at visit 4 demonstrated an average shift of less than 1 mm to higher values at visit 4 compared with visit 1 (Table 5). The intra-individual variability for all categories of the VRS was comparable in magnitude to the inter-individual variability observed at both visits (Tables 2 and 5). This resulted in a shift to higher or lower categories on the VAS by not more than 20.6% of patients using the cut-off points on the VAS as determined above (Table 6). Shifts occurred mainly between adjacent categories. Complete test-retest agreement on the VAS could be demonstrated in 61.0–91.4% of the patients depending on the VRS category.

Table 5.

Descriptive statistics of the difference between visit 1 and visit 4 in the assessment on the visual analogue scale (VAS) sorted by the scores of the verbal rating scale (VRS)

VRS score	Mean	SD	Median	Quartile
VRS score	Mean	SD	Median	Q1	Q3
None	0.1	4.0	0	0	0
Mild	0.4	10.4	1	−5	5
Moderate	0.6	15.6	0	−8	9
Severe	0.1	15.4	0	−9	9
Very severe	0.8	12.5	0	−6	7
Most severe imaginable	0.3	5.6	0	0	0

Number of patients with data at both visits: 1427.

Table 6.

Number and percentage of patients with a shift between visit 1 and visit 4 in the category defined by the cut-off points of the assessment on the visual analogue scale (VAS) at visit 1

VRS score	Category at visit 1 compared with visit 4
	Equal		Higher		Lower
	n	%	n	%	n	%
None	1304	91.4	67	4.7	56	3.9
Mild	930	65.2	231	16.2	266	18.6
Moderate	889	62.3	244	17.1	294	20.6
Severe	871	61.0	267	18.7	289	20.3
Very severe	885	62.0	262	18.4	280	19.6
Most severe imaginable	1250	87.6	68	4.8	109	7.6

Number of patients with data at both visits: 1427.

Discussion

Pain is a personal, subjective experience, which makes it difficult to define and measure (14).

The continuous VAS and discrete categorical VRS are the two most commonly used scales to assess pain intensity/severity (1,15,16,17). The most common VAS consists of a 10 cm horizontal line (used in this study) or vertical line (range 5–20 cm) (18–20).

Usually, the VRS consists of 3–11 adjectives reflecting degrees of pain severity/intensity; the specific adjectives, such as ‘none’ or ‘no pain’, ‘little pain’, ‘slight pain’, ‘mild pain’, ‘moderate pain’, ‘severe pain’, ‘very severe pain’, ‘extreme pain’, ‘unbearable pain’, and ‘worst pain ever’, vary from version to version (21–24). Certain terms, such as little, slight, or mild pain, are often used synonymously in VRS, even though it has been shown that VAS scores differ significantly between, for instance, slight and mild (25). For headache pain, the necessary number of VRS categories has not been systematically examined. In migraine studies a 4-grade VRS (none, mild, moderate, severe) (‘Glaxo-’) scale of headache pain measurement was mostly chosen, as this was sufficient to depict the effect of sumatriptan and consequently that of other triptans in terms of their clinical development (26,27). However, the question of whether, from the perspective of headache patients, these four categories are really sufficient to depict their perceived headache intensity remains open. After all, in the current Thomapyrin Study 65 out of 1,765 patients rated the intensity of their usual headache as ‘very severe’ (28). With only four VRS categories, these patients are forced to choose ‘severe pain’, even though this underestimates their pain intensity. Serlin et al. refute that patients should be limited to rating pain as mild, moderate or severe only, and determine that ‘three categories do not tell us all we wish to know about pain severity for clinical trials or for clinical use’ (29). Moreover, a number of researchers have reported that reliability and respondent preferences for categorical scales are maximized with 7–11-grade scales (30–32).

Patients who preferred VAS over VRS justified their preference through the insufficient number of pain categories of the VRS, stating that the VAS was ‘more accurate,’ and that it ‘gave better indication of their pain’ (33,34). To be able to depict as many pain intensities as possible on the VAS, in the current study, a 6-grade VRS was chosen, which in particular also measures the ‘severe’ to ‘most severe pain imaginable’ pain intensities and which is becoming more frequently applied (35,36). As a VRS, with an uneven number of categories, can increase the risk of a regression towards the mean (37) and more than seven word descriptors are considered to decrease the reliability of the VRS (38), a 6-grade VRS was chosen, as was used in other pain and headache studies (39,40). A systematic investigation among chronic pain patients showed that 10 - and 11-point NRS, compared with a 101-point NRS, provide sufficient levels of discrimination, whereas a 6-point NRS proved to be slightly worse and a 2 - and 4-point NRS proved to be substantially worse (41). The 11-point NRS was 55% more sensitive than the 4-grade VRS (none, mild, moderate, severe) in detecting clinically important differences (6). The sensitivity was compared in a simulation model of postoperative pain. The 4-grade VRS was less sensitive than the VAS but the 11-point NRS and the VAS were approximately equally sensitive (42). However, it should be noted that the 11-point NRS and the 100 mm VAS are sometimes interchangeable and sometimes not (43,44).

At visit 1, for approximately 75% of patients, the ascending order of the six VRS categories was in accordance with a corresponding ascending order of the VAS ratings (increase in the mm values). According to Pilgrim (26), the requirement of an unambiguous translatability into a range of languages was a reason for choosing the 4-grade VRS as a measurement instrument for pain intensity in the clinical development of sumatriptan. However, at visit 1, 12.6% of the patients mixed up the terms ‘leicht’ (mild) and ‘mäßig’ (moderate) with regard to their order on the VAS, meaning that for these patients, the term ‘mäßig’ (moderate) represented a lower pain intensity than ‘leicht’ (mild). This does not suggest uniform terminology, and therefore argues against an easy translatability of these pain intensity-describing adjectives in German and English and vice versa. It is unclear whether the German VRS categories ‘mäßig’, ‘mittel’ or ‘mäßig stark’ can be equated with the English ‘moderate’ in terms of their meaning for pain patients. Other researchers regarded the terms ‘mäßig’ and ‘mittelstark’ in German as two different pain categories (45). In contrast to ‘leicht, mäßig, stark (strong) or sehr stark (very strong)’, the term ‘mittel’, in the sense of a medium pain intensity, calls for the notion of two anchor points or extremes, of which the pain intensity can then be judged as lying in the middle (at the same distance from the two). The term ‘mittel’, at least in German, does not fit ‘organically’ into a series of ‘kein’ (none), ‘leicht’ (mild), ‘mäßig’ (moderate), ‘stark’ (strong), ‘sehr stark’ (very strong), etc. Equally, the fact that the English term ‘moderate’ is used in both 4-grade VRS (cf. most triptan studies) and 5-grade VRS (34,45–49) also does not speak for its terminological clarity in English. In French, too, the term ‘modéré’ is used both in 5-grade VRS (‘absente’, ‘faible’, ‘modérée’, ‘intense’ and ‘extrément intense’ douleur (50,51) and in 4-grade VRS (‘pas du tout’, ‘un peu’,’modérée’, ‘sévère’ (52,53). In the SF 36, the English term ‘moderate’ and its official translations (Italian ‘moderato’; Danish ‘middelstærker smerter’) is also located above the middle in the 6-grade VRS (54).

Some researchers see an advantage of the VRS in the fact that the descriptors derive from general vocabulary and that this leads to category judgements on the basis of word meanings (55). By contrast, it has been pointed out on various occasions that it is difficult to specify exactly what will be taken as severe, mild, and so on (46); thus, this is also the case for headaches, where the definition of Tfelt-Hansen (27) was adopted with little modification into the ‘definition of terms’ of the ICHD-II.

Various studies showed a clear correlation between VAS and VRS (5,16,34,56–58). However, if solely the VAS is used, it is unclear which area on a VAS corresponds to the VRS categories mild, moderate or severe pain intensity, or, as Collins et al. put it in the title of their publication: ‘What is moderate pain in millimetres?’ (59). As clinical pain studies often require ‘at least moderate pain intensity’ as an inclusion criterion, on various occasions, and apparently arbitrarily so according to Collins et al. (59), the criterion has been operationalized on a 100 mm VAS with the guideline>30 mm (60), >60 mm (61) and >75 mm (62). Gursoy et al. (63) classify a VAS score of 0–4 mm as ‘no pain’, 5–44 mm as ‘mild pain’, 45–74 mm as ‘moderate pain’ and 75–100 mm as ‘severe pain’, while Page et al. classify <10 mm as mild pain, 10–50 mm as moderate pain and >50 mm as severe pain (64). Similarly, in migraine and headache studies, VAS areas have been allocated to VRS categories without further justification: for instance 0 cm as no pain, 1–3 cm mild pain, 4–7 cm moderate pain and 8–10 cm severe pain (65), or 1–3 cm mild, 4–6 cm moderate, 7–9 cm severe, 10 cm very severe pain (66), whereby in the latter, the reference to ICHD-II is incorrect, as it contains no such classification (67).

The VAS mean (\pm SD in mm) of the VRS categories of our study (none 1.3 ± 1.6, mild 13.4 ± 8.5, moderate 28.6 ± 13.8, severe 67.3 ± 13.7, very severe 83.7 ± 11.2, and most severe imaginable 98.3 ± 5.4) differed in part from those found in other studies. In a study with post-operative pain on movement (4-grade VRS), the following results emerged (mean and range in mm): for no pain 0.5 (0–10), slight pain 20.3 (2–50), moderate pain 36.8 (0–92) and severe pain 59.7 (21–92) (68). A knee surgery study (4-grade VRS) showed the following: none 1.4 ± 2.6 (0–28), mild 27.8 ± 10.4 (2–60), moderate 55.8 ± 11.7 (22–88), and severe 86.6 ± 9.9 (52–100) (69); while a laparotomy study (4-grade VRS) provided the following: none 0.0 ± 0.0 (0–15), mild 26.8 ± 9.4 (2–60), moderate 58.2 ± 8.6 (32–84), and severe 82.5 ± 10.0 (62–100) (69). A study of 785 migraine patients with and without aura (4-grade VRS) showed the following median VAS scores: no headache 0 mm, mild pain 20 mm, moderate pain 55 mm, and severe pain 82 mm (9).

Central to the current study was the question of how the cut-off points should be selected to separate the overlaps of distributions of two consecutive pain intensity categories regarding sensitivity and specificity in as optimal a manner as possible. One procedure for simultaneously reaching as optimal a sensitivity and specificity as possible, ‘the point closest to perfect differentiation and the point farthest from none’, as Perkins and Schisterman (70) put it, is the construction of a ROC curve (13). In headache research, ROC analysis has rarely been applied to date (66,71).

To convert VAS into VRS scores, cut-off points are mandatory (72). In the current study, the cut-off points were determined on the basis of ROC curves as follows: for no pain 0–2 mm, mild pain 2–17 mm, moderate pain 17–47 mm, severe pain 47–77 mm, very severe pain 77–96 mm, most severe pain imaginable 96–100 mm. They show high sensitivity values for the certainty of decision-making between the VRS categories. In line with expectation, this was lower for the category pairs mild/moderate and severe/very severe than for the others. Jensen et al. indicate that 100 mm VAS ratings of 0 to 4 mm can be considered no pain; 5 to 44 mm as mild pain; 45 to 74 mm as moderate pain; and 75 to 100 mm as severe pain (69); these clearly differ from the results of the current study. In terms of methodology, in our study the patients were not aware of how many pain descriptors would be mentioned to them. The upper cut-off point of the category moderate pain, with 47 mm, almost corresponds to the upper cut-off point of the category mild pain of Jensen et al., with 44 mm. In 603 patients with groin hernia repair, on the basis of paired VRS and VAS data, Loos et al. found cut-off points (in mm) of 0–8 as no pain, 9–32 as mild, 33–71 as moderate, and >71 as severe pain (72). Serlin et al. identified the cut-off points between the categories of a 4-grade VRS on the basis of an 11-point NRS. They found that classifying a score of 1 to 4 as mild pain, 5 to 6 as moderate pain and 7 to 10 as severe pain (29). Paul et al. found that classifying 1–4 as mild pain, >4–7 as moderate pain and >7–10 as severe pain constituted the optimal cut-off points (11-point NRS and 4-grade VRS) (73), whereas Palos et al. reported the cut-off points 1.3 to 3.6 (mild), 4.3 to 6.5 (moderate) and 7.5 to 9.8 (severe) (74).

The test-retest agreement between visits 1 and 4 in the current study was good. By contrast, the test-retest reliability (stability) was lowest for 2-grade, 3-grade and 4-grade VRS and was significantly higher for scales with more response categories; the most reliable scores were derived from scales with 7, 8, 9, or 10 response categories (32).

There are several limitations of this study that should be noted. Although the calibration of VRS on VAS was performed with headache patients, it was done during a pain-free interval, which might be considered both a strength and weakness of the study. A strength might be seen in the fact that the patients did not only undertake one VRS category, but rather all six VRS descriptors in a relatively short time interval and under virtually identical conditions, and without knowledge of the number of descriptors to be rated. In each case, the information was gathered once, meaning that a possible influence through a multiple pain measurement by means of various scales (69) was ruled out. However, the rating in a pain-free interval might also be considered a weakness, if one assumes that during the sensation of pain, a verbal pain intensity descriptor would be assigned a different VAS score. Nevertheless, this is not necessarily likely. A more critical effect might be exerted by the differing pain history of the patients, that is, based on the intensity of the previously experienced pain, the terms, and thus also the anchor points of the VAS, might acquire a different meaning. Moreover, an indication dependency cannot be ruled out, according to which, for instance, headache patients have not yet experienced ‘very severe pain’ in comparison to cancer patients. Nevertheless, this applies to all scales for measuring pain intensity. For this reason, Dionne et al. suggest an individual responder analysis for pain (75), which takes into account this individual pain experience.

Which scale, then, is particularly recommended for measuring pain intensity? Both the 4-grade VRS and the 100 mm VAS are recommended for the measurement of headache intensity in the IHS guidelines for clinical trials for migraine und tension-type headache (7,8), but there is no discussion of their relative merits (9). In the comments on ‘intensity of headache’, it is merely established that ‘this verbal/numerical scale is simple and has been used successfully in RCTs [randomised controlled trials] for acute migraine treatment for many years. Visual analogue scales are probably most suitable for trials that include patients with mild and moderate headache’ (7). The comments on ‘headache intensity’ in the tension-type headache (TTH) guideline state that ‘experience with the VAS is limited in TTH trials. However, as mentioned above, the pain in TTH is usually mild or moderate. The categorical verbal scale, commonly used for migraine attacks, may thus not be sensitive enough’ (8). The medical textbook ‘The Headaches’ (3rd edition) notes only that ‘simple efficacy parameters’ or ‘a clinically relevant decrease of headache on a simple verbal scale’ should be used (76). In the ‘note for guidance on clinical investigation of medicinal products for treatment of nociceptive pain’ of the Committee for Proprietary Medicinal Products of the European Medicines Agency, it is stated regarding ‘pain measurement’ that: ‘From the regulatory point of view, no specific recommendation for choice of rating scale is made. The applicant should discuss and justify the reasons of his choice taking into consideration factors such as population demographics, characteristics (e.g. intensity, duration), associated pathology, and concomitant medication. There are several scales or methodologies to assess pain, but none of them are completely free of problems’ (77). However, in the gold standard of pain research, the ‘Textbook of Pain’, Melzack and Katz (1) and McGrath and Unruh (78) discuss in detail the advantages, limitations and disadvantages of the various pain measurement scales. Melzack and Katz conclude that ‘although VRS and NRS are simple to administer and have demonstrated reliability and validity, the advantages associated with VAS make VAS the measurement instrument of choice when unidimensional measure of pain is required’ (1). Besides limitations of this recommendation for older patients, there are also other opinions in this regard (79).

Clark et al. point out that ‘the choice for using either scale depends on several factors: the clinician’s goal and the setting in which the scale is to be used (clinical or research), the patient’s level of formal education, and the training time available in the setting’ (34). Chapman determines that ‘only a few pain researchers critically examine their own measurement practices. We normally follow the tradition within our parent disciplines’ (80), while Altman adds that ‘it may be sensible to use measures that are common in other trials in the field, although popularity alone is not sufficient reason to use a measure’ (81).

In conclusion, this study shows that the VRS categories cannot be presented in an equidistant manner on the VAS, and that, contrary to previous assumptions, pain intensity descriptors are less clear and can have different meanings in different languages. Therefore, both in the 3rd edition of the International Headache Classification (ICHD-III) and in the guidelines for clinical trials of patients with headache illnesses, rather than a 4-grade VRS, a 6-grade or higher level VRS or a VAS should be recommended, with correspondingly broadly defined anchor points.

Footnotes

Funding

This work was supported by Boehringer Ingelheim Pharma GmbH & Co. KG, Germany.

Conflicts of interest

BA and HP are employees of Boehringer Ingelheim Pharma GmbH & Co. KG, Germany. BP declares no conflict of interest. HCD received honoraria for participation in clinical trials, contribution to advisory boards or oral presentations from: Addex Pharma, Allergan, Almirall, AstraZeneca, Bayer Vital, Berlin Chemie, Coherex, CoLucid, Boehringer Ingelheim, Bristol-Myers Squibb, GlaxoSmithKline, Grünenthal, Janssen-Cilag, Lilly, La Roche, 3 M Medica, Medtronic, Menerini, Minster, MSD, Novartis, Johnson & Johnson, Pierre Fabre, Pfizer, Schaper and Brümmer, SanofiAventis and Weber & Weber. HCD has no ownership interest and does not own stocks of any pharmaceutical company.

References

Melzack

Katz

. Pain assessment in adult patients. In: McMahon

Klotzenburg

(eds) Wall and Melzack’s Textbook of Pain, 5th edn. Philadelphia: Elsevier, 2006; 291–304.

Peters

Patijn

Lamé

. Pain assessment in younger and older pain patients: psychometric properties and patient preference of five commonly used measures of pain intensity. Pain Med 2007; 7: 601–610.

Jensen

Chenc

Brugger

. Postsurgical pain outcome assessment. Pain 2002; 99: 101–109.

Jensen

Karoly

. Self-report scales and procedures for assessing pain in adults. In: Turk

Melzack

(eds) Handbook of pain assessment, 2nd edn. New York: Guilford Press, 2001; 15–34.

Duncan

Bushnell

Lavigne

. Comparison of verbal and visual analogue scales for measuring the intensity and unpleasantness of experimental pain. Pain 1989; 37: 295–303.

Kwong

Pathak

. Validation of the eleven-point pain scale in the measurement of migraine headache pain. Cephalalgia 2007; 27: 336–342.

Tfelt-Hansen

Block

Dahlöf

Diener

Ferrari

Goadsby

. Guidelines for controlled trials of drugs in migraine. 2nd ed. Cephalalgia 2000; 20: 765–786.

Bendtsen

Bigal

Cerbo

Diener

Holroyd

Lampl

. on behalf of the International Headache Society Clinical Trials Subcommittee: Guidelines for controlled trials of drugs in tension-type headache. 2nd ed. Cephalalgia 2009; 30: 1–16.

Lines

Vandormael

Malbecq

. Comparison of visual analog scale and categorical ratings of headache pain in a randomized controlled clinical trial with migraine patients. Pain 2001; 93: 185–190.

10.

Pilgrim

. Early-phase clinical trials in migraine: efficacy, safety and dose-finding. Cephalalgia 1995; (Suppl 15): 10–13.

11.

Diener

Pfaffenrath

Pageler

Peil

Aicher

. The fixed combination of acetylsalicylic acid, paracetamol and caffeine is more effective than single substances and dual combination for the treatment of headache: a multicentre, randomized, double-blind, single-dose, placebo-controlled parallel group study. Cephalalgia 2005; 25: 776–787.

12.

Headache Classification Committee of the International Headache Society. Classification and diagnostic criteria for headache disorders, cranial neuralgias and facial pain. Cephalalgia 1988; 8(Suppl 7): 1–96.

13.

Pepe

. The statistical evaluation of medical tests for classification and prediction, New York: Oxford University Press, 2003.

14.

McQuay

Moore

. Methods of therapeutic trials. In: McMahon

Klotzenburg

(eds) Wall and Melzack’s Textbook of Pain, 5th edn. Philadelphia: Elsevier, 2006; 415–425.

15.

Bond

Pilowsky

. Subjective assessment of pain and its relation to the administration of analgesics in cancer patients. J Psychosomatic Res 1966; 10: 203–208.

16.

Ohnhaus

Adler

. Methodological problems in the measurement of pain: a comparison between the verbal rating scale and the visual analogue scale. Pain 1975; 1: 379–384.

17.

Lara-Munoz

Ponce de Leon

Feinstein

Puentee

Wells

. Comparison of three rating scales for measuring subjective phenomena in clinical research. I. Use of experimentally controlled auditory stimuli. Arch Med Res 2004; 35: 43–48.

18.

Price

McGrath

Rafii

Buckingham

. The validation of visual analogue scales as ratio scale measures for chronic and experimental pain. Pain 1983; 17: 45–56.

19.

Seymour

Simpson

Charlton

Phillips

. An evaluation of length and end-phrase of visual analogue scales in dental pain. Pain 1985; 21: 177–185.

20.

Haefeli

Elfering

. Pain assessment. Euro Spine J 2006; 15: S17–S24.

21.

Shealy

M-C

. Behavioral techniques in the control of pain: a case for health maintenance vs. disease treatment. In: Weisenberg

Tursky

(eds) Pain – New perspectives in therapy and research, New York: Plenum Press, 1976; 21–33.

22.

Langley

Sheppeared

. Problems associated with pain measurement in arthritis: comparison of the visual analogue and verbal rating scales. Clin Experiment Rheumatol 1984; 2: 231–234.

23.

Ferraz

Quaresma

Aquino

Atra

Tugwell

Goldsmith

. Reliability of pain scales in the assessment of literate and illiterate patients with rheumatoid arthritis. J Rheumatol 1990; 17: 1022–1024.

24.

Dijkers

. Comparing quantification of pain severity by verbal rating and numeric rating scales. J Spinal Cord Med 2010; 33: 232–242.

25.

Sriwatanakul

Kelvie

Lasagna

. The quantification of pain: an analysis of words used to describe pain and analgesia in clinical trials. Clin Pharmacol Ther 1982; 32: 143–148.

26.

Pilgrim

. Methodology of clinical trials of sumatriptan in migraine and cluster headache. Eur Neurol 1991; 31: 295–299.

27.

Tfelt-Hansen

. Methodology of treatment trials for acute migraine drugs. In: Olesen

Tfelt-Hansen

(eds) Frontiers in headache research, vol. 6. Headache treatment: Trial Methodology and new drugs, Philadelphia: Lippincott-Raven Publ, 1997; 111–117.

28.

Diener

Peil

Aicher

. The fixed combination of acetylsalicylic acid, paracetamol and caffeine is effective and safe in the treatment of severe headache attacks: results from a multicentre, randomized, double-blind, single-dose, placebo-controlled parallel group study. Cephalalgia 2011; 31: 1466–1476.

29.

Serlin

Mendoza

Nakamura

Edwards

Cleeland

. When is cancer pain mild, moderate or severe? Grading pain severity by its interference with function. Pain 1995; 61: 277–284.

30.

Finn

. Effects of some variations in rating scale characteristics on the means and reliabilities of ratings. Educ Psychol Measurement 1972; 34: 885–892.

31.

Ramsay

. The effect of number of categories in rating scales on precision of estimation of scale values. Psychometrika 1973; 38: 513–533.

32.

Preston

Colman

. Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychologica 2000; 104: 1–15.

33.

von Grafenried

Nüesch

. Non-migrainous headache for the evaluation of oral analgesics. Br J Clin Pharmac 1980; 10: 225S–231S.

34.

Clark

Lavielle

Martinez . Learning from pain scales: patient perspective. J Rheumatol 2003; 30: 1584–1588.

35.

Kuball

Schüz

Gamm

Weber

. Bone marrow punctures and pain. Acute Pain 2004; 6: 9–14.

36.

Kunz

Mylius

Scharmann

Schepelman

Lautenbacher

. Influence of dementia on multiple components of pain. Eur J Pain 2009; 13: 317–325.

37.

Bland

Altman

. Some examples of regression towards the mean. Br Med J 1994; 309: 780–780.

38.

Guilford

. Psychometric methods, New York: McGraw-Hill, 1954.

39.

Weber

Schüz

Kuball

Gamm

Jage

. Schmerzerfassung bei invasiven diagnostischen Prozeduren – Vergleich einer elfstufigen numerischen Rangskala mit einer sechsstufigen verbalen Kategorieskala zur Schmerzmessung bei Knochenmarkpunktion. Schmerz 2005; 19: 513–520.

40.

Schachtel

Thoden

. Onset of action of ibuprofen in the treatment of muscle-contraction headache. Headache 1988; 28: 471–474.

41.

Jensen

Turner

Romano

. What is the maximum number of levels needed in pain intensity measurement? Pain 1994; 58: 387–392.

42.

Breivik

Björnsson

Skovlund

. A comparison of pain rating scales by sampling from clinical trial data. Clin J Pain 2000; 16: 22–28.

43.

Jensen

Chen

Brugger

. Postsurgical pain outcome assessment. Pain 2002; 99: 101–109.

44.

Marqui

Rico Duarte

Marine

Lauque

Sorum

. How patients and physicians rate patients’ pain in a French emergency department using a verbally administered numerical rating scale and a visual analog scale. Acute Pain 2008; 10: 31–37.

45.

Jage

. Medikamente gegen Krebsschmerzen, 4. Auflage. Stuttgart: Thieme, 2003.

46.

Dundee

. Clinical evaluation of mild analgesics. Br J Clin Pharmac 1980; 10: 329S–334S.

47.

Kremer

Atkinson

Ignelzi

. Measurement of pain: patient preference does not confound pain measurement. Pain 1981; 10: 241–248.

48.

Glassman

Soyka

Pollack

. Treatment of muscle contraction headache: Micrainin vs. Aspirin. Headache 1982; 22: 101–109.

49.

Bolognese

Schnitzer

Ehrich

. Response relationship of VAS and Likert scales in osteoarthritis efficacy measurement. Osteoarthritis Cartilage 2003; 11: 499–507.

50.

Serrie

. Evaluation d’une douleur chronique. In: Serrie

Thurel

(eds) La douleur en pratique quotidienne – Diagnostic et Traitements, 2^e edition, Rueil-Malmaison: Éditions Arnette, 2002; 62–84.

51.

Chauffour-Ader

Dayde

M-C

. Comprendre et soulager la douleur, Rueil-Malmaison: Wolters Kluver, 2008.

52.

Beloeil

Negre

. Mém douleur, Rueil-Malmaison: Groupe Liaisons Cedex, 2007.

53.

Cabot

Le May

Besner

. Revue critique des outils d’évaluation de la douleur chez une clientèle adulte souffrant de cancer. Recherche en soins infirmiers 2007; 90: 35–57.

54.

Caraceni A, Cherny N, Fainsinger R, Kaasa S, Poulain P, Radbruch L, De Conno F, and the Steering Committee of the EAPC research Network. Pain measurement tools and methods in clinical research in palliative care: recommendations of an expert working group of the European Association of Palliative Care. J Pain Symptom Manage 2002; 23: 239–255.

55.

Heft

Parker

. An experimental basis for revising the graphic rating scale for pain. Pain 1984; 19: 153–161.

56.

Wallenstein

Heidrich

III G

Kaiko

Houde

. Clinical evaluation of mild analgesics: the measurement of clinical pain. Br J Clin Pharmacol 1980; 10: 319S–327S.

57.

Linton

Götestam

. A clinical comparison of two pain scales: correlation, remembering chronic pain, and a measure of compliance. Pain 1983; 17: 57–65.

58.

Littman

Walker

Schneider

. Reassessment of verbal and visual analog ratings in analgesic studies. Clin Pharmacol Ther 1985; 38: 16–23.

59.

Collins

Moore

McQuay

. The visual analogue pain intensity scale: what is moderate pain in millimetres? Pain 1997; 72: 95–97.

60.

Seymour

Kelly

Hawesford

. The efficacy of ketoprofen and paracetamol (acetaminophen) in postoperative pain after third molar surgery. Br J Clin Pharmacol 1996; 41: 581–585.

61.

Stubhaug

Grimstad

Breivik

. Lack of analgesic effect of 50 and 100 mg oral tramadol after orthopaedic surgery: a randomized, double-blind, placebo and standard active drug comparison. Pain 1995; 62: 111–118.

62.

Curtis

Gartman

Green

. Utilization of ketorolac tromethamine for control of severe odontogenic pain. J Endod 1994; 20: 457–459.

63.

Gursoy

Ertugrul

Sahin

Tutuncu

Demirer

Demirag

. Needle-free delivery of lidocaine for reducing the pain associated with the fine-needle aspiration biopsy of thyroid nodules: time-saving and efficacious procedure. Thyroid 2007; 17: 317–321.

64.

Page

Paterson

Young

O’Dwyer

. Pain from primary inguinal hernia and the effect of repair on pain. Br J Surg 2002; 89: 1315–1318.

65.

Lampl

Huber

Haas

Rittberger

Diener

. Difference in triptan effect in patients with migraine and early allodynia. Cephalalgia 2008; 28: 1031–1038.

66.

Lundqvist

Benth

JŠ

Grande

Aaseth

Russell

. A vertical VAS is a valid instrument for monitoring headache pain intensity. Cephalalgia 2009; 29: 1034–1041.

67.

Headache Classification Subcommittee of the International Headache Society. The International Classification of Headache Disorders. 2nd edn. Cephalalgia 2004; 24(Suppl 1): 9–160.

68.

Briggs

Closs

. A descriptive study of the use of visual analogue scales and verbal rating scales for the assessment of postoperative pain in orthopedic patients. J Pain Symptom Manage 1999; 18: 438–446.

69.

Jensen

Chen

Brugger

. Interpretation of visual analog scale ratings and change scores: a reanalysis of two clinical trials of postoperative pain. J Pain 2003; 4: 407–414.

70.

Perkins

Schisterman

. The inconsistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristic curve. Am J Epidemiol 2006; 163: 670–675.

71.

Schürks

Kurth

Knorn

Pageler

Diener

H-C

. Predictors of hazardous alcohol consumption among patients with cluster headache. Cephalalgia 2006; 26: 623–627.

72.

Loos

MJA

Houterman

Scheltinga

MRM

Roumen

RMH

. Evaluating postherniorrhaphy groin pain: visual analogue or verbal rating scale? Hernia 2008; 12: 147–151.

73.

Paul

Zelman

Smith

Miaskowski

. Categorizing the severity of cancer pain: Further exploration of the establishment of cutpoints. Pain 2005; 113: 37–44.

74.

Palos

Mendoza

Mobley

Cantor

Cleeland

. Asking the community about cutpoints used to describe mild, moderate, and severe pain. J Pain 2006; 7: 49–56.

75.

Dionne

Bartoshuk

Mogil

Witter

. Individual responder analyses for pain: does one pain scale fit all? Trends Pharmacol Sci 2005; 26: 125–130.

76.

Dahlöf

CGH

Rolan

Tfelt-Hansen

. Principles of clinical pharmacology, randomized controlled clinical trials, and evidence-based medicine in headache. In: Olesen

Goadsby

Ramadan

Tfelt-Hansen

Welch

(eds) The Headaches, 3rd edn. Philadelphia: Lippincott Williams & Wilkins, 2006; 55–61.

77.

Committee for Proprietary Medicinal Products – European Medicines Agency. Note for guidance on clinical investigation of medicinal products for treatment of nociceptive pain. CPMP/EWP/612/00 , London: European Medicines Agency, 2002.

78.

McGrath

Unruh

. Measurement and assessment of paediatric pain. In: McMahon

Klotzenburg

(eds) Wall and Melzack’s Textbook of Pain, 5th edn. Philadelphia: Elsevier, 2009; 291–304.

79.

Lund

Lundeberg

Sandberg

Norrbrink Budh

Kowalski

Svensson

. Lack of interchangeability between visual analogue and verbal rating pain scales: a cross sectional description of pain etiology groups. BMC Med Res Methodol 2005; 5: 31–31.

80.

Chapman

. The concept of measurement. Coexisting theoretical perspectives. In: Chapman

Loeser

(eds) Issues in pain measurement, New York: Raven Press, 1–16, 1989.

81.

Altman

. General principles of outcome measures and analysis in clinical trials. In: Olesen

Tfelt-Hansen

(eds) Headache treatment: trial methodology and new drugs, Philadelphia: Lippincott-Raven, 1997; 27–33.