Development of a Vitamin B12 Deficiency Patient-Reported Outcome Measure for Clinical Practice and Research

Abstract

Background:

It is difficult to recognize vitamin B12 deficiency and to evaluate the effect of B12 treatment due to a broad range of variable clinical symptoms overlapping with other diseases and diagnostic biomarkers that quickly normalize during treatment. This poses a risk of delay in diagnosis and a challenge to uniformly monitor the effect of B12 treatment. There is a need for a new clinical outcome measure suitable for clinical practice and clinical evaluation studies.

Objective:

To develop a Patient-Reported Outcome Measure (PROM) which measures the severity of vitamin B12 deficiency symptoms.

Methods:

The B12 PROM was developed by (1) gathering input from experts and literature review to define a construct and develop a conceptual model, (2) processing input from health care providers, scientists, and patients to develop items and response options, and (3) improving items based on the feedback from laypersons, test interviews, semi-structured cognitive interviews with patients, and forward and backward translation (ENG-NL).

Results:

The B12 PROM includes 62 items grouped into 8 categories of symptoms related to vitamin B12 deficiency (General, Senses, Thinking, In limbs and/or face, Movement, Emotions, Mouth & Abdomen, Urinary tract & Reproductive organs). Cognitive interviews demonstrated good comprehensibility and comprehensiveness.

Conclusions:

This study is the first step in the development of a disease-specific PROM for vitamin B12 deficiency to measure the burden of symptoms. Further validation and reliability testing are necessary before the PROM can be applied in clinical practice and research.

Plain language title

Development of a Vitamin B12 Deficiency Questionnaire for Clinical Practice and Research

Plain language summary

This study is the first step in the development of a questionnaire for vitamin B12 deficiency to measure the severity of vitamin B12 deficiency symptoms. The questionnaire includes 62 items grouped into 8 categories of symptoms related to vitamin B12 deficiency (General, Senses, Thinking, In limbs and/or face, Movement, Emotions, Mouth & Abdomen, Urinary tract & Reproductive organs). Interviews with patients demonstrated good comprehensibility and comprehensiveness of the questionnaire. Further testing is necessary before the questionnaire can be applied in clinical practice and research.

Keywords

vitamin B12 deficiency Patient-Reported Outcome Measure PROM questionnaire symptom score clinical outcome measure patient monitoring tool severity of symptoms symptom burden diagnosis evaluation of treatment cognitive interviews content validity qualitative study

Introduction

Vitamin B12 deficiency requires early diagnosis and adequate treatment to prevent permanent neurological damage.¹ As B12 deficiency leads to a broad range of variable clinical symptoms^2
-4 and symptoms overlap with other diseases,⁵ it is challenging to recognize and monitor this disease. To our knowledge, there is no unequivocal measure to recognize and monitor the symptoms of vitamin B12 deficiency. This poses a risk of delay in diagnosis and a challenge to uniformly evaluate the effect of B12 treatment.

In the absence of an appropriate clinical outcome measure of vitamin B12 deficiency, there are few published clinical trials that measure the clinical effects of vitamin B12 deficiency treatment in patients. Most clinical trials focus on biochemical outcome measures as primary outcomes, such as the normalization of diagnostic biomarkers. However, diagnostic biomarkers in the blood may quickly normalize during treatment^6,7 while patients still experience B12-related symptoms. The few trials that evaluated the effect of B12 therapy on symptoms only evaluated a narrow range of symptoms without using a validated outcome measure.^8

-11

To improve the identification and monitoring of patients with symptoms of B12 deficiency, we aim to develop and validate a novel Patient-Reported Outcome Measure (PROM) specific to vitamin B12 deficiency which is suitable for clinical practice and clinical evaluation studies. A PROM is “a measurement based on a report that comes directly from the patient about the status of a patient’s health condition without amendment or interpretation of the patient’s response by a clinician or anyone else.”¹² The development of a PROM is a continuous process of evaluating and adapting draft versions. In this study we developed the construct of a novel PROM for vitamin B12 deficiency and assessed the content validity.

Methods

Study Design

Development of the B12 PROM followed the Rothrock guidance for developing a valid PROM.¹³ We developed the PROM in 3 phases: (1) Gathering input from experts and literature review to define a construct and develop a conceptual model; (2) Development of items and response options by reviewing the literature and processing input from health care providers, scientists, and patients; (3) Item improvement by processing feedback from laypersons, test interviews, semi-structured cognitive interviews with patients, and forward and backward translation. All methodological decisions and versions of the B12 PROM have been approved by consensus meetings in the steering board committee [KS, JV, CP].

The study was conducted in accordance with the Declaration of Helsinki. According to the Medical Research Involving Human Subjects Act (WMO) in the Netherlands, this study is exempt from ethics approval.

Gathering input

The definition of the construct was based on answering 3 questions: “What do you want to measure?”, “Which target population?” and “What is the purpose?”¹⁴ Researchers and health care providers (n = 9) provided their input to answer these questions. A literature search was performed to search for existing PROMs on B12 deficiency. Patient-Reported Outcome Measure experts (n = 4) were consulted to ask for advice on the usefulness of available generic PROMs to measure clinical outcomes of B12 deficiency and the development of the construct and conceptual model of a specific B12 PROM.

Development of items and response options

Item development was based on a literature review and expert opinions of health care providers, scientists, and patients. The literature review was based on a literature search in PubMed and Google Scholar [CP]. Search terms included Pernicious Anemia/Pernicious Anaemia/Vitamin B12 deficiency and symptoms/complaints. Search results covered a time frame from 1822 to 2016 and included clinical review papers, case studies, and patient surveys. Snowball search methods were used until saturation of symptoms was reached. Through multiple discussion rounds, the list of symptoms was reduced by (1) combining related symptoms into one item, (2) discarding medical test results that are reported by health care providers rather than patients, and (3) excluding symptoms that clinical experts regarded as extremely rare [CP, KS, SH, JaV]. This resulted in B12 PROM version 1.

All 5 patient representatives, appointed by the B12 Institute in Rotterdam, were invited to assess the acceptability and feasibility of the B12 PROM. All had received a diagnosis of vitamin B12 deficiency themselves and/or within their family in combination with a career as a health care provider or medical researcher. These patient representatives were invited to 2 video focus group meetings to discuss the acceptability and feasibility of completing the B12 PROM. The following subjects were discussed: (1) missing symptoms, (2) unnecessary or unrecognizable symptoms, (3) the order of the symptoms, (4) comprehensibility, (5) response options, and (6) the length of the questionnaire. The feedback was processed, and this resulted in B12 PROM version 2. This version was discussed in a similar manner with the head (n = 1) of vitamin B12 deficiency patients’ Facebook group (“B12 tekort, de vergeten ziekte”—“B12 deficiency, the forgotten illness”) with >16 000 members, who has extensive experience with social media platform discussions on B12 deficiency. New items were included in B12 PROM version 3 if they were recognized by at least 2 sources (literature, health care providers and/or patients) and prioritized by the steering board committee as frequently reported symptoms among vitamin B12 deficient patients.

Item improvement

Comprehensibility and readability of the B12 PROM were assessed by 2 laypersons without B12 deficiency [KS], resulting in B12 PROM version 4. Content validity was assessed by cognitive interviews with patients [KS]. A semi-structured interview guide was prepared based on the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) PRO Good Research Practices Task Force Report.¹⁵ A test interview with one layperson was performed to practice the think aloud method and probing method and identify potential problems in the comprehensibility, acceptability, and feasibility of the interview guide and B12 PROM. This feedback was used to improve the interview guide and create a more plain and easy-to-read format of the B12 PROM. This resulted in B12 PROM version 5. The test interview was repeated with the same layperson and the improved version of the interview guide and B12 PROM, which revealed that all issues were resolved. The steering board committee approved the interview guide and B12 PROM version 5 for initiating the cognitive interviews.

B12 deficient patients from the B12 Institute (secondary care center) were recruited to participate in the cognitive interviews. Purposive sampling was performed to ensure variation in age, sex, highest completed level of education, duration of B12 symptoms, and duration of B12 treatment. After receiving verbal informed consent, interviews were taken by a trained researcher [KS] via video calls (Microsoft Teams™), and the audio was recorded. The interviewer shared her screen such that each patient could see the B12 PROM with instructions, response options, and items. Patients were asked to complete the B12 PROM based on their own experience. The think aloud method and probing method were used during the interview to assess the comprehensibility of the PROM.^14,15 The interviewer asked follow-up questions to further explore the understanding and solutions to improve the clarity if a participant did not understand the instructions, response options and/or items. Comprehensiveness of the B12 PROM was assessed by asking 4 questions: (1) Are there any B12-related symptoms that have not been addressed in this questionnaire?; (2) How well do you think this questionnaire covers your complaints?; (3) How would you rate this questionnaire on a scale of 1 to 10? (1 is bad and 10 is excellent); (4) Do you consider this questionnaire to be complete?

Data analysis and B12 PROM adjustments occurred after every 4 interviews. Audio records were assessed, and the comprehensibility and comprehensiveness of the B12 PROM were improved accordingly. Symptoms were added to the B12 PROM if missing symptoms reported by patients were related to vitamin B12 deficiency in the literature. Recruitment of participants for interviews continued until the steering board committee reached consensus that saturation was achieved as no major improvements emerged from subsequent interview rounds.

For use in future international studies, forward and backward translators translated the Dutch B12 PROM emerging from the cognitive interviews into English based on the Guidelines for the Process of Cross-Cultural Adaptation of Self-Report Measures.¹⁶ Minor changes in item wording occurred in the final Dutch version of the B12 PROM based on the translators’ feedback.

Results

Gathering Input

Definition of the construct

We want to measure how much burden adults (18 years of age and older) experience from their vitamin B12 deficiency symptoms. We define symptom burden as a subjective measure of the severity of symptoms. The aim of the PROM is screening and monitoring: (1) a measurement instrument that can identify adults at risk of having a vitamin B12 deficiency by providing a score that discriminates between adults with and without symptoms related to vitamin B12 deficiency; (2) a measurement instrument to evaluate the effect of B12 deficiency treatment on symptoms related to vitamin B12 deficiency.

Conceptual model

No suitable clinical outcome measure was identified in the literature and experts concluded no generic PROMs could be used as clinical outcome measure for vitamin B12 deficiency because of the heterogeneous spectrum of symptoms related to vitamin B12 deficiency. The need for a disease-specific PROM for vitamin B12 deficiency, which covers the broad range of symptoms, was unanimously endorsed.

It was advised that the B12 PROM should be relatively short because vitamin B12 deficient patients suffer from concentration problems and fatigue. Rather than covering all symptoms related to vitamin B12 deficiency, it should include the most frequently reported symptoms per body system involved in the clinical presentation of vitamin B12 deficiency. Furthermore, it was decided to focus on adults because children and adolescents have different clinical manifestations of vitamin B12 deficiency.¹⁷

Development of Items and Response Options

The literature review resulted in a list of 118 symptoms related to vitamin B12 deficiency. This list of symptoms was converted to a list of 37 items with 5 response options (B12 PROM version 1). For example, “Difficulty falling asleep” and “Trouble staying asleep” have been combined into “Problems with sleeping,” and the items “Abnormal reflexes,” “Romberg sign,” “Babinski sign,” “Paranoia,” and “Osteoporosis” have been removed as these were more medical observations instead of symptoms that can be used to measure how much burden patients experience.

B12 PROM version 2 was generated based on the feedback of patient representatives (n = 5) and included new items, rephrasing existing items to improve clarity, removal of items, and categorization of symptoms in groups. For example, the item “Balance problems” was added, “Brain fog” was replaced by a more explanatory description, and “Diminished sense of vibration” was removed. It was advised that a questionnaire for vitamin B12 deficient patients should have a maximum completion time of 20 minutes.

Item generation was finalized in B12 PROM version 3 based on items that were frequently reported in the B12 Facebook group, such as “Frequent urination” and “Heavy feeling in the legs.” In addition, some items were adjusted to improve the clarity, readability, and feasibility.

Item Improvement

Cognitive interviews to assess the comprehensibility and comprehensiveness initiated with a B12 PROM version of 53 items. Changes were made to the wording of items and categories, and new items were added. For example, the item “Clumsiness (e.g., knocking things over or dropping them)” was included as a simple description of the medical term ataxia but it appeared that this item was not experienced as specific for vitamin B12 deficiency. Participants’ responses to this item were: “I think everyone suffers from that sometimes,” “Knocking things over depends on whether I’m tired,” and “I always do that. That’s nothing new.” As ataxia refers to problems with coordination that cause clumsy voluntary movements, this item was substituted by the item “Difficulty to accurately perform movements.” Another example was the adjustment of the item “Flat emotions” as a description of the medical term apathy because this description posed difficulties in understanding for multiple patients (n = 3). Two patients suggested the term “indifferent,” which ultimately led to the new item wording: “Less emotional (indifferent) in situations where you would normally be sad, angry, scared or happy.” In addition, new items mentioned by patients and the literature have been added to the B12 PROM, for example, “Hair loss,” “Cracking voice,” “Weakening and/or paralysis of limbs,” “Brittle nails,” “Involuntary limb movements,” “Restless legs,” and “Sores in the mouth (ulcers).”

After 12 interviews with patients (Table 1), saturation was reached as the last 4 interviews indicated good comprehensibility and comprehensiveness (Table 2).

Table 1.

Characteristics of Participants in Interviews (n = 12).

Age	Range 18-73 years 18-20 years (n = 1) 20-30 years (n = 3) 30-40 years (n = 1) 40-50 years (n = 2) 50-60 years (n = 3) 60-70 years (n = 1) >70 years (n = 1)
Sex	Female (n = 6) Male (n = 6)
Highest education level	Secondary education (n = 2) Vocational education and training (Dutch: MBO) (n = 3) Higher professional education (Dutch: HBO) (n = 4) University education (Dutch: WO) (n = 3)
Duration B12 symptoms	Range 1.5-35 years 1-2 years (n = 1) 2-5 years (n = 2) 5-10 years (n = 3) 10-15 years (n = 2) >15 years (n = 3) Difficult to estimate^a (n = 1)
Duration B12 treatment	Range 10 days-12 years 0-6 months (n = 5) 6 months-1 year (n = 1) 1-2 years (n = 2) 2-5 years (n = 2) 5-10 years (n = 1) >10 years (n = 1)
Comorbidities^b	None (n = 5) 1 (n = 6) 2 (n = 1)

^a One participant said “Complaints since childhood. I cannot say from what age I remember these complaints.”

^b The interviewer asked the participants about comorbidities. Participants mentioned the following comorbidities: cardiac arrhythmia, Hashimoto’s disease, hip dysplasia, delayed sleep phase syndrome, hereditary motor and sensory neuropathy (HMSN) type 1, chronic fatigue syndrome (CFS), irritable bowel syndrome (IBS), and functional neurological disorder (FND).

Table 2.

Responses From Patients (n = 4) in the Last 4 Cognitive Interviews (Interview Numbers #9-12).

Question	Response
How well do you think this questionnaire covers your complaints?	#9: “Pretty good. I really think everything is discussed that is possible and what I experience myself.” #10: “Yes, good. It feels like a celebration of recognition.” #11: “Very good overall. It does give a first impression into what my complaints are. Of course, you go into more detail during the consultation with a physician.” #12: “Yes, good.”
How would you rate this questionnaire on a scale of 1 to 10? (1 is bad and 10 is excellent)	#9: 9 #10: 8.5 #11: 8 #12: 9
Do you consider this questionnaire to be complete?	#9: “Yes.” #10: “Yes.” #11: “Yes, I think so if there is a question where you can add complaints not mentioned in the questionnaire.” #12: “There is the item ‘fatigue’ in the questionnaire, but I think ‘extreme fatigue’ is insufficiently highlighted. I think it’s complete after my additions.”

The Dutch B12 PROM emerging from the cognitive interviews was translated into English for international usage. The final version of the B12 PROM includes 62 items and 8 categories of symptoms (General, Senses, Thinking, In limbs and/or face, Movement, Emotions, Mouth & Abdomen, Urinary tract & Reproductive organs) (Table 3).

Table 3.

Items and Categories of Symptoms From the Final B12 PROM Version After Cognitive Interviews.^a

General	Movement
1. Fatigue	32. Unstable walking
2. Waking up tired after a night’s sleep	33. Difficulty to accurately perform movements
3. Problems with sleeping	34. Loss of strength
4. Need for afternoon naps	35. Muscle cramps
5. Heavy, fast and/or irregular heartbeat	36. Muscular pain
6. Pain or pressure on the chest	37. Pain in joints
7. Shortness of breath	38. Involuntary limb movements
8. Headache	39. Shaky hands
9. Dizziness	40. Heavy feeling in the legs
10. Limited energy	41. Restless legs
11. Hair loss	Emotions
12. Cracking voice	42. Mood swings
Senses	43. Gloomy
13. Less smell and/or taste	44. Quick to (almost) cry
14. Blurred vision (despite using glasses or lenses)	45. Easily irritated
15. Tinnitus (ringing in the ears)	46. Anxiety and/or panic
16. Cold hands and/or feet	47. Less emotional (indifferent) in situations where you would normally be sad, angry, scared, or happy
17. Balance problems	Mouth & Abdomen
Thinking	48. Decreased appetite
18. Difficulty concentrating	49. Sores in the mouth (ulcers)
19. Slow thinking	50. Painful and/or thick tongue
20. Unable to think clearly and/or foggy in the head	51. Stomach problems
21. Forgetful	52. Nausea
22. Difficulty finding a particular word	53. Bloated feeling
23. Easily overstimulated by light, sound and/or busy environment	54. Abdominal pain
24. Confused in thinking and speaking	55. Heartburn and/or acid reflux
In limbs and/or face	56. Diarrhea
25. Tingling, burning, and/or stabbing sensation	57. Constipation
26. Less sensation in limbs and/or skin	Urinary tract & Reproductive organs
27. Skin is painful when touched	58. Having a frequent urge to urinate
28. Numbness in certain spots	59. Undesirable urine loss
29. Skin and/or muscle twitching	60. Painful vaginal wall
30. Weakening and/or paralysis of limbs	61. Menstruation problems (past 3 months)
31. Brittle nails	62. Erection problems (past 3 months)

^a Instruction: “Indicate whether you have suffered from the following complaints in the past 7 days.” Response options: “Not, Little, Moderate, Much, Very much.”

Discussion

To our knowledge, we are developing the first PROM for vitamin B12 deficiency in adults. This PROM may be used to identify patients with B12 deficiency and to monitor and compare the effectiveness of B12 therapies. Consequently, this may lead to a shorter time to diagnosis and better therapy evaluation. This article outlined the first steps of PROM development: definition of the construct and the development of a conceptual model, development of items and response options, and item improvement.

All participants involved in the definition of the construct acknowledged the need to develop a B12-specific PROM, as there is no specific presentation of vitamin B12 deficiency. As symptoms may vary, it is difficult for health care providers to recognize patients who benefit from B12 testing. In addition, it is essential to assess the clinical outcome of B12 deficiency to evaluate the treatment, as laboratory testing of vitamin B12 and holotranscobalamin is not recommended for this purpose⁶ and elevated homocysteine and methylmalonic acid levels may already fall toward normal within 1 week of B12 treatment.¹⁸ In contrast, the recovery of clinical symptoms is slower, for example, neurological symptoms subside over weeks to months.¹⁹

The current version of our B12 PROM has several limitations. Firstly, the PROM was developed in the Dutch language. However, the initial literature search was performed in the English language, and all added symptoms were checked with English literature. In addition, this study was supervised by an international research committee that included native English speakers [CS, NW, DJ], and the B12 PROM was translated in a scientific manner. Secondly, the participating health care providers and patients were all linked to a single medical center, the B12 Institute in Rotterdam, specializing in diagnosing and treating vitamin B12 deficiency. This may lead to a bias toward more complex cases of B12 deficiency and cases not easily recognized by primary care doctors. In addition, several patients from the cognitive interviews had comorbidities with symptoms that overlap with vitamin B12 deficiency. Thirdly, this PROM has been focused on symptoms experienced by adults with B12 deficiency, as children and adolescents may present with different age-specific symptoms of B12 deficiency,¹⁷ and these symptoms were not included in this B12 PROM. Age-specific PROMs should be developed for children and adolescents. Fourthly, the development of a user manual with a scoring guideline and the evaluation of measurement properties are necessary before the implementation of the B12 PROM. Future studies need to assess the measurement properties (e.g., reliability, discriminative validity, and responsiveness) of this PROM.²⁰

This B12 deficiency PROM could have a significant impact on clinical practice and research in the future. This is the first step toward the development of (1) a screening tool to identify patients with symptoms of B12 deficiency, (2) a monitoring tool for both patients and health care providers to assess the severity of B12-related symptoms, and (3) a validated clinical outcome measure for vitamin B12 deficiency to be used in clinical trials to compare and evaluate effects of B12 treatments.

Supplemental Material

Supplemental Material, sj-pdf-1-fnb-10.1177_03795721241236284 - Development of a Vitamin B12 Deficiency Patient-Reported Outcome Measure for Clinical Practice and Research

Supplemental Material, sj-pdf-1-fnb-10.1177_03795721241236284 for Development of a Vitamin B12 Deficiency Patient-Reported Outcome Measure for Clinical Practice and Research by Kim I. M. Suijker, Clara H. M. Plattel, Catherine H. Seage, Nicola Ward, Delyth H. James and Jolande Y. Vis in Food and Nutrition Bulletin

Footnotes

Acknowledgments

The authors gratefully acknowledge everyone who provided feedback during the development of this B12 PROM: Sandra Heil (SH), Jacqueline Versteeg (JaV), the B12 Institute Research Foundation Patient Advisory Council, the head of the Facebook group “B12 tekort, de vergeten ziekte” (B12 deficiency, the forgotten illness), physicians and physician assistant from the B12 Institute, laypersons, participants in the interviews, forward and backward translators, Caroline Terwee, Monique Den Hollander-Ardon, Shannon van Hoorn, Alessandro Chiarotto, and Monique van Dijk. They thank practice manager Ank van Dorp from the B12 Institute for recruiting patients for the interviews and Jasper Drexhage for programming the digital version of the B12 PROM used during the interviews.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Kim I. M. Suijker

Clara H. M. Plattel

Catherine H. Seage

Nicola Ward

References

Hunt

Harrington

Robinson

. Vitamin B12 deficiency. BMJ. 2014;349:g5226. doi:10.1136/bmj.g5226

Stabler

SP.

Clinical practice. Vitamin B12 deficiency. N Engl J Med. 2013;368(2):149–160. doi:10.1056/NEJMcp1113996

Hooper

Hudson

Porter

McCaddon

. Patient journeys: diagnosis and treatment of pernicious anaemia. Br J Nurs. 2014;23(7):376–381. doi:10.12968/bjon.2014.23.7.376

Green

. Vitamin B12 deficiency from the perspective of a practicing hematologist. Blood. 2017;129(19):2603–2611. doi:10.1182/blood-2016-10-569186

Wolffenbuttel

BHR

Wouters

Heiner-Fokkema

van der Klauw

. The many faces of cobalamin (Vitamin B12) deficiency. Mayo Clin Proc Innov Qual Outcomes. 2019;3(2):200–214. doi:10.1016/j.mayocpiqo.2019.03.002

Carmel

. How I treat cobalamin (vitamin B12) deficiency. Blood. 2008;112(6):2214–2221. doi:10.1182/blood-2008-03-040253

Mansoor

Stea

Schneede

Reine

. Early biochemical and hematological response to intramuscular cyanocobalamin therapy in vitamin B12-deficient patients. Ann Nutr Metab. 2013;62(4):347–353. doi:10.1159/000347065

Kuzminski

Del Giacco

Allen

Stabler

Lindenbaum

. Effective treatment of cobalamin deficiency with oral cobalamin. Blood. 1998;92(4):1191–1198.

Bolaman

Kadikoylu

Yukselen

Yavasoglu

Barutca

Senturk

. Oral versus intramuscular cobalamin treatment in megaloblastic anemia: a single-center, prospective, randomized, open-label study. Clin Ther. 2003;25(12):3124–3134. doi:10.1016/s0149-2918(03)90096-8

10.

Sanz-Cuesta

Escortell-Mayor

Cura-Gonzalez

, et al. Oral versus intramuscular administration of vitamin B12 for vitamin B12 deficiency in primary care: a pragmatic, randomised, non-inferiority clinical trial (OB12). BMJ Open. 2020;10(8):e033687. doi:10.1136/bmjopen-2019-033687

11.

Wang

Qin

Song

Vidal-Alaball

Liu

. Oral vitamin B12 versus intramuscular vitamin B12 for vitamin B12 deficiency. Cochrane Database Syst Rev. 2018;3(3):CD004655. doi:10.1002/14651858.CD004655.pub3

12.

US Food and Drug Administration. Guidance for industry patient-reported outcome measures: use in medical product development to support labeling claims. 2009. Accessed June 21, 2022. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims

13.

Rothrock

Kaiser

Cella

. Developing a valid patient-reported outcome measure. Clin Pharmacol Ther. 2011;90(5):737–742. doi:10.1038/clpt.2011.195

14.

de Vet

HCW

Terwee

Mokkink

Knol

. Development of a measurement instrument. In: Measurement in Medicine: A Practical Guide. Cambridge University Press; 2011: 30–64.

15.

Patrick

Burke

Gwaltney

, et al. Content validity—establishing and reporting the evidence in newly developed Patient-Reported Outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: Part 2—Assessing respondent understanding. Value Health. 2011;14(8):978–988. doi:10.1016/j.jval.2011.06.013

16.

Beaton

Bombardier

Guillemin

Ferraz

. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25(24):3186–3191. doi:10.1097/00007632-200012150-00014

17.

Green

Allen

Bjørke-Monsen

, et al. Vitamin B12 deficiency. Nat Rev Dis Primers. 2017;3:17040. doi:10.1038/nrdp.2017.40

18.

Allen

Stabler

Savage

Lindenbaum

. Diagnosis of cobalamin deficiency I: usefulness of serum methylmalonic acid and total homocysteine concentrations. Am J Hematol. 1990;34(2):90–98. doi:10.1002/ajh.2830340204

19.

Healton

Savage

Brust

Garrett

Lindenbaum

. Neurologic aspects of cobalamin deficiency. Medicine (Baltimore). 1991;70(4):229–245. doi:10.1097/00005792-199107000-00001

20.

Mokkink

Prinsen

Patrick

, et al. COSMIN Study Design checklist for Patient-reported outcome measurement instruments. 2019. Accessed April 26, 2022. https://www.cosmin.nl/wp-content/uploads/COSMIN-study-designing-checklist_final.pdf#

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.40 MB