Investigating the structure and assessing the psychometric properties of Likert scales before their application is a prerequisite of scaling theory. However, there are instances in the literature where Likert scales that were developed as a multidimensional measurement of the underlying construct have been applied without first validating their structure and reporting on their reliability. In this paper, to discourage such applications, the required sequence of decisions for such an investigation and assessment based on current theory and practice is demonstrated using the Patient Satisfaction Questionnaire Short-form (PSQ-18), a six-dimensional Likert measurement instrument. Although the findings did not confirm the dimensionality of the PSQ-18, they did provide an overall valid and reliable scale, emphasizing the need of validating its structure before application.
AnagnostopoulosFYfantopoulosJMoustakiINiakasD (2013) Psychometric and Factor Analytic Evaluation of the 15D Health-related Quality of Life Instrument - The Case of Greece. Quality of Life Research22(8): 1973–86.
2.
BartholomewDJSteeleFMoustakiIGalbraithJ (2008) Analysis of Multivariate Social Science Data. London: Chapman & Hall/CRC.
3.
BollenKA (1989) Structural Equations with Latent Variables. New York: Wiley.
4.
BrownTA (2015) Confirmatory Factor Analysis for Applied Research (2nd edition). New York: Guilford Press.
5.
CarifioJPerlaRJ (2007) Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and their Antidotes. Journal of Social Sciences3(3): 106–16.
6.
CarifioJPerlaRJ (2008) Resolving the 50-year Debate around Using and Misusing Likert Scales. Medical Education42: 1150–51.
7.
ChouCPBentlerPM (1995) Estimates and Tests in Structural Equation Modeling. In: HoyleRH (ed.) Structural Equation Modeling - Concepts, Issues, and Applications. Thousand Oaks, CA: Sage, 37–55.
D’AgostinoRBBelangerAD’AgostinoRBJr (1990) A Suggestion for Using Powerful and Informative Tests of Normality. American Statistician44(4): 316–21.
10.
EndersCK (2010) Applied Missing Data Analysis. New York: Guilford Press.
11.
Erceg-HurnDMMirosevichVM (2008) Modern Robust Statistical Methods - An Easy Way to Maximize the Accuracy and Power of your Research. American Psychologist63(7): 591–601.
12.
FabrigarLRWegenerDTMacCallumRCStrahanEJ (1999) Evaluating the Use of Exploratory Factor Analysis in Psychological Research. Psychological Methods4(3): 272–99.
13.
FieldA (2013) Discovering Statistics Using IBM SPSS Statistics. London: Sage.
14.
FornellCLarckerDF (1981) Evaluating Structural Equation Models with Unobserved Variables and Measurement Error. Journal of Marketing Research18(1): 39–50.
15.
GanasegeranKPerianayagamWManafRAJadooSAAAl-DubaiSAR (2015) Patient Satisfaction in Malaysia’s Busiest Outpatient Medical Care. Scientific World Journal2015: 1–6.
GillLWhiteL (2009) A Critical Review of Patient Satisfaction. Leadership in Health Services22(1): 8–19.
18.
GöbRMcCollinsCRamalhotoMF (2007) Ordinal Methodology in the Analysis of Likert Scales. Quality & Quantity41(5): 601–26.
19.
GreenSBYangY (2009) Commentary on Coefficient Alpha - A Cautionary Tale. Psychometrika74(1): 121–35.
20.
HagedoornMUijlSGVan SonderenERanchorAVGrolBMOtterRKrolBVan den HeuvelWSandermanR (2003) Structure and Reliability of Ware’s Patient Satisfaction Questionnaire III - Patients’ Satisfaction with Oncological Care in the Netherlands. Medical Care41(2): 254–63.
21.
HarknessJAVillarAEdwardsB (2010) Translation, Adaptation and Design. In: HarknessJABraunMEdwardsBJohnsonTPLybergLMohlerPPhPennellBESmithTW (eds) Survey Methods in Multinational, Multiregional, and Multicultural Contexts. Hoboken, NJ: Wiley, 117–40.
22.
HawthorneG (2006) Review of Patient Satisfaction Measures. Canberra: Australian Government Department of Health and Ageing.
23.
HaytonJAllenDScarpelloV (2004) Factor Retention Decisions in Exploratory Factor Analysis - A Tutorial on Parallel Analysis. Organizational Research Methods7(2): 191–205.
24.
HuLBentlerPM (1999) Cutoff Criteria for Fit Indexes in Covariance Structure Analysis - Conventional Criteria versus New Alternatives. Structural Equation Modeling6(1): 1–55.
JamiesonS (2004) Likert Scales - How to (Ab)Use Them. Medical Education38(12):1217–8.
27.
KuzonWMUrbanchekMGMcCabeS (1996) The Seven Deadly Sins of Statistical Analysis. Annals of Plastic Surgery37(3): 265–72.
28.
LevesqueR (2012) Raynald’s SPSS Tools [computer software]. Available at: http://www.spsstools.net(accessed 5 February 2012).
29.
MarshallGNHaysRD (1994) The Patient Satisfaction Questionnaire Short-form (PSQ-18). Report no P-7865. Santa Monica, CA: RAND.
30.
MarshallGNHaysRDSherbourneCDWellsKB (1993) The Structure of Patient Satisfaction with Outpatient Medical Care. Psychological Assessment5(4): 477–83.
31.
MoserCKaltonG (1975) Survey Methods in Social Investigation. London: Heinemann Educational Books.
32.
MurphyKRDavidshoferCO (2001) Psychological Testing - Principles and Applications. New Jersey: Prentice-Hall.
33.
NormanG (2010) Likert Scales, Levels of Measurement and the “Laws” of Statistics. Advances in Health Sciences Education - Theory and Practice15(5): 625–32.
34.
NunnallyJCBernsteinIH (1994) Psychometric Theory. New York: McGraw-Hill.
35.
O’ConnorBP (2000) SPSS and SAS Programs for Determining the Number of Components using Parallel Analysis and Velicer’s MAP Test. Behavior Research Methods, Instruments, & Computers, 32(3): 396–402.
36.
PatilVHSinghSNMishraSDonavanT (2007) Parallel Analysis Engine to Aid Determining Number of Factors to Retain [computer software]. In: Instruction and Research Server, University of Kansas. Available at: http://ires.ku.edu/∼smishra/parallelengine.htm.
37.
PatilVHSinghSNMishrSDonavanT (2008) Efficient Theory Development and Factor Retention Criteria - A Case for Abandoning the ‘Eigenvalue greater than One’ Criterion. Journal of Business Research61(2): 162–70.
38.
PeabodyJWFlorentinoJShimkhadaRSolonOQuimboS (2010a) Quality Variation and Its Impact on Costs and Satisfaction - Evidence from the QIDS Study. Medical Care48(1): 25–30.
39.
PeabodyJWFlorentinoJShimkhadaRSolonOQuimboS (2010b) Response to Letter to the Editor Regarding, Quality Variation and Its Impact on Costs and Satisfaction - Evidence from the QIDS Study. Medical Care48(6): 570–71.
ReiseSWallerNComreyA (2000) Factor Analysis and Scale Revision. Psychological Assessment12(3): 287–97.
44.
RevelleWZinbargRE (2009) Coefficients Alpha, Beta, Omega, and the Glb - Comments on Sijtsma. Psychometrika74(1): 145–54.
45.
SchmittTA (2011) Current Methodological Considerations in Exploratory and Confirmatory Factor Analysis. Journal of Psychoeducational Assessment29(4): 304–22.
46.
SheikhK (2010) Quality of Care, Cost, and Patient Satisfaction. Medical Care48(6): 570.
47.
StevensJ (2002) Applied Multivariate Statistics for the Social Sciences. New Jersey: Lawrence Erlbaum.
48.
SymeonakiMMichalopoulouCKazaniA (2015) A Fuzzy Set Theory Solution to Combining Likert Items into a Single Overall Scale (or Subscales). Quality & Quantity49(2): 739–62.
49.
TabachnickBGFidellLS (2007) Using Multivariate Statistics. Upper Saddle River, NJ: Pearson Allyn & Bacon.
50.
ThayaparanAJMahdiE (2013) The Patient Satisfaction Questionnaire Short form (PSQ-18) as an Adaptable, Reliable, and Validated Tool for Use in Various Settings. Medical Education Online18: 21747.
51.
ThompsonB (2005) Exploratory and Confirmatory Factor Analysis - Understanding Concepts and Applications (2nd printing). Washington, DC: American Psychological Association.
52.
VrijhoefHJMBerbeeRWagnerEHSteutenLMG (2009) Quality of Integrated Chronic Care Measured by Patient Survey - Identification, Selection and Application of Most Appropriate Instruments. Health Expectations12(4): 417–29.
53.
WareJESnyderMKWrightWR (1976) Development and Validation of Scales to Measure Patient Satisfaction with Medical Care Services(Vol. I, Parts A and B, NTIS Publication No. PB 288-329 and 288-330). Springfield, VA: National Technical Information Service.
54.
WilkinDHallamLDoggettM (1992) Measures of Patient Satisfaction - Measures of Need and Outcome for Primary Health Care. Oxford: Oxford University Press.
55.
ZiegerMHagemannD (2015) Editorial - Testing the Unidimensionality of Items. European Journal of Psychological Assessment31(4): 231–37.
