Measuring Competence in APA Writing Style: Scale Development and Psychometric Evaluation of Indonesian University Students

Abstract

Background

The American Psychological Association (APA) style is a convention adopted by many universities worldwide. However, students find it difficult to apply the correct APA 7th edition format.

Objective

This study developed and validated a curriculum-based questionnaire to assess students’ competence in formatting journal articles according to APA style.

Method

We recruited 1,117 participants online (69.2% master's students and 30.8% doctoral students) from various Indonesian universities. Each participant completed our 30-item Student Competence in APA 7 Writing Style (SCAWS) scale. We used simple Rasch models to evaluate the scale's psychometric properties.

Results

The Rasch analysis confirmed model fit based on global-fit statistics. Totally, 27 of the 30 items were found to fit the Rasch model. The scale's reliability was .813, indicating strong internal consistency.

Conclusion

The 27-item SCAWS scale provides a valid measure suitable for use in educational and psychological research.

Teaching implications

The SCAWS scale appears to offer lecturers a useful instrument for diagnosing postgraduate students’ difficulties in understanding the APA 7th edition writing style and provides recommendations for addressing their weaknesses in specific areas.

Keywords

APA referencing style Rasch model scale development validation writing competencies

Get full access to this article

View all access options for this article.

References

Aitchison

Lee

(2006). Research writing: Problems and pedagogies. Teaching in Higher Education, 11(3), 265–278. https://doi.org/10.1080/13562510600680574

American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). American Psychological Association. https://doi.org/10.1037/0000165-000

APA Publications and Communications Board Working Group on Journal Article Reporting Standards. (2008). Reporting standards for research in psychology: Why do we need them? What might they be? American Psychologist, 63(9), 839–851. https://doi.org/10.1037/0003-066X.63.9.839

Appelbaum

Cooper

Kline

R. B.

Mayo-Wilson

Nezu

A. M.

Rao

S. M.

(2018). Journal article reporting standards for quantitative research in psychology: The APA publications and communications board task force report. American Psychologist, 73(1), 3–25. https://doi.org/10.1037/amp0000191

Bollen

K. A.

Davis

W. R.

(2009). Two rules of identification for structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 523–536. https://doi.org/10.1080/10705510903008261

Bonett

D. G.

(2010). Varying coefficient meta-analytic methods for alpha reliability. Psychological Methods, 15(4), 368–385. https://doi.org/10.1037/a0020142

Boone

W. J.

(2020). Rasch basics for the novice. In Khine

M. S.

(Ed.), Rasch measurement: Applications in quantitative educational research (pp. 9–30). Springer. https://doi.org/10.1007/978-981-15-1800-3_2

Bradley

Noble

Hendricks

(2021). Mastering the new APA publication manual: Teaching techniques. The Family Journal, 29(1), 4–9. https://doi.org/10.1177/1066480720954207

Castillo-Martínez

I. M.

Ramírez-Montoya

M. S.

(2021). Research competencies to develop academic Reading and writing: A systematic literature review. Frontiers in Education, 5, 576961. https://doi.org/10.3389/feduc.2020.576961

10.

Christensen

K. B.

Bjorner

J. B.

Kreiner

Petersen

J. H.

(2002). Testing unidimensionality in polytomous rasch models. Psychometrika, 67(4), 563–574. https://doi.org/10.1007/BF02295131

11.

Christensen

K. B.

Makransky

Horton

(2017). Critical values for yen's Q3: Identification of local dependence in the rasch model using residual correlations. Applied Psychological Measurement, 41(3), 178–194. https://doi.org/10.1177/0146621616677520

12.

Creamer

E. G.

(1999). Knowledge production, publication productivity, and intimate academic partnerships. The Journal of Higher Education, 70(3), 261–277. https://doi.org/10.2307/2649197

13.

Daniels

L. M.

Kennedy

K. E.

(2019). A three-pronged approach to helping students internalize APA style based in self-determination theory. The Canadian Journal for the Scholarship of Teaching and Learning, 10(2). https://doi.org/10.5206/cjsotl-rcacea.2019.2.8176

14.

Directorate General of Higher Education, Research, and Technology (DIKTI). (2012). Surat edaran Dirjen DIKTI No. 152/E/T/2012. Retrieved from https://sps.itb.ac.id/wp-content/uploads/sites/45/2015/12/syarat-lulus-publikasi-ilmiah.pdf

15.

Edwards

M. C.

Houts

C. R.

Cai

(2018). A diagnostic procedure to detect departures from local independence in item response theory models. Psychological Methods, 23(1), 138–149. https://doi.org/10.1037/met0000121

16.

Gaffney

A. L. H.

(2015). Revising and reflecting: How assessment of APA style evolved over two assessment cycles in an undergraduate communication program. Journal of Assessment and Institutional Effectiveness, 5(2), 148–167. https://doi.org/10.5325/jasseinsteffe.5.2.0148

17.

Gravett

Kinchin

I. M.

(2021). The role of academic referencing within students’ identity development. Journal of Further and Higher Education, 45(3), 377–388. https://doi.org/10.1080/0309877X.2020.1766665

18.

Hayat

Rahayu

Putra

M. D. K.

Sarifah

Puri

V. G. S.

Isa

(2023). Metacognitive skills assessment in research-proposal writing (MSARPW) in the Indonesian university context: Scale development and validation using multidimensional item response models. Jurnal Pengukuran Psikologi dan Pendidikan Indonesia, 12(1), 31–47. https://doi.org/10.15408/jp3i.v12i1.31679

19.

Himawan

K. K.

(2021). The question of ethical review for psychological research in Indonesia: Need and importance. ANIMA Indonesian Psychological Journal, 36(2), 326–340. https://doi.org/10.24123/aipj.v36i2.4059

20.

Holman

Lindeboom

Glas

C. A. W.

Vermeulen

de Haan

R. J.

(2003). Constructing an item bank using item response theory: The AMC linear disability score project. Health Services & Outcomes Research Methodology, 4(1), 19–33. https://doi.org/10.1023/A:1025824810390

21.

Irawan

D. E.

Rachmi

C. N.

Irawan

Abraham

Kusno

Multazam

M. T.

Rosada

K. K.

Nugroho

S. H.

Kusumah

Holidin

Aziz

N. A.

(2017). Penerapan open science di Indonesia agar riset lebih terbuka, mudah diakses, dan meningkatkan dampak saintifik. Berkala Ilmu Perpustakaan dan Informasi, 13(1), 25–36. https://doi.org/10.22146/bip.17054

22.

Karabatsos

(1998). Analyzing nonadditive conjoint structures: Compounding events by rasch model probabilities. Journal of Outcome Measurement, 2(3), 191–221.

23.

Karabatsos

(2000). A critique of rasch residual fit statistics. Journal of Applied Measurement, 1(2), 152–176.

24.

Kean

Bisson

E. F.

Brodke

D. S.

Biber

Gross

P. H.

(2018). An introduction to item response theory and rasch analysis: Application using the eating assessment tool (EAT-10). Brain Impairment, 19(1), 91–102. https://doi.org/10.1017/BrImp.2017.31

25.

Kim

Preston

K. P.

Jr. Braga

Fankhauser

S. C.

(2023). Increasing student confidence in writing: Integrating authentic manuscript writing into an online 8-week research program. Journal of Microbiology & Biology Education, 24(1), 1–10. https://doi.org/10.1128/jmbe.00199-22

26.

Landrum

R. E.

(2013). Writing in APA style: Faculty perspectives of competence and importance. Psychology Learning and Teaching, 12(3), 259–265. https://doi.org/10.2304/plat.2013.12.3.259

27.

Lawshe

C. H.

(1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x

28.

Lestari

P. W.

(2021). Peningkatan pengetahuan dosen rumpun ilmu kesehatan tentang etik penelitian. Jurnal Pengabdian Masyarakat Bakti Parahita, 2(2), 155–160. https://doi.org/10.54771/jpmbp.v2i02.234

29.

Levitt

H. M.

Bamberg

Creswell

J. W.

Frost

D. M.

Josselson

Suárez-Orozco

(2018). Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA. Publications and Communications Board Task Force Report. American Psychologist, 73(1), 26–46. https://doi.org/10.1037/amp0000151

30.

Luttrell

V. R.

Bufkin

J. L.

Eastman

V. J.

Miller

(2010). Teaching scientific writing: Measuring student learning in an intensive APA skills course. Teaching of Psychology, 37(3), 193–195. https://doi.org/10.1080/00986283.2010.488531

31.

Mair

(2018). Modern psychometrics with R. Springer International Publishing AG.

32.

Mair

Hatzinger

(2007). Extended rasch modeling: The eRm package for the application of IRT models in R. Journal of Statistical Software, 20(9), 1–20. https://doi.org/10.18637/jss.v020.i09

33.

Mandernach

B. J.

Zafonte

Taylor

(2016). Instructional strategies to improve college students’ APA style writing. International Journal of Teaching and Learning in Higher Education, 27(3), 407–412.

34.

National Research and Innovation Agency of the Republic of Indonesia (BRIN). (2022). Peraturan BRIN No. 22 Tahun 2022 tentang klirens etik riset. Retrieved from https://lppm.usm.ac.id/peraturan-brin-nomor-22-tahun-2022-tentang-klirens-etik-riset/

35.

Ningrum

R. W.

Hamamah

Sahiruddin

Rohmah

(2023). Academic writing difficulties for Indonesian students in pursuing postgraduate studies abroad. Premise: Journal of English Education and Applied Linguistics, 12(1), 93–109. https://doi.org/10.24127/pj.v12i1.5271

36.

Nosek

B. A.

Alter

Banks

G. C.

Borsboom

Bowman

S. D.

Breckler

S. J.

Buck

Chambers

C. D.

Chin

Christensen

Contestabile

Dafoe

Eich

Freese

Glennerster

Goroff

Green

D. P.

Hesse

Humphreys

Yarkoni

(2015). Promoting an open research culture: Author guidelines for journals could help to promote transparency, openness, and reproducibility. Science, 348(6242), 1422–1425. https://doi.org/10.1126/science.aab2374

37.

Padgett

R. N.

Morgan

G. B.

(2020). Using the eRm package for rasch modeling. Measurement: Interdisciplinary Research and Perspectives, 18(3), 163–176. https://doi.org/10.1080/15366367.2020.1732155

38.

Polit

D. F.

Beck

C. T.

Owen

S. V.

(2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459–467. https://doi.org/10.1002/nur.20199

39.

Putra

M. D. K.

(2024, May 13). Datasets: Measuring competence in APA Writing Style: Scale development and psychometric evaluation of Indonesian university students. https://osf.io/cb8jh/

40.

Rasch

(1960). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research.

41.

Richmond

A. S.

Boysen

G. A.

Gurung

R. A. R.

Tazeau

Y. N.

Meyers

S. A.

Sciutto

M. J.

(2014). Aspirational model teaching criteria for psychology. Teaching of Psychology, 41(4), 281–295. https://doi.org/10.1177/0098628314549699

42.

Rusdianto

Fitrawati

(2022). An analysis of students’ ability in paraphrasing a paragraph at the English department universitas Negeri padang. Journal of English Language Teaching, 11(3), 314–323. https://doi.org/10.24036/jelt.v11i3.119178

43.

Sandy

Shen

(2019). Publish to earn incentives: How do Indonesian professors respond to the new policy? Higher Education, 77(2), 247–263. https://doi.org/10.1007/s10734-018-0271-0

44.

Stellmack

M. A.

Konheim-Kalkstein

Y. L.

Manor

J. E.

Massey

A. R.

Schmitz

J. A. P.

(2009). An assessment of reliability and validity of a rubric for grading APA-style introductions. Teaching of Psychology, 36(2), 102–107. https://doi.org/10.1080/00986280902739776

45.

Stewart

(2023). A case for open science in Indonesia: Advancing digital literacy, scientific infrastructure, and leadership. Centre for Strategic and International Studies (CSIS). https://csis.or.id/publication/a-case-for-open-science-in-indonesia-advancing-digital-literacy-scientific-infrastructure-and-leadership/

46.

Sudiran

Adi

A. P. K.

(2023). An analysis of the students’ competence in writing abstract. ETERNAL: English Teaching Journal, 14(2), 177–186. https://doi.org/10.26877/eternal.v14i2.15495

47.

Swank

J. M.

Lambie

G. W.

(2016). Development of the research competencies scale. Measurement and Evaluation in Counseling and Development, 49(2), 91–108. https://doi.org/10.1177/0748175615625749

48.

Syazali

Erfan

Wahdah

N. F.

(2024). Efektivitas pembelajaran berbasis riset dalam pengembangan keterampilan menulis ilmiah mahasiswa: Studi komparatif mahasiswa tahun pertama dan mahasiswa tahun ketiga. BIOCHEPHY: Journal of Science Education, 4(2), 1062–1072. https://doi.org/10.52562/biochephy.v4i2.1380

49.

Tarigan

F. N.

Nasution

A. F.

Hasibuan

S. A.

(2023). Literasi data: Kemampuan dan kesulitan mahasiswa dalam penulisan dan publikasi artikel jurnal ilmiah. Jurnal Ilmiah KORPUS, 7(2), 212–218. https://doi.org/10.33369/jik.v7i2.29231

50.

Tesio

Caronni

Kumbhare

Scarano

(2024). Interpreting results from rasch analysis 1. The “most likely” measures coming from the model. Disability and Rehabilitation, 46(3), 591–603. https://doi.org/10.1080/09638288.2023.2169771

51.

UIN Syarif Hidayatullah Jakarta. (2022). Pedoman akademik UIN Syarif Hidayatullah Jakarta. Retrieved from https://asset.uinjkt.ac.id/uploads/assets/pedoman/pedoman-akademik-uin-2022-2023.pdf

52.

Universitas Gadjah Mada. (2021). APA referencing style—updated to 7th edition. Retrieved from https://lib.ugm.ac.id/en/apa-referencing-style-updated-to-7th-edition/

53.

Universitas Negeri Jakarta. (2020). Panduan penulisan tesis & disertasi. Retrieved from https://fip.unj.ac.id/s2bk/wp-content/uploads/2022/08/PENULISAN-BUKU-PANDUAN-TESIS-DAN-DISERTASI-1.pdf

54.

Universitas Negeri Jakarta. (2021). Rencana pembelajaran semester (RPS)–penulisan artikel dan jurnal ilmiah terindeks (academic writing). Retrieved from https://pps.unj.ac.id/wp-content/uploads/2022/05/RPS-PENULISAN-ARTIKEL-DAN-JURNAL-ILMIAH-TERINDEKS.pdf

55.

Wilson

(2023). Constructing measures: An item response modeling approach (2nd ed.). Routledge.

56.

Wilson

Draney

(2002). A technique for setting standards and maintaining them over time. In Nishisato

Baba

Bozdogan

Kanefugi

(Eds.), Measurement and multivariate analysis (pp. 325–332). Springer-Verlag.

57.

Wischgoll

(2017). Improving undergraduates’ and postgraduates’ academic writing skills with strategy training and feedback. Frontiers in Education, 2, 33. https://doi.org/10.3389/feduc.2017.00033

58.

Wright

Panchapakesan

(1969). A procedure for sample-free item analysis. Educational and Psychological Measurement, 29(1), 23–48. https://doi.org/10.1177/001316446902900102

59.

Wright

B. D.

Stone

(1999). Measurement essentials (2nd ed.). Wide Range, Inc.

60.

Wutsqaa

(2022, October 17). Cara menulis daftar pustaka dari jurnal berbagai format beserta contohnya. Detik.com. https://www.detik.com/sulsel/berita/d-6351363/cara-menulis-daftar-pustaka-dari-jurnal-berbagai-format-beserta-contohnya

61.

Yatin

S. F. M.

Mokhtar

Baharuddin

Ali

(2018). Approaches and strategies to learn and improve writing APA style. International Journal of Academic Research in Progressive Education and Development, 7(3), 74–85. https://doi.org/10.6007/IJARPED/v7-i3/4348

62.

Yulianto

J. E.

Abraham

Siaputra

I. B.

(2024). Ethics in psychological research and publication: The mocked essentials. ANIMA Indonesian Psychological Journal, 39(1), e01. https://doi.org/10.24123/aipj.v39i1.6323

63.

Zafonte

Parks-Stamm

E. J.

(2016). Effective instruction in APA style in blended and face-to-face classrooms. Scholarship of Teaching and Learning in Psychology, 2(3), 208–218. https://doi.org/10.1037/stl0000064

64.

Zeraati

Alavi

N. M.

(2014). Designing and validity evaluation of quality of nursing care scale in intensive care units. Journal of Nursing Measurement, 22(3), 461–471. https://doi.org/10.1891/1061-3749.22.3.461