This study examines the effectiveness of artificial intelligence (AI) in psychological report writing by comparing reports generated by human psychologists with those produced by OpenAI’s Generative Pre-trained Transformer Version 4 (ChatGPT-4). A total of 249 licensed psychologists evaluated the reports based on overall quality, readability, writing style, organization, summary quality, recommendations, preference, and willingness to sign off on the reports. Although human-generated reports were generally rated more favorably and participants expressed greater comfort in approving them, effect sizes were typically small. Two exceptions were noted: moderate effect sizes were found in favor of human-written summaries, while AI-generated reports showed moderate effect sizes for the quality of their recommendations. These findings suggest that AI shows potential for augmenting report writing. Comprehensive guidelines are necessary for the ethical and effective integration of AI into psychological practice. Further research is needed to enhance our understanding of AI’s role and capabilities in psychological assessment and reporting.
Artificial Analysis. (2024). Independent analysis of AI models and API providers. Artificial Analysis. https://artificialanalysis.ai/
5.
BaumK. T.von ThomsenC.ElamM.MurphyC.GerstleM.AustinC. A.BeebeD. A. (2017). Communication is key: The utility of a revised neuropsychological report format. The Clinical Neuropsychologist, 32(3), 345–367. https://doi.org/10.1080/13854046.2017.1413208
6.
BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society - Series B: Statistical Methodology, 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x
7.
BoettcherN. (2021). Studies of depression and anxiety using reddit as a data source: Scoping review. JMIR Mental Health, 8(11), 1–18. https://doi.org/10.2196%2F29487
8.
Bradley-JohnsonS.JohnsonC. M. (2006). A handbook for effective report writing (2nd ed.). PRO-ED.
9.
BurrC.MorleyJ.TaddeoM.FloridiL. (2020). Digital psychiatry: Risks and opportunities for public health and wellbeing. IEEE Transactions on Technology and Society, 1(1), 21–33. https://doi.org/10.1109/TTS.2020.2977059
10.
ChuZ.NiS.WangZ.FengX.LiC.HuX.ZhangW. (2024). History, development, and principles of large language models-an introductory survey. ArXiv. 1-25. https://doi.org/10.48550/arXiv.2402.06853
11.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
12.
DillmanD. A.SmythJ. D.ChristianL. M. (2014). Internet, phone, mail, and mixed mode surveys: The tailored design method (4th ed.). John Wiley & Sons Inc.
13.
DunnC.HunterJ.SteffesW.WhitneyZ.FossM.MamminoJ.NathooR.HawkinsS. D.DaneA.YungmannM. (2023). Artificial intelligence–derived dermatology case reports are indistinguishable from those written by humans: A single-blinded observer study. Journal of the American Academy of Dermatology, 89(2), 388–390. https://doi.org/10.1016/j.jaad.2023.04.005
EysenbachG. (2004). Improving the quality of web surveys: The checklist for reporting results of internet e-surveys (CHERRIES). Journal of Medical Internet Research, 6(3), Article e34. https://doi.org/10.2196/jmir.6.3.e34
16.
FarmerR.LockwoodA.GoforthA.ThomasC. (2024). Artificial intelligence in practice: Opportunities, challenges, and ethical considerations. Professional Psychology: Research and Practice, 5(1), 19–32. https://psycnet.apa.org/doi/10.1037/pro0000595
17.
FaulF.ErdfelderE.BuchnerA.LangA. G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. https://doi.org/10.3758/BRM.41.4.1149
18.
FilterK. J.EbsenS.DibosR. (2013). School psychology crossroads in America: Discrepancies between actual and preferred discrete practices and barriers to preferred practice. International Journal of Special Education, 28(1), 88–100. https://eric.ed.gov/?id=EJ1013688
GrahamS.DeppC.LeeE. E.NebekerC.TuX.KimH. C.JesteD. V. (2019). Artificial intelligence for mental health and mental illnesses: An overview. Current Psychiatry Reports, 21(11), 1–18. https://doi.org/10.1007%2Fs11920-019-1094-0
22.
GriffithL. E.CookD. J.GuyattG. H.CharlesC. A. (1999). Comparison of open and closed questionnaire formats in obtaining demographic information from Canadian general internists. Journal of Clinical Epidemiology, 52(10), 997–1005. https://doi.org/10.1016/s0895-4356(99)00106-7
23.
Groth-MarnatG.HorvathL. S. (2006). The psychological report: A review of current controversies. Journal of Clinical Psychology, 62(1), 73–81. https://doi.org/10.1002/jclp.20201
24.
HabichtJ.ViswanathanS.CarringtonB.HauserT. U.HarperR.RollwageM. (2024). Closing the accessibility gap to mental health treatment with a personalized self-referral chatbot. Nature Medicine, 30(2), 595–602. https://doi.org/10.1038/s41591-023-02766-x
25.
HarveyV. S. (2006). Variables affecting the clarity of psychological reports. Journal of Clinical Psychology, 62(1), 5–18. https://doi.org/10.1002/jclp.20196
26.
HassM.CarriereJ. A. (2014). Writing useful, accessible, and legally defensible psychoeducational reports. John Wiley & Sons.
27.
HatchS. G.GoodmanZ. T.VowelsL.HatchH. D.BrownA. L.GuttmanS.… BraithwaiteS. R. (2025). When ELIZA meets therapists: A Turing test for the heart and mind. PLOS Mental Health, 2(2), e0000145. https://doi.org/10.1371/journal.pmen.0000145
28.
HodsonN.WilliamsonS. (2024). Can large language models replace therapists? Evaluating performance at simple cognitive behavioral therapy tasks. JMIR AI, 3(1), 1–4. https://doi.org/10.2196/52500
29.
HollanderM.WolfeD. A.ChickenE. (2014). Nonparametric statistical methods. John Wiley & Sons.
LichtensteinR.EckerB. (2019). High-impact assessment reports for children and adolescents: A consumer-responsive approach. Guilford Publication.
32.
LockwoodA. B.CastleberryJ. (2024). Examining the capabilities of GPT-4 to write an APA-style school psychology paper. Contemporary School Psychology, 29(1), 1–8. https://doi.org/10.1007/s40688-024-00500-z
MastorasS. M.ClimieE. A.McCrimmonA. W.SchweanV. L. (2011). A C.L.E.A.R. Approach to report writing: A framework for improving the efficacy of psychoeducational reports. Canadian Journal of School Psychology, 26(2), 127–147. https://doi.org/10.1177/0829573511409722
37.
Mat HassanN.SalimH. S.AmaranS.YunusN. I.YusofN. A.DaudN.FryD. (2023). Prevalence of mental health problems among children with long COVID: A systematic review and meta-analysis. PLoS One, 18(5), 1–27. https://doi.org/10.1371/journal.pone.0282538
MoellB. (2024). Comparing the efficacy of GPT-4 and chat-GPT in mental health care: A blind assessment of large language models for psychological support. ArXiv, 1–14. https://doi.org/10.48550/arXiv.2405.09300
PintoD. S.NoronhaS. M.SaigalG.QuencerR. M. (2024). Comparison of an AI-generated case report with a human-written case report: Practical considerations for AI-assisted medical writing. Cureus, 16(5), 1–11. https://doi.org/10.7759/cureus.60461
RollwageM.HabichtJ.JuchemsK.CarringtonB.HauserT. U.HarperR. (2024). Conversational AI facilitates mental health assessments and is associated with improved recovery rates. BMJ Innovations, 10(1-2), 4–12. https://doi.org/10.1136/bmjinnov-2023-001110
46.
SrivastavaA.RastogiA.RaoA.ShoebA. A. M.AbidA.FischA.WangG. (2022). Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. ArXiv, 1–95. https://doi.org/10.48550/arXiv.2206.04615
TorousJ.BucciS.BellI. H.KessingL. V.Faurholt-JepsenM.WhelanP.CarvalhoA. F.KeshavanM.LinardonJ.FirthJ. (2021). The growing field of digital psychiatry: Current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry: Official Journal of the World Psychiatric Association (WPA), 20(3), 318–335. https://doi.org/10.1002/wps.20883
49.
WienerJ.CostarisL. (2012). Teaching psychological report writing: Content and process. Canadian Journal of School Psychology, 27(2), 119–135. https://doi.org/10.1177/0829573511418484
50.
ZellE.StrickhouserJ. E.SedikidesC.AlickeM. D. (2020). The better-than-average effect in comparative self-evaluation: A comprehensive review and meta-analysis. Psychological Bulletin, 146(2), 118–149. https://doi.org/10.1037/bul0000218
51.
ZiweiB. Y.ChuaH. N. (2019). An application for classifying depression in tweets. In Proceedings of the 2nd international conference on computing and big data (pp. 37–41). Association for Computing Machinery.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
