This paper explores the contributions and limitations of Generative AI (GenAI), including Chat GPT, in automated content analysis (ACA) in marketing. It compares the performance of several Chat GPT-driven ACA protocols with manual and automated ad hoc dictionary protocols, highlighting methodological and practical implications. The experiment uses a corpus of 992 responses from a consumer survey. It examines the quality of the different protocols for coding verbatim, but also for the design of thematic grids and ad hoc dictionaries. Performance is evaluated on validity and efficiency criteria. On the various key tasks of ACA, Chat GPT performs significantly worse than classical methods based on corpus analysis and researcher participation. The results show that GenAI has reduced validity, with frequent omissions and errors in the coding of specific data. GenAI can intervene in certain stages of ACA and offer time savings, but it requires rigorous and informed human supervision to ensure the validity of the results, highlighting the importance of a hybrid human-machine approach. The article provides a first systematic evaluation of GenAI applied to ACA in marketing, and proposes methodological recommendations, highlighting the fundamental differences between the algorithmic approach and the researchers’ approach based on real data.
AndlerD. (2023). Intelligence artificielle, intelligence humaine: la double énigme. Gallimard.
2.
BagS.TiwariM. K.ChanF. T. (2019). Predicting the consumer’s purchase intention of durable goods: An attribute-level analysis. Journal of Business Research, 94, 408–419. https://doi.org/10.1016/j.jbusres.2017.11.031
3.
BaliL. M.AnesburyZ. W.PhuaP.SharpB. (2024). How prevalent are suggestive brand names and distinctive assets? An AI-human approach. International Journal of Market Research, 66(5), Article 14707853241251954.
4.
BergerJ.HumphreysA.LudwigS.MoeW. W.NetzerO.SchweidelD. A. (2020). Uniting the tribes: Using text for marketing insight. Journal of Marketing, 84(1), 1–25. https://doi.org/10.1177/0022242919873106
5.
BijkerR.MerkourisS. S.DowlingN. A.RoddaS. N. (2024). Chat GPT for automated qualitative research: Content analysis. Journal of Medical Internet Research, 26, e59050. https://doi.org/10.2196/59050
6.
BonescuM.GinonE.TavillaV.BiguzziC.SutanA. (2023). Représentations de la convivialité: de l’imaginaire social du repas au bien-être alimentaire. Décisions Marketing, 109(1), 103–130. https://doi.org/10.3917/dm.109.0103
DaughertyT.EastinM. S.BrightL. (2008). Exploring consumer motivations for creating user-generated content. Journal of Interactive Advertising, 8(2), 16–25. https://doi.org/10.1080/15252019.2008.10722139
10.
GanassaliS. (2019). Status of brands in children’s consumption: What letters to Santa posted on La Poste website tell us. Psychology and Marketing, 36(1), 5–14. https://doi.org/10.1002/mar.21153
11.
Gavard-PerretM.-L.MoscarolaJ. (1998). Énoncé ou énonciation ? deux objets différents de l'analyse lexicale en marketing. Recherche et Applications en Marketing, 13(2), 31–47. https://doi.org/10.1177/076737019801300203
12.
GoyanesM.LopezosaC.JordáB. (2025). Thematic analysis of interview data with ChatGPT: Designing and testing a reliable research protocol for qualitative research. Quality and Quantity, 1–20.
13.
GrimmerJ.StewartB. M. (2013). Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political Analysis, 21(3), 267–297. https://doi.org/10.1093/pan/mps028
14.
HermannE.PuntoniS. (2024a). Artificial intelligence and consumer behavior: From predictive to generative AI. Journal of Business Research, 180, Article 114720. https://doi.org/10.1016/j.jbusres.2024.114720
15.
HermannE.PuntoniS. (2024b). Generative AI in marketing and principles for ethical design and deployment. Journal of Public Policy and Marketing, 44(3), 332–349. https://doi.org/10.1177/07439156241309874
16.
HumphreysA. (2022). Automated text analysis. In HomburgC.KlarmannM.VombergA. (Eds.), Handbook of market research (pp. 633–664): Springer.
17.
HumphreysA.WangR. J. H. (2018). Automated text analysis for consumer research. Journal of Consumer Research, 44(6), 1274–1306. https://doi.org/10.1093/jcr/ucx104
18.
JambuM. (1998). Datamining for better knowledge. A case study from the telecommunications industry. In FellowsD. (Ed.), The power of knowledge: From research findings to marketing intelligence - Proceedings of the 51st ESOMAR congress (pp. 153–164). ESOMAR.
19.
KessousA. (2015). Nostalgia and brands: A sweet rather than a bitter cultural evocation of the past. Journal of Marketing Management, 31(17-18), 1899–1923. https://doi.org/10.1080/0267257x.2015.1088889
20.
LahlouS.Collerie de BorélyA.BeaudouinV. (1993). Où en est la consommation aujourd’hui ? Une enquête sur le consommateur français des années 90, Cahier de recherche N°46. CREDOC.
21.
LeeL. W.DabirianA.McCarthyI. P.KietzmannJ. (2020). Making sense of text: Artificial intelligence-enabled content analysis. European Journal of Marketing, 54(3), 615–644. https://doi.org/10.1108/ejm-02-2019-0219
22.
LincolnY. S.GubaE. G. (1985). Naturalistic inquiry. Sage.
23.
MarianiM. M.Perez‐VegaR.WirtzJ. (2022). AI in marketing, consumer research and psychology: A systematic literature review and research agenda. Psychology and Marketing, 39(4), 755–776. https://doi.org/10.1002/mar.21619
24.
MorganD. L. (2023). Exploring the use of artificial intelligence for qualitative data analysis: The case of chat GPT. International Journal of Qualitative Methods, 22, Article 16094069231211248. https://doi.org/10.1177/16094069231211248
25.
MoscarolaJ. (2024). Coupler analyse de données textuelles et IA génératives pour augmenter le processus de recherche qualitative. in Actes des 17è Journées Internationales d’Analyse Statistique des Données Textuelles (JADT), Université Catholique de Louvain.
26.
NeuendorfK. A. (2018). Content analysis and thematic analysis. In BroughP. (Ed.), Advanced research methods for applied psychology (pp. 211–223): Routledge.
27.
NowellL. S.NorrisJ. M.WhiteD. E.MoulesN. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), Article 1609406917733847.
OpokuR. A.HultmanM.Saheli-SangariE. (2008). Positioning in market space: The evaluation of Swedish universities' online brand personalities. Journal of Marketing for Higher Education, 18(1), 124–144. https://doi.org/10.1080/08841240802100386
30.
PanB.MacLaurinT.CrottsJ. C. (2007). Travel blogs and the implications for destination marketing. Journal of Travel Research, 46(1), 35–45. https://doi.org/10.1177/0047287507302378
31.
PotterW. J.Levine-DonnersteinD. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research, 27(3), 258–284. https://doi.org/10.1080/00909889909365539
32.
RadfordA.WuJ.ChildR.LuanD.AmodeiD.SutskeverI. (2019). Language models are unsupervised multi task learners. OpenAI blog, 1(8), 9.
33.
RathjeS.MireaD. M.SucholutskyI.MarjiehR.RobertsonC. E.Van BavelJ. J. (2024). GPT is an effective tool for multilingual psychological text analysis. Proceedings of the National Academy of Sciences, 121(34).
34.
ReinertM. (1986). Un logiciel d'analyse lexicale. Les Cahiers de l'analyse des données, 11(4), 471–481.
35.
SaldañaJ. (2013). The coding manual for qualitative researchers. Sage Publications.
36.
SarstedtM.AdlerS. J.RauL.SchmittB. (2024). Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines. Psychology and Marketing, 41(6), 1254–1270. https://doi.org/10.1002/mar.21982
37.
SchulhoffS.IlieM.BalepurN.KahadzeK.LiuA.SiC.ResnikP. (2024). The prompt report: A systematic survey of prompt engineering techniques. arXiv preprint arXiv:2406.06608.
38.
TaboneW.De WinterJ. (2023). Using chat GPT for human-computer interaction research: A primer. Royal Society Open Science, 10(9), Article 231053. https://doi.org/10.1098/rsos.231053
39.
ThomazG. M.BizA. A.BettoniE. M.Mendes-FilhoL.BuhalisD. (2017). Content mining framework in social media: A FIFA world cup 2014 case analysis. Information & Management, 54(6), 786–801. https://doi.org/10.1016/j.im.2016.11.005
40.
TsaoH. Y.CampbellC. L.SandsS.FerraroC.MavrommatisA.LuS. (2020). A machine-learning based approach to measuring constructs through text analysis. European Journal of Marketing, 54(3), 511–524. https://doi.org/10.1108/ejm-01-2019-0084
41.
TurobovA.CoyleD.HardingV. (2024). Using ChatGPT for thematic analysis. University of Cambridge. Preprint arXiv:2405.08828.
42.
UbalV. O.LisjakM.MendeM. (2024). Cracking the consumers’ code: A framework for understanding the artificial intelligence-consumer interface. Current Opinion in Psychology, 58, Article 101832. https://doi.org/10.1016/j.copsyc.2024.101832
43.
VaidS.PuntoniS.KhodrA. (2023). Artificial intelligence and empirical consumer research: A topic modeling analysis. Journal of Business Research, 166, Article 114110. https://doi.org/10.1016/j.jbusres.2023.114110
44.
WuX.XiaoL.SunY.ZhangJ.MaT.HeL. (2022). A survey of human-in-the-loop for machine learning. Future Generation Computer Systems, 135, 364–381. https://doi.org/10.1016/j.future.2022.05.014
45.
XiangZ.SchwartzZ.GerdesJr. H.UysalM. (2015). What can big data and text analytics tell us about hotel guest experience and satisfaction?International Journal of Hospitality Management, 44, 120–130. https://doi.org/10.1016/j.ijhm.2014.10.013
46.
YeeL. B.Pei TanP. (2014). Gaining customer knowledge in low-cost airlines through text mining. Industrial Management & Data Systems, 114(9), 1344–1359. https://doi.org/10.1108/imds-07-2014-0225
