The rapid advancement of digital technologies, accelerated by the COVID-19 pandemic and the emergence of accessible artificial intelligence tools, has fundamentally transformed the evaluation landscape. This transformation presents both unprecedented opportunities and significant challenges for the evaluation community. Evaluators face critical questions about the appropriate use of digital technologies: How can we ensure proper application while maintaining established evaluation standards? How do we address issues of data ownership, algorithmic transparency, and ethical considerations? What knowledge and skills do evaluators need to navigate this digital transformation effectively? This special issue addresses three interconnected challenges in the digitalization of evaluation: methodological knowledge gaps, data and technology ownership issues, and ethical considerations. While digital technologies offer powerful tools for data collection, analysis, and stakeholder engagement, human oversight remains essential to ensure rigorous, ethical, and equitable evaluation practices.
ArceriNoFGiudiceOBattiatoS (2023) An innovative tool for uploading/scraping large image datasets on social networks. In: Paper presented at the IEEE international conference on metrology for eXtended reality, artificial intelligence and neural engineering, Milano, 25–27 October.
AzzamT (2023) Artificial intelligence and validity. New Directions for Evaluation2023(178–179): 85–95.
4.
BambergerM (2016) Integrating big data into the monitoring and evaluation of development programmes.
5.
Barbrook-JohnsonPCastellaniBHillsD, et al. (2021) Policy evaluation for a complex world: Practical methods and reflections from the UK Centre for the evaluation of complexity across the Nexus. Evaluation27(1): 4–17.
6.
BenYishayASayersRSinghK, et al. (2024) Irrigation strengthens climate resilience: Long-term evidence from Mali using satellites and surveys. PNAS Nexus3(2): pgae022.
7.
BlumenstockJCadamuroGOnR (2015) Predicting poverty and wealth from mobile phone metadata. Science350(6264): 1073–6.
8.
BonfiglioACamaioniBCartaV, et al. (2023) Estimating the common agricultural policy milestones and targets by neural networks. Evaluation and Program Planning99: 102296.
9.
BouyousfiSEOuedraogoM (2025) Artificial intelligence and big data-driven evaluation research and practices: A systematic literature review. Evaluation 31(3): 303–330.
10.
BrydaGCostaAP (2023) Qualitative research in digital era: Innovations, methodologies and collaborations. Social Sciences12(10). DOI: 10.3390/socsci12100570.
11.
BudziszewskaAPotlukaO (2025) The use of AI and technology by civil society organizations and its international implications. In: UgazioGMaricicM (eds) The Routledge Handbook of Artificial Intelligence and Philanthropy, 1 edn.London: Routledge, 94–106.
12.
DewiLCMeilianaChandraA (2019) Social media web scraping using social media developers API and Regex. Procedia Computer Science157: 444–9.
13.
DogucuMÇetinkaya-RundelM (2021) Web scraping in the statistics and data science curriculum: Challenges and opportunities. Journal of Statistics and Data Science Education29(Suppl. 1): S112–22.
14.
DongoICadinaleYAguileraA, et al. (2020) Web scraping versus Twitter API. In: Paper presented at the proceedings of the 22nd international conference on information integration and web-based applications & services, Chiang Mai, Thailand, 30 November–2 December.
15.
Fabra-MataJMygindJ (2019) Big data in evaluation: Experiences from using Twitter analysis to evaluate Norway’s contribution to the peace process in Colombia. Evaluation25(1): 6–22.
16.
FerrettiS (2023) Hacking by the prompt: Innovative ways to utilize ChatGPT for evaluators. New Directions for Evaluation2023(178–179): 73–84.
17.
FlammK (2017) Measuring Moore’s law: Evidence from price, cost, and quality indexes. In: Paper presented at the measuring and accounting for innovation in the twenty-first century, Washington, DC, 10–11 March 2017. Georgetown McDonough School of Business.
18.
Glez-PenaDLourencoALopez-FernandezH, et al. (2014) Web scraping technologies in an API world. Brief Bioinform15(5): 788–97.
19.
GreeneJC (1986) Participatory evaluation and the evaluation of social programs: Lessons learned. Advances in program evaluation. In: Paper presented at the annual meeting of the American educational research association, 16–20 April 1986. San Francisco, CA.
20.
GreeneJC (2000) Understanding social programs through evaluation. In: DenzinNKLincolnYS (eds) Handbook of Qualitative Research. London: Sage, 981–999.
21.
Hanson-DeFuscoJ (2023) What data counts in policymaking and programming evaluation—Relevant data sources for triangulation according to main epistemologies and philosophies within social science. Evaluation and Program Planning97: 102238.
22.
HavrdaMKlocekA (2023) Well—being impact assessment of artificial intelligence—A search for causality and proposal for an open platform for well-being impact assessment of AI systems. Evaluation and Program Planning99: 102294.
23.
HeadCBJasperPMcConnachieM, et al. (2023) Large language model applications for evaluation: Opportunities and ethical implications. New Directions for Evaluation2023(178–179): 33–46.
24.
HeinonenN-M (2025) Piloting text-mining in a mixed-method evaluation on human rights: Management propositions for small public administrations. Evaluation 31(3): 352–368.
25.
HlebowitshP (2021) Ralph Tyler, the Tyler Rationale, and the Idea of Educational Evaluation. Oxford: Oxford University Press.
26.
JamiesonVAzzamT (2012) The use of technology in evaluation practice. Journal of Multidisciplinary Evaluation8(18): 1–15.
27.
KaliskiB (2025) Moore’s Law. In: JajodiaSSamaratiPYungM (eds) Encyclopedia of Cryptography, Security and Privacy. Cham: Springer.
28.
KarthikeyanTSekaranKRanjithD, et al. (2019) Personalized content extraction and text classification using effective web scraping techniques. International Journal of Web Portals (IJWP)11(2): 41–52.
29.
KelleU (2004) Computer-assisted qualitative data analysis. In: SealeCGoboGGubriumJF, et al (eds) Qualitative Research Practice. London: Sage, 443.
30.
KochelTRWestMPToro-AlvarezMM (2025) Modeling a needs assessment approach for policymakers to investigate, understand, and reduce gun violence. Evaluation and Program Planning108: 102505.
31.
LeeuwFL (2024) The algorithmization of policy and society: The need for a realist evaluation approach. In: NielsenSBRinaldiFMPetersonGJ (eds) Artificial Intelligence and Evaluation: Emerging Technologies and their Implications for Evaluation. New York: Routledge, 242–265.
32.
LeeuwFLLeeuwB (2012) Cyber society and digital policies: Challenges to evaluation?Evaluation18(1): 111–27.
33.
Lefebvre-NaréFLemireSPeterssonGJ (2017) What is big data? In: PeterssonGJBreulJD (eds) Cyber Society, Big Data, and Evaluation. Piscataway, NJ: Transaction Publishers, 19–34.
34.
LuscombeADickKWalbyK (2022) Algorithmic thinking in the public interest: Navigating technical, legal, and ethical hurdles to web scraping in the social sciences. Quality & Quantity56(3): 1023–44.
35.
MarchioniniG (1995) Information Seeking in Electronic Environment. Cambridge: Cambridge University Press.
36.
MatlalaLS (2025) Navigating program evaluation amid health crises: Evaluator’s experiences on conducting virtual focus group discussions. Evaluation and Program Planning111: 102568.
37.
MillsDPudneySPevcinP, et al. (2022) Evidence-based public policy decision-making in smart cities: Does extant theory support achievement of city sustainability objectives?Sustainability14(1): 1–23.
38.
MilovanovićSBogdanovićZLabusA, et al. (2022) Social recruiting: An application of social network analysis for preselection of candidates. Data Technologies and Applications56(4): 536–57.
39.
Montrosse-MoorheadB (2023) Evaluation criteria for artificial intelligence. New Directions for Evaluation2023(178–179): 123–34.
40.
NielsenSBRinaldiFMPetersonGJ (eds) (2024) Artificial Intelligence and Evaluation: Emerging Technologies and Their Implications for Evaluation. New York: Routledge.
41.
PicciottoR (2020) Evaluation and the big data challenge. American Journal of Evaluation41(2): 166–81.
42.
PowellSCaldas CabralGMishanH (2025) A workflow for collecting and understanding stories at scale, supported by artificial intelligence. Evaluation 31(3): 394–411.
43.
PudneySMillsDAlaeiAR, et al. (2025) Evaluation of artificial intelligence–enhanced critical infrastructure systems: A conceptual framework. Evaluation 31(3): 412–443.
44.
RavehEOfekYBekkermanR, et al. (2020) Applying big data visualization to detect trends in 30 years of performance reports. Evaluation26(4): 516–40.
45.
ReidAM (2023) Vision for an equitable AI world: The role of evaluation and evaluators to incite change. New Directions for Evaluation2023(178–179): 111–21.
46.
RinaldiFMGiuffridaGNegreteT (2017) Real—time monitoring and evaluation—Emerging news as predictive process using Big Data-based approach. In: PeterssonGJBreulJD (eds) Cyber Society, Big Data, and Evaluation. New Brunswick, NJ: Transaction Publishers, 191–214.
47.
RompczykK (2025) Technological revolution in evaluation: Artificial intelligence and the adherence to evaluation standards. Evaluation 31(3): 331–351.
48.
SabarreNRBeckmannBBhaskaraS, et al. (2023) Using AI to disrupt business as usual in small evaluation firms. New Directions for Evaluation2023(178–179): 59–71.
ShalfJ (2020) The future of computing beyond Moore’s Law. Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences378(2166): 20190061.
51.
SinghKKMuirCSayersR, et al. (2024) Geospatial impact evaluation: A novel approach to assessing development interventions. Reference Module in Earth Systems and Environmental Sciences: 1–25.
52.
SrivastavaA (2022) Challenges for evaluation practices and innovative approaches: Lessons during COVID-19 pandemic. Evaluation and Program Planning92: 102095.
53.
StakeRE (1995) The Art of Case Study Research. Thousand Oaks, CA: Sage.
54.
StakeRE (2000) Program evaluation, particularly responsive evaluation. In: StufflebeamDLMadausGFKellaghanT (eds) Evaluation Models: Viewpoints on Educational and Human Services Evaluation. Dordrecht: Kluwer Academic Publishers, 343–362.
55.
SteeleJESundsoyPRPezzuloC, et al. (2017) Mapping poverty using mobile phone and satellite data. Journal of the Royal Society Interface14(127). DOI: 10.1098/rsif.2016.0690.
56.
ThorntonI (2023) A special delivery by a fork: Where does artificial intelligence come from?New Directions for Evaluation2023(178–179): 23–32.
57.
TiltonZLaVelleJMFordT, et al. (2023) Artificial intelligence and the future of evaluation education: Possibilities and prototypes. New Directions for Evaluation2023(178–179): 97–109.
58.
TylerRW (1949) Basic Principles of Curriculum and Instruction. Chicago, IL: The University of Chicago Press.
59.
VogelNTheisenCLeidigJP, et al. (2015) Mining mobile datasets to enable the fine-grained stochastic simulation of Ebola diffusion. Procedia Computer Science51: 765–74.
60.
WenckerTBorst-GraetzJNieklerA (2025) Text as data for evaluation: Natural language processing and large language models to generate novel insights from unstructured text data. Evaluation 31(3): 369–393.
61.
WesolowskiABengtssonLBuckeeCO, et al. (2014) West Africa human mobility models: Version 1.0.
