This paper presents a project evaluation framework for capacity building ICT projects, and describes an action research initiative in which the framework was piloted. The framework is a five stage process that draws on soft systems thinking and best practice in areas like ICTs for development and information systems evaluation. The action research was a collaborative undertaking between the author and an international non-governmental organization, and involved a project to develop information management solutions for four non-governmental organizations in Armenia.
Abu-SamahaA (2003) Soft evaluation: A systemic approach for post implementation review. In CanoJ.J., ed. Critical Reflections on Information Systems: A systemic approach. Hershey, PA: IGI Publishing, 136–157.
2.
AgarwalNRathodU (2006) Defining ‘success’ for software projects: An exploratory revelation. International Journal of Project Management24(4): 358–370.
3.
AgyemangGAwumbilaMUnermanJO’DwyerB (2009) NGO Accountability and Aid Delivery. London, Association of Chartered Certified Accountants.
4.
BarclayCOsei-BrysonKM (2010) Project performance development framework: An approach for developing performance criteria & measures for information systems (IS) projects. International Journal of Production Economics124(1): 272–292.
5.
BartisEMitevN (2008) A multiple narrative approach to information systems failure: A successful system that failed. European Journal of Information Systems17(2): 112–124.
6.
BrighamMHayesN (2013) Hybridity, consulting and e-development in the making: Inscribing new practices of impact assessment and value management. Information Technology for Development19(2): 112–132.
7.
BrysonJM (2012) Strategic Planning for Public and Nonprofit Organizations: A guide to strengthening and sustaining organizational achievement. Wiley.
8.
Cecez-KecmanovicDKautzKAbrahallR (2014) Reframing success and failure of information systems: A performative perspective. MIS Quarterly38(2): 561–588.
9.
ChecklandPScholesJ (1990) Soft Systems Methodology in Action. New York: John Wiley & Sons.
10.
ChenSOsmanMPengGC (2012) Information systems evaluation: Methodologies and practical case studies. In: NunesJMBIsaiasP (eds.) Information Systems Research and Exploring Social Artifacts: Approaches and Methodologies. Hershey, PA, IGI Global. pp. 333–354.
11.
CleaverF (2003) Paradoxes of participation: Questioning participatory approaches to development. In EdwardsMFowlerA (eds.) The Earthscan Reader on NGO Management. VA: Earthscan Publications Limited, 225–240.
12.
CrawfordPBryceP (2003) Project monitoring and evaluation: A method for enhancing the efficiency and effectiveness of aid project implementation. International Journal of Project Management21(5): 363–373.
13.
CronholmSGoldkuhlG (2003) Strategies for information systems evaluation – six generic types. Electronic Journal of Information Systems Evaluation6(2): 65–74.
14.
DialloAThuillierD (2005) The success of international development projects, trust and communication: An African perspective. International Journal of Project Management23(3): 237–252.
15.
DinizEHBaileyDEShollerD (2014) Achieving ICT4D project success by altering context, not technology. Information Technologies & International Development10(4): 15–29.
16.
DuncombeR (2011) Researching impact of mobile phones for development: Concepts, methods and lessons for practice. Information Technology for Development17(4): 268–288.
17.
EadeD (2007) Capacity building: Who builds whose capacity?Development in Practice17(4-5): 630–639.
18.
Fister GaleS (2011) Saving the world with project management–Not-for-profits can do good–For the world and their own ROI–By increasing transparency and accountability. PM Network-Project Management25(7): 24.
19.
FitzgeraldG (1998) Evaluating information systems projects: A multidimensional approach. Journal of Information Technology13: 15–27.
20.
GoliniRLandoniP (2014) International development projects by non-governmental organizations: An evaluation of the need for specific project management and appraisal tools. Impact Assessment and Project Appraisal32(2): 121–135.
21.
GrantAPlanteILeblancF (2002) The TEAM methodology for the evaluation of information systems in biomedicine. Computers in Biology and Medicine32: 195–207.
22.
HardmanJPaucar-CaceresA (2011) A soft systems methodology (SSM) based framework for evaluating managed learning environments. Systemic Practice and Action Research24(2): 165–185.
23.
HayesNWestrupC (2012) Power/knowledge and impact assessment: Creating new spaces for expertise in international development. New Technology, Work and Employment27(1): 9–22.
24.
HedmanJBorellA (2005) Broadening information systems evaluation through narratives. The Electronic Journal of Information Systems Evaluation8(2): 115–122.
25.
HeeksR (2002) Information systems and developing countries: Failure, success, and local improvisations. Information Society18(2): 101–113.
26.
HeeksR (2006) Health information systems: Failure, success and improvisation. International Journal of Medical Informatics75: 125–137.
27.
HughesJNiewenhuisL (2005) A Project Manager’s Guide to Evaluation. Institute Technology and Education (ITB), University of Bremen. (Evaluate Europe Handbook Series).
28.
IkaLA (2012) Project management for development in Africa: Why projects are failing and what can be done about it. Project Management Journal43(4): 27–41.
29.
IkaLADialloAThuillerD (2012) Critical success factors for World Bank projects: An empirical investigation. International Journal of Project Management30: 105–116.
30.
IraniZLovePED (2001) Information systems evaluation: Past, present and future. European Journal of Information Systems10: 183–188.
31.
KhangDBMoeTL (2008) Success criteria and factors for international development projects: A life-cycle-based framework. Project Management Journal39(1): 72–84.
32.
LagstenJGoldkuhlG (2008) Interpretative IS evaluation: Results and uses. The Electronic Journal Information Systems Evaluation11(2): 97–108.
33.
LiuS (2015) How team risk and planning and control risk moderate the effects of clan and self control on the process performance of IT projects: The perspective of user liaisons. Information Development31(1): 27–39.
34.
LiuSDengZ (2015) How environment risks moderate the effect of control on performance in information technology projects: Perspectives of project managers and user liaisons. International Journal of Information Management35(1): 80–97.
35.
MadonS (2005) Evaluating the developmental impact of e-governance initiatives: An exploratory framework. The Electronic Journal of Information Systems in Developing Countries20 (5): 1–13.
36.
MeyerCA (1997) The political economy of NGOs and information sharing. World Development25(7): 1127–1140.
37.
MidgleyG (2006) Systems thinking for evaluation. In: WilliamsBImamI (eds.) Systems Concepts in Evaluation: An Expert Anthology. Point Reyes CA: EdgePres. 11–34.
38.
MthokoHLPade-KheneC (2013) Towards a theoretical framework on ethical practice in ICT4D programmes. Information Development29(1): 36–53.
39.
Navarro FloresO (2011) Organizing by projects: A strategy for local development – The case of NGOs in a developing country. Project Management Journal42(6): 48–59.
40.
NgachoCDasD (2014) A performance evaluation framework of development projects: An empirical study of Constituency Development Fund (CDF) construction projects in Kenya. International Journal of Project Management32: 492–507.
41.
NkohkwoQIslamM (2013) Challenges to the successful implementation of e-government initiatives in Sub-Saharan Africa: A literature review. Electronic Journal of e-Government11(2): 253–267.
42.
ParadiceD (2009) Emerging Systems Approaches in Information Technologies: Concepts, theories, and applications. Hershey, PA, IGI Global.
43.
Pérez-BritoC (2009) Information as a development intervention: Reforming the culture of information systems in Guatemala. Information Development25(4): 272–282.
44.
ReasonPBradburyH (2008) The Sage Handbook of Action Research: Participative Inquiry and Practice, Los Angeles, London, New Delhi, Singapore: Sage Publications.
45.
SahlfeldM (2007) How does ICT work for development? A review of the challenges and opportunities. ATDF Journal4(1): 22–36.
46.
SchrammWL (1964) Mass Media and National Development: The role of information in the developing countries. Redwood City, CA, Stanford University Press.
47.
SerafeimidisVSmithsonS (2000) Information systems evaluation in practice: A case study of organisational change. Journal of Information Technology15: 93–105.
48.
SerafeimidisVSmithsonS (2003) Information systems evaluation as an organisational institution – experience from a case study. Information Systems Journal13: 251–274.
49.
SmillieI (1994) Changing partners: northern NGOs, northern governments. Voluntas: International Journal of Voluntary and Nonprofit Organizations5(2): 155–192.
50.
SmithsonSHirschheimR (1998) Analysing information systems evaluation: Another look at an old problem. European Journal of Information Systems7: 158–174.
51.
StockdaleRStandingC (2006) An interpretive approach to evaluating information systems: A content, context, process framework. European Journal of Operational Research173: 1090–1102.
52.
ThatcherJBBrowerRSMasonRM (2006) Organizational fields and the diffusion of information technologies within and across the nonprofit and public sectors. American Review of Public Administration36(4): 437–454.
53.
UnwinP (2009) ICT4D: Information and Communication Technology for Development. Cambridge, UK: Cambridge University Press.
54.
VasilecasOSaulisADereškevičiusS (2015) Evaluation of information systems procurement: Goal and task-driven approaches. Information Technology And Control35(3).
55.
WaltonMHeeksR (2011) Can a Process Approach Improve ICT4D Project Success?Manchester: Development Informatics Group, Institute for Development Policy and Management: unpublished.
56.
WinklhoferH (2002) A case for soft systems methodology: Information analysis and information systems evaluation during organizational change. Paper presented at ECIS, Gdansk, Poland, 6-8 June, 2002.
57.
ZhangWGutierrezOMathiesonK (2010) Information systems research in the nonprofit context: Challenges and opportunities. Communications of the Association for Information Systems27 (Article 1).
