This research seeks to determine whether IT leaders are willing to leverage Artificial Intelligence (AI) recommendations to augment their decision-making when considering which IT projects to select based on the projects’ contribution towards organizational goals and objectives and the IT leaders’ level of knowledge and trust in AI. The research design utilized a quantitative survey of 113 IT leaders to understand their willingness to accept AI recommendations that augment their own decision-making while considering organizational criteria for selecting the best IT projects to invest in. AI recommendations were measured based on factors contained within a conceptual AI-driven online recommendation system, while decision-making was measured based on factors such as uncertainty, consequences of decisions, information & goals, motivation, and self-regulation that were designed and validated to examine certain aspects that influence decision-making. IT project contributions were measured based on relevance, risk, reasonableness, basis research return, and business return associated with the project. The research results showed the relationship between IT Project Contributions and IT leader decision was significant in that the leader's perception of IT project contributions towards organizational goals and objectives has a positive influence on the IT project selection decision. Unfortunately, the remaining hypothesized relationships showed no significant outcomes. The research results did not show that IT Leader Decisions may be influenced by AI recommendations, nor the greater knowledge of AI an IT Leader has, the stronger the relationship will be in support of or against AI recommendations, nor that the greater trust in AI an IT Leader has, the stronger the relationship will be in support of or against AI recommendations. This research will provide important insights to understand IT project selection and human-AI collaboration by IT leaders in organizations.
AdadiABerradaM (2018) Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access6: 52138–52160.
2.
AgorWH (1986) The logic of intuition: How top executives make important decisions. Organizational Dynamics14(3): 5–18.
3.
AgrawalAGansJSGoldfarbA (2017) What to expect from artificial intelligence. MIT Sloan Management Review9: 23–27.
4.
AlamiHLehouxPJean-LouisD, et al. (2021) Organizational readiness for artificial intelligence in health care: Insights for decision-making and practice. Journal of Health Organization and Management35(1): 106–114.
5.
Alvarado-ValenciaJABarreroLH (2014) Reliance, trust and heuristics in judgmental forecasting. Computers in Human Behavior36: 102–113.
6.
ApteUSankarCSThakurM, et al. (1990) Reusability-Based strategy for development of information systems: Implementation experience of a bank. MIS Quarterly14(4): 421–433.
7.
AsoshehANalchigarSJamporazmeyM (2010) Information technology project evaluation: An integrated data envelopment analysis and balanced scorecard approach. Expert Systems with Applications37(8): 5931–5938.
8.
BainbridgeWAHartJKimES, et al. (2008) The effect of presence on human-robot interaction. RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication: 701–706. https://doi.org/10.1109/ROMAN.2008.4600749.
9.
BakerALPhillipsEKUllmanD, et al. (2018) Toward an understanding of trust repair in human-robot interaction: Current research and future directions. ACM Transactions on Interactive Intelligent Systems8(4): 1–30.
10.
BankerRDKauffmanRJ (1991) Reuse and productivity in integrated computer-aided software engineering: An empirical study. MIS Quarterly15(3): 75.
11.
BansalGNushiBKamarE, et al. (2019) Updates in human-AI teams: Understanding and addressing the performance/compatibility tradeoff. Proceedings of the AAAI Conference on Artificial Intelligence33: 2429–2437.
BleaseCKaptchukTJBernsteinMH, et al. (2019) Artificial intelligence and the future of primary care: Exploratory qualitative study of UK general Practitioners’ views. Journal of Medical Internet Research21(3): e12802.
14.
BolatBÇebiFTemurGT, et al. (2014) A fuzzy integrated approach for project selection. Journal of Enterprise Information Management27(3): 247–260.
15.
BraunTFungBCMIqbalF, et al. (2018) Security and privacy challenges in smart cities. Sustainable Cities and Society39: 499–507.
16.
BrynjolfssonEMcafeeA (2017) Artificial intelligence, for real. Artificial Intelligence31: 3–11.
17.
BurkeLAMillerMK (1999) Taking the mystery out of intuitive decision making. The Academy of Management Executive13(4): 91–99.
18.
BusenitzLWBarneyJB (1997) Differences between entrepreneurs and managers in large organizations: Biases and heuristics in strategic decision-making. Journal of Business Venturing12(1): 9–30.
19.
BussMD (1983) How to rank computer projects. Harvard Business Review61(1): 118–125.
20.
Cabrera-SánchezJ-PRamos-de-LunaICarvajal-TrujilloE, et al. (2020) Online recommendation systems: Factors influencing use in E-commerce. Sustainability12(21): 8888.
21.
CampbellA (1991) Brief case: Strategy and intuition—A conversation with Henry Mintzberg. Long Range Planning24(2): 108–110.
22.
ChenC-TChengH-L (2009) A comprehensive model for selecting information system project under fuzzy environment. International Journal of Project Management27(4): 389–399.
23.
ChongLZhangGGoucher-LambertK, et al. (2022) Human confidence in artificial intelligence and in themselves: The evolution and impact of confidence on adoption of AI advice. Computers in Human Behavior127: 107018.
24.
CooperRGEdgettSJKleinschmidtEJ (1999) New product portfolio management: Practices and performance. Journal of Product Innovation Management16(4): 333–351.
25.
CulleyKEMadhavanP (2013) A note of caution regarding anthropomorphism in HCI agents. Computers in Human Behavior29(3): 577–579.
26.
DalkeyN (1969) An experimental study of group opinion. Futures1(5): 408–426.
27.
DaneEPrattMG (2007) Exploring intuition and its role in managerial decision making. Academy of Management Review32(1): 33–54.
28.
DieboltVAzancotIBoisselF-H, et al. (2019) Artificial intelligence”: Which services, which applications, which results and which development today in clinical research? Which impact on the quality of care? Which recommendations?Therapies74(1): 155–164.
DzindoletMTPetersonSAPomrankyRA, et al. (2003) The role of trust in automation reliance. International Journal of Human-Computer Studies58(6): 697–718.
31.
EisenhardtKMBourgeoisLJ (1988) Politics of strategic decision making in high-velocity environments: Toward a midrange theory. Academy of Management Journal31(4): 737–770.
ElonenSArttoKA (2003) Problems in managing internal development projects in multi-project environments. International Journal of Project Management21(6): 395–402.
34.
FanXOhSMcNeeseM,et al. (2008). The influence of agent reliability on trust in human-agent collaboration. In: Proceedings of the 15th European conference on cognitive ergonomics the ergonomics of cool interaction – ECCE ‘08, pp.1. https://doi.org/10.1145/1473018.1473028
35.
FioriS (2011) Forms of bounded rationality: The reception and redefinition of Herbert A. Simon’s perspective. Review of Political Economy23(4): 587–612.
36.
FreedyADeVisserEWeltmanG, et al. (2007) Measurement of trust in human-robot collaboration. 2007 International Symposium on Collaborative Technologies and Systems: 106–114. https://doi.org/10.1109/CTS.2007.4621745.
37.
FulmerCAGelfandMJ (2012) At what level (and in whom) we trust: Trust across multiple organizational levels.
38.
GefenDKarahannaEStraubDW (2003) Trust and TAM in online shopping: An integrated model. MIS Quarterly27(1): 51–90.
39.
GuptaSKarAKBaabdullahA, et al. (2018) Big data with cognitive computing: A review for the future. International Journal of Information Management42: 78–89.
40.
HaenleinMKaplanA (2019) A brief history of artificial intelligence: On the past, present, and future of artificial intelligence. California Management Review61(4): 5–14.
41.
HenriksenADTraynorAJ (1999) A practical R&D project-selection scoring tool. IEEE Transactions on Engineering Management46(2): 158–170.
42.
HoffmanRRJohnsonMBradshawJM, et al. (2013) Trust in automation. IEEE Intelligent Systems28(1): 84–88.
43.
HollisVPekurovskyAWuE, et al. (2018) On being told how we feel: How algorithmic sensor feedback influences emotion perception. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies2(3): 1–31.
44.
HolzingerABiemannCPattichisCS, et al. (2017) What do we need to build explainable AI systems for the medical domain? (arXiv:1712.09923). arXiv. http://arxiv.org/abs/1712.09923.
45.
JhaSTopolEJ (2016) Adapting to artificial intelligence: Radiologists and pathologists as information specialists. JAMA316(22): 2353–2354.
46.
JordanMIMitchellTM (2015) Machine learning: Trends, perspectives, and prospects. Science349(6245): 255–260.
47.
KahnemanD (2003) Maps of bounded rationality: Psychology for behavioral economics. American Economic Review93(5): 1449–1475.
48.
KaracaOÇalışkanSADemirK (2021) Medical artificial intelligence readiness scale for medical students (MAIRS-MS) – development, validity and reliability study. BMC Medical Education21(1): 12.
49.
KerrIRBornfreundM (2005) Buddy bots: How Turing’s fast friends are undermining consumer privacy. Presence: Teleoperators and Virtual Environments14(6): 647–655.
50.
KimKBoellingLHaeslerS,et al. (2018). Does a digital assistant need a body? The influence of visual embodiment and social behavior on the perception of intelligent virtual agents in AR. In: 2018 IEEE international symposium on mixed and augmented reality (ISMAR), pp.105–114. https://doi.org/10.1109/ISMAR.2018.00039
51.
KnightFH (1921) Risk, Uncertainty and Profit. Boston and New York: Houghton Mifflin Co., The Riverside Press.
52.
KrollJA (2018) The fallacy of inscrutability. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences376(2133): 20180084.
53.
LangerEBlankAChanowitzB (1978) The mindlessness of ostensibly thoughtful action: The role of “placebic” information in interpersonal interaction. 8.
54.
LawrenceT (1991) Impacts of artificial intelligence on organizational decision making. Journal of Behavioral Decision Making4(3): 195–214.
55.
LeCunYBengioYHintonG (2015) Deep learning. Nature521(7553): 436–444.
56.
LeeJDSeeKA (2004) Trust in automation: Designing for appropriate reliance. Human Factors46(1): 50–80.
57.
LeeJWKimSH (2001) An integrated approach for interdependent information system project selection. International Journal of Project Management19(2): 111–118.
58.
LeeSM (1972) Goal Programming for Decision Analysis. Philadelphia: Auerbach Publishers.
59.
LiangCLiQ (2008) Enterprise information system project selection with regard to BOCR. International Journal of Project Management26(8): 810–820.
60.
LockeySGillespieNHolmD,et al. (2021). A review of trust in artificial intelligence: Challenges, vulnerabilities and future directions. In: Hawaii international conference on system sciences. https://doi.org/10.24251/HICSS.2021.664
61.
LucasHMooreJ (1976) A multiple-criterion scoring approach to information system project selection. INFOR: Information Systems and Operational Research14(1): 1–12.
62.
MahroofK (2019) A human-centric perspective exploring the readiness towards smart warehousing: The case of a large retail distribution warehouse. International Journal of Information Management45: 176–190.
63.
McCarthyJ (2004) What is artificial intelligence? 14.
64.
McknightDHCarterMThatcherJB, et al. (2011) Trust in a specific technology: An investigation of its components and measures. ACM Transactions on Management Information Systems2(2): 1–25.
65.
MeskendahlS (2010) The influence of business strategy on project portfolio management and its success—A conceptual framework. International Journal of Project Management28(8): 807–817.
66.
MilkmanKLChughDBazermanMH (2009) How can decision making be improved?Perspectives on Psychological Science4(4): 379–383.
67.
MoranSPantidiNBachourK, et al. (2013) Team reactions to voiced agent instructions in a pervasive game. In: Proceedings of the 2013 International conference on intelligent user interfaces - IUI ‘13, pp.371–382. https://doi.org/10.1145/2449396.2449445
OkeS (2004) Artificial intelligence: A review of the literature. In: Proceedings of the Fifth Asia Pacific industrial engineering and management systems conference.
70.
OztaysiB (2014) A decision model for information technology selection using AHP integrated TOPSIS-Grey: The case of content management systems. Knowledge-Based Systems70: 44–54.
71.
PanagiotopoulosIDimitrakopoulosG (2018) An empirical investigation on consumers’ intentions towards autonomous driving. Transportation Research Part C: Emerging Technologies95: 773–784.
72.
ParasuramanRMolloyRSinghIL (1993) Performance consequences of automation-induced “complacency.”. The International Journal of Aviation Psychology3(1): 1–23.
73.
ParasuramanRRileyV (1997) Humans and automation: Use, misuse, disuse, abuse. Human Factors: The Journal of the Human Factors and Ergonomics Society39(2): 230–253.
74.
PedersenK (2016) IT Project selection: Politics, experience and good friends. Electronic Journal of Information Systems Evaluation19(1): 55–70.
75.
PfefferJ (1981) Power in Organizations. Marshfield, MA: Pitman Publishing.
76.
PondyLR (1966) A systems theory of organizational conflict. Academy of Management Journal9(3): 246–256.
77.
RaiAConstantinidesPSarkerS (2019) Next-Generation digital platforms: Toward human–AI hybrids. MIS Quarterly43(1): iii–ix.
78.
RaischSKrakowskiS (2020) Artificial intelligence and management: the automation-augmentation paradox. 48.
79.
RealJCLealARoldánJL (2006) Information technology as a determinant of organizational learning and technological distinctive competencies. Industrial Marketing Management35(4): 505–521.
80.
Rinta-KahilaTPenttinenESalovaaraA, et al. (2018) Consequences of discontinuing knowledge work automation—surfacing of deskilling effects and methods of recovery. In: Hawaii international conference on system sciences. https://doi.org/10.24251/HICSS.2018.654.
81.
RousseauDM (2018) Making evidence-based organizational decisions in an uncertain world. Organizational Dynamics47(3): 135–146.
82.
SalemMLakatosGAmirabdollahianF, et al. (2015) Would you trust a (faulty) robot?: Effects of error, task type and personality on human-robot cooperation and trust. In: Proceedings of the tenth annual ACM/IEEE international conference on human-robot interaction, pp. 141–148. https://doi.org/10.1145/2696454.2696497.
83.
SallesAEversKFariscoM (2020) Anthropomorphism in AI. AJOB Neuroscience11(2): 88–95.
84.
SanthanamRKyparisisGJ (1996) A decision model for interdependent information system project selection. European Journal of Operational Research89(2): 380–399.
85.
Sanz de Acedo LizarragaMLSanz de Acedo BaquedanoMTSoria OliverM, et al. (2009) Development and validation of a decision-making questionnaire. British Journal of Guidance & Counselling37(3): 357–373.
86.
SchepmanARodwayP (2022) The general attitudes towards artificial intelligence scale (GAAIS): Confirmatory validation and associations with personality, corporate distrust, and general trust. International Journal of Human–Computer Interaction39: 1–18.
87.
ShresthaYRHeVFPuranamP (2021a) Algorithm supported induction for building theory: How can we use prediction models to theorize?Organization Science32(3): 856–880.
88.
ShresthaYRKrishnaVvon KroghG (2021b) Augmenting organizational decision-making with deep learning algorithms: Principles, promises, and challenges. Journal of Business Research123: 588–603.
89.
SiauKWangW (2018) Building trust in artificial intelligence, machine learning, and robotics. Cutter Business Technology Journal31(2): 8.
90.
SuttonSGArnoldVHoltM (2018) How much automation is too much? Keeping the human relevant in knowledge work. Journal of Emerging Technologies in Accounting15(2): 15–25.
91.
SwansonKMcCombDSmithJ, et al. (1991) The application software factory: Applying total quality techniques to systems development. MIS Quarterly15(4): 567–579.
92.
ThompsonJD (1967) Organizations in Action: Social Science Bases of Administrative Theory. University of Illinois at Urbana-Champaign's Academy for Entrepreneurial Leadership Historical Research Reference in Entrepreneurship: McGraw-Hill.
93.
TinglingPParentM (2004) An exploration of enterprise technology selection and evaluation. The Journal of Strategic Information Systems13(4): 329–354.
94.
TopolEJ (2019) High-performance medicine: The convergence of human and artificial intelligence. Nature Medicine25(1): 44–56.
95.
TrunkABirkelHHartmannE (2020) On the current state of combining human and artificial intelligence for strategic organizational decision making. Business Research13(3): 875–919.
96.
von KroghG (2018) Artificial intelligence in organizations: New opportunities for phenomenon-based theorizing. Academy of Management Discoveries4(4): 404–409.
97.
WagnerARBorensteinJHowardA (2018) Overtrust in the robotic age. Communications of the ACM61(9): 22–24.
98.
WallachH (2018) Computational social science ≠ computer science + social data. Communications of the ACM61(3): 42–44.
99.
WaytzAHeafnerJEpleyN (2014) The mind in the machine: Anthropomorphism increases trust in an autonomous vehicle. Journal of Experimental Social Psychology52: 113–117.
100.
WeinerBJ (2009) A theory of organizational readiness for change. Implementation Science4(1): 67.
101.
WeinerBJAmickHLeeS-YD (2008) Review: Conceptualization and measurement of organizational readiness for change: A review of the literature in health services research and other fields. Medical Care Research and Review65(4): 379–436.
102.
WellerA (2017) Challenges for Transparency. 8.
103.
ZadehLA (1965) Fuzzy sets. Information Control8: 338–353.
104.
ZhangGRainaACaganJ, et al. (2021) A cautionary tale about the impact of AI on human design teams. Design Studies72: 100990.
105.
ZhangYLiaoQVBellamyRKE (2020) Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp.295–305. https://doi.org/10.1145/3351095.3372852.
