Multiple Criteria to Evaluate Decision Support Outcomes

Abstract

The Integrative Decision Space (I-DecS) is a decision support tool designed to improve multinational mission planning through the evaluation of the impact of cultural influences on decision outcomes. A usability study was conducted to evaluate the prototype tool and to collect empirical data on teams using I-DecS in a laboratory setting. A key aspect of this effort was to identify a methodology to evaluate the benefit of the decision support provided, i.e., differences in mission plans produced with and without I-DecS. Multiple criteria were identified as important metrics of plan quality: mission specific criteria, simulated performance parameters, and command level course of action evaluation. This resulted in a comprehensive method to evaluate the mission plans and assess the effectiveness of the decision support tool. After the completion of the usability assessment with naïve users, I-DecS will be subjected to a more rigorous performance evaluation with experienced users.