Relationships Between Quantitative Measures of Evaluation Plan and Program Model Quality and a Qualitative Measure of Participant Perceptions of an Evaluation Capacity Building Approach

Abstract

Despite a heightened emphasis on building evaluation capacity and evaluation quality, there is a lack of tools available to identify high-quality evaluation. In the context of testing the Systems Evaluation Protocol (SEP), quantitative rubrics were designed and tested to assess the quality of evaluation plans and models. Interview data were also collected and analyzed using a priori codes. A mixed methods approach was used to synthesize quantitative and qualitative data and explore trends. Consistencies between data types were found for attitude and capacity, and disconnects were found for knowledge, cyberinfrastructure, time, and quality. This approach to data integration represents a novel way to tap the generative potential of divergence that arises when different methods produce contradictory results.