One of the most widely used communication tools in evaluation is the logic model. Despite its extensive use, there has been little research into the visualization aspect of the logic model. To assess the impact that design modifications would have on its effectiveness, we applied established visualization principles to revise a program model. Participants were randomly assigned to one of the six conditions to examine the effectiveness (i.e., visual efficiency, comprised of accuracy, response time, and mental effort; credibility; aesthetics) of variations to a logic model. The results demonstrated that the revisions to the model increased accuracy, perceived message credibility, and were considered more aesthetically pleasing; furthermore, revisions decreased mental effort and reduced the amount of time taken to review the model. Together, the findings from the study support the claim that visual efficiency can be improved by modifying a logic model’s formatting and design.
AzzamT. (2010). Evaluator responsiveness to stakeholders. American Journal of Evaluation, 31, 45–65.
2.
AzzamT.EvergreenS. (Eds.). (2013a). Data visualization, part 1: New directions for evaluation, number 139. San Francisco, CA: Jossey-Bass.
3.
AzzamT.EvergreenS. (Eds.). (2013b). Data visualization, part 2: New directions for evaluation, number 140. San Francisco, CA: Jossey-Bass.
4.
AzzamT.EvergreenS.GermuthA.A.KistlerS.J. (2013). Data visualization and evaluation. New Directions for Evaluation, 2013, 7–32. doi:10.1002/ev.20065
5.
BehrendT. S.SharekD. J.MeadeA. W.WiebeE. N. (2011). The viability of crowdsourcing for survey research. Behavior Research Methods, 43, 800.
6.
BertinJ. (1983). Semiology of graphics: Diagrams, networks, maps (BergW. J., Trans. ). Madison, WI: The University of Wisconsin Press.
7.
Bourdieu. (2007). Are program evaluators judges and/or knowledge brokers?Journal of Epidemiology and Community Health, 61, 1020. doi:10.1136/jech.2007.065573
8.
BrighamT. J. (2016). Feast for the eyes: An introduction to data visualization. Medical Reference Services Quarterly, 35, 215–223.
9.
BrownR. D.BraskampL. A.NewmanD. L. (1978). Evaluator credibility as a function of report style: Do jargon and data make a difference?Evaluation Quarterly, 2, 331–341.
10.
BuhrmesterM.KwangT.GoslingS. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data?Perspectives on Psychological Science, 6, 3–5.
11.
CaslerK.BickelL.HackettE. (2013). Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29, 2156–2160.
12.
CawthonN.MoereA. V. (2007). The effect of aesthetic on the usability of data visualization. Paper presented at the IEEE Conference on Information Visualization (IV ‘07 (pp. 637–648). Sacramento, CA.
ChenH. T. (2014). Practical program evaluation. Thousand Oaks, CA: Sage.
15.
ClevelandW. S. (1993). Visualizing data. New Jersey, NJ: Hobart Press.
16.
CooperE. A.FaridH. (2016). Does the Sun revolve around the Earth? A comparison between the general public and online survey respondents in basic scientific knowledge. Public Understanding of Science, 25, 146–153.
17.
CreswellJ. W.ClarkV. L. P. (2017). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
18.
DonaldsonS. I. (2001). Overcoming our negative reputation: Evaluation becomes known as a helping profession. American Journal of Evaluation, 22, 355–361.
19.
EvergreenS.MetznerC. (2013). Design principles for data visualization in evaluation. New Directions for Evaluation, 140, 5–20.
20.
EvergreenS. D. (2014). Presenting data effectively: Communicating your findings for maximum impact. Thousand Oaks, CA: Sage.
FewS.EdgeP. (2008). What ordinary people need most from information visualization today. Perceptual Edge: Visual Business Intelligence Newsletter, 1–7.
24.
FlanaginA. J.MetzgerM. J. (2003). The perceived credibility of personal web page information as influenced by the sex of the source. Computers in Human Behavior, 19, 683–701.
25.
FreedmanE. G.ShahP. (2002). Toward a model of knowledge-based graph comprehension. In HegartyM.MeyerB.NarayananN. H. (Eds.), Diagrammatic representation and inference (pp. 18–30). Berlin, Heidelberg: Springer-Verlag.
26.
FriendlyM. (2008). A brief history of data visualization. In ChenC.HardleW.UnwinA. (Eds.), Handbook of computational statistics: Data visualization (Vol. III). Berlin, Heidelberg: Springer-Verlag.
27.
FunnellS. C.RogersP. J. (2011). Purposeful program theory: Effective use of theories of change and logic models (Vol. 31). New Jersey, NJ: John Wiley.
28.
GreeneJ. C.MillettR. A.HopsonR. K. (2004). Evaluation as a democratizing practice. In BravemanE. T.ConstantineN. A.SlaterJ. K. (Eds.), Foundations and evaluations: Context and practices for effective philanthropy. San Francisco, CA: Jossey-Bass.
29.
GugiuP. C.Rodriguez-CamposL. (2007). Semi-structured interview protocol for constructing logic models. Evaluation and Program Planning, 30, 339–350.
30.
HeerJ.BostockM.OgievetskyV. (2010). A tour through the visualization zoo. Queue, 8, 20.
31.
HegartyM. (2011). The cognitive science of visual-spatial displays: Implications for design. Topics in Cognitive Science, 3, 446–474.
32.
HenryG. T. (1997). Creating effective graphs: Solutions for a variety of evaluation data. New Directions for Evaluation, 73, 1–109.
33.
HuangW.EadesP.HongS. H. (2009). Measuring effectiveness of graph visualizations: A cognitive load perspective. Information Visualization, 8, 139–152.
34.
HubermanA. M.MilesM. B. (1994). Data management and analysis methods. Thousand Oaks, CA: Sage.
35.
JulianD. A.JonesA.DeyoD. (1995). Open systems evaluation and the logic model: Program planning and evaluation tools. Evaluation and Program Planning, 18, 333–341.
36.
KaplanS. A.GarrettK. E. (2005). The use of logic models by community based initiatives. Evaluation and Program Planning, 28, 167–172. doi:10.1016/j.evalprogplan.200.09.002
37.
KnowltonL. W.PhillipsC. C. (2012). The logic model guidebook: Better strategies for great results. Thousand Oaks, CA: Sage.
38.
KurosuM.KashimuraK. (1995, May). Apparent usability vs. inherent usability: Experimental analysis on the determinants of the apparent usability. Conference companion on human factors in computing systems (pp. 292–293). Denver, CO: ACM.
McLaughlinJ. A.JordanG. B. (1999). Logic models: A tool for telling your program’s performance story. Evaluation and Program Planning, 22, 65–72.
41.
MillarA.SimeoneR. S.CarnevaleJ. T. (2001). Logic models: A system tool for performance management. Evaluation and Program Planning, 24(1), 73–81.
42.
MoereA. V.PurchaseH. (2011). On the role of design in information visualization. Information Visualization, 10, 356–371.
43.
MoereA. V.TomitschM.WimmerC.ChristophB.GrechenigT. (2012). Evaluating the effect of style in information visualization. IEEE Transactions on Visualization and Computer Graphics, 18, 2739–2748.
44.
NormanD. (2002). Emotion & design: Attractive things work better. Interactions, 9, 36–42.
45.
PaasF. G. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84, 429.
46.
PaolacciG.ChandlerJ. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23, 184–188.
47.
PattonM. Q. (1999). Enhancing the quality and credibility of qualitative analysis. Health Services Research, 34, 1189.
48.
RengerR.TitcombA. (2002). A three-step approach to teaching logic models. The American Journal of Evaluation, 23, 493–503.
49.
RushB.OgborneA. (1991). Program logic models: Expanding their role and structure for program planning and evaluation. The Canadian Journal of Program Evaluation, 6, 95.
50.
SalimunC.PurchaseH. C.SimmonsD. R.BrewsterS. (2010, October). The effect of aesthetically pleasing composition on visual search performance. Proceedings of the Sixth Nordic Conference on Human-Computer Interaction: Extending Boundaries (pp. 422–431). Reykjavik, Iceland: ACM.
51.
TractinskyN.KatzA. S.IkarD. (2000). What is beautiful is usable. Interacting With Computers, 13, 127–145.
52.
TufteE. (1997). Visual explanations: Images and quantities, evidence and narrative. Cheshire, CT: Graphics Press.
53.
TufteE. (2001). The visual display of quantitative information (2nd ed.). Cheshire, CT: Graphics Press.
W.K. Kellogg Foundation. (2004). Using logic models to bring together planning, evaluation, and action: Logic model development guide. Battle Creek, MI: Author.
56.
WolcottH. F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oaks, CA: Sage.
57.
ZhuY. (2007, November). Measuring effective data visualization. International Symposium on Visual Computing (pp. 652–661). Berlin, Heidelberg: Springer.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
