Evaluating,Comparing,and Best Practice in Electronic Portfolio System Use

Abstract

Electronic portfolios (e-portfolios) are commonly positioned to show evidence of student learning with formative and summative assessment benefits. At the University of Auckland teacher education program, two e-portfolio systems were adopted to document preservice teacher's course work and to attest to the Graduating Teacher Standards. This article seeks to (a) evaluate e-portfolio systems, (b) compare the MyPortfolio (Mahara) and Google Sites systems, and (c) provide recommendations for best practice in system use. The software system comparison between the MyPortfolio (Mahara) and Google Sites system was based on the essential technology features for system use. User satisfaction and usability data were obtained through self-reported surveys (N = 192). Open-ended questions (N = 192) and semistructured interviews with a sample of course participants (N = 12) ascertained the best practice approach for system use. Mean scores were higher for the MyPortfolio (Mahara) system. One-way analysis of variance indicated that the differences were statistically significant.

Keywords

electronic portfolio system higher education teacher education e-portfolio system comparison educational technology evaluation

Get full access to this article

View all access options for this article.

References

Abrami, P., & Barrett, H. (2005). Directions for research and development on electronic portfolios. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 31(3). Retrieved from http://www.cjlt.ca/index.php/cjlt/article/view/92/86%26num%3B1.

Baeten

Dochy

Struyven

(2008) Students’ approaches to learning and assessment preferences in a portfolio-based learning environment. Instructional Science 36(5–6): 359–374. doi:10.1007/s11251-008-9060-y.

Balaban, I., & Bubas, G. (2010). Educational potentials of eportfolio systems: Student evaluations of Mahara and Elgg. Proceedings of the ITI 2010 32nd Int. Conf. on Information Technology Interfaces, 329–336. Retrieved from http://hnk.ffzg.hr/bibl/iti2010/pdf/08%20Technology%20Enhanced%20Learning/08-01-163.pdf.

Barrett, H. (2011). Balancing the two faces of e-portfolios. British Columbia Ministry of Education. Innovations in Education (2nd Ed.). Retrieved from http://electronicportfolios.org/balance/balancingarticle2.pdf.

Bartlett

(2009) A five-step model for enhancing electronic teaching portfolios. In: Adamy

Milman

N. B.

(eds) Evaluating electronic portfolios in teacher education, Charlotte, NC: Information Age Publishing, INC, pp. 49–62.

Beck

R. J.

Livne

N. L.

Bear

S. L.

(2005) Teachers’ self-assessment of the effects of formative and summative electronic portfolios on professional development. European Journal of Teacher Education 28(3): 221–244. doi:10.1.1.598.9042.

Brown

G. T. L.

(2004) Measuring attitude with positively packed self-report ratings: Comparison of agreement and frequency scales. Psychological Reports 94(3): 1015–1024. doi:10.2466/pr0.94.3.1015-1024.

Buzzetto-More

Alade

(2008) The pentagonal e-portfolio model for selecting, adopting, building, and implementing an e-portfolio. Journal of Information Technology Education 7: 184–208.

Buzzetto-More

N. A.

Sweat-Guy

(2007) A comparative analysis of common e-portfolio features and available platforms. Issues in Informing Science & Technology 4: 327.

10.

Challis, D. (2005). Towards the mature e-portfolio: Some implications for higher education. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 31(3). Retrieved from http://www.cjlt.ca/index.php/cjlt/article/view/93/87.

11.

Chau, J. (2007). A developer’s challenges on an e-portfolio journey. Proceedings ascilite Singapore 2007, Concise paper, 145–148.

12.

Chau

Cheng

(2010) Towards understanding the potential of e-portfolios for independent learning: A qualitative study. Australasian Journal of Educational Technology 26(7): 932–950. Retrieved from http://www.ascilite.org.au/ajet/ajet26/chau.html.

13.

Cohen

(1992) A power primer. Psychological Bulletin 112(1): 155–159.

14.

Cuban

Kirkpatrick

Peck

(2001) High access and low use of technologies in high school classrooms: Explaining an apparent paradox. American Educational Research Journal 38(4): 813–834.

15.

Deneen

Brown

G. T.

Bond

T. G.

Shroff

(2013) Understanding outcome-based education changes in teacher education: Evaluation of a new instrument with preliminary findings. Asia-Pacific Journal of Teacher Education 41(4): 441–456.

16.

Dzida

(1996) International usability standards. ACM Computing Surveys (CSUR) 28(1): 173–175.

17.

Edwards, R. (2015). E-portfolio environment design principles in practice: A case study of a collaborative project in technology teacher education. Australasian Journal of Technology Education, 2(1). Retrieved from http://www.ajte.org/index.php/AJTE/article/viewFile/32/16.

18.

Gaide

(2006) ePortfolio: Supercharge performance-based student assessment. Distance Education Report 10(2): 14–16.

19.

Hamp-Lyons

Condon

(2000) Assessing the portfolio: Principles for practice, theory, and research, Cresskill, NJ: Hampton Press.

20.

Hartnell-Young

Morriss

(2006) Digital portfolios: Powerful tools for promoting professional growth and reflection, Thousand Oaks, CA: Corwin Press.

21.

Hattie

J. A.

Brown

G. T. L.

Ward

Irving

S. E.

Keegan

P. J.

(2006) Formative Evaluation of an educational assessment technology innovation: Developers’ insights into assessment tools for teaching and learning (asTTle). Journal of MultiDisciplinary Evaluation 5: 1–54.

22.

Havelock

(2009) Too new a tale to tell? Issues in evaluating e-portfolio systems and implementations. In: Adamy

Milman

N. B.

(eds) Evaluating electronic portfolios in teacher education, Charlotte, NC: Information Age Publishing, INC, pp. 63–90.

23.

Hecht

M. L.

(2002) A research odyssey: Toward the development of communication theory of identity. Communication Monographs 60(1): 76–82. doi:10.1080/03637759309376297.

24.

Imhof

Picard

(2009) Views on using portfolio in teacher education. Teaching and Teacher Education 25(1): 149–154. doi:10.1016/j.tate.2008.08.001.

25.

Ives

Olson

Baroudi

J. J.

(1983) The measurement of user information satisfaction. Communications of the AMC 26(10): 785–793.

26.

Jafari

(2004) The “Sticky” ePortfolio system. Tackling challenges & identifying attributes. Educause Review 39(4): 38–49.

27.

Jamieson

(2004) Likert scales: How to (ab)use them. Medical Education 38(12): 1217–1218. doi:10.1111/j.1365-2929.2004.02012.x.

28.

Lam

T. C. M.

Klockars

A. J.

(1982) Anchor point effects on the equivalence of questionnaire items. Journal of Educational Measurement 19(4): 317–322.

29.

Lambert

Corrin

(2007) Moving towards a university wide implementation of an ePortfolio tool. Australasian Journal of Educational Technology 23(1): 1–16.

30.

Laxman

(2010) A conceptual framework mapping the application of information search strategies to well and ill structured problem solving. Computers & Education 55: 513–526. doi:10.1016/j.compedu. 2010.02.014.

31.

Lee, M., & Pohio, L. (2012 November). An e-portfolio theoretical approach for Provisionally Registered Teachers (pp. 550–559). Presented at the ascilite 2012, Wellington, New Zealand. Retrieved from http://www.ascilite.org/conferences/Wellington12/2012/images/custom/asclite2012_proceedings.pdf.

32.

Lorenzo

Ittelson

(2005) An overview of e-portfolios. Educause Learning Initiative 1: 1–27. Retrieved from http://ctl.tedu.edu.tr/sites/default/files/content_files/eportfolioeducausedocument.pdf.

33.

Love

McKean

Gathercoal

(2004) Portfolios to webportfolios and beyond: Levels of maturation. EDUCAUSE Quarterly 2: 24–37. Retrieved from http://net.educause.edu/ir/library/pdf/EQM0423.pdf.

34.

Mishra

Koehler

(2006) Technological pedagogical content knowledge: A framework for teacher knowledge. The Teachers College Record 108(6): 1017–1054.

35.

New Zealand Teachers Council (2010) Registered teacher criteria, Wellington, New Zealand: New Zealand Teachers Council.

36.

Norman

(2010) Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Science Education 15(5): 625–632. doi:10.1007/s10459-010-9222-y.

37.

Oliver

Whelan

(2011) Designing an e-portfolio for assurance of learning focusing on adoptability and learning analytics. Australasian Journal of Education Technology 27(6): 1026–1041.

38.

Reaves

C. C.

(1992) Quantitative research for the behavioral sciences, New York, NY: John Wiley & Sons, Inc.

39.

Redmond

Devine

Basson

(2014) Exploring discipline differentiation in online discussion participation. Australasian Journal of Educational Technology 30(2): 122–135. Retrieved from http://eprints.usq.edu.au/24224/2/Redmond_Devine_basson_AJET_2014_PV.pdf.

40.

San Jose, D. L. (2014). Does technology matter? Students’ and teachers’ experiences of electronic portfolio (e portfolio) systems in teacher education. In B. Hegarty, J. McDonald, & S.-K. Loke (Eds.), Rhetoric and reality: Critical perspectives on educational technology. Proceedings ascilite Dunedin 2014 (pp. 584–588). Retrieved from http://www.ascilite.org/conferences/dunedin2014/files/concisepapers/194-San-Jose.pdf.

41.

San Jose

D. L.

Kelleher

(2009) Measuring ecoshock and affective learning: A comparison of student responses to online and face-to-face learning ecologies. MERLOT Journal of Online Learning and Teaching 5(3): 469–476.

42.

Shroff

Deneen

C. C.

Lim

C. P.

(2014) Student ownership of learning using e-portfolio or career development. Journal of Information Systems Technology & Planning 7(18): 75–90.

43.

Shroff

R. H.

Deneen

C. C.

E. M. W.

(2011) Analysis of the technology acceptance model in examining students’ behavioural intention to use an e-portfolio system. Australasian Journal of Educational Technology 27(4): 600–618.

44.

Swan

(2009) Examining barriers in faculty adoption of an e-portfolio system. Australasian Journal of Educational Technology 25(5): 627–644.

45.

Teoh

K.-K.

(2011) An examination of critical success factors in the implementation of ePortfolios in universities. Journal of Academic Language & Learning 5(2): A60–A72.

46.

Vlachopoulos, P., & Wheeler, A. (2013). Joining the dots: Using structured e-portfolio assignments to enhanced reflection. 30th ascilite Conference 2013 Proceedings, 879–882.

47.

Zaharias

Poylymenakou

(2009) Developing a usability evaluation method for e-learning applications: Beyond functional usability. International Journal of Human-Computer Interaction 25(1): 75–98. doi:10.1080/10447310802546716.