GUI testing is essential to provide validity and quality of system response, but applying it to a development is not straightforward: it is time consuming, requires specialized personnel, and involves complex activities that sometimes are implemented manually. GUI testing tools help supporting these processes. However, integrating them into software projects may be troublesome, mainly due to the diversity of GUI platforms and operating systems in use. This work presents the design and implementation of Open HMI Tester (OHT), an application framework for the automation of testing processes based on GUI introspection. It is cross-platform, and provides an adaptable design aimed at supporting major event-based GUI platforms. It can also be integrated into ongoing and legacy developments using dynamic library preloading. OHT provides a robust and extensible basis to implement GUI testing tools. A capture and replay approach has been implemented as proof of concept. Introspection is used to capture essential GUI and interaction data. It is used also to simulate real human interaction in order to increase robustness and tolerance to changes between testing iterations. OHT is being actively developed by the Open-source Community and, as shown in this paper, it is ready to be used in current software projects.
Testona: Test design using the classification tree method. http://www.testona.net/ [Online; accessed July 2016].
3.
AhoP., MenzN., RätyT. and SchieferdeckerI., Automated Java GUI Modeling for Model-Based Testing Purposes, In ITNG, IEEE Computer Society, 2011, pp. 268–273.
4.
AmalfitanoD., FasolinoA.R., TramontanaP., CarmineS.D. and MemonA.M., Using gui ripping for automated testing of android applications. In GoedickeM., MenziesT.and SaekiM., editors, ASE, 2012, pp. 258–261. ACM.
5.
CaswellM., AravamudhanV. and WilsonK., Introduction to jfcUnit. Retrieved August, 4, 2004.
6.
ChangT.-H., YehT. and MillerR.C., GUI Testing Using Computer Vision. In MynattE.D., SchonerD., FitzpatrickG., HudsonS.E., EdwardsW. Keith and RoddenT., editors, CHI, ACM, 2010, pp. 1535–1544.
7.
ChoiW., NeculaG.C. and SenK., Guided gui testing of android apps with minimal restart and approximate learning. In
HoskingA.L., EugsterP.Th. and LopesC.V., editors, OOPSLA, ACM, 2013, pp. 623–640.
8.
ArissO., XuD., DandeyS., VenderB., McCleanP.E. and SlatorB.M., A Systematic Capture and Replay Strategy for Testing Complex GUI Based Java Applications. In LatifiS., editor, ITNG, IEEE Computer Society, 2010, pp. 1038–1043.
9.
GrechanikM., XieQ. and FuC., Maintaining and evolving GUI-directed test scripts. In ICSE, IEEE, 2009, pp. 408–418.
10.
GrossF., FraserG. and ZellerA., Search-based system testing: High coverage, no false alarms. In HeimdahlM.P.E. and SuZhendong, editors, ISSTA, ACM, 2012, pp. 67–77.
11.
HorngS.-J., SuM.-Y. and TsaiJ.-G., A dynamic backdoor detection system based on dynamic link libraries, International Journal of Business and Systems Research2(3) (2008), 244–257.
12.
HustedT. and MassolV., JUnit in Action, Manning Publications, 2004.
13.
KaramM.R., DascaluS.M. and HaziméR.H., Challenges and opportunities for improving code-based testing of graphical user interfaces, Journal of Computational Methods in Sciences and Engineering6(5-6) (2006), 379–388.
KruseP.M., NasarekJ. and FernandezN.C., Systematic testing of web applications with the classification tree method. In XVII Iberoamerican Conference on Software Engineering (CIbSE 2014), 2014.
16.
LeonD., PodgurskiA. and WhiteL.J., Multivariate visualization in observation-based testing. In GhezziC., JazayeriM. and WolfA.L., editors, ICSE, ACM, 2000, pp. 116–125.
17.
LiuC.-H., LuC.-Y., ChengS.-J., ChangK.-Y., HsiaoY.-C. and ChuW.-M., Capturereplay testing for android applications. In Computer, Consumer and Control (IS3C), 2014 International Symposium on, 2014, pp. 1129–1132.
18.
NavarroP.L.M., HillmannS., MöllerS., RuizD.S. and PérezG.M., Run-time model based framework for automatic evaluation of multimodal interfaces, Journal on Multimodal User Interfaces8(4) (2014), 399–427.
19.
NavarroP.L.M., PérezG.M. and RuizD.S., A context-aware interaction model for the analysis of users QoE in mobile environments, International Journal of Human-Computer Interaction30(12) (2014), 946–964.
20.
NavarroP.L.M., RuizD.S. and PérezG.M.,A Proposal for Automatic Testing of GUIs Based on Annotated Use Cases. Adv. Software Engineering, Special Issue on Software Test Automation, 2010.
21.
NavarroP.L.M., RuizD.S. and PérezG.M., Open HMI tester: An open and cross-platform architecture for GUI testing and certification, International Journal of Computer Systems Science and Engineering (IJCSSE), Special Issue on Open Source Certification25(4) (2010), 283–296.
22.
NavarroP.L.M., RuizD.S. and PérezG.M., A lightweight framework for dynamic gui data verification based on scripts, Softw Test, Verif Reliab26(2) (2016), 95–118.
23.
MemonA., GUI testing: Pitfalls and process, IEEE Computer35(8) (2002), 87–88.
24.
MemonA., An event-flow model of GUI-based applications for testing, Software Testing Verification and Reliability17(3) (2007), 137–157.
25.
MemonA., BanerjeeI. and NagarajanA., GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing. In DeursenA.V., StrouliaE. and StoreyM.-A.D., editors, WCRE, IEEE Computer Society, 2003, pp. 260–269.
26.
MemonA.M., BanerjeeI., HashmiN. and NagarajanA., DART: A Framework for Regression Testing “Nightly/daily Builds” of GUI Applications. In ICSM, IEEE Computer Society, 2003, pp. 410–419.
27.
MemonA.M., SoffaM.L. and PollackM.E., Coverage criteria for GUI testing. In ESEC/SIGSOFT FSE, 2001, pp. 256–267.
28.
MingsongC., XiaokangQ. and XuandongL., Automatic test case generation for UML activity diagrams JAVA. In AST ’06: Proceedings of the 2006 International Workshop on Automation of Software Test, New York, NY, USA, ACM, 2006, pp. 2–8.
29.
Nokia Corporation. Qt: Cross-platform application and UI framework, 2012.
30.
OrsoA. and KennedyB., Selective capture and replay of program executions, ACM SIGSOFT Software Engineering Notes30(4) (2005), 1–7.
31.
PaivaA.C.R., FariaJ.C.P. and VidalR.F.A.M., Towards the integration of visual and formal models for GUI testing, Electr Notes Theor Comput Sci190(2) (2007), 99–111.
32.
Renas Reda. Robotium: User scenario testing for Android. http://www.robotium.org/ [Online; accessed July 2016].
33.
StevenJ., ChandraP., FleckB. and PodgurskiA., jrapture: A capture/replay tool for observation-based testing, In ISSTA, 2000, pp. 158–167.
34.
The GTK+ Team. The GIMP Toolkit (GTK), version 2.x. http://www.gtk.org [Online; accessed July 2016].
35.
VieiraM., LeducJ., HaslingB., SubramanyanR. and KazmeierJ., Automation of GUI Testing Using a Model-driven Approach. In
ZhuH., HorganJ.R., CheungS.-C. and LiJ.J., editors, AST, ACM, 2006, pp. 9–14.
36.
WhiteL. and AlmezenH., Generating test cases for GUI responsibilities using complete interaction sequences, IEEE Transactions on SMC Associate Editors (2000), 110–123.
37.
WhiteL., AlmezenH. and AlzeidiN., User-Based Testing of GUI Sequences and Their Interactions. IEEE | 12th International Symposium on Software Reliability Engineering (ISSRE’01), 2001, pp. 54–65.
38.
XieQ. and MemonA.M., Studing the fault-detection effectiveness of GUI test cases for rapidly envolving software, IEEE Computer Society (2005).
39.
XieQ. and MemonA.M., Designing and comparing automated test oracles for GUI-based software applications, ACM Trans Softw Eng Methodol16(1) (2007).
40.
YangW., PrasadM.R. and XieT., A grey-box approach for automated gui-model generation of mobile applications. In CortellessaV. and VarróD., editors, FASE, volume 7793of Lecture Notes in Computer Science, Springer, 2013, pp. 250–265.
41.
YehT., ChangT.-H. and MillerR.C., Sikuli: Using GUI screenshots for search and automation. In WilsonA.D. and GuimbretiéreF., editors, UIST, ACM, 2009, pp. 183–192.
42.
YuanX. and MemonA.M., Using GUI Run-Time State as Feedback to Generate Test Cases, In ICSE, IEEE Computer Society, 2007, pp. 396–405.
