Test effort estimation and prediction of traditional and rapid release models using machine learning algorithms

Abstract

Recently, many software companies have shifted to shorter release cycles from the traditional multi-month release cycle. Evolution and transition of release cycles may affect the test effort in the system. This paper analyses 25 traditional releases containing 1210 classes and 69 rapid releases containing 2616 classes of four Open Source Java systems. Correlations between 48 Object Oriented metrics and 2 test metrics were evaluated to identify the best indicators of test effort. The results show that (i) correlation between OO and test metrics remain irrespective of release models, (ii) test effort required in Rapid Release (RR) models (shorter release cycles) is slightly more as compared to Traditional Release (TR) models, (iii) Out of 18 machine learning algorithms instance based machine learning algorithms IBK and K star followed by Multi-Layer Perceptron (MLP) and additive regression are able to predict the test effort accurately in classes.

Keywords

Release cycles machine learning prediction software metrics test effort

Get full access to this article

View all access options for this article.

References

Kaur

and Kaur

, Statistical comparison of modelling methods for software maintainability prediction, International Journal of Software Engineering and Knowledge Engineering23(6) (2013), 743–774.

Apache Software Foundation (ASF), http://www.apache.org/ .

Croux

and Dehon

, Influence functions of the Spearman and Kendall correlation measures, Statistical Methods and Applications19(4) (2010), 497–515.

Gamma

, Agile, open source, distributed, and on-time inside the eclipse development process, International Conference on Software, (Keynote)2005.

Khomh

, Adams

, Dhaliwal

and Zou

, Understanding the impact of rapid releases on software quality, Empirical Software Engineering20(2) (2015), 336–373.

Khomh

, Dhaliwal

, Zou

and Adams

, Do faster releases improve software quality? An empirical case study of Mozilla Firefox, Proceedings of the 9th Working Conference on Mining Software Repositories2012, pp. 179–188.

Toure

, Badri

and Lamontagne

, A metrics suite for JUnit test code: A multiple case study on open source software, Journal of Software Engineering Research and Development2(1) (2014), 14–26.

Toure

, Badri

and Lamontagne

, Predicting different levels of the unit testing effort of classes using source code metrics: A multiple case study on open-source software, Innovations in Systems and Software Engineering14(1) (2018), 15–46.

Github , http://ww.github.com .

10.

IntellijIdea, https://www.jetbrains.com/idea .

11.

Demsar

, Statistical comparisons of classifiers over multiple data sets, Journal of Machine Learning Research7 (2006), 1–30.

12.

Jira, http://www.jira.atlassian.org .

13.

Beck

, Extreme programming explained: Embrace change. 2000Professional Addison-Wesley.

14.

Stewart

K.J.

, Darcy

D.P.

and Daniel

S.L.

, Observations on patterns of development in open source software projects, ACM SIGSOFT Software Engineering Notes30(4) (2005), 1–5.

15.

Aggarwal

K.K.

, Singh

, Kaur

and Malhotra

, Empirical study of object-oriented metrics, Journal of Object Technology5(8) (2006), 149–173.

16.

Badri

, Badri

and Toure

, An empirical analysis of lack of cohesion metrics for predicting testability of classes, International Journal of Software Engineering and Its Applications5(2) (2011), 69–86.

17.

Badri

, Badri

and Toure

, Exploring empirically the relationship between lack of cohesion and testability in object-oriented systems, International Conference on Advanced Software Engineering and Its Applications2010, pp. 78–92.

18.

Badri

and Toure

, Empirical analysis of object-oriented design metrics for predicting unit testing effort of classes, Journal of Software Engineering and Applications5(7) (2012), 513–524.

19.

Badri

Toure

and Lamontagne

, Predicting unit test effort levels of classes: An exploratory study based on multinomial logistic regression modeling, Procedia Computer Science62 (2015), 529–538.

20.

Badri

, Badri

, Flageol

and Toure

, Investigating the Accuracy of Test Code Size Prediction using Use Case Metrics and Machine Learning Algorithms: An Empirical Study, International Conference on Machine Learning and Soft Computing2017, pp. 25–33.

21.

Bruntink

and Van

, Deursen, An empirical study into class testability, Journal of Systems and Software79(9) (2006), 1219–1232.

22.

Bruntink

and Van

, Deursen, Predicting class testability using object-oriented metrics, International Workshop on Source Code Analysis and Manipulation (2004), pp. 136–145.

23.

Friedman

, A comparison of alternative tests of significance for the problem of m rankings, The Annals of Mathematical Statistics11(1) (1940), 86–92.

24.

Leon

, Van

Deursen

Zaidman

and Bruntink

, On the Interplay Between Software Testing and Evolution and its Effect on Program Comprehension, (2008), 173–202.

25.

Marschall

, Transforming a six month release cycle to continuous ow, in, pp, Proceedings of the Conference of AGILE2007, 395–400.

26.

Mäntylä

M.V.

, Adams

, Khomh

, Engström

and Petersen

, On rapid releases and software testing: A case study and a semi-systematic literature review, Empirical Software Engineering20(5) (2015), 1384–1425.

27.

Baysal

, Davis

and Godfrey

M.W.

, A tale of two browsers, in Proceedings of the 8th Working Conference on Mining Software Repositories2011, pp. 238–241.

28.

Petersen and Wohlin

, A comparison of issues and advantages in agile and incremental development between state of the art and an industrial case, Journal of Systems and Software82(9) (2009), 1479–1490.

29.

Malhotra

, An empirical framework for defect prediction using machine learning techniques with Android software, Applied Soft Computing49 (2016), 1034–1050.

30.

Kuppuswami

, Vivekanandan

, Ramaswamy

and Rodrigues

, The effects of individual XP practices on software development effort, ACM SIGSOFT Software Engineering Notes28(6) (2003), 6–6.

31.

Lessmann

, Baesens

, Mues

and Pietsch

, Benchmarking classification models for software defect prediction: A proposed framework and novel findings, IEEE Transactions on Software Engineering34(4) (2008), 485–496.

32.

McIlroy

, Ali

and Hassan

A.E.

, Fresh apps: An empirical study of frequently-updated mobile apps in the Google play store, Empirical Software Engineering21(3) (2016), 1346–1370.

33.

Shapiro

S.S.

and Wilk

M.B.

, An analysis of variance test for normality, Biometrika52(3) (1965), 591–611.

34.

Shankland

Google ethos speeds up Chrome release cycle, 2010 http://cnet.co/wlS24U .

35.

Otte

, Moreton

and Knoell

H.D.

, Applied quality assurance methods under the open source development model, in Proceedings of the 32nd Annual IEEE International Computer Software and Applications Conference (COMPSAC)2008, pp. 1247–1252.

36.

Waikato Environment for Knowledge Analysis, www.cs.waikato.ac.nz/ml/weka/ .

37.

Singh

and Saha

, Predicting testability of eclipse: A case study, Journal of Software Engineering4(2) (2010), 122–136.

38.

Zhou

, Leung

, Song

, Zhao

, Lu

, Chen

and Xu

, An in-depth investigation into the relationships between structural metrics and unit testability in object-oriented systems, Science China Information Sciences55(12) (2012), 2800–2815.