Kanban, which is an agile process methodology as well as a means to implement lean principles, has been growing as a project management framework across a range of domains, including manufacturing, software development and data science. This paper explores, for teams using Kanban, the ability to predict low team performance. The prediction is based on an analytical model that uses specific project metrics that can be collected via the team’s visual Kanban board. Specifically, data from 80 teams was used to build and test machine learning models that predict teams at risk for delivering low quality results. The model developed was significantly better than the baseline situation of thinking that all teams were at risk. While this analysis was done within a data science project context, the results are likely applicable across a range of information system projects.
AhmadM. O.MarkkulaJ., & OivoM. (2013). Kanban in software development: A systematic literature review. In 2013 39th Euromicro Conference on Software Engineering and Advanced Applications, Santander, Spain, 2013, 9-16.
2.
AhmadM. O.KuvajaP.OivoM., & MarkkulaJ. (2016). Transition of software maintenance teams from Scrum to Kanban. In 49th Hawaii International Conference on System Sciences (HICSS), 5427-5436.
3.
AhmadM. O.DennehyD.ConboyK., & OivoM. (2018). Kanban in software engineering: A systematic mapping study. The Journal of Systems and Software, 137,, 96-113.
4.
Al-AidarosH. A., & OmarM. (2017). Software project management approaches for monitoring work-in-progress: A review. Journal of Engineering and Applied Sciences, 12(15), 3851-3857.
5.
AndersonD. (2010). Kanban: Successful Evolutionary Change For Your Technology Business. Blue Hole Press, Sequim, Washington.
6.
BaronJ.A. (1994). Too bad it isn’t true. Medical Decision Making, 14(2), 107-107. doi: 10.1177/0272989X9401400202. PMID 8028462.
7.
BensonJ. (2011) Kanban for Short Intense Projects: How We Used Kanban to Visualize Our Hiring Process Workflow and Make Our Lives Easier, Accessed July 18, 2019. http://personalkanban.com/pk/designpatterns/kanban-for-short-intense-projects-how-we-used-kanban-to-visualize-our-hiring-process-workflow-and-make-our-lives-easier/.
8.
BeranderP. (2004). Using students as subjects in requirements prioritization. Proceedings, International Symposium Empirical Software Engineering 2004. ISESE’04, 167-176.
9.
BladtJ., & FilbinB. (2013). Know the Difference Between Your Data and Your Metricshttps://hbr.org/2013/03/know-the-difference-between-yo
10.
BoykoE. J. (1994). Ruling out or ruling in disease with the most sensitive or specific diagnostic test: short cut or wrong turn? Medical Decision Making, 14(2), 175-179. doi:10.1177/0272989X9401400210. PMID 8028470.
11.
CatalC. & DiriB. (2009). A systematic review of software fault prediction studies. Expert Systems with Applications, 36(4), 7346-7354.
12.
DvirD.LipovetskyS.ShenharA., & TishlerA. (1998). In search of project classification: a non-universal approach to project success factors. Research Policy, 27, 915-935.
13.
DvirD.LipovetskyS.ShenharA., & TishlerA. (2003). What is really important for project success? A refined, multivariate, comprehensive analysis. International Journal of Management and Decision Making, 4(4), 382-404.
14.
El-HaleesA. (2008). Mining student data to analyze learning behavior: A case study, International Arab Conference of Information Technology (ACIT2008), Tunisia, 2008.
15.
FeitelsonD. G. (Dec 2015). Using students as experimental subjects in software engineering research – a review and discussion of the evidence. ArXiv:1512.08409 [cs.SE].
16.
FentonN. E., & PfleegerS. L. (1998). Software Metrics: A Rigorous and Practical Approach. 2nd edition, PWS Publishing Co. Boston, MA, USA.
17.
HildebrandtH.GurevichP.StukeH.KastrupA., & StukeH. (2017). Neuropsychological testing and machine learning distinguish alzheimer’s disease from other causes for cognitive impairment. Frontiers in Aging Neuroscience. 9. doi: 10.3389/fnagi.2017.00114.
18.
HongS.ChenG., & HuS. (2009). Evaluation of the sensitivity and specificity of qualitative diagnostic test under normal assumption. Model Assisted Statistics and Applications. 4. doi: 10.3233/MAS-2009-0113.
19.
HöstM.RegnellB., & WohlinC. (2000). Using Students as Subjects — A Comparative Study of Students and Professionals in Lead-Time Impact Assessment. Empirical Software Engineering, 5(3), 201-214.
HuaJ.XiongZ.LoweyJ.SuhE., & DoughertyE. R. (2005). Optimal number of features as a function of sample size for various classification rules. Bioinformatics, 21(8), 1509-1515. https://doi.org/10.1093/bioinformatics/bti171
22.
JamesG.WittenD.HastieT., & TibshiraniR. (2013). An Introduction to Statistical Learning with Applications in R.
23.
KabakchievaD. (2013). Predicting Student Performance by Using Data Mining Methods for Classification. Cybernetics and Information Technologies, 13, 1.
24.
KahnemanD., & TverskyA. (1979). Intuitive prediction: biases and corrective procedures. TIMS Studies in Management Science12, 313-327.
25.
KirovskaN., & KoceskiS. (2015). Usage of Kanban methodology at software development teams. Journal of Applied Economics and Business, 3(3), 25-34. ISSN 1857-8721.
26.
KitchenhamB. A.PfleegerS. L.PickardL. M.JonesP. W.HoaglinD. C.EmamK. E., & RosenbergJ. (2002). Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering, 28(8), 721-734.
27.
KupiainenE.MantylaM. V., & ItkonenJ. (2014). Why are industrial agile teams using metrics and how do they use them? in: Proceedings of the 5th International Workshop on Emerging Trends in Software Metrics. ACM, 2014, 23-29.
28.
KupiainenE., Mäntyla M.V., & Itkonen J. (2015). Using metrics in agile and lean software development – a systematic literature review of industrial studies. Information and Software Technology, 62, 143-163.
29.
LambertR. (2014). Using Kanban in the recruitment process, The Social Tester. Accessed 18 July 2019. http://thesocialtester.co.uk/kanban-and-recruitment/
30.
LandryJ., & McDanielR. (2015). Agile preparation within a traditional project management course, agile preparation within a traditional project management course, 2015 Proceedings of the EDSIG Conference, (2015).
31.
LeiH.GanjeizadehF.JayachandranP. K., & OzcanP. (2017). A statistical analysis of the effects of Scrum and Kanban on software development projects. Robotics and Computer-Integrated Manufacturing, 43, 59-67.
32.
LevesqueL. L.WilsonJ. M., & WholeyD. R. (2001). Cognitive divergence and shared mental models in software development project teams. Journal of Organizational Behavior, 22(2), Special Issue: Shared Cognition, 135-144.
33.
LeybournE. (2013). Directing the Agile Organisation: A Lean Approach to Business Management. London: IT Governance Publishing: 160-66.
34.
LipovetskyS.TishlerA.DvirD., & ShenharA. (1997). The relative importance of project success dimensions. R&D Management, 27, 97-106.
35.
MahničV., & ZabkarN. (2012). Measuring progress of scrum-based software projects. Electronics and Electrical Engineering, 18(8), 73-76.
36.
MahničV. (2013). Applying Kanban Principles to Software Development, Proceedings of the IT for Practice 2013 16th International Conference on Information Technology for Practice.
37.
MiddletonP., & JoyceD. (2012). Lean Software Management: BBC Worldwide Case Study, Engineering Management. IEEE Transactions, 59(1), 20-32.
38.
Minaei-BidgoliB.KashyD. A.KortemeyerG., & PunchW. F. (2003). Predicting student performance: an application of data mining methods with an educational Web-based system. 33rd Annual Frontiers in Education, 2003. FIE 2003, Westminster, CO, 2003, pp. T2A-13 doi: 10.1109/FIE.2003.1263284
39.
MooreR.ReffK.GrahamJ., & HackersonB. (2007). Scrum at a Fortune 500 Manufacturing Company. Agile 2007 (AGILE 2007), Washington, DC, 2007, 175-180.
40.
MujtabaS.FeldtR., & PetersenK. (2010). Waste and lead time reduction in a software product customization process with value stream maps. Proceedings of the Australian Software Engineering Conference, ASWEC, 139-148, Auckland.
41.
NakazawaS. & TanakaT. (2015). Prototype of Kanban Tool and Preliminary Evaluation of Visualizing Method for Task Assignment. In 2015 International Conference on Computer Application Technologies, Matsue, 2015, 48-49.
42.
NakazawaS. & TanakaT. (2016). Development and Application of Kanban Tool Visualizing the Work in Progress. In 2016 5th IIAI International Congress on Advanced Applied Informatics (IIAI-AAI), Kumamoto, 2016, 908-913.
43.
OhnoT. (1988). Toyota Production System: beyond large-scale production. Productivity Press, 1988.
44.
OsmanbegovićE., & SuljićM. (2012). Data mining approach for predicting student performance. Economic Review – Journal of Economics and Business, X(1), May 2012.
45.
OstergaardK. (2016). Applying Kanban principles to electronic resource acquisitions with Trello. Journal of Electronic Resources Librarianship, 28(1), 48-52.
46.
ParikhR.MathaiA.ParikhS.SekharG. C., & ThomasR. (2008). Understanding and using sensitivity, specificity and predictive values. Indian Journal of Ophthalmology, 2008 Jan-Feb; 56(1), 45-50.
47.
PedregosaF.VaroquauxG.GramfortA.MichelV.ThirionB.GriselO.BlondelM.PrettenhoferP.WeissR.DubourgV.VanderplasJ.PassosA.CournapeauD.PerrotM., & DuchesnayE. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research12, 2825-2830.
48.
PetersenK., & WohlinC. (2010). The effect of moving from a plan-driven to an incremental software development approach with agile practices: An industrial case study. Empirical Software Engineering, 15(6), 654-693.
49.
PetersenK., & WohlinC. (2011). Measuring the flow in lean software development. Software - Practice and Experience, 41(9), 975-996.
50.
PewsnerD.BattagliaM.MinderC.MarxA.BucherH. C., & EggerM. (2004). Ruling a diagnosis in or out with “SpPIn” and “SnNOut”: a note of caution. BMJ, 329(7459), 209-213. doi:10.1136/bmj.329.7459.209. PMC 487735. PMID 15271832
51.
Project Management Institute (PMI). (2004). A Guide to the project management body of knowledge, third edition (PMBOK® guide). Newtown Square, PA: Project Management Institute.
52.
PowerK. (2014). Metrics for Understanding Flow, Agile Software Development Conference (Agile 2014), Orlando, FL, USA.
53.
PowersD. (2011). Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation. Journal of Machine Learning Technologies, 2(1), 37-63.
54.
PulfordK.Kuntzmann-CombellesA., & ShirlawS. (1995). A quantitative approach to software management: The AMI handbook, Addison-Wesley Longman Publishing Co., Inc., 1995.
55.
PressmanR. S., & MaximB. R. (2015). Software Engineering: A Practitioner’s Approach, 8th Edition. McGraw-Hill, 2015.
56.
RunesonP. (2003). Using students as experiment subjects – an analysis on graduate and freshmen student data. Proceedings of the 7th International Conference on Empirical Assessment & Evaluation in Software Engineering, 95-102.
57.
SalmanI.MisirliA.T., & JuristoN. (2015). Are students representatives of professionals in software engineering experiments? Proceedings of the 37th International Conference on Software Engineering, 1, 666-676.
58.
SaltzJ. (2015). The need for new processes, methodologies and tools to support big data teams and improve big data project effectiveness. in: Big Data (Big Data), 2015 IEEE International Conference on, Santa Clara, CA, USA, 2015, 2066-2071.
59.
SaltzJ.ShamshurinI., & ConnorsC. (2017). Predicting data science sociotechnical execution challenges by categorizing data science projects, Journal of the Association for Information Science and Technology (JASIST).
60.
SaltzJ.ShamshurinI., & CrowstonK. (2017). Comparing Data Science Project Management Methodologies via a Controlled Experiment. In 2017 50th Hawaii International Conference on System Sciences, Waikoloa, HI, USA, 2017, 1013-1022.
SharmaD., Yadav U.B., & SharmaP. (2009). The concept of sensitivity and specificity in relationto two types of errors and its application in medical research. Journal of Reliability and Statistical Studies, 2(2), 53-8.
64.
ShenB., & JuD. (2007). On the measurement of agility in software process. Lecture Notes in Computer Science, 4470 LNCS, 25-36.
65.
ShenharA.TishlerA.DvirD.LipovetskyS., & LechlerT. (2002). Refining the search for project success factors: a multivariate, typological approach. R&D Management, 32(2), 111-126.
66.
SjøbergD.I.K.AndaB.ArisholmE.DybaT.JorgensenM.KarahasanovicA.KorenE. F., & VokacM. (2002). Conducting realistic experiments in software engineering. Proceedings International Symposium on Empirical Software Engineering, 1325.
67.
SjøbergD.I.K.HannayJ. E.HansenO.KampenesV. B.KarahasanovicA.LiborgN. K., & RekdalA. C. (2005). A survey of controlled experiments in software engineering. IEEE Transactions on Software Engineering, 31(9), 733-753.
68.
SutherlandJ. (2004). Agile development: lessons learned from the first Scrum, Cutter Agile Project Management Advisory Service. Executive Update, 5, 20.
69.
SugimoriY.KusunokiK.ChoF., & UchikawaS. (1977). Toyota production system and kanban system materialization of just-in-time and respect-for-human system. International Journal of Production Research15(6), 553-564.
70.
SvahnbergM.AurumA., & WohlinC. (2008). Using students as subjects – an empirical evaluation. In Proceedings of the Second ACMIEEE international symposium on Empirical software engineering and measurement, 288-290.
71.
TaibiD.LenarduzziV.JanesA.LiukkunenK., & AhmadM. (2017). Comparing Requirements Decomposition Within the Scrum, Scrum with Kanban, XP, and Banana Development Processes. XP 2017: Agile Processes in Software Engineering and Extreme Programming, 68-83.
72.
TichyW. F. (2000). Hints for reviewing empirical work in software engineering. Empirical Software Engineering5(4), 309-312.
73.
TishlerA.DvirD.ShenharA., & LipovetskyS. (1996). Identifying critical success factors of defense development projects: a multivariate analysis. Technological Forecasting and Social Change, 51, 151-171.
74.
TrapaV., & RaoS. (2006). T3 - tool for monitoring agile development,Proceedings - AGILE Conference, 2006, 243-248, Minneapolis, MN.
75.
TudorD. & WalterG. (2006). Using an agile approach in a large, traditional organization. In Proceedings - AGILE Conference, 2006, 367-373, Minneapolis, MN.
76.
VersionOne Inc. (2018). The 12th annual state of agile report. Technical Report. VersionOne Inc. https://explore.versionone.com/state-of-agile/versionone-12th-annual-state-of-agile-report.
77.
VersionOne, Inc. (2016). The 10th annual state of agile survey. Technical Report. VersionOne Inc. http://www.agile247.pl/wp-content/uploads/2016/04/VersionOne-10th-Annual-State-of-Agile-Report.pdf.
78.
WolfT.SchroterA.DamianD., & NguyenT. (2009). Predicting build failures using social network analysis on developer communication. In Proceedings of the 31st International Conference on Software Engineering (ICSE ‘09). IEEE Computer Society, Washington, DC, USA, 1-11. doi: 10.1109/ICSE.2009.5070503.
79.
YerushalmyJ. (1947). Statistical problems in assessing methods of medical diagnosis with special reference to x-ray techniques. Public Health Reports, 62(2), 1432-39. doi: 10.1177/0272989X9401400202. PMID 8028462.
80.
ZhangJ. G. (2011). Schedule management method study of middle and small software projects. in: 2nd IEEE International Conference on Artificial Intelligence, Management Science and Electronic Commerce, Dengleng, China, 1495-1498.
