System safety is closely related to system reliability. Safety requirements many times are translated to reliability requirements. Nowadays, software systems exist in many engineering systems. However, there is no consensus method for software reliability estimation. On the contrary, there is an increasing interest in estimating the software reliability due to concerns for safety-critical systems. In this article, we try to close the gap by proposing a systematic and probabilistic method to estimate the software reliability based on software test data.
IAEA. IAEA Standard SSR-2/1. Safety of nuclear power plant: design. 2016. Vienna: IAEA.
2.
IAEA. Application of probabilistic safety assessment (PSA) for nuclear power plants. IAEA technical report IAEA-TECDOC-1200, February2001. Vienna: IAEA.
3.
IAEA. Development and application of level-1 probabilistic safety assessment for nuclear power plant. IAEA Special safety guide no. SSG-3, June2010. Vienna: IAEA.
4.
LawrenceJD.Software reliability and safety in nuclear reactor protection systems. NUREG/CR-6001, 11June1993. Rockville, MD: U.S. Nuclear Regulatory Commission.
5.
ParnasDLAsmisGJKMadeyJ. Assessment of safety-critical software in nuclear power plants. Nucl Safe1991; 32(2): 189–198.
6.
LevesonNGHarveyPR. Analyzing software safety. IEEE Trans Soft Eng1983; 9: 569–579.
7.
FarrW. Software reliability modeling survey. In: LyuMR (ed.) Handbook of software reliability engineering. New York: IEEE Computer Society Press; McGraw-Hill, 1996, pp.71–117.
8.
HuangCYLinCT. Software reliability analysis by considering fault dependency and debugging time lag. IEEE Trans Reliab2006; 55(2): 436–450.
9.
KapurPKPhamHAnandS, et al. A unified approach for developing software reliability growth models in the presence of imperfect debugging and error generation. IEEE Trans Reliab2011; 60(1): 331–340.
10.
KuoS-YHuangC-YLyuMR. Framework for modeling software reliability, using various testing-efforts and fault-detection rate. IEEE Trans Reliab2001; 50: 310–320.
SongKYChangIJPhamH. A software reliability model with a Weibull fault detection rate function subject to operating environments. Appl Sci2017; 7: 983.
13.
WangWLHemmingerTLTangMH. A moving average non-homogeneous Poisson process reliability growth model to account for software with repair and system structure. IEEE Trans Reliab2007; 56(3): 411–421.
14.
MillerKWMorellLJNoonanRE, et al. Estimating the probability of failure when testing reveals no failure. IEEE Trans Soft Eng1992; 18: 33–43.
15.
CalinescuRGhezziCJohnsonK, et al. Formal verification with confidence intervals to establish quality of service properties of software systems. IEEE Trans Reliab2016; 65(1): 107–125.
16.
BishopPBloomfieldRLittlewoodB, et al. A conservative bound for the probability of failure of a 1-out-of-2 protection system with one hardware-only and one software-based protection train. Reliab Eng Syst Safe2014; 130: 61–68.
17.
BishopPPovyakaloA. Deriving a frequentist conservative confidence bound for probability of failure per demand for systems with different operational and test profile. Reliab Eng Syst Safe2017; 158: 246–253.
18.
LittlewoodBRushbyP. Reasoning about the reliability of diverse two-channel system in which one channel is “possible perfect.” IEEE Trans Soft Eng2012; 38: 1178–1194.
19.
LvJYinBBCaiKY. Estimating confidence interval of software reliability with adaptive testing strategy. J Syst Soft2014; 97: 192–306.
20.
MayJHughesGLunnAD. Reliability estimation from appropriate testing of plant protection software. J Soft Eng1995; 10: 206–218.
21.
PressSJ. Bayesian statistics: principles, models, and applications. New York: John Wiley & Sons, 1989.
22.
GokhaleSS. Architecture-based software reliability analysis: overview and limitations. IEEE Trans Depend Sec Comput2007; 4: 32–40.
23.
Goševa-PopstojanovaKTrivediKS. Architecture-based approach to reliability assessment of software systems. Perform Eval2001; 45: 179–204.
24.
ClarkeEMWingJM. Formal methods: state of the art and future directions. ACM Comput Surv1996; 28(4): 626–643.
YangY. A flow network model for software reliability assessment. In: Proceedings of 6th American nuclear society international topical meeting on nuclear plant instrumentation, control, and human-machine interface technologies, Knoxville, TN, 5–9 April 2009. New York: Curran Associates.
27.
YangYSydnorR. Reliability estimation for a digital instrument and control system. Nucl Eng Tech2012; 44(4): 405–414.
28.
ZouBYangMYangJ, et al. Reliability analysis and allocation: development of a hierarchical structure modeling platform in I&C system software life cycle. Nucl Eng Des2018; 328: 345–352.
29.
BishopPBloomfieldRLittlewoodB, et al. Toward a formalism for conservative claims about the dependability of software-based systems. IEEE Trans Soft Eng2011; 37: 708–717.
30.
MusaJD. Operational profiles in software-reliability engineering. IEEE Soft1993; 10(2): 14–32.
31.
ZhaoXLittlewoodBPovyakaloA, et al. Modeling the probability of failure on demand (pfd) of 1-out-of-2 system in which one channel is “quasi-perfect.”Reliab Eng Syst Safe2017; 158: 230–245.
32.
McCabeT. A complexity measure. IEEE Trans Soft Eng1976; 2(4): 308–320.
33.
KurfessFJLankalaMVantipalliA, et al. A toolset for the reengineering of complex computer systems. In: Proceedings of international conference and workshop on engineering of computer-based systems, Monterey, CA, 24–28 March 1997, pp.97–104. New York: IEEE.
34.
ChuTLYueMMartinez-GuridiG, et al. Review of quantitative software reliability methods (BNL-94074-2010). New York: Brookhaven National Laboratory, 2010.
35.
SenKAghaG. CUTE and jCUTE: concolic unit testing and explicit path model-checking tools. In: International conference on computer aided verification, vol. 6, Seattle, WA, 17–20 August 2006. Berlin: Springer.
36.
GelmanACarlinJBSternHS, et al. Bayesian data analysis. Boca Raton, FL: Chapman and Hall/CRC, 2004.
