RULES-3 Plus is a member of the RULES family of simple inductive learning algorithms with successful engineering applications. However, it requires modification in order to be a practical tool for problems involving large data sets. In particular, efficient mechanisms are needed for handling continuous attributes and noisy data. This article presents a new rule induction algorithm called RULES-6, which is derived from the RULES-3 Plus algorithm. The algorithm employs a fast and noise-tolerant search method for extracting IF-THEN rules from examples. It also uses simple and effective methods for rule evaluation and handling of continuous attributes. A detailed empirical evaluation of the algorithm is reported in this paper. The results presented demonstrate the strong performance of the algorithm.
BrahaD.Data mining for design and manufacturing: Methods and applications, 2001 (Kluwer Academic Publishers, Boston).
2.
MonostoriL.AI and machine learning techniques for managing complexity, changes and uncertainties in manufacturing. In Proceedings of the 15th Triennial World Congress, Barcelona, Spain, 2002, pp. 119–130.
3.
PhamD. T.AfifyA. A.Machine learning techniques and their applications in manufacturing. Proc. Instn Mech. Engrs. Part B: J. Engineering Manufacture, 2005, 219(B5), 395–412.
4.
BreimanL.FriedmanJ. H.OlshenR. A.StoneC. J.Classification and regression trees, 1984 (Belmont, Wadsworth).
5.
QuinlanJ. R.Learning efficient classification procedures and their application to chess endgames. In Machine learning: An artificial intelligence approach (Eds MichalskiR. S.CarbonellJ. G.MitchellT. M.), 1983, Vol. 1, pp. 463–482 (Tioga Publishing Co., Palo Alto, CA).
6.
QuinlanJ. R.Induction of decision trees. Mach. Learn., 1986, 1, 81–106.
7.
QuinlanJ. R.C4.5: Programs for machine learning, 1993 (Morgan Kaufmann, San Mateo, CA).
8.
ISL. Clementine data mining package, 1998 (SPSS UK Ltd., Surrey, UK).
9.
RuleQuest. Data Mining Tools C5.0. Pty Ltd, 30 Athena Avenue, St Ives NSW 2075, Australia, available from: http://www.rulequest.com/see5-info.html. accessed 1 February 2003.
10.
MehtaM.AgrawalR.RissanenJ.SLIQ: A fast scalable classifier for data mining. In Proceedings of the fifth International Conference on Extending database technology, Avignon, France, 1996, pp. 18–32.
11.
ShaferJ.AgrawalR.MehtaM.SPRINT: A scalable parallel classifier for data mining. In Proceedings of the 22nd International Conference on Very large data bases (VLDB), Mumbai (Bombay), India, 1996, pp. 544–555.
12.
RastogiR.ShimK.PUBLIC: A decision tree classifier that integrates building and pruning. In Proceedings of the 24th International Conference on Very large data bases (VLDB), New York, USA, 1998, pp. 404–415.
13.
MichalskiR. S.On the quasi-minimal solution of the general covering problem. In Proceedings of the fifth International Symposium on Information processing (FCIP 69), Bled, Yugoslavia, 1969, A3 (Switching Circuits), pp. 125–128.
14.
MichalskiR. S.MozeticI.HongJ.LavracN.The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. American association of artificial intelligence, 1986, pp. 1041–1045 (Morgan Kaufmann, Los Altos, CA).
15.
MichalskiR. S.KaufmanK. A.The AQ19 system for machine learning and pattern discovery: A general description and user guide. Reports of the Machine Learning and Inference Laboratory, MLI 01–2, George Mason University, Fairfax, VA, USA, 2001.
ClarkP.BoswellR.Rule induction with CN2: Some recent improvements. In Proceedings of the fifth European Conference on Artificial intelligence, Porto, Portugal, 1991, pp. 151–163.
18.
CohenW. W.Fast effective rule induction. In Proceedings of the 12th International Conference on Machine learning, Lake Tahoe City, California, USA, 1995, pp. 115–123 (Morgan Kaufmann, San Francisco, CA).
19.
CohenW. W.SingerY.A simple, fast and effective rule learner. In Proceedings of the 16th National Conference on Artificial intelligence, Menlo Park, CA, 1999, pp. 335–342 (AAAI/MIT Press, Menlo Park, CA).
LeeC.Generating classification rules from databases. In Proceedings of the Ninth Conference on Application of artificial intelligence in engineering, PA, USA, 1994, pp. 205–212.
26.
PhamD. T.DimovS. S.An algorithm for incremental inductive learning. Proc. Instn. Mech. Engrs. PartB:J. Engineering Manufacture, 1997, 211, 239–249.
27.
PhamD. T.BigotS.DimovS. S.RULES-5: A rule induction algorithm for problems involving continuous attributes. Proc. Instn Mech. Engrs. Part C:J. Mechanical Engineering Science, 2003, 217(C12), pp. 1273–1286.
28.
PhamD. T.DimovS. S.An approach to concurrent engineering. Proc. Instn Mech. Engrs. Part B: J. Engineering Manufacture, 1998, 212(B1), 13–27.
29.
JenningsN. R.Automated visual inspection of engine valve stem seals. Internal Report, University of Wales Cardiff, Cardiff, UK, 1996.
DomingosP.A unified approach to concept learning. PhD Thesis, University of California, Irvine, 1997.
32.
HolteR. C.AckerL. E.PorterB. W.Concept learning and the problem of small disjuncts. In Proceedings of the 11th International Joint Conference on Artificial intelligence, Detroit, Michigan, USA, 1989, pp. 813–818.
33.
WeissG. M.Learning with rare cases and small disjuncts. In Proceedings of the 12th International Conference on Machine learning, Lake Tahoe City, California, USA, 1995, pp. 558–565 (Morgan Kaufmann, San Francisco, CA).
34.
WeissG. M.HirshH.The problem with noise and small disjuncts. In Proceedings of the 15th International Conference on Machine learning, Madison, Wisconsin, USA, 1998, pp. 574–578 (Morgan Kaufmann, San Francisco, CA).
35.
FraymanY.TingK. M.WangL.A fuzzy neural network for data mining: Dealing with the problem of small disjuncts. In IEEE International Joint Conference on Neural networks (IJCNN-99), Washington, DC, 1999, 4, pp. 2490–2493.
36.
WeissG. M.HirshH.A quantitative study of small disjuncts. In Proceedings of the 17th National Conference on Artificial intelligence, Austin, Texas, 2000, pp. 665–670.
37.
WeissS.IndurkhyaN.Reduced complexity rule induction. In Proceedings of the 12th International Joint Conference on Artificial intelligence, Sydney, Australia, 1991 pp. 678–684 (Morgan Kaufmann, San Francisco, CA).
38.
CendrowskaJ.PRISM: An algorithm for inducing modular rules. Int. J. Man-Mach Studies, 1987, 27, 349–370.
39.
FürnkranzJ.WidmerG.Incremental reduced error pruning. In Proceedings of the 11th International Conference on Artificial intelligence, Amsterdam, The Netherlands, 1994, pp. 453–457.
WebbG.Systematic search for categorical attribute-value data-driven machine learning. In AI 94-Proceedings of the sixth Australian joint conference on artificial intelligence (Eds RowlesC.LiuH.FooN.), Melbourne, 1993, pp. 342–347 (World Scientific, Chicago).
42.
WebbG.OPUS: An efficient admissible algorithm for unordered search. J. Artif. Intell. Res., 1995, 3, 431–465.
43.
SegalR. B.Machine learning as massive search. PhD Thesis, Department of Computer Science and Engineering, University of Washington, 1997.
44.
CestnikB.Estimating probabilities: A crucial task in machine learning. In Proceedings of the Third European Conference on Artificial intelligence (ECAI-90), Stockholm, Sweden, 1990, pp. 147–149 (Pitman, London).
45.
AfifyA. A.Design and analysis of scalable rule induction systems. PhD Thesis, University of Wales Cardiff, School of Engineering, Systems Engineering Division, Cardiff, UK, 2004.
46.
GoodI. J.The estimation of probabilities: An essay on modern Bayesian methods, 1965 (MIT Press, Cambridge, MA).
47.
MitchellT. M.Machine learning, 1997 (McGraw Hill, New York).
48.
HoK. M.ScottP. D.Zeta: A global method for discretization of continuous variables. IEEE Trans. Knowl. DataEng., 1997, 9 (5), 718–730.
49.
ZighedD. A.RakotomalalaR.FeschetF.Optimal multiple intervals discretization of continuous attributes for supervised learning. In Proceedings of the Third International Conference on Knowledge discovery and data mining (KDD-97), Newport Beach, California, USA, 1997, pp. 295–298 (AAAI Press, Menlo Park, CA).
50.
FrankE.WittenI. H.Making better use of global discretization. In Proceedings of the 15th International Conference on Machine learning, Madison, Wisconsin, USA, 1998, pp. 152–160.
51.
TrautzschS.PernerP.Multi-interval discretization methods for decision tree learning. In Advances in pattern recognition (Eds AminA.DoriD.PudilP.FreemanH.), 1998, 1451, pp. 475–482 (Springer-Verlag, Germany).
52.
AnA.CerconeN.Discretisation of continuous attributes for learning classification rules. In Proceedings of the Third Pacific-Asia Conference on Knowledge discovery and data mining (PAKDD-99), Kyoto, Japan, 1999, pp. 509–514.
53.
WongA. K. C.ChiuD. K. Y.Synthesizing statistical knowledge from incomplete mixed-mode data. IEEE Trans. Pattern Anal. Mach. Intell., 1987, 9 (6), 796–805.
54.
HolteR. C.Very simple classification rules perform well on most commonly used data sets. Mach. Learn., 1993, 11, 63–90.
55.
CatlettJ.On changing continuous attributes into ordered discrete attributes. In Proceedings of the European Working Session on Learning, Porto, Portugal, 1991, pp. 164–178 (Springer-Verlag, Germany).
56.
FayyadU. M.IraniK. B.Multi-interval discretization of continuous-valued attributes for classification. In Proceedings of the 13th International Joint Conference on Artificial intelligence, Chambery, France, 1993, pp. 1022–1027.
57.
FayyadU. M.IraniK. B.On the handling of continuous-valued attributes in decision tree generation. Mach. Learn., 1992, 8, 87–102.
58.
MaassW.Efficient agnostic PAC-learning with simple hypotheses. In Proceedings of the Seventh Annual ACM Conference on Computational learning theory, New Brunswick, New Jersey, USA, 1994, pp. 67–75.
59.
AuerP.HolteR. C.MaassW.Theory and application of agnostic PAC-learning with small decision trees. In Proceedings of the 12th International Conference on Machine learning, Lake Tahoe City, California, USA, 1995, pp. 21–29 (Morgan Kaufmann, San Francisco, CA).
60.
FultonT.KasifS.SalzbergS.Efficient algorithms for finding multi-way splits for decision trees. In Proceedings of the 12th International Conference on Machine learning, Lake Tahoe City, California, USA, 1995, pp. 244–251 (Morgan Kaufmann, San Francisco CA).
61.
BirkendorfA.On fast and simple algorithms for finding maximal subarrays and applications on computational learning theory. In Lecture notes in artificial intelligence, 1997, 1208, pp. 198–209 (Springer-Verlag, Heidelberg).
62.
RousuJ.Efficient range partitioning in classification learning. PhD Thesis, Department of Computer Science, University of Helsinki, Finland, 2001.
63.
CaiZ.Technical aspects of data mining. PhD Thesis, University of Wales Cardiff, Cardiff, UK, 2001.
64.
DoughertyJ.KohaviR.SahamiM.Supervised and unsupervised discretization of continuous features. In Proceedings of the 12th International Conference on Machine learning, Lake Tahoe City, California, USA, 1995, pp. 194–202 (Morgan Kaufmann, San Francisco, CA).
65.
VenturaD.MartinezT. R.An empirical comparison of discretization methods. In Proceedings of the 10th International Symposium on Computer and information sciences, Aydin, Turkey, 1995, pp. 443–450.
66.
KohaviR.SahamiM.Error-based and entropy-based discretization of continuous features. In Proceedings of the second International Conference on Knowledge discovery in databases (Eds SimondisE.FayyadU.), 1996, pp. 114–119 (AAAI Press, Menlo Park, CA).
67.
BlakeC. L.MerzC. J.UCI repository of machine learning databases. University of California, Department of Information and Computer Science, Irvine, CA, 1998, available from http://www.ics.uci.edu/εmlearn/MLRepository.html.
