Machine learning algorithms designed for engineering applications must be able to handle numerical attributes, particularly attributes with real (or continuous) values. Many algorithms deal with continuous-valued attributes by discretizing them before starting the learning process. This paper describes a new approach for discretization of continuous-valued attributes during the learning process. Incorporating discretization within the learning process has the advantage of taking into account the bias inherent in the learning system as well as the interactions between the different attributes. Experiments have demonstrated that the proposed method, when used in conjunction with the SRI rule induction algorithm developed by the authors, improves the accuracy of the induced model.
MichieD.SpiegelhalterD. J.TaylorC. C.Machine learning, neural and statistical classification, 1994 (Ellis Horwood, New York).
2.
MitchellT. M.Machine learning, 1997 (McGraw Hill, New York).
3.
HanJ.KamberM.Data mining: concepts and techniques, 2001 (Academic Press, USA).
4.
KlösgenW.ŻytkowJ. M.Handbook of data mining and knowledge Discovery, 2002 (Oxford University Press, New York).
5.
PhamD. T.AfifyA. A.Machine learning: Techniques and trends. In Proceedings of the Ninth International Workshop on Systems, Signals and Image Processing (IWSSIP-02), Manchester Town Hall, UK, 2002, pp. 12–36 (World Scientific Publishing Co., London).
6.
BrahaD.Data mining for design and manufacturing: methods and applications, 2001 (Kluwer Academic Publishers, Boston).
7.
MonostoriLAI and machine learning techniques for managing complexity, changes and uncertainties in manufacturing. Eng. Appl. Artif. Intell., 2003, 16 (3), 227–291.
8.
PhamD. T.AfifyA. A.Machine learning techniques and their applications in manufacturing. Proc. Instn Mech. Engrs. Part B: J. Engineering Manufacture, 2005, 219 (5), 395–412.
9.
TingK. M.Discretization of continuous-valued attributes and instance-based learning. Technical report 491, Basser Department of Computer Science, University of Sydney, Sydney, Australia, 1994.
10.
WangK.LiuB.Concurrent discretisation of multiple attributes. In Proceedings of the Fifth Pacific Rim International Conference on Artificial Intelligence, Singapore, 1998, pp. 250–259.
HoK. M.ScottP. D.Zeta: A global method for discretization of continuous variables. IEEE Trans. Knowl. Data Eng., 1997, 9 (5), 718–730.
13.
JunB. H.KimC. S.KimJA new criterion in selection and discretisation of attributes for the generation of decision trees. IEEE Trans. Pattern Anal. Machine Intell., 1997, 19 (12), 1371–1375.
14.
KontkanenP.MyllymakiP.SilanderT.TirriH.A Bayesian approach to discretisation. In Proceedings of the European Symposium on Intelligent Techniques, Bari, Italy, 1997, pp. 265–268.
15.
LiuJ.SetionoRFeature selection via discretisation. IEEE Trans. Knowl. Data Eng., 1997, 9 (4), 642–645.
16.
ZighedD. A.RakotomalalaR.FeschetF.Optimal multiple intervals discretization of continuous attributes for supervised learning. In Proceedings of the Third International Conference on Knowledge Discovery and Data Mining (KDD-97), Newport Beach, California, USA, 1997, pp. 295–298 (AAAI Press, Menlo Park, CA).
17.
FrankE.WittenI. H.Making better use of global discretization. In Proceedings of the 15th International Conference on Machine Learning, Madison, Wisconsin, USA, 1998, pp. 152–160.
18.
TrautzschS.PernerP.Multi-interval discretization methods for decision tree learning. In Advances in pattern recognition (Eds AminA.DoriD.PudilP.FreemanH.), Lecture Notes in Computer Science, 1998, Vol. 1451, pp. 475–482 (Springer-Verlag, Germany).
19.
AnA.CerconeN.Discretisation of continuous attributes for learning classification rules. In Proceedings of the Third Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD-99), Kyoto, Japan, 1999, pp. 509–514.
20.
PengYIntelligent condition monitoring using fuzzy inductive learning. J. Intell. Manuf., 2004, 15, 373–380.
21.
CatlettJ.Megainduction: machine learning on very large databases. PhD Thesis, School of Computer Science, University of Technology, Sydney, Australia, 1991.
22.
VenturaD.On discretization as a preprocessing step for supervised learning models. MSc Thesis, Brigham Young University, Provo, UT, 1995.
23.
PhamD. T.AfifyA. A.SRI: A scalable rule induction algorithm. Proc. Instn Mech. Engrs, Part C: J. Mechanical Engineering Science, 2004 (submitted for publication).
24.
ChingJ. Y.WongA. K. C.ChanC. C.Class dependent discretization for inductive learning from continuous and mixed-mode data. IEEE Trans. Pattern Anal. Machine Intell., 1995, 17 (7), 641–651.
25.
DoughertyJ.KohaviR.SahamiM.Supervised and unsupervised discretization of continuous features. In Proceedings of the 12th International Conference on Machine Learning, Lake Tahoe City, CA, 1995, pp. 194–202 (Morgan Kaufmann, San Francisco, CA, USA).
26.
LiuH.HussainF.TanC. L.DashM.Discretization: An enabling technique. Data Min. Knowl. Disc., 2002, 6, 393–423.
27.
QuinlanJ. R.C4.5: Programs for machine learning, 1993 (Morgan Kaufmann, San Mateo, CA, USA).
28.
KurganL.CiosK. J.Discretisation algorithm that uses class-attribute independence maximization. In Proceedings of the 17th International Conference on Artificial Intelligence (ICAI 2001), Las Vegas, Nevada, 2001, pp. 980–987.
29.
WongA. K. C.ChiuD. K. Y.Synthesizing statistical knowledge from incomplete mixed-mode data. IEEE Trans. Pattern Anal. Machine Intell., 1987, 9 (6), 796–805.
30.
ChmielewskiM. R.Grzymala-BusseJ. W.Global discretization of continuous attributes as preprocessing for machine learning. In Proceedings of the Third International Workshop on Rough Sets and Soft Computing, San Jose, CA, 1994, pp. 294–301.
31.
KerberR.ChiMerge: Discretization of numeric attributes. In Proceedings of the 10th National Conference on Artificial Intelligence, San Jose, CA, 1992, 123–128.
32.
RicheldiM.RossottoM.Class-driven statistical discretization of continuous attributes. In Proceedings of the 8th European Conference on Machine Learning, Heraclion, Crete, Greece, 1995, pp. 335–338 (Springer-Verlag, Berlin).
33.
CatlettJ.On changing continuous attributes into ordered discrete attributes. In Proceedings of the European Working Session on Learning, Porto, Portugal, 1991, pp. 164–178 (Springer-Verlag, Berlin).
34.
FayyadU. M.IraniK. B.Multi-interval discretization of continuous-valued attributes for classification. In Proceedings of the 13th International Joint Conference on Artificial Intelligence, Chambery, France, 1993, pp. 1022–1027.
35.
PfahringerB.Compression-based discretization of continuous attributes. In Proceedings of the 12th International Conference on Machine Learning, Lake Tahoe City, CA, 1995, pp. 456–463 (Morgan Kaufmann, San Francisco, CA, USA).
36.
CerquidesJ.de MantarasLopez R.Proposal and empirical comparison of a parallelizable distance based discretization method. In Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, Newport Beach, CA, 1997, pp. 139–142.
37.
QuinlanJ. R.Induction of decision trees. Mach. Learn., 1986, 1, 81–106.
38.
QuinlanJ. R.RivestR. L.Inferring decision trees using Minimum Description Length principle. Inform. Comp., 1989, 80, 227–248.
39.
FayyadU. M.IraniK. B.On the handling of continuous-valued attributes in decision tree generation. Mach. Learn., 1992, 8, 87–102.
40.
HolteR. C.Very simple classification rules perform well on most commonly used data sets. Mach. Learn., 1993, 11, 63–90.
41.
MaassW.Efficient agnostic PAC-learning with simple hypotheses. In Proceedings of the 7th Annual ACM Conference on Computational Learning Theory, New Brunswick, NJ, USA, 1994, pp. 67–75.
42.
AuerP.HolteR. C.MaassW.Theory and application of agnostic PAC-learning with small decision trees. In Proceedings of the 12th International Conference on Machine Learning, Lake Tahoe City, CA, 1995, pp. 21–29 (Morgan Kaufmann, San Francisco, CA, USA).
43.
FultonT.KasifS.SalzbergS.Efficient algorithms for finding multi-way splits for decision trees. In Proceedings of the 12th International Conference on Machine Learning, Lake Tahoe City, CA, 1995, pp. 244–251 (Morgan Kaufmann, San Francisco, CA, USA).
44.
BirkendorfAOn fast and simple algorithms for finding maximal subarrays and applications on computational learning theory. Lecture Notes in Artificial Intell., 1997, Vol. 1208, pp. 198–209 (Springer-Verlag, Heidelberg).
45.
ElomaaT.RousuJSpeeding up the search for optimal partitions. Lecture Notes in Artificial Intell., 1999, Vol. 1704, pp. 89–97 (Springer-Verlag, Heidelberg).
46.
RousuJ.Efficient range partitioning in classification learning. PhD Thesis, Department of Computer Science, University of Helsinki, Finland, 2001.
47.
CaiZTechnical aspects of data mining. PhD Thesis, University of Wales Cardiff, Cardiff, UK, 2001.
48.
GoodI. J.The estimation of probabilities: an essay on modern Bayesian methods, 1965 (MIT Press, Cambridge, MA, USA).
49.
KohaviR.SahamiM.Error-based and entropy-based discretization of continuous features. In Proceedings of the Second International Conference on Knowledge Discovery in Databases, Montreal, Canada, 1996, pp. 114–119 (AAAI Press, Menlo Park, CA, USA).
50.
BlakeC. L.MerzC. J.UCI repository of machine learning databases. University of California, Department of Information and Computer Science, Irvine, CA, 1998, available from: http://www.ics.uci.edu/mlearn/MLRepository.html.
51.
QuinlanJ. R.Improved use of continuous attributes in C4.5. J. Artif. Intell. Res., 1996, 4, 77–90.
52.
TrautzschS.PernerP.A comparison of different multi-interval discretization methods for decision tree learning. Institute of Computer Vision and Applied Computer Science, Germany, 1996, available from: http://lareina.imise.uni-leipzig.de/∼perner.
53.
ElomaaT.RousuJGeneral and efficient multisplitting of numerical attributes. Mach. Learn., 1999, 36 (3), 201–244.
54.
QuinlanJ. R.Learning efficient classification procedures and their application to chess endgames. In Machine Learning: An Artificial Intelligence Approach (Eds MichalskiR. S.CarbonellJ. G.MitchellT. M.), 1983, Vol. 1, pp. 463–482 (Tioga Publishing Co., Palo Alto, CA, USA).
55.
BerkaP.BruhaI.Empirical comparison of various discretization procedures. Int. J. Pattern Recog. Artif. Intel., 1998, 12 (7), 1017–1032.
