Optimal estimator of hypothesis probability for data mining problems with small samples

Andrzej Piegat ; Marek Landowski

International Journal of Applied Mathematics and Computer Science, Tome 22 (2012), p. 629-645 / Harvested from The Polish Digital Mathematics Library

Résumé

The paper presents a new (to the best of the authors' knowledge) estimator of probability called the "Epₕ√2 completeness estimator" along with a theoretical derivation of its optimality. The estimator is especially suitable for a small number of sample items, which is the feature of many real problems characterized by data insufficiency. The control parameter of the estimator is not assumed in an a priori, subjective way, but was determined on the basis of an optimization criterion (the least absolute errors).The estimator was compared with the universally used frequency estimator of probability and with Cestnik's m-estimator with respect to accuracy. The comparison was realized both theoretically and experimentally. The results show the superiority of the Epₕ√2 completeness estimator over the frequency estimator for the probability interval pₕ ∈ (0.1, 0.9). The frequency estimator is better for pₕ ∈ [0, 0.1] and pₕ ∈ [0.9, 1].

Détail

Publié le : 2012-01-01

Zbl 1302.93206

EUDML-ID : urn:eudml:doc:244052

Bibliographie

[000] Ben-Haim, Y. (2006). Info-gap Decision Theory, Elsevier, Oxford/Amsterdam.

[001] Burdzy, K. (2009). The Search for Certainty. On the Clash of Science and Philosophy of Probability, World Scientific, Singapore. | Zbl 1330.60003

[002] Burdzy, K. (2011a). Blog on the book The Search for Certainty. On the Clash of Science and Philosophy of Probability, http://search4certainty.blogspot.com/.

[003] Burdzy, K. (2011b). Philosophy of probability, Website, http://www.math.washington.edu/∼burdzy/philosophy/.

[004] Carnap, R. (1952). Logical Foundations of Probability, University Press, Chicago, IL. | Zbl 0040.07001

[005] Cestnik, B. (1990). Estimating probabilities: A crucial task in machine learning, in L. Aiello (Ed.), ECAI'90, Pitman, London, pp. 147-149.

[006] Cestnik, B. (1991). Estimating Probabilities in Machine Learning, Ph.D. thesis, Faculty of Computer and Information Science, University of Ljubljana, Ljubljana.

[007] Chernoff, H. (1952). A measure of asymptotic efficiency for test of a hypothesis based on the sum of observations, Annals of Mathematical Statistics 23(4): 493-507. | Zbl 0048.11804

[008] Cichosz, P. (2000). Learning Systems, Wydawnictwa NaukowoTechniczne, Warsaw, (in Polish). | Zbl 0930.93048

[009] Cios, K. and Kurgan, L. (2001). SPECT heart data set, UCI Machine Learning Repository, http://archive.ics.uci.edu/ml/datasets/spect+heart.

[010] De Finetti, B. (1975). Theory of Probability: A Critical Introductory Treatment, Willey, London. | Zbl 0328.60003

[011] Dubois, D. and Prade, H. (1988). Possibility Theory, Plenum Press, New York/NY, London. | Zbl 0645.68108

[012] Furnkranz, J. and Flach, P.A. (2005). Roc'n'rule learning: Towards a better understanding of covering algorithms, Machine Learning 58(1): 39-77. | Zbl 1075.68071

[013] Hajek, A. (2010). Interpretations of probability, in E.N. Zalta, (Ed.), The Stanford Encyclopedia of Philosophy, http://plato.stanford.edu/entries/probability-interpret/.

[014] Khrennikov, A. (1999). Interpretations of Probability, Brill Academic Pub., Utrecht/Boston, MA. | Zbl 0998.81508

[015] Klirr, G.J. and Yuan, B. (1996). Fuzzy Sets, Fuzzy Logic, and Fuzzy Systems. Selected Papers by Lotfi Zadeh, World Scientific, Singapore.

[016] Laplace, P.S. (1814, English edition 1951). A Philosophical Essay on Probabilities, Dover Publication, New York/NY. | Zbl 11.0242.01

[017] Larose, D.T. (2010). Discovering Statistics, W.H. Freeman and Company, New York, NY.

[018] Piegat, A. (2011a). Uncertainty of probability, in K.T. Atanassov, M. Baczyński, J. Drewniak, J. Kacprzyk, M. Krawczak, E. Schmidt, M. Wygralak and S. Zadrożny (Eds.) Recent Advances in Fuzzy Sets, Intuitionistic Fuzzy Sets, Generalized Nets and Related Topics, Vol. I: Foundations, IBS PAN, Warsaw, pp. 159-173.

[019] Piegat, A. (2011b). Basic lecture on completeness interpretation of probability, Website, http://kmsiims.wi.zut.edu.pl/pobierz-pliki/cat view/47-publikacje.

[020] Polkowski, L. (2002). Rough Sets, Physica-Verlag, Heidelberg/New York, NY.

[021] Popper, K.R. (1957). The propensity interpretation of the calculus of probability and the quantum theory, in S. Korner (Ed.), Observation and Interpretation: A Symposium of Philosophers and Physicists, Butterworth Scientific Publications, London, pp. 65-70.

[022] Rocchi, P. (2003). The Structural Theory of Probability: New Ideas from Computer Science on the Ancient Problem of Probability Interpretation, Kluwer Academic/Plenum Publishers, New York, NY. | Zbl 1069.60001

[023] Rokach, L. and Maimon, O. (2008). Data Mining with Decision Trees: Theory and Applications, Machine Perception and Artificial Intelligence, Vol. 69, World Scientific Publishing, Singapore. | Zbl 1154.68098

[024] Shafer, G. (1976). A Mathematical Theory of Evidence, Princetown University Press, Princetown, NJ . | Zbl 0359.62002

[025] Siegler, R.S. (1976). Three aspects of cognitive development, Cognitive Psychology 8(4): 481-520.

[026] Siegler, R.S. (1994). Balance scale weight & distance database, UCI Machine Learning Repository, http://archive.ics.uci.edu/ml/datasets/balance+scale.

[027] Sulzmann, J.N. and Furnkranz, J. (2009). An empirical comparison of probability estimation techniques for probabilistic rules, in J. Gama, J. Santos Costa, A.M. Jorge and P. Brazdil (Eds.), Proceedings of the 12th International Conference on Discovery Science (DS-09), Springer-Verlag, Heidelberg/New York, NY, pp. 317-331.

[028] Sulzmann, J.N. and Furnkranz, J. (2010). Probability estimation and aggregation for rule learning, Technical Report TUDKE-201-03, Knowledge Engineering Group, TU Darmstadt, Darmstadt.

[029] von Mises, R. (1957). Probability, Statistics and the Truth, Macmillan, Dover/New York, NY. | Zbl 0080.33801

[030] Witten, I.H. and Frank, E. (2005). Data Mining, Elsevier, Amsterdam. | Zbl 1076.68555

[031] Zadeh, L.A. (1965). Fuzzy sets, Information and Control 8(3): 338-353. | Zbl 0139.24606

[032] Ziarko, W. (1999). Decision making with probabilistic decision tables, in N. Zhong (Ed.), New Directions in Rough Sets, Data Mining, and Granular-Soft Computing, Proceedings of the 7th International Workshop, RSFDGrC99, Yamaguchi, Japan, Springer-Verlag, Berlin/Heidelberg, New York, NY, pp. 463-471.