In this article we mainly define the information entropy [3], [11] and prove some its basic properties. First, we discuss some properties on four kinds of transformation functions between vector and matrix. The transformation functions are LineVec2Mx, ColVec2Mx, Vec2DiagMx and Mx2FinS. Mx2FinS is a horizontal concatenation operator for a given matrix, treating rows of the given matrix as finite sequences, yielding a new finite sequence by horizontally joining each row of the given matrix in order to index. Then we define each concept of information entropy for a probability sequence and two kinds of probability matrices, joint and conditional, that are defined in article [25]. Further, we discuss some properties of information entropy including Shannon's lemma, maximum property, additivity and super-additivity properties.
@article{bwmeta1.element.doi-10_2478_v10037-007-0012-9,
author = {Bo Zhang and Yatsuka Nakamura},
title = {Definition and some Properties of Information Entropy},
journal = {Formalized Mathematics},
volume = {15},
year = {2007},
pages = {111-119},
language = {en},
url = {http://dml.mathdoc.fr/item/bwmeta1.element.doi-10_2478_v10037-007-0012-9}
}
Bo Zhang; Yatsuka Nakamura. Definition and some Properties of Information Entropy. Formalized Mathematics, Tome 15 (2007) pp. 111-119. http://gdmltest.u-ga.fr/item/bwmeta1.element.doi-10_2478_v10037-007-0012-9/
[1] Grzegorz Bancerek. The ordinal numbers. Formalized Mathematics, 1(1):91-96, 1990.
[2] Grzegorz Bancerek and Krzysztof Hryniewiecki. Segments of natural numbers and finite sequences. Formalized Mathematics, 1(1):107-114, 1990.
[3] P. Billingsley. Ergodic Theory and Information. John Wiley & Sons, 1964. | Zbl 0184.43301
[4] Czesław Byliński. Binary operations. Formalized Mathematics, 1(1):175-180, 1990.
[5] Czesław Byliński. Finite sequences and tuples of elements of a non-empty sets. Formalized Mathematics, 1(3):529-536, 1990.
[6] Czesław Byliński. Functions and their basic properties. Formalized Mathematics, 1(1):55-65, 1990.
[7] Czesław Byliński. Functions from a set to a set. Formalized Mathematics, 1(1):153-164, 1990.
[8] Czesław Byliński. Some basic properties of sets. Formalized Mathematics, 1(1):47-53, 1990.
[9] Czesław Byliński. The sum and product of finite sequences of real numbers. Formalized Mathematics, 1(4):661-668, 1990.
[10] Agata Darmochwał. The Euclidean space. Formalized Mathematics, 2(4):599-603, 1991.
[11] Shigeichi Hirasawa. Information Theory. Baifukan CO., 1996.
[12] Katarzyna Jankowska. Matrices. Abelian group of matrices. Formalized Mathematics, 2(4):475-480, 1991.
[13] Artur Korniłowicz. On the real valued functions. Formalized Mathematics, 13(1):181-187, 2005.
[14] Jarosław Kotowicz. Real sequences and basic operations on them. Formalized Mathematics, 1(2):269-272, 1990.
[15] Rafał Kwiatek. Factorial and Newton coefficients. Formalized Mathematics, 1(5):887-890, 1990.
[16] Yatsuka Nakamura, Nobuyuki Tamaura, and Wenpai Chang. A theory of matrices of real elements. Formalized Mathematics, 14(1):21-28, 2006.
[17] Library Committee of the Association of Mizar Users. Binary operations on numbers. To appear in Formalized Mathematics.
[18] Konrad Raczkowski and Andrzej Nedzusiak. Real exponents and logarithms. Formalized Mathematics, 2(2):213-216, 1991.
[19] Yasunari Shidama. The Taylor expansions. Formalized Mathematics, 12(2):195-200, 2004.
[20] Andrzej Trybulec. Subsets of complex numbers. To appear in Formalized Mathematics.
[21] Andrzej Trybulec. Tarski Grothendieck set theory. Formalized Mathematics, 1(1):9-11, 1990.
[22] Wojciech A. Trybulec. Vectors in real linear space. Formalized Mathematics, 1(2):291-296, 1990.
[23] Zinaida Trybulec. Properties of subsets. Formalized Mathematics, 1(1):67-71, 1990.
[24] Edmund Woronowicz. Relations and their basic properties. Formalized Mathematics, 1(1):73-83, 1990.
[25] Bo Zhang and Yatsuka Nakamura. The definition of finite sequences and matrices of probability, and addition of matrices of real elements. Formalized Mathematics, 14(3):101-108, 2006.