Information theory

Many computational musicology analyses rely on probabilistic modeling and information theory. HumdrumR includes functions to make these sorts of analyses quick and easy. These functions are closely connected to our distribution functions, which can be used to calculate/estimate the probability of data observations.

Details

The most fundamental tools of information theory are statistics that characterize probability distributions. Thus, they are descriptive statistics, which describe a distribution (usually, the distribution of values in your data) using a single number. Such information-theoretic descriptive statistics can be computed using the entropy() (joint or conditional entropy), xentropy() (cross entropy), kld() (Kullback–Leibler divergence), and mutual() (mutual information) functions. In contrast, other information theory metrics are calculated "point-wise": one value for each data observation. Our point-wise information theory functions are data.table::like() (likelihood), info() (information content), pentropy() (pointwise conditional entropy), and pmutual() (pointwise mutual information).

Note that all of these functions calculate or utilize empirical statistics—i.e., they describe your data. They are not (necessarily) representative of the "true" information content in real music. They may be used as estimates of the "true" entropy of music we study, but this assumes that our sample is representative and that our probabilistic models make sense (i.e., make valid assumptions).