Compression of random binary data

[Steve D'Aprano]

On Sun, 29 Oct 2017 02:31 pm, Gregory Ewing wrote:

> Steve D'Aprano wrote:
>> I don't think that's right. The entropy of a single message is a
>> well-defined quantity, formally called the self-information.
>> 
>> https://en.wikipedia.org/wiki/Self-information
> 
> True, but it still depends on knowing (or assuming) the
> probability of getting that particular message out of
> the set of all possible messages.

Indeed.


> This is *not* what danceswithnumbers did when he
> calculated the "entropy" of his example bit sequences.
> He didn't define the set they were drawn from or
> what their probabilities were.

I'm not defending or supporting Danceswithnumbers in his confusion. I'm just
pointing out that an entropy-like measure of information for individual
messages does exist, and people do often call it "entropy".



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.