Lower probability signifies higher information. How come ?

"More unlikely that X would equal x , the more informative would be the message." Is this sounding a bit weird ? Well, its not !

It can be understood with an easy logic that if the no. of outcomes of an event is less, one needs to be more confident in selecting from that lot. More surety, symbolizes more information.

For e.g. in a basket of 12 fruits,there are 3 oranges and 9 apples.
Then,
P(orange) = 3/12
P(apple)=9/12

If I am selecting or getting an orange, that means I need to have more information at hand so that I can pick an item which was more rare or less probable. Choosing an apple means, I can be a bit liberal than in oranges case and I have some room for mistake, hence suggests lesser information than previous case.