When an average tells the wrong story
숫자가 틀리지 않아도, 비교는 틀릴 수 있다.
Imagine that you are choosing between two puzzle games. A review says that players finish Game B more often than Game A. That sounds like a useful measure of difficulty. But before drawing a conclusion, you ask a second question: did the two games attract the same kinds of players?
Here is a made-up example. Each game offers easy and hard levels. In Game A, nine of ten players finish an easy level, and thirty of one hundred finish a hard level. Its success rates are therefore ninety percent and thirty percent. In Game B, eighty of one hundred finish an easy level, while two of ten finish a hard level. Its rates are eighty percent and twenty percent.
Within both categories, Game A has the higher success rate. However, add the groups together. Game A has thirty-nine successes out of one hundred and ten attempts. Game B has eighty-two successes out of the same number of attempts. The overall rate is about thirty-five percent for A and seventy-five percent for B. The order has reversed.
No one changed the numbers. The difference comes from the mixture of attempts. Most players in Game A tried hard levels. Most players in Game B tried easy ones. The overall rates combine performance with the distribution of difficulty. They answer a different question from the rates within each category.
This kind of reversal is called Simpson’s paradox. A pattern found in separate groups can reverse when the groups are combined. It is a warning about interpretation, rather than evidence that arithmetic is unreliable. A summary can be correct and still hide information that matters to a decision.
For a game developer, the next step would depend on the question. If you want to describe what happened to all players, the overall rate is useful. If you want to compare difficulty for similar attempts, the categories may be more informative. Neither view automatically proves what would happen if the same player switched games. A causal claim needs more than a comparison of percentages.
Try writing your own review of these imaginary games. “B had more completed attempts overall” is supported by the example. “B was easier for every kind of player” is not. The second sentence quietly changes the claim. It moves from a result for a mixed population to a claim about individual experience.
An average is a compact answer. Before trusting it, make the question equally clear. Ask who is included, how the groups differ, and which comparison you actually need. Learning to read a number means learning to notice the choices behind it.
이 글의 참고 자료
Stanford Encyclopedia of Philosophy · Simpson’s Paradox ↗Microsoft · Software Testing Paradoxes ↗공개 자료를 참고해 새로 작성한 학습용 글입니다. 비유와 가상의 예는 설명을 위한 구성입니다.