Dataset Used for the Split Calculations
This example builds a Decision Tree for the target Play Tennis using four input attributes: Outlook, Temp, Humidity, and Wind.
| Day |
Outlook |
Temp |
Humidity |
Wind |
Play
Tennis |
| D1 | Sunny | Hot | High | Weak | No |
| D2 | Sunny | Hot | High | Strong | No |
| D3 | Overcast | Hot | High | Weak | Yes |
| D4 | Rain | Mild | High | Weak | Yes |
| D5 | Rain | Cool | Normal | Weak | Yes |
| D6 | Rain | Cool | Normal | Strong | No |
| D7 | Overcast | Cool | Normal | Strong | Yes |
| D8 | Sunny | Mild | High | Weak | No |
| D9 | Sunny | Cool | Normal | Weak | Yes |
| D10 | Rain | Mild | Normal | Weak | Yes |
| D11 | Sunny | Mild | Normal | Strong | Yes |
| D12 | Overcast | Mild | High | Strong | Yes |
| D13 | Overcast | Hot | Normal | Weak | Yes |
| D14 | Rain | Mild | High | Strong | No |
Class Count at Root
There are 9 Yes records and 5 No records, so the root set is:
S = [9+, 5-]
Core Formulas
Entropy(S) = -p(+) log2 p(+) - p(-) log2 p(-)
Gain(S, A) = Entropy(S) - Σ (|Sv| / |S|) Entropy(Sv)
A split is preferred when it produces the highest information gain.
Step 1: Entropy of the Full Dataset
Entropy(S) = -(9/14) log2(9/14) - (5/14) log2(5/14)
= 0.94
Step 2: Compare All Candidate Root Attributes
Outlook
Values(Outlook) = {Sunny, Overcast, Rain}
Ssunny = [2+, 3-] Entropy(Ssunny) = 0.971
Sovercast = [4+, 0-] Entropy(Sovercast) = 0
Srain = [3+, 2-] Entropy(Srain) = 0.971
Gain(S, Outlook) = 0.94 - (5/14)(0.971) - (4/14)(0) - (5/14)(0.971)
= 0.2464
Temp
Values(Temp) = {Hot, Mild, Cool}
Shot = [2+, 2-] Entropy(Shot) = 1.0
Smild = [4+, 2-] Entropy(Smild) = 0.9183
Scool = [3+, 1-] Entropy(Scool) = 0.8113
Gain(S, Temp) = 0.94 - (4/14)(1.0) - (6/14)(0.9183) - (4/14)(0.8113)
= 0.0289
Humidity
Values(Humidity) = {High, Normal}
Shigh = [3+, 4-] Entropy(Shigh) = 0.9852
Snormal = [6+, 1-] Entropy(Snormal) = 0.5916
Gain(S, Humidity) = 0.94 - (7/14)(0.9852) - (7/14)(0.5916)
= 0.1516
Wind
Values(Wind) = {Strong, Weak}
Sstrong = [3+, 3-] Entropy(Sstrong) = 1.0
Sweak = [6+, 2-] Entropy(Sweak) = 0.8113
Gain(S, Wind) = 0.94 - (6/14)(1.0) - (8/14)(0.8113)
= 0.0478
Root Decision
Outlook has the highest gain:
Gain(S, Outlook) = 0.2464
So the root node becomes Outlook.
Step 3: Solve the Sunny Branch
The records under Sunny are D1, D2, D8, D9, and D11.
Ssunny = [2+, 3-]
Entropy(Ssunny) = -(2/5) log2(2/5) - (3/5) log2(3/5)
= 0.97
Sunny Branch: Split by Temp
Shot = [0+, 2-] Entropy(Shot) = 0
Smild = [1+, 1-] Entropy(Smild) = 1.0
Scool = [1+, 0-] Entropy(Scool) = 0
Gain(Ssunny, Temp) = 0.97 - (2/5)(0) - (2/5)(1.0) - (1/5)(0)
= 0.570
Sunny Branch: Split by Humidity
Shigh = [0+, 3-] Entropy(Shigh) = 0
Snormal = [2+, 0-] Entropy(Snormal) = 0
Gain(Ssunny, Humidity) = 0.97 - (3/5)(0) - (2/5)(0)
= 0.97
Sunny Branch: Split by Wind
Sstrong = [1+, 1-] Entropy(Sstrong) = 1.0
Sweak = [1+, 2-] Entropy(Sweak) = 0.9183
Gain(Ssunny, Wind) = 0.97 - (2/5)(1.0) - (3/5)(0.9183)
= 0.0192
Highest gain in the Sunny branch comes from Humidity.
Sunny → Humidity
High → No
Normal → Yes
Step 4: Solve the Remaining Root Branches
Overcast Branch
The Overcast records are D3, D7, D12, and D13.
Sovercast = [4+, 0-]
Entropy(Sovercast) = 0
This branch is already pure, so it directly becomes:
Overcast → Yes
Rain Branch
The records under Rain are D4, D5, D6, D10, and D14.
Srain = [3+, 2-]
Entropy(Srain) = -(3/5) log2(3/5) - (2/5) log2(2/5)
= 0.97
Rain Branch: Split by Temp
Shot = [0+, 0-] Entropy(Shot) = 0
Smild = [2+, 1-] Entropy(Smild) = 0.9183
Scool = [1+, 1-] Entropy(Scool) = 1.0
Gain(Srain, Temp) = 0.97 - (0/5)(0) - (3/5)(0.9183) - (2/5)(1.0)
= 0.0192
Rain Branch: Split by Humidity
Shigh = [1+, 1-] Entropy(Shigh) = 1.0
Snormal = [2+, 1-] Entropy(Snormal) = 0.9183
Gain(Srain, Humidity) = 0.97 - (2/5)(1.0) - (3/5)(0.9183)
= 0.0192
Rain Branch: Split by Wind
Sstrong = [0+, 2-] Entropy(Sstrong) = 0
Sweak = [3+, 0-] Entropy(Sweak) = 0
Gain(Srain, Wind) = 0.97 - (2/5)(0) - (3/5)(0)
= 0.97
Highest gain in the Rain branch comes from Wind.
Rain → Wind
Strong → No
Weak → Yes
Final Tree Structure
Root: Outlook
If Outlook = Sunny, split on Humidity
If Humidity = High, predict No
If Humidity = Normal, predict Yes
If Outlook = Overcast, predict Yes
If Outlook = Rain, split on Wind
If Wind = Strong, predict No
If Wind = Weak, predict Yes
Equivalent Rule Set
1. If Outlook = Overcast, then Play Tennis = Yes
2. If Outlook = Sunny and Humidity = High, then Play Tennis = No
3. If Outlook = Sunny and Humidity = Normal, then Play Tennis = Yes
4. If Outlook = Rain and Wind = Strong, then Play Tennis = No
5. If Outlook = Rain and Wind = Weak, then Play Tennis = Yes
What This Calculation Shows
- Entropy measures impurity at the current node.
- Information gain measures how much impurity is reduced after a split.
- The tree always chooses the attribute with the highest gain at each step.
- Pure child nodes have entropy 0, so they become terminal leaves.
Interpretation: the final tree is small because two second-level splits are enough to make every branch pure.