How does Khiops handle target value grouping, and why is it useful for classification tasks?

[lucaurelien]

This discussion is based on a question received via our contact form:
“Khiops has a feature to group target classes for classification tasks. How does this work, and in what scenarios is it most beneficial?”

[lucaurelien]

Khiops’ target value grouping functionality is designed to address challenges when dealing with a large number of target classes. When the number of classes is high, it can be difficult to discriminate between all classes, especially with insufficient number of instances (i.e. with sparse input data).

To address this, Khiops reduces data sparsity by grouping target classes. In practice:

1. Univariate Preprocessing: Target class groupings occur during the preprocessing step. Khiops determines the optimal grouping of target values for each variable based on the correlations between the data. The grouped classes are treated as distinguishable from other groups, but classes within the same group are considered indistinguishable.
2. Class-level predictions: The Selective Naive Bayes (SNB) predictor uses the univariate preparations to make precise, class-by-class predictions. While groupings simplify univariate preprocessing, the final model still predicts probabilities for individual classes, not groups.

Example: Grouped Target Probabilities

In the following example, the explanatory variable x has two distinct values {v1, v2}, and the target variable y has eight classes {A, B, C, D, E, F, G, H}, which are clustered into three groups during preprocessing.

For x = v1:

- P(y ∈ {A, B} | x = v1) = 0.1  
- P(y ∈ {C, D, E, F} | x = v1) = 0.75 
- P(y ∈ {G, H} | x = v1) = 0.15

For x = v2:

P(y ∈ {A, B} | x = v2) = 0.6  
P(y ∈ {C, D, E, F} | x = v2) = 0.1
P(y ∈ {G, H} | x = v2) = 0.3

In this example, the classifier identifies that distinguishing between groups ({A, B}, {C, D, E, F}, {G, H}) is feasible, but it cannot reliably separate classes within the same group during univariate preparation.

Because this grouping is done independently for each explanatory variable, a different grouping could occur for another explanatory variable z. For example:

For z = v0:

P(y ∈ {A, D, H} | z = v0) = 0.6  
P(y ∈ {B, C, E, F, G} | z = v0) = 0.4

The SNB predictor later uses this variable-specific grouped information from all explanatory variables to make precise, individual class predictions.

Additional Insights

In some cases, it may be beneficial to globally reduce the number of target classes for the entire problem. The univariate preparation reports generated by Khiops can help data miners identify frequently occurring groupings, guiding decisions about merging or removing target classes altogether.