OrdinalEncoder handle encoded_missing_value and unknown_value
#1132
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xadupre
approved these changes
Nov 14, 2024
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
LLukas22
force-pushed
the
ordinal_encoded_missing_value
branch
from
efc8da3
to
4c23ba8
Compare
Signed-off-by: Lukas Kreussel <[email protected]>
Signed-off-by: Lukas Kreussel <[email protected]>
Signed-off-by: Lukas Kreussel <[email protected]>
LLukas22
force-pushed
the
ordinal_encoded_missing_value
branch
from
4c23ba8
to
e7e8996
Compare
Signed-off-by: Lukas Kreussel <[email protected]>
|
Re-signed my commits, this should be good to go now 👍 |
xadupre
approved these changes
Nov 14, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the current version of
sklearn-onnx, theencoded_missing_valueandunknown_valueparameters of theOrdinalEncoderin scikit-learn are not properly handled. Specifically, these parameters are ignored during the ONNX model conversion.For example, if we create an
OrdinalEncoderwithencoded_missing_valueset to42and fit it on the following data:np.array([["a"], ["b"], ["c"], ["d"], [np.nan]], dtype=np.object_), scikit-learn produces the expected output:[0, 1, 2, 3, 42]. However, the converted ONNX model does not respect theencoded_missing_valueparameter, leading to an unexpected result:[0, 1, 2, 3, 4].Similarly, the
unknown_valueparameter is also ignored during conversion, which affects the expected output. To address this issue, thedefault_int64parameter of the ONNXLabelEncoderneeds to be set when anunknown_valueis specified.I have included some tests to demonstrate this behavior and implemented a simple fix to resolve the issue.