Skip to content

Commit

Permalink
Add prediction schema description.
Browse files Browse the repository at this point in the history
  • Loading branch information
kamilest committed Oct 11, 2024
1 parent 30d5631 commit 039b421
Showing 1 changed file with 31 additions and 2 deletions.
33 changes: 31 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,11 +111,40 @@ conforming to MEDS binary classification prediction schema:
./MEDS-DEV/src/MEDS_DEV/helpers/generate_predictions.sh $MEDS_ROOT_DIR $TASK_NAME
```

In order to work with the evaluation package (see the next section),
the model's outputs must conform to the _prediction schema_:

```python
prediction = pa.schema(
[
("subject_id", pa.int64()),
("prediction_time", pa.timestamp("us")),
("boolean_value", pa.bool_()),
("predicted_boolean_value", pa.bool_()),
("predicted_boolean_probability", pa.float64()),
]
)

Prediction = TypedDict(
"Prediction",
{
"subject_id": int,
"prediction_time": datetime.datetime,
"boolean_value": bool,
"predicted_boolean_value": bool,
"predicted_boolean_probability": bool,
},
total=False,
)
```

TODO: make the predicted values/probabilities optional and evaluate metrics based on availability of these
values

### Evaluate the model

You can use the `meds-evaluation` package by running `meds-evaluation-cli` and providing the path to
predictions
dataframe as well as the output directory. For example,
predictions dataframe as well as the output directory. For example,

```bash
meds-evaluation-cli \
Expand Down

0 comments on commit 039b421

Please sign in to comment.