Template for adding a new model

.pre-commit-config.yaml

@@ -125,8 +125,4 @@ repos:
       - id: nbqa-isort
         args: ["--profile=black"]
       - id: nbqa-flake8
-        args:
-          [
-            "--extend-ignore=E203,E402,E501,F401,F841",
-            "--exclude=logs/*,data/*",
-          ]
+        args: ["--extend-ignore=E203,E402,E501,F401,F841", "--exclude=logs/*,data/*"]

README.md

@@ -1,5 +1,8 @@
 # The MEDS Dynamic Extensible Validation (MEDS-DEV) Benchmark: Re-thinking Reproducibility and Validation in ML for Health
 
+> \[!WARNING\]
+> MEDS-DEV currently only supports binary classification tasks.
+
 This repository contains the dataset, task, model training recipes, and results for the MEDS-DEV benchmarking
 effort for EHR machine learning.
 
@@ -131,8 +134,6 @@ Note this package currently supports binary classification only.
 
 ### To Add a Model
 
-TODO
-
 ### To Add a Dataset
 
 TODO

src/MEDS_DEV/templates/README.md

@@ -0,0 +1,4 @@
+# README templates
+
+This directory contains README templates for contributing to the MEDS-DEV repository with new
+models, datasets, tasks, and results.

src/MEDS_DEV/templates/model.md

@@ -0,0 +1,31 @@
+# \[New model\] \*\*Add your model name here\*\*
+
+> \[!Warning\]
+> MEDS-DEV currently only supports binary classification tasks and therefore only accepts binary
+> classification models for now.
+
+## Description
+
+Add a brief description of your model, including the details of its design and its intended purpose
+(what sorts of tasks is it designed for).
+
+## Dependencies
+
+Add any dependencies that your model may have.
+
+## Running the model
+
+Please provide a step-by-step instructions for running your model, including installation,
+details on the commands or a remote API, and any parameters.
+
+## Resources and links
+
+Add a link to the repository to your model's implementation and any other useful resources
+(e.g. documentation).
+
+## Checklist
+
+Please make sure your model conforms to the MEDS-DEV API, including
+
+- [ ] It is compatible with the most recent version of the MEDS data format.
+- [ ] Its outputs are compatible with the most recent version of the MEDS-DEV evaluation schema.