Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Template for adding a new model #21

Merged
merged 10 commits into from
Oct 24, 2024
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# The MEDS Dynamic Extensible Validation (MEDS-DEV) Benchmark: Re-thinking Reproducibility and Validation in ML for Health

> \[!WARNING\]
> MEDS-DEV currently only supports binary classification tasks.

This repository contains the dataset, task, model training recipes, and results for the MEDS-DEV benchmarking
effort for EHR machine learning.

Expand Down Expand Up @@ -131,8 +134,6 @@ Note this package currently supports binary classification only.

### To Add a Model

TODO

### To Add a Dataset

TODO
Expand Down
4 changes: 4 additions & 0 deletions src/MEDS_DEV/templates/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# README templates

This directory contains README templates for contributing to the MEDS-DEV repository with new
models, datasets, tasks, and results.
Empty file.
67 changes: 67 additions & 0 deletions src/MEDS_DEV/templates/model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# \[New model\] \*\*Add your model name here\*\*

> \[!Warning\]
> MEDS-DEV currently only supports binary classification tasks and therefore only accepts binary
> classification models for now.

## Description

Provide a brief description of your model, including:

- Model architecture and key components
- Intended purpose and target tasks
- Any notable features or innovations
- Limitations or constraints

## Computational Requirements

Provide information about the computational requirements of your model, including:

- Hardware specifications used during testing (e.g., GPU type, memory)
- Computational resources required for training and inference
- Performance characteristics (e.g., training speed)

## Dependencies

List all dependencies required to run your model. For each dependency, provide:

- Name of the dependency
- Version number (if applicable)
- Any specific installation instructions (if needed)

Example:

- Python 3.8+
- TensorFlow 2.4.0
- NumPy 1.19.5
- Pandas 1.2.0

## Running the model

Please provide detailed instructions for running your model, including:

- Installation steps
- Command-line instructions or API usage
- Description of all parameters and their default values
- Troubleshooting tips (if applicable)

## Resources and links

Please provide the following:

- Link to the model's implementation repository
- Link to the model's documentation (if separate from the repository)
- Any relevant research papers or articles
- Links to pre-trained model weights (if applicable)
- Any additional resources that would be helpful for users

## Checklist

Please ensure your model conforms to the MEDS-DEV API by checking the following:

- [ ] It is compatible with the most recent version of the MEDS data format.
- [ ] Its outputs are compatible with the most recent version of the MEDS-evaluation schema.
- [ ] All required dependencies are clearly listed and versioned.
- [ ] The model can be run using the provided instructions without errors.
- [ ] The model documentation is complete and follows this template.
- [ ] (If applicable) The model has been tested on a sample dataset provided by MEDS-DEV.
Empty file.
Empty file added src/MEDS_DEV/templates/task.md
Empty file.
Loading