Repositories list

• inference

  Public
  Reference implementations of MLPerf™ inference benchmarks

  benchmarkmachine-learning

  Python

  •

  Apache License 2.0

  •536•1.2k•157•13•Updated Dec 4, 2024Dec 4, 2024

• dynabench

  Public
  Python

  •

  MIT License

  •16•20•7•5•Updated Dec 4, 2024Dec 4, 2024

• algorithmic-efficiency

  Public
  MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.

  Python

  •

  Apache License 2.0

  •71•336•33•8•Updated Dec 4, 2024Dec 4, 2024

• mlperf-automations

  Public
  This repository contains the automations and scripts used to automate MLPerf benchmarks (mainly MLPerf inference for now)

  automationmlperf

  Python

  •

  Apache License 2.0

  •24•0•45•1•Updated Dec 4, 2024Dec 4, 2024

• cm4mlops

  Public
  A collection of portable, reusable and cross-platform automation recipes (CM scripts) to make it easier to build and benchmark AI systems across diverse models, data sets, software and hardware

  workflowdevopsmachine-learning+ 6automationcross-platformreusableworkflow-automationmlopsmlperf

  Python

  •

  Apache License 2.0

  •24•15•10•0•Updated Dec 4, 2024Dec 4, 2024

• mlperf_inference_test_submissions_v5.0

  Public
  Mermaid

  •1•0•0•0•Updated Dec 4, 2024Dec 4, 2024

• croissant

  Public
  Croissant is a high-level format for machine learning datasets that brings together four rich layers.

  machine-learningschema-orgjson-ld+ 1datasets

  Python

  •

  Apache License 2.0

  •42•459•117•16•Updated Dec 4, 2024Dec 4, 2024

• modelbench

  Public
  Run safety benchmarks against AI models and view detailed reports showing how well they performed.

  Python

  •

  Apache License 2.0

  •11•62•153•4•Updated Dec 3, 2024Dec 3, 2024

• logging

  Public
  MLPerf™ logging library

  Python

  •

  Apache License 2.0

  •46•31•40•2•Updated Dec 3, 2024Dec 3, 2024

• ck

  Public
  Collective Knowledge (CK) and Collective Mind (CM): educational community projects to learn how to run AI, ML and other emerging workloads in a more efficient and cost-effective way across diverse models, datasets, software and hardware using MLPerf and CM automations

  educationautomationoptimization+ 16best-practicescollaborationcmmodularityworkflowsportabilityreusability+ 9

  Python

  •

  Apache License 2.0

  •118•609•12•0•Updated Dec 3, 2024Dec 3, 2024

• inference_policies

  Public
  Issues related to MLPerf™ Inference policies, including rules and suggested changes

  Apache License 2.0

  •52•57•18•3•Updated Dec 3, 2024Dec 3, 2024

• mobile_app_open

  Public
  Mobile App Open

  C++

  •

  Apache License 2.0

  •23•47•31•4•Updated Dec 3, 2024Dec 3, 2024

• datasets-knowledge-graphs

  Public
  Apache License 2.0

  •0•0•0•0•Updated Dec 3, 2024Dec 3, 2024

• medperf

  Public
  An open benchmarking platform for medical artificial intelligence using Federated Evaluation.

  Python

  •

  Apache License 2.0

  •32•146•54•23•Updated Dec 2, 2024Dec 2, 2024

• cm4abtf

  Public
  CM interface and automation recipes for ABTF

  workflowautomation

  Python

  •

  Apache License 2.0

  •4•2•2•0•Updated Nov 29, 2024Nov 29, 2024

• common-crawl-dmlr

  Public
  Apache License 2.0

  •0•0•0•1•Updated Nov 26, 2024Nov 26, 2024

• tiny

  Public
  MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers

  C

  •

  Apache License 2.0

  •89•360•13•5•Updated Nov 26, 2024Nov 26, 2024

• GaNDLF

  Public
  A generalizable application framework for segmentation, regression, and classification using PyTorch

  machine-learningframeworkdeep-learning+ 10regressionpytorchmedical-imagingclassificationsegmentationdata-augmentationbiomedical-image-processing+ 3

  Python

  •

  Apache License 2.0

  •80•166•16•2•Updated Nov 21, 2024Nov 21, 2024

• cm4mlperf-inference

  Public
  Apache License 2.0

  •36•1•2•2•Updated Nov 21, 2024Nov 21, 2024

• training_policies

  Public
  Issues related to MLPerf™ training policies, including rules and suggested changes

  Python

  •

  Apache License 2.0

  •66•93•109•3•Updated Nov 21, 2024Nov 21, 2024

• policies

  Public
  General policies for MLPerf™ including submission rules, coding standards, etc.

  Python

  •

  Apache License 2.0

  •54•28•35•38•Updated Nov 19, 2024Nov 19, 2024

• chakra

  Public
  Repository for MLCommons Chakra schema and tools

  Python

  •

  Apache License 2.0

  •39•71•22•10•Updated Nov 16, 2024Nov 16, 2024

• submissions_algorithms

  Public
  Apache License 2.0

  •1•1•1•1•Updated Nov 13, 2024Nov 13, 2024

• training_results_v4.1

  Public
  This repository contains the results and code for the MLPerf™ Training v4.1 benchmark.

  Python

  •2•3•1•0•Updated Nov 13, 2024Nov 13, 2024

• power-dev

  Public
  Dev repo for power measurement for the MLPerf™ benchmarks

  Python

  •

  Apache License 2.0

  •25•18•10•4•Updated Nov 12, 2024Nov 12, 2024

• cla-bot

  Public
  TypeScript

  •

  Apache License 2.0

  •2•0•0•0•Updated Nov 7, 2024Nov 7, 2024

• inference_results_v4.0

  Public
  This repository contains the results and code for the MLPerf™ Inference v4.0 benchmark.

  Apache License 2.0

  •17•10•5•5•Updated Nov 7, 2024Nov 7, 2024

• GaNDLF-Synth

  Public
  Extension for GaNDLF [gandlf.org] to enable synthesis.

  medical-imagingsynthesis

  Python

  •

  Apache License 2.0

  •2•4•6•0•Updated Nov 7, 2024Nov 7, 2024

• template

  Public template
  Apache License 2.0

  •0•0•0•0•Updated Nov 1, 2024Nov 1, 2024

• training

  Public
  Reference implementations of MLPerf™ training benchmarks

  benchmarkmachine-learning

  Python

  •

  Apache License 2.0

  •562•1.6k•18•42•Updated Oct 17, 2024Oct 17, 2024