Collective Knowledge (CK) in an educational project to help researchers and engineers automate their repetitive, tedious and time-consuming tasks to build, run, benchmark and optimize AI, ML and other applications and systems across diverse and continuously changing models, data, software and hardware.
CK consists of several sub-projects:
-
Collective Mind framework (CM) - a very lightweight Python-based framework with minimal dependencies to help users implement, share and reuse cross-platform automation recipes to build, benchmark and optimize applications on any platform with any software and hardware.
-
CM4MLOPS - a collection of portable, extensible and technology-agnostic automation recipes with a human-friendly interface (aka CM scripts) to unify and automate all the manual steps required to compose, run, benchmark and optimize complex ML/AI applications on diverse platforms with any software and hardware: see online catalog at CK playground, online MLCommons catalog
-
CM4ABTF - a unified CM interface and automation recipes to run automotive benchmark across different models, data sets, software and hardware from different vendors.
-
CMX (the next generation of CM and CM4MLOps) - we are developing the next generation of CM to make it simpler and more flexible based on user feedback. Please follow this project here.
-
Collective Knowledge Playground - a unified platform to list CM scripts similar to PYPI, aggregate AI/ML Systems benchmarking results in a reproducible format with CM workflows, and organize public optimization challenges and reproducibility initiatives to co-design more efficient and cost-effiective software and hardware for emerging workloads.
-
Artifact Evaluation - automating artifact evaluation and reproducibility initiatives at ML and systems conferences.
- Copyright (c) 2021-2024 MLCommons
- Copyright (c) 2014-2021 cTuning foundation
- CM/CM4Research: Grigori Fursin
- CM4MLOps: Arjun Suresh and Anandhu Sooraj
- CMX (the next generation of CM) Grigori Fursin
If you found the CM automation framework helpful, kindly reference this article: [ ArXiv ], [ BibTex ].
To learn more about the motivation behind CK and CM technology, please explore the following presentations:
- "Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments": [ ArXiv ]
- ACM REP'23 keynote about the MLCommons CM automation framework: [ slides ]
- ACM TechTalk'21 about Collective Knowledge project: [ YouTube ] [ slides ]
- CM installation GUI
- CM Getting Started Guide and FAQ
- Full documentation
- CM development tasks
- CM and CK history
The open-source Collective Knowledge project (CK, CM, CM4MLOps/CM4MLPerf, CM4Research and CMX) was created by Grigori Fursin and sponsored by cTuning.org, OctoAI and HiPEAC. Grigori donated CK to MLCommons to benefit the community and to advance its development as a collaborative, community-driven effort. We thank MLCommons and FlexAI for supporting this project, as well as our dedicated volunteers and collaborators for their feedback and contributions!