MLCommons CM v1.0.0 - the next generation of the MLCommons Collective Knowledge framework
This is the stable release of the MLCommons Collective Mind framework v1.0.1 with reusable and portable MLOps components - the next generation of the MLCommons Collective Knowledge framework developed to modularize AI/ML Systems and automate their benchmarking, optimization and design space exploration based on the mature MLPerf methodology.
After donating the CK framework to the MLCommons, we have been developing this portable workflow automation technology as a community effort within the open education workgroup to modularize MLPerf and make it easier to plug in real-world tasks, models, data sets, software and hardware from the cloud to the edge.
We are very glad to see that more than 80% of all performance results and more than 95% of all power results were automated by the MLCommons CK v2.6.1 in the latest MLPerf inference round thanks to submissions from Qualcomm, Krai, Dell, HPE and Lenovo!
We invite you to join our public workgroup to continue developing this portable workflow framework and reusable automation for MLOps and DevOps as a community effort to:
- develop an open-source educational toolkit to make it easier to plug any real-world ML & AI tasks, models, data sets, software and hardware into the MLPerf benchmarking infrastructure;
- automate design space exploration of diverse ML/SW/HW stacks to trade off performance, accuracy, energy, size and costs;
- help end-users reproduce MLPerf results and deploy the most suitable ML/SW/HW stacks in production;
- support collaborative and reproducible research.
(C)opyright MLCommons 2022