reproduce-micro-paper-2023-victima

Click here to see the table of contents.

• About
• Summary
• Reuse this script in your project
  • Install CM automation language
  • Check CM script flags
  • Run this script from command line
  • Run this script from Python
  • Run this script via GUI
  • Run this script via Docker (beta)
• Customization
  • Variations
  • Script flags mapped to environment
  • Default environment
• Script workflow, dependencies and native scripts
• Script output
• New environment keys (filter)
• New environment keys auto-detected from customize
• Maintainers

Note that this README is automatically generated - don't edit!

About

See extra notes from the authors and contributors.

Summary

• Category: Reproducibility and artifact evaluation.
• CM GitHub repository: mlcommons@ck
• GitHub directory for this script: GitHub
• CM meta description for this script: _cm.yaml
• CM "database" tags to find this script: reproduce,project,paper,micro,micro-2023,victima
• Output cached? False

---

Reuse this script in your project

Install CM automation language

• Installation guide
• CM intro

Pull CM repository with this automation

cm pull repo mlcommons@ck

Run this script from command line

1. cm run script --tags=reproduce,project,paper,micro,micro-2023,victima[,variations] [--input_flags]

2. cmr "reproduce project paper micro micro-2023 victima[ variations]" [--input_flags]

• variations can be seen here

• input_flags can be seen here

Run this script from Python

Click here to expand this section.

import cmind

r = cmind.access({'action':'run'
                  'automation':'script',
                  'tags':'reproduce,project,paper,micro,micro-2023,victima'
                  'out':'con',
                  ...
                  (other input keys for this script)
                  ...
                 })

if r['return']>0:
    print (r['error'])

Run this script via GUI

cmr "cm gui" --script="reproduce,project,paper,micro,micro-2023,victima"

Use this online GUI to generate CM CMD.

Run this script via Docker (beta)

cm docker script "reproduce project paper micro micro-2023 victima[ variations]" [--input_flags]

---

Customization

Variations

• No group (any variation can be selected)

  Click here to expand this section.

  • _install_deps
    • Workflow:
  • _plot
    • Workflow:
  • _run
    • Workflow:

Script flags mapped to environment

Click here to expand this section.

• --container=value → CM_VICTIMA_CONTAINER=value
• --job_manager=value → CM_VICTIMA_JOB_MANAGER=value

Above CLI flags can be used in the Python CM API as follows:

r=cm.access({... , "container":...}

Default environment

Click here to expand this section.

These keys can be updated via --env.KEY=VALUE or env dictionary in @input.json or using script flags.

• CM_VICTIMA_JOB_MANAGER: native
• CM_VICTIMA_CONTAINER: docker

---

Script workflow, dependencies and native scripts

Click here to expand this section.

1. Read "deps" on other CM scripts from meta
   • detect,os
     • CM script: detect-os
   • get,python
     • CM names: --adr.['python', 'python3']...
     • CM script: get-python3
   • get,git,repo,_repo.https://github.com/CMU-SAFARI/Victima
     • CM script: get-git-repo
2. Run "preprocess" function from customize.py
3. Read "prehook_deps" on other CM scripts from meta
4. Run native script if exists
   • run.sh
5. Read "posthook_deps" on other CM scripts from meta
6. Run "postrocess" function from customize.py
7. Read "post_deps" on other CM scripts from meta

---

Script output

cmr "reproduce project paper micro micro-2023 victima[,variations]" [--input_flags] -j

New environment keys (filter)

New environment keys auto-detected from customize

---

Maintainers

• Open MLCommons taskforce on automation and reproducibility