Skip to content
This repository has been archived by the owner on Jan 31, 2022. It is now read-only.

Commit

Permalink
Create a notebook to train a model using AutoML
Browse files Browse the repository at this point in the history 
* Refactor some of the notebook setup into notebook_setup.py to make it reusable

* For the kpt package to launch a notebook
  Add kpt setters to properly set most values.
  Add an application resource so that links show up in the application dashboard.
  • Loading branch information
@jlewi
jlewi authored and Jeremy Lewi committed May 1, 2020
1 parent b4b9b4b commit ba7a52c
Show file tree
Hide file tree
Showing 11 changed files with 8,976 additions and 100 deletions.
80 changes: 10 additions & 70 deletions Issue_Triage/notebooks/triage.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7955,13 +7955,7 @@
"state:Issue needs triage:\n",
"\t Issue needs one of the priorities ['priority/p0', 'priority/p1', 'priority/p2', 'priority/p3']\n",
"\t Issue needs an area label\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
"INFO|2020-04-20T13:40:49|Issue https://github.com/kubeflow/examples/issues/780 already in triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|751|\n",
"INFO|2020-04-20T13:40:49|Issue https://github.com/kubeflow/kubeflow/issues/4894:\n",
"state:Issue needs triage:\n",
Expand Down Expand Up @@ -8063,13 +8057,7 @@
"state:Issue needs triage:\n",
"\t Issue needs an area label\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
"INFO|2020-04-20T13:40:49|Issue https://github.com/kubeflow/kfctl/issues/278 already in triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|751|\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO|2020-04-20T13:40:49|Issue https://github.com/kubeflow/kfctl/issues/278 already in triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|751|\n",
"INFO|2020-04-20T13:40:49|Issue https://github.com/kubeflow/kubeflow/issues/4845:\n",
"state:Issue needs triage:\n",
"\t Issue needs one of the priorities ['priority/p0', 'priority/p1', 'priority/p2', 'priority/p3']\n",
Expand Down Expand Up @@ -8171,13 +8159,7 @@
"INFO|2020-04-20T13:40:51|Issue https://github.com/kubeflow/kfctl/issues/265:\n",
"state:Issue needs triage:\n",
"\t Issue needs an area label\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
"INFO|2020-04-20T13:40:51|Issue https://github.com/kubeflow/kfctl/issues/265 already in triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|751|\n",
"INFO|2020-04-20T13:40:51|Issue https://github.com/kubeflow/kubeflow/issues/4820:\n",
"state:Issue needs triage:\n",
Expand Down Expand Up @@ -8281,13 +8263,7 @@
"INFO|2020-04-20T13:40:51|Issue https://github.com/kubeflow/website/issues/1643:\n",
"state:Issue needs triage:\n",
"\t Issue needs an area label\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
"INFO|2020-04-20T13:40:51|Issue https://github.com/kubeflow/website/issues/1643 already in triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|751|\n",
"INFO|2020-04-20T13:40:51|Issue https://github.com/kubeflow/website/issues/1626:\n",
"state:Issue doesn't need attention.\n",
Expand Down Expand Up @@ -8377,13 +8353,7 @@
"INFO|2020-04-20T13:41:03|Issue https://github.com/kubeflow/kfserving/issues/546:\n",
"state:Issue doesn't need attention.\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
"INFO|2020-04-20T13:41:03|Issue https://github.com/kubeflow/kfserving/issues/546 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO|2020-04-20T13:41:03|Issue https://github.com/kubeflow/kfserving/issues/546 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n",
"INFO|2020-04-20T13:41:04|Issue https://github.com/kubeflow/fairing/issues/417:\n",
"state:Issue doesn't need attention.\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
Expand Down Expand Up @@ -8471,13 +8441,7 @@
"INFO|2020-04-20T13:41:26|Issue https://github.com/kubeflow/fairing/issues/380 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n",
"INFO|2020-04-20T13:41:26|Issue https://github.com/kubeflow/kfserving/issues/353:\n",
"state:Issue doesn't need attention.\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
"INFO|2020-04-20T13:41:26|Issue https://github.com/kubeflow/kfserving/issues/353 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n",
"INFO|2020-04-20T13:41:27|Issue https://github.com/kubeflow/kfserving/issues/347:\n",
"state:Issue doesn't need attention.\n",
Expand Down Expand Up @@ -8563,13 +8527,7 @@
"INFO|2020-04-20T13:41:42|Issue https://github.com/kubeflow/metadata/issues/119 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n",
"INFO|2020-04-20T13:41:43|Issue https://github.com/kubeflow/kfserving/issues/310:\n",
"state:Issue doesn't need attention.\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
"INFO|2020-04-20T13:41:43|Issue https://github.com/kubeflow/kfserving/issues/310 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n",
"INFO|2020-04-20T13:41:43|Issue https://github.com/kubeflow/metadata/issues/117:\n",
"state:Issue doesn't need attention.\n",
Expand Down Expand Up @@ -8659,13 +8617,7 @@
"INFO|2020-04-20T13:41:55|Issue https://github.com/kubeflow/pytorch-operator/issues/190 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n",
"INFO|2020-04-20T13:41:56|Issue https://github.com/kubeflow/fairing/issues/314:\n",
"state:Issue doesn't need attention.\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
"INFO|2020-04-20T13:41:56|Issue https://github.com/kubeflow/fairing/issues/314 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n",
"INFO|2020-04-20T13:41:57|Issue https://github.com/kubeflow/metadata/issues/88:\n",
"state:Issue doesn't need attention.\n",
Expand Down Expand Up @@ -8751,13 +8703,7 @@
"INFO|2020-04-20T13:42:12|Issue https://github.com/kubeflow/website/issues/836:\n",
"state:Issue doesn't need attention.\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
"INFO|2020-04-20T13:42:12|Issue https://github.com/kubeflow/website/issues/836 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO|2020-04-20T13:42:12|Issue https://github.com/kubeflow/website/issues/836 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n",
"INFO|2020-04-20T13:42:13|Issue https://github.com/kubeflow/tf-operator/issues/1033:\n",
"state:Issue doesn't need attention.\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
Expand Down Expand Up @@ -8848,13 +8794,7 @@
"INFO|2020-04-20T13:42:24|Issue https://github.com/kubeflow/tf-operator/issues/999:\n",
"state:Issue doesn't need attention.\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
"INFO|2020-04-20T13:42:24|Issue https://github.com/kubeflow/tf-operator/issues/999 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO|2020-04-20T13:42:24|Issue https://github.com/kubeflow/tf-operator/issues/999 remove from triage project|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|734|\n",
"INFO|2020-04-20T13:42:25|Issue https://github.com/kubeflow/kfserving/issues/82:\n",
"state:Issue doesn't need attention.\n",
"|/home/jovyan/git_kubeflow-code-intelligence/py/issue_triage/triage.py|677|\n",
Expand Down
9 changes: 2 additions & 7 deletions Label_Microservice/notebooks/Label_k8s_issues_with_MLP.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,6 @@
"## Background\n",
"In this notebook, we show how to feed the embeddings from the language model into the MLP classifier. Then, we take the github repo, `kubernetes/kubernetes`, as an example. We do transfer learning and show the results.\n",
"\n",
"* TODO(jlewi): This notebook is duplicative with repo_mlp.ipynb. It looks like this might have contained\n",
"the original training code which has been refactored in repo_mlp.ipynb. But unlike repo_mlp.ipynb this\n",
"notebook contains code for model evaluation. We should probably combine them and remove duplication.\n",
"\n",
"## Data\n",
"**combined_sig_df.pkl**\n",
"https://storage.googleapis.com/issue_label_bot/notebook_files/combined_sig_df.pkl\n",
Expand Down Expand Up @@ -455,7 +451,6 @@
"source": [
"from sklearn.metrics import roc_auc_score\n",
"\n",
"# TODO(jlewi): I moved this into mlp.py\n",
"def calculate_auc(predictions):\n",
" auc_scores = []\n",
" counts = []\n",
Expand Down Expand Up @@ -5659,9 +5654,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
"version": "3.6.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
"nbformat_minor": 2
}
Loading

0 comments on commit ba7a52c

Please sign in to comment.