Skip to content

Commit

Permalink
Merge pull request #83 from ds-modules/cg_update
Browse files Browse the repository at this point in the history
Testing a widget to calculate cloud cost
  • Loading branch information
balajialg authored Sep 17, 2024
2 parents 9dd932f + a8cdacd commit 45a1ea6
Show file tree
Hide file tree
Showing 3 changed files with 224 additions and 1 deletion.
5 changes: 5 additions & 0 deletions _toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,11 @@ parts:
title: Download Datahub home directory contents as archive
- file: workflow/performance_issue.md
title: Best Practices to Avoid Performance Issues
- file: workflow/calculate-compute-cost.ipynb
title: Calculate GCP cost for compute intensive scenarios
sections:
- url: https://dev.datahub.berkeley.edu/user/balajialwar/voila/render/data100%20cloud%20cost%20calculator/data100_cloudcost_calculator.ipynb?
title: "Launch Cloud Cost Calculator Widget"
- file: workflow/securely-push-github
title: Securely Push Changes to Github
- file: technology/using-ai-llm
Expand Down
218 changes: 218 additions & 0 deletions workflow/calculate-compute-cost.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "d3e5b963-b7e9-4ee1-8732-a56da4dc8149",
"metadata": {},
"source": [
"<h1 style=\"text-align: center;\">Data 100 GCP cloud cost simulator widget</h1>\n",
"\n",
"<p style=\"text-align: center;\">\n",
"This interactive widget allows you to calculate the cost for running CPU heavy workloads in standard virtual machine vs CPU optimized machine\n",
"</p>\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "7dc9f40d-8a30-4344-9606-2ba071e67e89",
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"\n",
"# Data for the table\n",
"data = {\n",
" \"Type of node\": [\n",
" \"n2-highcpu-32\", \"n2-highcpu-32\", \"n2-highcpu-32\", \"n2-highcpu-32\", \"n2-highcpu-32\", \"n2-highcpu-32\", \"n2-highcpu-96\", \"n2-highcpu-96\", \"n2-highcpu-96\", \"n2-highcpu-96\", \"n2-highcpu-96\", \"n2-highcpu-96\", \n",
" \"n2-highmem-32\", \"n2-highmem-32\", \"n2-highmem-32\", \"n2-highmem-32\", \"n2-highmem-32\", \"n2-highmem-32\"\n",
" ],\n",
" \"Number of CPUs allocated per student\": [\n",
" \"4 CPU\", \"4 CPU\", \"4 CPU\", \"2 CPU\", \"2 CPU\", \"2 CPU\", \"4 CPU\", \"4 CPU\", \"4 CPU\", \"2 CPU\", \"2 CPU\", \"2 CPU\", \"4 CPU\", \"4 CPU\", \"4 CPU\", \"2 CPU\", \"2 CPU\", \"2 CPU\"\n",
" ],\n",
" \"Estimate of the total number of students running CPU based workloads\": [\n",
" 1200, 600, 400, 1200, 600, 400, 1200, 600, 400, 1200, 600, 400, 1200, 600, 400, 1200, 600, 400\n",
" ],\n",
" \"# of students allocated per node\": [\n",
" 8, 8, 8, 16, 16, 16, 24, 24, 24, 48, 48, 48, 8, 8, 8, 16, 16, 16\n",
" ],\n",
" \"Total number of nodes allocated\": [\n",
" 150, 75, 50, 75, 38, 25, 50, 25, 17, 25, 13, 8, 150, 75, 38, 75, 38, 25\n",
" ],\n",
" \"Node cost per day\": [\n",
" 27.9, 27.9, 27.9, 27.9, 27.9, 27.9, 83.73, 83.73, 83.73, 83.73, 83.73, 83.73, 51, 51, 51, 51, 51, 51\n",
" ],\n",
" \"Number of days\": [\n",
" 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12\n",
" ],\n",
" \"Total Cost (12 days)\": [\n",
" 50220, 25110, 16740, 25110, 12555, 8370, 50238, 25119, 16746, 25119, 12559.5, 8373, 91800, 45900, 23256, 45900, 23256, 15300\n",
" ]\n",
"}\n",
"\n",
"# Creating the DataFrame\n",
"df = pd.DataFrame(data)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "51161385-0713-4254-bd1a-19691edfb967",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "1306431292e14695bc008300cd77cb15",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"VBox(children=(VBox(children=(VBox(children=(Dropdown(description='Node Type:', layout=Layout(width='80%'), op…"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import ipywidgets as widgets\n",
"from IPython.display import display\n",
"from ipywidgets import Layout\n",
"\n",
"# Pricing data for the latest instance costs in the Iowa Central region (us-central1)\n",
"NODE_PRICING = {\n",
" 'n2-highcpu-32': 27.93, # Latest price for n2-highcpu-32\n",
" 'n2-highcpu-96': 83.73, # Latest price for n2-highcpu-96\n",
" 'n2-highmem-32': 50.31, # Latest price for n2-highmem-32\n",
"}\n",
"\n",
"# Function to calculate the total number of nodes and total cost\n",
"def calculate_total_cost(node_type, num_cpus, num_students, num_days):\n",
" # Extract the number of CPUs in the node from the node type (e.g., 32 or 96)\n",
" node_capacity = int(node_type.split('-')[-1])\n",
" \n",
" # Calculate the number of students allocated per node\n",
" students_per_node = node_capacity / num_cpus\n",
" \n",
" # Calculate the total number of nodes needed\n",
" total_nodes = num_students / students_per_node\n",
" \n",
" # Get the node cost from the pricing dictionary\n",
" node_cost = NODE_PRICING[node_type]\n",
" \n",
" # Calculate the total cost for the given number of days\n",
" total_cost = total_nodes * node_cost * num_days\n",
" \n",
" return total_nodes, total_cost\n",
"\n",
"# Callback function for the button\n",
"def on_button_click(b):\n",
" node_type = node_type_dropdown.value\n",
" num_cpus = int(num_cpus_dropdown.value.split()[0]) # Extracting the number of CPUs (2 or 4)\n",
" num_students = int(students_input.value)\n",
" num_days = int(num_days_input.value)\n",
" \n",
" # Calculate total nodes and total cost\n",
" total_nodes, total_cost = calculate_total_cost(node_type, num_cpus, num_students, num_days)\n",
" \n",
" # Show result in text box\n",
" result_text.value = f\"Total nodes allocated: {total_nodes:.2f}\\nTotal cost for {num_days} days: ${total_cost:.2f}\"\n",
"\n",
"# Widget elements\n",
"node_type_dropdown = widgets.Dropdown(\n",
" options=['n2-highcpu-32', 'n2-highcpu-96', 'n2-highmem-32'],\n",
" value='n2-highcpu-32',\n",
" description='Node Type:',\n",
" style={'description_width': 'initial'},\n",
" layout=Layout(width='80%')\n",
")\n",
"\n",
"num_cpus_dropdown = widgets.Dropdown(\n",
" options=['2 CPU', '4 CPU'],\n",
" value='4 CPU',\n",
" description='CPUs per Student:',\n",
" style={'description_width': 'initial'},\n",
" layout=Layout(width='80%')\n",
")\n",
"\n",
"students_input = widgets.IntText(\n",
" value=1200,\n",
" description='Total Students:',\n",
" style={'description_width': 'initial'},\n",
" layout=Layout(width='80%')\n",
")\n",
"\n",
"num_days_input = widgets.IntText(\n",
" value=12,\n",
" description='Number of Days:',\n",
" style={'description_width': 'initial'},\n",
" layout=Layout(width='80%')\n",
")\n",
"\n",
"calculate_button = widgets.Button(\n",
" description='Calculate Total Cost',\n",
" button_style='success',\n",
" layout=Layout(width='50%') # Set button width to 50% of the container width\n",
")\n",
"\n",
"# Improved result text box with larger size, center alignment, and padding for better spacing\n",
"result_text = widgets.Textarea(\n",
" value='',\n",
" description='Result:',\n",
" disabled=True,\n",
" layout=Layout(width='80%', height='100px', padding='10px'),\n",
" style={'description_width': 'initial', 'text-align': 'center'}\n",
")\n",
"\n",
"# Attach the click event handler\n",
"calculate_button.on_click(on_button_click)\n",
"\n",
"# Organize widgets in a vertical and horizontal layout with proper spacing\n",
"inputs_box = widgets.VBox([\n",
" widgets.VBox([node_type_dropdown, num_cpus_dropdown], layout=Layout(align_items='center', padding='10px')),\n",
" widgets.VBox([students_input, num_days_input], layout=Layout(align_items='center', padding='10px')),\n",
" calculate_button\n",
"], layout=Layout(padding='10px', align_items='center', width='100%'))\n",
"\n",
"# Final layout for displaying widgets and result\n",
"final_layout = widgets.VBox([\n",
" inputs_box,\n",
" result_text\n",
"], layout=Layout(padding='10px', align_items='center'))\n",
"\n",
"# Display the entire widget setup\n",
"display(final_layout)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a1180ae6-6c56-43cd-b622-5d14818dbe09",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
2 changes: 1 addition & 1 deletion workflow/download_notebook_as_pdf.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,6 @@ This method is used to embed an image as an attachment within a Jupyter notebook
````

```{warning}
All image links referred as part of the Jupyter Notebook should be adhering to https and not http. Serving http content within https pages are referred to as mixed content requests. Modern browsers consider a lot of these sorts of requests to be a security risk and will refuse to load them when using default security settings. For more information, check [here](https://www.cloudflare.com/learning/ssl/what-is-mixed-content/)
All image links referred as part of the Jupyter Notebook should be adhering to https and not http. Serving http content within https pages are referred to as mixed content requests. Modern browsers consider these requests to be a security risk and will refuse to load them when using default security settings. For more information, check [here](https://www.cloudflare.com/learning/ssl/what-is-mixed-content/)
````

0 comments on commit 45a1ea6

Please sign in to comment.