Skip to content

Commit

Permalink
Repo init
Browse files Browse the repository at this point in the history
  • Loading branch information
@joefiorini
joefiorini committed Jul 17, 2023
0 parents commit becf453
Show file tree
Hide file tree
Showing 51 changed files with 15,715 additions and 0 deletions.
14 changes: 14 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@

__pycache__
.virtual_documents/System Monitor/requirements.ipynb
.virtual_documents/System Monitor/tests.ipynb
/models
.ipynb_checkpoints
/db
/data

**/db
/url-summarizer/url-summarizer
System Monitor/system-monitor
System Monitor/service-logs/*
!System Monitor/service-logs/collect-logs.fish
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"python.analysis.typeCheckingMode": "basic"
}
137 changes: 137 additions & 0 deletions Gradient Descent/linear-regression.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"> https://chat.openai.com/share/1a8d2884-7cf4-4f26-a3b6-87fdb254d492\n",
"\n",
"**Objective:** The goal of this lab is to reinforce your understanding of gradient descent by implementing it from scratch to solve a simple linear regression problem. You will be using Python, Jupyter notebooks, and PyTorch for this assignment.\n",
"\n",
"**Problem Description:** You are given a dataset with two variables, x and y, where y is a linear function of x with some added noise. Your task is to find the line of best fit for this data using gradient descent.\n",
"\n",
"**Instructions:**\n",
"\n",
"1. **Data Generation:** Generate a synthetic dataset of 100 points for this task. You can use the torch.rand function to generate x and then define y as y = 2x + 3 + noise, where noise is a random value added to each point to simulate real-world data. The coefficients 2 and 3 are the true weight and bias, respectively, that you will try to learn with gradient descent.\n",
"\n",
"2. **Model Definition:** Define a simple linear regression model y = wx + b, where w is the weight and b is the bias. Initialize w and b to any values of your choice.\n",
"\n",
"3. **Loss Function:** Define the mean squared error loss function, which is the function you will minimize using gradient descent.\n",
"\n",
"4. **Gradient Descent:** Implement the gradient descent algorithm. At each step, compute the gradients of the loss with respect to w and b, and then update w and b in the direction that reduces the loss. Repeat this process for a fixed number of iterations, or until w and b converge to the true values within a certain tolerance.\n",
"\n",
"5. **Evaluation:** Plot the original data along with the line of best fit found by your model. Also, plot the loss over time to see how it decreases as the model learns.\n",
"\n",
"**Questions to Consider:**\n",
"\n",
"- How do different initial values of w and b affect the number of iterations needed for convergence?\n",
"- How does the learning rate affect the speed of convergence and the final result?\n",
"- What happens if the learning rate is set too high or too low?\n",
"- How does the model perform if you increase the amount of noise in the data?\n",
"- Remember, you can ask for help at any time if you're unsure about how to proceed. Good luck!"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"from torch.testing import assert_close\n",
"\n",
"device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
"\n",
"xs = torch.randn(100, 1) * 10\n",
"noise = torch.randn(100, 1) * 3\n",
"ys = torch.nn.functional.linear(xs, torch.tensor([[2.0]]), torch.tensor([3.0]) + noise)\n",
"\n",
"\n",
"assert_close(ys, 2 * xs + 3 + noise)\n",
"kak"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([100, 1])\n",
"torch.Size([1, 1])\n",
"torch.Size([1])\n"
]
},
{
"ename": "RuntimeError",
"evalue": "assigned grad has data of a different size",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mRuntimeError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[43], line 29\u001b[0m\n\u001b[1;32m 27\u001b[0m b\u001b[39m.\u001b[39mgrad\u001b[39m.\u001b[39mzero_()\n\u001b[1;32m 28\u001b[0m \u001b[39mreturn\u001b[39;00m w, b\n\u001b[0;32m---> 29\u001b[0m linear_regression(xs, ys, epochs\u001b[39m=\u001b[39;49m\u001b[39m100\u001b[39;49m, learning_rate\u001b[39m=\u001b[39;49m\u001b[39m0.001\u001b[39;49m)\n",
"Cell \u001b[0;32mIn[43], line 22\u001b[0m, in \u001b[0;36mlinear_regression\u001b[0;34m(xs, ys, epochs, learning_rate)\u001b[0m\n\u001b[1;32m 20\u001b[0m y_pred \u001b[39m=\u001b[39m \u001b[39m2\u001b[39m \u001b[39m*\u001b[39m xs \u001b[39m+\u001b[39m \u001b[39m3\u001b[39m \u001b[39m+\u001b[39m noise\n\u001b[1;32m 21\u001b[0m loss \u001b[39m=\u001b[39m mean_squared_loss(y_pred, ys)\n\u001b[0;32m---> 22\u001b[0m loss\u001b[39m.\u001b[39;49mgrad \u001b[39m=\u001b[39m compute_gradient(loss, [w, b])\n\u001b[1;32m 23\u001b[0m \u001b[39mwith\u001b[39;00m torch\u001b[39m.\u001b[39mno_grad():\n\u001b[1;32m 24\u001b[0m w \u001b[39m-\u001b[39m\u001b[39m=\u001b[39m learning_rate \u001b[39m*\u001b[39m w\u001b[39m.\u001b[39mgrad\n",
"\u001b[0;31mRuntimeError\u001b[0m: assigned grad has data of a different size"
]
}
],
"source": [
"initial_weight = 1.0\n",
"initial_bias = 1.0\n",
"\n",
"def mean_squared_loss(y_pred, y_true):\n",
" return ((y_pred - y_true) ** 2).mean()\n",
"\n",
"def compute_gradient(loss, params):\n",
" # Compute the gradient of loss with respect to params\n",
" # For y=wx+b (using mean-squared loss function) the gradient is:\n",
" # dw = 2*x*(wx+b-y)\n",
" return 2 * xs * (params[0] * xs + params[1] - ys)\n",
"\n",
"def linear_regression(xs, ys, epochs, learning_rate):\n",
" w = torch.tensor([[initial_weight]], dtype=torch.float32, requires_grad=False)\n",
" b = torch.tensor([initial_bias], dtype=torch.float32, requires_grad=False)\n",
" print(xs.shape)\n",
" print(w.shape)\n",
" print(b.shape)\n",
" for epoch in range(epochs):\n",
" y_pred = 2 * xs + 3 + noise\n",
" loss = mean_squared_loss(y_pred, ys)\n",
" loss.grad = compute_gradient(loss, [w, b])\n",
" with torch.no_grad():\n",
" w -= learning_rate * w.grad\n",
" b -= learning_rate * b.grad\n",
" w.grad.zero_()\n",
" b.grad.zero_()\n",
" return w, b\n",
"linear_regression(xs, ys, epochs=100, learning_rate=0.001)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.11"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
143 changes: 143 additions & 0 deletions IMDb/pip.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"notebookRunGroups": {
"groupValue": ""
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Usage: \n",
" pip <command> [options]\n",
"\n",
"Commands:\n",
" install Install packages.\n",
" download Download packages.\n",
" uninstall Uninstall packages.\n",
" freeze Output installed packages in requirements format.\n",
" inspect Inspect the python environment.\n",
" list List installed packages.\n",
" show Show information about installed packages.\n",
" check Verify installed packages have compatible dependencies.\n",
" config Manage local and global configuration.\n",
" search Search PyPI for packages.\n",
" cache Inspect and manage pip's wheel cache.\n",
" index Inspect information available from package indexes.\n",
" wheel Build wheels from your requirements.\n",
" hash Compute hashes of package archives.\n",
" completion A helper command used for command completion.\n",
" debug Show information useful for debugging.\n",
" help Show help for commands.\n",
"\n",
"General Options:\n",
" -h, --help Show help.\n",
" --debug Let unhandled exceptions propagate outside the\n",
" main subroutine, instead of logging them to\n",
" stderr.\n",
" --isolated Run pip in an isolated mode, ignoring\n",
" environment variables and user configuration.\n",
" --require-virtualenv Allow pip to only run in a virtual environment;\n",
" exit with an error otherwise.\n",
" --python <python> Run pip with the specified Python interpreter.\n",
" -v, --verbose Give more output. Option is additive, and can be\n",
" used up to 3 times.\n",
" -V, --version Show version and exit.\n",
" -q, --quiet Give less output. Option is additive, and can be\n",
" used up to 3 times (corresponding to WARNING,\n",
" ERROR, and CRITICAL logging levels).\n",
" --log <path> Path to a verbose appending log.\n",
" --no-input Disable prompting for input.\n",
" --keyring-provider <keyring_provider>\n",
" Enable the credential lookup via the keyring\n",
" library if user input is allowed. Specify which\n",
" mechanism to use [disabled, import, subprocess].\n",
" (default: disabled)\n",
" --proxy <proxy> Specify a proxy in the form\n",
" scheme://[user:passwd@]proxy.server:port.\n",
" --retries <retries> Maximum number of retries each connection should\n",
" attempt (default 5 times).\n",
" --timeout <sec> Set the socket timeout (default 15 seconds).\n",
" --exists-action <action> Default action when a path already exists:\n",
" (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.\n",
" --trusted-host <hostname> Mark this host or host:port pair as trusted,\n",
" even though it does not have valid or any HTTPS.\n",
" --cert <path> Path to PEM-encoded CA certificate bundle. If\n",
" provided, overrides the default. See 'SSL\n",
" Certificate Verification' in pip documentation\n",
" for more information.\n",
" --client-cert <path> Path to SSL client certificate, a single file\n",
" containing the private key and the certificate\n",
" in PEM format.\n",
" --cache-dir <dir> Store the cache data in <dir>.\n",
" --no-cache-dir Disable the cache.\n",
" --disable-pip-version-check\n",
" Don't periodically check PyPI to determine\n",
" whether a new version of pip is available for\n",
" download. Implied with --no-index.\n",
" --no-color Suppress colored output.\n",
" --no-python-version-warning\n",
" Silence deprecation warnings for upcoming\n",
" unsupported Pythons.\n",
" --use-feature <feature> Enable new functionality, that may be backward\n",
" incompatible.\n",
" --use-deprecated <feature> Enable deprecated functionality, that will be\n",
" removed in the future.\n"
]
}
],
"source": [
"!pip --help"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting portalocker\n",
" Using cached portalocker-2.7.0-py2.py3-none-any.whl (15 kB)\n",
"Installing collected packages: portalocker\n",
"Successfully installed portalocker-2.7.0\n"
]
}
],
"source": [
"%pip install portalocker\n",
"%"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.11"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading

0 comments on commit becf453

Please sign in to comment.