python/environment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Environment\n",
    "> [Main Table of Contents](../README.md)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## In this notebook\n",
    "- Anaconda vs Miniconda\n",
    "- Install Anaconda or Miniconda in WSL Ubuntu \n",
    "- Conda (package manager, virtual environment manager)\n",
    "\t- `.condarc` \n",
    "\t- Package manager terminal commands\n",
    "\t- Virtual environment terminal commands\n",
    "\t- Special note on install Java into active virtual environment\n",
    "- Pip\n",
    "\t- Package manager\n",
    "\t- Commands list\n",
    "- Conda vs Pip\n",
    "- Run python programs from the command line"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Anaconda vs Miniconda\n",
    "- Anaconda (don't use)\n",
    "    - A collection of ~350 data science packages\n",
    "    - Installs conda, python, its dependencies, ~350 packages\n",
    "    - I have found this install not useful since I use conda environments and manually install very specific packages for each environment\n",
    "- Miniconda\n",
    "    - Only installs conda, python, its dependencies"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Anaconda or Miniconda in WSL Ubuntu  \n",
    "\n",
    "0. Pick [Anaconda](https://repo.anaconda.com/archive/) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html) release  \n",
    "\n",
    "1. Download Anaconda release into WSL2 current directory\n",
    "\n",
    "|Terminal command |Example|  \n",
    "| --- | --- |  \n",
    "|`wget https://repo.anaconda.com/archive/<Anaconda_version_for_64bit_linux>.sh` | `wget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh`  |    \n",
    "\n",
    "2. Install script  \n",
    "\n",
    "\n",
    "|Terminal command | Example  |  \n",
    "|--- | ---  |  \n",
    "|`bash <Anaconda_version_for_64bit_linux>.sh` | `bash Anaconda3-2022.10-Linux-x86_64.sh`  |  \n",
    "\n",
    "3. Accept the license and `yes` if it asks to run `conda init`\n",
    "\n",
    "4. Close the terminal and reopen it to reload .bash configs"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conda (Package manager)\n",
    "- Functions include package manger, virtual environment manager\n",
    "- Automatically install packages into every new environment by configuring `.condarc` \n",
    "    - If need to create an environment without the default packages use `--no-default-packages` flag\n",
    "- Virtual environment manager\n",
    "    - Use `conda` virtual environments\n",
    "    - Can easily install any version of python\n",
    "    - Don't have to hassle with `pyenv`, `venv` and trying to get proper file structure to maintain the environments\n",
    "- Can use `pip` to install into conda environments\n",
    "    - GOTCHA: *make sure to use pip after all conda installs* [SEE REF HERE](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#using-pip-in-an-environment)\n",
    "        - If I need to use `conda install ...` *after* a `pip install ...` re-create the virtual environment first and do all conda installs first!\n",
    "    - GOTCHA: Make sure to `which pip` to make sure it's the pip in the conda environnment.\n",
    "        - Better yet, if new conda environment, be sure to `conda deactivate` then `conda activate <env_name>` before using `pip install <pck_name>` to make sure pip is installing to the local conda environment, otherwise, may install to global environment\n",
    "    - If running `conda install ...` after `pip install ...` then re-create the environment to make sure all conda installs are before pip installs"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [.condarc](https://conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html#overview)\n",
    "- Automatically install packages into every new environment by configuring `.condarc` \n",
    "- If need to create an environment without the default packages use `--no-default-packages` flag"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# .condarc file example\n",
    "\n",
    "# Remove comments below\n",
    "# create_default_packages:\n",
    "#   - pip\n",
    "#   - ipython\n",
    "#   - scipy=0.15.0"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- conda package manager commands  \n",
    "\n",
    "|Terminal commands | Example | Explanation |\n",
    "|--- | --- | ---  |\n",
    "|`conda update conda` | -| Update conda|\n",
    "|`conda search` |- |List of all packages available for Anaconda|\n",
    "|`conda search <pckg_name>` |-| Check if specific package is available for installation|\n",
    "|`conda env export > <filename>.yml` | `conda env export > my_environment.yml` | Save list of project dependencies to YAML file|\n",
    "    "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- conda virtual environment commands\n",
    "\n",
    "\n",
    "Terminal commands | Example | Explanation  \n",
    "--- | --- | ---  \n",
    "`conda create -n <env_name> python=<version> -y` | `conda create -n test_project python=3.8.5 -y` | Create new virtual environment for specified python version<br>'-y' tells command line to say 'yes' to all prompts that follow. Time saver.\n",
    "`conda create -n <env_name> python=<version> <pckg_name>=<version> <pckg_name> ...` | `conda create -n test_project python=3.7 scipy=0.15.0 astroid babel` | Create new virtual environment for specified python version and install additional dependencies\n",
    "`conda env create -f <filename>.yml` | `conda env create -f environment.yml` | Create new virtual environment from YAML file<br>'-f' stands for filename\n",
    "`conda env update –f <filename>.yml –n <env_name>` | `conda env update –f environment.yml –n test_project` | Update an environment using a YAML file\n",
    "`conda env export > <filename>.yml`<br>2. `conda env export -n <env-name> > <filename>.yml`<br>3. `conda env export --from-history > <filename>.yml` |1.`conda env export > environment.yml`<br>2. `conda env export -n test_project environment.yml`<br>3. `conda env export --from-history > environment_cross.yml`| Save project dependencies to YAML file<br>1. if already in activated environment<br>2. When not in activated environment<br>3. When working across platforms iow: re-creating environment on different os. GOTCHA: doesn't include pip installed packages<br><br>If pip was installed with conda, this command will list both conda and pip packages<br>Similar to `pip freeze`\n",
    "`conda env list` | -| Get list of virtual environments\n",
    "`conda activate <env_name>` | `conda activate test_project` | Activate virtual environment\n",
    "`conda deactivate` | -| Deactivate virtual environment\n",
    "`conda remove --name <env_name> --all` | `conda remove --name test_project --all` | Remove and complete clean the environment\n",
    "`conda list` |-| List all installed packages in the active environment\n",
    "`conda list -n <env_name>` | - | List all installed packages in a deactived environment \n",
    "`conda list <pckg_name>` | `conda list pandas` | Get info on specific package that is installed the active environment"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Special note on installing java 8 which is required for spark-related packages\n",
    "\n",
    "Terminal command | Explanation\n",
    "--- | --- \n",
    "`conda install -c conda-forge openjdk=8` | Install java version 8 into the active virtual environment"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pip \n",
    "- Package manager"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- pip package manager commands  \n",
    "\n",
    "Terminal commands | Explanation\n",
    "--- | ---\n",
    "`pip install\\|uninstall -r <path>/requirements.txt` | Install\\|Uninstall dependencies from requirements.txt file into system python\n",
    "`pip install\\|uninstall <package_name>` | Install\\|Uninstall one dependency into system python\n",
    "`<path_specific_python_dir>/python3 -m pip install\\|uninstall -r <path>/requirements.txt` | Install\\|Uninstall dependencies from requirements.txt file into specific python\n",
    "`<path_specific_python_dir>/python3 -m pip install\\|uninstall <package_name>` | Install\\|Uninstall one dependency into specific python\n",
    "`pip freeze --local > <fileName>.txt` | Save list of project dependencies to file"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conda vs Pip\n",
    "- [Article](https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run python programs from the command line\n",
    "\n",
    "```python\n",
    "python3 -m <packageName>\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "notes",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9 (main, Jan 11 2023, 15:21:40) [GCC 11.2.0]"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "b8500f9483930a0c66dce4ac8684f923d2fbeba6afd3eeecf71190e0e74d8118"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}