fine_tuning_sql/notebooks/4_upload_and_train.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 4: Upload the dataset and train in Sambastudio\n",
    "This notebook is a guide that goes over the steps you need to follow to train a model on the pretraining and fine-tuning datasets in Sambastudio GUI."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Upload dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First you need to upload the previous Generated datasets to Sambastudio\n",
    "\n",
    "- Go to your sambastudio environments and click on the Datasets section\n",
    "\n",
    "    ![](../images/1_dataset_selector.png)\n",
    "\n",
    "- Click on on the upload dataset button\n",
    "\n",
    "    ![](../images/2_add_dataset.png)\n",
    "\n",
    "- Fill with with your dataset details including the name, details, and the applicable ML apps (model families you want to be trained with your dataset), select local upload and select the output folder of the Generative data preparation step \n",
    "\n",
    "    ![](../images/3_dataset_details.png)\n",
    "\n",
    "- Finally you will be able to see the information of uploaded  datasets in the datasets dashboard\n",
    "\n",
    "    ![](../images/4_datasets_overview.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Find more information in [SambaStudio datasets documentation](https://docs.sambanova.ai/sambastudio/latest/add-datasets.html#_add_a_dataset_using_the_gui)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a project"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next step you will need to do is create a project in which you will create your training jobs.\n",
    "\n",
    "- First go projects section \n",
    "\n",
    "    ![](../images/5_projects_selector.png)\n",
    "\n",
    "- Click in the create project button\n",
    "\n",
    "    ![](../images/6_add_project.png)\n",
    "\n",
    "- Fill with your project name and details\n",
    " \n",
    "    ![](../images/7_project_creation.png)\n",
    "\n",
    "- Click on save and continue button"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Find more information in [SambaStudio projects documentation](https://docs.sambanova.ai/sambastudio/latest/projects.html#_create_a_project_using_the_gui)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a training Job"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you should create and start the training job\n",
    "\n",
    "- Click on create job button\n",
    "\n",
    "    ![](../images/8_job_creation.png)\n",
    "\n",
    "- Then fill the job details selecting training job, and filling job name, ML app, description the specific model to train and the dataset you wan to use for the training job \n",
    "\n",
    "    ![](../images/9_job_details.png)\n",
    "\n",
    "> For pretraining we will use **Llama7B Base** as the starting model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then set the hyperparameters for the training job\n",
    "\n",
    "#### Pre-training\n",
    "\n",
    "![](../images/10_pretraining_SN.png)\n",
    "\n",
    "#### Fine-tuning\n",
    "\n",
    "![](../images/11_fine_tuning_SN.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Click on start trining job and you will be able to see a dashboard view of the training job process as shown bellow\n",
    "\n",
    "    ![](../images/12_training_process.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Find more information in [SambaStudio training jobs documentation](https://docs.sambanova.ai/sambastudio/latest/training.html#_create_a_train_model_job_using_the_gui)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save your model checkpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Depending on the save steps parameters, the job will store checkpoints of your trained model, when you have a model with low training loss you can store specific model checkpoints in the model hub to be able to deploy it or do further fine-tuning \n",
    "\n",
    "- for this only click in the menu of the checkpoint of your preference and click on save to model hub button\n",
    "\n",
    "    ![](../images/13_checkpoints_save.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Find more information in [SambaStudio checkpoints documentation](https://docs.sambanova.ai/sambastudio/latest/training.html#_saveadd_checkpoints_to_the_model_hub)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "finetuning_env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}