05_finetune/02_fine_tune_dolly_llama2_sagemaker_cluster.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fine-tune LLaMA 2 on Amazon SageMaker\n",
    "\n",
    "In this example, we are going to learn how to fine-tune [LLaMA 2](https://huggingface.co/meta-llama/Llama-2-7b-hf) suing Amazon SageMaker. [LLaMA 2](https://huggingface.co/meta-llama/Llama-2-70b-hf) is the next version of the [LLaMA](https://arxiv.org/abs/2302.13971). \n",
    "\n",
    "Compared to the V1 model, it is trained on more data - 2T tokens and supports context length window upto 4K tokens. Learn more about LLaMa 2 in the [\"\"]() blog post.\n",
    "\n",
    "In Detail you will learn how to:\n",
    "1. Setup Development Environment\n",
    "2. Load and prepare the dataset\n",
    "3. Fine-Tune LLaMA 7B on Amazon SageMaker with p4d.24xlarge\n",
    "4. Deploy Fine-tuned LLM on Amazon SageMaker\n",
    "\n",
    "### Access LLaMA 2\n",
    "\n",
    "Before we can start training we have to make sure that we accepted the license of [llama 2](https://huggingface.co/meta-llama/Llama-2-70-hf) to be able to use it. You can accept the license by clicking on the Agree and access repository button on the model page at: \n",
    "\n",
    "## 1. Setup Development Environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "spyder 5.3.3 requires pyqt5<5.16, which is not installed.\n",
      "spyder 5.3.3 requires pyqtwebengine<5.16, which is not installed.\n",
      "distributed 2022.7.0 requires tornado<6.2,>=6.0.3, but you have tornado 6.4 which is incompatible.\n",
      "spyder 5.3.3 requires ipython<8.0.0,>=7.31.1, but you have ipython 8.18.1 which is incompatible.\n",
      "spyder 5.3.3 requires pylint<3.0,>=2.5.0, but you have pylint 3.0.2 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0m\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install \"transformers==4.31.0\" \"datasets[s3]==2.13.0\" sagemaker --upgrade --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To access any LLaMA 2 asset we need to login into our hugging face account. We can do this by running the following command:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are going to use Sagemaker in a local environment. You need access to an IAM Role with the required permissions for Sagemaker. You can find [here](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html) more about it.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/sagemaker/config.yaml\n",
      "sagemaker.config INFO - Not applying SDK defaults from location: /root/.config/sagemaker/config.yaml\n",
      "sagemaker role arn: arn:aws:iam::079002598131:role/service-role/AmazonSageMaker-ExecutionRole-20220804T150518\n",
      "sagemaker bucket: sagemaker-us-east-1-079002598131\n",
      "sagemaker session region: us-east-1\n"
     ]
    }
   ],
   "source": [
    "import sagemaker\n",
    "import boto3\n",
    "sess = sagemaker.Session()\n",
    "\n",
    "# sagemaker session bucket -> used for uploading data, models and logs\n",
    "# sagemaker will automatically create this bucket if it not exists\n",
    "sagemaker_session_bucket=None\n",
    "if sagemaker_session_bucket is None and sess is not None:\n",
    "    # set to default bucket if a bucket name is not given\n",
    "    sagemaker_session_bucket = sess.default_bucket()\n",
    "\n",
    "try:\n",
    "    role = sagemaker.get_execution_role()\n",
    "except ValueError:\n",
    "    iam = boto3.client('iam')\n",
    "    role = iam.get_role(RoleName='sagemaker_execution_role')['Role']['Arn']\n",
    "\n",
    "sess = sagemaker.Session(default_bucket=sagemaker_session_bucket)\n",
    "\n",
    "print(f\"sagemaker role arn: {role}\")\n",
    "print(f\"sagemaker bucket: {sess.default_bucket()}\")\n",
    "print(f\"sagemaker session region: {sess.boto_region_name}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Load and prepare the dataset\n",
    "\n",
    "we will use the [dolly](https://huggingface.co/datasets/databricks/databricks-dolly-15k) an open source dataset of instruction-following records generated by thousands of Databricks employees in several of the behavioral categories outlined in the [InstructGPT paper](https://arxiv.org/abs/2203.02155), including brainstorming, classification, closed QA, generation, information extraction, open QA, and summarization.\n",
    "\n",
    "```python\n",
    "{\n",
    "  \"instruction\": \"What is world of warcraft\",\n",
    "  \"context\": \"\",\n",
    "  \"response\": \"World of warcraft is a massive online multi player role playing game. It was released in 2004 by bizarre entertainment\"\n",
    "}\n",
    "```\n",
    "\n",
    "To load the `samsum` dataset, we use the `load_dataset()` method from the 🤗 Datasets library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Found cached dataset json (/root/.cache/huggingface/datasets/databricks___json/databricks--databricks-dolly-15k-7427aa6e57c34282/0.0.0/e347ab1c932092252e717ff3f949105a4dd28b27e842dd53157d2f72e276c2e4)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dataset size: 15011\n",
      "{'instruction': 'What is AWS EMR?', 'context': '', 'response': 'Amazon EMR (previously called Amazon Elastic MapReduce) is a managed cluster platform that simplifies running big data frameworks, such as Apache Hadoop and Apache Spark , on AWS to process and analyze vast amounts of data.', 'category': 'open_qa'}\n"
     ]
    }
   ],
   "source": [
    "from datasets import load_dataset\n",
    "from random import randrange\n",
    "\n",
    "# Load dataset from the hub\n",
    "dataset = load_dataset(\"databricks/databricks-dolly-15k\", split=\"train\")\n",
    "\n",
    "print(f\"dataset size: {len(dataset)}\")\n",
    "print(dataset[randrange(len(dataset))])\n",
    "\n",
    "# dataset size: 15011"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To instruct tune our model we need to convert our structured examples into a collection of tasks described via instructions. We define a `formatting_function` that takes a sample and returns a string with our format instruction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def format_dolly(sample):\n",
    "    instruction = f\"### Instruction\\n{sample['instruction']}\"\n",
    "    context = f\"### Context\\n{sample['context']}\" if len(sample[\"context\"]) > 0 else None\n",
    "    response = f\"### Answer\\n{sample['response']}\"\n",
    "    # join all the parts together\n",
    "    prompt = \"\\n\\n\".join([i for i in [instruction, context, response] if i is not None])\n",
    "    return prompt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lets test our formatting function on a random example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "### Instruction\n",
      "Do tomatoes belong in Louisiana-style gumbo?\n",
      "\n",
      "### Answer\n",
      "The answer is it depends.  In traditional Louisiana-style gumbo recipes from the area known for Cajun (Acadiana) cuisine, tomatoes are not commonly part of the recipe.  However, in the area known for Creole (New Orleans) cuisine tomatoes are commonly part of the recipe.\n"
     ]
    }
   ],
   "source": [
    "from random import randrange\n",
    "\n",
    "print(format_dolly(dataset[randrange(len(dataset))]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition, to formatting our samples we also want to pack multiple samples to one sequence to have a more efficient training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoTokenizer\n",
    "\n",
    "#model_id = \"meta-llama/Llama-2-13b-hf\" # gated\n",
    "model_id = \"NousResearch/Llama-2-7b-hf\" # not gated\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_id)\n",
    "tokenizer.pad_token = tokenizer.eos_token"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define some helper functions to pack our samples into sequences of a given length and then tokenize them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Loading cached processed dataset at /root/.cache/huggingface/datasets/databricks___json/databricks--databricks-dolly-15k-7427aa6e57c34282/0.0.0/e347ab1c932092252e717ff3f949105a4dd28b27e842dd53157d2f72e276c2e4/cache-a9e8aa13e41a7fc8.arrow\n",
      "Loading cached processed dataset at /root/.cache/huggingface/datasets/databricks___json/databricks--databricks-dolly-15k-7427aa6e57c34282/0.0.0/e347ab1c932092252e717ff3f949105a4dd28b27e842dd53157d2f72e276c2e4/cache-9286c08bce910be1.arrow\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "### Instruction\n",
      "Identify the bird from the list: Mouse, Balearic shearwater, Giraffe\n",
      "\n",
      "### Answer\n",
      "Balearic shearwater</s>\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Loading cached processed dataset at /root/.cache/huggingface/datasets/databricks___json/databricks--databricks-dolly-15k-7427aa6e57c34282/0.0.0/e347ab1c932092252e717ff3f949105a4dd28b27e842dd53157d2f72e276c2e4/cache-75f16e9e4c95140b.arrow\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of samples: 1591\n"
     ]
    }
   ],
   "source": [
    "from random import randint\n",
    "from itertools import chain\n",
    "from functools import partial\n",
    "\n",
    "# template dataset to add prompt to each sample\n",
    "def template_dataset(sample):\n",
    "    sample[\"text\"] = f\"{format_dolly(sample)}{tokenizer.eos_token}\"\n",
    "    return sample\n",
    "\n",
    "# apply prompt template per sample\n",
    "dataset = dataset.map(template_dataset, remove_columns=list(dataset.features))\n",
    "# print random sample\n",
    "print(dataset[randint(0, len(dataset))][\"text\"])\n",
    "\n",
    "# empty list to save remainder from batches to use in next batch\n",
    "remainder = {\"input_ids\": [], \"attention_mask\": [], \"token_type_ids\": []}\n",
    "\n",
    "def chunk(sample, chunk_length=2048):\n",
    "    # define global remainder variable to save remainder from batches to use in next batch\n",
    "    global remainder\n",
    "    # Concatenate all texts and add remainder from previous batch\n",
    "    concatenated_examples = {k: list(chain(*sample[k])) for k in sample.keys()}\n",
    "    concatenated_examples = {k: remainder[k] + concatenated_examples[k] for k in concatenated_examples.keys()}\n",
    "    # get total number of tokens for batch\n",
    "    batch_total_length = len(concatenated_examples[list(sample.keys())[0]])\n",
    "\n",
    "    # get max number of chunks for batch\n",
    "    if batch_total_length >= chunk_length:\n",
    "        batch_chunk_length = (batch_total_length // chunk_length) * chunk_length\n",
    "\n",
    "    # Split by chunks of max_len.\n",
    "    result = {\n",
    "        k: [t[i : i + chunk_length] for i in range(0, batch_chunk_length, chunk_length)]\n",
    "        for k, t in concatenated_examples.items()\n",
    "    }\n",
    "    # add remainder to global variable for next batch\n",
    "    remainder = {k: concatenated_examples[k][batch_chunk_length:] for k in concatenated_examples.keys()}\n",
    "    # prepare labels\n",
    "    result[\"labels\"] = result[\"input_ids\"].copy()\n",
    "    return result\n",
    "\n",
    "\n",
    "# tokenize and chunk dataset\n",
    "lm_dataset = dataset.map(\n",
    "    lambda sample: tokenizer(sample[\"text\"]), batched=True, remove_columns=list(dataset.features)\n",
    ").map(\n",
    "    partial(chunk, chunk_length=2048),\n",
    "    batched=True,\n",
    ")\n",
    "\n",
    "# Print total number of samples\n",
    "print(f\"Total number of samples: {len(lm_dataset)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we processed the datasets we are going to use the new [FileSystem integration](https://huggingface.co/docs/datasets/filesystems) to upload our dataset to S3. We are using the `sess.default_bucket()`, adjust this if you want to store the dataset in a different S3 bucket. We will use the S3 path later in our training script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Saving the dataset (0/1 shards):   0%|          | 0/1591 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "uploaded data to:\n",
      "training dataset to: s3://sagemaker-us-east-1-079002598131/processed/llama/dolly/train\n"
     ]
    }
   ],
   "source": [
    "# save train_dataset to s3\n",
    "training_input_path = f's3://{sess.default_bucket()}/processed/llama/dolly/train'\n",
    "lm_dataset.save_to_disk(training_input_path)\n",
    "\n",
    "print(\"uploaded data to:\")\n",
    "print(f\"training dataset to: {training_input_path}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Fine-Tune LLaMA 7B on Amazon SageMaker with p4d.24xlarge\n",
    "\n",
    "We prepared a [train.py](./scripts/train.py), which trains our model. \n",
    "\n",
    "In order to create a sagemaker training job we need an `HuggingFace` Estimator. The Estimator handles end-to-end Amazon SageMaker training and deployment tasks. The Estimator manages the infrastructure use. \n",
    "\n",
    "SagMaker takes care of starting and managing all the required ec2 instances for us, provides the correct huggingface container, uploads the provided scripts and downloads the data from our S3 bucket into the container at `/opt/ml/input/data`. Then, it starts the training job by running."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from sagemaker.huggingface import HuggingFace\n",
    "from huggingface_hub import HfFolder\n",
    "\n",
    "# define Training Job Name \n",
    "job_name = f'huggingface-lora-{time.strftime(\"%Y-%m-%d-%H-%M-%S\", time.localtime())}'\n",
    "\n",
    "# hyperparameters, which are passed into the training job\n",
    "hyperparameters ={\n",
    "  'model_id': model_id,                             # pre-trained model\n",
    "  'dataset_path': '/opt/ml/input/data/training',    # path where sagemaker will save training dataset\n",
    "  'epochs': 3,                                      # number of training epochs\n",
    "  'per_device_train_batch_size': 2,                 # batch size for training\n",
    "  'lr': 2e-4,                                       # learning rate used during training\n",
    "}\n",
    "\n",
    "# create the Estimator\n",
    "huggingface_estimator = HuggingFace(\n",
    "    entry_point          = 'train.py',        # train script\n",
    "    source_dir           = 'scripts',         # directory which includes all the files needed for training\n",
    "    instance_type        = 'ml.p4d.24xlarge', # instances type used for the training job\n",
    "    instance_count       = 1,                 # the number of instances used for training\n",
    "    base_job_name        = job_name,          # the name of the training job\n",
    "    role                 = role,              # Iam role used in training job to access AWS ressources, e.g. S3\n",
    "    volume_size          = 300,               # the size of the EBS volume in GB\n",
    "    transformers_version = '4.28',            # the transformers version used in the training job\n",
    "    pytorch_version      = '2.0',             # the pytorch_version version used in the training job\n",
    "    py_version           = 'py310',           # the python version used in the training job\n",
    "    hyperparameters      =  hyperparameters,  # the hyperparameters passed to the training job\n",
    "    environment          = { \"HUGGINGFACE_HUB_CACHE\": \"/tmp/.cache\" }, # set env variable to cache models in /tmp\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now start our training job, with the `.fit()` method passing our S3 path to the training script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker.image_uris:image_uri is not presented, retrieving image_uri based on instance_type, framework etc.\n",
      "INFO:sagemaker:Creating training-job with name: huggingface-lora-2024-01-01-19-55-37-2024-01-01-19-55-38-266\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2024-01-01 19:55:38 Starting - Starting the training job\n",
      "2024-01-01 19:55:38 Pending - Training job waiting for capacity......\n",
      "2024-01-01 19:56:23 Pending - Preparing the instances for training.................................\n",
      "2024-01-01 20:01:54 Downloading - Downloading input data...\n",
      "2024-01-01 20:02:24 Downloading - Downloading the training image.....................\n",
      "2024-01-01 20:06:05 Training - Training image download completed. Training in progress........\u001b[34mbash: cannot set terminal process group (-1): Inappropriate ioctl for device\u001b[0m\n",
      "\u001b[34mbash: no job control in this shell\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:13,031 sagemaker-training-toolkit INFO     Imported framework sagemaker_pytorch_container.training\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:13,125 sagemaker-training-toolkit INFO     No Neurons detected (normal if no neurons installed)\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:13,133 sagemaker_pytorch_container.training INFO     Block until all host DNS lookups succeed.\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:13,134 sagemaker_pytorch_container.training INFO     Invoking user training script.\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:14,584 sagemaker-training-toolkit INFO     Installing dependencies from requirements.txt:\u001b[0m\n",
      "\u001b[34m/opt/conda/bin/python3.10 -m pip install -r requirements.txt\u001b[0m\n",
      "\u001b[34mCollecting transformers==4.31.0 (from -r requirements.txt (line 1))\u001b[0m\n",
      "\u001b[34mDownloading transformers-4.31.0-py3-none-any.whl (7.4 MB)\u001b[0m\n",
      "\u001b[34m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.4/7.4 MB 44.8 MB/s eta 0:00:00\u001b[0m\n",
      "\u001b[34mCollecting accelerate==0.21.0 (from -r requirements.txt (line 3))\u001b[0m\n",
      "\u001b[34mDownloading accelerate-0.21.0-py3-none-any.whl (244 kB)\u001b[0m\n",
      "\u001b[34m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 244.2/244.2 kB 43.0 MB/s eta 0:00:00\u001b[0m\n",
      "\u001b[34mCollecting safetensors>=0.3.1 (from -r requirements.txt (line 5))\u001b[0m\n",
      "\u001b[34mDownloading safetensors-0.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\u001b[0m\n",
      "\u001b[34m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 44.9 MB/s eta 0:00:00\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: tokenizers>=0.13.3 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 6)) (0.13.3)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers==4.31.0->-r requirements.txt (line 1)) (3.12.0)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: huggingface-hub<1.0,>=0.14.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.31.0->-r requirements.txt (line 1)) (0.14.1)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers==4.31.0->-r requirements.txt (line 1)) (1.23.5)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers==4.31.0->-r requirements.txt (line 1)) (23.1)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.31.0->-r requirements.txt (line 1)) (5.4.1)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers==4.31.0->-r requirements.txt (line 1)) (2023.5.5)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers==4.31.0->-r requirements.txt (line 1)) (2.28.2)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers==4.31.0->-r requirements.txt (line 1)) (4.65.0)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from accelerate==0.21.0->-r requirements.txt (line 3)) (5.9.5)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: torch>=1.10.0 in /opt/conda/lib/python3.10/site-packages (from accelerate==0.21.0->-r requirements.txt (line 3)) (2.0.0)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: fsspec in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.14.1->transformers==4.31.0->-r requirements.txt (line 1)) (2023.5.0)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.14.1->transformers==4.31.0->-r requirements.txt (line 1)) (4.5.0)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 3)) (1.11.1)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 3)) (3.1)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 3)) (3.1.2)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers==4.31.0->-r requirements.txt (line 1)) (3.1.0)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers==4.31.0->-r requirements.txt (line 1)) (3.4)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers==4.31.0->-r requirements.txt (line 1)) (1.26.15)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers==4.31.0->-r requirements.txt (line 1)) (2023.5.7)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 3)) (2.1.2)\u001b[0m\n",
      "\u001b[34mRequirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 3)) (1.3.0)\u001b[0m\n",
      "\u001b[34mInstalling collected packages: safetensors, transformers, accelerate\u001b[0m\n",
      "\u001b[34mAttempting uninstall: transformers\u001b[0m\n",
      "\u001b[34mFound existing installation: transformers 4.28.1\u001b[0m\n",
      "\u001b[34mUninstalling transformers-4.28.1:\u001b[0m\n",
      "\u001b[34mSuccessfully uninstalled transformers-4.28.1\u001b[0m\n",
      "\u001b[34mAttempting uninstall: accelerate\u001b[0m\n",
      "\u001b[34mFound existing installation: accelerate 0.19.0\u001b[0m\n",
      "\u001b[34mUninstalling accelerate-0.19.0:\u001b[0m\n",
      "\u001b[34mSuccessfully uninstalled accelerate-0.19.0\u001b[0m\n",
      "\u001b[34mSuccessfully installed accelerate-0.21.0 safetensors-0.4.1 transformers-4.31.0\u001b[0m\n",
      "\u001b[34mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n",
      "\u001b[34m[notice] A new release of pip is available: 23.1.2 -> 23.3.2\u001b[0m\n",
      "\u001b[34m[notice] To update, run: pip install --upgrade pip\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:22,304 sagemaker-training-toolkit INFO     Waiting for the process to finish and give a return code.\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:22,304 sagemaker-training-toolkit INFO     Done waiting for a return code. Received 0 from exiting process.\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:22,420 sagemaker-training-toolkit INFO     No Neurons detected (normal if no neurons installed)\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:22,527 sagemaker-training-toolkit INFO     No Neurons detected (normal if no neurons installed)\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:22,626 sagemaker-training-toolkit INFO     No Neurons detected (normal if no neurons installed)\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:22,635 sagemaker-training-toolkit INFO     Invoking user script\u001b[0m\n",
      "\u001b[34mTraining Env:\u001b[0m\n",
      "\u001b[34m{\n",
      "    \"additional_framework_parameters\": {},\n",
      "    \"channel_input_dirs\": {\n",
      "        \"training\": \"/opt/ml/input/data/training\"\n",
      "    },\n",
      "    \"current_host\": \"algo-1\",\n",
      "    \"current_instance_group\": \"homogeneousCluster\",\n",
      "    \"current_instance_group_hosts\": [\n",
      "        \"algo-1\"\n",
      "    ],\n",
      "    \"current_instance_type\": \"ml.p4d.24xlarge\",\n",
      "    \"distribution_hosts\": [],\n",
      "    \"distribution_instance_groups\": [],\n",
      "    \"framework_module\": \"sagemaker_pytorch_container.training:main\",\n",
      "    \"hosts\": [\n",
      "        \"algo-1\"\n",
      "    ],\n",
      "    \"hyperparameters\": {\n",
      "        \"dataset_path\": \"/opt/ml/input/data/training\",\n",
      "        \"epochs\": 3,\n",
      "        \"lr\": 0.0002,\n",
      "        \"model_id\": \"NousResearch/Llama-2-7b-hf\",\n",
      "        \"per_device_train_batch_size\": 2\n",
      "    },\n",
      "    \"input_config_dir\": \"/opt/ml/input/config\",\n",
      "    \"input_data_config\": {\n",
      "        \"training\": {\n",
      "            \"TrainingInputMode\": \"File\",\n",
      "            \"S3DistributionType\": \"FullyReplicated\",\n",
      "            \"RecordWrapperType\": \"None\"\n",
      "        }\n",
      "    },\n",
      "    \"input_dir\": \"/opt/ml/input\",\n",
      "    \"instance_groups\": [\n",
      "        \"homogeneousCluster\"\n",
      "    ],\n",
      "    \"instance_groups_dict\": {\n",
      "        \"homogeneousCluster\": {\n",
      "            \"instance_group_name\": \"homogeneousCluster\",\n",
      "            \"instance_type\": \"ml.p4d.24xlarge\",\n",
      "            \"hosts\": [\n",
      "                \"algo-1\"\n",
      "            ]\n",
      "        }\n",
      "    },\n",
      "    \"is_hetero\": false,\n",
      "    \"is_master\": true,\n",
      "    \"is_modelparallel_enabled\": null,\n",
      "    \"is_smddpmprun_installed\": true,\n",
      "    \"job_name\": \"huggingface-lora-2024-01-01-19-55-37-2024-01-01-19-55-38-266\",\n",
      "    \"log_level\": 20,\n",
      "    \"master_hostname\": \"algo-1\",\n",
      "    \"model_dir\": \"/opt/ml/model\",\n",
      "    \"module_dir\": \"s3://sagemaker-us-east-1-079002598131/huggingface-lora-2024-01-01-19-55-37-2024-01-01-19-55-38-266/source/sourcedir.tar.gz\",\n",
      "    \"module_name\": \"train\",\n",
      "    \"network_interface_name\": \"eth0\",\n",
      "    \"num_cpus\": 96,\n",
      "    \"num_gpus\": 8,\n",
      "    \"num_neurons\": 0,\n",
      "    \"output_data_dir\": \"/opt/ml/output/data\",\n",
      "    \"output_dir\": \"/opt/ml/output\",\n",
      "    \"output_intermediate_dir\": \"/opt/ml/output/intermediate\",\n",
      "    \"resource_config\": {\n",
      "        \"current_host\": \"algo-1\",\n",
      "        \"current_instance_type\": \"ml.p4d.24xlarge\",\n",
      "        \"current_group_name\": \"homogeneousCluster\",\n",
      "        \"hosts\": [\n",
      "            \"algo-1\"\n",
      "        ],\n",
      "        \"instance_groups\": [\n",
      "            {\n",
      "                \"instance_group_name\": \"homogeneousCluster\",\n",
      "                \"instance_type\": \"ml.p4d.24xlarge\",\n",
      "                \"hosts\": [\n",
      "                    \"algo-1\"\n",
      "                ]\n",
      "            }\n",
      "        ],\n",
      "        \"network_interface_name\": \"eth0\"\n",
      "    },\n",
      "    \"user_entry_point\": \"train.py\"\u001b[0m\n",
      "\u001b[34m}\u001b[0m\n",
      "\u001b[34mEnvironment variables:\u001b[0m\n",
      "\u001b[34mSM_HOSTS=[\"algo-1\"]\u001b[0m\n",
      "\u001b[34mSM_NETWORK_INTERFACE_NAME=eth0\u001b[0m\n",
      "\u001b[34mSM_HPS={\"dataset_path\":\"/opt/ml/input/data/training\",\"epochs\":3,\"lr\":0.0002,\"model_id\":\"NousResearch/Llama-2-7b-hf\",\"per_device_train_batch_size\":2}\u001b[0m\n",
      "\u001b[34mSM_USER_ENTRY_POINT=train.py\u001b[0m\n",
      "\u001b[34mSM_FRAMEWORK_PARAMS={}\u001b[0m\n",
      "\u001b[34mSM_RESOURCE_CONFIG={\"current_group_name\":\"homogeneousCluster\",\"current_host\":\"algo-1\",\"current_instance_type\":\"ml.p4d.24xlarge\",\"hosts\":[\"algo-1\"],\"instance_groups\":[{\"hosts\":[\"algo-1\"],\"instance_group_name\":\"homogeneousCluster\",\"instance_type\":\"ml.p4d.24xlarge\"}],\"network_interface_name\":\"eth0\"}\u001b[0m\n",
      "\u001b[34mSM_INPUT_DATA_CONFIG={\"training\":{\"RecordWrapperType\":\"None\",\"S3DistributionType\":\"FullyReplicated\",\"TrainingInputMode\":\"File\"}}\u001b[0m\n",
      "\u001b[34mSM_OUTPUT_DATA_DIR=/opt/ml/output/data\u001b[0m\n",
      "\u001b[34mSM_CHANNELS=[\"training\"]\u001b[0m\n",
      "\u001b[34mSM_CURRENT_HOST=algo-1\u001b[0m\n",
      "\u001b[34mSM_CURRENT_INSTANCE_TYPE=ml.p4d.24xlarge\u001b[0m\n",
      "\u001b[34mSM_CURRENT_INSTANCE_GROUP=homogeneousCluster\u001b[0m\n",
      "\u001b[34mSM_CURRENT_INSTANCE_GROUP_HOSTS=[\"algo-1\"]\u001b[0m\n",
      "\u001b[34mSM_INSTANCE_GROUPS=[\"homogeneousCluster\"]\u001b[0m\n",
      "\u001b[34mSM_INSTANCE_GROUPS_DICT={\"homogeneousCluster\":{\"hosts\":[\"algo-1\"],\"instance_group_name\":\"homogeneousCluster\",\"instance_type\":\"ml.p4d.24xlarge\"}}\u001b[0m\n",
      "\u001b[34mSM_DISTRIBUTION_INSTANCE_GROUPS=[]\u001b[0m\n",
      "\u001b[34mSM_IS_HETERO=false\u001b[0m\n",
      "\u001b[34mSM_MODULE_NAME=train\u001b[0m\n",
      "\u001b[34mSM_LOG_LEVEL=20\u001b[0m\n",
      "\u001b[34mSM_FRAMEWORK_MODULE=sagemaker_pytorch_container.training:main\u001b[0m\n",
      "\u001b[34mSM_INPUT_DIR=/opt/ml/input\u001b[0m\n",
      "\u001b[34mSM_INPUT_CONFIG_DIR=/opt/ml/input/config\u001b[0m\n",
      "\u001b[34mSM_OUTPUT_DIR=/opt/ml/output\u001b[0m\n",
      "\u001b[34mSM_NUM_CPUS=96\u001b[0m\n",
      "\u001b[34mSM_NUM_GPUS=8\u001b[0m\n",
      "\u001b[34mSM_NUM_NEURONS=0\u001b[0m\n",
      "\u001b[34mSM_MODEL_DIR=/opt/ml/model\u001b[0m\n",
      "\u001b[34mSM_MODULE_DIR=s3://sagemaker-us-east-1-079002598131/huggingface-lora-2024-01-01-19-55-37-2024-01-01-19-55-38-266/source/sourcedir.tar.gz\u001b[0m\n",
      "\u001b[34mSM_TRAINING_ENV={\"additional_framework_parameters\":{},\"channel_input_dirs\":{\"training\":\"/opt/ml/input/data/training\"},\"current_host\":\"algo-1\",\"current_instance_group\":\"homogeneousCluster\",\"current_instance_group_hosts\":[\"algo-1\"],\"current_instance_type\":\"ml.p4d.24xlarge\",\"distribution_hosts\":[],\"distribution_instance_groups\":[],\"framework_module\":\"sagemaker_pytorch_container.training:main\",\"hosts\":[\"algo-1\"],\"hyperparameters\":{\"dataset_path\":\"/opt/ml/input/data/training\",\"epochs\":3,\"lr\":0.0002,\"model_id\":\"NousResearch/Llama-2-7b-hf\",\"per_device_train_batch_size\":2},\"input_config_dir\":\"/opt/ml/input/config\",\"input_data_config\":{\"training\":{\"RecordWrapperType\":\"None\",\"S3DistributionType\":\"FullyReplicated\",\"TrainingInputMode\":\"File\"}},\"input_dir\":\"/opt/ml/input\",\"instance_groups\":[\"homogeneousCluster\"],\"instance_groups_dict\":{\"homogeneousCluster\":{\"hosts\":[\"algo-1\"],\"instance_group_name\":\"homogeneousCluster\",\"instance_type\":\"ml.p4d.24xlarge\"}},\"is_hetero\":false,\"is_master\":true,\"is_modelparallel_enabled\":null,\"is_smddpmprun_installed\":true,\"job_name\":\"huggingface-lora-2024-01-01-19-55-37-2024-01-01-19-55-38-266\",\"log_level\":20,\"master_hostname\":\"algo-1\",\"model_dir\":\"/opt/ml/model\",\"module_dir\":\"s3://sagemaker-us-east-1-079002598131/huggingface-lora-2024-01-01-19-55-37-2024-01-01-19-55-38-266/source/sourcedir.tar.gz\",\"module_name\":\"train\",\"network_interface_name\":\"eth0\",\"num_cpus\":96,\"num_gpus\":8,\"num_neurons\":0,\"output_data_dir\":\"/opt/ml/output/data\",\"output_dir\":\"/opt/ml/output\",\"output_intermediate_dir\":\"/opt/ml/output/intermediate\",\"resource_config\":{\"current_group_name\":\"homogeneousCluster\",\"current_host\":\"algo-1\",\"current_instance_type\":\"ml.p4d.24xlarge\",\"hosts\":[\"algo-1\"],\"instance_groups\":[{\"hosts\":[\"algo-1\"],\"instance_group_name\":\"homogeneousCluster\",\"instance_type\":\"ml.p4d.24xlarge\"}],\"network_interface_name\":\"eth0\"},\"user_entry_point\":\"train.py\"}\u001b[0m\n",
      "\u001b[34mSM_USER_ARGS=[\"--dataset_path\",\"/opt/ml/input/data/training\",\"--epochs\",\"3\",\"--lr\",\"0.0002\",\"--model_id\",\"NousResearch/Llama-2-7b-hf\",\"--per_device_train_batch_size\",\"2\"]\u001b[0m\n",
      "\u001b[34mSM_OUTPUT_INTERMEDIATE_DIR=/opt/ml/output/intermediate\u001b[0m\n",
      "\u001b[34mSM_CHANNEL_TRAINING=/opt/ml/input/data/training\u001b[0m\n",
      "\u001b[34mSM_HP_DATASET_PATH=/opt/ml/input/data/training\u001b[0m\n",
      "\u001b[34mSM_HP_EPOCHS=3\u001b[0m\n",
      "\u001b[34mSM_HP_LR=0.0002\u001b[0m\n",
      "\u001b[34mSM_HP_MODEL_ID=NousResearch/Llama-2-7b-hf\u001b[0m\n",
      "\u001b[34mSM_HP_PER_DEVICE_TRAIN_BATCH_SIZE=2\u001b[0m\n",
      "\u001b[34mPYTHONPATH=/opt/ml/code:/opt/conda/bin:/opt/conda/lib/python310.zip:/opt/conda/lib/python3.10:/opt/conda/lib/python3.10/lib-dynload:/opt/conda/lib/python3.10/site-packages\u001b[0m\n",
      "\u001b[34mInvoking script with the following command:\u001b[0m\n",
      "\u001b[34m/opt/conda/bin/python3.10 train.py --dataset_path /opt/ml/input/data/training --epochs 3 --lr 0.0002 --model_id NousResearch/Llama-2-7b-hf --per_device_train_batch_size 2\u001b[0m\n",
      "\u001b[34m2024-01-01 20:07:22,662 sagemaker-training-toolkit INFO     Exceptions not imported for SageMaker TF as Tensorflow is not installed.\u001b[0m\n",
      "\u001b[34mDownloading config.json:   0%|          | 0.00/583 [00:00<?, ?B/s]\u001b[0m\n",
      "\u001b[34mDownloading config.json: 100%|██████████| 583/583 [00:00<00:00, 5.81MB/s]\u001b[0m\n",
      "\u001b[34mDownloading (…)fetensors.index.json:   0%|          | 0.00/26.8k [00:00<?, ?B/s]\u001b[0m\n",
      "\u001b[34mDownloading (…)fetensors.index.json: 100%|██████████| 26.8k/26.8k [00:00<00:00, 63.2MB/s]\u001b[0m\n",
      "\u001b[34mDownloading shards:   0%|          | 0/2 [00:00<?, ?it/s]\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   0%|          | 0.00/9.98G [00:00<?, ?B/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   0%|          | 41.9M/9.98G [00:00<00:28, 352MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   1%|          | 83.9M/9.98G [00:00<00:30, 322MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   1%|▏         | 126M/9.98G [00:00<00:28, 349MB/s] #033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   2%|▏         | 168M/9.98G [00:00<00:30, 327MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   2%|▏         | 220M/9.98G [00:00<00:27, 359MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   3%|▎         | 262M/9.98G [00:00<00:29, 328MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   3%|▎         | 304M/9.98G [00:00<00:27, 346MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   3%|▎         | 346M/9.98G [00:01<00:39, 246MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   4%|▍         | 388M/9.98G [00:01<00:36, 265MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   4%|▍         | 440M/9.98G [00:01<00:31, 305MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   5%|▍         | 482M/9.98G [00:01<00:29, 326MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   5%|▌         | 524M/9.98G [00:01<00:29, 320MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   6%|▌         | 566M/9.98G [00:01<00:29, 323MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   6%|▌         | 619M/9.98G [00:01<00:26, 352MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   7%|▋         | 661M/9.98G [00:02<00:28, 332MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   7%|▋         | 703M/9.98G [00:02<00:27, 336MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   7%|▋         | 744M/9.98G [00:02<00:26, 354MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   8%|▊         | 786M/9.98G [00:02<00:26, 348MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   8%|▊         | 839M/9.98G [00:02<00:24, 373MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   9%|▉         | 881M/9.98G [00:02<00:23, 385MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   9%|▉         | 933M/9.98G [00:02<00:22, 397MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  10%|▉         | 986M/9.98G [00:02<00:22, 404MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  10%|█         | 1.04G/9.98G [00:02<00:21, 419MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  11%|█         | 1.09G/9.98G [00:03<00:22, 388MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  11%|█▏        | 1.13G/9.98G [00:03<00:23, 379MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  12%|█▏        | 1.17G/9.98G [00:03<00:25, 343MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  12%|█▏        | 1.22G/9.98G [00:03<00:26, 333MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  13%|█▎        | 1.26G/9.98G [00:03<00:25, 338MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  13%|█▎        | 1.30G/9.98G [00:03<00:26, 323MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  13%|█▎        | 1.34G/9.98G [00:03<00:28, 304MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  14%|█▍        | 1.38G/9.98G [00:04<00:26, 320MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  14%|█▍        | 1.43G/9.98G [00:04<00:24, 344MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  15%|█▍        | 1.47G/9.98G [00:04<00:23, 359MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  15%|█▌        | 1.51G/9.98G [00:04<00:22, 375MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  16%|█▌        | 1.55G/9.98G [00:04<00:22, 381MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  16%|█▌        | 1.59G/9.98G [00:04<00:22, 367MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  16%|█▋        | 1.64G/9.98G [00:04<00:23, 361MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  17%|█▋        | 1.69G/9.98G [00:04<00:21, 389MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  17%|█▋        | 1.73G/9.98G [00:05<00:23, 356MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  18%|█▊        | 1.77G/9.98G [00:05<00:23, 356MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  18%|█▊        | 1.81G/9.98G [00:05<00:22, 363MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  19%|█▊        | 1.86G/9.98G [00:05<00:22, 358MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  19%|█▉        | 1.90G/9.98G [00:05<00:22, 361MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  19%|█▉        | 1.94G/9.98G [00:05<00:21, 374MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  20%|█▉        | 1.98G/9.98G [00:05<00:21, 367MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  20%|██        | 2.02G/9.98G [00:05<00:21, 375MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  21%|██        | 2.07G/9.98G [00:05<00:21, 362MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  21%|██        | 2.11G/9.98G [00:06<00:21, 360MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  22%|██▏       | 2.15G/9.98G [00:06<00:22, 353MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  22%|██▏       | 2.20G/9.98G [00:06<00:20, 378MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  23%|██▎       | 2.25G/9.98G [00:06<00:19, 399MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  23%|██▎       | 2.31G/9.98G [00:06<00:18, 404MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  24%|██▎       | 2.36G/9.98G [00:06<00:18, 420MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  24%|██▍       | 2.41G/9.98G [00:06<00:17, 435MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  25%|██▍       | 2.46G/9.98G [00:06<00:16, 456MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  25%|██▌       | 2.52G/9.98G [00:06<00:15, 473MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  26%|██▌       | 2.57G/9.98G [00:07<00:15, 483MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  26%|██▋       | 2.62G/9.98G [00:07<00:15, 476MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  27%|██▋       | 2.67G/9.98G [00:07<00:15, 457MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  27%|██▋       | 2.73G/9.98G [00:07<00:16, 444MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  28%|██▊       | 2.78G/9.98G [00:07<00:17, 409MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  28%|██▊       | 2.82G/9.98G [00:07<00:17, 409MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  29%|██▊       | 2.86G/9.98G [00:07<00:18, 391MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  29%|██▉       | 2.90G/9.98G [00:07<00:18, 380MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  30%|██▉       | 2.95G/9.98G [00:08<00:19, 360MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  30%|██▉       | 2.99G/9.98G [00:08<00:20, 340MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  30%|███       | 3.04G/9.98G [00:08<00:19, 349MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  31%|███       | 3.08G/9.98G [00:08<00:22, 311MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  31%|███▏      | 3.12G/9.98G [00:08<00:21, 324MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  32%|███▏      | 3.17G/9.98G [00:08<00:20, 327MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  32%|███▏      | 3.21G/9.98G [00:08<00:20, 331MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  33%|███▎      | 3.25G/9.98G [00:08<00:19, 347MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  33%|███▎      | 3.30G/9.98G [00:09<00:18, 365MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  34%|███▎      | 3.34G/9.98G [00:09<00:18, 365MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  34%|███▍      | 3.39G/9.98G [00:09<00:18, 362MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  34%|███▍      | 3.44G/9.98G [00:09<00:16, 396MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  35%|███▍      | 3.48G/9.98G [00:09<00:16, 395MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  35%|███▌      | 3.52G/9.98G [00:09<00:17, 375MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  36%|███▌      | 3.57G/9.98G [00:09<00:17, 371MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  36%|███▌      | 3.61G/9.98G [00:09<00:17, 372MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  37%|███▋      | 3.66G/9.98G [00:10<00:16, 389MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  37%|███▋      | 3.71G/9.98G [00:10<00:15, 404MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  38%|███▊      | 3.76G/9.98G [00:10<00:14, 435MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  38%|███▊      | 3.82G/9.98G [00:10<00:14, 412MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  39%|███▉      | 3.87G/9.98G [00:10<00:15, 407MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  39%|███▉      | 3.91G/9.98G [00:10<00:15, 386MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  40%|███▉      | 3.95G/9.98G [00:10<00:16, 362MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  40%|████      | 4.00G/9.98G [00:10<00:16, 355MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  40%|████      | 4.04G/9.98G [00:11<00:17, 349MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  41%|████      | 4.08G/9.98G [00:11<00:16, 363MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  41%|████▏     | 4.12G/9.98G [00:11<00:21, 267MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  42%|████▏     | 4.15G/9.98G [00:11<00:21, 277MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  42%|████▏     | 4.18G/9.98G [00:11<00:21, 275MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  42%|████▏     | 4.23G/9.98G [00:11<00:19, 291MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  43%|████▎     | 4.27G/9.98G [00:11<00:19, 299MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  43%|████▎     | 4.31G/9.98G [00:11<00:17, 319MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  44%|████▎     | 4.35G/9.98G [00:12<00:17, 329MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  44%|████▍     | 4.40G/9.98G [00:12<00:15, 361MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  45%|████▍     | 4.45G/9.98G [00:12<00:15, 351MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  45%|████▍     | 4.49G/9.98G [00:12<00:17, 318MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  46%|████▌     | 4.54G/9.98G [00:12<00:15, 348MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  46%|████▌     | 4.58G/9.98G [00:12<00:15, 344MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  46%|████▋     | 4.62G/9.98G [00:12<00:18, 293MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  47%|████▋     | 4.68G/9.98G [00:13<00:15, 332MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  47%|████▋     | 4.72G/9.98G [00:13<00:15, 344MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  48%|████▊     | 4.76G/9.98G [00:13<00:15, 342MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  48%|████▊     | 4.80G/9.98G [00:13<00:14, 361MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  49%|████▊     | 4.85G/9.98G [00:13<00:13, 372MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  49%|████▉     | 4.90G/9.98G [00:13<00:14, 353MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  50%|████▉     | 4.94G/9.98G [00:13<00:14, 352MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  50%|████▉     | 4.98G/9.98G [00:13<00:15, 331MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  50%|█████     | 5.02G/9.98G [00:14<00:14, 342MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  51%|█████     | 5.06G/9.98G [00:14<00:14, 350MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  51%|█████     | 5.11G/9.98G [00:14<00:13, 360MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  52%|█████▏    | 5.16G/9.98G [00:14<00:12, 383MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  52%|█████▏    | 5.20G/9.98G [00:14<00:12, 368MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  53%|█████▎    | 5.24G/9.98G [00:14<00:13, 361MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  53%|█████▎    | 5.28G/9.98G [00:14<00:13, 344MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  53%|█████▎    | 5.34G/9.98G [00:14<00:12, 365MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  54%|█████▍    | 5.39G/9.98G [00:15<00:11, 391MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  55%|█████▍    | 5.44G/9.98G [00:15<00:11, 409MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  55%|█████▌    | 5.49G/9.98G [00:15<00:10, 425MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  56%|█████▌    | 5.55G/9.98G [00:15<00:09, 444MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  56%|█████▌    | 5.60G/9.98G [00:15<00:10, 436MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  57%|█████▋    | 5.65G/9.98G [00:15<00:10, 396MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  57%|█████▋    | 5.69G/9.98G [00:15<00:11, 379MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  58%|█████▊    | 5.75G/9.98G [00:15<00:10, 390MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  58%|█████▊    | 5.79G/9.98G [00:16<00:10, 383MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  58%|█████▊    | 5.83G/9.98G [00:16<00:11, 372MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  59%|█████▉    | 5.87G/9.98G [00:16<00:11, 367MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  59%|█████▉    | 5.91G/9.98G [00:16<00:10, 375MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  60%|█████▉    | 5.96G/9.98G [00:16<00:10, 371MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  60%|██████    | 6.01G/9.98G [00:16<00:10, 395MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  61%|██████    | 6.05G/9.98G [00:16<00:11, 336MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  61%|██████    | 6.09G/9.98G [00:16<00:10, 354MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  61%|██████▏   | 6.13G/9.98G [00:16<00:10, 360MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  62%|██████▏   | 6.20G/9.98G [00:17<00:09, 410MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  63%|██████▎   | 6.24G/9.98G [00:17<00:09, 399MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  63%|██████▎   | 6.28G/9.98G [00:17<00:09, 401MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  63%|██████▎   | 6.32G/9.98G [00:17<00:09, 399MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  64%|██████▍   | 6.38G/9.98G [00:17<00:08, 414MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  64%|██████▍   | 6.42G/9.98G [00:17<00:08, 398MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  65%|██████▍   | 6.46G/9.98G [00:17<00:08, 392MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  65%|██████▌   | 6.51G/9.98G [00:17<00:08, 414MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  66%|██████▌   | 6.56G/9.98G [00:17<00:08, 423MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  66%|██████▋   | 6.62G/9.98G [00:18<00:07, 438MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  67%|██████▋   | 6.67G/9.98G [00:18<00:07, 446MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  67%|██████▋   | 6.72G/9.98G [00:18<00:08, 404MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  68%|██████▊   | 6.76G/9.98G [00:18<00:08, 365MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  68%|██████▊   | 6.81G/9.98G [00:18<00:08, 364MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  69%|██████▊   | 6.85G/9.98G [00:18<00:08, 377MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  69%|██████▉   | 6.89G/9.98G [00:18<00:08, 363MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  69%|██████▉   | 6.93G/9.98G [00:19<00:08, 342MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  70%|██████▉   | 6.97G/9.98G [00:19<00:08, 354MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  70%|███████   | 7.01G/9.98G [00:19<00:09, 317MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  71%|███████   | 7.06G/9.98G [00:19<00:08, 336MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  71%|███████   | 7.10G/9.98G [00:19<00:09, 304MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  72%|███████▏  | 7.14G/9.98G [00:19<00:08, 326MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  72%|███████▏  | 7.18G/9.98G [00:19<00:08, 330MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  72%|███████▏  | 7.22G/9.98G [00:19<00:08, 340MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  73%|███████▎  | 7.28G/9.98G [00:20<00:07, 381MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  73%|███████▎  | 7.32G/9.98G [00:20<00:06, 381MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  74%|███████▍  | 7.36G/9.98G [00:20<00:07, 374MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  74%|███████▍  | 7.40G/9.98G [00:20<00:06, 380MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  75%|███████▍  | 7.46G/9.98G [00:20<00:06, 400MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  75%|███████▌  | 7.50G/9.98G [00:20<00:06, 391MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  76%|███████▌  | 7.55G/9.98G [00:20<00:06, 398MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  76%|███████▌  | 7.59G/9.98G [00:20<00:06, 397MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  77%|███████▋  | 7.63G/9.98G [00:20<00:05, 398MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  77%|███████▋  | 7.69G/9.98G [00:21<00:05, 412MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  77%|███████▋  | 7.73G/9.98G [00:21<00:05, 410MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  78%|███████▊  | 7.77G/9.98G [00:21<00:05, 409MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  78%|███████▊  | 7.82G/9.98G [00:21<00:05, 423MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  79%|███████▉  | 7.87G/9.98G [00:21<00:07, 284MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  79%|███████▉  | 7.92G/9.98G [00:21<00:06, 311MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  80%|███████▉  | 7.96G/9.98G [00:21<00:06, 335MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  80%|████████  | 8.01G/9.98G [00:21<00:05, 359MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  81%|████████  | 8.05G/9.98G [00:22<00:05, 365MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  81%|████████  | 8.10G/9.98G [00:22<00:05, 354MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  82%|████████▏ | 8.14G/9.98G [00:22<00:05, 344MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  82%|████████▏ | 8.18G/9.98G [00:22<00:05, 346MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  82%|████████▏ | 8.22G/9.98G [00:22<00:06, 252MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  83%|████████▎ | 8.25G/9.98G [00:22<00:06, 259MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  83%|████████▎ | 8.28G/9.98G [00:22<00:06, 264MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  83%|████████▎ | 8.33G/9.98G [00:23<00:05, 300MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  84%|████████▍ | 8.37G/9.98G [00:23<00:05, 322MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  84%|████████▍ | 8.42G/9.98G [00:23<00:04, 359MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  85%|████████▍ | 8.46G/9.98G [00:23<00:04, 363MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  85%|████████▌ | 8.51G/9.98G [00:23<00:03, 389MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  86%|████████▌ | 8.56G/9.98G [00:23<00:03, 368MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  86%|████████▋ | 8.61G/9.98G [00:23<00:03, 389MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  87%|████████▋ | 8.65G/9.98G [00:23<00:03, 397MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  87%|████████▋ | 8.69G/9.98G [00:23<00:03, 387MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  88%|████████▊ | 8.73G/9.98G [00:24<00:03, 388MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  88%|████████▊ | 8.78G/9.98G [00:24<00:03, 371MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  88%|████████▊ | 8.83G/9.98G [00:24<00:02, 385MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  89%|████████▉ | 8.87G/9.98G [00:24<00:03, 334MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  89%|████████▉ | 8.92G/9.98G [00:24<00:02, 361MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  90%|████████▉ | 8.97G/9.98G [00:24<00:02, 368MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  90%|█████████ | 9.01G/9.98G [00:24<00:02, 371MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  91%|█████████ | 9.05G/9.98G [00:24<00:02, 362MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  91%|█████████ | 9.10G/9.98G [00:25<00:02, 404MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  92%|█████████▏| 9.15G/9.98G [00:25<00:01, 428MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  92%|█████████▏| 9.21G/9.98G [00:25<00:01, 433MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  93%|█████████▎| 9.26G/9.98G [00:25<00:01, 422MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  93%|█████████▎| 9.31G/9.98G [00:25<00:01, 425MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  94%|█████████▍| 9.36G/9.98G [00:25<00:01, 376MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  94%|█████████▍| 9.41G/9.98G [00:25<00:01, 384MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  95%|█████████▍| 9.45G/9.98G [00:25<00:01, 350MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  95%|█████████▌| 9.49G/9.98G [00:26<00:01, 356MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  96%|█████████▌| 9.53G/9.98G [00:26<00:01, 342MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  96%|█████████▌| 9.57G/9.98G [00:26<00:01, 355MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  96%|█████████▋| 9.62G/9.98G [00:26<00:01, 339MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  97%|█████████▋| 9.66G/9.98G [00:26<00:00, 330MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  97%|█████████▋| 9.70G/9.98G [00:26<00:00, 350MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  98%|█████████▊| 9.74G/9.98G [00:26<00:00, 359MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  98%|█████████▊| 9.79G/9.98G [00:26<00:00, 387MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  99%|█████████▊| 9.85G/9.98G [00:27<00:00, 403MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  99%|█████████▉| 9.89G/9.98G [00:27<00:00, 404MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors: 100%|█████████▉| 9.93G/9.98G [00:27<00:00, 373MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors: 100%|██████████| 9.98G/9.98G [00:27<00:00, 393MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors: 100%|██████████| 9.98G/9.98G [00:27<00:00, 364MB/s]\u001b[0m\n",
      "\u001b[34mDownloading shards:  50%|█████     | 1/2 [00:27<00:27, 27.48s/it]\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   0%|          | 0.00/3.50G [00:00<?, ?B/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   1%|▏         | 52.4M/3.50G [00:00<00:06, 503MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   3%|▎         | 105M/3.50G [00:00<00:08, 409MB/s] #033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   4%|▍         | 157M/3.50G [00:00<00:09, 357MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   6%|▌         | 210M/3.50G [00:00<00:08, 392MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   7%|▋         | 252M/3.50G [00:00<00:08, 369MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:   8%|▊         | 294M/3.50G [00:00<00:09, 353MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  10%|▉         | 336M/3.50G [00:00<00:08, 365MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  11%|█         | 377M/3.50G [00:01<00:08, 365MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  12%|█▏        | 430M/3.50G [00:01<00:07, 392MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  13%|█▎        | 472M/3.50G [00:01<00:07, 380MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  15%|█▍        | 514M/3.50G [00:01<00:07, 383MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  16%|█▌        | 566M/3.50G [00:01<00:07, 402MB/s]\u001b[0m\n",
      "\u001b[34m#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  17%|█▋        | 608M/3.50G [00:01<00:07, 379MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  19%|█▊        | 650M/3.50G [00:01<00:07, 358MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  20%|█▉        | 692M/3.50G [00:01<00:07, 365MB/s]\u001b[0m\n",
      "\u001b[34m#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  21%|██        | 734M/3.50G [00:01<00:07, 375MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  22%|██▏       | 786M/3.50G [00:02<00:07, 384MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  24%|██▎       | 828M/3.50G [00:02<00:07, 374MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  25%|██▍       | 870M/3.50G [00:02<00:07, 367MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  26%|██▌       | 912M/3.50G [00:02<00:06, 379MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  27%|██▋       | 954M/3.50G [00:02<00:06, 375MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  29%|██▉       | 1.01G/3.50G [00:02<00:06, 366MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  30%|██▉       | 1.05G/3.50G [00:02<00:06, 378MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  31%|███       | 1.09G/3.50G [00:02<00:06, 364MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  32%|███▏      | 1.13G/3.50G [00:03<00:06, 341MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  34%|███▍      | 1.18G/3.50G [00:03<00:06, 371MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  35%|███▌      | 1.24G/3.50G [00:03<00:05, 390MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  37%|███▋      | 1.29G/3.50G [00:03<00:05, 403MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  38%|███▊      | 1.34G/3.50G [00:03<00:05, 411MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  40%|███▉      | 1.38G/3.50G [00:03<00:06, 351MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  41%|████      | 1.43G/3.50G [00:03<00:05, 352MB/s]\u001b[0m\n",
      "\u001b[34m#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  42%|████▏     | 1.47G/3.50G [00:03<00:05, 357MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  43%|████▎     | 1.51G/3.50G [00:04<00:05, 368MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  44%|████▍     | 1.55G/3.50G [00:04<00:05, 381MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  46%|████▌     | 1.59G/3.50G [00:04<00:04, 383MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  47%|████▋     | 1.65G/3.50G [00:04<00:04, 393MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  48%|████▊     | 1.69G/3.50G [00:04<00:04, 378MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  50%|████▉     | 1.74G/3.50G [00:04<00:04, 404MB/s]\u001b[0m\n",
      "\u001b[34m#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  51%|█████     | 1.79G/3.50G [00:04<00:03, 435MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  53%|█████▎    | 1.85G/3.50G [00:04<00:03, 426MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  54%|█████▍    | 1.90G/3.50G [00:04<00:03, 435MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  56%|█████▌    | 1.95G/3.50G [00:05<00:03, 402MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  57%|█████▋    | 1.99G/3.50G [00:05<00:04, 361MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  58%|█████▊    | 2.03G/3.50G [00:05<00:03, 369MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  60%|█████▉    | 2.09G/3.50G [00:05<00:03, 393MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  61%|██████    | 2.13G/3.50G [00:05<00:03, 394MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  62%|██████▏   | 2.17G/3.50G [00:05<00:03, 369MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  63%|██████▎   | 2.21G/3.50G [00:05<00:03, 350MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  64%|██████▍   | 2.25G/3.50G [00:05<00:03, 349MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  66%|██████▌   | 2.30G/3.50G [00:06<00:03, 367MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  67%|██████▋   | 2.34G/3.50G [00:06<00:03, 352MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  68%|██████▊   | 2.39G/3.50G [00:06<00:02, 382MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  70%|██████▉   | 2.44G/3.50G [00:06<00:02, 396MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  71%|███████▏  | 2.50G/3.50G [00:06<00:02, 414MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  73%|███████▎  | 2.55G/3.50G [00:06<00:02, 435MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  74%|███████▍  | 2.60G/3.50G [00:06<00:02, 433MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  76%|███████▌  | 2.65G/3.50G [00:06<00:01, 439MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  77%|███████▋  | 2.71G/3.50G [00:07<00:01, 424MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  79%|███████▉  | 2.76G/3.50G [00:07<00:01, 381MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  80%|███████▉  | 2.80G/3.50G [00:07<00:01, 357MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  81%|████████  | 2.84G/3.50G [00:07<00:01, 334MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  82%|████████▏ | 2.88G/3.50G [00:07<00:01, 345MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  84%|████████▎ | 2.93G/3.50G [00:07<00:01, 357MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  85%|████████▍ | 2.97G/3.50G [00:07<00:01, 350MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  86%|████████▌ | 3.01G/3.50G [00:07<00:01, 318MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  87%|████████▋ | 3.05G/3.50G [00:08<00:01, 334MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  88%|████████▊ | 3.09G/3.50G [00:08<00:01, 301MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  89%|████████▉ | 3.12G/3.50G [00:08<00:01, 280MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  90%|█████████ | 3.16G/3.50G [00:08<00:01, 269MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  91%|█████████ | 3.19G/3.50G [00:08<00:01, 247MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  93%|█████████▎| 3.24G/3.50G [00:08<00:00, 302MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  94%|█████████▍| 3.28G/3.50G [00:08<00:00, 317MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  95%|█████████▌| 3.33G/3.50G [00:09<00:00, 366MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  96%|█████████▋| 3.38G/3.50G [00:09<00:00, 343MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  98%|█████████▊| 3.43G/3.50G [00:09<00:00, 372MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors:  99%|█████████▉| 3.48G/3.50G [00:09<00:00, 407MB/s]#033[A\u001b[0m\n",
      "\u001b[34mDownloading (…)of-00002.safetensors: 100%|██████████| 3.50G/3.50G [00:09<00:00, 370MB/s]\u001b[0m\n",
      "\u001b[34mDownloading shards: 100%|██████████| 2/2 [00:36<00:00, 16.89s/it]\u001b[0m\n",
      "\u001b[34mDownloading shards: 100%|██████████| 2/2 [00:36<00:00, 18.48s/it]\u001b[0m\n",
      "\u001b[34mLoading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]\u001b[0m\n",
      "\u001b[34mLoading checkpoint shards:  50%|█████     | 1/2 [00:06<00:06,  6.73s/it]\u001b[0m\n",
      "\u001b[34mLoading checkpoint shards: 100%|██████████| 2/2 [00:09<00:00,  4.23s/it]\u001b[0m\n",
      "\u001b[34mLoading checkpoint shards: 100%|██████████| 2/2 [00:09<00:00,  4.61s/it]\u001b[0m\n",
      "\u001b[34mDownloading generation_config.json:   0%|          | 0.00/179 [00:00<?, ?B/s]\u001b[0m\n",
      "\u001b[34mDownloading generation_config.json: 100%|██████████| 179/179 [00:00<00:00, 1.98MB/s]\u001b[0m\n",
      "\u001b[34m/opt/conda/lib/python3.10/site-packages/transformers/optimization.py:411: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
      "  warnings.warn(\u001b[0m\n",
      "\u001b[34m0%|          | 0/2388 [00:00<?, ?it/s]\u001b[0m\n",
      "\u001b[34m0%|          | 1/2388 [00:03<2:12:03,  3.32s/it]\u001b[0m\n",
      "\u001b[34m0%|          | 2/2388 [00:05<1:45:50,  2.66s/it]\u001b[0m\n",
      "\u001b[34m0%|          | 3/2388 [00:07<1:37:48,  2.46s/it]\u001b[0m\n",
      "\u001b[34m0%|          | 4/2388 [00:09<1:34:02,  2.37s/it]\u001b[0m\n",
      "\u001b[34m0%|          | 5/2388 [00:12<1:31:56,  2.31s/it]\u001b[0m\n",
      "\u001b[34m0%|          | 6/2388 [00:14<1:30:39,  2.28s/it]\u001b[0m\n",
      "\u001b[34m0%|          | 7/2388 [00:16<1:29:50,  2.26s/it]\u001b[0m\n",
      "\u001b[34m0%|          | 8/2388 [00:18<1:29:16,  2.25s/it]\u001b[0m\n",
      "\u001b[34m0%|          | 9/2388 [00:21<1:28:52,  2.24s/it]\u001b[0m\n",
      "\u001b[34m0%|          | 10/2388 [00:23<1:28:36,  2.24s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 2.3035, 'learning_rate': 0.00019916247906197655, 'epoch': 0.01}\u001b[0m\n",
      "\u001b[34m0%|          | 10/2388 [00:23<1:28:36,  2.24s/it]\u001b[0m\n",
      "\u001b[34m0%|          | 11/2388 [00:25<1:28:24,  2.23s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 12/2388 [00:27<1:28:15,  2.23s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 13/2388 [00:29<1:28:08,  2.23s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 14/2388 [00:32<1:28:02,  2.23s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 15/2388 [00:34<1:27:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 16/2388 [00:36<1:27:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 17/2388 [00:38<1:27:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 18/2388 [00:41<1:27:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 19/2388 [00:43<1:27:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 20/2388 [00:45<1:27:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.71, 'learning_rate': 0.0001983249581239531, 'epoch': 0.03}\u001b[0m\n",
      "\u001b[34m1%|          | 20/2388 [00:45<1:27:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 21/2388 [00:47<1:27:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 22/2388 [00:49<1:27:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 23/2388 [00:52<1:27:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 24/2388 [00:54<1:27:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 25/2388 [00:56<1:27:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 26/2388 [00:58<1:27:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 27/2388 [01:01<1:27:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 28/2388 [01:03<1:27:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m1%|          | 29/2388 [01:05<1:27:02,  2.21s/it]\u001b[0m\n",
      "\u001b[34m1%|▏         | 30/2388 [01:07<1:26:58,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 8.9335, 'learning_rate': 0.00019748743718592964, 'epoch': 0.04}\u001b[0m\n",
      "\u001b[34m1%|▏         | 30/2388 [01:07<1:26:58,  2.21s/it]\u001b[0m\n",
      "\u001b[34m1%|▏         | 31/2388 [01:09<1:26:57,  2.21s/it]\u001b[0m\n",
      "\u001b[34m1%|▏         | 32/2388 [01:12<1:26:54,  2.21s/it]\u001b[0m\n",
      "\u001b[34m1%|▏         | 33/2388 [01:14<1:26:52,  2.21s/it]\u001b[0m\n",
      "\u001b[34m1%|▏         | 34/2388 [01:16<1:26:50,  2.21s/it]\u001b[0m\n",
      "\u001b[34m1%|▏         | 35/2388 [01:18<1:26:47,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 36/2388 [01:20<1:26:44,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 37/2388 [01:23<1:26:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 38/2388 [01:25<1:26:38,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 39/2388 [01:27<1:26:35,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 40/2388 [01:29<1:26:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 8.6527, 'learning_rate': 0.0001966499162479062, 'epoch': 0.05}\u001b[0m\n",
      "\u001b[34m2%|▏         | 40/2388 [01:29<1:26:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 41/2388 [01:31<1:26:30,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 42/2388 [01:34<1:26:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 43/2388 [01:36<1:26:24,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 44/2388 [01:38<1:26:21,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 45/2388 [01:40<1:26:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 46/2388 [01:43<1:26:16,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 47/2388 [01:45<1:26:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 48/2388 [01:47<1:26:11,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 49/2388 [01:49<1:26:08,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 50/2388 [01:51<1:26:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 9.1475, 'learning_rate': 0.00019581239530988274, 'epoch': 0.06}\u001b[0m\n",
      "\u001b[34m2%|▏         | 50/2388 [01:51<1:26:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 51/2388 [01:54<1:26:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 52/2388 [01:56<1:26:03,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 53/2388 [01:58<1:26:00,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 54/2388 [02:00<1:25:58,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 55/2388 [02:02<1:25:56,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 56/2388 [02:05<1:25:54,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 57/2388 [02:07<1:25:51,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 58/2388 [02:09<1:25:49,  2.21s/it]\u001b[0m\n",
      "\u001b[34m2%|▏         | 59/2388 [02:11<1:25:47,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 60/2388 [02:13<1:25:44,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.8626, 'learning_rate': 0.0001949748743718593, 'epoch': 0.08}\u001b[0m\n",
      "\u001b[34m3%|▎         | 60/2388 [02:14<1:25:44,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 61/2388 [02:16<1:25:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 62/2388 [02:18<1:25:40,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 63/2388 [02:20<1:25:39,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 64/2388 [02:22<1:25:36,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 65/2388 [02:25<1:25:34,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 66/2388 [02:27<1:25:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 67/2388 [02:29<1:25:30,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 68/2388 [02:31<1:25:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 69/2388 [02:33<1:25:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 70/2388 [02:36<1:25:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.6718, 'learning_rate': 0.00019413735343383584, 'epoch': 0.09}\u001b[0m\n",
      "\u001b[34m3%|▎         | 70/2388 [02:36<1:25:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 71/2388 [02:38<1:25:22,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 72/2388 [02:40<1:25:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 73/2388 [02:42<1:25:17,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 74/2388 [02:44<1:25:16,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 75/2388 [02:47<1:25:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 76/2388 [02:49<1:25:11,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 77/2388 [02:51<1:25:09,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 78/2388 [02:53<1:25:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 79/2388 [02:55<1:25:04,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 80/2388 [02:58<1:25:02,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.5304, 'learning_rate': 0.0001932998324958124, 'epoch': 0.1}\u001b[0m\n",
      "\u001b[34m3%|▎         | 80/2388 [02:58<1:25:02,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 81/2388 [03:00<1:25:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 82/2388 [03:02<1:24:59,  2.21s/it]\u001b[0m\n",
      "\u001b[34m3%|▎         | 83/2388 [03:04<1:24:57,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▎         | 84/2388 [03:07<1:24:54,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▎         | 85/2388 [03:09<1:24:52,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▎         | 86/2388 [03:11<1:24:49,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▎         | 87/2388 [03:13<1:24:47,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▎         | 88/2388 [03:15<1:24:46,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▎         | 89/2388 [03:18<1:24:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 90/2388 [03:20<1:24:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.3042, 'learning_rate': 0.00019246231155778894, 'epoch': 0.11}\u001b[0m\n",
      "\u001b[34m4%|▍         | 90/2388 [03:20<1:24:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 91/2388 [03:22<1:24:40,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 92/2388 [03:24<1:24:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 93/2388 [03:26<1:24:35,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 94/2388 [03:29<1:24:33,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 95/2388 [03:31<1:24:30,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 96/2388 [03:33<1:24:28,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 97/2388 [03:35<1:24:26,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 98/2388 [03:38<1:24:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 99/2388 [03:40<1:24:21,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 100/2388 [03:42<1:24:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2977, 'learning_rate': 0.0001916247906197655, 'epoch': 0.13}\u001b[0m\n",
      "\u001b[34m4%|▍         | 100/2388 [03:42<1:24:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 101/2388 [03:44<1:24:18,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 102/2388 [03:46<1:24:16,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 103/2388 [03:49<1:24:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 104/2388 [03:51<1:24:10,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 105/2388 [03:53<1:24:08,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 106/2388 [03:55<1:24:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m4%|▍         | 107/2388 [03:57<1:24:03,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 108/2388 [04:00<1:24:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 109/2388 [04:02<1:23:59,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 110/2388 [04:04<1:23:56,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.1958, 'learning_rate': 0.00019078726968174204, 'epoch': 0.14}\u001b[0m\n",
      "\u001b[34m5%|▍         | 110/2388 [04:04<1:23:56,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 111/2388 [04:06<1:23:55,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 112/2388 [04:08<1:23:53,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 113/2388 [04:11<1:23:50,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 114/2388 [04:13<1:23:48,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 115/2388 [04:15<1:23:46,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 116/2388 [04:17<1:23:44,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 117/2388 [04:20<1:23:42,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 118/2388 [04:22<1:23:40,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▍         | 119/2388 [04:24<1:23:38,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 120/2388 [04:26<1:23:36,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.3386, 'learning_rate': 0.0001899497487437186, 'epoch': 0.15}\u001b[0m\n",
      "\u001b[34m5%|▌         | 120/2388 [04:26<1:23:36,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 121/2388 [04:28<1:23:34,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 122/2388 [04:31<1:23:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 123/2388 [04:33<1:23:29,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 124/2388 [04:35<1:23:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 125/2388 [04:37<1:23:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 126/2388 [04:39<1:23:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 127/2388 [04:42<1:23:20,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 128/2388 [04:44<1:23:18,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 129/2388 [04:46<1:23:17,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 130/2388 [04:48<1:23:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2853, 'learning_rate': 0.00018911222780569514, 'epoch': 0.16}\u001b[0m\n",
      "\u001b[34m5%|▌         | 130/2388 [04:48<1:23:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m5%|▌         | 131/2388 [04:50<1:23:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 132/2388 [04:53<1:23:10,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 133/2388 [04:55<1:23:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 134/2388 [04:57<1:23:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 135/2388 [04:59<1:23:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 136/2388 [05:02<1:23:03,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 137/2388 [05:04<1:23:00,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 138/2388 [05:06<1:22:56,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 139/2388 [05:08<1:22:53,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 140/2388 [05:10<1:22:51,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2568, 'learning_rate': 0.0001882747068676717, 'epoch': 0.18}\u001b[0m\n",
      "\u001b[34m6%|▌         | 140/2388 [05:10<1:22:51,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 141/2388 [05:13<1:22:49,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 142/2388 [05:15<1:22:46,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 143/2388 [05:17<1:22:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 144/2388 [05:19<1:22:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 145/2388 [05:21<1:22:39,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 146/2388 [05:24<1:22:36,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 147/2388 [05:26<1:22:34,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 148/2388 [05:28<1:22:31,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▌         | 149/2388 [05:30<1:22:29,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▋         | 150/2388 [05:33<1:22:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2178, 'learning_rate': 0.00018743718592964824, 'epoch': 0.19}\u001b[0m\n",
      "\u001b[34m6%|▋         | 150/2388 [05:33<1:22:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▋         | 151/2388 [05:35<1:22:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▋         | 152/2388 [05:37<1:22:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▋         | 153/2388 [05:39<1:22:21,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▋         | 154/2388 [05:41<1:22:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m6%|▋         | 155/2388 [05:44<1:22:18,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 156/2388 [05:46<1:22:15,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 157/2388 [05:48<1:22:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 158/2388 [05:50<1:22:12,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 159/2388 [05:52<1:22:10,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 160/2388 [05:55<1:22:08,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.1973, 'learning_rate': 0.0001865996649916248, 'epoch': 0.2}\u001b[0m\n",
      "\u001b[34m7%|▋         | 160/2388 [05:55<1:22:08,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 161/2388 [05:57<1:22:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 162/2388 [05:59<1:22:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 163/2388 [06:01<1:22:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 164/2388 [06:03<1:22:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 165/2388 [06:06<1:21:57,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 166/2388 [06:08<1:21:54,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 167/2388 [06:10<1:21:51,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 168/2388 [06:12<1:21:48,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 169/2388 [06:15<1:21:46,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 170/2388 [06:17<1:21:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.285, 'learning_rate': 0.00018576214405360134, 'epoch': 0.21}\u001b[0m\n",
      "\u001b[34m7%|▋         | 170/2388 [06:17<1:21:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 171/2388 [06:19<1:21:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 172/2388 [06:21<1:21:38,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 173/2388 [06:23<1:21:36,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 174/2388 [06:26<1:21:33,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 175/2388 [06:28<1:21:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 176/2388 [06:30<1:21:30,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 177/2388 [06:32<1:21:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 178/2388 [06:34<1:21:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m7%|▋         | 179/2388 [06:37<1:21:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 180/2388 [06:39<1:21:22,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2457, 'learning_rate': 0.0001849246231155779, 'epoch': 0.23}\u001b[0m\n",
      "\u001b[34m8%|▊         | 180/2388 [06:39<1:21:22,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 181/2388 [06:41<1:21:20,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 182/2388 [06:43<1:21:18,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 183/2388 [06:45<1:21:15,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 184/2388 [06:48<1:21:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 185/2388 [06:50<1:21:12,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 186/2388 [06:52<1:21:10,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 187/2388 [06:54<1:21:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 188/2388 [06:57<1:21:04,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 189/2388 [06:59<1:21:02,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 190/2388 [07:01<1:21:00,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2775, 'learning_rate': 0.00018408710217755444, 'epoch': 0.24}\u001b[0m\n",
      "\u001b[34m8%|▊         | 190/2388 [07:01<1:21:00,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 191/2388 [07:03<1:20:59,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 192/2388 [07:05<1:20:56,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 193/2388 [07:08<1:20:53,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 194/2388 [07:10<1:20:51,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 195/2388 [07:12<1:20:49,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 196/2388 [07:14<1:20:47,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 197/2388 [07:16<1:20:44,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 198/2388 [07:19<1:20:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 199/2388 [07:21<1:20:39,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 200/2388 [07:23<1:20:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2982, 'learning_rate': 0.000183249581239531, 'epoch': 0.25}\u001b[0m\n",
      "\u001b[34m8%|▊         | 200/2388 [07:23<1:20:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 201/2388 [07:25<1:20:35,  2.21s/it]\u001b[0m\n",
      "\u001b[34m8%|▊         | 202/2388 [07:27<1:20:33,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▊         | 203/2388 [07:30<1:20:31,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▊         | 204/2388 [07:32<1:20:28,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▊         | 205/2388 [07:34<1:20:26,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▊         | 206/2388 [07:36<1:20:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▊         | 207/2388 [07:39<1:20:21,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▊         | 208/2388 [07:41<1:20:18,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 209/2388 [07:43<1:20:16,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 210/2388 [07:45<1:20:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2523, 'learning_rate': 0.00018241206030150754, 'epoch': 0.26}\u001b[0m\n",
      "\u001b[34m9%|▉         | 210/2388 [07:45<1:20:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 211/2388 [07:47<1:20:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 212/2388 [07:50<1:20:10,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 213/2388 [07:52<1:20:08,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 214/2388 [07:54<1:20:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 215/2388 [07:56<1:20:03,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 216/2388 [07:58<1:20:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 217/2388 [08:01<1:19:59,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 218/2388 [08:03<1:19:57,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 219/2388 [08:05<1:19:54,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 220/2388 [08:07<1:19:52,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2799, 'learning_rate': 0.0001815745393634841, 'epoch': 0.28}\u001b[0m\n",
      "\u001b[34m9%|▉         | 220/2388 [08:07<1:19:52,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 221/2388 [08:09<1:19:50,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 222/2388 [08:12<1:19:48,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 223/2388 [08:14<1:19:45,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 224/2388 [08:16<1:19:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 225/2388 [08:18<1:19:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m9%|▉         | 226/2388 [08:21<1:19:39,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 227/2388 [08:23<1:19:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 228/2388 [08:25<1:19:34,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 229/2388 [08:27<1:19:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 230/2388 [08:29<1:19:29,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2341, 'learning_rate': 0.00018073701842546063, 'epoch': 0.29}\u001b[0m\n",
      "\u001b[34m10%|▉         | 230/2388 [08:29<1:19:29,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 231/2388 [08:32<1:19:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 232/2388 [08:34<1:19:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 233/2388 [08:36<1:19:22,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 234/2388 [08:38<1:19:20,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 235/2388 [08:40<1:19:17,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 236/2388 [08:43<1:19:15,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 237/2388 [08:45<1:19:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|▉         | 238/2388 [08:47<1:19:11,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 239/2388 [08:49<1:19:09,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 240/2388 [08:51<1:19:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2133, 'learning_rate': 0.0001798994974874372, 'epoch': 0.3}\u001b[0m\n",
      "\u001b[34m10%|█         | 240/2388 [08:52<1:19:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 241/2388 [08:54<1:19:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 242/2388 [08:56<1:19:03,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 243/2388 [08:58<1:19:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 244/2388 [09:00<1:18:59,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 245/2388 [09:03<1:18:57,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 246/2388 [09:05<1:18:55,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 247/2388 [09:07<1:18:52,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 248/2388 [09:09<1:18:50,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 249/2388 [09:11<1:18:47,  2.21s/it]\u001b[0m\n",
      "\u001b[34m10%|█         | 250/2388 [09:14<1:18:45,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2531, 'learning_rate': 0.00017906197654941373, 'epoch': 0.31}\u001b[0m\n",
      "\u001b[34m10%|█         | 250/2388 [09:14<1:18:45,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 251/2388 [09:16<1:18:44,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 252/2388 [09:18<1:18:42,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 253/2388 [09:20<1:18:40,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 254/2388 [09:22<1:18:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 255/2388 [09:25<1:18:35,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 256/2388 [09:27<1:18:33,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 257/2388 [09:29<1:18:31,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 258/2388 [09:31<1:18:29,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 259/2388 [09:34<1:18:28,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 260/2388 [09:36<1:18:26,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2298, 'learning_rate': 0.0001782244556113903, 'epoch': 0.33}\u001b[0m\n",
      "\u001b[34m11%|█         | 260/2388 [09:36<1:18:26,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 261/2388 [09:38<1:18:24,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 262/2388 [09:40<1:18:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 263/2388 [09:42<1:18:21,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 264/2388 [09:45<1:18:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 265/2388 [09:47<1:18:16,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 266/2388 [09:49<1:18:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 267/2388 [09:51<1:18:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█         | 268/2388 [09:53<1:18:11,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█▏        | 269/2388 [09:56<1:18:09,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█▏        | 270/2388 [09:58<1:18:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2248, 'learning_rate': 0.00017738693467336683, 'epoch': 0.34}\u001b[0m\n",
      "\u001b[34m11%|█▏        | 270/2388 [09:58<1:18:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█▏        | 271/2388 [10:00<1:18:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█▏        | 272/2388 [10:02<1:18:03,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█▏        | 273/2388 [10:04<1:18:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m11%|█▏        | 274/2388 [10:07<1:17:58,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 275/2388 [10:09<1:17:56,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 276/2388 [10:11<1:17:54,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 277/2388 [10:13<1:17:51,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 278/2388 [10:16<1:17:49,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 279/2388 [10:18<1:17:46,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 280/2388 [10:20<1:17:44,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.1855, 'learning_rate': 0.0001765494137353434, 'epoch': 0.35}\u001b[0m\n",
      "\u001b[34m12%|█▏        | 280/2388 [10:20<1:17:44,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 281/2388 [10:22<1:17:42,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 282/2388 [10:24<1:17:40,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 283/2388 [10:27<1:17:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 284/2388 [10:29<1:17:35,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 285/2388 [10:31<1:17:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 286/2388 [10:33<1:17:30,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 287/2388 [10:35<1:17:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 288/2388 [10:38<1:17:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 289/2388 [10:40<1:17:22,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 290/2388 [10:42<1:17:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2446, 'learning_rate': 0.00017571189279731993, 'epoch': 0.36}\u001b[0m\n",
      "\u001b[34m12%|█▏        | 290/2388 [10:42<1:17:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 291/2388 [10:44<1:17:17,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 292/2388 [10:47<1:17:15,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 293/2388 [10:49<1:17:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 294/2388 [10:51<1:17:10,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 295/2388 [10:53<1:17:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 296/2388 [10:55<1:17:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 297/2388 [10:58<1:17:02,  2.21s/it]\u001b[0m\n",
      "\u001b[34m12%|█▏        | 298/2388 [11:00<1:17:00,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 299/2388 [11:02<1:16:58,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 300/2388 [11:04<1:16:55,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2057, 'learning_rate': 0.0001748743718592965, 'epoch': 0.38}\u001b[0m\n",
      "\u001b[34m13%|█▎        | 300/2388 [11:04<1:16:55,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 301/2388 [11:06<1:16:53,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 302/2388 [11:09<1:16:51,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 303/2388 [11:11<1:16:48,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 304/2388 [11:13<1:16:46,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 305/2388 [11:15<1:16:44,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 306/2388 [11:17<1:16:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 307/2388 [11:20<1:16:39,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 308/2388 [11:22<1:16:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 309/2388 [11:24<1:16:35,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 310/2388 [11:26<1:16:33,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2178, 'learning_rate': 0.00017403685092127303, 'epoch': 0.39}\u001b[0m\n",
      "\u001b[34m13%|█▎        | 310/2388 [11:26<1:16:33,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 311/2388 [11:29<1:16:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 312/2388 [11:31<1:16:30,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 313/2388 [11:33<1:16:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 314/2388 [11:35<1:16:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 315/2388 [11:37<1:16:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 316/2388 [11:40<1:16:21,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 317/2388 [11:42<1:16:18,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 318/2388 [11:44<1:16:16,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 319/2388 [11:46<1:16:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 320/2388 [11:48<1:16:12,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2351, 'learning_rate': 0.0001731993299832496, 'epoch': 0.4}\u001b[0m\n",
      "\u001b[34m13%|█▎        | 320/2388 [11:48<1:16:12,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 321/2388 [11:51<1:16:10,  2.21s/it]\u001b[0m\n",
      "\u001b[34m13%|█▎        | 322/2388 [11:53<1:16:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▎        | 323/2388 [11:55<1:16:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▎        | 324/2388 [11:57<1:16:02,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▎        | 325/2388 [11:59<1:16:00,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▎        | 326/2388 [12:02<1:15:58,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▎        | 327/2388 [12:04<1:15:56,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▎        | 328/2388 [12:06<1:15:54,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 329/2388 [12:08<1:15:52,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 330/2388 [12:11<1:15:50,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2631, 'learning_rate': 0.00017236180904522613, 'epoch': 0.41}\u001b[0m\n",
      "\u001b[34m14%|█▍        | 330/2388 [12:11<1:15:50,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 331/2388 [12:13<1:15:48,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 332/2388 [12:15<1:15:46,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 333/2388 [12:17<1:15:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 334/2388 [12:19<1:15:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 335/2388 [12:22<1:15:38,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 336/2388 [12:24<1:15:36,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 337/2388 [12:26<1:15:34,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 338/2388 [12:28<1:15:31,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 339/2388 [12:30<1:15:29,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 340/2388 [12:33<1:15:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2186, 'learning_rate': 0.0001715242881072027, 'epoch': 0.43}\u001b[0m\n",
      "\u001b[34m14%|█▍        | 340/2388 [12:33<1:15:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 341/2388 [12:35<1:15:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 342/2388 [12:37<1:15:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 343/2388 [12:39<1:15:21,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 344/2388 [12:41<1:15:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 345/2388 [12:44<1:15:17,  2.21s/it]\u001b[0m\n",
      "\u001b[34m14%|█▍        | 346/2388 [12:46<1:15:15,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 347/2388 [12:48<1:15:12,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 348/2388 [12:50<1:15:10,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 349/2388 [12:53<1:15:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 350/2388 [12:55<1:15:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2811, 'learning_rate': 0.00017068676716917923, 'epoch': 0.44}\u001b[0m\n",
      "\u001b[34m15%|█▍        | 350/2388 [12:55<1:15:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 351/2388 [12:57<1:15:03,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 352/2388 [12:59<1:15:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 353/2388 [13:01<1:14:59,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 354/2388 [13:04<1:14:56,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 355/2388 [13:06<1:14:54,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 356/2388 [13:08<1:14:52,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 357/2388 [13:10<1:14:49,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▍        | 358/2388 [13:12<1:14:47,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 359/2388 [13:15<1:14:45,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 360/2388 [13:17<1:14:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2735, 'learning_rate': 0.0001698492462311558, 'epoch': 0.45}\u001b[0m\n",
      "\u001b[34m15%|█▌        | 360/2388 [13:17<1:14:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 361/2388 [13:19<1:14:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 362/2388 [13:21<1:14:38,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 363/2388 [13:23<1:14:36,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 364/2388 [13:26<1:14:34,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 365/2388 [13:28<1:14:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 366/2388 [13:30<1:14:30,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 367/2388 [13:32<1:14:29,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 368/2388 [13:35<1:14:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 369/2388 [13:37<1:14:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m15%|█▌        | 370/2388 [13:39<1:14:24,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2897, 'learning_rate': 0.00016901172529313233, 'epoch': 0.46}\u001b[0m\n",
      "\u001b[34m15%|█▌        | 370/2388 [13:39<1:14:24,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 371/2388 [13:41<1:14:22,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 372/2388 [13:43<1:14:20,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 373/2388 [13:46<1:14:17,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 374/2388 [13:48<1:14:15,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 375/2388 [13:50<1:14:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 376/2388 [13:52<1:14:11,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 377/2388 [13:54<1:14:08,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 378/2388 [13:57<1:14:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 379/2388 [13:59<1:14:03,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 380/2388 [14:01<1:14:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2432, 'learning_rate': 0.0001681742043551089, 'epoch': 0.48}\u001b[0m\n",
      "\u001b[34m16%|█▌        | 380/2388 [14:01<1:14:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 381/2388 [14:03<1:13:59,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 382/2388 [14:06<1:13:57,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 383/2388 [14:08<1:13:55,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 384/2388 [14:10<1:13:52,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 385/2388 [14:12<1:13:50,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 386/2388 [14:14<1:13:48,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 387/2388 [14:17<1:13:45,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▌        | 388/2388 [14:19<1:13:42,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▋        | 389/2388 [14:21<1:13:40,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▋        | 390/2388 [14:23<1:13:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2745, 'learning_rate': 0.00016733668341708543, 'epoch': 0.49}\u001b[0m\n",
      "\u001b[34m16%|█▋        | 390/2388 [14:23<1:13:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▋        | 391/2388 [14:25<1:13:35,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▋        | 392/2388 [14:28<1:13:33,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▋        | 393/2388 [14:30<1:13:31,  2.21s/it]\u001b[0m\n",
      "\u001b[34m16%|█▋        | 394/2388 [14:32<1:13:28,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 395/2388 [14:34<1:13:26,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 396/2388 [14:36<1:13:24,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 397/2388 [14:39<1:13:22,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 398/2388 [14:41<1:13:20,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 399/2388 [14:43<1:13:17,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 400/2388 [14:45<1:13:15,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2111, 'learning_rate': 0.000166499162479062, 'epoch': 0.5}\u001b[0m\n",
      "\u001b[34m17%|█▋        | 400/2388 [14:45<1:13:15,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 401/2388 [14:48<1:13:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 402/2388 [14:50<1:13:11,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 403/2388 [14:52<1:13:09,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 404/2388 [14:54<1:13:07,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 405/2388 [14:56<1:13:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 406/2388 [14:59<1:13:03,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 407/2388 [15:01<1:13:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 408/2388 [15:03<1:12:58,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 409/2388 [15:05<1:12:56,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 410/2388 [15:07<1:12:54,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.266, 'learning_rate': 0.00016566164154103853, 'epoch': 0.52}\u001b[0m\n",
      "\u001b[34m17%|█▋        | 410/2388 [15:07<1:12:54,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 411/2388 [15:10<1:12:52,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 412/2388 [15:12<1:12:50,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 413/2388 [15:14<1:12:47,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 414/2388 [15:16<1:12:45,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 415/2388 [15:18<1:12:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 416/2388 [15:21<1:12:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m17%|█▋        | 417/2388 [15:23<1:12:38,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 418/2388 [15:25<1:12:36,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 419/2388 [15:27<1:12:34,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 420/2388 [15:30<1:12:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2261, 'learning_rate': 0.0001648241206030151, 'epoch': 0.53}\u001b[0m\n",
      "\u001b[34m18%|█▊        | 420/2388 [15:30<1:12:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 421/2388 [15:32<1:12:30,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 422/2388 [15:34<1:12:28,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 423/2388 [15:36<1:12:26,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 424/2388 [15:38<1:12:24,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 425/2388 [15:41<1:12:22,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 426/2388 [15:43<1:12:20,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 427/2388 [15:45<1:12:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 428/2388 [15:47<1:12:16,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 429/2388 [15:49<1:12:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 430/2388 [15:52<1:12:11,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.26, 'learning_rate': 0.00016398659966499162, 'epoch': 0.54}\u001b[0m\n",
      "\u001b[34m18%|█▊        | 430/2388 [15:52<1:12:11,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 431/2388 [15:54<1:12:09,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 432/2388 [15:56<1:12:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 433/2388 [15:58<1:12:04,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 434/2388 [16:01<1:12:02,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 435/2388 [16:03<1:12:00,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 436/2388 [16:05<1:11:57,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 437/2388 [16:07<1:11:55,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 438/2388 [16:09<1:11:53,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 439/2388 [16:12<1:11:50,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 440/2388 [16:14<1:11:48,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.3325, 'learning_rate': 0.0001631490787269682, 'epoch': 0.55}\u001b[0m\n",
      "\u001b[34m18%|█▊        | 440/2388 [16:14<1:11:48,  2.21s/it]\u001b[0m\n",
      "\u001b[34m18%|█▊        | 441/2388 [16:16<1:11:46,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▊        | 442/2388 [16:18<1:11:44,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▊        | 443/2388 [16:20<1:11:42,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▊        | 444/2388 [16:23<1:11:39,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▊        | 445/2388 [16:25<1:11:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▊        | 446/2388 [16:27<1:11:35,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▊        | 447/2388 [16:29<1:11:34,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 448/2388 [16:31<1:11:32,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 449/2388 [16:34<1:11:30,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 450/2388 [16:36<1:11:29,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.4419, 'learning_rate': 0.00016231155778894472, 'epoch': 0.57}\u001b[0m\n",
      "\u001b[34m19%|█▉        | 450/2388 [16:36<1:11:29,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 451/2388 [16:38<1:11:28,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 452/2388 [16:40<1:11:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 453/2388 [16:43<1:11:21,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 454/2388 [16:45<1:11:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 455/2388 [16:47<1:11:16,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 456/2388 [16:49<1:11:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 457/2388 [16:51<1:11:12,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 458/2388 [16:54<1:11:09,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 459/2388 [16:56<1:11:08,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 460/2388 [16:58<1:11:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.6386, 'learning_rate': 0.0001614740368509213, 'epoch': 0.58}\u001b[0m\n",
      "\u001b[34m19%|█▉        | 460/2388 [16:58<1:11:06,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 461/2388 [17:00<1:11:04,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 462/2388 [17:02<1:11:02,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 463/2388 [17:05<1:11:01,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 464/2388 [17:07<1:10:59,  2.21s/it]\u001b[0m\n",
      "\u001b[34m19%|█▉        | 465/2388 [17:09<1:10:56,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 466/2388 [17:11<1:10:53,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 467/2388 [17:14<1:10:50,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 468/2388 [17:16<1:10:48,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 469/2388 [17:18<1:10:46,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 470/2388 [17:20<1:10:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.6554, 'learning_rate': 0.00016063651591289782, 'epoch': 0.59}\u001b[0m\n",
      "\u001b[34m20%|█▉        | 470/2388 [17:20<1:10:43,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 471/2388 [17:22<1:10:41,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 472/2388 [17:25<1:10:39,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 473/2388 [17:27<1:10:37,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 474/2388 [17:29<1:10:35,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 475/2388 [17:31<1:10:33,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 476/2388 [17:33<1:10:31,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|█▉        | 477/2388 [17:36<1:10:29,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 478/2388 [17:38<1:10:27,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 479/2388 [17:40<1:10:25,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 480/2388 [17:42<1:10:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.3932, 'learning_rate': 0.00015979899497487439, 'epoch': 0.6}\u001b[0m\n",
      "\u001b[34m20%|██        | 480/2388 [17:42<1:10:23,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 481/2388 [17:45<1:10:22,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 482/2388 [17:47<1:10:20,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 483/2388 [17:49<1:10:18,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 484/2388 [17:51<1:10:16,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 485/2388 [17:53<1:10:14,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 486/2388 [17:56<1:10:12,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 487/2388 [17:58<1:10:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 488/2388 [18:00<1:10:08,  2.21s/it]\u001b[0m\n",
      "\u001b[34m20%|██        | 489/2388 [18:02<1:10:05,  2.21s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 490/2388 [18:04<1:10:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.265, 'learning_rate': 0.00015896147403685092, 'epoch': 0.62}\u001b[0m\n",
      "\u001b[34m21%|██        | 490/2388 [18:04<1:10:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 491/2388 [18:07<1:10:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 492/2388 [18:09<1:10:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 493/2388 [18:11<1:09:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 494/2388 [18:13<1:09:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 495/2388 [18:16<1:09:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 496/2388 [18:18<1:09:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 497/2388 [18:20<1:09:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 498/2388 [18:22<1:09:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 499/2388 [18:24<1:09:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 500/2388 [18:27<1:09:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.2917, 'learning_rate': 0.00015812395309882749, 'epoch': 0.63}\u001b[0m\n",
      "\u001b[34m21%|██        | 500/2388 [18:27<1:09:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 501/2388 [18:29<1:09:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 502/2388 [18:31<1:09:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 503/2388 [18:33<1:09:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 504/2388 [18:35<1:09:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 505/2388 [18:38<1:09:30,  2.21s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 506/2388 [18:40<1:09:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██        | 507/2388 [18:42<1:09:26,  2.21s/it]\u001b[0m\n",
      "\u001b[34m21%|██▏       | 508/2388 [18:44<1:09:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██▏       | 509/2388 [18:47<1:09:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██▏       | 510/2388 [18:49<1:09:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.1965, 'learning_rate': 0.00015728643216080402, 'epoch': 0.64}\u001b[0m\n",
      "\u001b[34m21%|██▏       | 510/2388 [18:49<1:09:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██▏       | 511/2388 [18:51<1:09:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██▏       | 512/2388 [18:53<1:09:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m21%|██▏       | 513/2388 [18:55<1:09:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 514/2388 [18:58<1:09:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 515/2388 [19:00<1:09:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 516/2388 [19:02<1:09:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 517/2388 [19:04<1:09:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 518/2388 [19:06<1:09:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 519/2388 [19:09<1:09:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 520/2388 [19:11<1:09:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.1614, 'learning_rate': 0.00015644891122278058, 'epoch': 0.65}\u001b[0m\n",
      "\u001b[34m22%|██▏       | 520/2388 [19:11<1:09:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 521/2388 [19:13<1:08:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 522/2388 [19:15<1:08:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 523/2388 [19:18<1:08:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 524/2388 [19:20<1:08:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 525/2388 [19:22<1:08:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 526/2388 [19:24<1:08:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 527/2388 [19:26<1:08:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 528/2388 [19:29<1:08:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 529/2388 [19:31<1:08:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 530/2388 [19:33<1:08:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.9128, 'learning_rate': 0.00015561139028475712, 'epoch': 0.67}\u001b[0m\n",
      "\u001b[34m22%|██▏       | 530/2388 [19:33<1:08:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 531/2388 [19:35<1:08:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 532/2388 [19:38<1:08:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 533/2388 [19:40<1:08:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 534/2388 [19:42<1:08:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 535/2388 [19:44<1:08:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 536/2388 [19:46<1:08:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m22%|██▏       | 537/2388 [19:49<1:08:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 538/2388 [19:51<1:08:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 539/2388 [19:53<1:08:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 540/2388 [19:55<1:08:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 7.053, 'learning_rate': 0.00015477386934673368, 'epoch': 0.68}\u001b[0m\n",
      "\u001b[34m23%|██▎       | 540/2388 [19:55<1:08:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 541/2388 [19:57<1:08:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 542/2388 [20:00<1:08:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 543/2388 [20:02<1:08:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 544/2388 [20:04<1:08:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 545/2388 [20:06<1:08:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 546/2388 [20:09<1:08:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 547/2388 [20:11<1:08:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 548/2388 [20:13<1:08:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 549/2388 [20:15<1:07:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 550/2388 [20:17<1:07:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.9357, 'learning_rate': 0.00015393634840871022, 'epoch': 0.69}\u001b[0m\n",
      "\u001b[34m23%|██▎       | 550/2388 [20:17<1:07:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 551/2388 [20:20<1:07:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 552/2388 [20:22<1:07:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 553/2388 [20:24<1:07:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 554/2388 [20:26<1:07:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 555/2388 [20:29<1:07:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 556/2388 [20:31<1:07:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 557/2388 [20:33<1:07:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 558/2388 [20:35<1:07:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 559/2388 [20:37<1:07:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 560/2388 [20:40<1:07:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.7321, 'learning_rate': 0.00015309882747068678, 'epoch': 0.7}\u001b[0m\n",
      "\u001b[34m23%|██▎       | 560/2388 [20:40<1:07:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m23%|██▎       | 561/2388 [20:42<1:07:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▎       | 562/2388 [20:44<1:07:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▎       | 563/2388 [20:46<1:07:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▎       | 564/2388 [20:49<1:07:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▎       | 565/2388 [20:51<1:07:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▎       | 566/2388 [20:53<1:07:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▎       | 567/2388 [20:55<1:07:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 568/2388 [20:57<1:07:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 569/2388 [21:00<1:07:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 570/2388 [21:02<1:07:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.5731, 'learning_rate': 0.00015226130653266332, 'epoch': 0.72}\u001b[0m\n",
      "\u001b[34m24%|██▍       | 570/2388 [21:02<1:07:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 571/2388 [21:04<1:07:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 572/2388 [21:06<1:07:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 573/2388 [21:09<1:07:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 574/2388 [21:11<1:07:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 575/2388 [21:13<1:07:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 576/2388 [21:15<1:07:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 577/2388 [21:17<1:07:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 578/2388 [21:20<1:07:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 579/2388 [21:22<1:06:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 580/2388 [21:24<1:06:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.492, 'learning_rate': 0.00015142378559463988, 'epoch': 0.73}\u001b[0m\n",
      "\u001b[34m24%|██▍       | 580/2388 [21:24<1:06:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 581/2388 [21:26<1:06:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 582/2388 [21:29<1:06:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 583/2388 [21:31<1:06:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 584/2388 [21:33<1:06:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m24%|██▍       | 585/2388 [21:35<1:06:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 586/2388 [21:37<1:06:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 587/2388 [21:40<1:06:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 588/2388 [21:42<1:06:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 589/2388 [21:44<1:06:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 590/2388 [21:46<1:06:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.3774, 'learning_rate': 0.00015058626465661642, 'epoch': 0.74}\u001b[0m\n",
      "\u001b[34m25%|██▍       | 590/2388 [21:46<1:06:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 591/2388 [21:48<1:06:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 592/2388 [21:51<1:06:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 593/2388 [21:53<1:06:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 594/2388 [21:55<1:06:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 595/2388 [21:57<1:06:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▍       | 596/2388 [22:00<1:06:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 597/2388 [22:02<1:06:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 598/2388 [22:04<1:06:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 599/2388 [22:06<1:06:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 600/2388 [22:08<1:06:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.3635, 'learning_rate': 0.00014974874371859298, 'epoch': 0.75}\u001b[0m\n",
      "\u001b[34m25%|██▌       | 600/2388 [22:09<1:06:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 601/2388 [22:11<1:06:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 602/2388 [22:13<1:06:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 603/2388 [22:15<1:06:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 604/2388 [22:17<1:06:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 605/2388 [22:20<1:06:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 606/2388 [22:22<1:05:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 607/2388 [22:24<1:05:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m25%|██▌       | 608/2388 [22:26<1:05:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 609/2388 [22:28<1:05:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 610/2388 [22:31<1:05:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.056, 'learning_rate': 0.00014891122278056952, 'epoch': 0.77}\u001b[0m\n",
      "\u001b[34m26%|██▌       | 610/2388 [22:31<1:05:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 611/2388 [22:33<1:05:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 612/2388 [22:35<1:05:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 613/2388 [22:37<1:05:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 614/2388 [22:40<1:05:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 615/2388 [22:42<1:05:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 616/2388 [22:44<1:05:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 617/2388 [22:46<1:05:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 618/2388 [22:48<1:05:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 619/2388 [22:51<1:05:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 620/2388 [22:53<1:05:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.1386, 'learning_rate': 0.00014807370184254608, 'epoch': 0.78}\u001b[0m\n",
      "\u001b[34m26%|██▌       | 620/2388 [22:53<1:05:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 621/2388 [22:55<1:05:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 622/2388 [22:57<1:05:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 623/2388 [23:00<1:05:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 624/2388 [23:02<1:05:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 625/2388 [23:04<1:05:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▌       | 626/2388 [23:06<1:05:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▋       | 627/2388 [23:08<1:05:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▋       | 628/2388 [23:11<1:05:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▋       | 629/2388 [23:13<1:05:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▋       | 630/2388 [23:15<1:05:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.9728, 'learning_rate': 0.00014723618090452262, 'epoch': 0.79}\u001b[0m\n",
      "\u001b[34m26%|██▋       | 630/2388 [23:15<1:05:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▋       | 631/2388 [23:17<1:05:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m26%|██▋       | 632/2388 [23:20<1:05:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 633/2388 [23:22<1:04:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 634/2388 [23:24<1:04:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 635/2388 [23:26<1:04:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 636/2388 [23:28<1:04:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 637/2388 [23:31<1:04:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 638/2388 [23:33<1:04:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 639/2388 [23:35<1:04:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 640/2388 [23:37<1:04:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.0362, 'learning_rate': 0.00014639865996649918, 'epoch': 0.8}\u001b[0m\n",
      "\u001b[34m27%|██▋       | 640/2388 [23:37<1:04:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 641/2388 [23:40<1:04:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 642/2388 [23:42<1:04:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 643/2388 [23:44<1:04:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 644/2388 [23:46<1:04:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 645/2388 [23:48<1:04:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 646/2388 [23:51<1:04:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 647/2388 [23:53<1:04:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 648/2388 [23:55<1:04:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 649/2388 [23:57<1:04:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 650/2388 [24:00<1:04:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.1057, 'learning_rate': 0.00014556113902847571, 'epoch': 0.82}\u001b[0m\n",
      "\u001b[34m27%|██▋       | 650/2388 [24:00<1:04:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 651/2388 [24:02<1:04:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 652/2388 [24:04<1:04:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 653/2388 [24:06<1:04:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 654/2388 [24:08<1:04:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 655/2388 [24:11<1:04:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m27%|██▋       | 656/2388 [24:13<1:04:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 657/2388 [24:15<1:04:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 658/2388 [24:17<1:04:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 659/2388 [24:20<1:03:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 660/2388 [24:22<1:03:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 6.0703, 'learning_rate': 0.00014472361809045228, 'epoch': 0.83}\u001b[0m\n",
      "\u001b[34m28%|██▊       | 660/2388 [24:22<1:03:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 661/2388 [24:24<1:03:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 662/2388 [24:26<1:03:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 663/2388 [24:28<1:03:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 664/2388 [24:31<1:03:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 665/2388 [24:33<1:03:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 666/2388 [24:35<1:03:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 667/2388 [24:37<1:03:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 668/2388 [24:40<1:03:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 669/2388 [24:42<1:03:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 670/2388 [24:44<1:03:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.9209, 'learning_rate': 0.0001438860971524288, 'epoch': 0.84}\u001b[0m\n",
      "\u001b[34m28%|██▊       | 670/2388 [24:44<1:03:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 671/2388 [24:46<1:03:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 672/2388 [24:48<1:03:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 673/2388 [24:51<1:03:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 674/2388 [24:53<1:03:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 675/2388 [24:55<1:03:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 676/2388 [24:57<1:03:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 677/2388 [25:00<1:03:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 678/2388 [25:02<1:03:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 679/2388 [25:04<1:03:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m28%|██▊       | 680/2388 [25:06<1:03:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.8874, 'learning_rate': 0.00014304857621440538, 'epoch': 0.85}\u001b[0m\n",
      "\u001b[34m28%|██▊       | 680/2388 [25:06<1:03:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▊       | 681/2388 [25:08<1:03:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▊       | 682/2388 [25:11<1:03:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▊       | 683/2388 [25:13<1:03:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▊       | 684/2388 [25:15<1:03:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▊       | 685/2388 [25:17<1:03:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▊       | 686/2388 [25:20<1:02:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 687/2388 [25:22<1:02:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 688/2388 [25:24<1:02:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 689/2388 [25:26<1:02:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 690/2388 [25:28<1:02:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.9447, 'learning_rate': 0.0001422110552763819, 'epoch': 0.87}\u001b[0m\n",
      "\u001b[34m29%|██▉       | 690/2388 [25:28<1:02:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 691/2388 [25:31<1:02:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 692/2388 [25:33<1:02:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 693/2388 [25:35<1:02:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 694/2388 [25:37<1:02:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 695/2388 [25:40<1:02:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 696/2388 [25:42<1:02:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 697/2388 [25:44<1:02:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 698/2388 [25:46<1:02:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 699/2388 [25:48<1:02:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 700/2388 [25:51<1:02:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.9446, 'learning_rate': 0.00014137353433835848, 'epoch': 0.88}\u001b[0m\n",
      "\u001b[34m29%|██▉       | 700/2388 [25:51<1:02:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 701/2388 [25:53<1:02:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 702/2388 [25:55<1:02:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 703/2388 [25:57<1:02:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m29%|██▉       | 704/2388 [25:59<1:02:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 705/2388 [26:02<1:02:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 706/2388 [26:04<1:02:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 707/2388 [26:06<1:02:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 708/2388 [26:08<1:02:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 709/2388 [26:11<1:02:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 710/2388 [26:13<1:02:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.7327, 'learning_rate': 0.000140536013400335, 'epoch': 0.89}\u001b[0m\n",
      "\u001b[34m30%|██▉       | 710/2388 [26:13<1:02:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 711/2388 [26:15<1:02:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 712/2388 [26:17<1:02:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 713/2388 [26:19<1:02:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 714/2388 [26:22<1:01:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 715/2388 [26:24<1:01:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|██▉       | 716/2388 [26:26<1:01:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 717/2388 [26:28<1:01:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 718/2388 [26:31<1:01:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 719/2388 [26:33<1:01:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 720/2388 [26:35<1:01:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.9589, 'learning_rate': 0.00013969849246231157, 'epoch': 0.9}\u001b[0m\n",
      "\u001b[34m30%|███       | 720/2388 [26:35<1:01:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 721/2388 [26:37<1:01:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 722/2388 [26:39<1:01:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 723/2388 [26:42<1:01:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 724/2388 [26:44<1:01:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 725/2388 [26:46<1:01:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 726/2388 [26:48<1:01:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 727/2388 [26:51<1:01:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m30%|███       | 728/2388 [26:53<1:01:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 729/2388 [26:55<1:01:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 730/2388 [26:57<1:01:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.8589, 'learning_rate': 0.0001388609715242881, 'epoch': 0.92}\u001b[0m\n",
      "\u001b[34m31%|███       | 730/2388 [26:57<1:01:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 731/2388 [26:59<1:01:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 732/2388 [27:02<1:01:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 733/2388 [27:04<1:01:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 734/2388 [27:06<1:01:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 735/2388 [27:08<1:01:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 736/2388 [27:11<1:01:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 737/2388 [27:13<1:01:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 738/2388 [27:15<1:01:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 739/2388 [27:17<1:01:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 740/2388 [27:19<1:01:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.7046, 'learning_rate': 0.00013802345058626467, 'epoch': 0.93}\u001b[0m\n",
      "\u001b[34m31%|███       | 740/2388 [27:20<1:01:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 741/2388 [27:22<1:01:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 742/2388 [27:24<1:00:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 743/2388 [27:26<1:00:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 744/2388 [27:28<1:00:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 745/2388 [27:31<1:00:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███       | 746/2388 [27:33<1:00:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███▏      | 747/2388 [27:35<1:00:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███▏      | 748/2388 [27:37<1:00:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███▏      | 749/2388 [27:39<1:00:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███▏      | 750/2388 [27:42<1:00:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.6608, 'learning_rate': 0.0001371859296482412, 'epoch': 0.94}\u001b[0m\n",
      "\u001b[34m31%|███▏      | 750/2388 [27:42<1:00:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███▏      | 751/2388 [27:44<1:00:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m31%|███▏      | 752/2388 [27:46<1:00:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 753/2388 [27:48<1:00:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 754/2388 [27:51<1:00:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 755/2388 [27:53<1:00:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 756/2388 [27:55<1:00:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 757/2388 [27:57<1:00:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 758/2388 [27:59<1:00:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 759/2388 [28:02<1:00:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 760/2388 [28:04<1:00:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.8839, 'learning_rate': 0.00013634840871021777, 'epoch': 0.95}\u001b[0m\n",
      "\u001b[34m32%|███▏      | 760/2388 [28:04<1:00:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 761/2388 [28:06<1:00:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 762/2388 [28:08<1:00:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 763/2388 [28:11<1:00:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 764/2388 [28:13<1:00:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 765/2388 [28:15<1:00:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 766/2388 [28:17<1:00:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 767/2388 [28:19<1:00:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 768/2388 [28:22<1:00:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 769/2388 [28:24<59:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 770/2388 [28:26<59:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.7656, 'learning_rate': 0.0001355108877721943, 'epoch': 0.97}\u001b[0m\n",
      "\u001b[34m32%|███▏      | 770/2388 [28:26<59:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 771/2388 [28:28<59:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 772/2388 [28:31<59:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 773/2388 [28:33<59:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 774/2388 [28:35<59:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 775/2388 [28:37<59:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m32%|███▏      | 776/2388 [28:39<59:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 777/2388 [28:42<59:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 778/2388 [28:44<59:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 779/2388 [28:46<59:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 780/2388 [28:48<59:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.7394, 'learning_rate': 0.00013467336683417087, 'epoch': 0.98}\u001b[0m\n",
      "\u001b[34m33%|███▎      | 780/2388 [28:48<59:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 781/2388 [28:51<59:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 782/2388 [28:53<59:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 783/2388 [28:55<59:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 784/2388 [28:57<59:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 785/2388 [28:59<59:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 786/2388 [29:02<59:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 787/2388 [29:04<59:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 788/2388 [29:06<59:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 789/2388 [29:08<59:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 790/2388 [29:11<59:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.707, 'learning_rate': 0.0001338358458961474, 'epoch': 0.99}\u001b[0m\n",
      "\u001b[34m33%|███▎      | 790/2388 [29:11<59:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 791/2388 [29:13<59:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 792/2388 [29:15<59:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 793/2388 [29:17<59:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 794/2388 [29:19<59:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 795/2388 [29:22<58:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 796/2388 [29:23<50:57,  1.92s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 797/2388 [29:25<53:19,  2.01s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 798/2388 [29:27<54:58,  2.07s/it]\u001b[0m\n",
      "\u001b[34m33%|███▎      | 799/2388 [29:30<56:06,  2.12s/it]\u001b[0m\n",
      "\u001b[34m34%|███▎      | 800/2388 [29:32<56:53,  2.15s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.5197, 'learning_rate': 0.00013299832495812397, 'epoch': 1.01}\u001b[0m\n",
      "\u001b[34m34%|███▎      | 800/2388 [29:32<56:53,  2.15s/it]\u001b[0m\n",
      "\u001b[34m34%|███▎      | 801/2388 [29:34<57:26,  2.17s/it]\u001b[0m\n",
      "\u001b[34m34%|███▎      | 802/2388 [29:36<57:48,  2.19s/it]\u001b[0m\n",
      "\u001b[34m34%|███▎      | 803/2388 [29:38<58:03,  2.20s/it]\u001b[0m\n",
      "\u001b[34m34%|███▎      | 804/2388 [29:41<58:13,  2.21s/it]\u001b[0m\n",
      "\u001b[34m34%|███▎      | 805/2388 [29:43<58:19,  2.21s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 806/2388 [29:45<58:22,  2.21s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 807/2388 [29:47<58:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 808/2388 [29:50<58:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 809/2388 [29:52<58:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 810/2388 [29:54<58:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.5269, 'learning_rate': 0.0001321608040201005, 'epoch': 1.02}\u001b[0m\n",
      "\u001b[34m34%|███▍      | 810/2388 [29:54<58:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 811/2388 [29:56<58:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 812/2388 [29:58<58:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 813/2388 [30:01<58:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 814/2388 [30:03<58:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 815/2388 [30:05<58:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 816/2388 [30:07<58:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 817/2388 [30:10<58:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 818/2388 [30:12<58:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 819/2388 [30:14<58:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 820/2388 [30:16<58:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.5541, 'learning_rate': 0.00013132328308207707, 'epoch': 1.03}\u001b[0m\n",
      "\u001b[34m34%|███▍      | 820/2388 [30:16<58:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 821/2388 [30:19<58:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 822/2388 [30:21<58:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m34%|███▍      | 823/2388 [30:23<57:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 824/2388 [30:25<57:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 825/2388 [30:27<57:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 826/2388 [30:30<57:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 827/2388 [30:32<57:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 828/2388 [30:34<57:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 829/2388 [30:36<57:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 830/2388 [30:39<57:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4781, 'learning_rate': 0.0001304857621440536, 'epoch': 1.04}\u001b[0m\n",
      "\u001b[34m35%|███▍      | 830/2388 [30:39<57:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 831/2388 [30:41<57:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 832/2388 [30:43<57:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 833/2388 [30:45<57:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 834/2388 [30:47<57:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▍      | 835/2388 [30:50<57:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 836/2388 [30:52<57:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 837/2388 [30:54<57:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 838/2388 [30:56<57:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 839/2388 [30:59<57:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 840/2388 [31:01<57:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.5243, 'learning_rate': 0.00012964824120603017, 'epoch': 1.06}\u001b[0m\n",
      "\u001b[34m35%|███▌      | 840/2388 [31:01<57:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 841/2388 [31:03<57:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 842/2388 [31:05<57:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 843/2388 [31:07<57:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 844/2388 [31:10<57:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 845/2388 [31:12<57:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 846/2388 [31:14<57:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m35%|███▌      | 847/2388 [31:16<57:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 848/2388 [31:19<57:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 849/2388 [31:21<57:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 850/2388 [31:23<56:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4769, 'learning_rate': 0.0001288107202680067, 'epoch': 1.07}\u001b[0m\n",
      "\u001b[34m36%|███▌      | 850/2388 [31:23<56:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 851/2388 [31:25<56:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 852/2388 [31:27<56:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 853/2388 [31:30<56:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 854/2388 [31:32<56:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 855/2388 [31:34<56:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 856/2388 [31:36<56:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 857/2388 [31:39<56:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 858/2388 [31:41<56:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 859/2388 [31:43<56:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 860/2388 [31:45<56:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.5456, 'learning_rate': 0.00012797319932998327, 'epoch': 1.08}\u001b[0m\n",
      "\u001b[34m36%|███▌      | 860/2388 [31:45<56:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 861/2388 [31:47<56:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 862/2388 [31:50<56:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 863/2388 [31:52<56:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 864/2388 [31:54<56:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▌      | 865/2388 [31:56<56:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▋      | 866/2388 [31:59<56:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▋      | 867/2388 [32:01<56:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▋      | 868/2388 [32:03<56:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▋      | 869/2388 [32:05<56:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▋      | 870/2388 [32:07<56:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4867, 'learning_rate': 0.0001271356783919598, 'epoch': 1.09}\u001b[0m\n",
      "\u001b[34m36%|███▋      | 870/2388 [32:07<56:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m36%|███▋      | 871/2388 [32:10<56:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 872/2388 [32:12<56:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 873/2388 [32:14<56:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 874/2388 [32:16<56:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 875/2388 [32:19<56:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 876/2388 [32:21<56:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 877/2388 [32:23<55:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 878/2388 [32:25<55:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 879/2388 [32:27<55:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 880/2388 [32:30<55:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.3976, 'learning_rate': 0.00012629815745393637, 'epoch': 1.11}\u001b[0m\n",
      "\u001b[34m37%|███▋      | 880/2388 [32:30<55:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 881/2388 [32:32<55:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 882/2388 [32:34<55:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 883/2388 [32:36<55:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 884/2388 [32:39<55:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 885/2388 [32:41<55:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 886/2388 [32:43<55:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 887/2388 [32:45<55:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 888/2388 [32:47<55:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 889/2388 [32:50<55:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 890/2388 [32:52<55:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4305, 'learning_rate': 0.0001254606365159129, 'epoch': 1.12}\u001b[0m\n",
      "\u001b[34m37%|███▋      | 890/2388 [32:52<55:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 891/2388 [32:54<55:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 892/2388 [32:56<55:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 893/2388 [32:59<55:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 894/2388 [33:01<55:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m37%|███▋      | 895/2388 [33:03<55:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 896/2388 [33:05<55:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 897/2388 [33:07<55:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 898/2388 [33:10<55:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 899/2388 [33:12<55:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 900/2388 [33:14<55:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4454, 'learning_rate': 0.00012462311557788947, 'epoch': 1.13}\u001b[0m\n",
      "\u001b[34m38%|███▊      | 900/2388 [33:14<55:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 901/2388 [33:16<55:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 902/2388 [33:19<55:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 903/2388 [33:21<55:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 904/2388 [33:23<54:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 905/2388 [33:25<54:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 906/2388 [33:27<54:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 907/2388 [33:30<54:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 908/2388 [33:32<54:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 909/2388 [33:34<54:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 910/2388 [33:36<54:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4096, 'learning_rate': 0.000123785594639866, 'epoch': 1.14}\u001b[0m\n",
      "\u001b[34m38%|███▊      | 910/2388 [33:36<54:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 911/2388 [33:39<54:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 912/2388 [33:41<54:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 913/2388 [33:43<54:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 914/2388 [33:45<54:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 915/2388 [33:47<54:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 916/2388 [33:50<54:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 917/2388 [33:52<54:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 918/2388 [33:54<54:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m38%|███▊      | 919/2388 [33:56<54:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▊      | 920/2388 [33:59<54:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4684, 'learning_rate': 0.00012294807370184256, 'epoch': 1.16}\u001b[0m\n",
      "\u001b[34m39%|███▊      | 920/2388 [33:59<54:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▊      | 921/2388 [34:01<54:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▊      | 922/2388 [34:03<54:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▊      | 923/2388 [34:05<54:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▊      | 924/2388 [34:07<54:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▊      | 925/2388 [34:10<54:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 926/2388 [34:12<54:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 927/2388 [34:14<54:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 928/2388 [34:16<54:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 929/2388 [34:19<54:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 930/2388 [34:21<54:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4755, 'learning_rate': 0.0001221105527638191, 'epoch': 1.17}\u001b[0m\n",
      "\u001b[34m39%|███▉      | 930/2388 [34:21<54:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 931/2388 [34:23<53:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 932/2388 [34:25<53:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 933/2388 [34:27<53:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 934/2388 [34:30<53:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 935/2388 [34:32<53:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 936/2388 [34:34<53:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 937/2388 [34:36<53:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 938/2388 [34:39<53:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 939/2388 [34:41<53:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 940/2388 [34:43<53:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4307, 'learning_rate': 0.00012127303182579566, 'epoch': 1.18}\u001b[0m\n",
      "\u001b[34m39%|███▉      | 940/2388 [34:43<53:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 941/2388 [34:45<53:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 942/2388 [34:47<53:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m39%|███▉      | 943/2388 [34:50<53:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 944/2388 [34:52<53:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 945/2388 [34:54<53:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 946/2388 [34:56<53:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 947/2388 [34:59<53:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 948/2388 [35:01<53:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 949/2388 [35:03<53:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 950/2388 [35:05<53:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4402, 'learning_rate': 0.00012043551088777221, 'epoch': 1.19}\u001b[0m\n",
      "\u001b[34m40%|███▉      | 950/2388 [35:05<53:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 951/2388 [35:07<53:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 952/2388 [35:10<53:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 953/2388 [35:12<53:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 954/2388 [35:14<53:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|███▉      | 955/2388 [35:16<53:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 956/2388 [35:19<53:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 957/2388 [35:21<53:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 958/2388 [35:23<52:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 959/2388 [35:25<52:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 960/2388 [35:27<52:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.4528, 'learning_rate': 0.00011959798994974876, 'epoch': 1.21}\u001b[0m\n",
      "\u001b[34m40%|████      | 960/2388 [35:28<52:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 961/2388 [35:30<52:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 962/2388 [35:32<52:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 963/2388 [35:34<52:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 964/2388 [35:36<52:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 965/2388 [35:39<52:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 966/2388 [35:41<52:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m40%|████      | 967/2388 [35:43<52:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 968/2388 [35:45<52:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 969/2388 [35:47<52:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 970/2388 [35:50<52:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.3553, 'learning_rate': 0.00011876046901172531, 'epoch': 1.22}\u001b[0m\n",
      "\u001b[34m41%|████      | 970/2388 [35:50<52:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 971/2388 [35:52<52:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 972/2388 [35:54<52:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 973/2388 [35:56<52:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 974/2388 [35:59<52:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 975/2388 [36:01<52:25,  2.23s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 976/2388 [36:03<52:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 977/2388 [36:05<52:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 978/2388 [36:07<52:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 979/2388 [36:10<52:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 980/2388 [36:12<52:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.292, 'learning_rate': 0.00011792294807370186, 'epoch': 1.23}\u001b[0m\n",
      "\u001b[34m41%|████      | 980/2388 [36:12<52:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 981/2388 [36:14<52:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 982/2388 [36:16<52:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 983/2388 [36:19<52:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 984/2388 [36:21<52:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████      | 985/2388 [36:23<51:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████▏     | 986/2388 [36:25<51:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████▏     | 987/2388 [36:27<51:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████▏     | 988/2388 [36:30<51:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████▏     | 989/2388 [36:32<51:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████▏     | 990/2388 [36:34<51:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.3475, 'learning_rate': 0.00011708542713567841, 'epoch': 1.24}\u001b[0m\n",
      "\u001b[34m41%|████▏     | 990/2388 [36:34<51:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m41%|████▏     | 991/2388 [36:36<51:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 992/2388 [36:39<51:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 993/2388 [36:41<51:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 994/2388 [36:43<51:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 995/2388 [36:45<51:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 996/2388 [36:47<51:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 997/2388 [36:50<51:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 998/2388 [36:52<51:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 999/2388 [36:54<51:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1000/2388 [36:56<51:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.3495, 'learning_rate': 0.00011624790619765496, 'epoch': 1.26}\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1000/2388 [36:56<51:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1001/2388 [36:59<51:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1002/2388 [37:01<51:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1003/2388 [37:03<51:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1004/2388 [37:05<51:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1005/2388 [37:08<51:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1006/2388 [37:10<51:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1007/2388 [37:12<51:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1008/2388 [37:14<51:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1009/2388 [37:16<51:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1010/2388 [37:19<51:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.3206, 'learning_rate': 0.00011541038525963151, 'epoch': 1.27}\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1010/2388 [37:19<51:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1011/2388 [37:21<51:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1012/2388 [37:23<50:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1013/2388 [37:25<50:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m42%|████▏     | 1014/2388 [37:28<50:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1015/2388 [37:30<50:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1016/2388 [37:32<50:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1017/2388 [37:34<50:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1018/2388 [37:36<50:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1019/2388 [37:39<50:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1020/2388 [37:41<50:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.2557, 'learning_rate': 0.00011457286432160806, 'epoch': 1.28}\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1020/2388 [37:41<50:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1021/2388 [37:43<50:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1022/2388 [37:45<50:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1023/2388 [37:48<50:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1024/2388 [37:50<50:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1025/2388 [37:52<50:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1026/2388 [37:54<50:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1027/2388 [37:56<50:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1028/2388 [37:59<50:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1029/2388 [38:01<50:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1030/2388 [38:03<50:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.433, 'learning_rate': 0.00011373534338358461, 'epoch': 1.29}\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1030/2388 [38:03<50:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1031/2388 [38:05<50:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1032/2388 [38:08<50:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1033/2388 [38:10<50:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1034/2388 [38:12<50:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1035/2388 [38:14<50:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1036/2388 [38:16<50:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1037/2388 [38:19<50:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m43%|████▎     | 1038/2388 [38:21<50:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▎     | 1039/2388 [38:23<49:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▎     | 1040/2388 [38:25<49:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.2703, 'learning_rate': 0.00011289782244556116, 'epoch': 1.31}\u001b[0m\n",
      "\u001b[34m44%|████▎     | 1040/2388 [38:25<49:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▎     | 1041/2388 [38:28<49:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▎     | 1042/2388 [38:30<49:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▎     | 1043/2388 [38:32<49:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▎     | 1044/2388 [38:34<49:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1045/2388 [38:36<49:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1046/2388 [38:39<49:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1047/2388 [38:41<49:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1048/2388 [38:43<49:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1049/2388 [38:45<49:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1050/2388 [38:48<49:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.2601, 'learning_rate': 0.00011206030150753771, 'epoch': 1.32}\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1050/2388 [38:48<49:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1051/2388 [38:50<49:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1052/2388 [38:52<49:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1053/2388 [38:54<49:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1054/2388 [38:56<49:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1055/2388 [38:59<49:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1056/2388 [39:01<49:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1057/2388 [39:03<49:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1058/2388 [39:05<49:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1059/2388 [39:08<49:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1060/2388 [39:10<49:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.228, 'learning_rate': 0.00011122278056951426, 'epoch': 1.33}\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1060/2388 [39:10<49:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1061/2388 [39:12<49:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m44%|████▍     | 1062/2388 [39:14<49:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1063/2388 [39:16<49:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1064/2388 [39:19<49:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1065/2388 [39:21<49:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1066/2388 [39:23<49:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1067/2388 [39:25<48:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1068/2388 [39:28<48:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1069/2388 [39:30<48:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1070/2388 [39:32<48:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.1461, 'learning_rate': 0.00011038525963149081, 'epoch': 1.34}\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1070/2388 [39:32<48:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1071/2388 [39:34<48:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1072/2388 [39:36<48:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1073/2388 [39:39<48:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▍     | 1074/2388 [39:41<48:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1075/2388 [39:43<48:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1076/2388 [39:45<48:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1077/2388 [39:48<48:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1078/2388 [39:50<48:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1079/2388 [39:52<48:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1080/2388 [39:54<48:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.312, 'learning_rate': 0.00010954773869346736, 'epoch': 1.36}\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1080/2388 [39:54<48:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1081/2388 [39:56<48:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1082/2388 [39:59<48:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1083/2388 [40:01<48:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1084/2388 [40:03<48:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1085/2388 [40:05<48:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m45%|████▌     | 1086/2388 [40:08<48:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1087/2388 [40:10<48:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1088/2388 [40:12<48:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1089/2388 [40:14<48:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1090/2388 [40:16<48:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.1963, 'learning_rate': 0.0001087102177554439, 'epoch': 1.37}\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1090/2388 [40:17<48:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1091/2388 [40:19<48:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1092/2388 [40:21<47:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1093/2388 [40:23<47:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1094/2388 [40:25<47:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1095/2388 [40:28<47:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1096/2388 [40:30<47:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1097/2388 [40:32<47:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1098/2388 [40:34<47:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1099/2388 [40:36<47:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1100/2388 [40:39<47:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.3357, 'learning_rate': 0.00010787269681742044, 'epoch': 1.38}\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1100/2388 [40:39<47:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1101/2388 [40:41<47:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1102/2388 [40:43<47:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1103/2388 [40:45<47:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▌     | 1104/2388 [40:48<47:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▋     | 1105/2388 [40:50<47:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▋     | 1106/2388 [40:52<47:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▋     | 1107/2388 [40:54<47:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▋     | 1108/2388 [40:56<47:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▋     | 1109/2388 [40:59<47:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m46%|████▋     | 1110/2388 [41:01<47:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.1478, 'learning_rate': 0.00010703517587939699, 'epoch': 1.39}\u001b[0m\n",
      "\u001b[34m46%|████▋     | 1110/2388 [41:01<47:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1111/2388 [41:03<47:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1112/2388 [41:05<47:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1113/2388 [41:08<47:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1114/2388 [41:10<47:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1115/2388 [41:12<47:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1116/2388 [41:14<47:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1117/2388 [41:16<47:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1118/2388 [41:19<46:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1119/2388 [41:21<46:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1120/2388 [41:23<46:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.2873, 'learning_rate': 0.00010619765494137354, 'epoch': 1.41}\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1120/2388 [41:23<46:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1121/2388 [41:25<46:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1122/2388 [41:28<46:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1123/2388 [41:30<46:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1124/2388 [41:32<46:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1125/2388 [41:34<46:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1126/2388 [41:36<46:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1127/2388 [41:39<46:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1128/2388 [41:41<46:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1129/2388 [41:43<46:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1130/2388 [41:45<46:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.1323, 'learning_rate': 0.00010536013400335009, 'epoch': 1.42}\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1130/2388 [41:45<46:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1131/2388 [41:48<46:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1132/2388 [41:50<46:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1133/2388 [41:52<46:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m47%|████▋     | 1134/2388 [41:54<46:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1135/2388 [41:56<46:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1136/2388 [41:59<46:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1137/2388 [42:01<46:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1138/2388 [42:03<46:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1139/2388 [42:05<46:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1140/2388 [42:07<46:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.1256, 'learning_rate': 0.00010452261306532664, 'epoch': 1.43}\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1140/2388 [42:08<46:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1141/2388 [42:10<46:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1142/2388 [42:12<46:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1143/2388 [42:14<46:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1144/2388 [42:16<46:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1145/2388 [42:19<45:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1146/2388 [42:21<45:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1147/2388 [42:23<45:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1148/2388 [42:25<45:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1149/2388 [42:27<45:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1150/2388 [42:30<45:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.1283, 'learning_rate': 0.00010368509212730319, 'epoch': 1.44}\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1150/2388 [42:30<45:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1151/2388 [42:32<45:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1152/2388 [42:34<45:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1153/2388 [42:36<45:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1154/2388 [42:39<45:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1155/2388 [42:41<45:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1156/2388 [42:43<45:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1157/2388 [42:45<45:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m48%|████▊     | 1158/2388 [42:47<45:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▊     | 1159/2388 [42:50<45:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▊     | 1160/2388 [42:52<45:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.2533, 'learning_rate': 0.00010284757118927974, 'epoch': 1.46}\u001b[0m\n",
      "\u001b[34m49%|████▊     | 1160/2388 [42:52<45:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▊     | 1161/2388 [42:54<45:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▊     | 1162/2388 [42:56<45:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▊     | 1163/2388 [42:59<45:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▊     | 1164/2388 [43:01<45:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1165/2388 [43:03<45:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1166/2388 [43:05<45:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1167/2388 [43:07<45:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1168/2388 [43:10<45:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1169/2388 [43:12<45:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1170/2388 [43:14<45:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.3146, 'learning_rate': 0.00010201005025125629, 'epoch': 1.47}\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1170/2388 [43:14<45:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1171/2388 [43:16<45:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1172/2388 [43:19<44:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1173/2388 [43:21<44:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1174/2388 [43:23<44:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1175/2388 [43:25<44:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1176/2388 [43:27<44:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1177/2388 [43:30<44:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1178/2388 [43:32<44:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1179/2388 [43:34<44:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1180/2388 [43:36<44:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.1259, 'learning_rate': 0.00010117252931323284, 'epoch': 1.48}\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1180/2388 [43:36<44:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1181/2388 [43:39<44:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m49%|████▉     | 1182/2388 [43:41<44:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1183/2388 [43:43<44:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1184/2388 [43:45<44:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1185/2388 [43:47<44:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1186/2388 [43:50<44:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1187/2388 [43:52<44:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1188/2388 [43:54<44:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1189/2388 [43:56<44:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1190/2388 [43:59<44:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.2177, 'learning_rate': 0.00010033500837520939, 'epoch': 1.49}\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1190/2388 [43:59<44:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1191/2388 [44:01<44:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1192/2388 [44:03<44:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|████▉     | 1193/2388 [44:05<44:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1194/2388 [44:07<44:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1195/2388 [44:10<44:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1196/2388 [44:12<44:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1197/2388 [44:14<44:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1198/2388 [44:16<44:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1199/2388 [44:18<44:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1200/2388 [44:21<43:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.2228, 'learning_rate': 9.949748743718594e-05, 'epoch': 1.51}\u001b[0m\n",
      "\u001b[34m50%|█████     | 1200/2388 [44:21<43:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1201/2388 [44:23<43:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1202/2388 [44:25<43:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1203/2388 [44:27<43:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1204/2388 [44:30<43:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m50%|█████     | 1205/2388 [44:32<43:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1206/2388 [44:34<43:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1207/2388 [44:36<43:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1208/2388 [44:38<43:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1209/2388 [44:41<43:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1210/2388 [44:43<43:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0501, 'learning_rate': 9.865996649916249e-05, 'epoch': 1.52}\u001b[0m\n",
      "\u001b[34m51%|█████     | 1210/2388 [44:43<43:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1211/2388 [44:45<43:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1212/2388 [44:47<43:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1213/2388 [44:50<43:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1214/2388 [44:52<43:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1215/2388 [44:54<43:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1216/2388 [44:56<43:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1217/2388 [44:58<43:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1218/2388 [45:01<43:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1219/2388 [45:03<43:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1220/2388 [45:05<43:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.1367, 'learning_rate': 9.782244556113904e-05, 'epoch': 1.53}\u001b[0m\n",
      "\u001b[34m51%|█████     | 1220/2388 [45:05<43:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1221/2388 [45:07<43:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1222/2388 [45:10<43:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████     | 1223/2388 [45:12<43:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████▏    | 1224/2388 [45:14<43:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████▏    | 1225/2388 [45:16<43:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████▏    | 1226/2388 [45:18<43:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████▏    | 1227/2388 [45:21<42:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████▏    | 1228/2388 [45:23<42:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m51%|█████▏    | 1229/2388 [45:25<42:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1230/2388 [45:27<42:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0319, 'learning_rate': 9.698492462311559e-05, 'epoch': 1.55}\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1230/2388 [45:27<42:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1231/2388 [45:30<42:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1232/2388 [45:32<42:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1233/2388 [45:34<42:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1234/2388 [45:36<42:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1235/2388 [45:38<42:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1236/2388 [45:41<42:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1237/2388 [45:43<42:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1238/2388 [45:45<42:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1239/2388 [45:47<42:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1240/2388 [45:50<42:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.182, 'learning_rate': 9.614740368509214e-05, 'epoch': 1.56}\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1240/2388 [45:50<42:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1241/2388 [45:52<42:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1242/2388 [45:54<42:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1243/2388 [45:56<42:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1244/2388 [45:58<42:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1245/2388 [46:01<42:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1246/2388 [46:03<42:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1247/2388 [46:05<42:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1248/2388 [46:07<42:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1249/2388 [46:10<42:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1250/2388 [46:12<42:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0475, 'learning_rate': 9.530988274706868e-05, 'epoch': 1.57}\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1250/2388 [46:12<42:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1251/2388 [46:14<42:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1252/2388 [46:16<42:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m52%|█████▏    | 1253/2388 [46:18<42:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1254/2388 [46:21<41:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1255/2388 [46:23<41:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1256/2388 [46:25<41:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1257/2388 [46:27<41:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1258/2388 [46:30<41:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1259/2388 [46:32<41:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1260/2388 [46:34<41:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.1666, 'learning_rate': 9.447236180904523e-05, 'epoch': 1.58}\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1260/2388 [46:34<41:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1261/2388 [46:36<41:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1262/2388 [46:38<41:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1263/2388 [46:41<41:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1264/2388 [46:43<41:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1265/2388 [46:45<41:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1266/2388 [46:47<41:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1267/2388 [46:50<41:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1268/2388 [46:52<41:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1269/2388 [46:54<41:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1270/2388 [46:56<41:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9668, 'learning_rate': 9.363484087102178e-05, 'epoch': 1.6}\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1270/2388 [46:56<41:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1271/2388 [46:58<41:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1272/2388 [47:01<41:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1273/2388 [47:03<41:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1274/2388 [47:05<41:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1275/2388 [47:07<41:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1276/2388 [47:10<41:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m53%|█████▎    | 1277/2388 [47:12<41:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▎    | 1278/2388 [47:14<41:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▎    | 1279/2388 [47:16<41:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▎    | 1280/2388 [47:18<41:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0121, 'learning_rate': 9.279731993299833e-05, 'epoch': 1.61}\u001b[0m\n",
      "\u001b[34m54%|█████▎    | 1280/2388 [47:19<41:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▎    | 1281/2388 [47:21<41:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▎    | 1282/2388 [47:23<40:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▎    | 1283/2388 [47:25<40:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1284/2388 [47:27<40:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1285/2388 [47:30<40:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1286/2388 [47:32<40:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1287/2388 [47:34<40:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1288/2388 [47:36<40:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1289/2388 [47:38<40:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1290/2388 [47:41<40:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0399, 'learning_rate': 9.195979899497488e-05, 'epoch': 1.62}\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1290/2388 [47:41<40:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1291/2388 [47:43<40:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1292/2388 [47:45<40:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1293/2388 [47:47<40:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1294/2388 [47:50<40:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1295/2388 [47:52<40:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1296/2388 [47:54<40:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1297/2388 [47:56<40:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1298/2388 [47:58<40:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1299/2388 [48:01<40:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1300/2388 [48:03<40:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9978, 'learning_rate': 9.112227805695143e-05, 'epoch': 1.63}\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1300/2388 [48:03<40:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m54%|█████▍    | 1301/2388 [48:05<40:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1302/2388 [48:07<40:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1303/2388 [48:10<40:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1304/2388 [48:12<40:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1305/2388 [48:14<40:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1306/2388 [48:16<40:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1307/2388 [48:18<40:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1308/2388 [48:21<39:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1309/2388 [48:23<39:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1310/2388 [48:25<39:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0511, 'learning_rate': 9.028475711892798e-05, 'epoch': 1.65}\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1310/2388 [48:25<39:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1311/2388 [48:27<39:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1312/2388 [48:30<39:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▍    | 1313/2388 [48:32<39:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1314/2388 [48:34<39:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1315/2388 [48:36<40:28,  2.26s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1316/2388 [48:39<40:13,  2.25s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1317/2388 [48:41<40:01,  2.24s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1318/2388 [48:43<39:52,  2.24s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1319/2388 [48:45<39:45,  2.23s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1320/2388 [48:48<39:40,  2.23s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0894, 'learning_rate': 8.944723618090453e-05, 'epoch': 1.66}\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1320/2388 [48:48<39:40,  2.23s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1321/2388 [48:50<39:35,  2.23s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1322/2388 [48:52<39:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1323/2388 [48:54<39:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1324/2388 [48:56<39:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m55%|█████▌    | 1325/2388 [48:59<39:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1326/2388 [49:01<39:23,  2.23s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1327/2388 [49:03<39:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1328/2388 [49:05<39:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1329/2388 [49:08<39:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1330/2388 [49:10<39:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9271, 'learning_rate': 8.860971524288108e-05, 'epoch': 1.67}\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1330/2388 [49:10<39:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1331/2388 [49:12<39:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1332/2388 [49:14<39:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1333/2388 [49:16<39:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1334/2388 [49:19<39:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1335/2388 [49:21<38:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1336/2388 [49:23<38:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1337/2388 [49:25<38:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1338/2388 [49:28<38:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1339/2388 [49:30<38:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1340/2388 [49:32<38:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9233, 'learning_rate': 8.777219430485763e-05, 'epoch': 1.68}\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1340/2388 [49:32<38:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1341/2388 [49:34<38:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1342/2388 [49:36<38:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▌    | 1343/2388 [49:39<38:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▋    | 1344/2388 [49:41<38:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▋    | 1345/2388 [49:43<38:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▋    | 1346/2388 [49:45<38:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▋    | 1347/2388 [49:48<38:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▋    | 1348/2388 [49:50<38:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m56%|█████▋    | 1349/2388 [49:52<38:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1350/2388 [49:54<38:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0444, 'learning_rate': 8.693467336683418e-05, 'epoch': 1.7}\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1350/2388 [49:54<38:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1351/2388 [49:56<38:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1352/2388 [49:59<38:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1353/2388 [50:01<38:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1354/2388 [50:03<38:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1355/2388 [50:05<38:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1356/2388 [50:08<38:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1357/2388 [50:10<38:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1358/2388 [50:12<38:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1359/2388 [50:14<38:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1360/2388 [50:16<38:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0301, 'learning_rate': 8.609715242881073e-05, 'epoch': 1.71}\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1360/2388 [50:16<38:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1361/2388 [50:19<38:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1362/2388 [50:21<37:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1363/2388 [50:23<37:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1364/2388 [50:25<37:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1365/2388 [50:27<37:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1366/2388 [50:30<37:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1367/2388 [50:32<37:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1368/2388 [50:34<37:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1369/2388 [50:36<37:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1370/2388 [50:39<37:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0271, 'learning_rate': 8.525963149078728e-05, 'epoch': 1.72}\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1370/2388 [50:39<37:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1371/2388 [50:41<37:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1372/2388 [50:43<37:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m57%|█████▋    | 1373/2388 [50:45<37:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1374/2388 [50:47<37:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1375/2388 [50:50<37:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1376/2388 [50:52<37:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1377/2388 [50:54<37:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1378/2388 [50:56<37:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1379/2388 [50:59<37:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1380/2388 [51:01<37:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.8796, 'learning_rate': 8.442211055276383e-05, 'epoch': 1.73}\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1380/2388 [51:01<37:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1381/2388 [51:03<37:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1382/2388 [51:05<37:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1383/2388 [51:07<37:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1384/2388 [51:10<37:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1385/2388 [51:12<37:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1386/2388 [51:14<37:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1387/2388 [51:16<37:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1388/2388 [51:19<37:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1389/2388 [51:21<36:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1390/2388 [51:23<36:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 5.0331, 'learning_rate': 8.358458961474038e-05, 'epoch': 1.75}\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1390/2388 [51:23<36:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1391/2388 [51:25<36:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1392/2388 [51:27<36:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1393/2388 [51:30<36:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1394/2388 [51:32<36:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1395/2388 [51:34<36:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m58%|█████▊    | 1396/2388 [51:36<36:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▊    | 1397/2388 [51:39<36:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▊    | 1398/2388 [51:41<36:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▊    | 1399/2388 [51:43<36:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▊    | 1400/2388 [51:45<36:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9146, 'learning_rate': 8.274706867671693e-05, 'epoch': 1.76}\u001b[0m\n",
      "\u001b[34m59%|█████▊    | 1400/2388 [51:45<36:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▊    | 1401/2388 [51:47<36:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▊    | 1402/2388 [51:50<36:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1403/2388 [51:52<36:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1404/2388 [51:54<36:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1405/2388 [51:56<36:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1406/2388 [51:59<36:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1407/2388 [52:01<36:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1408/2388 [52:03<36:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1409/2388 [52:05<36:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1410/2388 [52:07<36:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.8347, 'learning_rate': 8.190954773869348e-05, 'epoch': 1.77}\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1410/2388 [52:08<36:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1411/2388 [52:10<36:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1412/2388 [52:12<36:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1413/2388 [52:14<36:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1414/2388 [52:16<36:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1415/2388 [52:19<36:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1416/2388 [52:21<35:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1417/2388 [52:23<35:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1418/2388 [52:25<35:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1419/2388 [52:27<35:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1420/2388 [52:30<35:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.8532, 'learning_rate': 8.107202680067003e-05, 'epoch': 1.78}\u001b[0m\n",
      "\u001b[34m59%|█████▉    | 1420/2388 [52:30<35:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1421/2388 [52:32<35:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1422/2388 [52:34<35:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1423/2388 [52:36<35:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1424/2388 [52:39<35:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1425/2388 [52:41<35:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1426/2388 [52:43<35:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1427/2388 [52:45<35:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1428/2388 [52:47<35:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1429/2388 [52:50<35:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1430/2388 [52:52<35:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.91, 'learning_rate': 8.023450586264658e-05, 'epoch': 1.8}\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1430/2388 [52:52<35:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1431/2388 [52:54<35:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|█████▉    | 1432/2388 [52:56<35:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1433/2388 [52:59<35:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1434/2388 [53:01<35:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1435/2388 [53:03<35:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1436/2388 [53:05<35:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1437/2388 [53:07<35:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1438/2388 [53:10<35:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1439/2388 [53:12<35:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1440/2388 [53:14<35:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9509, 'learning_rate': 7.939698492462313e-05, 'epoch': 1.81}\u001b[0m\n",
      "\u001b[34m60%|██████    | 1440/2388 [53:14<35:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1441/2388 [53:16<35:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1442/2388 [53:19<35:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1443/2388 [53:21<34:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m60%|██████    | 1444/2388 [53:23<34:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1445/2388 [53:25<34:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1446/2388 [53:27<34:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1447/2388 [53:30<34:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1448/2388 [53:32<34:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1449/2388 [53:34<34:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1450/2388 [53:36<34:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9448, 'learning_rate': 7.855946398659967e-05, 'epoch': 1.82}\u001b[0m\n",
      "\u001b[34m61%|██████    | 1450/2388 [53:36<34:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1451/2388 [53:39<34:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1452/2388 [53:41<34:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1453/2388 [53:43<34:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1454/2388 [53:45<34:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1455/2388 [53:47<34:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1456/2388 [53:50<34:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1457/2388 [53:52<34:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1458/2388 [53:54<34:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1459/2388 [53:56<34:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1460/2388 [53:59<34:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.8593, 'learning_rate': 7.772194304857622e-05, 'epoch': 1.83}\u001b[0m\n",
      "\u001b[34m61%|██████    | 1460/2388 [53:59<34:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1461/2388 [54:01<34:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████    | 1462/2388 [54:03<34:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████▏   | 1463/2388 [54:05<34:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████▏   | 1464/2388 [54:07<34:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████▏   | 1465/2388 [54:10<34:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████▏   | 1466/2388 [54:12<34:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████▏   | 1467/2388 [54:14<34:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m61%|██████▏   | 1468/2388 [54:16<34:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1469/2388 [54:19<34:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1470/2388 [54:21<34:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9139, 'learning_rate': 7.688442211055277e-05, 'epoch': 1.85}\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1470/2388 [54:21<34:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1471/2388 [54:23<33:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1472/2388 [54:25<33:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1473/2388 [54:28<33:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1474/2388 [54:30<33:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1475/2388 [54:32<33:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1476/2388 [54:34<33:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1477/2388 [54:36<33:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1478/2388 [54:39<33:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1479/2388 [54:41<33:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1480/2388 [54:43<33:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9439, 'learning_rate': 7.604690117252932e-05, 'epoch': 1.86}\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1480/2388 [54:43<33:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1481/2388 [54:45<33:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1482/2388 [54:48<33:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1483/2388 [54:50<33:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1484/2388 [54:52<33:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1485/2388 [54:54<33:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1486/2388 [54:56<33:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1487/2388 [54:59<33:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1488/2388 [55:01<33:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1489/2388 [55:03<33:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1490/2388 [55:05<33:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.7867, 'learning_rate': 7.520938023450587e-05, 'epoch': 1.87}\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1490/2388 [55:05<33:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1491/2388 [55:08<33:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m62%|██████▏   | 1492/2388 [55:10<33:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1493/2388 [55:12<33:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1494/2388 [55:14<33:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1495/2388 [55:16<33:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1496/2388 [55:19<33:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1497/2388 [55:21<33:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1498/2388 [55:23<32:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1499/2388 [55:25<32:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1500/2388 [55:28<32:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.8901, 'learning_rate': 7.437185929648241e-05, 'epoch': 1.88}\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1500/2388 [55:28<32:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1501/2388 [55:30<32:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1502/2388 [55:32<32:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1503/2388 [55:34<32:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1504/2388 [55:36<32:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1505/2388 [55:39<32:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1506/2388 [55:41<32:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1507/2388 [55:43<32:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1508/2388 [55:45<32:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1509/2388 [55:48<32:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1510/2388 [55:50<32:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9417, 'learning_rate': 7.353433835845896e-05, 'epoch': 1.9}\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1510/2388 [55:50<32:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1511/2388 [55:52<32:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1512/2388 [55:54<32:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1513/2388 [55:56<32:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1514/2388 [55:59<32:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1515/2388 [56:01<32:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m63%|██████▎   | 1516/2388 [56:03<32:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▎   | 1517/2388 [56:05<32:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▎   | 1518/2388 [56:08<32:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▎   | 1519/2388 [56:10<32:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▎   | 1520/2388 [56:12<32:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.8703, 'learning_rate': 7.269681742043551e-05, 'epoch': 1.91}\u001b[0m\n",
      "\u001b[34m64%|██████▎   | 1520/2388 [56:12<32:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▎   | 1521/2388 [56:14<32:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▎   | 1522/2388 [56:16<32:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1523/2388 [56:19<32:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1524/2388 [56:21<32:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1525/2388 [56:23<31:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1526/2388 [56:25<31:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1527/2388 [56:28<31:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1528/2388 [56:30<31:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1529/2388 [56:32<31:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1530/2388 [56:34<31:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9005, 'learning_rate': 7.185929648241206e-05, 'epoch': 1.92}\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1530/2388 [56:34<31:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1531/2388 [56:36<31:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1532/2388 [56:39<31:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1533/2388 [56:41<31:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1534/2388 [56:43<31:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1535/2388 [56:45<31:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1536/2388 [56:48<31:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1537/2388 [56:50<31:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1538/2388 [56:52<31:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1539/2388 [56:54<31:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1540/2388 [56:57<31:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.8576, 'learning_rate': 7.102177554438861e-05, 'epoch': 1.93}\u001b[0m\n",
      "\u001b[34m64%|██████▍   | 1540/2388 [56:57<31:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1541/2388 [56:59<31:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1542/2388 [57:01<31:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1543/2388 [57:03<31:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1544/2388 [57:05<31:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1545/2388 [57:08<31:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1546/2388 [57:10<31:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1547/2388 [57:12<31:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1548/2388 [57:14<31:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1549/2388 [57:17<31:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1550/2388 [57:19<31:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.7697, 'learning_rate': 7.018425460636516e-05, 'epoch': 1.95}\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1550/2388 [57:19<31:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1551/2388 [57:21<31:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▍   | 1552/2388 [57:23<30:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1553/2388 [57:25<30:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1554/2388 [57:28<30:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1555/2388 [57:30<30:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1556/2388 [57:32<30:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1557/2388 [57:34<30:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1558/2388 [57:37<30:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1559/2388 [57:39<30:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1560/2388 [57:41<30:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.7677, 'learning_rate': 6.93467336683417e-05, 'epoch': 1.96}\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1560/2388 [57:41<30:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1561/2388 [57:43<30:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1562/2388 [57:45<30:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1563/2388 [57:48<30:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m65%|██████▌   | 1564/2388 [57:50<30:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1565/2388 [57:52<30:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1566/2388 [57:54<30:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1567/2388 [57:57<30:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1568/2388 [57:59<30:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1569/2388 [58:01<30:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1570/2388 [58:03<30:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.7798, 'learning_rate': 6.850921273031826e-05, 'epoch': 1.97}\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1570/2388 [58:03<30:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1571/2388 [58:05<30:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1572/2388 [58:08<30:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1573/2388 [58:10<30:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1574/2388 [58:12<30:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1575/2388 [58:14<30:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1576/2388 [58:17<30:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1577/2388 [58:19<30:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1578/2388 [58:21<30:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1579/2388 [58:23<29:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1580/2388 [58:25<29:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.9645, 'learning_rate': 6.76716917922948e-05, 'epoch': 1.98}\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1580/2388 [58:25<29:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1581/2388 [58:28<29:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▌   | 1582/2388 [58:30<29:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▋   | 1583/2388 [58:32<29:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▋   | 1584/2388 [58:34<29:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▋   | 1585/2388 [58:37<29:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▋   | 1586/2388 [58:39<29:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▋   | 1587/2388 [58:41<29:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m66%|██████▋   | 1588/2388 [58:43<29:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1589/2388 [58:45<29:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1590/2388 [58:48<29:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.7386, 'learning_rate': 6.683417085427135e-05, 'epoch': 2.0}\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1590/2388 [58:48<29:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1591/2388 [58:50<29:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1592/2388 [58:51<25:29,  1.92s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1593/2388 [58:53<26:39,  2.01s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1594/2388 [58:56<27:28,  2.08s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1595/2388 [58:58<28:01,  2.12s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1596/2388 [59:00<28:23,  2.15s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1597/2388 [59:02<28:38,  2.17s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1598/2388 [59:04<28:48,  2.19s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1599/2388 [59:07<28:54,  2.20s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1600/2388 [59:09<28:58,  2.21s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.7144, 'learning_rate': 6.59966499162479e-05, 'epoch': 2.01}\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1600/2388 [59:09<28:58,  2.21s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1601/2388 [59:11<29:00,  2.21s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1602/2388 [59:13<29:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1603/2388 [59:16<29:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1604/2388 [59:18<29:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1605/2388 [59:20<28:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1606/2388 [59:22<28:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1607/2388 [59:24<28:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1608/2388 [59:27<28:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1609/2388 [59:29<28:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1610/2388 [59:31<28:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5078, 'learning_rate': 6.515912897822445e-05, 'epoch': 2.02}\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1610/2388 [59:31<28:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m67%|██████▋   | 1611/2388 [59:33<28:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1612/2388 [59:36<28:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1613/2388 [59:38<28:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1614/2388 [59:40<28:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1615/2388 [59:42<28:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1616/2388 [59:44<28:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1617/2388 [59:47<28:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1618/2388 [59:49<28:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1619/2388 [59:51<28:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1620/2388 [59:53<28:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.6053, 'learning_rate': 6.4321608040201e-05, 'epoch': 2.04}\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1620/2388 [59:53<28:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1621/2388 [59:56<28:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1622/2388 [59:58<28:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1623/2388 [1:00:00<28:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1624/2388 [1:00:02<28:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1625/2388 [1:00:04<28:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1626/2388 [1:00:07<28:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1627/2388 [1:00:09<28:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1628/2388 [1:00:11<28:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1629/2388 [1:00:13<28:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1630/2388 [1:00:16<28:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4976, 'learning_rate': 6.348408710217755e-05, 'epoch': 2.05}\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1630/2388 [1:00:16<28:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1631/2388 [1:00:18<28:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1632/2388 [1:00:20<28:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1633/2388 [1:00:22<27:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1634/2388 [1:00:25<27:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m68%|██████▊   | 1635/2388 [1:00:27<27:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▊   | 1636/2388 [1:00:29<27:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▊   | 1637/2388 [1:00:31<27:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▊   | 1638/2388 [1:00:33<27:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▊   | 1639/2388 [1:00:36<27:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▊   | 1640/2388 [1:00:38<27:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.6087, 'learning_rate': 6.26465661641541e-05, 'epoch': 2.06}\u001b[0m\n",
      "\u001b[34m69%|██████▊   | 1640/2388 [1:00:38<27:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▊   | 1641/2388 [1:00:40<27:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1642/2388 [1:00:42<27:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1643/2388 [1:00:45<27:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1644/2388 [1:00:47<27:35,  2.23s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1645/2388 [1:00:49<27:33,  2.23s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1646/2388 [1:00:51<27:31,  2.23s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1647/2388 [1:00:53<27:29,  2.23s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1648/2388 [1:00:56<27:26,  2.23s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1649/2388 [1:00:58<27:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1650/2388 [1:01:00<27:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5623, 'learning_rate': 6.180904522613065e-05, 'epoch': 2.07}\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1650/2388 [1:01:00<27:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1651/2388 [1:01:02<27:20,  2.23s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1652/2388 [1:01:05<27:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1653/2388 [1:01:07<27:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1654/2388 [1:01:09<27:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1655/2388 [1:01:11<27:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1656/2388 [1:01:13<27:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1657/2388 [1:01:16<27:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1658/2388 [1:01:18<27:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m69%|██████▉   | 1659/2388 [1:01:20<27:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1660/2388 [1:01:22<26:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4837, 'learning_rate': 6.097152428810721e-05, 'epoch': 2.09}\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1660/2388 [1:01:22<26:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1661/2388 [1:01:25<26:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1662/2388 [1:01:27<26:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1663/2388 [1:01:29<26:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1664/2388 [1:01:31<26:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1665/2388 [1:01:33<26:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1666/2388 [1:01:36<26:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1667/2388 [1:01:38<26:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1668/2388 [1:01:40<26:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1669/2388 [1:01:42<26:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1670/2388 [1:01:45<26:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4921, 'learning_rate': 6.013400335008376e-05, 'epoch': 2.1}\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1670/2388 [1:01:45<26:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|██████▉   | 1671/2388 [1:01:47<26:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1672/2388 [1:01:49<26:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1673/2388 [1:01:51<26:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1674/2388 [1:01:53<26:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1675/2388 [1:01:56<26:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1676/2388 [1:01:58<26:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1677/2388 [1:02:00<26:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1678/2388 [1:02:02<26:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1679/2388 [1:02:05<26:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1680/2388 [1:02:07<26:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5524, 'learning_rate': 5.929648241206031e-05, 'epoch': 2.11}\u001b[0m\n",
      "\u001b[34m70%|███████   | 1680/2388 [1:02:07<26:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1681/2388 [1:02:09<26:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1682/2388 [1:02:11<26:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m70%|███████   | 1683/2388 [1:02:13<26:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1684/2388 [1:02:16<26:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1685/2388 [1:02:18<26:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1686/2388 [1:02:20<26:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1687/2388 [1:02:22<25:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1688/2388 [1:02:25<25:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1689/2388 [1:02:27<25:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1690/2388 [1:02:29<25:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.6271, 'learning_rate': 5.8458961474036856e-05, 'epoch': 2.12}\u001b[0m\n",
      "\u001b[34m71%|███████   | 1690/2388 [1:02:29<25:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1691/2388 [1:02:31<25:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1692/2388 [1:02:33<25:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1693/2388 [1:02:36<25:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1694/2388 [1:02:38<25:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1695/2388 [1:02:40<25:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1696/2388 [1:02:42<25:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1697/2388 [1:02:45<25:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1698/2388 [1:02:47<25:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1699/2388 [1:02:49<25:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1700/2388 [1:02:51<25:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.572, 'learning_rate': 5.7621440536013406e-05, 'epoch': 2.14}\u001b[0m\n",
      "\u001b[34m71%|███████   | 1700/2388 [1:02:51<25:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████   | 1701/2388 [1:02:54<25:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████▏  | 1702/2388 [1:02:56<25:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████▏  | 1703/2388 [1:02:58<25:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████▏  | 1704/2388 [1:03:00<25:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████▏  | 1705/2388 [1:03:02<25:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████▏  | 1706/2388 [1:03:05<25:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m71%|███████▏  | 1707/2388 [1:03:07<25:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1708/2388 [1:03:09<25:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1709/2388 [1:03:11<25:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1710/2388 [1:03:14<25:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5009, 'learning_rate': 5.6783919597989955e-05, 'epoch': 2.15}\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1710/2388 [1:03:14<25:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1711/2388 [1:03:16<25:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1712/2388 [1:03:18<25:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1713/2388 [1:03:20<25:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1714/2388 [1:03:22<24:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1715/2388 [1:03:25<24:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1716/2388 [1:03:27<24:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1717/2388 [1:03:29<24:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1718/2388 [1:03:31<24:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1719/2388 [1:03:34<24:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1720/2388 [1:03:36<24:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.6223, 'learning_rate': 5.5946398659966505e-05, 'epoch': 2.16}\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1720/2388 [1:03:36<24:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1721/2388 [1:03:38<24:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1722/2388 [1:03:40<24:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1723/2388 [1:03:42<24:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1724/2388 [1:03:45<24:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1725/2388 [1:03:47<24:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1726/2388 [1:03:49<24:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1727/2388 [1:03:51<24:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1728/2388 [1:03:54<24:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1729/2388 [1:03:56<24:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1730/2388 [1:03:58<24:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.517, 'learning_rate': 5.5108877721943054e-05, 'epoch': 2.17}\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1730/2388 [1:03:58<24:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m72%|███████▏  | 1731/2388 [1:04:00<24:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1732/2388 [1:04:02<24:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1733/2388 [1:04:05<24:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1734/2388 [1:04:07<24:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1735/2388 [1:04:09<24:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1736/2388 [1:04:11<24:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1737/2388 [1:04:14<24:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1738/2388 [1:04:16<24:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1739/2388 [1:04:18<24:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1740/2388 [1:04:20<24:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4959, 'learning_rate': 5.4271356783919604e-05, 'epoch': 2.19}\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1740/2388 [1:04:20<24:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1741/2388 [1:04:22<23:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1742/2388 [1:04:25<23:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1743/2388 [1:04:27<23:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1744/2388 [1:04:29<23:52,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1745/2388 [1:04:31<23:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1746/2388 [1:04:34<23:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1747/2388 [1:04:36<23:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1748/2388 [1:04:38<23:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1749/2388 [1:04:40<23:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1750/2388 [1:04:42<23:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5146, 'learning_rate': 5.3433835845896153e-05, 'epoch': 2.2}\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1750/2388 [1:04:43<23:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1751/2388 [1:04:45<23:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1752/2388 [1:04:47<23:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1753/2388 [1:04:49<23:32,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1754/2388 [1:04:51<23:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m73%|███████▎  | 1755/2388 [1:04:54<23:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▎  | 1756/2388 [1:04:56<23:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▎  | 1757/2388 [1:04:58<23:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▎  | 1758/2388 [1:05:00<23:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▎  | 1759/2388 [1:05:03<23:19,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▎  | 1760/2388 [1:05:05<23:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.3891, 'learning_rate': 5.25963149078727e-05, 'epoch': 2.21}\u001b[0m\n",
      "\u001b[34m74%|███████▎  | 1760/2388 [1:05:05<23:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▎  | 1761/2388 [1:05:07<23:15,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1762/2388 [1:05:09<23:13,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1763/2388 [1:05:11<23:10,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1764/2388 [1:05:14<23:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1765/2388 [1:05:16<23:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1766/2388 [1:05:18<23:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1767/2388 [1:05:20<23:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1768/2388 [1:05:23<22:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1769/2388 [1:05:25<22:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1770/2388 [1:05:27<22:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5665, 'learning_rate': 5.175879396984925e-05, 'epoch': 2.22}\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1770/2388 [1:05:27<22:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1771/2388 [1:05:29<22:52,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1772/2388 [1:05:31<22:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1773/2388 [1:05:34<22:48,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1774/2388 [1:05:36<22:46,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1775/2388 [1:05:38<22:43,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1776/2388 [1:05:40<22:42,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1777/2388 [1:05:43<22:39,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1778/2388 [1:05:45<22:37,  2.23s/it]\u001b[0m\n",
      "\u001b[34m74%|███████▍  | 1779/2388 [1:05:47<22:35,  2.23s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1780/2388 [1:05:49<22:33,  2.23s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5631, 'learning_rate': 5.09212730318258e-05, 'epoch': 2.24}\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1780/2388 [1:05:49<22:33,  2.23s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1781/2388 [1:05:51<22:31,  2.23s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1782/2388 [1:05:54<22:28,  2.23s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1783/2388 [1:05:56<22:26,  2.23s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1784/2388 [1:05:58<22:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1785/2388 [1:06:00<22:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1786/2388 [1:06:03<22:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1787/2388 [1:06:05<22:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1788/2388 [1:06:07<22:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1789/2388 [1:06:09<22:12,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1790/2388 [1:06:11<22:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5365, 'learning_rate': 5.008375209380235e-05, 'epoch': 2.25}\u001b[0m\n",
      "\u001b[34m75%|███████▍  | 1790/2388 [1:06:12<22:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1791/2388 [1:06:14<22:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1792/2388 [1:06:16<22:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1793/2388 [1:06:18<22:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1794/2388 [1:06:20<22:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1795/2388 [1:06:23<21:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1796/2388 [1:06:25<21:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1797/2388 [1:06:27<21:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1798/2388 [1:06:29<21:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1799/2388 [1:06:31<21:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1800/2388 [1:06:34<21:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5276, 'learning_rate': 4.92462311557789e-05, 'epoch': 2.26}\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1800/2388 [1:06:34<21:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1801/2388 [1:06:36<21:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m75%|███████▌  | 1802/2388 [1:06:38<21:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1803/2388 [1:06:40<21:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1804/2388 [1:06:43<21:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1805/2388 [1:06:45<21:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1806/2388 [1:06:47<21:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1807/2388 [1:06:49<21:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1808/2388 [1:06:52<21:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1809/2388 [1:06:54<21:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1810/2388 [1:06:56<21:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4912, 'learning_rate': 4.840871021775545e-05, 'epoch': 2.27}\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1810/2388 [1:06:56<21:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1811/2388 [1:06:58<21:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1812/2388 [1:07:00<21:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1813/2388 [1:07:03<21:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1814/2388 [1:07:05<21:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1815/2388 [1:07:07<21:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1816/2388 [1:07:09<21:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1817/2388 [1:07:12<21:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1818/2388 [1:07:14<21:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1819/2388 [1:07:16<21:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1820/2388 [1:07:18<21:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.431, 'learning_rate': 4.7571189279732e-05, 'epoch': 2.29}\u001b[0m\n",
      "\u001b[34m76%|███████▌  | 1820/2388 [1:07:18<21:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▋  | 1821/2388 [1:07:20<21:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▋  | 1822/2388 [1:07:23<20:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▋  | 1823/2388 [1:07:25<20:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▋  | 1824/2388 [1:07:27<20:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▋  | 1825/2388 [1:07:29<20:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m76%|███████▋  | 1826/2388 [1:07:32<20:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1827/2388 [1:07:34<20:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1828/2388 [1:07:36<20:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1829/2388 [1:07:38<20:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1830/2388 [1:07:40<20:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5481, 'learning_rate': 4.673366834170855e-05, 'epoch': 2.3}\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1830/2388 [1:07:40<20:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1831/2388 [1:07:43<20:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1832/2388 [1:07:45<20:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1833/2388 [1:07:47<20:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1834/2388 [1:07:49<20:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1835/2388 [1:07:52<20:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1836/2388 [1:07:54<20:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1837/2388 [1:07:56<20:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1838/2388 [1:07:58<20:23,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1839/2388 [1:08:00<20:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1840/2388 [1:08:03<20:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4565, 'learning_rate': 4.58961474036851e-05, 'epoch': 2.31}\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1840/2388 [1:08:03<20:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1841/2388 [1:08:05<20:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1842/2388 [1:08:07<20:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1843/2388 [1:08:09<20:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1844/2388 [1:08:12<20:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1845/2388 [1:08:14<20:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1846/2388 [1:08:16<20:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1847/2388 [1:08:18<20:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1848/2388 [1:08:20<20:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1849/2388 [1:08:23<19:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1850/2388 [1:08:25<19:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4808, 'learning_rate': 4.505862646566165e-05, 'epoch': 2.32}\u001b[0m\n",
      "\u001b[34m77%|███████▋  | 1850/2388 [1:08:25<19:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1851/2388 [1:08:27<19:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1852/2388 [1:08:29<19:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1853/2388 [1:08:32<19:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1854/2388 [1:08:34<19:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1855/2388 [1:08:36<19:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1856/2388 [1:08:38<19:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1857/2388 [1:08:40<19:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1858/2388 [1:08:43<19:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1859/2388 [1:08:45<19:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1860/2388 [1:08:47<19:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4905, 'learning_rate': 4.42211055276382e-05, 'epoch': 2.34}\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1860/2388 [1:08:47<19:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1861/2388 [1:08:49<19:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1862/2388 [1:08:52<19:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1863/2388 [1:08:54<19:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1864/2388 [1:08:56<19:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1865/2388 [1:08:58<19:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1866/2388 [1:09:00<19:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1867/2388 [1:09:03<19:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1868/2388 [1:09:05<19:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1869/2388 [1:09:07<19:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1870/2388 [1:09:09<19:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5841, 'learning_rate': 4.338358458961474e-05, 'epoch': 2.35}\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1870/2388 [1:09:09<19:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1871/2388 [1:09:12<19:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1872/2388 [1:09:14<19:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1873/2388 [1:09:16<19:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m78%|███████▊  | 1874/2388 [1:09:18<19:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▊  | 1875/2388 [1:09:20<19:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▊  | 1876/2388 [1:09:23<18:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▊  | 1877/2388 [1:09:25<18:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▊  | 1878/2388 [1:09:27<18:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▊  | 1879/2388 [1:09:29<18:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▊  | 1880/2388 [1:09:32<18:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5357, 'learning_rate': 4.254606365159129e-05, 'epoch': 2.36}\u001b[0m\n",
      "\u001b[34m79%|███████▊  | 1880/2388 [1:09:32<18:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1881/2388 [1:09:34<18:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1882/2388 [1:09:36<18:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1883/2388 [1:09:38<18:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1884/2388 [1:09:40<18:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1885/2388 [1:09:43<18:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1886/2388 [1:09:45<18:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1887/2388 [1:09:47<18:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1888/2388 [1:09:49<18:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1889/2388 [1:09:52<18:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1890/2388 [1:09:54<18:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4809, 'learning_rate': 4.170854271356784e-05, 'epoch': 2.37}\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1890/2388 [1:09:54<18:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1891/2388 [1:09:56<18:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1892/2388 [1:09:58<18:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1893/2388 [1:10:00<18:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1894/2388 [1:10:03<18:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1895/2388 [1:10:05<18:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1896/2388 [1:10:07<18:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1897/2388 [1:10:09<18:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m79%|███████▉  | 1898/2388 [1:10:12<18:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1899/2388 [1:10:14<18:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1900/2388 [1:10:16<18:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.5104, 'learning_rate': 4.087102177554439e-05, 'epoch': 2.39}\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1900/2388 [1:10:16<18:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1901/2388 [1:10:18<18:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1902/2388 [1:10:20<18:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1903/2388 [1:10:23<17:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1904/2388 [1:10:25<17:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1905/2388 [1:10:27<17:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1906/2388 [1:10:29<17:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1907/2388 [1:10:32<17:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1908/2388 [1:10:34<17:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1909/2388 [1:10:36<17:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1910/2388 [1:10:38<17:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4432, 'learning_rate': 4.003350083752094e-05, 'epoch': 2.4}\u001b[0m\n",
      "\u001b[34m80%|███████▉  | 1910/2388 [1:10:38<17:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1911/2388 [1:10:40<17:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1912/2388 [1:10:43<17:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1913/2388 [1:10:45<17:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1914/2388 [1:10:47<17:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1915/2388 [1:10:49<17:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1916/2388 [1:10:52<17:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1917/2388 [1:10:54<17:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1918/2388 [1:10:56<17:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1919/2388 [1:10:58<17:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1920/2388 [1:11:00<17:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4274, 'learning_rate': 3.919597989949749e-05, 'epoch': 2.41}\u001b[0m\n",
      "\u001b[34m80%|████████  | 1920/2388 [1:11:00<17:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1921/2388 [1:11:03<17:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m80%|████████  | 1922/2388 [1:11:05<17:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1923/2388 [1:11:07<17:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1924/2388 [1:11:09<17:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1925/2388 [1:11:12<17:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1926/2388 [1:11:14<17:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1927/2388 [1:11:16<17:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1928/2388 [1:11:18<17:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1929/2388 [1:11:20<17:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1930/2388 [1:11:23<16:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.3789, 'learning_rate': 3.835845896147404e-05, 'epoch': 2.42}\u001b[0m\n",
      "\u001b[34m81%|████████  | 1930/2388 [1:11:23<16:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1931/2388 [1:11:25<16:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1932/2388 [1:11:27<16:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1933/2388 [1:11:29<16:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1934/2388 [1:11:32<16:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1935/2388 [1:11:34<16:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1936/2388 [1:11:36<16:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1937/2388 [1:11:38<16:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1938/2388 [1:11:40<16:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1939/2388 [1:11:43<16:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████  | 1940/2388 [1:11:45<16:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.34, 'learning_rate': 3.752093802345059e-05, 'epoch': 2.44}\u001b[0m\n",
      "\u001b[34m81%|████████  | 1940/2388 [1:11:45<16:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████▏ | 1941/2388 [1:11:47<16:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████▏ | 1942/2388 [1:11:49<16:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████▏ | 1943/2388 [1:11:52<16:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████▏ | 1944/2388 [1:11:54<16:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████▏ | 1945/2388 [1:11:56<16:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m81%|████████▏ | 1946/2388 [1:11:58<16:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1947/2388 [1:12:00<16:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1948/2388 [1:12:03<16:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1949/2388 [1:12:05<16:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1950/2388 [1:12:07<16:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.575, 'learning_rate': 3.668341708542714e-05, 'epoch': 2.45}\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1950/2388 [1:12:07<16:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1951/2388 [1:12:09<16:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1952/2388 [1:12:12<16:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1953/2388 [1:12:14<16:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1954/2388 [1:12:16<16:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1955/2388 [1:12:18<16:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1956/2388 [1:12:20<16:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1957/2388 [1:12:23<15:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1958/2388 [1:12:25<15:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1959/2388 [1:12:27<15:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1960/2388 [1:12:29<15:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4926, 'learning_rate': 3.5845896147403686e-05, 'epoch': 2.46}\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1960/2388 [1:12:29<15:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1961/2388 [1:12:32<15:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1962/2388 [1:12:34<15:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1963/2388 [1:12:36<15:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1964/2388 [1:12:38<15:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1965/2388 [1:12:40<15:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1966/2388 [1:12:43<15:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1967/2388 [1:12:45<15:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1968/2388 [1:12:47<15:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1969/2388 [1:12:49<15:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1970/2388 [1:12:52<15:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4037, 'learning_rate': 3.5008375209380236e-05, 'epoch': 2.47}\u001b[0m\n",
      "\u001b[34m82%|████████▏ | 1970/2388 [1:12:52<15:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1971/2388 [1:12:54<15:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1972/2388 [1:12:56<15:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1973/2388 [1:12:58<15:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1974/2388 [1:13:01<15:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1975/2388 [1:13:03<15:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1976/2388 [1:13:05<15:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1977/2388 [1:13:07<15:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1978/2388 [1:13:09<15:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1979/2388 [1:13:12<15:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1980/2388 [1:13:14<15:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4626, 'learning_rate': 3.4170854271356785e-05, 'epoch': 2.49}\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1980/2388 [1:13:14<15:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1981/2388 [1:13:16<15:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1982/2388 [1:13:18<15:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1983/2388 [1:13:21<15:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1984/2388 [1:13:23<14:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1985/2388 [1:13:25<14:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1986/2388 [1:13:27<14:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1987/2388 [1:13:29<14:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1988/2388 [1:13:32<14:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1989/2388 [1:13:34<14:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1990/2388 [1:13:36<14:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2107, 'learning_rate': 3.3333333333333335e-05, 'epoch': 2.5}\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1990/2388 [1:13:36<14:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1991/2388 [1:13:38<14:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1992/2388 [1:13:41<14:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m83%|████████▎ | 1993/2388 [1:13:43<14:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▎ | 1994/2388 [1:13:45<14:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▎ | 1995/2388 [1:13:47<14:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▎ | 1996/2388 [1:13:49<14:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▎ | 1997/2388 [1:13:52<14:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▎ | 1998/2388 [1:13:54<14:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▎ | 1999/2388 [1:13:56<14:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2000/2388 [1:13:58<14:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4174, 'learning_rate': 3.2495812395309884e-05, 'epoch': 2.51}\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2000/2388 [1:13:58<14:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2001/2388 [1:14:01<14:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2002/2388 [1:14:03<14:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2003/2388 [1:14:05<14:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2004/2388 [1:14:07<14:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2005/2388 [1:14:09<14:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2006/2388 [1:14:12<14:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2007/2388 [1:14:14<14:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2008/2388 [1:14:16<14:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2009/2388 [1:14:18<14:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2010/2388 [1:14:21<14:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.3895, 'learning_rate': 3.1658291457286434e-05, 'epoch': 2.53}\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2010/2388 [1:14:21<14:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2011/2388 [1:14:23<13:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2012/2388 [1:14:25<13:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2013/2388 [1:14:27<13:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2014/2388 [1:14:29<13:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2015/2388 [1:14:32<13:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2016/2388 [1:14:34<13:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m84%|████████▍ | 2017/2388 [1:14:36<13:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2018/2388 [1:14:38<13:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2019/2388 [1:14:41<13:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2020/2388 [1:14:43<13:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4003, 'learning_rate': 3.0820770519262983e-05, 'epoch': 2.54}\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2020/2388 [1:14:43<13:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2021/2388 [1:14:45<13:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2022/2388 [1:14:47<13:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2023/2388 [1:14:49<13:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2024/2388 [1:14:52<13:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2025/2388 [1:14:54<13:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2026/2388 [1:14:56<13:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2027/2388 [1:14:58<13:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2028/2388 [1:15:01<13:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▍ | 2029/2388 [1:15:03<13:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2030/2388 [1:15:05<13:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2404, 'learning_rate': 2.998324958123953e-05, 'epoch': 2.55}\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2030/2388 [1:15:05<13:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2031/2388 [1:15:07<13:14,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2032/2388 [1:15:09<13:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2033/2388 [1:15:12<13:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2034/2388 [1:15:14<13:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2035/2388 [1:15:16<13:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2036/2388 [1:15:18<13:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2037/2388 [1:15:21<13:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2038/2388 [1:15:23<12:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2039/2388 [1:15:25<12:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2040/2388 [1:15:27<12:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.1652, 'learning_rate': 2.914572864321608e-05, 'epoch': 2.56}\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2040/2388 [1:15:27<12:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m85%|████████▌ | 2041/2388 [1:15:29<12:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2042/2388 [1:15:32<12:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2043/2388 [1:15:34<12:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2044/2388 [1:15:36<12:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2045/2388 [1:15:38<12:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2046/2388 [1:15:41<12:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2047/2388 [1:15:43<12:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2048/2388 [1:15:45<12:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2049/2388 [1:15:47<12:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2050/2388 [1:15:49<12:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4431, 'learning_rate': 2.830820770519263e-05, 'epoch': 2.58}\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2050/2388 [1:15:50<12:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2051/2388 [1:15:52<12:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2052/2388 [1:15:54<12:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2053/2388 [1:15:56<12:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2054/2388 [1:15:58<12:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2055/2388 [1:16:01<12:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2056/2388 [1:16:03<12:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2057/2388 [1:16:05<12:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2058/2388 [1:16:07<12:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▌ | 2059/2388 [1:16:10<12:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▋ | 2060/2388 [1:16:12<12:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2487, 'learning_rate': 2.7470686767169178e-05, 'epoch': 2.59}\u001b[0m\n",
      "\u001b[34m86%|████████▋ | 2060/2388 [1:16:12<12:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▋ | 2061/2388 [1:16:14<12:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▋ | 2062/2388 [1:16:16<12:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▋ | 2063/2388 [1:16:18<12:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▋ | 2064/2388 [1:16:21<12:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m86%|████████▋ | 2065/2388 [1:16:23<11:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2066/2388 [1:16:25<11:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2067/2388 [1:16:27<11:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2068/2388 [1:16:30<11:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2069/2388 [1:16:32<11:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2070/2388 [1:16:34<11:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4226, 'learning_rate': 2.6633165829145728e-05, 'epoch': 2.6}\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2070/2388 [1:16:34<11:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2071/2388 [1:16:36<11:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2072/2388 [1:16:38<11:43,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2073/2388 [1:16:41<11:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2074/2388 [1:16:43<11:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2075/2388 [1:16:45<11:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2076/2388 [1:16:47<11:34,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2077/2388 [1:16:50<11:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2078/2388 [1:16:52<11:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2079/2388 [1:16:54<11:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2080/2388 [1:16:56<11:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.1242, 'learning_rate': 2.5795644891122277e-05, 'epoch': 2.61}\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2080/2388 [1:16:56<11:25,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2081/2388 [1:16:58<11:23,  2.23s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2082/2388 [1:17:01<11:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2083/2388 [1:17:03<11:18,  2.23s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2084/2388 [1:17:05<11:16,  2.23s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2085/2388 [1:17:07<11:14,  2.23s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2086/2388 [1:17:10<11:12,  2.23s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2087/2388 [1:17:12<11:09,  2.23s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2088/2388 [1:17:14<11:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m87%|████████▋ | 2089/2388 [1:17:16<11:05,  2.22s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2090/2388 [1:17:18<11:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4398, 'learning_rate': 2.4958123953098827e-05, 'epoch': 2.63}\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2090/2388 [1:17:19<11:03,  2.22s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2091/2388 [1:17:21<11:00,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2092/2388 [1:17:23<10:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2093/2388 [1:17:25<10:56,  2.22s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2094/2388 [1:17:27<10:54,  2.22s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2095/2388 [1:17:30<10:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2096/2388 [1:17:32<10:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2097/2388 [1:17:34<10:47,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2098/2388 [1:17:36<10:45,  2.22s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2099/2388 [1:17:39<10:43,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2100/2388 [1:17:41<10:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4541, 'learning_rate': 2.4120603015075376e-05, 'epoch': 2.64}\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2100/2388 [1:17:41<10:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2101/2388 [1:17:43<10:38,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2102/2388 [1:17:45<10:36,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2103/2388 [1:17:47<10:34,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2104/2388 [1:17:50<10:31,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2105/2388 [1:17:52<10:29,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2106/2388 [1:17:54<10:27,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2107/2388 [1:17:56<10:25,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2108/2388 [1:17:59<10:23,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2109/2388 [1:18:01<10:20,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2110/2388 [1:18:03<10:18,  2.23s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2234, 'learning_rate': 2.3283082077051926e-05, 'epoch': 2.65}\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2110/2388 [1:18:03<10:18,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2111/2388 [1:18:05<10:16,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2112/2388 [1:18:07<10:14,  2.23s/it]\u001b[0m\n",
      "\u001b[34m88%|████████▊ | 2113/2388 [1:18:10<10:11,  2.23s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▊ | 2114/2388 [1:18:12<10:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▊ | 2115/2388 [1:18:14<10:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▊ | 2116/2388 [1:18:16<10:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▊ | 2117/2388 [1:18:19<10:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▊ | 2118/2388 [1:18:21<10:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▊ | 2119/2388 [1:18:23<09:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2120/2388 [1:18:25<09:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.3528, 'learning_rate': 2.2445561139028475e-05, 'epoch': 2.66}\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2120/2388 [1:18:25<09:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2121/2388 [1:18:27<09:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2122/2388 [1:18:30<09:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2123/2388 [1:18:32<09:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2124/2388 [1:18:34<09:47,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2125/2388 [1:18:36<09:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2126/2388 [1:18:39<09:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2127/2388 [1:18:41<09:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2128/2388 [1:18:43<09:38,  2.23s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2129/2388 [1:18:45<09:36,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2130/2388 [1:18:47<09:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2901, 'learning_rate': 2.1608040201005025e-05, 'epoch': 2.68}\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2130/2388 [1:18:47<09:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2131/2388 [1:18:50<09:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2132/2388 [1:18:52<09:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2133/2388 [1:18:54<09:27,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2134/2388 [1:18:56<09:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2135/2388 [1:18:59<09:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2136/2388 [1:19:01<09:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m89%|████████▉ | 2137/2388 [1:19:03<09:18,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2138/2388 [1:19:05<09:16,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2139/2388 [1:19:07<09:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2140/2388 [1:19:10<09:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2191, 'learning_rate': 2.0770519262981574e-05, 'epoch': 2.69}\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2140/2388 [1:19:10<09:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2141/2388 [1:19:12<09:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2142/2388 [1:19:14<09:07,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2143/2388 [1:19:16<09:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2144/2388 [1:19:19<09:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2145/2388 [1:19:21<09:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2146/2388 [1:19:23<08:58,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2147/2388 [1:19:25<08:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2148/2388 [1:19:27<08:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|████████▉ | 2149/2388 [1:19:30<08:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2150/2388 [1:19:32<08:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.3594, 'learning_rate': 1.9932998324958124e-05, 'epoch': 2.7}\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2150/2388 [1:19:32<08:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2151/2388 [1:19:34<08:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2152/2388 [1:19:36<08:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2153/2388 [1:19:39<08:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2154/2388 [1:19:41<08:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2155/2388 [1:19:43<08:38,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2156/2388 [1:19:45<08:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2157/2388 [1:19:48<08:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2158/2388 [1:19:50<08:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2159/2388 [1:19:52<08:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2160/2388 [1:19:54<08:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2865, 'learning_rate': 1.9095477386934673e-05, 'epoch': 2.71}\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2160/2388 [1:19:54<08:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m90%|█████████ | 2161/2388 [1:19:56<08:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2162/2388 [1:19:59<08:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2163/2388 [1:20:01<08:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2164/2388 [1:20:03<08:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2165/2388 [1:20:05<08:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2166/2388 [1:20:07<08:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2167/2388 [1:20:10<08:10,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2168/2388 [1:20:12<08:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2169/2388 [1:20:14<08:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2170/2388 [1:20:16<08:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.4022, 'learning_rate': 1.8257956448911223e-05, 'epoch': 2.73}\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2170/2388 [1:20:16<08:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2171/2388 [1:20:19<08:01,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2172/2388 [1:20:21<07:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2173/2388 [1:20:23<07:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2174/2388 [1:20:25<07:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2175/2388 [1:20:27<07:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2176/2388 [1:20:30<07:50,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2177/2388 [1:20:32<07:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2178/2388 [1:20:34<07:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████ | 2179/2388 [1:20:36<07:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████▏| 2180/2388 [1:20:39<07:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2008, 'learning_rate': 1.7420435510887772e-05, 'epoch': 2.74}\u001b[0m\n",
      "\u001b[34m91%|█████████▏| 2180/2388 [1:20:39<07:41,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████▏| 2181/2388 [1:20:41<07:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████▏| 2182/2388 [1:20:43<07:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████▏| 2183/2388 [1:20:45<07:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████▏| 2184/2388 [1:20:47<07:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m91%|█████████▏| 2185/2388 [1:20:50<07:30,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2186/2388 [1:20:52<07:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2187/2388 [1:20:54<07:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2188/2388 [1:20:56<07:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2189/2388 [1:20:59<07:21,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2190/2388 [1:21:01<07:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.1812, 'learning_rate': 1.6582914572864322e-05, 'epoch': 2.75}\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2190/2388 [1:21:01<07:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2191/2388 [1:21:03<07:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2192/2388 [1:21:05<07:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2193/2388 [1:21:07<07:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2194/2388 [1:21:10<07:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2195/2388 [1:21:12<07:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2196/2388 [1:21:14<07:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2197/2388 [1:21:16<07:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2198/2388 [1:21:19<07:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2199/2388 [1:21:21<06:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2200/2388 [1:21:23<06:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.1643, 'learning_rate': 1.574539363484087e-05, 'epoch': 2.76}\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2200/2388 [1:21:23<06:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2201/2388 [1:21:25<06:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2202/2388 [1:21:27<06:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2203/2388 [1:21:30<06:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2204/2388 [1:21:32<06:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2205/2388 [1:21:34<06:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2206/2388 [1:21:36<06:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2207/2388 [1:21:39<06:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m92%|█████████▏| 2208/2388 [1:21:41<06:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2209/2388 [1:21:43<06:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2210/2388 [1:21:45<06:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.3018, 'learning_rate': 1.490787269681742e-05, 'epoch': 2.78}\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2210/2388 [1:21:45<06:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2211/2388 [1:21:47<06:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2212/2388 [1:21:50<06:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2213/2388 [1:21:52<06:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2214/2388 [1:21:54<06:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2215/2388 [1:21:56<06:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2216/2388 [1:21:59<06:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2217/2388 [1:22:01<06:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2218/2388 [1:22:03<06:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2219/2388 [1:22:05<06:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2220/2388 [1:22:07<06:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2596, 'learning_rate': 1.407035175879397e-05, 'epoch': 2.79}\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2220/2388 [1:22:07<06:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2221/2388 [1:22:10<06:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2222/2388 [1:22:12<06:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2223/2388 [1:22:14<06:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2224/2388 [1:22:16<06:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2225/2388 [1:22:19<06:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2226/2388 [1:22:21<05:59,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2227/2388 [1:22:23<05:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2228/2388 [1:22:25<05:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2229/2388 [1:22:27<05:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2230/2388 [1:22:30<05:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.3561, 'learning_rate': 1.323283082077052e-05, 'epoch': 2.8}\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2230/2388 [1:22:30<05:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2231/2388 [1:22:32<05:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m93%|█████████▎| 2232/2388 [1:22:34<05:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▎| 2233/2388 [1:22:36<05:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▎| 2234/2388 [1:22:39<05:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▎| 2235/2388 [1:22:41<05:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▎| 2236/2388 [1:22:43<05:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▎| 2237/2388 [1:22:45<05:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▎| 2238/2388 [1:22:47<05:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2239/2388 [1:22:50<05:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2240/2388 [1:22:52<05:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.3535, 'learning_rate': 1.239530988274707e-05, 'epoch': 2.81}\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2240/2388 [1:22:52<05:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2241/2388 [1:22:54<05:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2242/2388 [1:22:56<05:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2243/2388 [1:22:59<05:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2244/2388 [1:23:01<05:19,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2245/2388 [1:23:03<05:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2246/2388 [1:23:05<05:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2247/2388 [1:23:07<05:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2248/2388 [1:23:10<05:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2249/2388 [1:23:12<05:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2250/2388 [1:23:14<05:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2682, 'learning_rate': 1.1557788944723619e-05, 'epoch': 2.83}\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2250/2388 [1:23:14<05:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2251/2388 [1:23:16<05:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2252/2388 [1:23:19<05:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2253/2388 [1:23:21<05:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2254/2388 [1:23:23<04:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2255/2388 [1:23:25<04:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m94%|█████████▍| 2256/2388 [1:23:27<04:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2257/2388 [1:23:30<04:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2258/2388 [1:23:32<04:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2259/2388 [1:23:34<04:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2260/2388 [1:23:36<04:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2296, 'learning_rate': 1.0720268006700168e-05, 'epoch': 2.84}\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2260/2388 [1:23:36<04:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2261/2388 [1:23:39<04:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2262/2388 [1:23:41<04:39,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2263/2388 [1:23:43<04:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2264/2388 [1:23:45<04:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2265/2388 [1:23:47<04:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2266/2388 [1:23:50<04:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2267/2388 [1:23:52<04:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▍| 2268/2388 [1:23:54<04:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2269/2388 [1:23:56<04:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2270/2388 [1:23:59<04:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.3224, 'learning_rate': 9.882747068676718e-06, 'epoch': 2.85}\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2270/2388 [1:23:59<04:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2271/2388 [1:24:01<04:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2272/2388 [1:24:03<04:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2273/2388 [1:24:05<04:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2274/2388 [1:24:07<04:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2275/2388 [1:24:10<04:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2276/2388 [1:24:12<04:09,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2277/2388 [1:24:14<04:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2278/2388 [1:24:16<04:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2279/2388 [1:24:19<04:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2280/2388 [1:24:21<04:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2001, 'learning_rate': 9.045226130653267e-06, 'epoch': 2.86}\u001b[0m\n",
      "\u001b[34m95%|█████████▌| 2280/2388 [1:24:21<04:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2281/2388 [1:24:23<03:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2282/2388 [1:24:25<03:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2283/2388 [1:24:27<03:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2284/2388 [1:24:30<03:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2285/2388 [1:24:32<03:49,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2286/2388 [1:24:34<03:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2287/2388 [1:24:36<03:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2288/2388 [1:24:39<03:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2289/2388 [1:24:41<03:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2290/2388 [1:24:43<03:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2891, 'learning_rate': 8.207705192629817e-06, 'epoch': 2.88}\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2290/2388 [1:24:43<03:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2291/2388 [1:24:45<03:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2292/2388 [1:24:48<03:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2293/2388 [1:24:50<03:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2294/2388 [1:24:52<03:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2295/2388 [1:24:54<03:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2296/2388 [1:24:56<03:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2297/2388 [1:24:59<03:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▌| 2298/2388 [1:25:01<03:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▋| 2299/2388 [1:25:03<03:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▋| 2300/2388 [1:25:05<03:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.1487, 'learning_rate': 7.370184254606366e-06, 'epoch': 2.89}\u001b[0m\n",
      "\u001b[34m96%|█████████▋| 2300/2388 [1:25:05<03:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▋| 2301/2388 [1:25:08<03:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▋| 2302/2388 [1:25:10<03:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▋| 2303/2388 [1:25:12<03:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m96%|█████████▋| 2304/2388 [1:25:14<03:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2305/2388 [1:25:16<03:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2306/2388 [1:25:19<03:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2307/2388 [1:25:21<03:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2308/2388 [1:25:23<02:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2309/2388 [1:25:25<02:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2310/2388 [1:25:28<02:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.3348, 'learning_rate': 6.532663316582915e-06, 'epoch': 2.9}\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2310/2388 [1:25:28<02:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2311/2388 [1:25:30<02:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2312/2388 [1:25:32<02:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2313/2388 [1:25:34<02:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2314/2388 [1:25:36<02:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2315/2388 [1:25:39<02:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2316/2388 [1:25:41<02:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2317/2388 [1:25:43<02:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2318/2388 [1:25:45<02:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2319/2388 [1:25:48<02:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2320/2388 [1:25:50<02:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2821, 'learning_rate': 5.6951423785594645e-06, 'epoch': 2.91}\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2320/2388 [1:25:50<02:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2321/2388 [1:25:52<02:29,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2322/2388 [1:25:54<02:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2323/2388 [1:25:56<02:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2324/2388 [1:25:59<02:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2325/2388 [1:26:01<02:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2326/2388 [1:26:03<02:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2327/2388 [1:26:05<02:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m97%|█████████▋| 2328/2388 [1:26:08<02:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2329/2388 [1:26:10<02:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2330/2388 [1:26:12<02:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.0814, 'learning_rate': 4.857621440536014e-06, 'epoch': 2.93}\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2330/2388 [1:26:12<02:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2331/2388 [1:26:14<02:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2332/2388 [1:26:16<02:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2333/2388 [1:26:19<02:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2334/2388 [1:26:21<02:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2335/2388 [1:26:23<01:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2336/2388 [1:26:25<01:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2337/2388 [1:26:28<01:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2338/2388 [1:26:30<01:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2339/2388 [1:26:32<01:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2340/2388 [1:26:34<01:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.0996, 'learning_rate': 4.020100502512563e-06, 'epoch': 2.94}\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2340/2388 [1:26:34<01:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2341/2388 [1:26:36<01:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2342/2388 [1:26:39<01:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2343/2388 [1:26:41<01:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2344/2388 [1:26:43<01:37,  2.23s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2345/2388 [1:26:45<01:35,  2.23s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2346/2388 [1:26:48<01:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2347/2388 [1:26:50<01:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2348/2388 [1:26:52<01:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2349/2388 [1:26:54<01:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2350/2388 [1:26:57<01:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2997, 'learning_rate': 3.1825795644891126e-06, 'epoch': 2.95}\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2350/2388 [1:26:57<01:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2351/2388 [1:26:59<01:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m98%|█████████▊| 2352/2388 [1:27:01<01:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▊| 2353/2388 [1:27:03<01:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▊| 2354/2388 [1:27:05<01:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▊| 2355/2388 [1:27:08<01:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▊| 2356/2388 [1:27:10<01:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▊| 2357/2388 [1:27:12<01:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▊| 2358/2388 [1:27:14<01:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2359/2388 [1:27:17<01:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2360/2388 [1:27:19<01:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2416, 'learning_rate': 2.3450586264656617e-06, 'epoch': 2.96}\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2360/2388 [1:27:19<01:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2361/2388 [1:27:21<01:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2362/2388 [1:27:23<00:57,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2363/2388 [1:27:25<00:55,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2364/2388 [1:27:28<00:53,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2365/2388 [1:27:30<00:51,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2366/2388 [1:27:32<00:48,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2367/2388 [1:27:34<00:46,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2368/2388 [1:27:37<00:44,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2369/2388 [1:27:39<00:42,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2370/2388 [1:27:41<00:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.2702, 'learning_rate': 1.507537688442211e-06, 'epoch': 2.98}\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2370/2388 [1:27:41<00:40,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2371/2388 [1:27:43<00:37,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2372/2388 [1:27:45<00:35,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2373/2388 [1:27:48<00:33,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2374/2388 [1:27:50<00:31,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2375/2388 [1:27:52<00:28,  2.22s/it]\u001b[0m\n",
      "\u001b[34m99%|█████████▉| 2376/2388 [1:27:54<00:26,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2377/2388 [1:27:57<00:24,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2378/2388 [1:27:59<00:22,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2379/2388 [1:28:01<00:20,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2380/2388 [1:28:03<00:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m{'loss': 4.1113, 'learning_rate': 6.700167504187605e-07, 'epoch': 2.99}\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2380/2388 [1:28:03<00:17,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2381/2388 [1:28:05<00:15,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2382/2388 [1:28:08<00:13,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2383/2388 [1:28:10<00:11,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2384/2388 [1:28:12<00:08,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2385/2388 [1:28:14<00:06,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2386/2388 [1:28:17<00:04,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|█████████▉| 2387/2388 [1:28:19<00:02,  2.22s/it]\u001b[0m\n",
      "\u001b[34m100%|██████████| 2388/2388 [1:28:20<00:00,  1.92s/it]\u001b[0m\n",
      "\u001b[34m{'train_runtime': 5300.5119, 'train_samples_per_second': 0.9, 'train_steps_per_second': 0.451, 'train_loss': 5.464830675716176, 'epoch': 3.0}\u001b[0m\n",
      "\u001b[34m100%|██████████| 2388/2388 [1:28:20<00:00,  1.92s/it]\u001b[0m\n",
      "\u001b[34m100%|██████████| 2388/2388 [1:28:20<00:00,  2.22s/it]\u001b[0m\n",
      "\u001b[34mDownloading tokenizer_config.json:   0%|          | 0.00/746 [00:00<?, ?B/s]\u001b[0m\n",
      "\u001b[34mDownloading tokenizer_config.json: 100%|██████████| 746/746 [00:00<00:00, 7.23MB/s]\u001b[0m\n",
      "\u001b[34mDownloading tokenizer.model:   0%|          | 0.00/500k [00:00<?, ?B/s]\u001b[0m\n",
      "\u001b[34mDownloading tokenizer.model: 100%|██████████| 500k/500k [00:00<00:00, 73.4MB/s]\u001b[0m\n",
      "\u001b[34mDownloading tokenizer.json:   0%|          | 0.00/1.84M [00:00<?, ?B/s]\u001b[0m\n",
      "\u001b[34mDownloading tokenizer.json: 100%|██████████| 1.84M/1.84M [00:00<00:00, 41.5MB/s]\u001b[0m\n",
      "\u001b[34mDownloading (…)cial_tokens_map.json:   0%|          | 0.00/435 [00:00<?, ?B/s]\u001b[0m\n",
      "\u001b[34mDownloading (…)cial_tokens_map.json: 100%|██████████| 435/435 [00:00<00:00, 5.90MB/s]\u001b[0m\n",
      "\u001b[34m2024-01-01 21:37:32,636 sagemaker-training-toolkit INFO     Waiting for the process to finish and give a return code.\u001b[0m\n",
      "\u001b[34m2024-01-01 21:37:32,636 sagemaker-training-toolkit INFO     Done waiting for a return code. Received 0 from exiting process.\u001b[0m\n",
      "\u001b[34m2024-01-01 21:37:32,636 sagemaker-training-toolkit INFO     Reporting training SUCCESS\u001b[0m\n",
      "\n",
      "2024-01-01 21:37:39 Uploading - Uploading generated training model\n",
      "2024-01-01 21:38:55 Completed - Training job completed\n",
      "Training seconds: 5821\n",
      "Billable seconds: 5821\n"
     ]
    }
   ],
   "source": [
    "# define a data input dictonary with our uploaded s3 uris\n",
    "data = {'training': training_input_path}\n",
    "\n",
    "# starting the train job with our uploaded datasets as input\n",
    "huggingface_estimator.fit(data, wait=True)"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "vscode": {
   "interpreter": {
    "hash": "2d58e898dde0263bc564c6968b04150abacfd33eed9b19aaa8e45c040360e146"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}