In [1]:
!git clone -b dev https://github.com/utensil/Alpaca-CoT.git code
Cloning into 'code'... remote: Enumerating objects: 771, done. remote: Counting objects: 100% (112/112), done. remote: Compressing objects: 100% (64/64), done. remote: Total 771 (delta 64), reused 82 (delta 48), pack-reused 659 Receiving objects: 100% (771/771), 127.13 MiB | 21.25 MiB/s, done. Resolving deltas: 100% (413/413), done.
In [2]:
!python -m pip install bitsandbytes
!python -m pip install datasets
!python -m pip install git+https://github.com/huggingface/transformers.git
!python -m pip install peft
!python -m pip install sentencepiece
!python -m pip install gradio
Requirement already satisfied: bitsandbytes in /usr/local/lib/python3.10/dist-packages (0.38.1) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip is available: 23.0.1 -> 23.1 [notice] To update, run: python -m pip install --upgrade pip Requirement already satisfied: datasets in /usr/local/lib/python3.10/dist-packages (2.11.0) Requirement already satisfied: huggingface-hub<1.0.0,>=0.11.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.13.3) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0) Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (1.5.3) Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.18.0) Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.24.2) Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from datasets) (0.70.14) Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (23.0) Requirement already satisfied: dill<0.3.7,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.3.6) Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.28.2) Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets) (3.2.0) Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (2023.3.0) Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.8.4) Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.65.0) Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (11.0.0) Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (3.0.1) Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (22.2.0) Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.2) Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.8.2) Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1) Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.4) Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.3) Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0.0,>=0.11.0->datasets) (3.10.4) Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0.0,>=0.11.0->datasets) (4.5.0) Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests>=2.19.0->datasets) (2019.11.28) Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests>=2.19.0->datasets) (2.8) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (1.26.15) Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2022.7.1) Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2) Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.1->pandas->datasets) (1.14.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip is available: 23.0.1 -> 23.1 [notice] To update, run: python -m pip install --upgrade pip Collecting git+https://github.com/huggingface/transformers.git Cloning https://github.com/huggingface/transformers.git to /tmp/pip-req-build-aybwnf6s Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers.git /tmp/pip-req-build-aybwnf6s Resolved https://github.com/huggingface/transformers.git to commit 84a6570e7bce91ba7d18c0782186241c5f1fde75 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.29.0.dev0) (2023.3.23) Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.29.0.dev0) (3.10.4) Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.29.0.dev0) (0.13.2) Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.29.0.dev0) (1.24.2) Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.29.0.dev0) (4.65.0) Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.29.0.dev0) (0.13.3) Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.29.0.dev0) (2.28.2) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.29.0.dev0) (6.0) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.29.0.dev0) (23.0) Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers==4.29.0.dev0) (4.5.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.29.0.dev0) (1.26.15) Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->transformers==4.29.0.dev0) (2.8) Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->transformers==4.29.0.dev0) (2019.11.28) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.29.0.dev0) (3.0.1) Building wheels for collected packages: transformers Building wheel for transformers (pyproject.toml) ... done Created wheel for transformers: filename=transformers-4.29.0.dev0-py3-none-any.whl size=6934831 sha256=a89150c4cb678a65adce0abee01082308edfc9159aa7c9846beee88fbc44fad1 Stored in directory: /tmp/pip-ephem-wheel-cache-2qbv5vto/wheels/e7/9c/5b/e1a9c8007c343041e61cc484433d512ea9274272e3fcbe7c16 Successfully built transformers Installing collected packages: transformers Attempting uninstall: transformers Found existing installation: transformers 4.28.1 Uninstalling transformers-4.28.1: Successfully uninstalled transformers-4.28.1 Successfully installed transformers-4.29.0.dev0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip is available: 23.0.1 -> 23.1 [notice] To update, run: python -m pip install --upgrade pip Requirement already satisfied: peft in /usr/local/lib/python3.10/dist-packages (0.3.0.dev0) Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (from peft) (0.18.0) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from peft) (23.0) Requirement already satisfied: torch>=1.13.0 in /usr/local/lib/python3.10/dist-packages (from peft) (1.13.1+cu116) Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from peft) (6.0) Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (from peft) (4.29.0.dev0) Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from peft) (1.24.2) Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from peft) (5.9.4) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.13.0->peft) (4.5.0) Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (0.13.3) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (2023.3.23) Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (2.28.2) Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (3.10.4) Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (0.13.2) Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (4.65.0) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->peft) (3.0.1) Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->transformers->peft) (2.8) Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->transformers->peft) (2019.11.28) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->peft) (1.26.15) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip is available: 23.0.1 -> 23.1 [notice] To update, run: python -m pip install --upgrade pip Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (0.1.97) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip is available: 23.0.1 -> 23.1 [notice] To update, run: python -m pip install --upgrade pip Requirement already satisfied: gradio in /usr/local/lib/python3.10/dist-packages (3.25.0) Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from gradio) (1.24.2) Requirement already satisfied: gradio-client>=0.0.8 in /usr/local/lib/python3.10/dist-packages (from gradio) (0.1.3) Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from gradio) (1.5.3) Requirement already satisfied: fastapi in /usr/local/lib/python3.10/dist-packages (from gradio) (0.95.0) Requirement already satisfied: python-multipart in /usr/local/lib/python3.10/dist-packages (from gradio) (0.0.6) Requirement already satisfied: websockets>=10.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (10.4) Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from gradio) (0.23.3) Requirement already satisfied: aiofiles in /usr/local/lib/python3.10/dist-packages (from gradio) (22.1.0) Requirement already satisfied: mdit-py-plugins<=0.3.3 in /usr/local/lib/python3.10/dist-packages (from gradio) (0.3.3) Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from gradio) (6.0) Requirement already satisfied: uvicorn in /usr/local/lib/python3.10/dist-packages (from gradio) (0.21.1) Requirement already satisfied: huggingface-hub>=0.13.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (0.13.3) Requirement already satisfied: markupsafe in /usr/local/lib/python3.10/dist-packages (from gradio) (2.1.2) Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.2.0) Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from gradio) (3.8.4) Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from gradio) (3.7.1) Requirement already satisfied: pillow in /usr/local/lib/python3.10/dist-packages (from gradio) (9.5.0) Requirement already satisfied: pydantic in /usr/local/lib/python3.10/dist-packages (from gradio) (1.10.7) Requirement already satisfied: orjson in /usr/local/lib/python3.10/dist-packages (from gradio) (3.8.8) Requirement already satisfied: ffmpy in /usr/local/lib/python3.10/dist-packages (from gradio) (0.3.0) Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.1.2) Requirement already satisfied: pydub in /usr/local/lib/python3.10/dist-packages (from gradio) (0.25.1) Requirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (4.2.2) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from gradio) (4.5.0) Requirement already satisfied: semantic-version in /usr/local/lib/python3.10/dist-packages (from gradio) (2.10.0) Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from gradio) (2.28.2) Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio) (4.17.3) Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio) (0.4) Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio) (0.12.0) Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gradio-client>=0.0.8->gradio) (23.0) Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client>=0.0.8->gradio) (2023.3.0) Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.0->gradio) (4.65.0) Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.0->gradio) (3.10.4) Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio) (0.1.2) Requirement already satisfied: linkify-it-py<3,>=1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio) (2.0.0) Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->gradio) (2.8.2) Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->gradio) (2022.7.1) Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio) (4.0.2) Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio) (1.3.3) Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio) (3.0.1) Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio) (1.3.1) Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio) (6.0.4) Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio) (1.8.2) Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio) (22.2.0) Requirement already satisfied: starlette<0.27.0,>=0.26.1 in /usr/local/lib/python3.10/dist-packages (from fastapi->gradio) (0.26.1) Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio) (1.3.0) Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in /usr/local/lib/python3.10/dist-packages (from httpx->gradio) (1.5.0) Requirement already satisfied: httpcore<0.17.0,>=0.15.0 in /usr/local/lib/python3.10/dist-packages (from httpx->gradio) (0.16.3) Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from httpx->gradio) (2019.11.28) Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio) (3.0.9) Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio) (4.39.2) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio) (0.11.0) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio) (1.4.4) Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio) (1.0.7) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->gradio) (1.26.15) Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->gradio) (2.8) Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn->gradio) (8.1.3) Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.10/dist-packages (from uvicorn->gradio) (0.14.0) Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/lib/python3.10/dist-packages (from httpcore<0.17.0,>=0.15.0->httpx->gradio) (3.6.2) Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio) (0.19.3) Requirement already satisfied: uc-micro-py in /usr/local/lib/python3.10/dist-packages (from linkify-it-py<3,>=1->markdown-it-py[linkify]>=2.0.0->gradio) (1.0.1) Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.1->pandas->gradio) (1.14.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [notice] A new release of pip is available: 23.0.1 -> 23.1 [notice] To update, run: python -m pip install --upgrade pip
In [3]:
# https://github.com/ymcui/Chinese-LLaMA-Alpaca/issues/160
!python -m pip install --force-reinstall peft
Collecting peft
Using cached peft-0.2.0-py3-none-any.whl (40 kB)
Collecting accelerate
Using cached accelerate-0.18.0-py3-none-any.whl (215 kB)
Collecting numpy>=1.17
Using cached numpy-1.24.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Collecting psutil
Downloading psutil-5.9.5-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (282 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 282.1/282.1 kB 6.5 MB/s eta 0:00:0000:01
Collecting pyyaml
Using cached PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
Collecting transformers
Using cached transformers-4.28.1-py3-none-any.whl (7.0 MB)
Collecting packaging>=20.0
Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting torch>=1.13.0
Downloading torch-2.0.0-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 619.9/619.9 MB 10.1 MB/s eta 0:00:0000:0100:01
Collecting jinja2
Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting nvidia-nccl-cu11==2.14.3
Downloading nvidia_nccl_cu11-2.14.3-py3-none-manylinux1_x86_64.whl (177.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 177.1/177.1 MB 24.6 MB/s eta 0:00:0000:0100:01
Collecting nvidia-cusparse-cu11==11.7.4.91
Downloading nvidia_cusparse_cu11-11.7.4.91-py3-none-manylinux1_x86_64.whl (173.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 173.2/173.2 MB 26.2 MB/s eta 0:00:0000:0100:01
Collecting typing-extensions
Using cached typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Collecting sympy
Downloading sympy-1.11.1-py3-none-any.whl (6.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 70.5 MB/s eta 0:00:00a 0:00:01
Collecting nvidia-cuda-nvrtc-cu11==11.7.99
Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.0/21.0 MB 59.4 MB/s eta 0:00:0000:0100:01
Collecting nvidia-cudnn-cu11==8.5.0.96
Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 557.1/557.1 MB 11.4 MB/s eta 0:00:0000:0100:01
Collecting nvidia-cufft-cu11==10.9.0.58
Downloading nvidia_cufft_cu11-10.9.0.58-py3-none-manylinux1_x86_64.whl (168.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 168.4/168.4 MB 21.5 MB/s eta 0:00:0000:0100:01
Collecting networkx
Downloading networkx-3.1-py3-none-any.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 48.4 MB/s eta 0:00:00a 0:00:01
Collecting triton==2.0.0
Downloading triton-2.0.0-1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.3/63.3 MB 32.5 MB/s eta 0:00:0000:0100:01
Collecting nvidia-cublas-cu11==11.10.3.66
Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.1/317.1 MB 16.6 MB/s eta 0:00:0000:0100:01
Collecting nvidia-cuda-runtime-cu11==11.7.99
Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 849.3/849.3 kB 92.5 MB/s eta 0:00:00
Collecting nvidia-nvtx-cu11==11.7.91
Downloading nvidia_nvtx_cu11-11.7.91-py3-none-manylinux1_x86_64.whl (98 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.6/98.6 kB 28.4 MB/s eta 0:00:00
Collecting nvidia-curand-cu11==10.2.10.91
Downloading nvidia_curand_cu11-10.2.10.91-py3-none-manylinux1_x86_64.whl (54.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.6/54.6 MB 37.4 MB/s eta 0:00:0000:0100:01
Collecting nvidia-cusolver-cu11==11.4.0.1
Downloading nvidia_cusolver_cu11-11.4.0.1-2-py3-none-manylinux1_x86_64.whl (102.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.6/102.6 MB 32.1 MB/s eta 0:00:0000:0100:01
Collecting nvidia-cuda-cupti-cu11==11.7.101
Downloading nvidia_cuda_cupti_cu11-11.7.101-py3-none-manylinux1_x86_64.whl (11.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.8/11.8 MB 60.8 MB/s eta 0:00:0000:0100:01
Collecting filelock
Downloading filelock-3.12.0-py3-none-any.whl (10 kB)
Collecting wheel
Using cached wheel-0.40.0-py3-none-any.whl (64 kB)
Collecting setuptools
Using cached setuptools-67.6.1-py3-none-any.whl (1.1 MB)
Collecting cmake
Using cached cmake-3.26.3-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (24.0 MB)
Collecting lit
Downloading lit-16.0.1.tar.gz (137 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 137.9/137.9 kB 31.7 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting regex!=2019.12.17
Using cached regex-2023.3.23-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (769 kB)
Collecting tqdm>=4.27
Using cached tqdm-4.65.0-py3-none-any.whl (77 kB)
Collecting huggingface-hub<1.0,>=0.11.0
Downloading huggingface_hub-0.13.4-py3-none-any.whl (200 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 200.1/200.1 kB 40.3 MB/s eta 0:00:00
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1
Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 64.3 MB/s eta 0:00:0000:0100:01
Collecting requests
Using cached requests-2.28.2-py3-none-any.whl (62 kB)
Collecting MarkupSafe>=2.0
Using cached MarkupSafe-2.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting idna<4,>=2.5
Downloading idna-3.4-py3-none-any.whl (61 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 14.9 MB/s eta 0:00:00
Collecting charset-normalizer<4,>=2
Downloading charset_normalizer-3.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.3/199.3 kB 49.2 MB/s eta 0:00:00
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.15-py2.py3-none-any.whl (140 kB)
Collecting certifi>=2017.4.17
Downloading certifi-2022.12.7-py3-none-any.whl (155 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 33.4 MB/s eta 0:00:00
Collecting mpmath>=0.19
Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 80.4 MB/s eta 0:00:00
Building wheels for collected packages: lit
Building wheel for lit (setup.py) ... done
Created wheel for lit: filename=lit-16.0.1-py3-none-any.whl size=88174 sha256=fa76e1d6d49feb9def971129dbd241ffc17e1943318206f2bd59f0110a2ceabd
Stored in directory: /root/.cache/pip/wheels/33/c2/b7/b91592eb5167b4293827b97fb02d52686773dded13f7fb1054
Successfully built lit
Installing collected packages: tokenizers, mpmath, lit, cmake, wheel, urllib3, typing-extensions, tqdm, sympy, setuptools, regex, pyyaml, psutil, packaging, nvidia-nccl-cu11, nvidia-cufft-cu11, nvidia-cuda-nvrtc-cu11, numpy, networkx, MarkupSafe, idna, filelock, charset-normalizer, certifi, requests, nvidia-nvtx-cu11, nvidia-cusparse-cu11, nvidia-curand-cu11, nvidia-cuda-runtime-cu11, nvidia-cuda-cupti-cu11, nvidia-cublas-cu11, jinja2, nvidia-cusolver-cu11, nvidia-cudnn-cu11, huggingface-hub, transformers, triton, torch, accelerate, peft
Attempting uninstall: tokenizers
Found existing installation: tokenizers 0.13.2
Uninstalling tokenizers-0.13.2:
Successfully uninstalled tokenizers-0.13.2
Attempting uninstall: wheel
Found existing installation: wheel 0.38.4
Uninstalling wheel-0.38.4:
Successfully uninstalled wheel-0.38.4
Attempting uninstall: urllib3
Found existing installation: urllib3 1.26.15
Uninstalling urllib3-1.26.15:
Successfully uninstalled urllib3-1.26.15
Attempting uninstall: typing-extensions
Found existing installation: typing_extensions 4.5.0
Uninstalling typing_extensions-4.5.0:
Successfully uninstalled typing_extensions-4.5.0
Attempting uninstall: tqdm
Found existing installation: tqdm 4.65.0
Uninstalling tqdm-4.65.0:
Successfully uninstalled tqdm-4.65.0
Attempting uninstall: setuptools
Found existing installation: setuptools 67.3.2
Uninstalling setuptools-67.3.2:
Successfully uninstalled setuptools-67.3.2
Attempting uninstall: regex
Found existing installation: regex 2023.3.23
Uninstalling regex-2023.3.23:
Successfully uninstalled regex-2023.3.23
Attempting uninstall: pyyaml
Found existing installation: PyYAML 6.0
Uninstalling PyYAML-6.0:
Successfully uninstalled PyYAML-6.0
Attempting uninstall: psutil
Found existing installation: psutil 5.9.4
Uninstalling psutil-5.9.4:
Successfully uninstalled psutil-5.9.4
Attempting uninstall: packaging
Found existing installation: packaging 23.0
Uninstalling packaging-23.0:
Successfully uninstalled packaging-23.0
Attempting uninstall: numpy
Found existing installation: numpy 1.24.2
Uninstalling numpy-1.24.2:
Successfully uninstalled numpy-1.24.2
Attempting uninstall: MarkupSafe
Found existing installation: MarkupSafe 2.1.2
Uninstalling MarkupSafe-2.1.2:
Successfully uninstalled MarkupSafe-2.1.2
Attempting uninstall: idna
Found existing installation: idna 2.8
Uninstalling idna-2.8:
Successfully uninstalled idna-2.8
Attempting uninstall: filelock
Found existing installation: filelock 3.10.4
Uninstalling filelock-3.10.4:
Successfully uninstalled filelock-3.10.4
Attempting uninstall: charset-normalizer
Found existing installation: charset-normalizer 3.0.1
Uninstalling charset-normalizer-3.0.1:
Successfully uninstalled charset-normalizer-3.0.1
Attempting uninstall: certifi
Found existing installation: certifi 2019.11.28
Uninstalling certifi-2019.11.28:
Successfully uninstalled certifi-2019.11.28
Attempting uninstall: requests
Found existing installation: requests 2.28.2
Uninstalling requests-2.28.2:
Successfully uninstalled requests-2.28.2
Attempting uninstall: jinja2
Found existing installation: Jinja2 3.1.2
Uninstalling Jinja2-3.1.2:
Successfully uninstalled Jinja2-3.1.2
Attempting uninstall: huggingface-hub
Found existing installation: huggingface-hub 0.13.3
Uninstalling huggingface-hub-0.13.3:
Successfully uninstalled huggingface-hub-0.13.3
Attempting uninstall: transformers
Found existing installation: transformers 4.29.0.dev0
Uninstalling transformers-4.29.0.dev0:
Successfully uninstalled transformers-4.29.0.dev0
Attempting uninstall: torch
Found existing installation: torch 1.13.1+cu116
Uninstalling torch-1.13.1+cu116:
Successfully uninstalled torch-1.13.1+cu116
Attempting uninstall: accelerate
Found existing installation: accelerate 0.18.0
Uninstalling accelerate-0.18.0:
Successfully uninstalled accelerate-0.18.0
Attempting uninstall: peft
Found existing installation: peft 0.3.0.dev0
Uninstalling peft-0.3.0.dev0:
Successfully uninstalled peft-0.3.0.dev0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchvision 0.14.1+cu116 requires torch==1.13.1, but you have torch 2.0.0 which is incompatible.
torchaudio 0.13.1+cu116 requires torch==1.13.1, but you have torch 2.0.0 which is incompatible.
Successfully installed MarkupSafe-2.1.2 accelerate-0.18.0 certifi-2022.12.7 charset-normalizer-3.1.0 cmake-3.26.3 filelock-3.12.0 huggingface-hub-0.13.4 idna-3.4 jinja2-3.1.2 lit-16.0.1 mpmath-1.3.0 networkx-3.1 numpy-1.24.2 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 packaging-23.1 peft-0.2.0 psutil-5.9.5 pyyaml-6.0 regex-2023.3.23 requests-2.28.2 setuptools-67.6.1 sympy-1.11.1 tokenizers-0.13.3 torch-2.0.0 tqdm-4.65.0 transformers-4.28.1 triton-2.0.0 typing-extensions-4.5.0 urllib3-1.26.15 wheel-0.40.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.0.1 -> 23.1
[notice] To update, run: python -m pip install --upgrade pip
In [4]:
!pip install wandb
Collecting wandb
Downloading wandb-0.14.2-py3-none-any.whl (2.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 26.3 MB/s eta 0:00:0000:010:01
Collecting appdirs>=1.4.3
Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Requirement already satisfied: PyYAML in /usr/local/lib/python3.10/dist-packages (from wandb) (6.0)
Collecting docker-pycreds>=0.4.0
Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from wandb) (67.6.1)
Collecting sentry-sdk>=1.0.0
Downloading sentry_sdk-1.19.1-py2.py3-none-any.whl (199 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.2/199.2 kB 40.0 MB/s eta 0:00:00
Collecting setproctitle
Downloading setproctitle-1.3.2-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)
Collecting pathtools
Downloading pathtools-0.1.2.tar.gz (11 kB)
Preparing metadata (setup.py) ... done
Collecting protobuf!=4.21.0,<5,>=3.19.0
Downloading protobuf-4.22.3-cp37-abi3-manylinux2014_x86_64.whl (302 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 302.4/302.4 kB 55.2 MB/s eta 0:00:00
Requirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb) (5.9.5)
Collecting GitPython!=3.1.29,>=1.0.0
Downloading GitPython-3.1.31-py3-none-any.whl (184 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 184.3/184.3 kB 41.2 MB/s eta 0:00:00
Requirement already satisfied: requests<3,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb) (2.28.2)
Requirement already satisfied: Click!=8.0.0,>=7.0 in /usr/local/lib/python3.10/dist-packages (from wandb) (8.1.3)
Requirement already satisfied: six>=1.4.0 in /usr/lib/python3/dist-packages (from docker-pycreds>=0.4.0->wandb) (1.14.0)
Collecting gitdb<5,>=4.0.1
Downloading gitdb-4.0.10-py3-none-any.whl (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.7/62.7 kB 15.8 MB/s eta 0:00:00
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb) (2022.12.7)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb) (3.1.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.0.0->wandb) (1.26.15)
Collecting smmap<6,>=3.0.1
Downloading smmap-5.0.0-py3-none-any.whl (24 kB)
Building wheels for collected packages: pathtools
Building wheel for pathtools (setup.py) ... done
Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8791 sha256=479f8088a9cc5c56259f171255f9768e7b97f3aca5cde00485cbb144df78367a
Stored in directory: /root/.cache/pip/wheels/e7/f3/22/152153d6eb222ee7a56ff8617d80ee5207207a8c00a7aab794
Successfully built pathtools
Installing collected packages: pathtools, appdirs, smmap, setproctitle, sentry-sdk, protobuf, docker-pycreds, gitdb, GitPython, wandb
Successfully installed GitPython-3.1.31 appdirs-1.4.4 docker-pycreds-0.4.0 gitdb-4.0.10 pathtools-0.1.2 protobuf-4.22.3 sentry-sdk-1.19.1 setproctitle-1.3.2 smmap-5.0.0 wandb-0.14.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.0.1 -> 23.1
[notice] To update, run: python3 -m pip install --upgrade pip
In [6]:
!wandb login
wandb: Appending key for api.wandb.ai to your netrc file: /root/.netrc
In [7]:
%env WANDB_PROJECT=Alpaca-CoT
env: WANDB_PROJECT=Alpaca-CoT
In [10]:
# download data git repository
!GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/datasets/QingyiSi/Alpaca-CoT data
Cloning into 'data'... remote: Enumerating objects: 2167, done. remote: Counting objects: 100% (2167/2167), done. remote: Compressing objects: 100% (2111/2111), done. remote: Total 2167 (delta 159), reused 1613 (delta 0), pack-reused 0 Receiving objects: 100% (2167/2167), 21.86 MiB | 5.89 MiB/s, done. Resolving deltas: 100% (159/159), done.
In [11]:
# pull specify data and move it，change <alpaca/> to your preference
!cd /workspace/data && git config core.sparsecheckout true && git config lfs.fetchinclude alpaca/
!cd /workspace/data && git lfs pull
!cp /workspace/data/alpaca/*.json /workspace/code/data/
!ls -lh /workspace/code/data/
total 44Mng LFS objects: 100% (2/2), 46 MB | 18 MB/s -rw-r--r-- 1 root root 22M Apr 19 04:01 alpaca_data.json -rw-r--r-- 1 root root 22M Apr 19 04:01 alpaca_data_cleaned.json drwxr-xr-x 2 root root 228 Apr 19 03:57 formatted_cot_data drwxr-xr-x 2 root root 260 Apr 19 03:57 origin_cot_data
In [12]:
!python3 /workspace/llm-playground/helper/upload.py
Working directory changed to: /workspace/llm-playground/helper/.. /workspace/llm-playground/storage is already a clone of https://huggingface.co/datasets/utensil/storage. Make sure you pull the latest changes with `repo.git_pull()`. Nothing to upload, exiting...
In [13]:
!cp -r /workspace/llm-playground/storage/saved_models /workspace/code/
In [14]:
!ls /workspace/code/saved_models/llama-7b-hf_alpaca
adapter_config.json adapter_model.bin checkpoint-351 checkpoint-390
In [16]:
# instruction finetuning
!cd /workspace/code && python uniform_finetune.py --model_type llama --model_name_or_path decapoda-research/llama-7b-hf --data alpaca --lora_target_modules q_proj v_proj --per_gpu_train_batch_size 4 --learning_rate 3e-4 --epochs 1 --report_to wandb --resume_from_checkpoint /workspace/code/saved_models/llama-7b-hf_alpaca/checkpoint-390
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib'), PosixPath('/usr/local/nvidia/lib64')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('module'), PosixPath('//matplotlib_inline.backend_inline')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('PygmalionAI/pygmalion-6b')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('noninteractiveSHELL=/bin/bash')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 116
CUDA SETUP: Loading binary /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so...
Namespace(size=None, data='alpaca', local_rank=-1, model_type='llama', model_name_or_path='decapoda-research/llama-7b-hf', per_gpu_train_batch_size=4, gradient_accumulation_steps=32, epochs=1, learning_rate=0.0003, cutoff_len=512, lora_r=8, lora_alpha=16, lora_dropout=0.05, val_set_size=2000, lora_target_modules=['q_proj', 'v_proj'], resume_from_checkpoint='/workspace/code/saved_models/llama-7b-hf_alpaca/checkpoint-390')
Downloading and preparing dataset json/default to /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e...
Downloading data files: 100%|███████████████████| 1/1 [00:00<00:00, 6307.22it/s]
Extracting data files: 100%|████████████████████| 1/1 [00:00<00:00, 1757.14it/s]
Dataset json downloaded and prepared to /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e. Subsequent calls will reuse this data.
100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 621.56it/s]
DatasetDict({
train: Dataset({
features: ['instruction', 'input', 'output'],
num_rows: 51942
})
})
Overriding torch_dtype=None with `torch_dtype=torch.float16` due to requirements of `bitsandbytes` to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning.
Downloading (…)lve/main/config.json: 100%|██████| 427/427 [00:00<00:00, 437kB/s]
Downloading (…)model.bin.index.json: 100%|█| 25.5k/25.5k [00:00<00:00, 20.9MB/s]
Downloading shards: 0%| | 0/33 [00:00<?, ?it/s]
Downloading (…)l-00001-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00001-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 81.0MB/s]
Downloading (…)l-00001-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 79.4MB/s]
Downloading (…)l-00001-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 81.6MB/s]
Downloading (…)l-00001-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:04, 79.1MB/s]
Downloading (…)l-00001-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:04, 80.2MB/s]
Downloading (…)l-00001-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:04, 78.0MB/s]
Downloading (…)l-00001-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:04, 77.2MB/s]
Downloading (…)l-00001-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:04, 78.1MB/s]
Downloading (…)l-00001-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:04, 76.7MB/s]
Downloading (…)l-00001-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 77.8MB/s]
Downloading (…)l-00001-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 79.3MB/s]
Downloading (…)l-00001-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:03, 78.4MB/s]
Downloading (…)l-00001-of-00033.bin: 34%|█ | 136M/405M [00:01<00:03, 79.2MB/s]
Downloading (…)l-00001-of-00033.bin: 36%|█ | 147M/405M [00:01<00:03, 79.6MB/s]
Downloading (…)l-00001-of-00033.bin: 39%|█▏ | 157M/405M [00:01<00:03, 79.9MB/s]
Downloading (…)l-00001-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:02, 79.5MB/s]
Downloading (…)l-00001-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:02, 79.0MB/s]
Downloading (…)l-00001-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 77.3MB/s]
Downloading (…)l-00001-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 78.1MB/s]
Downloading (…)l-00001-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 77.2MB/s]
Downloading (…)l-00001-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:02, 75.3MB/s]
Downloading (…)l-00001-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:02, 74.0MB/s]
Downloading (…)l-00001-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 75.0MB/s]
Downloading (…)l-00001-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:01, 76.8MB/s]
Downloading (…)l-00001-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:01, 73.4MB/s]
Downloading (…)l-00001-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 75.7MB/s]
Downloading (…)l-00001-of-00033.bin: 70%|██ | 283M/405M [00:03<00:01, 76.0MB/s]
Downloading (…)l-00001-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 77.3MB/s]
Downloading (…)l-00001-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 79.3MB/s]
Downloading (…)l-00001-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 78.1MB/s]
Downloading (…)l-00001-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:01, 77.9MB/s]
Downloading (…)l-00001-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 77.8MB/s]
Downloading (…)l-00001-of-00033.bin: 85%|██▌| 346M/405M [00:04<00:00, 78.8MB/s]
Downloading (…)l-00001-of-00033.bin: 88%|██▋| 357M/405M [00:04<00:00, 77.4MB/s]
Downloading (…)l-00001-of-00033.bin: 91%|██▋| 367M/405M [00:04<00:00, 76.3MB/s]
Downloading (…)l-00001-of-00033.bin: 93%|██▊| 377M/405M [00:04<00:00, 76.7MB/s]
Downloading (…)l-00001-of-00033.bin: 96%|██▉| 388M/405M [00:04<00:00, 77.3MB/s]
Downloading (…)l-00001-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 77.4MB/s]
Downloading shards: 3%|▋ | 1/33 [00:05<02:54, 5.46s/it]
Downloading (…)l-00002-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00002-of-00033.bin: 3%| | 10.5M/405M [00:00<00:05, 77.2MB/s]
Downloading (…)l-00002-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 77.1MB/s]
Downloading (…)l-00002-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:05, 71.7MB/s]
Downloading (…)l-00002-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:04, 74.2MB/s]
Downloading (…)l-00002-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:04, 76.8MB/s]
Downloading (…)l-00002-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:04, 79.1MB/s]
Downloading (…)l-00002-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:04, 80.4MB/s]
Downloading (…)l-00002-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:03, 80.4MB/s]
Downloading (…)l-00002-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:03, 81.5MB/s]
Downloading (…)l-00002-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 80.8MB/s]
Downloading (…)l-00002-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 81.1MB/s]
Downloading (…)l-00002-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:03, 81.9MB/s]
Downloading (…)l-00002-of-00033.bin: 34%|█ | 136M/405M [00:01<00:03, 78.7MB/s]
Downloading (…)l-00002-of-00033.bin: 36%|█ | 147M/405M [00:01<00:04, 63.7MB/s]
Downloading (…)l-00002-of-00033.bin: 39%|█▏ | 157M/405M [00:02<00:03, 65.5MB/s]
Downloading (…)l-00002-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 69.5MB/s]
Downloading (…)l-00002-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:03, 71.2MB/s]
Downloading (…)l-00002-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 73.9MB/s]
Downloading (…)l-00002-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 73.4MB/s]
Downloading (…)l-00002-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 75.0MB/s]
Downloading (…)l-00002-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:02, 75.2MB/s]
Downloading (…)l-00002-of-00033.bin: 57%|█▋ | 231M/405M [00:03<00:02, 77.5MB/s]
Downloading (…)l-00002-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 78.6MB/s]
Downloading (…)l-00002-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:01, 78.5MB/s]
Downloading (…)l-00002-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:01, 78.0MB/s]
Downloading (…)l-00002-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 78.0MB/s]
Downloading (…)l-00002-of-00033.bin: 70%|██ | 283M/405M [00:03<00:01, 78.3MB/s]
Downloading (…)l-00002-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 70.9MB/s]
Downloading (…)l-00002-of-00033.bin: 75%|██▎| 304M/405M [00:04<00:01, 71.5MB/s]
Downloading (…)l-00002-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 73.2MB/s]
Downloading (…)l-00002-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:01, 69.5MB/s]
Downloading (…)l-00002-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 72.3MB/s]
Downloading (…)l-00002-of-00033.bin: 85%|██▌| 346M/405M [00:04<00:00, 75.0MB/s]
Downloading (…)l-00002-of-00033.bin: 88%|██▋| 357M/405M [00:04<00:00, 76.6MB/s]
Downloading (…)l-00002-of-00033.bin: 91%|██▋| 367M/405M [00:04<00:00, 78.9MB/s]
Downloading (…)l-00002-of-00033.bin: 93%|██▊| 377M/405M [00:04<00:00, 79.7MB/s]
Downloading (…)l-00002-of-00033.bin: 96%|██▉| 388M/405M [00:05<00:00, 77.3MB/s]
Downloading (…)l-00002-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 75.7MB/s]
Downloading shards: 6%|█▍ | 2/33 [00:11<02:52, 5.56s/it]
Downloading (…)l-00003-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00003-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 96.0MB/s]
Downloading (…)l-00003-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 86.6MB/s]
Downloading (…)l-00003-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 92.5MB/s]
Downloading (…)l-00003-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 95.4MB/s]
Downloading (…)l-00003-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 97.3MB/s]
Downloading (…)l-00003-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:03, 98.8MB/s]
Downloading (…)l-00003-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:03, 99.6MB/s]
Downloading (…)l-00003-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 99.8MB/s]
Downloading (…)l-00003-of-00033.bin: 23%|▍ | 94.4M/405M [00:00<00:03, 99.3MB/s]
Downloading (…)l-00003-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 99.8MB/s]
Downloading (…)l-00003-of-00033.bin: 28%|█▏ | 115M/405M [00:01<00:02, 100MB/s]
Downloading (…)l-00003-of-00033.bin: 31%|█▏ | 126M/405M [00:01<00:02, 100MB/s]
Downloading (…)l-00003-of-00033.bin: 34%|█▎ | 136M/405M [00:01<00:02, 100MB/s]
Downloading (…)l-00003-of-00033.bin: 36%|█▍ | 147M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00003-of-00033.bin: 39%|█▌ | 157M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00003-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 99.6MB/s]
Downloading (…)l-00003-of-00033.bin: 44%|█▎ | 178M/405M [00:01<00:02, 99.3MB/s]
Downloading (…)l-00003-of-00033.bin: 47%|█▍ | 189M/405M [00:01<00:02, 99.5MB/s]
Downloading (…)l-00003-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 99.7MB/s]
Downloading (…)l-00003-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:01, 99.9MB/s]
Downloading (…)l-00003-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:01, 97.6MB/s]
Downloading (…)l-00003-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 99.1MB/s]
Downloading (…)l-00003-of-00033.bin: 60%|██▍ | 241M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00003-of-00033.bin: 62%|██▍ | 252M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00003-of-00033.bin: 65%|██▌ | 262M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00003-of-00033.bin: 67%|██ | 273M/405M [00:02<00:01, 99.9MB/s]
Downloading (…)l-00003-of-00033.bin: 70%|██▊ | 283M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00003-of-00033.bin: 73%|██▉ | 294M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00003-of-00033.bin: 75%|███ | 304M/405M [00:03<00:00, 101MB/s]
Downloading (…)l-00003-of-00033.bin: 78%|███ | 315M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00003-of-00033.bin: 80%|███▏| 325M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00003-of-00033.bin: 83%|███▎| 336M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00003-of-00033.bin: 85%|███▍| 346M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00003-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 99.7MB/s]
Downloading (…)l-00003-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 99.9MB/s]
Downloading (…)l-00003-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 98.6MB/s]
Downloading (…)l-00003-of-00033.bin: 96%|██▉| 388M/405M [00:03<00:00, 99.2MB/s]
Downloading (…)l-00003-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 99.3MB/s]
Downloading shards: 9%|██▏ | 3/33 [00:15<02:29, 4.99s/it]
Downloading (…)l-00004-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00004-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 79.6MB/s]
Downloading (…)l-00004-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 76.9MB/s]
Downloading (…)l-00004-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 78.9MB/s]
Downloading (…)l-00004-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:04, 78.4MB/s]
Downloading (…)l-00004-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:04, 79.1MB/s]
Downloading (…)l-00004-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:04, 78.9MB/s]
Downloading (…)l-00004-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:04, 79.7MB/s]
Downloading (…)l-00004-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:04, 78.3MB/s]
Downloading (…)l-00004-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:03, 79.1MB/s]
Downloading (…)l-00004-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 77.9MB/s]
Downloading (…)l-00004-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 78.9MB/s]
Downloading (…)l-00004-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:03, 78.1MB/s]
Downloading (…)l-00004-of-00033.bin: 34%|█ | 136M/405M [00:01<00:03, 77.7MB/s]
Downloading (…)l-00004-of-00033.bin: 36%|█ | 147M/405M [00:01<00:03, 75.9MB/s]
Downloading (…)l-00004-of-00033.bin: 39%|█▏ | 157M/405M [00:02<00:03, 77.3MB/s]
Downloading (…)l-00004-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 77.3MB/s]
Downloading (…)l-00004-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:02, 79.2MB/s]
Downloading (…)l-00004-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 79.8MB/s]
Downloading (…)l-00004-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 79.6MB/s]
Downloading (…)l-00004-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 79.7MB/s]
Downloading (…)l-00004-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:02, 80.5MB/s]
Downloading (…)l-00004-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:02, 75.5MB/s]
Downloading (…)l-00004-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 76.6MB/s]
Downloading (…)l-00004-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:01, 78.0MB/s]
Downloading (…)l-00004-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:01, 77.8MB/s]
Downloading (…)l-00004-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 77.4MB/s]
Downloading (…)l-00004-of-00033.bin: 70%|██ | 283M/405M [00:03<00:01, 76.4MB/s]
Downloading (…)l-00004-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 75.5MB/s]
Downloading (…)l-00004-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 76.6MB/s]
Downloading (…)l-00004-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 76.8MB/s]
Downloading (…)l-00004-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:01, 78.2MB/s]
Downloading (…)l-00004-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 78.6MB/s]
Downloading (…)l-00004-of-00033.bin: 85%|██▌| 346M/405M [00:04<00:00, 78.1MB/s]
Downloading (…)l-00004-of-00033.bin: 88%|██▋| 357M/405M [00:04<00:00, 77.8MB/s]
Downloading (…)l-00004-of-00033.bin: 91%|██▋| 367M/405M [00:04<00:00, 70.5MB/s]
Downloading (…)l-00004-of-00033.bin: 93%|██▊| 377M/405M [00:04<00:00, 73.7MB/s]
Downloading (…)l-00004-of-00033.bin: 96%|██▉| 388M/405M [00:05<00:00, 68.9MB/s]
Downloading (…)l-00004-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 74.9MB/s]
Downloading shards: 12%|██▉ | 4/33 [00:21<02:31, 5.24s/it]
Downloading (…)l-00005-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00005-of-00033.bin: 3%| | 10.5M/405M [00:00<00:05, 75.1MB/s]
Downloading (…)l-00005-of-00033.bin: 5%| | 21.0M/405M [00:00<00:05, 76.0MB/s]
Downloading (…)l-00005-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:07, 47.2MB/s]
Downloading (…)l-00005-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:06, 55.6MB/s]
Downloading (…)l-00005-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:05, 60.6MB/s]
Downloading (…)l-00005-of-00033.bin: 16%|▎ | 62.9M/405M [00:01<00:05, 64.6MB/s]
Downloading (…)l-00005-of-00033.bin: 18%|▎ | 73.4M/405M [00:01<00:05, 65.5MB/s]
Downloading (…)l-00005-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:04, 68.4MB/s]
Downloading (…)l-00005-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:04, 69.4MB/s]
Downloading (…)l-00005-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:04, 71.6MB/s]
Downloading (…)l-00005-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 73.4MB/s]
Downloading (…)l-00005-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:03, 70.0MB/s]
Downloading (…)l-00005-of-00033.bin: 34%|█ | 136M/405M [00:02<00:03, 70.9MB/s]
Downloading (…)l-00005-of-00033.bin: 36%|█ | 147M/405M [00:02<00:03, 73.8MB/s]
Downloading (…)l-00005-of-00033.bin: 39%|█▏ | 157M/405M [00:02<00:03, 75.6MB/s]
Downloading (…)l-00005-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 75.7MB/s]
Downloading (…)l-00005-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:03, 74.7MB/s]
Downloading (…)l-00005-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 75.0MB/s]
Downloading (…)l-00005-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:03, 67.9MB/s]
Downloading (…)l-00005-of-00033.bin: 52%|█▌ | 210M/405M [00:03<00:02, 70.6MB/s]
Downloading (…)l-00005-of-00033.bin: 54%|█▋ | 220M/405M [00:03<00:02, 73.3MB/s]
Downloading (…)l-00005-of-00033.bin: 57%|█▋ | 231M/405M [00:03<00:02, 69.2MB/s]
Downloading (…)l-00005-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 62.9MB/s]
Downloading (…)l-00005-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:02, 66.7MB/s]
Downloading (…)l-00005-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:02, 67.6MB/s]
Downloading (…)l-00005-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 68.3MB/s]
Downloading (…)l-00005-of-00033.bin: 70%|██ | 283M/405M [00:04<00:01, 66.6MB/s]
Downloading (…)l-00005-of-00033.bin: 73%|██▏| 294M/405M [00:04<00:01, 59.4MB/s]
Downloading (…)l-00005-of-00033.bin: 75%|██▎| 304M/405M [00:04<00:01, 61.2MB/s]
Downloading (…)l-00005-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 65.5MB/s]
Downloading (…)l-00005-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:01, 68.7MB/s]
Downloading (…)l-00005-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 71.3MB/s]
Downloading (…)l-00005-of-00033.bin: 85%|██▌| 346M/405M [00:05<00:00, 71.0MB/s]
Downloading (…)l-00005-of-00033.bin: 88%|██▋| 357M/405M [00:05<00:00, 73.5MB/s]
Downloading (…)l-00005-of-00033.bin: 91%|██▋| 367M/405M [00:05<00:00, 75.5MB/s]
Downloading (…)l-00005-of-00033.bin: 93%|██▊| 377M/405M [00:05<00:00, 77.2MB/s]
Downloading (…)l-00005-of-00033.bin: 96%|██▉| 388M/405M [00:05<00:00, 77.1MB/s]
Downloading (…)l-00005-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 69.3MB/s]
Downloading shards: 15%|███▋ | 5/33 [00:27<02:35, 5.56s/it]
Downloading (…)l-00006-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00006-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 96.4MB/s]
Downloading (…)l-00006-of-00033.bin: 5%| | 21.0M/405M [00:00<00:03, 99.7MB/s]
Downloading (…)l-00006-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 101MB/s]
Downloading (…)l-00006-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 99.0MB/s]
Downloading (…)l-00006-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 99.8MB/s]
Downloading (…)l-00006-of-00033.bin: 16%|▍ | 62.9M/405M [00:00<00:03, 101MB/s]
Downloading (…)l-00006-of-00033.bin: 18%|▌ | 73.4M/405M [00:00<00:03, 101MB/s]
Downloading (…)l-00006-of-00033.bin: 21%|▌ | 83.9M/405M [00:00<00:03, 101MB/s]
Downloading (…)l-00006-of-00033.bin: 23%|▋ | 94.4M/405M [00:00<00:03, 101MB/s]
Downloading (…)l-00006-of-00033.bin: 26%|█ | 105M/405M [00:01<00:02, 100MB/s]
Downloading (…)l-00006-of-00033.bin: 28%|█▏ | 115M/405M [00:01<00:02, 100MB/s]
Downloading (…)l-00006-of-00033.bin: 31%|█▏ | 126M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00006-of-00033.bin: 34%|█▎ | 136M/405M [00:01<00:02, 102MB/s]
Downloading (…)l-00006-of-00033.bin: 36%|█▍ | 147M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00006-of-00033.bin: 39%|█▌ | 157M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00006-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 99.6MB/s]
Downloading (…)l-00006-of-00033.bin: 44%|█▊ | 178M/405M [00:01<00:02, 100MB/s]
Downloading (…)l-00006-of-00033.bin: 47%|█▍ | 189M/405M [00:01<00:02, 94.6MB/s]
Downloading (…)l-00006-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 96.4MB/s]
Downloading (…)l-00006-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 97.0MB/s]
Downloading (…)l-00006-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:01, 98.1MB/s]
Downloading (…)l-00006-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 99.0MB/s]
Downloading (…)l-00006-of-00033.bin: 60%|█▊ | 241M/405M [00:02<00:01, 98.0MB/s]
Downloading (…)l-00006-of-00033.bin: 62%|█▊ | 252M/405M [00:02<00:01, 98.4MB/s]
Downloading (…)l-00006-of-00033.bin: 65%|█▉ | 262M/405M [00:02<00:01, 99.6MB/s]
Downloading (…)l-00006-of-00033.bin: 67%|██▋ | 273M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00006-of-00033.bin: 70%|██▊ | 283M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00006-of-00033.bin: 73%|██▏| 294M/405M [00:02<00:01, 99.9MB/s]
Downloading (…)l-00006-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 95.5MB/s]
Downloading (…)l-00006-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 97.2MB/s]
Downloading (…)l-00006-of-00033.bin: 80%|██▍| 325M/405M [00:03<00:00, 98.4MB/s]
Downloading (…)l-00006-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 99.9MB/s]
Downloading (…)l-00006-of-00033.bin: 85%|███▍| 346M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00006-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 99.2MB/s]
Downloading (…)l-00006-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 98.5MB/s]
Downloading (…)l-00006-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 98.4MB/s]
Downloading (…)l-00006-of-00033.bin: 96%|██▉| 388M/405M [00:03<00:00, 99.0MB/s]
Downloading (…)l-00006-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 99.2MB/s]
Downloading shards: 18%|████▎ | 6/33 [00:31<02:18, 5.14s/it]
Downloading (…)l-00007-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00007-of-00033.bin: 3%| | 10.5M/405M [00:00<00:06, 62.4MB/s]
Downloading (…)l-00007-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 81.2MB/s]
Downloading (…)l-00007-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 89.3MB/s]
Downloading (…)l-00007-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 94.2MB/s]
Downloading (…)l-00007-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 96.4MB/s]
Downloading (…)l-00007-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:03, 93.1MB/s]
Downloading (…)l-00007-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:03, 95.5MB/s]
Downloading (…)l-00007-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 94.9MB/s]
Downloading (…)l-00007-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:03, 95.9MB/s]
Downloading (…)l-00007-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 97.8MB/s]
Downloading (…)l-00007-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:02, 98.8MB/s]
Downloading (…)l-00007-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:02, 99.8MB/s]
Downloading (…)l-00007-of-00033.bin: 34%|█▎ | 136M/405M [00:01<00:02, 100MB/s]
Downloading (…)l-00007-of-00033.bin: 36%|█▍ | 147M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00007-of-00033.bin: 39%|█▌ | 157M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00007-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 95.6MB/s]
Downloading (…)l-00007-of-00033.bin: 44%|█▎ | 178M/405M [00:01<00:02, 97.6MB/s]
Downloading (…)l-00007-of-00033.bin: 47%|█▍ | 189M/405M [00:01<00:02, 97.6MB/s]
Downloading (…)l-00007-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 98.8MB/s]
Downloading (…)l-00007-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:01, 99.9MB/s]
Downloading (…)l-00007-of-00033.bin: 54%|██▏ | 220M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00007-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 97.9MB/s]
Downloading (…)l-00007-of-00033.bin: 60%|█▊ | 241M/405M [00:02<00:01, 98.0MB/s]
Downloading (…)l-00007-of-00033.bin: 62%|█▊ | 252M/405M [00:02<00:01, 97.9MB/s]
Downloading (…)l-00007-of-00033.bin: 65%|█▉ | 262M/405M [00:02<00:01, 98.4MB/s]
Downloading (…)l-00007-of-00033.bin: 67%|██ | 273M/405M [00:02<00:01, 98.2MB/s]
Downloading (…)l-00007-of-00033.bin: 70%|██ | 283M/405M [00:02<00:01, 96.6MB/s]
Downloading (…)l-00007-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 98.0MB/s]
Downloading (…)l-00007-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 99.0MB/s]
Downloading (…)l-00007-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 99.7MB/s]
Downloading (…)l-00007-of-00033.bin: 80%|██▍| 325M/405M [00:03<00:00, 99.9MB/s]
Downloading (…)l-00007-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 99.4MB/s]
Downloading (…)l-00007-of-00033.bin: 85%|███▍| 346M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00007-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 95.8MB/s]
Downloading (…)l-00007-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 97.4MB/s]
Downloading (…)l-00007-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 97.4MB/s]
Downloading (…)l-00007-of-00033.bin: 96%|██▉| 388M/405M [00:04<00:00, 95.2MB/s]
Downloading (…)l-00007-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 96.9MB/s]
Downloading shards: 21%|█████ | 7/33 [00:35<02:07, 4.89s/it]
Downloading (…)l-00008-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00008-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 79.0MB/s]
Downloading (…)l-00008-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 77.7MB/s]
Downloading (…)l-00008-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 78.1MB/s]
Downloading (…)l-00008-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:04, 79.7MB/s]
Downloading (…)l-00008-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:04, 77.0MB/s]
Downloading (…)l-00008-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:04, 77.7MB/s]
Downloading (…)l-00008-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:04, 79.4MB/s]
Downloading (…)l-00008-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:04, 80.0MB/s]
Downloading (…)l-00008-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:04, 77.5MB/s]
Downloading (…)l-00008-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 76.9MB/s]
Downloading (…)l-00008-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 79.0MB/s]
Downloading (…)l-00008-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:03, 79.5MB/s]
Downloading (…)l-00008-of-00033.bin: 34%|█ | 136M/405M [00:01<00:03, 77.4MB/s]
Downloading (…)l-00008-of-00033.bin: 36%|█ | 147M/405M [00:01<00:03, 77.6MB/s]
Downloading (…)l-00008-of-00033.bin: 39%|█▏ | 157M/405M [00:02<00:03, 78.1MB/s]
Downloading (…)l-00008-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 79.0MB/s]
Downloading (…)l-00008-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:02, 79.4MB/s]
Downloading (…)l-00008-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 76.9MB/s]
Downloading (…)l-00008-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 69.4MB/s]
Downloading (…)l-00008-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 71.9MB/s]
Downloading (…)l-00008-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:02, 74.3MB/s]
Downloading (…)l-00008-of-00033.bin: 57%|█▋ | 231M/405M [00:03<00:02, 73.1MB/s]
Downloading (…)l-00008-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 62.1MB/s]
Downloading (…)l-00008-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:02, 66.8MB/s]
Downloading (…)l-00008-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:02, 68.0MB/s]
Downloading (…)l-00008-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 71.5MB/s]
Downloading (…)l-00008-of-00033.bin: 70%|██ | 283M/405M [00:03<00:01, 72.4MB/s]
Downloading (…)l-00008-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 74.6MB/s]
Downloading (…)l-00008-of-00033.bin: 75%|██▎| 304M/405M [00:04<00:01, 75.2MB/s]
Downloading (…)l-00008-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 77.3MB/s]
Downloading (…)l-00008-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:01, 77.4MB/s]
Downloading (…)l-00008-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 78.0MB/s]
Downloading (…)l-00008-of-00033.bin: 85%|██▌| 346M/405M [00:04<00:00, 78.2MB/s]
Downloading (…)l-00008-of-00033.bin: 88%|██▋| 357M/405M [00:04<00:00, 71.1MB/s]
Downloading (…)l-00008-of-00033.bin: 91%|██▋| 367M/405M [00:04<00:00, 74.5MB/s]
Downloading (…)l-00008-of-00033.bin: 93%|██▊| 377M/405M [00:05<00:00, 76.4MB/s]
Downloading (…)l-00008-of-00033.bin: 96%|██▉| 388M/405M [00:05<00:00, 77.3MB/s]
Downloading (…)l-00008-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 72.8MB/s]
Downloading shards: 24%|█████▊ | 8/33 [00:41<02:10, 5.20s/it]
Downloading (…)l-00009-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00009-of-00033.bin: 3%| | 10.5M/405M [00:01<00:44, 8.90MB/s]
Downloading (…)l-00009-of-00033.bin: 5%| | 21.0M/405M [00:01<00:33, 11.5MB/s]
Downloading (…)l-00009-of-00033.bin: 8%|▏ | 31.5M/405M [00:02<00:27, 13.4MB/s]
Downloading (…)l-00009-of-00033.bin: 10%|▏ | 41.9M/405M [00:03<00:27, 13.2MB/s]
Downloading (…)l-00009-of-00033.bin: 13%|▎ | 52.4M/405M [00:04<00:30, 11.7MB/s]
Downloading (…)l-00009-of-00033.bin: 16%|▎ | 62.9M/405M [00:05<00:31, 10.9MB/s]
Downloading (…)l-00009-of-00033.bin: 18%|▎ | 73.4M/405M [00:06<00:31, 10.5MB/s]
Downloading (…)l-00009-of-00033.bin: 21%|▍ | 83.9M/405M [00:07<00:31, 10.2MB/s]
Downloading (…)l-00009-of-00033.bin: 23%|▍ | 94.4M/405M [00:08<00:30, 10.3MB/s]
Downloading (…)l-00009-of-00033.bin: 26%|▊ | 105M/405M [00:09<00:28, 10.4MB/s]
Downloading (…)l-00009-of-00033.bin: 28%|▊ | 115M/405M [00:10<00:28, 10.2MB/s]
Downloading (…)l-00009-of-00033.bin: 31%|▉ | 126M/405M [00:11<00:27, 10.3MB/s]
Downloading (…)l-00009-of-00033.bin: 34%|█ | 136M/405M [00:12<00:25, 10.4MB/s]
Downloading (…)l-00009-of-00033.bin: 36%|█ | 147M/405M [00:13<00:24, 10.4MB/s]
Downloading (…)l-00009-of-00033.bin: 39%|█▏ | 157M/405M [00:14<00:23, 10.5MB/s]
Downloading (…)l-00009-of-00033.bin: 41%|█▏ | 168M/405M [00:15<00:22, 10.6MB/s]
Downloading (…)l-00009-of-00033.bin: 44%|█▎ | 178M/405M [00:16<00:20, 10.9MB/s]
Downloading (…)l-00009-of-00033.bin: 47%|█▍ | 189M/405M [00:17<00:19, 11.4MB/s]
Downloading (…)l-00009-of-00033.bin: 49%|█▍ | 199M/405M [00:18<00:17, 11.8MB/s]
Downloading (…)l-00009-of-00033.bin: 52%|█▌ | 210M/405M [00:18<00:15, 12.5MB/s]
Downloading (…)l-00009-of-00033.bin: 54%|█▋ | 220M/405M [00:19<00:13, 13.3MB/s]
Downloading (…)l-00009-of-00033.bin: 57%|█▋ | 231M/405M [00:20<00:12, 14.1MB/s]
Downloading (…)l-00009-of-00033.bin: 60%|█▊ | 241M/405M [00:20<00:10, 15.0MB/s]
Downloading (…)l-00009-of-00033.bin: 62%|█▊ | 252M/405M [00:21<00:09, 16.1MB/s]
Downloading (…)l-00009-of-00033.bin: 65%|█▉ | 262M/405M [00:21<00:08, 17.0MB/s]
Downloading (…)l-00009-of-00033.bin: 67%|██ | 273M/405M [00:22<00:07, 18.3MB/s]
Downloading (…)l-00009-of-00033.bin: 70%|██ | 283M/405M [00:22<00:06, 19.5MB/s]
Downloading (…)l-00009-of-00033.bin: 73%|██▏| 294M/405M [00:23<00:05, 20.6MB/s]
Downloading (…)l-00009-of-00033.bin: 75%|██▎| 304M/405M [00:23<00:04, 21.9MB/s]
Downloading (…)l-00009-of-00033.bin: 78%|██▎| 315M/405M [00:24<00:03, 23.1MB/s]
Downloading (…)l-00009-of-00033.bin: 80%|██▍| 325M/405M [00:24<00:03, 24.6MB/s]
Downloading (…)l-00009-of-00033.bin: 83%|██▍| 336M/405M [00:24<00:02, 25.8MB/s]
Downloading (…)l-00009-of-00033.bin: 85%|██▌| 346M/405M [00:25<00:02, 28.1MB/s]
Downloading (…)l-00009-of-00033.bin: 88%|██▋| 357M/405M [00:25<00:01, 28.8MB/s]
Downloading (…)l-00009-of-00033.bin: 91%|██▋| 367M/405M [00:25<00:01, 30.6MB/s]
Downloading (…)l-00009-of-00033.bin: 93%|██▊| 377M/405M [00:26<00:00, 31.0MB/s]
Downloading (…)l-00009-of-00033.bin: 96%|██▉| 388M/405M [00:26<00:00, 32.4MB/s]
Downloading (…)l-00009-of-00033.bin: 98%|██▉| 398M/405M [00:26<00:00, 34.0MB/s]
Downloading (…)l-00009-of-00033.bin: 100%|███| 405M/405M [00:26<00:00, 15.1MB/s]
Downloading shards: 27%|██████▌ | 9/33 [01:09<04:52, 12.17s/it]
Downloading (…)l-00010-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00010-of-00033.bin: 3%| | 10.5M/405M [00:00<00:08, 45.0MB/s]
Downloading (…)l-00010-of-00033.bin: 5%| | 21.0M/405M [00:00<00:06, 59.4MB/s]
Downloading (…)l-00010-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:04, 83.5MB/s]
Downloading (…)l-00010-of-00033.bin: 16%|▍ | 62.9M/405M [00:00<00:03, 109MB/s]
Downloading (…)l-00010-of-00033.bin: 23%|▋ | 94.4M/405M [00:00<00:02, 147MB/s]
Downloading (…)l-00010-of-00033.bin: 31%|█▏ | 126M/405M [00:00<00:01, 179MB/s]
Downloading (…)l-00010-of-00033.bin: 39%|█▌ | 157M/405M [00:01<00:02, 107MB/s]
Downloading (…)l-00010-of-00033.bin: 44%|█▊ | 178M/405M [00:01<00:01, 121MB/s]
Downloading (…)l-00010-of-00033.bin: 52%|██ | 210M/405M [00:01<00:01, 150MB/s]
Downloading (…)l-00010-of-00033.bin: 60%|██▍ | 241M/405M [00:01<00:00, 174MB/s]
Downloading (…)l-00010-of-00033.bin: 67%|██▋ | 273M/405M [00:01<00:00, 196MB/s]
Downloading (…)l-00010-of-00033.bin: 75%|███ | 304M/405M [00:02<00:00, 214MB/s]
Downloading (…)l-00010-of-00033.bin: 83%|███▎| 336M/405M [00:02<00:00, 228MB/s]
Downloading (…)l-00010-of-00033.bin: 91%|███▋| 367M/405M [00:02<00:00, 232MB/s]
Downloading (…)l-00010-of-00033.bin: 100%|████| 405M/405M [00:02<00:00, 167MB/s]
Downloading shards: 30%|██████▉ | 10/33 [01:11<03:32, 9.23s/it]
Downloading (…)l-00011-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00011-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 97.3MB/s]
Downloading (…)l-00011-of-00033.bin: 5%| | 21.0M/405M [00:00<00:03, 97.7MB/s]
Downloading (…)l-00011-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 98.5MB/s]
Downloading (…)l-00011-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 97.3MB/s]
Downloading (…)l-00011-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 98.0MB/s]
Downloading (…)l-00011-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:03, 97.3MB/s]
Downloading (…)l-00011-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:03, 98.4MB/s]
Downloading (…)l-00011-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 98.3MB/s]
Downloading (…)l-00011-of-00033.bin: 23%|▍ | 94.4M/405M [00:00<00:03, 98.7MB/s]
Downloading (…)l-00011-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 99.2MB/s]
Downloading (…)l-00011-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 96.4MB/s]
Downloading (…)l-00011-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:02, 97.8MB/s]
Downloading (…)l-00011-of-00033.bin: 34%|█ | 136M/405M [00:01<00:02, 98.5MB/s]
Downloading (…)l-00011-of-00033.bin: 36%|█ | 147M/405M [00:01<00:02, 94.0MB/s]
Downloading (…)l-00011-of-00033.bin: 39%|█▏ | 157M/405M [00:01<00:02, 94.0MB/s]
Downloading (…)l-00011-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 94.4MB/s]
Downloading (…)l-00011-of-00033.bin: 44%|█▎ | 178M/405M [00:01<00:02, 95.7MB/s]
Downloading (…)l-00011-of-00033.bin: 47%|█▍ | 189M/405M [00:01<00:02, 93.8MB/s]
Downloading (…)l-00011-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 94.4MB/s]
Downloading (…)l-00011-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 95.9MB/s]
Downloading (…)l-00011-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:01, 97.1MB/s]
Downloading (…)l-00011-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 98.0MB/s]
Downloading (…)l-00011-of-00033.bin: 60%|█▊ | 241M/405M [00:02<00:01, 98.1MB/s]
Downloading (…)l-00011-of-00033.bin: 62%|█▊ | 252M/405M [00:02<00:01, 97.9MB/s]
Downloading (…)l-00011-of-00033.bin: 65%|█▉ | 262M/405M [00:02<00:01, 98.2MB/s]
Downloading (…)l-00011-of-00033.bin: 67%|██ | 273M/405M [00:02<00:01, 98.2MB/s]
Downloading (…)l-00011-of-00033.bin: 70%|██ | 283M/405M [00:02<00:01, 99.3MB/s]
Downloading (…)l-00011-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 98.5MB/s]
Downloading (…)l-00011-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 99.1MB/s]
Downloading (…)l-00011-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 99.4MB/s]
Downloading (…)l-00011-of-00033.bin: 80%|███▏| 325M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00011-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 99.4MB/s]
Downloading (…)l-00011-of-00033.bin: 85%|███▍| 346M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00011-of-00033.bin: 88%|███▌| 357M/405M [00:03<00:00, 101MB/s]
Downloading (…)l-00011-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 99.1MB/s]
Downloading (…)l-00011-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 99.8MB/s]
Downloading (…)l-00011-of-00033.bin: 96%|██▉| 388M/405M [00:03<00:00, 99.5MB/s]
Downloading (…)l-00011-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 97.6MB/s]
Downloading shards: 33%|███████▋ | 11/33 [01:16<02:50, 7.74s/it]
Downloading (…)l-00012-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00012-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 94.0MB/s]
Downloading (…)l-00012-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 94.0MB/s]
Downloading (…)l-00012-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 96.0MB/s]
Downloading (…)l-00012-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 96.3MB/s]
Downloading (…)l-00012-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 97.2MB/s]
Downloading (…)l-00012-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:03, 97.6MB/s]
Downloading (…)l-00012-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:03, 98.0MB/s]
Downloading (…)l-00012-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 97.2MB/s]
Downloading (…)l-00012-of-00033.bin: 23%|▍ | 94.4M/405M [00:00<00:03, 96.4MB/s]
Downloading (…)l-00012-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 91.2MB/s]
Downloading (…)l-00012-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 89.5MB/s]
Downloading (…)l-00012-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:03, 90.4MB/s]
Downloading (…)l-00012-of-00033.bin: 34%|█ | 136M/405M [00:01<00:02, 90.1MB/s]
Downloading (…)l-00012-of-00033.bin: 36%|█ | 147M/405M [00:01<00:02, 91.9MB/s]
Downloading (…)l-00012-of-00033.bin: 39%|█▏ | 157M/405M [00:01<00:02, 93.0MB/s]
Downloading (…)l-00012-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 94.3MB/s]
Downloading (…)l-00012-of-00033.bin: 44%|█▎ | 178M/405M [00:01<00:02, 93.8MB/s]
Downloading (…)l-00012-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 93.8MB/s]
Downloading (…)l-00012-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 92.9MB/s]
Downloading (…)l-00012-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 93.9MB/s]
Downloading (…)l-00012-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:01, 94.7MB/s]
Downloading (…)l-00012-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 94.2MB/s]
Downloading (…)l-00012-of-00033.bin: 60%|█▊ | 241M/405M [00:02<00:01, 93.3MB/s]
Downloading (…)l-00012-of-00033.bin: 62%|█▊ | 252M/405M [00:02<00:01, 94.6MB/s]
Downloading (…)l-00012-of-00033.bin: 65%|█▉ | 262M/405M [00:02<00:01, 95.6MB/s]
Downloading (…)l-00012-of-00033.bin: 67%|██ | 273M/405M [00:02<00:01, 93.9MB/s]
Downloading (…)l-00012-of-00033.bin: 70%|██ | 283M/405M [00:03<00:01, 95.1MB/s]
Downloading (…)l-00012-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 94.3MB/s]
Downloading (…)l-00012-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 95.1MB/s]
Downloading (…)l-00012-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 95.4MB/s]
Downloading (…)l-00012-of-00033.bin: 80%|██▍| 325M/405M [00:03<00:00, 95.7MB/s]
Downloading (…)l-00012-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 97.1MB/s]
Downloading (…)l-00012-of-00033.bin: 85%|██▌| 346M/405M [00:03<00:00, 97.9MB/s]
Downloading (…)l-00012-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 97.2MB/s]
Downloading (…)l-00012-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 97.8MB/s]
Downloading (…)l-00012-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 96.7MB/s]
Downloading (…)l-00012-of-00033.bin: 96%|██▉| 388M/405M [00:04<00:00, 96.4MB/s]
Downloading (…)l-00012-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 94.7MB/s]
Downloading shards: 36%|████████▎ | 12/33 [01:20<02:21, 6.74s/it]
Downloading (…)l-00013-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00013-of-00033.bin: 3%| | 10.5M/405M [00:00<00:31, 12.6MB/s]
Downloading (…)l-00013-of-00033.bin: 5%| | 21.0M/405M [00:01<00:16, 22.6MB/s]
Downloading (…)l-00013-of-00033.bin: 8%|▏ | 31.5M/405M [00:01<00:13, 28.1MB/s]
Downloading (…)l-00013-of-00033.bin: 10%|▏ | 41.9M/405M [00:01<00:11, 31.2MB/s]
Downloading (…)l-00013-of-00033.bin: 13%|▎ | 52.4M/405M [00:01<00:10, 33.3MB/s]
Downloading (…)l-00013-of-00033.bin: 16%|▎ | 62.9M/405M [00:02<00:08, 38.0MB/s]
Downloading (…)l-00013-of-00033.bin: 18%|▎ | 73.4M/405M [00:02<00:08, 38.1MB/s]
Downloading (…)l-00013-of-00033.bin: 21%|▍ | 83.9M/405M [00:02<00:07, 41.6MB/s]
Downloading (…)l-00013-of-00033.bin: 23%|▍ | 94.4M/405M [00:02<00:07, 40.6MB/s]
Downloading (…)l-00013-of-00033.bin: 26%|▊ | 105M/405M [00:03<00:06, 43.2MB/s]
Downloading (…)l-00013-of-00033.bin: 28%|▊ | 115M/405M [00:03<00:06, 42.1MB/s]
Downloading (…)l-00013-of-00033.bin: 31%|▉ | 126M/405M [00:03<00:06, 40.7MB/s]
Downloading (…)l-00013-of-00033.bin: 34%|█ | 136M/405M [00:03<00:06, 43.2MB/s]
Downloading (…)l-00013-of-00033.bin: 36%|█ | 147M/405M [00:04<00:06, 41.8MB/s]
Downloading (…)l-00013-of-00033.bin: 39%|█▏ | 157M/405M [00:04<00:05, 44.0MB/s]
Downloading (…)l-00013-of-00033.bin: 41%|█▏ | 168M/405M [00:04<00:05, 42.4MB/s]
Downloading (…)l-00013-of-00033.bin: 44%|█▎ | 178M/405M [00:04<00:05, 44.7MB/s]
Downloading (…)l-00013-of-00033.bin: 47%|█▍ | 189M/405M [00:04<00:05, 42.4MB/s]
Downloading (…)l-00013-of-00033.bin: 49%|█▍ | 199M/405M [00:05<00:05, 40.8MB/s]
Downloading (…)l-00013-of-00033.bin: 52%|█▌ | 210M/405M [00:05<00:04, 39.5MB/s]
Downloading (…)l-00013-of-00033.bin: 54%|█▋ | 220M/405M [00:05<00:04, 38.8MB/s]
Downloading (…)l-00013-of-00033.bin: 57%|█▋ | 231M/405M [00:06<00:04, 42.1MB/s]
Downloading (…)l-00013-of-00033.bin: 60%|█▊ | 241M/405M [00:06<00:04, 40.6MB/s]
Downloading (…)l-00013-of-00033.bin: 62%|█▊ | 252M/405M [00:06<00:03, 39.7MB/s]
Downloading (…)l-00013-of-00033.bin: 65%|█▉ | 262M/405M [00:06<00:03, 42.5MB/s]
Downloading (…)l-00013-of-00033.bin: 67%|██ | 273M/405M [00:07<00:03, 40.5MB/s]
Downloading (…)l-00013-of-00033.bin: 70%|██ | 283M/405M [00:07<00:03, 39.9MB/s]
Downloading (…)l-00013-of-00033.bin: 73%|██▏| 294M/405M [00:07<00:02, 43.0MB/s]
Downloading (…)l-00013-of-00033.bin: 75%|██▎| 304M/405M [00:07<00:02, 41.3MB/s]
Downloading (…)l-00013-of-00033.bin: 78%|██▎| 315M/405M [00:08<00:02, 44.3MB/s]
Downloading (…)l-00013-of-00033.bin: 80%|██▍| 325M/405M [00:08<00:01, 42.1MB/s]
Downloading (…)l-00013-of-00033.bin: 83%|██▍| 336M/405M [00:08<00:01, 40.5MB/s]
Downloading (…)l-00013-of-00033.bin: 85%|██▌| 346M/405M [00:08<00:01, 39.4MB/s]
Downloading (…)l-00013-of-00033.bin: 88%|██▋| 357M/405M [00:09<00:01, 42.0MB/s]
Downloading (…)l-00013-of-00033.bin: 91%|██▋| 367M/405M [00:09<00:00, 40.5MB/s]
Downloading (…)l-00013-of-00033.bin: 93%|██▊| 377M/405M [00:09<00:00, 39.5MB/s]
Downloading (…)l-00013-of-00033.bin: 96%|██▉| 388M/405M [00:09<00:00, 42.2MB/s]
Downloading (…)l-00013-of-00033.bin: 98%|██▉| 398M/405M [00:10<00:00, 41.3MB/s]
Downloading (…)l-00013-of-00033.bin: 100%|███| 405M/405M [00:10<00:00, 39.5MB/s]
Downloading shards: 39%|█████████ | 13/33 [01:31<02:40, 8.00s/it]
Downloading (…)l-00014-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00014-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 95.0MB/s]
Downloading (…)l-00014-of-00033.bin: 5%| | 21.0M/405M [00:00<00:03, 96.2MB/s]
Downloading (…)l-00014-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 97.3MB/s]
Downloading (…)l-00014-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 95.1MB/s]
Downloading (…)l-00014-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 95.7MB/s]
Downloading (…)l-00014-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:03, 94.2MB/s]
Downloading (…)l-00014-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:03, 94.1MB/s]
Downloading (…)l-00014-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 94.7MB/s]
Downloading (…)l-00014-of-00033.bin: 23%|▍ | 94.4M/405M [00:00<00:03, 96.4MB/s]
Downloading (…)l-00014-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 94.1MB/s]
Downloading (…)l-00014-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 95.6MB/s]
Downloading (…)l-00014-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:02, 96.9MB/s]
Downloading (…)l-00014-of-00033.bin: 34%|█ | 136M/405M [00:01<00:02, 97.3MB/s]
Downloading (…)l-00014-of-00033.bin: 36%|█ | 147M/405M [00:01<00:02, 97.7MB/s]
Downloading (…)l-00014-of-00033.bin: 39%|█▏ | 157M/405M [00:01<00:02, 96.9MB/s]
Downloading (…)l-00014-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 95.2MB/s]
Downloading (…)l-00014-of-00033.bin: 44%|█▎ | 178M/405M [00:01<00:02, 95.6MB/s]
Downloading (…)l-00014-of-00033.bin: 47%|█▍ | 189M/405M [00:01<00:02, 96.8MB/s]
Downloading (…)l-00014-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 94.3MB/s]
Downloading (…)l-00014-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 94.7MB/s]
Downloading (…)l-00014-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:01, 96.7MB/s]
Downloading (…)l-00014-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 95.7MB/s]
Downloading (…)l-00014-of-00033.bin: 60%|█▊ | 241M/405M [00:02<00:01, 96.1MB/s]
Downloading (…)l-00014-of-00033.bin: 62%|█▊ | 252M/405M [00:02<00:01, 96.3MB/s]
Downloading (…)l-00014-of-00033.bin: 65%|█▉ | 262M/405M [00:02<00:01, 95.6MB/s]
Downloading (…)l-00014-of-00033.bin: 67%|██ | 273M/405M [00:02<00:01, 95.4MB/s]
Downloading (…)l-00014-of-00033.bin: 70%|██ | 283M/405M [00:02<00:01, 96.7MB/s]
Downloading (…)l-00014-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 97.5MB/s]
Downloading (…)l-00014-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 92.8MB/s]
Downloading (…)l-00014-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 92.9MB/s]
Downloading (…)l-00014-of-00033.bin: 80%|██▍| 325M/405M [00:03<00:00, 95.4MB/s]
Downloading (…)l-00014-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 95.1MB/s]
Downloading (…)l-00014-of-00033.bin: 85%|██▌| 346M/405M [00:03<00:00, 95.8MB/s]
Downloading (…)l-00014-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 95.8MB/s]
Downloading (…)l-00014-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 92.2MB/s]
Downloading (…)l-00014-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 94.3MB/s]
Downloading (…)l-00014-of-00033.bin: 96%|██▉| 388M/405M [00:04<00:00, 95.8MB/s]
Downloading (…)l-00014-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 95.7MB/s]
Downloading shards: 42%|█████████▊ | 14/33 [01:36<02:11, 6.94s/it]
Downloading (…)l-00015-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00015-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 97.9MB/s]
Downloading (…)l-00015-of-00033.bin: 5%|▏ | 21.0M/405M [00:00<00:03, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 10%|▎ | 41.9M/405M [00:00<00:03, 103MB/s]
Downloading (…)l-00015-of-00033.bin: 13%|▍ | 52.4M/405M [00:00<00:03, 103MB/s]
Downloading (…)l-00015-of-00033.bin: 16%|▍ | 62.9M/405M [00:00<00:03, 102MB/s]
Downloading (…)l-00015-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:03, 99.2MB/s]
Downloading (…)l-00015-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 98.9MB/s]
Downloading (…)l-00015-of-00033.bin: 23%|▍ | 94.4M/405M [00:00<00:03, 97.3MB/s]
Downloading (…)l-00015-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 98.2MB/s]
Downloading (…)l-00015-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:02, 99.6MB/s]
Downloading (…)l-00015-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:02, 99.8MB/s]
Downloading (…)l-00015-of-00033.bin: 34%|█▎ | 136M/405M [00:01<00:02, 100MB/s]
Downloading (…)l-00015-of-00033.bin: 36%|█▍ | 147M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 39%|█▌ | 157M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 41%|█▋ | 168M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 44%|█▊ | 178M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 47%|█▊ | 189M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 49%|█▉ | 199M/405M [00:01<00:02, 100MB/s]
Downloading (…)l-00015-of-00033.bin: 52%|██ | 210M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 54%|██▏ | 220M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 57%|██▎ | 231M/405M [00:02<00:01, 102MB/s]
Downloading (…)l-00015-of-00033.bin: 60%|██▍ | 241M/405M [00:02<00:01, 102MB/s]
Downloading (…)l-00015-of-00033.bin: 62%|██▍ | 252M/405M [00:02<00:01, 102MB/s]
Downloading (…)l-00015-of-00033.bin: 65%|██▌ | 262M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 67%|██▋ | 273M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00015-of-00033.bin: 70%|██ | 283M/405M [00:02<00:01, 99.3MB/s]
Downloading (…)l-00015-of-00033.bin: 73%|██▏| 294M/405M [00:02<00:01, 95.7MB/s]
Downloading (…)l-00015-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 96.6MB/s]
Downloading (…)l-00015-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 97.4MB/s]
Downloading (…)l-00015-of-00033.bin: 80%|██▍| 325M/405M [00:03<00:00, 98.3MB/s]
Downloading (…)l-00015-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 97.7MB/s]
Downloading (…)l-00015-of-00033.bin: 85%|██▌| 346M/405M [00:03<00:00, 97.0MB/s]
Downloading (…)l-00015-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 98.3MB/s]
Downloading (…)l-00015-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 98.9MB/s]
Downloading (…)l-00015-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 99.6MB/s]
Downloading (…)l-00015-of-00033.bin: 96%|██▉| 388M/405M [00:03<00:00, 96.5MB/s]
Downloading (…)l-00015-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 99.4MB/s]
Downloading shards: 45%|██████████▍ | 15/33 [01:40<01:50, 6.14s/it]
Downloading (…)l-00016-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00016-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 81.9MB/s]
Downloading (…)l-00016-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 91.5MB/s]
Downloading (…)l-00016-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 94.3MB/s]
Downloading (…)l-00016-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 94.3MB/s]
Downloading (…)l-00016-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 96.0MB/s]
Downloading (…)l-00016-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:03, 97.1MB/s]
Downloading (…)l-00016-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:03, 91.4MB/s]
Downloading (…)l-00016-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 92.6MB/s]
Downloading (…)l-00016-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:03, 92.9MB/s]
Downloading (…)l-00016-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 95.1MB/s]
Downloading (…)l-00016-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:02, 97.0MB/s]
Downloading (…)l-00016-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:02, 98.1MB/s]
Downloading (…)l-00016-of-00033.bin: 34%|█ | 136M/405M [00:01<00:02, 98.9MB/s]
Downloading (…)l-00016-of-00033.bin: 36%|█ | 147M/405M [00:01<00:02, 98.5MB/s]
Downloading (…)l-00016-of-00033.bin: 39%|█▏ | 157M/405M [00:01<00:02, 99.5MB/s]
Downloading (…)l-00016-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 95.5MB/s]
Downloading (…)l-00016-of-00033.bin: 44%|█▎ | 178M/405M [00:01<00:02, 97.0MB/s]
Downloading (…)l-00016-of-00033.bin: 47%|█▍ | 189M/405M [00:01<00:02, 97.6MB/s]
Downloading (…)l-00016-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 97.9MB/s]
Downloading (…)l-00016-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:01, 98.0MB/s]
Downloading (…)l-00016-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:01, 99.3MB/s]
Downloading (…)l-00016-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 99.6MB/s]
Downloading (…)l-00016-of-00033.bin: 60%|█▊ | 241M/405M [00:02<00:01, 98.8MB/s]
Downloading (…)l-00016-of-00033.bin: 62%|█▊ | 252M/405M [00:02<00:01, 98.9MB/s]
Downloading (…)l-00016-of-00033.bin: 65%|█▉ | 262M/405M [00:02<00:01, 97.5MB/s]
Downloading (…)l-00016-of-00033.bin: 67%|██ | 273M/405M [00:02<00:01, 98.6MB/s]
Downloading (…)l-00016-of-00033.bin: 70%|██ | 283M/405M [00:02<00:01, 99.5MB/s]
Downloading (…)l-00016-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 99.7MB/s]
Downloading (…)l-00016-of-00033.bin: 75%|███ | 304M/405M [00:03<00:01, 100MB/s]
Downloading (…)l-00016-of-00033.bin: 78%|███ | 315M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00016-of-00033.bin: 80%|██▍| 325M/405M [00:03<00:00, 90.8MB/s]
Downloading (…)l-00016-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 89.6MB/s]
Downloading (…)l-00016-of-00033.bin: 85%|██▌| 346M/405M [00:03<00:00, 92.8MB/s]
Downloading (…)l-00016-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 94.9MB/s]
Downloading (…)l-00016-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 96.6MB/s]
Downloading (…)l-00016-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 96.8MB/s]
Downloading (…)l-00016-of-00033.bin: 96%|██▉| 388M/405M [00:04<00:00, 96.6MB/s]
Downloading (…)l-00016-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 96.6MB/s]
Downloading shards: 48%|███████████▏ | 16/33 [01:44<01:35, 5.61s/it]
Downloading (…)l-00017-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00017-of-00033.bin: 3%| | 10.5M/405M [00:00<00:05, 78.8MB/s]
Downloading (…)l-00017-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 90.4MB/s]
Downloading (…)l-00017-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 93.8MB/s]
Downloading (…)l-00017-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 96.0MB/s]
Downloading (…)l-00017-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 97.4MB/s]
Downloading (…)l-00017-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:03, 98.2MB/s]
Downloading (…)l-00017-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:03, 96.9MB/s]
Downloading (…)l-00017-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 97.8MB/s]
Downloading (…)l-00017-of-00033.bin: 23%|▍ | 94.4M/405M [00:00<00:03, 97.9MB/s]
Downloading (…)l-00017-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 97.1MB/s]
Downloading (…)l-00017-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:02, 97.5MB/s]
Downloading (…)l-00017-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:02, 97.3MB/s]
Downloading (…)l-00017-of-00033.bin: 34%|█ | 136M/405M [00:01<00:02, 97.7MB/s]
Downloading (…)l-00017-of-00033.bin: 36%|█ | 147M/405M [00:01<00:02, 98.5MB/s]
Downloading (…)l-00017-of-00033.bin: 39%|█▏ | 157M/405M [00:01<00:02, 97.8MB/s]
Downloading (…)l-00017-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 98.7MB/s]
Downloading (…)l-00017-of-00033.bin: 44%|█▎ | 178M/405M [00:01<00:02, 98.0MB/s]
Downloading (…)l-00017-of-00033.bin: 47%|█▍ | 189M/405M [00:01<00:02, 98.3MB/s]
Downloading (…)l-00017-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 98.9MB/s]
Downloading (…)l-00017-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:01, 99.4MB/s]
Downloading (…)l-00017-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:01, 99.9MB/s]
Downloading (…)l-00017-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 98.9MB/s]
Downloading (…)l-00017-of-00033.bin: 60%|█▊ | 241M/405M [00:02<00:01, 99.8MB/s]
Downloading (…)l-00017-of-00033.bin: 62%|██▍ | 252M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00017-of-00033.bin: 65%|█▉ | 262M/405M [00:02<00:01, 95.6MB/s]
Downloading (…)l-00017-of-00033.bin: 67%|██ | 273M/405M [00:02<00:01, 96.3MB/s]
Downloading (…)l-00017-of-00033.bin: 70%|██ | 283M/405M [00:02<00:01, 95.8MB/s]
Downloading (…)l-00017-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 96.4MB/s]
Downloading (…)l-00017-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 97.0MB/s]
Downloading (…)l-00017-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 98.1MB/s]
Downloading (…)l-00017-of-00033.bin: 80%|██▍| 325M/405M [00:03<00:00, 98.7MB/s]
Downloading (…)l-00017-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 98.7MB/s]
Downloading (…)l-00017-of-00033.bin: 85%|██▌| 346M/405M [00:03<00:00, 99.4MB/s]
Downloading (…)l-00017-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 91.4MB/s]
Downloading (…)l-00017-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 93.3MB/s]
Downloading (…)l-00017-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 95.8MB/s]
Downloading (…)l-00017-of-00033.bin: 96%|██▉| 388M/405M [00:03<00:00, 97.3MB/s]
Downloading (…)l-00017-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 97.2MB/s]
Downloading shards: 52%|███████████▊ | 17/33 [01:49<01:23, 5.24s/it]
Downloading (…)l-00018-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00018-of-00033.bin: 3%| | 10.5M/405M [00:00<00:05, 66.3MB/s]
Downloading (…)l-00018-of-00033.bin: 5%| | 21.0M/405M [00:00<00:05, 73.3MB/s]
Downloading (…)l-00018-of-00033.bin: 10%|▎ | 41.9M/405M [00:00<00:03, 101MB/s]
Downloading (…)l-00018-of-00033.bin: 16%|▍ | 62.9M/405M [00:00<00:02, 124MB/s]
Downloading (…)l-00018-of-00033.bin: 23%|▋ | 94.4M/405M [00:00<00:01, 156MB/s]
Downloading (…)l-00018-of-00033.bin: 31%|█▏ | 126M/405M [00:00<00:01, 174MB/s]
Downloading (…)l-00018-of-00033.bin: 39%|█▌ | 157M/405M [00:01<00:01, 192MB/s]
Downloading (…)l-00018-of-00033.bin: 44%|█▊ | 178M/405M [00:01<00:02, 112MB/s]
Downloading (…)l-00018-of-00033.bin: 49%|█▉ | 199M/405M [00:01<00:01, 111MB/s]
Downloading (…)l-00018-of-00033.bin: 57%|██▎ | 231M/405M [00:01<00:01, 140MB/s]
Downloading (…)l-00018-of-00033.bin: 65%|██▌ | 262M/405M [00:01<00:00, 167MB/s]
Downloading (…)l-00018-of-00033.bin: 73%|██▉ | 294M/405M [00:01<00:00, 190MB/s]
Downloading (…)l-00018-of-00033.bin: 80%|███▏| 325M/405M [00:02<00:00, 209MB/s]
Downloading (…)l-00018-of-00033.bin: 88%|███▌| 357M/405M [00:02<00:00, 224MB/s]
Downloading (…)l-00018-of-00033.bin: 100%|████| 405M/405M [00:02<00:00, 167MB/s]
Downloading shards: 55%|████████████▌ | 18/33 [01:51<01:06, 4.46s/it]
Downloading (…)l-00019-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00019-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 93.7MB/s]
Downloading (…)l-00019-of-00033.bin: 5%| | 21.0M/405M [00:00<00:03, 96.8MB/s]
Downloading (…)l-00019-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 99.0MB/s]
Downloading (…)l-00019-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 99.4MB/s]
Downloading (…)l-00019-of-00033.bin: 13%|▍ | 52.4M/405M [00:00<00:03, 100MB/s]
Downloading (…)l-00019-of-00033.bin: 16%|▍ | 62.9M/405M [00:00<00:03, 101MB/s]
Downloading (…)l-00019-of-00033.bin: 18%|▌ | 73.4M/405M [00:00<00:03, 100MB/s]
Downloading (…)l-00019-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 98.5MB/s]
Downloading (…)l-00019-of-00033.bin: 23%|▍ | 94.4M/405M [00:00<00:03, 98.3MB/s]
Downloading (…)l-00019-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 98.3MB/s]
Downloading (…)l-00019-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 96.4MB/s]
Downloading (…)l-00019-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:02, 97.1MB/s]
Downloading (…)l-00019-of-00033.bin: 34%|█ | 136M/405M [00:01<00:02, 96.9MB/s]
Downloading (…)l-00019-of-00033.bin: 36%|█ | 147M/405M [00:01<00:02, 98.3MB/s]
Downloading (…)l-00019-of-00033.bin: 39%|█▏ | 157M/405M [00:01<00:02, 99.3MB/s]
Downloading (…)l-00019-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 99.5MB/s]
Downloading (…)l-00019-of-00033.bin: 44%|█▊ | 178M/405M [00:01<00:02, 100MB/s]
Downloading (…)l-00019-of-00033.bin: 47%|█▊ | 189M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00019-of-00033.bin: 49%|█▉ | 199M/405M [00:02<00:02, 101MB/s]
Downloading (…)l-00019-of-00033.bin: 52%|██ | 210M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00019-of-00033.bin: 54%|██▏ | 220M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00019-of-00033.bin: 57%|██▎ | 231M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00019-of-00033.bin: 60%|██▍ | 241M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00019-of-00033.bin: 62%|██▍ | 252M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00019-of-00033.bin: 65%|██▌ | 262M/405M [00:02<00:01, 101MB/s]
Downloading (…)l-00019-of-00033.bin: 67%|██▋ | 273M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00019-of-00033.bin: 70%|██▊ | 283M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00019-of-00033.bin: 73%|██▉ | 294M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00019-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 99.7MB/s]
Downloading (…)l-00019-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 96.5MB/s]
Downloading (…)l-00019-of-00033.bin: 80%|██▍| 325M/405M [00:03<00:00, 97.8MB/s]
Downloading (…)l-00019-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 98.0MB/s]
Downloading (…)l-00019-of-00033.bin: 85%|██▌| 346M/405M [00:03<00:00, 98.4MB/s]
Downloading (…)l-00019-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 99.0MB/s]
Downloading (…)l-00019-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 98.0MB/s]
Downloading (…)l-00019-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 98.4MB/s]
Downloading (…)l-00019-of-00033.bin: 96%|██▉| 388M/405M [00:03<00:00, 98.6MB/s]
Downloading (…)l-00019-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 98.9MB/s]
Downloading shards: 58%|█████████████▏ | 19/33 [01:56<01:01, 4.41s/it]
Downloading (…)l-00020-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00020-of-00033.bin: 3%| | 10.5M/405M [00:00<00:05, 77.2MB/s]
Downloading (…)l-00020-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 113MB/s]
Downloading (…)l-00020-of-00033.bin: 13%|▍ | 52.4M/405M [00:00<00:02, 141MB/s]
Downloading (…)l-00020-of-00033.bin: 21%|▌ | 83.9M/405M [00:00<00:01, 180MB/s]
Downloading (…)l-00020-of-00033.bin: 28%|█▏ | 115M/405M [00:00<00:01, 209MB/s]
Downloading (…)l-00020-of-00033.bin: 36%|█▍ | 147M/405M [00:00<00:01, 223MB/s]
Downloading (…)l-00020-of-00033.bin: 44%|█▊ | 178M/405M [00:00<00:00, 239MB/s]
Downloading (…)l-00020-of-00033.bin: 52%|██ | 210M/405M [00:01<00:00, 202MB/s]
Downloading (…)l-00020-of-00033.bin: 60%|█▊ | 241M/405M [00:01<00:01, 90.5MB/s]
Downloading (…)l-00020-of-00033.bin: 67%|██▋ | 273M/405M [00:01<00:01, 112MB/s]
Downloading (…)l-00020-of-00033.bin: 75%|███ | 304M/405M [00:02<00:00, 136MB/s]
Downloading (…)l-00020-of-00033.bin: 80%|███▏| 325M/405M [00:02<00:00, 148MB/s]
Downloading (…)l-00020-of-00033.bin: 88%|███▌| 357M/405M [00:02<00:00, 170MB/s]
Downloading (…)l-00020-of-00033.bin: 100%|████| 405M/405M [00:02<00:00, 160MB/s]
Downloading shards: 61%|█████████████▉ | 20/33 [01:58<00:50, 3.91s/it]
Downloading (…)l-00021-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00021-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 88.2MB/s]
Downloading (…)l-00021-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 110MB/s]
Downloading (…)l-00021-of-00033.bin: 13%|▍ | 52.4M/405M [00:00<00:02, 137MB/s]
Downloading (…)l-00021-of-00033.bin: 18%|▌ | 73.4M/405M [00:00<00:02, 159MB/s]
Downloading (…)l-00021-of-00033.bin: 26%|█ | 105M/405M [00:00<00:01, 198MB/s]
Downloading (…)l-00021-of-00033.bin: 34%|█▎ | 136M/405M [00:00<00:01, 219MB/s]
Downloading (…)l-00021-of-00033.bin: 41%|█▋ | 168M/405M [00:00<00:01, 234MB/s]
Downloading (…)l-00021-of-00033.bin: 49%|█▉ | 199M/405M [00:00<00:00, 240MB/s]
Downloading (…)l-00021-of-00033.bin: 57%|██▎ | 231M/405M [00:01<00:01, 104MB/s]
Downloading (…)l-00021-of-00033.bin: 62%|█▊ | 252M/405M [00:01<00:01, 90.5MB/s]
Downloading (…)l-00021-of-00033.bin: 70%|██▊ | 283M/405M [00:02<00:01, 117MB/s]
Downloading (…)l-00021-of-00033.bin: 78%|███ | 315M/405M [00:02<00:00, 146MB/s]
Downloading (…)l-00021-of-00033.bin: 85%|███▍| 346M/405M [00:02<00:00, 169MB/s]
Downloading (…)l-00021-of-00033.bin: 93%|███▋| 377M/405M [00:02<00:00, 191MB/s]
Downloading (…)l-00021-of-00033.bin: 100%|████| 405M/405M [00:02<00:00, 159MB/s]
Downloading shards: 64%|██████████████▋ | 21/33 [02:01<00:42, 3.56s/it]
Downloading (…)l-00022-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00022-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 81.9MB/s]
Downloading (…)l-00022-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 82.9MB/s]
Downloading (…)l-00022-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 81.0MB/s]
Downloading (…)l-00022-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:04, 80.8MB/s]
Downloading (…)l-00022-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:04, 81.1MB/s]
Downloading (…)l-00022-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:04, 81.0MB/s]
Downloading (…)l-00022-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:04, 82.5MB/s]
Downloading (…)l-00022-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:04, 77.1MB/s]
Downloading (…)l-00022-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:04, 77.6MB/s]
Downloading (…)l-00022-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 78.0MB/s]
Downloading (…)l-00022-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 76.2MB/s]
Downloading (…)l-00022-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:03, 78.0MB/s]
Downloading (…)l-00022-of-00033.bin: 34%|█ | 136M/405M [00:01<00:03, 78.1MB/s]
Downloading (…)l-00022-of-00033.bin: 36%|█ | 147M/405M [00:01<00:03, 78.9MB/s]
Downloading (…)l-00022-of-00033.bin: 39%|█▏ | 157M/405M [00:01<00:03, 77.3MB/s]
Downloading (…)l-00022-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 77.9MB/s]
Downloading (…)l-00022-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:02, 79.7MB/s]
Downloading (…)l-00022-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 80.3MB/s]
Downloading (…)l-00022-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 75.9MB/s]
Downloading (…)l-00022-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 77.7MB/s]
Downloading (…)l-00022-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:02, 79.0MB/s]
Downloading (…)l-00022-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:02, 78.9MB/s]
Downloading (…)l-00022-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 78.5MB/s]
Downloading (…)l-00022-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:01, 79.7MB/s]
Downloading (…)l-00022-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:01, 79.5MB/s]
Downloading (…)l-00022-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 81.0MB/s]
Downloading (…)l-00022-of-00033.bin: 70%|██ | 283M/405M [00:03<00:01, 80.5MB/s]
Downloading (…)l-00022-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 80.4MB/s]
Downloading (…)l-00022-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 80.3MB/s]
Downloading (…)l-00022-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:01, 77.4MB/s]
Downloading (…)l-00022-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:00, 80.8MB/s]
Downloading (…)l-00022-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 80.6MB/s]
Downloading (…)l-00022-of-00033.bin: 85%|██▌| 346M/405M [00:04<00:00, 78.9MB/s]
Downloading (…)l-00022-of-00033.bin: 88%|██▋| 357M/405M [00:04<00:00, 78.8MB/s]
Downloading (…)l-00022-of-00033.bin: 91%|██▋| 367M/405M [00:04<00:00, 79.9MB/s]
Downloading (…)l-00022-of-00033.bin: 93%|██▊| 377M/405M [00:04<00:00, 80.7MB/s]
Downloading (…)l-00022-of-00033.bin: 96%|██▉| 388M/405M [00:04<00:00, 80.6MB/s]
Downloading (…)l-00022-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 79.4MB/s]
Downloading shards: 67%|███████████████▎ | 22/33 [02:06<00:44, 4.09s/it]
Downloading (…)l-00023-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00023-of-00033.bin: 3%| | 10.5M/405M [00:00<00:05, 77.0MB/s]
Downloading (…)l-00023-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 80.6MB/s]
Downloading (…)l-00023-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 79.7MB/s]
Downloading (…)l-00023-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:05, 64.2MB/s]
Downloading (…)l-00023-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:05, 67.1MB/s]
Downloading (…)l-00023-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:04, 70.9MB/s]
Downloading (…)l-00023-of-00033.bin: 18%|▎ | 73.4M/405M [00:01<00:04, 73.9MB/s]
Downloading (…)l-00023-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:04, 74.5MB/s]
Downloading (…)l-00023-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:04, 74.9MB/s]
Downloading (…)l-00023-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 76.9MB/s]
Downloading (…)l-00023-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 75.0MB/s]
Downloading (…)l-00023-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:03, 71.9MB/s]
Downloading (…)l-00023-of-00033.bin: 34%|█ | 136M/405M [00:01<00:03, 74.2MB/s]
Downloading (…)l-00023-of-00033.bin: 36%|█ | 147M/405M [00:02<00:03, 70.3MB/s]
Downloading (…)l-00023-of-00033.bin: 39%|█▏ | 157M/405M [00:02<00:03, 73.4MB/s]
Downloading (…)l-00023-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 69.8MB/s]
Downloading (…)l-00023-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:03, 72.3MB/s]
Downloading (…)l-00023-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 74.6MB/s]
Downloading (…)l-00023-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 69.8MB/s]
Downloading (…)l-00023-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 68.4MB/s]
Downloading (…)l-00023-of-00033.bin: 54%|█▋ | 220M/405M [00:03<00:03, 60.8MB/s]
Downloading (…)l-00023-of-00033.bin: 57%|█▋ | 231M/405M [00:03<00:02, 60.4MB/s]
Downloading (…)l-00023-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 65.6MB/s]
Downloading (…)l-00023-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:02, 67.4MB/s]
Downloading (…)l-00023-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:02, 68.1MB/s]
Downloading (…)l-00023-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 67.5MB/s]
Downloading (…)l-00023-of-00033.bin: 70%|██ | 283M/405M [00:04<00:01, 66.7MB/s]
Downloading (…)l-00023-of-00033.bin: 73%|██▏| 294M/405M [00:04<00:01, 72.0MB/s]
Downloading (…)l-00023-of-00033.bin: 75%|██▎| 304M/405M [00:04<00:01, 72.2MB/s]
Downloading (…)l-00023-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 71.8MB/s]
Downloading (…)l-00023-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:01, 73.8MB/s]
Downloading (…)l-00023-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 74.5MB/s]
Downloading (…)l-00023-of-00033.bin: 85%|██▌| 346M/405M [00:04<00:00, 71.9MB/s]
Downloading (…)l-00023-of-00033.bin: 88%|██▋| 357M/405M [00:05<00:00, 61.9MB/s]
Downloading (…)l-00023-of-00033.bin: 91%|██▋| 367M/405M [00:05<00:00, 66.5MB/s]
Downloading (…)l-00023-of-00033.bin: 93%|██▊| 377M/405M [00:05<00:00, 70.3MB/s]
Downloading (…)l-00023-of-00033.bin: 96%|██▉| 388M/405M [00:05<00:00, 73.4MB/s]
Downloading (…)l-00023-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 70.0MB/s]
Downloading shards: 70%|████████████████ | 23/33 [02:12<00:46, 4.66s/it]
Downloading (…)l-00024-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00024-of-00033.bin: 3%| | 10.5M/405M [00:00<00:05, 74.2MB/s]
Downloading (…)l-00024-of-00033.bin: 5%| | 21.0M/405M [00:00<00:05, 69.2MB/s]
Downloading (…)l-00024-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:05, 72.7MB/s]
Downloading (…)l-00024-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:04, 73.3MB/s]
Downloading (…)l-00024-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:04, 76.5MB/s]
Downloading (…)l-00024-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:04, 75.7MB/s]
Downloading (…)l-00024-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:04, 76.6MB/s]
Downloading (…)l-00024-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:04, 77.6MB/s]
Downloading (…)l-00024-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:04, 71.6MB/s]
Downloading (…)l-00024-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:04, 64.7MB/s]
Downloading (…)l-00024-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:04, 65.2MB/s]
Downloading (…)l-00024-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:04, 67.1MB/s]
Downloading (…)l-00024-of-00033.bin: 34%|█ | 136M/405M [00:01<00:04, 66.8MB/s]
Downloading (…)l-00024-of-00033.bin: 36%|█ | 147M/405M [00:02<00:03, 69.6MB/s]
Downloading (…)l-00024-of-00033.bin: 39%|█▏ | 157M/405M [00:02<00:03, 74.1MB/s]
Downloading (…)l-00024-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 74.4MB/s]
Downloading (…)l-00024-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:03, 74.8MB/s]
Downloading (…)l-00024-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 74.0MB/s]
Downloading (…)l-00024-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 75.2MB/s]
Downloading (…)l-00024-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 73.5MB/s]
Downloading (…)l-00024-of-00033.bin: 54%|█▋ | 220M/405M [00:03<00:02, 74.9MB/s]
Downloading (…)l-00024-of-00033.bin: 57%|█▋ | 231M/405M [00:03<00:02, 76.2MB/s]
Downloading (…)l-00024-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 75.8MB/s]
Downloading (…)l-00024-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:02, 74.9MB/s]
Downloading (…)l-00024-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:01, 76.5MB/s]
Downloading (…)l-00024-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 78.2MB/s]
Downloading (…)l-00024-of-00033.bin: 70%|██ | 283M/405M [00:03<00:01, 78.8MB/s]
Downloading (…)l-00024-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 79.8MB/s]
Downloading (…)l-00024-of-00033.bin: 75%|██▎| 304M/405M [00:04<00:01, 77.2MB/s]
Downloading (…)l-00024-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 79.4MB/s]
Downloading (…)l-00024-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:00, 80.0MB/s]
Downloading (…)l-00024-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 80.5MB/s]
Downloading (…)l-00024-of-00033.bin: 85%|██▌| 346M/405M [00:04<00:00, 81.5MB/s]
Downloading (…)l-00024-of-00033.bin: 88%|██▋| 357M/405M [00:04<00:00, 80.8MB/s]
Downloading (…)l-00024-of-00033.bin: 91%|██▋| 367M/405M [00:04<00:00, 75.1MB/s]
Downloading (…)l-00024-of-00033.bin: 93%|██▊| 377M/405M [00:05<00:00, 78.4MB/s]
Downloading (…)l-00024-of-00033.bin: 96%|██▉| 388M/405M [00:05<00:00, 77.6MB/s]
Downloading (…)l-00024-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 75.1MB/s]
Downloading shards: 73%|████████████████▋ | 24/33 [02:18<00:44, 4.95s/it]
Downloading (…)l-00025-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00025-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 88.7MB/s]
Downloading (…)l-00025-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 94.4MB/s]
Downloading (…)l-00025-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 95.5MB/s]
Downloading (…)l-00025-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 97.7MB/s]
Downloading (…)l-00025-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 95.2MB/s]
Downloading (…)l-00025-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:03, 96.6MB/s]
Downloading (…)l-00025-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:03, 96.5MB/s]
Downloading (…)l-00025-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 97.0MB/s]
Downloading (…)l-00025-of-00033.bin: 23%|▍ | 94.4M/405M [00:00<00:03, 98.1MB/s]
Downloading (…)l-00025-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 97.9MB/s]
Downloading (…)l-00025-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:02, 98.5MB/s]
Downloading (…)l-00025-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:02, 98.7MB/s]
Downloading (…)l-00025-of-00033.bin: 34%|█ | 136M/405M [00:01<00:02, 97.5MB/s]
Downloading (…)l-00025-of-00033.bin: 39%|█▌ | 157M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00025-of-00033.bin: 41%|█▋ | 168M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00025-of-00033.bin: 44%|█▎ | 178M/405M [00:01<00:02, 96.7MB/s]
Downloading (…)l-00025-of-00033.bin: 47%|█▍ | 189M/405M [00:01<00:02, 96.5MB/s]
Downloading (…)l-00025-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 96.4MB/s]
Downloading (…)l-00025-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 96.3MB/s]
Downloading (…)l-00025-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:01, 94.3MB/s]
Downloading (…)l-00025-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 96.9MB/s]
Downloading (…)l-00025-of-00033.bin: 60%|█▊ | 241M/405M [00:02<00:01, 96.4MB/s]
Downloading (…)l-00025-of-00033.bin: 65%|█▉ | 262M/405M [00:02<00:01, 99.2MB/s]
Downloading (…)l-00025-of-00033.bin: 67%|██ | 273M/405M [00:02<00:01, 98.0MB/s]
Downloading (…)l-00025-of-00033.bin: 70%|██ | 283M/405M [00:02<00:01, 98.1MB/s]
Downloading (…)l-00025-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 97.6MB/s]
Downloading (…)l-00025-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 96.4MB/s]
Downloading (…)l-00025-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 95.8MB/s]
Downloading (…)l-00025-of-00033.bin: 80%|██▍| 325M/405M [00:03<00:00, 97.0MB/s]
Downloading (…)l-00025-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 97.5MB/s]
Downloading (…)l-00025-of-00033.bin: 85%|██▌| 346M/405M [00:03<00:00, 97.8MB/s]
Downloading (…)l-00025-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 98.2MB/s]
Downloading (…)l-00025-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 98.2MB/s]
Downloading (…)l-00025-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 95.5MB/s]
Downloading (…)l-00025-of-00033.bin: 96%|██▉| 388M/405M [00:03<00:00, 97.8MB/s]
Downloading (…)l-00025-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 97.2MB/s]
Downloading shards: 76%|█████████████████▍ | 25/33 [02:22<00:38, 4.77s/it]
Downloading (…)l-00026-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00026-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 90.4MB/s]
Downloading (…)l-00026-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 95.9MB/s]
Downloading (…)l-00026-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 98.0MB/s]
Downloading (…)l-00026-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:03, 99.0MB/s]
Downloading (…)l-00026-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 99.5MB/s]
Downloading (…)l-00026-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:03, 99.4MB/s]
Downloading (…)l-00026-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:03, 99.5MB/s]
Downloading (…)l-00026-of-00033.bin: 21%|▍ | 83.9M/405M [00:00<00:03, 97.7MB/s]
Downloading (…)l-00026-of-00033.bin: 23%|▍ | 94.4M/405M [00:00<00:03, 98.4MB/s]
Downloading (…)l-00026-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 99.2MB/s]
Downloading (…)l-00026-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:02, 99.1MB/s]
Downloading (…)l-00026-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:02, 98.8MB/s]
Downloading (…)l-00026-of-00033.bin: 34%|█ | 136M/405M [00:01<00:02, 99.3MB/s]
Downloading (…)l-00026-of-00033.bin: 36%|█ | 147M/405M [00:01<00:02, 99.7MB/s]
Downloading (…)l-00026-of-00033.bin: 39%|█▏ | 157M/405M [00:01<00:02, 99.6MB/s]
Downloading (…)l-00026-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 98.2MB/s]
Downloading (…)l-00026-of-00033.bin: 44%|█▎ | 178M/405M [00:01<00:02, 95.9MB/s]
Downloading (…)l-00026-of-00033.bin: 47%|█▍ | 189M/405M [00:01<00:02, 97.2MB/s]
Downloading (…)l-00026-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 96.8MB/s]
Downloading (…)l-00026-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:01, 97.6MB/s]
Downloading (…)l-00026-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:01, 98.8MB/s]
Downloading (…)l-00026-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 99.2MB/s]
Downloading (…)l-00026-of-00033.bin: 60%|█▊ | 241M/405M [00:02<00:01, 98.1MB/s]
Downloading (…)l-00026-of-00033.bin: 62%|█▊ | 252M/405M [00:02<00:01, 97.6MB/s]
Downloading (…)l-00026-of-00033.bin: 65%|█▉ | 262M/405M [00:02<00:01, 98.6MB/s]
Downloading (…)l-00026-of-00033.bin: 67%|██ | 273M/405M [00:02<00:01, 98.5MB/s]
Downloading (…)l-00026-of-00033.bin: 70%|██ | 283M/405M [00:02<00:01, 97.6MB/s]
Downloading (…)l-00026-of-00033.bin: 73%|██▏| 294M/405M [00:02<00:01, 98.3MB/s]
Downloading (…)l-00026-of-00033.bin: 75%|███ | 304M/405M [00:03<00:01, 100MB/s]
Downloading (…)l-00026-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 98.6MB/s]
Downloading (…)l-00026-of-00033.bin: 83%|███▎| 336M/405M [00:03<00:00, 101MB/s]
Downloading (…)l-00026-of-00033.bin: 85%|███▍| 346M/405M [00:03<00:00, 101MB/s]
Downloading (…)l-00026-of-00033.bin: 88%|███▌| 357M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00026-of-00033.bin: 91%|███▋| 367M/405M [00:03<00:00, 100MB/s]
Downloading (…)l-00026-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 93.7MB/s]
Downloading (…)l-00026-of-00033.bin: 96%|██▉| 388M/405M [00:03<00:00, 92.9MB/s]
Downloading (…)l-00026-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 96.9MB/s]
Downloading shards: 79%|██████████████████ | 26/33 [02:27<00:32, 4.65s/it]
Downloading (…)l-00027-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00027-of-00033.bin: 3%| | 10.5M/405M [00:00<00:05, 72.5MB/s]
Downloading (…)l-00027-of-00033.bin: 5%| | 21.0M/405M [00:00<00:05, 73.2MB/s]
Downloading (…)l-00027-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 75.1MB/s]
Downloading (…)l-00027-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:04, 77.1MB/s]
Downloading (…)l-00027-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:04, 78.2MB/s]
Downloading (…)l-00027-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:04, 80.4MB/s]
Downloading (…)l-00027-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:04, 81.6MB/s]
Downloading (…)l-00027-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:03, 81.5MB/s]
Downloading (…)l-00027-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:03, 81.7MB/s]
Downloading (…)l-00027-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:03, 81.2MB/s]
Downloading (…)l-00027-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:03, 80.9MB/s]
Downloading (…)l-00027-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:03, 80.3MB/s]
Downloading (…)l-00027-of-00033.bin: 34%|█ | 136M/405M [00:01<00:03, 81.8MB/s]
Downloading (…)l-00027-of-00033.bin: 36%|█ | 147M/405M [00:01<00:03, 78.8MB/s]
Downloading (…)l-00027-of-00033.bin: 39%|█▏ | 157M/405M [00:01<00:03, 77.3MB/s]
Downloading (…)l-00027-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 74.0MB/s]
Downloading (…)l-00027-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:03, 73.3MB/s]
Downloading (…)l-00027-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:03, 70.9MB/s]
Downloading (…)l-00027-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 73.3MB/s]
Downloading (…)l-00027-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 75.5MB/s]
Downloading (…)l-00027-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:02, 77.8MB/s]
Downloading (…)l-00027-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:02, 75.9MB/s]
Downloading (…)l-00027-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 68.7MB/s]
Downloading (…)l-00027-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:02, 65.4MB/s]
Downloading (…)l-00027-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:02, 69.6MB/s]
Downloading (…)l-00027-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 70.8MB/s]
Downloading (…)l-00027-of-00033.bin: 70%|██ | 283M/405M [00:03<00:01, 67.5MB/s]
Downloading (…)l-00027-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 68.0MB/s]
Downloading (…)l-00027-of-00033.bin: 75%|██▎| 304M/405M [00:04<00:01, 68.8MB/s]
Downloading (…)l-00027-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 72.0MB/s]
Downloading (…)l-00027-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:01, 74.0MB/s]
Downloading (…)l-00027-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 72.1MB/s]
Downloading (…)l-00027-of-00033.bin: 85%|██▌| 346M/405M [00:04<00:00, 59.0MB/s]
Downloading (…)l-00027-of-00033.bin: 88%|██▋| 357M/405M [00:04<00:00, 62.8MB/s]
Downloading (…)l-00027-of-00033.bin: 91%|██▋| 367M/405M [00:05<00:00, 63.3MB/s]
Downloading (…)l-00027-of-00033.bin: 93%|██▊| 377M/405M [00:05<00:00, 66.7MB/s]
Downloading (…)l-00027-of-00033.bin: 96%|██▉| 388M/405M [00:05<00:00, 70.0MB/s]
Downloading (…)l-00027-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 72.9MB/s]
Downloading shards: 82%|██████████████████▊ | 27/33 [02:32<00:29, 4.98s/it]
Downloading (…)l-00028-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00028-of-00033.bin: 3%| | 10.5M/405M [00:00<00:05, 77.9MB/s]
Downloading (…)l-00028-of-00033.bin: 5%| | 21.0M/405M [00:00<00:04, 77.3MB/s]
Downloading (…)l-00028-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 74.8MB/s]
Downloading (…)l-00028-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:05, 71.2MB/s]
Downloading (…)l-00028-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:05, 64.8MB/s]
Downloading (…)l-00028-of-00033.bin: 16%|▎ | 62.9M/405M [00:01<00:07, 48.8MB/s]
Downloading (…)l-00028-of-00033.bin: 18%|▎ | 73.4M/405M [00:01<00:07, 42.6MB/s]
Downloading (…)l-00028-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:06, 48.5MB/s]
Downloading (…)l-00028-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:06, 47.1MB/s]
Downloading (…)l-00028-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:05, 51.7MB/s]
Downloading (…)l-00028-of-00033.bin: 28%|▊ | 115M/405M [00:02<00:05, 54.6MB/s]
Downloading (…)l-00028-of-00033.bin: 31%|▉ | 126M/405M [00:02<00:04, 60.4MB/s]
Downloading (…)l-00028-of-00033.bin: 34%|█ | 136M/405M [00:02<00:04, 62.1MB/s]
Downloading (…)l-00028-of-00033.bin: 36%|█ | 147M/405M [00:02<00:04, 56.1MB/s]
Downloading (…)l-00028-of-00033.bin: 39%|█▏ | 157M/405M [00:02<00:04, 59.7MB/s]
Downloading (…)l-00028-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 64.9MB/s]
Downloading (…)l-00028-of-00033.bin: 44%|█▎ | 178M/405M [00:03<00:03, 69.0MB/s]
Downloading (…)l-00028-of-00033.bin: 47%|█▍ | 189M/405M [00:03<00:03, 70.3MB/s]
Downloading (…)l-00028-of-00033.bin: 49%|█▍ | 199M/405M [00:03<00:02, 73.5MB/s]
Downloading (…)l-00028-of-00033.bin: 52%|█▌ | 210M/405M [00:03<00:02, 71.5MB/s]
Downloading (…)l-00028-of-00033.bin: 54%|█▋ | 220M/405M [00:03<00:02, 73.5MB/s]
Downloading (…)l-00028-of-00033.bin: 57%|█▋ | 231M/405M [00:03<00:02, 74.3MB/s]
Downloading (…)l-00028-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 73.7MB/s]
Downloading (…)l-00028-of-00033.bin: 62%|█▊ | 252M/405M [00:04<00:02, 73.7MB/s]
Downloading (…)l-00028-of-00033.bin: 65%|█▉ | 262M/405M [00:04<00:01, 74.9MB/s]
Downloading (…)l-00028-of-00033.bin: 67%|██ | 273M/405M [00:04<00:01, 76.5MB/s]
Downloading (…)l-00028-of-00033.bin: 70%|██ | 283M/405M [00:04<00:01, 78.5MB/s]
Downloading (…)l-00028-of-00033.bin: 73%|██▏| 294M/405M [00:04<00:01, 78.5MB/s]
Downloading (…)l-00028-of-00033.bin: 75%|██▎| 304M/405M [00:04<00:01, 78.2MB/s]
Downloading (…)l-00028-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 78.1MB/s]
Downloading (…)l-00028-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:01, 79.1MB/s]
Downloading (…)l-00028-of-00033.bin: 83%|██▍| 336M/405M [00:05<00:00, 79.5MB/s]
Downloading (…)l-00028-of-00033.bin: 85%|██▌| 346M/405M [00:05<00:00, 79.8MB/s]
Downloading (…)l-00028-of-00033.bin: 88%|██▋| 357M/405M [00:05<00:00, 74.7MB/s]
Downloading (…)l-00028-of-00033.bin: 91%|██▋| 367M/405M [00:05<00:00, 72.1MB/s]
Downloading (…)l-00028-of-00033.bin: 93%|██▊| 377M/405M [00:05<00:00, 72.7MB/s]
Downloading (…)l-00028-of-00033.bin: 96%|██▉| 388M/405M [00:05<00:00, 74.4MB/s]
Downloading (…)l-00028-of-00033.bin: 100%|███| 405M/405M [00:06<00:00, 66.1MB/s]
Downloading shards: 85%|███████████████████▌ | 28/33 [02:39<00:26, 5.39s/it]
Downloading (…)l-00029-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00029-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 80.0MB/s]
Downloading (…)l-00029-of-00033.bin: 5%| | 21.0M/405M [00:00<00:05, 76.5MB/s]
Downloading (…)l-00029-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 80.1MB/s]
Downloading (…)l-00029-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:04, 82.0MB/s]
Downloading (…)l-00029-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:04, 82.7MB/s]
Downloading (…)l-00029-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:04, 80.0MB/s]
Downloading (…)l-00029-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:04, 70.4MB/s]
Downloading (…)l-00029-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:04, 71.9MB/s]
Downloading (…)l-00029-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:04, 73.6MB/s]
Downloading (…)l-00029-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:04, 74.3MB/s]
Downloading (…)l-00029-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:04, 72.0MB/s]
Downloading (…)l-00029-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:03, 74.9MB/s]
Downloading (…)l-00029-of-00033.bin: 34%|█ | 136M/405M [00:01<00:03, 76.1MB/s]
Downloading (…)l-00029-of-00033.bin: 36%|█ | 147M/405M [00:01<00:03, 75.0MB/s]
Downloading (…)l-00029-of-00033.bin: 39%|█▏ | 157M/405M [00:02<00:03, 72.9MB/s]
Downloading (…)l-00029-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 76.4MB/s]
Downloading (…)l-00029-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:02, 78.0MB/s]
Downloading (…)l-00029-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 79.1MB/s]
Downloading (…)l-00029-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 74.3MB/s]
Downloading (…)l-00029-of-00033.bin: 52%|█▌ | 210M/405M [00:02<00:02, 74.6MB/s]
Downloading (…)l-00029-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:02, 76.3MB/s]
Downloading (…)l-00029-of-00033.bin: 57%|█▋ | 231M/405M [00:03<00:02, 78.0MB/s]
Downloading (…)l-00029-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 78.4MB/s]
Downloading (…)l-00029-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:01, 80.0MB/s]
Downloading (…)l-00029-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:01, 80.8MB/s]
Downloading (…)l-00029-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 78.3MB/s]
Downloading (…)l-00029-of-00033.bin: 70%|██ | 283M/405M [00:03<00:01, 78.2MB/s]
Downloading (…)l-00029-of-00033.bin: 73%|██▏| 294M/405M [00:03<00:01, 77.3MB/s]
Downloading (…)l-00029-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 77.6MB/s]
Downloading (…)l-00029-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 76.0MB/s]
Downloading (…)l-00029-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:01, 76.8MB/s]
Downloading (…)l-00029-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 77.6MB/s]
Downloading (…)l-00029-of-00033.bin: 85%|██▌| 346M/405M [00:04<00:00, 77.1MB/s]
Downloading (…)l-00029-of-00033.bin: 88%|██▋| 357M/405M [00:04<00:00, 79.0MB/s]
Downloading (…)l-00029-of-00033.bin: 91%|██▋| 367M/405M [00:04<00:00, 74.6MB/s]
Downloading (…)l-00029-of-00033.bin: 93%|██▊| 377M/405M [00:04<00:00, 74.8MB/s]
Downloading (…)l-00029-of-00033.bin: 96%|██▉| 388M/405M [00:05<00:00, 76.2MB/s]
Downloading (…)l-00029-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 76.6MB/s]
Downloading shards: 88%|████████████████████▏ | 29/33 [02:44<00:21, 5.42s/it]
Downloading (…)l-00030-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00030-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 80.1MB/s]
Downloading (…)l-00030-of-00033.bin: 5%| | 21.0M/405M [00:00<00:05, 75.2MB/s]
Downloading (…)l-00030-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:04, 76.3MB/s]
Downloading (…)l-00030-of-00033.bin: 10%|▏ | 41.9M/405M [00:00<00:04, 77.9MB/s]
Downloading (…)l-00030-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:04, 78.2MB/s]
Downloading (…)l-00030-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:04, 75.4MB/s]
Downloading (…)l-00030-of-00033.bin: 18%|▎ | 73.4M/405M [00:00<00:04, 77.1MB/s]
Downloading (…)l-00030-of-00033.bin: 21%|▍ | 83.9M/405M [00:01<00:04, 71.1MB/s]
Downloading (…)l-00030-of-00033.bin: 23%|▍ | 94.4M/405M [00:01<00:04, 73.0MB/s]
Downloading (…)l-00030-of-00033.bin: 26%|▊ | 105M/405M [00:01<00:04, 67.6MB/s]
Downloading (…)l-00030-of-00033.bin: 28%|▊ | 115M/405M [00:01<00:06, 48.0MB/s]
Downloading (…)l-00030-of-00033.bin: 31%|▉ | 126M/405M [00:01<00:05, 52.3MB/s]
Downloading (…)l-00030-of-00033.bin: 34%|█ | 136M/405M [00:02<00:04, 57.8MB/s]
Downloading (…)l-00030-of-00033.bin: 36%|█ | 147M/405M [00:02<00:04, 63.0MB/s]
Downloading (…)l-00030-of-00033.bin: 39%|█▏ | 157M/405M [00:02<00:03, 66.5MB/s]
Downloading (…)l-00030-of-00033.bin: 41%|█▏ | 168M/405M [00:02<00:03, 70.9MB/s]
Downloading (…)l-00030-of-00033.bin: 44%|█▎ | 178M/405M [00:02<00:03, 72.4MB/s]
Downloading (…)l-00030-of-00033.bin: 47%|█▍ | 189M/405M [00:02<00:02, 74.3MB/s]
Downloading (…)l-00030-of-00033.bin: 49%|█▍ | 199M/405M [00:02<00:02, 76.4MB/s]
Downloading (…)l-00030-of-00033.bin: 52%|█▌ | 210M/405M [00:03<00:02, 69.4MB/s]
Downloading (…)l-00030-of-00033.bin: 54%|█▋ | 220M/405M [00:03<00:02, 68.4MB/s]
Downloading (…)l-00030-of-00033.bin: 57%|█▋ | 231M/405M [00:03<00:02, 71.3MB/s]
Downloading (…)l-00030-of-00033.bin: 60%|█▊ | 241M/405M [00:03<00:02, 71.7MB/s]
Downloading (…)l-00030-of-00033.bin: 62%|█▊ | 252M/405M [00:03<00:02, 74.5MB/s]
Downloading (…)l-00030-of-00033.bin: 65%|█▉ | 262M/405M [00:03<00:01, 75.2MB/s]
Downloading (…)l-00030-of-00033.bin: 67%|██ | 273M/405M [00:03<00:01, 76.7MB/s]
Downloading (…)l-00030-of-00033.bin: 70%|██ | 283M/405M [00:04<00:01, 76.7MB/s]
Downloading (…)l-00030-of-00033.bin: 73%|██▏| 294M/405M [00:04<00:01, 77.5MB/s]
Downloading (…)l-00030-of-00033.bin: 75%|██▎| 304M/405M [00:04<00:01, 73.5MB/s]
Downloading (…)l-00030-of-00033.bin: 78%|██▎| 315M/405M [00:04<00:01, 74.9MB/s]
Downloading (…)l-00030-of-00033.bin: 80%|██▍| 325M/405M [00:04<00:01, 76.6MB/s]
Downloading (…)l-00030-of-00033.bin: 83%|██▍| 336M/405M [00:04<00:00, 76.6MB/s]
Downloading (…)l-00030-of-00033.bin: 85%|██▌| 346M/405M [00:04<00:00, 76.9MB/s]
Downloading (…)l-00030-of-00033.bin: 88%|██▋| 357M/405M [00:05<00:00, 76.5MB/s]
Downloading (…)l-00030-of-00033.bin: 91%|██▋| 367M/405M [00:05<00:00, 78.0MB/s]
Downloading (…)l-00030-of-00033.bin: 93%|██▊| 377M/405M [00:05<00:00, 78.7MB/s]
Downloading (…)l-00030-of-00033.bin: 96%|██▉| 388M/405M [00:05<00:00, 71.4MB/s]
Downloading (…)l-00030-of-00033.bin: 100%|███| 405M/405M [00:05<00:00, 71.3MB/s]
Downloading shards: 91%|████████████████████▉ | 30/33 [02:50<00:16, 5.56s/it]
Downloading (…)l-00031-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00031-of-00033.bin: 3%| | 10.5M/405M [00:00<00:05, 66.1MB/s]
Downloading (…)l-00031-of-00033.bin: 5%| | 21.0M/405M [00:00<00:05, 75.6MB/s]
Downloading (…)l-00031-of-00033.bin: 10%|▎ | 41.9M/405M [00:00<00:03, 103MB/s]
Downloading (…)l-00031-of-00033.bin: 16%|▍ | 62.9M/405M [00:00<00:02, 130MB/s]
Downloading (…)l-00031-of-00033.bin: 21%|▌ | 83.9M/405M [00:00<00:02, 153MB/s]
Downloading (…)l-00031-of-00033.bin: 28%|█▏ | 115M/405M [00:00<00:01, 193MB/s]
Downloading (…)l-00031-of-00033.bin: 36%|█▍ | 147M/405M [00:00<00:01, 216MB/s]
Downloading (…)l-00031-of-00033.bin: 44%|█▊ | 178M/405M [00:01<00:01, 224MB/s]
Downloading (…)l-00031-of-00033.bin: 52%|█▌ | 210M/405M [00:01<00:02, 90.5MB/s]
Downloading (…)l-00031-of-00033.bin: 60%|██▍ | 241M/405M [00:01<00:01, 115MB/s]
Downloading (…)l-00031-of-00033.bin: 67%|██▋ | 273M/405M [00:02<00:00, 135MB/s]
Downloading (…)l-00031-of-00033.bin: 75%|███ | 304M/405M [00:02<00:00, 153MB/s]
Downloading (…)l-00031-of-00033.bin: 83%|███▎| 336M/405M [00:02<00:00, 170MB/s]
Downloading (…)l-00031-of-00033.bin: 91%|███▋| 367M/405M [00:02<00:00, 189MB/s]
Downloading (…)l-00031-of-00033.bin: 100%|████| 405M/405M [00:02<00:00, 151MB/s]
Downloading shards: 94%|█████████████████████▌ | 31/33 [02:53<00:09, 4.76s/it]
Downloading (…)l-00032-of-00033.bin: 0%| | 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00032-of-00033.bin: 3%| | 10.5M/405M [00:00<00:04, 97.4MB/s]
Downloading (…)l-00032-of-00033.bin: 5%|▏ | 21.0M/405M [00:00<00:03, 100MB/s]
Downloading (…)l-00032-of-00033.bin: 8%|▏ | 31.5M/405M [00:00<00:03, 99.2MB/s]
Downloading (…)l-00032-of-00033.bin: 13%|▎ | 52.4M/405M [00:00<00:03, 98.7MB/s]
Downloading (…)l-00032-of-00033.bin: 16%|▎ | 62.9M/405M [00:00<00:03, 98.4MB/s]
Downloading (…)l-00032-of-00033.bin: 21%|▌ | 83.9M/405M [00:00<00:03, 100MB/s]
Downloading (…)l-00032-of-00033.bin: 23%|▋ | 94.4M/405M [00:00<00:03, 101MB/s]
Downloading (…)l-00032-of-00033.bin: 26%|█ | 105M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00032-of-00033.bin: 28%|█▏ | 115M/405M [00:01<00:02, 102MB/s]
Downloading (…)l-00032-of-00033.bin: 31%|█▏ | 126M/405M [00:01<00:02, 102MB/s]
Downloading (…)l-00032-of-00033.bin: 34%|█▎ | 136M/405M [00:01<00:02, 102MB/s]
Downloading (…)l-00032-of-00033.bin: 36%|█▍ | 147M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00032-of-00033.bin: 39%|█▌ | 157M/405M [00:01<00:02, 101MB/s]
Downloading (…)l-00032-of-00033.bin: 41%|█▏ | 168M/405M [00:01<00:02, 97.1MB/s]
Downloading (…)l-00032-of-00033.bin: 44%|█▎ | 178M/405M [00:01<00:02, 98.4MB/s]
Downloading (…)l-00032-of-00033.bin: 47%|█▍ | 189M/405M [00:01<00:02, 99.4MB/s]
Downloading (…)l-00032-of-00033.bin: 49%|█▍ | 199M/405M [00:01<00:02, 99.8MB/s]
Downloading (…)l-00032-of-00033.bin: 52%|██ | 210M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00032-of-00033.bin: 54%|█▋ | 220M/405M [00:02<00:01, 99.6MB/s]
Downloading (…)l-00032-of-00033.bin: 57%|█▋ | 231M/405M [00:02<00:01, 99.6MB/s]
Downloading (…)l-00032-of-00033.bin: 60%|█▊ | 241M/405M [00:02<00:01, 98.2MB/s]
Downloading (…)l-00032-of-00033.bin: 62%|█▊ | 252M/405M [00:02<00:01, 96.6MB/s]
Downloading (…)l-00032-of-00033.bin: 67%|██▋ | 273M/405M [00:02<00:01, 100MB/s]
Downloading (…)l-00032-of-00033.bin: 70%|██ | 283M/405M [00:02<00:01, 97.7MB/s]
Downloading (…)l-00032-of-00033.bin: 73%|██▏| 294M/405M [00:02<00:01, 97.4MB/s]
Downloading (…)l-00032-of-00033.bin: 75%|██▎| 304M/405M [00:03<00:01, 97.1MB/s]
Downloading (…)l-00032-of-00033.bin: 78%|██▎| 315M/405M [00:03<00:00, 98.1MB/s]
Downloading (…)l-00032-of-00033.bin: 80%|██▍| 325M/405M [00:03<00:00, 96.8MB/s]
Downloading (…)l-00032-of-00033.bin: 83%|██▍| 336M/405M [00:03<00:00, 98.0MB/s]
Downloading (…)l-00032-of-00033.bin: 85%|██▌| 346M/405M [00:03<00:00, 95.0MB/s]
Downloading (…)l-00032-of-00033.bin: 88%|██▋| 357M/405M [00:03<00:00, 97.0MB/s]
Downloading (…)l-00032-of-00033.bin: 91%|██▋| 367M/405M [00:03<00:00, 98.4MB/s]
Downloading (…)l-00032-of-00033.bin: 93%|██▊| 377M/405M [00:03<00:00, 99.1MB/s]
Downloading (…)l-00032-of-00033.bin: 96%|██▉| 388M/405M [00:03<00:00, 93.2MB/s]
Downloading (…)l-00032-of-00033.bin: 100%|███| 405M/405M [00:04<00:00, 98.2MB/s]
Downloading shards: 97%|██████████████████████▎| 32/33 [02:58<00:04, 4.75s/it]
Downloading (…)l-00033-of-00033.bin: 0%| | 0.00/524M [00:00<?, ?B/s]
Downloading (…)l-00033-of-00033.bin: 2%| | 10.5M/524M [00:00<00:05, 96.3MB/s]
Downloading (…)l-00033-of-00033.bin: 4%| | 21.0M/524M [00:00<00:05, 99.0MB/s]
Downloading (…)l-00033-of-00033.bin: 6%|▏ | 31.5M/524M [00:00<00:04, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 8%|▏ | 41.9M/524M [00:00<00:04, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 10%|▎ | 52.4M/524M [00:00<00:04, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 12%|▎ | 62.9M/524M [00:00<00:04, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 14%|▍ | 73.4M/524M [00:00<00:04, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 16%|▍ | 83.9M/524M [00:00<00:04, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 18%|▌ | 94.4M/524M [00:00<00:04, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 20%|▊ | 105M/524M [00:01<00:04, 102MB/s]
Downloading (…)l-00033-of-00033.bin: 22%|▉ | 115M/524M [00:01<00:04, 102MB/s]
Downloading (…)l-00033-of-00033.bin: 24%|▉ | 126M/524M [00:01<00:03, 102MB/s]
Downloading (…)l-00033-of-00033.bin: 26%|█ | 136M/524M [00:01<00:03, 102MB/s]
Downloading (…)l-00033-of-00033.bin: 28%|█ | 147M/524M [00:01<00:03, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 30%|█▏ | 157M/524M [00:01<00:03, 102MB/s]
Downloading (…)l-00033-of-00033.bin: 32%|█▎ | 168M/524M [00:01<00:03, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 34%|█▎ | 178M/524M [00:01<00:03, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 36%|█▍ | 189M/524M [00:01<00:03, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 38%|█▌ | 199M/524M [00:01<00:03, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 40%|█▌ | 210M/524M [00:02<00:03, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 42%|█▋ | 220M/524M [00:02<00:03, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 44%|█▊ | 231M/524M [00:02<00:02, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 46%|█▊ | 241M/524M [00:02<00:02, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 48%|█▉ | 252M/524M [00:02<00:02, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 50%|█▉ | 262M/524M [00:02<00:02, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 52%|██ | 273M/524M [00:02<00:02, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 54%|██▏ | 283M/524M [00:02<00:02, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 56%|██▏ | 294M/524M [00:02<00:02, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 58%|██▎ | 304M/524M [00:03<00:02, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 60%|██▍ | 315M/524M [00:03<00:02, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 62%|██▍ | 325M/524M [00:03<00:01, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 64%|██▌ | 336M/524M [00:03<00:01, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 66%|██▋ | 346M/524M [00:03<00:01, 102MB/s]
Downloading (…)l-00033-of-00033.bin: 68%|██▋ | 357M/524M [00:03<00:01, 102MB/s]
Downloading (…)l-00033-of-00033.bin: 70%|██▊ | 367M/524M [00:03<00:01, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 72%|██▏| 377M/524M [00:03<00:01, 99.8MB/s]
Downloading (…)l-00033-of-00033.bin: 74%|██▉ | 388M/524M [00:03<00:01, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 76%|███ | 398M/524M [00:03<00:01, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 78%|███ | 409M/524M [00:04<00:01, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 80%|███▏| 419M/524M [00:04<00:01, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 82%|███▎| 430M/524M [00:04<00:00, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 84%|███▎| 440M/524M [00:04<00:00, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 86%|███▍| 451M/524M [00:04<00:00, 100MB/s]
Downloading (…)l-00033-of-00033.bin: 88%|███▌| 461M/524M [00:04<00:00, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 90%|██▋| 472M/524M [00:04<00:00, 97.8MB/s]
Downloading (…)l-00033-of-00033.bin: 94%|███▊| 493M/524M [00:04<00:00, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 96%|███▊| 503M/524M [00:04<00:00, 101MB/s]
Downloading (…)l-00033-of-00033.bin: 98%|██▉| 514M/524M [00:05<00:00, 98.7MB/s]
Downloading (…)l-00033-of-00033.bin: 100%|████| 524M/524M [00:05<00:00, 101MB/s]
Downloading shards: 100%|███████████████████████| 33/33 [03:03<00:00, 5.57s/it]
Loading checkpoint shards: 100%|████████████████| 33/33 [00:07<00:00, 4.28it/s]
Downloading (…)neration_config.json: 100%|██████| 124/124 [00:00<00:00, 121kB/s]
Downloading tokenizer.model: 100%|███████████| 500k/500k [00:00<00:00, 20.3MB/s]
Downloading (…)cial_tokens_map.json: 100%|███| 2.00/2.00 [00:00<00:00, 1.88kB/s]
Downloading (…)okenizer_config.json: 100%|█████| 141/141 [00:00<00:00, 73.6kB/s]
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
trainable params: 4194304 || all params: 6742609920 || trainable%: 0.06220594176090199
***** Running training *****
Num Epochs = 1
Instantaneous batch size per GPU = 4
Gradient Accumulation steps = 32
Total train batch size (w. parallel, distributed & accumulation) = 128
Total optimization steps = 390
Saving steps = 39
/usr/local/lib/python3.10/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning
warnings.warn(
wandb: Currently logged in as: utensil. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.14.2
wandb: Run data is saved locally in /workspace/code/wandb/run-20230419_040758-bc95sx2m
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run genial-puddle-1
wandb: ⭐️ View project at https://wandb.ai/utensil/Alpaca-CoT
wandb: 🚀 View run at https://wandb.ai/utensil/Alpaca-CoT/runs/bc95sx2m
{'loss': 1.4114, 'learning_rate': 0.00015384615384615382, 'epoch': 0.05}
10%|████ | 39/390 [17:32<2:39:37, 27.29s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:48, 5.16it/s]
1%|▌ | 3/250 [00:00<01:04, 3.85it/s]
2%|▋ | 4/250 [00:01<02:00, 2.04it/s]
2%|▊ | 5/250 [00:02<01:53, 2.16it/s]
2%|█ | 6/250 [00:02<01:57, 2.07it/s]
3%|█▏ | 7/250 [00:02<01:49, 2.21it/s]
3%|█▍ | 8/250 [00:03<01:40, 2.42it/s]
4%|█▌ | 9/250 [00:04<02:13, 1.80it/s]
4%|█▋ | 10/250 [00:04<01:59, 2.00it/s]
4%|█▊ | 11/250 [00:04<01:53, 2.10it/s]
5%|██ | 12/250 [00:05<01:49, 2.17it/s]
5%|██▏ | 13/250 [00:05<01:40, 2.35it/s]
6%|██▎ | 14/250 [00:06<01:46, 2.23it/s]
6%|██▌ | 15/250 [00:06<01:46, 2.22it/s]
6%|██▋ | 16/250 [00:07<01:52, 2.08it/s]
7%|██▊ | 17/250 [00:07<01:57, 1.99it/s]
7%|███ | 18/250 [00:08<02:02, 1.89it/s]
8%|███▏ | 19/250 [00:08<01:58, 1.95it/s]
8%|███▎ | 20/250 [00:09<01:50, 2.09it/s]
8%|███▌ | 21/250 [00:09<01:55, 1.99it/s]
9%|███▋ | 22/250 [00:10<01:43, 2.21it/s]
9%|███▊ | 23/250 [00:10<01:30, 2.51it/s]
10%|████ | 24/250 [00:10<01:26, 2.61it/s]
10%|████▏ | 25/250 [00:11<01:23, 2.69it/s]
10%|████▎ | 26/250 [00:11<01:37, 2.29it/s]
11%|████▌ | 27/250 [00:11<01:28, 2.53it/s]
11%|████▋ | 28/250 [00:12<01:20, 2.75it/s]
12%|████▊ | 29/250 [00:12<01:14, 2.95it/s]
12%|█████ | 30/250 [00:12<01:13, 3.01it/s]
12%|█████▏ | 31/250 [00:13<01:11, 3.08it/s]
13%|█████▍ | 32/250 [00:13<01:38, 2.22it/s]
13%|█████▌ | 33/250 [00:14<01:31, 2.36it/s]
14%|█████▋ | 34/250 [00:14<01:29, 2.41it/s]
14%|█████▉ | 35/250 [00:15<01:25, 2.52it/s]
14%|██████ | 36/250 [00:15<01:55, 1.86it/s]
15%|██████▏ | 37/250 [00:16<01:42, 2.07it/s]
15%|██████▍ | 38/250 [00:16<01:36, 2.19it/s]
16%|██████▌ | 39/250 [00:17<01:37, 2.17it/s]
16%|██████▋ | 40/250 [00:17<01:28, 2.38it/s]
16%|██████▉ | 41/250 [00:18<01:44, 2.01it/s]
17%|███████ | 42/250 [00:18<01:37, 2.13it/s]
17%|███████▏ | 43/250 [00:18<01:31, 2.25it/s]
18%|███████▍ | 44/250 [00:19<01:35, 2.15it/s]
18%|███████▌ | 45/250 [00:19<01:40, 2.04it/s]
18%|███████▋ | 46/250 [00:20<01:29, 2.29it/s]
19%|███████▉ | 47/250 [00:20<01:22, 2.45it/s]
19%|████████ | 48/250 [00:21<01:25, 2.37it/s]
20%|████████▏ | 49/250 [00:21<01:32, 2.17it/s]
20%|████████▍ | 50/250 [00:21<01:22, 2.44it/s]
20%|████████▌ | 51/250 [00:22<01:23, 2.38it/s]
21%|████████▋ | 52/250 [00:22<01:17, 2.56it/s]
21%|████████▉ | 53/250 [00:23<01:13, 2.68it/s]
22%|█████████ | 54/250 [00:23<01:13, 2.67it/s]
22%|█████████▏ | 55/250 [00:24<01:42, 1.91it/s]
22%|█████████▍ | 56/250 [00:24<01:32, 2.09it/s]
23%|█████████▌ | 57/250 [00:24<01:21, 2.36it/s]
23%|█████████▋ | 58/250 [00:25<01:20, 2.40it/s]
24%|█████████▉ | 59/250 [00:25<01:20, 2.38it/s]
24%|██████████ | 60/250 [00:26<01:15, 2.52it/s]
24%|██████████▏ | 61/250 [00:26<01:10, 2.68it/s]
25%|██████████▍ | 62/250 [00:26<01:14, 2.52it/s]
25%|██████████▌ | 63/250 [00:27<01:30, 2.07it/s]
26%|██████████▊ | 64/250 [00:27<01:23, 2.23it/s]
26%|██████████▉ | 65/250 [00:28<01:18, 2.35it/s]
26%|███████████ | 66/250 [00:28<01:13, 2.52it/s]
27%|███████████▎ | 67/250 [00:28<01:09, 2.62it/s]
27%|███████████▍ | 68/250 [00:29<01:09, 2.61it/s]
28%|███████████▌ | 69/250 [00:29<01:06, 2.73it/s]
28%|███████████▊ | 70/250 [00:30<01:06, 2.71it/s]
28%|███████████▉ | 71/250 [00:30<01:07, 2.67it/s]
29%|████████████ | 72/250 [00:30<01:07, 2.63it/s]
29%|████████████▎ | 73/250 [00:31<01:06, 2.68it/s]
30%|████████████▍ | 74/250 [00:31<01:08, 2.58it/s]
30%|████████████▌ | 75/250 [00:31<01:03, 2.74it/s]
30%|████████████▊ | 76/250 [00:32<01:00, 2.90it/s]
31%|████████████▉ | 77/250 [00:32<01:08, 2.52it/s]
31%|█████████████ | 78/250 [00:33<01:05, 2.61it/s]
32%|█████████████▎ | 79/250 [00:33<01:30, 1.89it/s]
32%|█████████████▍ | 80/250 [00:34<01:21, 2.08it/s]
32%|█████████████▌ | 81/250 [00:34<01:19, 2.12it/s]
33%|█████████████▊ | 82/250 [00:35<01:14, 2.24it/s]
33%|█████████████▉ | 83/250 [00:35<01:10, 2.37it/s]
34%|██████████████ | 84/250 [00:35<01:05, 2.54it/s]
34%|██████████████▎ | 85/250 [00:36<01:21, 2.01it/s]
34%|██████████████▍ | 86/250 [00:36<01:14, 2.19it/s]
35%|██████████████▌ | 87/250 [00:37<01:16, 2.13it/s]
35%|██████████████▊ | 88/250 [00:37<01:08, 2.36it/s]
36%|██████████████▉ | 89/250 [00:38<01:14, 2.16it/s]
36%|███████████████ | 90/250 [00:38<01:09, 2.29it/s]
36%|███████████████▎ | 91/250 [00:39<01:13, 2.16it/s]
37%|███████████████▍ | 92/250 [00:39<01:06, 2.36it/s]
37%|███████████████▌ | 93/250 [00:39<01:00, 2.60it/s]
38%|███████████████▊ | 94/250 [00:40<01:09, 2.25it/s]
38%|███████████████▉ | 95/250 [00:40<01:04, 2.39it/s]
38%|████████████████▏ | 96/250 [00:41<01:03, 2.42it/s]
39%|████████████████▎ | 97/250 [00:41<01:01, 2.49it/s]
39%|████████████████▍ | 98/250 [00:41<01:00, 2.51it/s]
40%|████████████████▋ | 99/250 [00:42<00:56, 2.66it/s]
40%|████████████████▍ | 100/250 [00:42<00:56, 2.66it/s]
40%|████████████████▌ | 101/250 [00:42<00:53, 2.81it/s]
41%|████████████████▋ | 102/250 [00:43<01:00, 2.46it/s]
41%|████████████████▉ | 103/250 [00:43<00:58, 2.51it/s]
42%|█████████████████ | 104/250 [00:44<01:02, 2.34it/s]
42%|█████████████████▏ | 105/250 [00:44<00:58, 2.46it/s]
42%|█████████████████▍ | 106/250 [00:45<00:56, 2.56it/s]
43%|█████████████████▌ | 107/250 [00:45<00:55, 2.60it/s]
43%|█████████████████▋ | 108/250 [00:45<00:52, 2.69it/s]
44%|█████████████████▉ | 109/250 [00:46<00:51, 2.75it/s]
44%|██████████████████ | 110/250 [00:46<00:53, 2.60it/s]
44%|██████████████████▏ | 111/250 [00:47<00:56, 2.48it/s]
45%|██████████████████▎ | 112/250 [00:47<00:53, 2.57it/s]
45%|██████████████████▌ | 113/250 [00:47<00:48, 2.80it/s]
46%|██████████████████▋ | 114/250 [00:48<00:47, 2.87it/s]
46%|██████████████████▊ | 115/250 [00:48<00:44, 3.02it/s]
46%|███████████████████ | 116/250 [00:48<00:46, 2.89it/s]
47%|███████████████████▏ | 117/250 [00:49<00:47, 2.80it/s]
47%|███████████████████▎ | 118/250 [00:49<00:45, 2.91it/s]
48%|███████████████████▌ | 119/250 [00:49<00:47, 2.78it/s]
48%|███████████████████▋ | 120/250 [00:50<00:44, 2.89it/s]
48%|███████████████████▊ | 121/250 [00:50<00:43, 2.99it/s]
49%|████████████████████ | 122/250 [00:50<00:41, 3.06it/s]
49%|████████████████████▏ | 123/250 [00:51<00:45, 2.82it/s]
50%|████████████████████▎ | 124/250 [00:51<00:43, 2.91it/s]
50%|████████████████████▌ | 125/250 [00:51<00:50, 2.48it/s]
50%|████████████████████▋ | 126/250 [00:52<00:47, 2.62it/s]
51%|████████████████████▊ | 127/250 [00:52<00:42, 2.87it/s]
51%|████████████████████▉ | 128/250 [00:52<00:41, 2.95it/s]
52%|█████████████████████▏ | 129/250 [00:53<00:56, 2.13it/s]
52%|█████████████████████▎ | 130/250 [00:54<00:55, 2.17it/s]
52%|█████████████████████▍ | 131/250 [00:54<00:56, 2.11it/s]
53%|█████████████████████▋ | 132/250 [00:55<00:58, 2.01it/s]
53%|█████████████████████▊ | 133/250 [00:55<00:52, 2.25it/s]
54%|█████████████████████▉ | 134/250 [00:55<00:51, 2.24it/s]
54%|██████████████████████▏ | 135/250 [00:56<00:47, 2.41it/s]
54%|██████████████████████▎ | 136/250 [00:56<00:48, 2.35it/s]
55%|██████████████████████▍ | 137/250 [00:57<00:48, 2.34it/s]
55%|██████████████████████▋ | 138/250 [00:57<00:47, 2.37it/s]
56%|██████████████████████▊ | 139/250 [00:57<00:45, 2.46it/s]
56%|██████████████████████▉ | 140/250 [00:58<00:42, 2.60it/s]
56%|███████████████████████ | 141/250 [00:58<00:40, 2.67it/s]
57%|███████████████████████▎ | 142/250 [00:58<00:39, 2.74it/s]
57%|███████████████████████▍ | 143/250 [00:59<00:39, 2.70it/s]
58%|███████████████████████▌ | 144/250 [00:59<00:36, 2.89it/s]
58%|███████████████████████▊ | 145/250 [01:00<00:42, 2.50it/s]
58%|███████████████████████▉ | 146/250 [01:00<00:47, 2.18it/s]
59%|████████████████████████ | 147/250 [01:01<00:42, 2.40it/s]
59%|████████████████████████▎ | 148/250 [01:01<00:49, 2.07it/s]
60%|████████████████████████▍ | 149/250 [01:02<00:44, 2.26it/s]
60%|████████████████████████▌ | 150/250 [01:02<00:43, 2.32it/s]
60%|████████████████████████▊ | 151/250 [01:02<00:39, 2.50it/s]
61%|████████████████████████▉ | 152/250 [01:03<00:37, 2.59it/s]
61%|█████████████████████████ | 153/250 [01:03<00:35, 2.72it/s]
62%|█████████████████████████▎ | 154/250 [01:03<00:32, 2.91it/s]
62%|█████████████████████████▍ | 155/250 [01:04<00:36, 2.61it/s]
62%|█████████████████████████▌ | 156/250 [01:04<00:40, 2.32it/s]
63%|█████████████████████████▋ | 157/250 [01:05<00:39, 2.37it/s]
63%|█████████████████████████▉ | 158/250 [01:05<00:36, 2.53it/s]
64%|██████████████████████████ | 159/250 [01:05<00:35, 2.55it/s]
64%|██████████████████████████▏ | 160/250 [01:06<00:33, 2.66it/s]
64%|██████████████████████████▍ | 161/250 [01:06<00:32, 2.74it/s]
65%|██████████████████████████▌ | 162/250 [01:06<00:30, 2.84it/s]
65%|██████████████████████████▋ | 163/250 [01:07<00:29, 2.96it/s]
66%|██████████████████████████▉ | 164/250 [01:07<00:28, 3.04it/s]
66%|███████████████████████████ | 165/250 [01:08<00:41, 2.04it/s]
66%|███████████████████████████▏ | 166/250 [01:08<00:38, 2.20it/s]
67%|███████████████████████████▍ | 167/250 [01:09<00:36, 2.28it/s]
67%|███████████████████████████▌ | 168/250 [01:09<00:33, 2.47it/s]
68%|███████████████████████████▋ | 169/250 [01:09<00:33, 2.45it/s]
68%|███████████████████████████▉ | 170/250 [01:10<00:32, 2.46it/s]
68%|████████████████████████████ | 171/250 [01:10<00:30, 2.55it/s]
69%|████████████████████████████▏ | 172/250 [01:11<00:31, 2.50it/s]
69%|████████████████████████████▎ | 173/250 [01:11<00:32, 2.40it/s]
70%|████████████████████████████▌ | 174/250 [01:11<00:30, 2.52it/s]
70%|████████████████████████████▋ | 175/250 [01:12<00:30, 2.43it/s]
70%|████████████████████████████▊ | 176/250 [01:12<00:34, 2.15it/s]
71%|█████████████████████████████ | 177/250 [01:13<00:32, 2.25it/s]
71%|█████████████████████████████▏ | 178/250 [01:13<00:28, 2.54it/s]
72%|█████████████████████████████▎ | 179/250 [01:13<00:26, 2.72it/s]
72%|█████████████████████████████▌ | 180/250 [01:14<00:25, 2.75it/s]
72%|█████████████████████████████▋ | 181/250 [01:14<00:32, 2.09it/s]
73%|█████████████████████████████▊ | 182/250 [01:15<00:34, 2.00it/s]
73%|██████████████████████████████ | 183/250 [01:16<00:33, 2.02it/s]
74%|██████████████████████████████▏ | 184/250 [01:16<00:30, 2.20it/s]
74%|██████████████████████████████▎ | 185/250 [01:16<00:27, 2.38it/s]
74%|██████████████████████████████▌ | 186/250 [01:17<00:24, 2.62it/s]
75%|██████████████████████████████▋ | 187/250 [01:17<00:28, 2.23it/s]
75%|██████████████████████████████▊ | 188/250 [01:18<00:28, 2.17it/s]
76%|██████████████████████████████▉ | 189/250 [01:18<00:25, 2.43it/s]
76%|███████████████████████████████▏ | 190/250 [01:18<00:25, 2.34it/s]
76%|███████████████████████████████▎ | 191/250 [01:19<00:30, 1.92it/s]
77%|███████████████████████████████▍ | 192/250 [01:19<00:26, 2.20it/s]
77%|███████████████████████████████▋ | 193/250 [01:20<00:23, 2.39it/s]
78%|███████████████████████████████▊ | 194/250 [01:20<00:21, 2.59it/s]
78%|███████████████████████████████▉ | 195/250 [01:20<00:20, 2.65it/s]
78%|████████████████████████████████▏ | 196/250 [01:21<00:20, 2.64it/s]
79%|████████████████████████████████▎ | 197/250 [01:21<00:23, 2.23it/s]
79%|████████████████████████████████▍ | 198/250 [01:22<00:27, 1.88it/s]
80%|████████████████████████████████▋ | 199/250 [01:22<00:24, 2.10it/s]
80%|████████████████████████████████▊ | 200/250 [01:23<00:21, 2.31it/s]
80%|████████████████████████████████▉ | 201/250 [01:23<00:19, 2.45it/s]
81%|█████████████████████████████████▏ | 202/250 [01:24<00:19, 2.47it/s]
81%|█████████████████████████████████▎ | 203/250 [01:24<00:19, 2.35it/s]
82%|█████████████████████████████████▍ | 204/250 [01:24<00:18, 2.53it/s]
82%|█████████████████████████████████▌ | 205/250 [01:25<00:20, 2.22it/s]
82%|█████████████████████████████████▊ | 206/250 [01:25<00:18, 2.32it/s]
83%|█████████████████████████████████▉ | 207/250 [01:26<00:19, 2.18it/s]
83%|██████████████████████████████████ | 208/250 [01:26<00:18, 2.33it/s]
84%|██████████████████████████████████▎ | 209/250 [01:27<00:19, 2.06it/s]
84%|██████████████████████████████████▍ | 210/250 [01:27<00:20, 1.94it/s]
84%|██████████████████████████████████▌ | 211/250 [01:28<00:20, 1.89it/s]
85%|██████████████████████████████████▊ | 212/250 [01:28<00:19, 1.92it/s]
85%|██████████████████████████████████▉ | 213/250 [01:29<00:17, 2.07it/s]
86%|███████████████████████████████████ | 214/250 [01:29<00:16, 2.15it/s]
86%|███████████████████████████████████▎ | 215/250 [01:30<00:18, 1.93it/s]
86%|███████████████████████████████████▍ | 216/250 [01:30<00:16, 2.09it/s]
87%|███████████████████████████████████▌ | 217/250 [01:31<00:15, 2.13it/s]
87%|███████████████████████████████████▊ | 218/250 [01:31<00:16, 1.97it/s]
88%|███████████████████████████████████▉ | 219/250 [01:32<00:14, 2.11it/s]
88%|████████████████████████████████████ | 220/250 [01:32<00:13, 2.23it/s]
88%|████████████████████████████████████▏ | 221/250 [01:32<00:12, 2.40it/s]
89%|████████████████████████████████████▍ | 222/250 [01:33<00:11, 2.43it/s]
89%|████████████████████████████████████▌ | 223/250 [01:33<00:12, 2.14it/s]
90%|████████████████████████████████████▋ | 224/250 [01:34<00:11, 2.30it/s]
90%|████████████████████████████████████▉ | 225/250 [01:34<00:09, 2.51it/s]
90%|█████████████████████████████████████ | 226/250 [01:35<00:09, 2.43it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:35<00:08, 2.63it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:35<00:08, 2.75it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:36<00:07, 2.94it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:36<00:06, 3.07it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:36<00:06, 2.78it/s]
93%|██████████████████████████████████████ | 232/250 [01:37<00:06, 2.68it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:37<00:06, 2.81it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:37<00:05, 2.93it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:38<00:04, 3.05it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:38<00:04, 2.93it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:38<00:04, 2.90it/s]
95%|███████████████████████████████████████ | 238/250 [01:39<00:04, 2.91it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:39<00:03, 2.83it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:40<00:04, 2.43it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:40<00:03, 2.57it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:41<00:03, 2.21it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:41<00:03, 2.18it/s]
98%|████████████████████████████████████████ | 244/250 [01:41<00:02, 2.32it/s]
98%|████████████████████████████████████████▏| 245/250 [01:42<00:02, 2.29it/s]
98%|████████████████████████████████████████▎| 246/250 [01:42<00:01, 2.54it/s]
99%|████████████████████████████████████████▌| 247/250 [01:43<00:01, 2.43it/s]
99%|████████████████████████████████████████▋| 248/250 [01:43<00:00, 2.40it/s]
100%|████████████████████████████████████████▊| 249/250 [01:43<00:00, 2.57it/s]
100%|█████████████████████████████████████████| 250/250 [01:44<00:00, 2.29it/s]
{'eval_loss': 1.1480391025543213, 'eval_runtime': 105.0239, 'eval_samples_per_second': 19.043, 'eval_steps_per_second': 2.38, 'epoch': 0.1}
10%|████ | 39/390 [19:17<2:39:37, 27.29s/it]
{'loss': 1.1733, 'learning_rate': 0.00029914529914529915, 'epoch': 0.1}
10%|████ | 40/390 [19:44<5:43:10, 58.83s/it]wandb: Network error (ReadTimeout), entering retry loop.
{'loss': 1.1086, 'learning_rate': 0.00028205128205128203, 'epoch': 0.15}
20%|████████ | 78/390 [36:45<2:19:33, 26.84s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:48, 5.15it/s]
1%|▌ | 3/250 [00:00<01:04, 3.85it/s]
2%|▋ | 4/250 [00:01<02:00, 2.04it/s]
2%|▊ | 5/250 [00:02<01:53, 2.15it/s]
2%|█ | 6/250 [00:02<01:57, 2.07it/s]
3%|█▏ | 7/250 [00:02<01:49, 2.21it/s]
3%|█▍ | 8/250 [00:03<01:40, 2.42it/s]
4%|█▌ | 9/250 [00:04<02:14, 1.80it/s]
4%|█▋ | 10/250 [00:04<01:59, 2.00it/s]
4%|█▊ | 11/250 [00:04<01:54, 2.10it/s]
5%|██ | 12/250 [00:05<01:49, 2.17it/s]
5%|██▏ | 13/250 [00:05<01:41, 2.35it/s]
6%|██▎ | 14/250 [00:06<01:48, 2.18it/s]
6%|██▌ | 15/250 [00:06<01:47, 2.18it/s]
6%|██▋ | 16/250 [00:07<01:53, 2.06it/s]
7%|██▊ | 17/250 [00:07<01:57, 1.98it/s]
7%|███ | 18/250 [00:08<02:03, 1.88it/s]
8%|███▏ | 19/250 [00:08<01:59, 1.94it/s]
8%|███▎ | 20/250 [00:09<01:50, 2.08it/s]
8%|███▌ | 21/250 [00:09<01:55, 1.98it/s]
9%|███▋ | 22/250 [00:10<01:43, 2.20it/s]
9%|███▊ | 23/250 [00:10<01:30, 2.50it/s]
10%|████ | 24/250 [00:10<01:26, 2.61it/s]
10%|████▏ | 25/250 [00:11<01:23, 2.69it/s]
10%|████▎ | 26/250 [00:11<01:37, 2.29it/s]
11%|████▌ | 27/250 [00:12<01:27, 2.54it/s]
11%|████▋ | 28/250 [00:12<01:20, 2.75it/s]
12%|████▊ | 29/250 [00:12<01:14, 2.95it/s]
12%|█████ | 30/250 [00:12<01:13, 3.00it/s]
12%|█████▏ | 31/250 [00:13<01:11, 3.07it/s]
13%|█████▍ | 32/250 [00:13<01:38, 2.22it/s]
13%|█████▌ | 33/250 [00:14<01:32, 2.36it/s]
14%|█████▋ | 34/250 [00:14<01:29, 2.40it/s]
14%|█████▉ | 35/250 [00:15<01:25, 2.52it/s]
14%|██████ | 36/250 [00:15<01:55, 1.85it/s]
15%|██████▏ | 37/250 [00:16<01:43, 2.06it/s]
15%|██████▍ | 38/250 [00:16<01:37, 2.18it/s]
16%|██████▌ | 39/250 [00:17<01:37, 2.15it/s]
16%|██████▋ | 40/250 [00:17<01:28, 2.38it/s]
16%|██████▉ | 41/250 [00:18<01:44, 2.00it/s]
17%|███████ | 42/250 [00:18<01:38, 2.12it/s]
17%|███████▏ | 43/250 [00:18<01:32, 2.24it/s]
18%|███████▍ | 44/250 [00:19<01:36, 2.14it/s]
18%|███████▌ | 45/250 [00:20<01:40, 2.03it/s]
18%|███████▋ | 46/250 [00:20<01:29, 2.28it/s]
19%|███████▉ | 47/250 [00:20<01:23, 2.45it/s]
19%|████████ | 48/250 [00:21<01:25, 2.37it/s]
20%|████████▏ | 49/250 [00:21<01:33, 2.16it/s]
20%|████████▍ | 50/250 [00:22<01:22, 2.43it/s]
20%|████████▌ | 51/250 [00:22<01:24, 2.37it/s]
21%|████████▋ | 52/250 [00:22<01:17, 2.56it/s]
21%|████████▉ | 53/250 [00:23<01:13, 2.67it/s]
22%|█████████ | 54/250 [00:23<01:13, 2.66it/s]
22%|█████████▏ | 55/250 [00:24<01:42, 1.91it/s]
22%|█████████▍ | 56/250 [00:24<01:32, 2.09it/s]
23%|█████████▌ | 57/250 [00:25<01:22, 2.35it/s]
23%|█████████▋ | 58/250 [00:25<01:20, 2.38it/s]
24%|█████████▉ | 59/250 [00:25<01:20, 2.36it/s]
24%|██████████ | 60/250 [00:26<01:15, 2.50it/s]
24%|██████████▏ | 61/250 [00:26<01:10, 2.67it/s]
25%|██████████▍ | 62/250 [00:26<01:14, 2.51it/s]
25%|██████████▌ | 63/250 [00:27<01:30, 2.06it/s]
26%|██████████▊ | 64/250 [00:28<01:23, 2.23it/s]
26%|██████████▉ | 65/250 [00:28<01:18, 2.34it/s]
26%|███████████ | 66/250 [00:28<01:13, 2.51it/s]
27%|███████████▎ | 67/250 [00:29<01:10, 2.61it/s]
27%|███████████▍ | 68/250 [00:29<01:09, 2.60it/s]
28%|███████████▌ | 69/250 [00:29<01:06, 2.72it/s]
28%|███████████▊ | 70/250 [00:30<01:06, 2.70it/s]
28%|███████████▉ | 71/250 [00:30<01:07, 2.66it/s]
29%|████████████ | 72/250 [00:30<01:07, 2.62it/s]
29%|████████████▎ | 73/250 [00:31<01:06, 2.67it/s]
30%|████████████▍ | 74/250 [00:31<01:08, 2.58it/s]
30%|████████████▌ | 75/250 [00:32<01:04, 2.73it/s]
30%|████████████▊ | 76/250 [00:32<01:00, 2.89it/s]
31%|████████████▉ | 77/250 [00:32<01:08, 2.52it/s]
31%|█████████████ | 78/250 [00:33<01:06, 2.60it/s]
32%|█████████████▎ | 79/250 [00:34<01:30, 1.88it/s]
32%|█████████████▍ | 80/250 [00:34<01:22, 2.07it/s]
32%|█████████████▌ | 81/250 [00:34<01:19, 2.12it/s]
33%|█████████████▊ | 82/250 [00:35<01:14, 2.24it/s]
33%|█████████████▉ | 83/250 [00:35<01:10, 2.37it/s]
34%|██████████████ | 84/250 [00:35<01:05, 2.53it/s]
34%|██████████████▎ | 85/250 [00:36<01:24, 1.96it/s]
34%|██████████████▍ | 86/250 [00:37<01:16, 2.15it/s]
35%|██████████████▌ | 87/250 [00:37<01:17, 2.10it/s]
35%|██████████████▊ | 88/250 [00:37<01:09, 2.34it/s]
36%|██████████████▉ | 89/250 [00:38<01:15, 2.14it/s]
36%|███████████████ | 90/250 [00:38<01:10, 2.27it/s]
36%|███████████████▎ | 91/250 [00:39<01:14, 2.14it/s]
37%|███████████████▍ | 92/250 [00:39<01:07, 2.35it/s]
37%|███████████████▌ | 93/250 [00:40<01:00, 2.59it/s]
38%|███████████████▊ | 94/250 [00:40<01:09, 2.24it/s]
38%|███████████████▉ | 95/250 [00:40<01:05, 2.38it/s]
38%|████████████████▏ | 96/250 [00:41<01:03, 2.41it/s]
39%|████████████████▎ | 97/250 [00:41<01:01, 2.48it/s]
39%|████████████████▍ | 98/250 [00:42<01:00, 2.51it/s]
40%|████████████████▋ | 99/250 [00:42<00:56, 2.66it/s]
40%|████████████████▍ | 100/250 [00:42<00:56, 2.66it/s]
40%|████████████████▌ | 101/250 [00:43<00:53, 2.81it/s]
41%|████████████████▋ | 102/250 [00:43<01:00, 2.46it/s]
41%|████████████████▉ | 103/250 [00:44<00:58, 2.51it/s]
42%|█████████████████ | 104/250 [00:44<01:02, 2.33it/s]
42%|█████████████████▏ | 105/250 [00:44<00:58, 2.46it/s]
42%|█████████████████▍ | 106/250 [00:45<00:56, 2.56it/s]
43%|█████████████████▌ | 107/250 [00:45<00:55, 2.59it/s]
43%|█████████████████▋ | 108/250 [00:45<00:52, 2.69it/s]
44%|█████████████████▉ | 109/250 [00:46<00:51, 2.74it/s]
44%|██████████████████ | 110/250 [00:46<00:53, 2.60it/s]
44%|██████████████████▏ | 111/250 [00:47<00:56, 2.48it/s]
45%|██████████████████▎ | 112/250 [00:47<00:53, 2.56it/s]
45%|██████████████████▌ | 113/250 [00:47<00:49, 2.80it/s]
46%|██████████████████▋ | 114/250 [00:48<00:47, 2.86it/s]
46%|██████████████████▊ | 115/250 [00:48<00:44, 3.01it/s]
46%|███████████████████ | 116/250 [00:48<00:46, 2.89it/s]
47%|███████████████████▏ | 117/250 [00:49<00:47, 2.79it/s]
47%|███████████████████▎ | 118/250 [00:49<00:45, 2.90it/s]
48%|███████████████████▌ | 119/250 [00:49<00:47, 2.77it/s]
48%|███████████████████▋ | 120/250 [00:50<00:45, 2.88it/s]
48%|███████████████████▊ | 121/250 [00:50<00:43, 2.98it/s]
49%|████████████████████ | 122/250 [00:50<00:41, 3.05it/s]
49%|████████████████████▏ | 123/250 [00:51<00:45, 2.82it/s]
50%|████████████████████▎ | 124/250 [00:51<00:43, 2.90it/s]
50%|████████████████████▌ | 125/250 [00:52<00:50, 2.48it/s]
50%|████████████████████▋ | 126/250 [00:52<00:47, 2.61it/s]
51%|████████████████████▊ | 127/250 [00:52<00:42, 2.87it/s]
51%|████████████████████▉ | 128/250 [00:53<00:41, 2.94it/s]
52%|█████████████████████▏ | 129/250 [00:53<00:56, 2.12it/s]
52%|█████████████████████▎ | 130/250 [00:54<00:55, 2.17it/s]
52%|█████████████████████▍ | 131/250 [00:54<00:56, 2.11it/s]
53%|█████████████████████▋ | 132/250 [00:55<00:58, 2.01it/s]
53%|█████████████████████▊ | 133/250 [00:55<00:52, 2.24it/s]
54%|█████████████████████▉ | 134/250 [00:56<00:51, 2.23it/s]
54%|██████████████████████▏ | 135/250 [00:56<00:47, 2.42it/s]
54%|██████████████████████▎ | 136/250 [00:56<00:48, 2.35it/s]
55%|██████████████████████▍ | 137/250 [00:57<00:48, 2.34it/s]
55%|██████████████████████▋ | 138/250 [00:57<00:47, 2.37it/s]
56%|██████████████████████▊ | 139/250 [00:58<00:45, 2.45it/s]
56%|██████████████████████▉ | 140/250 [00:58<00:42, 2.59it/s]
56%|███████████████████████ | 141/250 [00:58<00:40, 2.66it/s]
57%|███████████████████████▎ | 142/250 [00:59<00:39, 2.73it/s]
57%|███████████████████████▍ | 143/250 [00:59<00:39, 2.69it/s]
58%|███████████████████████▌ | 144/250 [00:59<00:36, 2.88it/s]
58%|███████████████████████▊ | 145/250 [01:00<00:42, 2.49it/s]
58%|███████████████████████▉ | 146/250 [01:00<00:47, 2.18it/s]
59%|████████████████████████ | 147/250 [01:01<00:42, 2.40it/s]
59%|████████████████████████▎ | 148/250 [01:01<00:49, 2.07it/s]
60%|████████████████████████▍ | 149/250 [01:02<00:44, 2.26it/s]
60%|████████████████████████▌ | 150/250 [01:02<00:43, 2.32it/s]
60%|████████████████████████▊ | 151/250 [01:03<00:39, 2.50it/s]
61%|████████████████████████▉ | 152/250 [01:03<00:37, 2.58it/s]
61%|█████████████████████████ | 153/250 [01:03<00:35, 2.71it/s]
62%|█████████████████████████▎ | 154/250 [01:03<00:33, 2.90it/s]
62%|█████████████████████████▍ | 155/250 [01:04<00:36, 2.61it/s]
62%|█████████████████████████▌ | 156/250 [01:04<00:40, 2.31it/s]
63%|█████████████████████████▋ | 157/250 [01:05<00:39, 2.36it/s]
63%|█████████████████████████▉ | 158/250 [01:05<00:36, 2.52it/s]
64%|██████████████████████████ | 159/250 [01:06<00:35, 2.55it/s]
64%|██████████████████████████▏ | 160/250 [01:06<00:33, 2.66it/s]
64%|██████████████████████████▍ | 161/250 [01:06<00:32, 2.75it/s]
65%|██████████████████████████▌ | 162/250 [01:07<00:30, 2.85it/s]
65%|██████████████████████████▋ | 163/250 [01:07<00:29, 2.96it/s]
66%|██████████████████████████▉ | 164/250 [01:07<00:28, 3.04it/s]
66%|███████████████████████████ | 165/250 [01:08<00:41, 2.03it/s]
66%|███████████████████████████▏ | 166/250 [01:08<00:38, 2.19it/s]
67%|███████████████████████████▍ | 167/250 [01:09<00:36, 2.27it/s]
67%|███████████████████████████▌ | 168/250 [01:09<00:33, 2.46it/s]
68%|███████████████████████████▋ | 169/250 [01:10<00:33, 2.44it/s]
68%|███████████████████████████▉ | 170/250 [01:10<00:32, 2.46it/s]
68%|████████████████████████████ | 171/250 [01:10<00:31, 2.55it/s]
69%|████████████████████████████▏ | 172/250 [01:11<00:31, 2.49it/s]
69%|████████████████████████████▎ | 173/250 [01:11<00:32, 2.40it/s]
70%|████████████████████████████▌ | 174/250 [01:12<00:30, 2.52it/s]
70%|████████████████████████████▋ | 175/250 [01:12<00:30, 2.42it/s]
70%|████████████████████████████▊ | 176/250 [01:13<00:34, 2.15it/s]
71%|█████████████████████████████ | 177/250 [01:13<00:32, 2.24it/s]
71%|█████████████████████████████▏ | 178/250 [01:13<00:28, 2.53it/s]
72%|█████████████████████████████▎ | 179/250 [01:14<00:26, 2.67it/s]
72%|█████████████████████████████▌ | 180/250 [01:14<00:25, 2.72it/s]
72%|█████████████████████████████▋ | 181/250 [01:15<00:33, 2.08it/s]
73%|█████████████████████████████▊ | 182/250 [01:15<00:34, 1.98it/s]
73%|██████████████████████████████ | 183/250 [01:16<00:33, 2.01it/s]
74%|██████████████████████████████▏ | 184/250 [01:16<00:30, 2.19it/s]
74%|██████████████████████████████▎ | 185/250 [01:16<00:27, 2.37it/s]
74%|██████████████████████████████▌ | 186/250 [01:17<00:24, 2.61it/s]
75%|██████████████████████████████▋ | 187/250 [01:17<00:28, 2.23it/s]
75%|██████████████████████████████▊ | 188/250 [01:18<00:28, 2.17it/s]
76%|██████████████████████████████▉ | 189/250 [01:18<00:25, 2.43it/s]
76%|███████████████████████████████▏ | 190/250 [01:19<00:25, 2.34it/s]
76%|███████████████████████████████▎ | 191/250 [01:19<00:30, 1.92it/s]
77%|███████████████████████████████▍ | 192/250 [01:20<00:26, 2.19it/s]
77%|███████████████████████████████▋ | 193/250 [01:20<00:23, 2.39it/s]
78%|███████████████████████████████▊ | 194/250 [01:20<00:21, 2.58it/s]
78%|███████████████████████████████▉ | 195/250 [01:21<00:20, 2.65it/s]
78%|████████████████████████████████▏ | 196/250 [01:21<00:20, 2.65it/s]
79%|████████████████████████████████▎ | 197/250 [01:22<00:23, 2.23it/s]
79%|████████████████████████████████▍ | 198/250 [01:22<00:27, 1.89it/s]
80%|████████████████████████████████▋ | 199/250 [01:23<00:24, 2.10it/s]
80%|████████████████████████████████▊ | 200/250 [01:23<00:21, 2.31it/s]
80%|████████████████████████████████▉ | 201/250 [01:23<00:19, 2.46it/s]
81%|█████████████████████████████████▏ | 202/250 [01:24<00:19, 2.47it/s]
81%|█████████████████████████████████▎ | 203/250 [01:24<00:19, 2.35it/s]
82%|█████████████████████████████████▍ | 204/250 [01:25<00:18, 2.53it/s]
82%|█████████████████████████████████▌ | 205/250 [01:25<00:20, 2.22it/s]
82%|█████████████████████████████████▊ | 206/250 [01:26<00:19, 2.31it/s]
83%|█████████████████████████████████▉ | 207/250 [01:26<00:20, 2.11it/s]
83%|██████████████████████████████████ | 208/250 [01:27<00:18, 2.27it/s]
84%|██████████████████████████████████▎ | 209/250 [01:27<00:20, 2.04it/s]
84%|██████████████████████████████████▍ | 210/250 [01:28<00:20, 1.92it/s]
84%|██████████████████████████████████▌ | 211/250 [01:28<00:20, 1.88it/s]
85%|██████████████████████████████████▊ | 212/250 [01:29<00:19, 1.91it/s]
85%|██████████████████████████████████▉ | 213/250 [01:29<00:17, 2.06it/s]
86%|███████████████████████████████████ | 214/250 [01:30<00:16, 2.14it/s]
86%|███████████████████████████████████▎ | 215/250 [01:30<00:18, 1.93it/s]
86%|███████████████████████████████████▍ | 216/250 [01:31<00:16, 2.09it/s]
87%|███████████████████████████████████▌ | 217/250 [01:31<00:15, 2.09it/s]
87%|███████████████████████████████████▊ | 218/250 [01:32<00:16, 1.94it/s]
88%|███████████████████████████████████▉ | 219/250 [01:32<00:14, 2.10it/s]
88%|████████████████████████████████████ | 220/250 [01:32<00:13, 2.21it/s]
88%|████████████████████████████████████▏ | 221/250 [01:33<00:12, 2.39it/s]
89%|████████████████████████████████████▍ | 222/250 [01:33<00:11, 2.42it/s]
89%|████████████████████████████████████▌ | 223/250 [01:34<00:12, 2.13it/s]
90%|████████████████████████████████████▋ | 224/250 [01:34<00:11, 2.30it/s]
90%|████████████████████████████████████▉ | 225/250 [01:34<00:09, 2.51it/s]
90%|█████████████████████████████████████ | 226/250 [01:35<00:09, 2.42it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:35<00:08, 2.64it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:36<00:07, 2.75it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:36<00:07, 2.94it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:36<00:06, 3.08it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:37<00:06, 2.78it/s]
93%|██████████████████████████████████████ | 232/250 [01:37<00:06, 2.69it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:37<00:06, 2.82it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:38<00:05, 2.93it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:38<00:04, 3.05it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:38<00:04, 2.93it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:39<00:04, 2.90it/s]
95%|███████████████████████████████████████ | 238/250 [01:39<00:04, 2.91it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:39<00:03, 2.83it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:40<00:04, 2.43it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:40<00:03, 2.57it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:41<00:03, 2.22it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:41<00:03, 2.18it/s]
98%|████████████████████████████████████████ | 244/250 [01:42<00:02, 2.33it/s]
98%|████████████████████████████████████████▏| 245/250 [01:42<00:02, 2.29it/s]
98%|████████████████████████████████████████▎| 246/250 [01:42<00:01, 2.54it/s]
99%|████████████████████████████████████████▌| 247/250 [01:43<00:01, 2.44it/s]
99%|████████████████████████████████████████▋| 248/250 [01:43<00:00, 2.41it/s]
100%|████████████████████████████████████████▊| 249/250 [01:44<00:00, 2.58it/s]
100%|█████████████████████████████████████████| 250/250 [01:44<00:00, 2.29it/s]
{'eval_loss': 1.1117446422576904, 'eval_runtime': 105.3346, 'eval_samples_per_second': 18.987, 'eval_steps_per_second': 2.373, 'epoch': 0.2}
20%|████████ | 78/390 [38:30<2:19:33, 26.84s/it]
{'loss': 1.108, 'learning_rate': 0.00026495726495726497, 'epoch': 0.21}
{'loss': 1.1071, 'learning_rate': 0.00024786324786324785, 'epoch': 0.26}
30%|███████████▋ | 117/390 [56:08<2:01:40, 26.74s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:48, 5.14it/s]
1%|▌ | 3/250 [00:00<01:04, 3.84it/s]
2%|▋ | 4/250 [00:01<02:00, 2.04it/s]
2%|▊ | 5/250 [00:02<01:53, 2.15it/s]
2%|█ | 6/250 [00:02<01:57, 2.07it/s]
3%|█▏ | 7/250 [00:02<01:49, 2.21it/s]
3%|█▍ | 8/250 [00:03<01:40, 2.41it/s]
4%|█▌ | 9/250 [00:04<02:14, 1.80it/s]
4%|█▋ | 10/250 [00:04<02:00, 2.00it/s]
4%|█▊ | 11/250 [00:04<01:53, 2.10it/s]
5%|██ | 12/250 [00:05<01:49, 2.17it/s]
5%|██▏ | 13/250 [00:05<01:40, 2.35it/s]
6%|██▎ | 14/250 [00:06<01:46, 2.22it/s]
6%|██▌ | 15/250 [00:06<01:46, 2.21it/s]
6%|██▋ | 16/250 [00:07<01:52, 2.08it/s]
7%|██▊ | 17/250 [00:07<01:57, 1.99it/s]
7%|███ | 18/250 [00:08<02:02, 1.89it/s]
8%|███▏ | 19/250 [00:08<01:58, 1.95it/s]
8%|███▎ | 20/250 [00:09<01:50, 2.09it/s]
8%|███▌ | 21/250 [00:09<01:55, 1.99it/s]
9%|███▋ | 22/250 [00:10<01:43, 2.20it/s]
9%|███▊ | 23/250 [00:10<01:30, 2.50it/s]
10%|████ | 24/250 [00:10<01:26, 2.60it/s]
10%|████▏ | 25/250 [00:11<01:23, 2.68it/s]
10%|████▎ | 26/250 [00:11<01:37, 2.29it/s]
11%|████▌ | 27/250 [00:11<01:28, 2.53it/s]
11%|████▋ | 28/250 [00:12<01:20, 2.74it/s]
12%|████▊ | 29/250 [00:12<01:14, 2.95it/s]
12%|█████ | 30/250 [00:12<01:13, 3.00it/s]
12%|█████▏ | 31/250 [00:13<01:11, 3.06it/s]
13%|█████▍ | 32/250 [00:13<01:38, 2.22it/s]
13%|█████▌ | 33/250 [00:14<01:32, 2.36it/s]
14%|█████▋ | 34/250 [00:14<01:29, 2.41it/s]
14%|█████▉ | 35/250 [00:15<01:25, 2.52it/s]
14%|██████ | 36/250 [00:15<01:55, 1.86it/s]
15%|██████▏ | 37/250 [00:16<01:42, 2.07it/s]
15%|██████▍ | 38/250 [00:16<01:36, 2.19it/s]
16%|██████▌ | 39/250 [00:17<01:37, 2.16it/s]
16%|██████▋ | 40/250 [00:17<01:28, 2.38it/s]
16%|██████▉ | 41/250 [00:18<01:44, 2.00it/s]
17%|███████ | 42/250 [00:18<01:38, 2.12it/s]
17%|███████▏ | 43/250 [00:18<01:32, 2.24it/s]
18%|███████▍ | 44/250 [00:19<01:36, 2.13it/s]
18%|███████▌ | 45/250 [00:20<01:40, 2.03it/s]
18%|███████▋ | 46/250 [00:20<01:29, 2.28it/s]
19%|███████▉ | 47/250 [00:20<01:23, 2.44it/s]
19%|████████ | 48/250 [00:21<01:25, 2.36it/s]
20%|████████▏ | 49/250 [00:21<01:33, 2.16it/s]
20%|████████▍ | 50/250 [00:21<01:22, 2.42it/s]
20%|████████▌ | 51/250 [00:22<01:24, 2.37it/s]
21%|████████▋ | 52/250 [00:22<01:17, 2.55it/s]
21%|████████▉ | 53/250 [00:23<01:13, 2.67it/s]
22%|█████████ | 54/250 [00:23<01:13, 2.66it/s]
22%|█████████▏ | 55/250 [00:24<01:42, 1.90it/s]
22%|█████████▍ | 56/250 [00:24<01:32, 2.09it/s]
23%|█████████▌ | 57/250 [00:25<01:22, 2.35it/s]
23%|█████████▋ | 58/250 [00:25<01:20, 2.39it/s]
24%|█████████▉ | 59/250 [00:25<01:20, 2.37it/s]
24%|██████████ | 60/250 [00:26<01:15, 2.51it/s]
24%|██████████▏ | 61/250 [00:26<01:10, 2.67it/s]
25%|██████████▍ | 62/250 [00:26<01:15, 2.51it/s]
25%|██████████▌ | 63/250 [00:27<01:30, 2.06it/s]
26%|██████████▊ | 64/250 [00:28<01:23, 2.22it/s]
26%|██████████▉ | 65/250 [00:28<01:19, 2.34it/s]
26%|███████████ | 66/250 [00:28<01:13, 2.51it/s]
27%|███████████▎ | 67/250 [00:29<01:10, 2.61it/s]
27%|███████████▍ | 68/250 [00:29<01:09, 2.60it/s]
28%|███████████▌ | 69/250 [00:29<01:06, 2.72it/s]
28%|███████████▊ | 70/250 [00:30<01:06, 2.70it/s]
28%|███████████▉ | 71/250 [00:30<01:07, 2.66it/s]
29%|████████████ | 72/250 [00:30<01:07, 2.62it/s]
29%|████████████▎ | 73/250 [00:31<01:06, 2.66it/s]
30%|████████████▍ | 74/250 [00:31<01:08, 2.57it/s]
30%|████████████▌ | 75/250 [00:32<01:04, 2.72it/s]
30%|████████████▊ | 76/250 [00:32<01:00, 2.88it/s]
31%|████████████▉ | 77/250 [00:32<01:08, 2.51it/s]
31%|█████████████ | 78/250 [00:33<01:06, 2.59it/s]
32%|█████████████▎ | 79/250 [00:34<01:31, 1.88it/s]
32%|█████████████▍ | 80/250 [00:34<01:22, 2.07it/s]
32%|█████████████▌ | 81/250 [00:34<01:20, 2.11it/s]
33%|█████████████▊ | 82/250 [00:35<01:15, 2.23it/s]
33%|█████████████▉ | 83/250 [00:35<01:10, 2.36it/s]
34%|██████████████ | 84/250 [00:35<01:05, 2.53it/s]
34%|██████████████▎ | 85/250 [00:36<01:22, 2.01it/s]
34%|██████████████▍ | 86/250 [00:37<01:14, 2.19it/s]
35%|██████████████▌ | 87/250 [00:37<01:16, 2.12it/s]
35%|██████████████▊ | 88/250 [00:37<01:08, 2.35it/s]
36%|██████████████▉ | 89/250 [00:38<01:14, 2.15it/s]
36%|███████████████ | 90/250 [00:38<01:10, 2.27it/s]
36%|███████████████▎ | 91/250 [00:39<01:14, 2.14it/s]
37%|███████████████▍ | 92/250 [00:39<01:07, 2.35it/s]
37%|███████████████▌ | 93/250 [00:40<01:00, 2.58it/s]
38%|███████████████▊ | 94/250 [00:40<01:09, 2.24it/s]
38%|███████████████▉ | 95/250 [00:40<01:05, 2.38it/s]
38%|████████████████▏ | 96/250 [00:41<01:04, 2.41it/s]
39%|████████████████▎ | 97/250 [00:41<01:01, 2.48it/s]
39%|████████████████▍ | 98/250 [00:42<01:00, 2.51it/s]
40%|████████████████▋ | 99/250 [00:42<00:56, 2.66it/s]
40%|████████████████▍ | 100/250 [00:42<00:56, 2.66it/s]
40%|████████████████▌ | 101/250 [00:43<00:53, 2.81it/s]
41%|████████████████▋ | 102/250 [00:43<01:00, 2.46it/s]
41%|████████████████▉ | 103/250 [00:44<00:58, 2.51it/s]
42%|█████████████████ | 104/250 [00:44<01:02, 2.33it/s]
42%|█████████████████▏ | 105/250 [00:44<00:58, 2.46it/s]
42%|█████████████████▍ | 106/250 [00:45<00:56, 2.56it/s]
43%|█████████████████▌ | 107/250 [00:45<00:55, 2.59it/s]
43%|█████████████████▋ | 108/250 [00:45<00:52, 2.68it/s]
44%|█████████████████▉ | 109/250 [00:46<00:51, 2.73it/s]
44%|██████████████████ | 110/250 [00:46<00:54, 2.59it/s]
44%|██████████████████▏ | 111/250 [00:47<00:56, 2.47it/s]
45%|██████████████████▎ | 112/250 [00:47<00:54, 2.55it/s]
45%|██████████████████▌ | 113/250 [00:47<00:49, 2.79it/s]
46%|██████████████████▋ | 114/250 [00:48<00:47, 2.85it/s]
46%|██████████████████▊ | 115/250 [00:48<00:44, 3.01it/s]
46%|███████████████████ | 116/250 [00:48<00:46, 2.88it/s]
47%|███████████████████▏ | 117/250 [00:49<00:47, 2.78it/s]
47%|███████████████████▎ | 118/250 [00:49<00:45, 2.90it/s]
48%|███████████████████▌ | 119/250 [00:49<00:47, 2.77it/s]
48%|███████████████████▋ | 120/250 [00:50<00:44, 2.89it/s]
48%|███████████████████▊ | 121/250 [00:50<00:43, 2.99it/s]
49%|████████████████████ | 122/250 [00:50<00:41, 3.06it/s]
49%|████████████████████▏ | 123/250 [00:51<00:45, 2.82it/s]
50%|████████████████████▎ | 124/250 [00:51<00:43, 2.90it/s]
50%|████████████████████▌ | 125/250 [00:52<00:50, 2.47it/s]
50%|████████████████████▋ | 126/250 [00:52<00:47, 2.61it/s]
51%|████████████████████▊ | 127/250 [00:52<00:42, 2.87it/s]
51%|████████████████████▉ | 128/250 [00:53<00:41, 2.94it/s]
52%|█████████████████████▏ | 129/250 [00:53<00:57, 2.12it/s]
52%|█████████████████████▎ | 130/250 [00:54<00:55, 2.17it/s]
52%|█████████████████████▍ | 131/250 [00:54<00:56, 2.11it/s]
53%|█████████████████████▋ | 132/250 [00:55<00:58, 2.01it/s]
53%|█████████████████████▊ | 133/250 [00:55<00:52, 2.24it/s]
54%|█████████████████████▉ | 134/250 [00:56<00:51, 2.24it/s]
54%|██████████████████████▏ | 135/250 [00:56<00:47, 2.42it/s]
54%|██████████████████████▎ | 136/250 [00:56<00:48, 2.35it/s]
55%|██████████████████████▍ | 137/250 [00:57<00:48, 2.34it/s]
55%|██████████████████████▋ | 138/250 [00:57<00:47, 2.37it/s]
56%|██████████████████████▊ | 139/250 [00:58<00:45, 2.45it/s]
56%|██████████████████████▉ | 140/250 [00:58<00:42, 2.59it/s]
56%|███████████████████████ | 141/250 [00:58<00:40, 2.66it/s]
57%|███████████████████████▎ | 142/250 [00:59<00:39, 2.73it/s]
57%|███████████████████████▍ | 143/250 [00:59<00:39, 2.69it/s]
58%|███████████████████████▌ | 144/250 [00:59<00:36, 2.88it/s]
58%|███████████████████████▊ | 145/250 [01:00<00:42, 2.49it/s]
58%|███████████████████████▉ | 146/250 [01:00<00:47, 2.18it/s]
59%|████████████████████████ | 147/250 [01:01<00:43, 2.39it/s]
59%|████████████████████████▎ | 148/250 [01:01<00:49, 2.06it/s]
60%|████████████████████████▍ | 149/250 [01:02<00:44, 2.26it/s]
60%|████████████████████████▌ | 150/250 [01:02<00:43, 2.32it/s]
60%|████████████████████████▊ | 151/250 [01:02<00:39, 2.50it/s]
61%|████████████████████████▉ | 152/250 [01:03<00:37, 2.58it/s]
61%|█████████████████████████ | 153/250 [01:03<00:35, 2.71it/s]
62%|█████████████████████████▎ | 154/250 [01:03<00:33, 2.90it/s]
62%|█████████████████████████▍ | 155/250 [01:04<00:36, 2.61it/s]
62%|█████████████████████████▌ | 156/250 [01:04<00:40, 2.31it/s]
63%|█████████████████████████▋ | 157/250 [01:05<00:39, 2.36it/s]
63%|█████████████████████████▉ | 158/250 [01:05<00:36, 2.52it/s]
64%|██████████████████████████ | 159/250 [01:06<00:35, 2.55it/s]
64%|██████████████████████████▏ | 160/250 [01:06<00:33, 2.66it/s]
64%|██████████████████████████▍ | 161/250 [01:06<00:32, 2.75it/s]
65%|██████████████████████████▌ | 162/250 [01:07<00:30, 2.84it/s]
65%|██████████████████████████▋ | 163/250 [01:07<00:29, 2.95it/s]
66%|██████████████████████████▉ | 164/250 [01:07<00:28, 3.04it/s]
66%|███████████████████████████ | 165/250 [01:08<00:41, 2.03it/s]
66%|███████████████████████████▏ | 166/250 [01:08<00:38, 2.20it/s]
67%|███████████████████████████▍ | 167/250 [01:09<00:36, 2.27it/s]
67%|███████████████████████████▌ | 168/250 [01:09<00:33, 2.47it/s]
68%|███████████████████████████▋ | 169/250 [01:10<00:33, 2.44it/s]
68%|███████████████████████████▉ | 170/250 [01:10<00:32, 2.46it/s]
68%|████████████████████████████ | 171/250 [01:10<00:30, 2.55it/s]
69%|████████████████████████████▏ | 172/250 [01:11<00:31, 2.49it/s]
69%|████████████████████████████▎ | 173/250 [01:11<00:32, 2.40it/s]
70%|████████████████████████████▌ | 174/250 [01:12<00:30, 2.52it/s]
70%|████████████████████████████▋ | 175/250 [01:12<00:30, 2.42it/s]
70%|████████████████████████████▊ | 176/250 [01:13<00:34, 2.15it/s]
71%|█████████████████████████████ | 177/250 [01:13<00:32, 2.24it/s]
71%|█████████████████████████████▏ | 178/250 [01:13<00:28, 2.53it/s]
72%|█████████████████████████████▎ | 179/250 [01:14<00:26, 2.71it/s]
72%|█████████████████████████████▌ | 180/250 [01:14<00:25, 2.74it/s]
72%|█████████████████████████████▋ | 181/250 [01:15<00:32, 2.09it/s]
73%|█████████████████████████████▊ | 182/250 [01:15<00:34, 2.00it/s]
73%|██████████████████████████████ | 183/250 [01:16<00:33, 2.02it/s]
74%|██████████████████████████████▏ | 184/250 [01:16<00:30, 2.20it/s]
74%|██████████████████████████████▎ | 185/250 [01:16<00:27, 2.38it/s]
74%|██████████████████████████████▌ | 186/250 [01:17<00:24, 2.61it/s]
75%|██████████████████████████████▋ | 187/250 [01:17<00:28, 2.23it/s]
75%|██████████████████████████████▊ | 188/250 [01:18<00:28, 2.17it/s]
76%|██████████████████████████████▉ | 189/250 [01:18<00:25, 2.43it/s]
76%|███████████████████████████████▏ | 190/250 [01:19<00:25, 2.34it/s]
76%|███████████████████████████████▎ | 191/250 [01:19<00:30, 1.92it/s]
77%|███████████████████████████████▍ | 192/250 [01:20<00:26, 2.19it/s]
77%|███████████████████████████████▋ | 193/250 [01:20<00:23, 2.39it/s]
78%|███████████████████████████████▊ | 194/250 [01:20<00:21, 2.58it/s]
78%|███████████████████████████████▉ | 195/250 [01:21<00:20, 2.64it/s]
78%|████████████████████████████████▏ | 196/250 [01:21<00:20, 2.66it/s]
79%|████████████████████████████████▎ | 197/250 [01:22<00:23, 2.24it/s]
79%|████████████████████████████████▍ | 198/250 [01:22<00:27, 1.89it/s]
80%|████████████████████████████████▋ | 199/250 [01:23<00:24, 2.11it/s]
80%|████████████████████████████████▊ | 200/250 [01:23<00:21, 2.32it/s]
80%|████████████████████████████████▉ | 201/250 [01:23<00:19, 2.46it/s]
81%|█████████████████████████████████▏ | 202/250 [01:24<00:19, 2.47it/s]
81%|█████████████████████████████████▎ | 203/250 [01:24<00:19, 2.36it/s]
82%|█████████████████████████████████▍ | 204/250 [01:25<00:18, 2.53it/s]
82%|█████████████████████████████████▌ | 205/250 [01:25<00:20, 2.22it/s]
82%|█████████████████████████████████▊ | 206/250 [01:26<00:18, 2.32it/s]
83%|█████████████████████████████████▉ | 207/250 [01:26<00:19, 2.19it/s]
83%|██████████████████████████████████ | 208/250 [01:26<00:18, 2.33it/s]
84%|██████████████████████████████████▎ | 209/250 [01:27<00:19, 2.07it/s]
84%|██████████████████████████████████▍ | 210/250 [01:28<00:20, 1.94it/s]
84%|██████████████████████████████████▌ | 211/250 [01:28<00:20, 1.90it/s]
85%|██████████████████████████████████▊ | 212/250 [01:29<00:19, 1.92it/s]
85%|██████████████████████████████████▉ | 213/250 [01:29<00:17, 2.07it/s]
86%|███████████████████████████████████ | 214/250 [01:29<00:16, 2.15it/s]
86%|███████████████████████████████████▎ | 215/250 [01:30<00:18, 1.94it/s]
86%|███████████████████████████████████▍ | 216/250 [01:31<00:16, 2.09it/s]
87%|███████████████████████████████████▌ | 217/250 [01:31<00:15, 2.13it/s]
87%|███████████████████████████████████▊ | 218/250 [01:32<00:16, 1.96it/s]
88%|███████████████████████████████████▉ | 219/250 [01:32<00:14, 2.11it/s]
88%|████████████████████████████████████ | 220/250 [01:32<00:13, 2.22it/s]
88%|████████████████████████████████████▏ | 221/250 [01:33<00:12, 2.39it/s]
89%|████████████████████████████████████▍ | 222/250 [01:33<00:11, 2.42it/s]
89%|████████████████████████████████████▌ | 223/250 [01:34<00:13, 2.07it/s]
90%|████████████████████████████████████▋ | 224/250 [01:34<00:11, 2.25it/s]
90%|████████████████████████████████████▉ | 225/250 [01:34<00:10, 2.47it/s]
90%|█████████████████████████████████████ | 226/250 [01:35<00:09, 2.40it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:35<00:08, 2.62it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:35<00:08, 2.74it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:36<00:07, 2.93it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:36<00:06, 3.08it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:36<00:06, 2.78it/s]
93%|██████████████████████████████████████ | 232/250 [01:37<00:06, 2.68it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:37<00:06, 2.82it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:38<00:05, 2.93it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:38<00:04, 3.05it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:38<00:04, 2.93it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:39<00:04, 2.90it/s]
95%|███████████████████████████████████████ | 238/250 [01:39<00:04, 2.91it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:39<00:03, 2.83it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:40<00:04, 2.44it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:40<00:03, 2.57it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:41<00:03, 2.21it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:41<00:03, 2.18it/s]
98%|████████████████████████████████████████ | 244/250 [01:42<00:02, 2.32it/s]
98%|████████████████████████████████████████▏| 245/250 [01:42<00:02, 2.30it/s]
98%|████████████████████████████████████████▎| 246/250 [01:42<00:01, 2.55it/s]
99%|████████████████████████████████████████▌| 247/250 [01:43<00:01, 2.44it/s]
99%|████████████████████████████████████████▋| 248/250 [01:43<00:00, 2.41it/s]
100%|████████████████████████████████████████▊| 249/250 [01:44<00:00, 2.59it/s]
100%|█████████████████████████████████████████| 250/250 [01:44<00:00, 2.30it/s]
{'eval_loss': 1.1009880304336548, 'eval_runtime': 105.2386, 'eval_samples_per_second': 19.004, 'eval_steps_per_second': 2.376, 'epoch': 0.3}
30%|███████████▋ | 117/390 [57:53<2:01:40, 26.74s/it]
{'loss': 1.1041, 'learning_rate': 0.00023076923076923076, 'epoch': 0.31}
{'loss': 1.0885, 'learning_rate': 0.00021367521367521365, 'epoch': 0.36}
39%|██████████████▌ | 153/390 [1:14:02<1:44:22, 26.42s/it]^C
In [17]:
# instruction finetuning
!cd /workspace/code && python uniform_finetune.py --model_type llama --model_name_or_path decapoda-research/llama-7b-hf --data alpaca --lora_target_modules q_proj v_proj --per_gpu_train_batch_size 4 --learning_rate 3e-4 --epochs 1 --report_to wandb --resume_from_checkpoint latest
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib64'), PosixPath('/usr/local/nvidia/lib')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('module'), PosixPath('//matplotlib_inline.backend_inline')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('PygmalionAI/pygmalion-6b')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('noninteractiveSHELL=/bin/bash')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/workspace/Untitled.ipynb')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0'), PosixPath('/usr/local/cuda/lib64/libcudart.so')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so.11.0
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 116
CUDA SETUP: Loading binary /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so...
Namespace(size=None, data='alpaca', local_rank=-1, model_type='llama', model_name_or_path='decapoda-research/llama-7b-hf', per_gpu_train_batch_size=4, gradient_accumulation_steps=32, epochs=1, learning_rate=0.0003, cutoff_len=512, lora_r=8, lora_alpha=16, lora_dropout=0.05, val_set_size=2000, lora_target_modules=['q_proj', 'v_proj'], resume_from_checkpoint='latest')
Found cached dataset json (/root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e)
100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 473.40it/s]
DatasetDict({
train: Dataset({
features: ['instruction', 'input', 'output'],
num_rows: 51942
})
})
Overriding torch_dtype=None with `torch_dtype=torch.float16` due to requirements of `bitsandbytes` to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning.
Loading checkpoint shards: 100%|████████████████| 33/33 [00:07<00:00, 4.15it/s]
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
trainable params: 4194304 || all params: 6742609920 || trainable%: 0.06220594176090199
Loading cached split indices for dataset at /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e/cache-d9cf6a7263a2a9b6.arrow and /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e/cache-1833a2e8bfe038bc.arrow
***** Running training *****
Num Epochs = 1
Instantaneous batch size per GPU = 4
Gradient Accumulation steps = 32
Total train batch size (w. parallel, distributed & accumulation) = 128
Total optimization steps = 390
Saving steps = 39
/usr/local/lib/python3.10/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning
warnings.warn(
wandb: Currently logged in as: utensil. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.14.2
wandb: Run data is saved locally in /workspace/code/wandb/run-20230419_052401-rzum7ifl
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run smart-armadillo-2
wandb: ⭐️ View project at https://wandb.ai/utensil/Alpaca-CoT
wandb: 🚀 View run at https://wandb.ai/utensil/Alpaca-CoT/runs/rzum7ifl
0%| | 1/390 [00:29<3:08:07, 29.02s/it]^C
Traceback (most recent call last):
File "/workspace/code/uniform_finetune.py", line 349, in <module>
train(args)
File "/workspace/code/uniform_finetune.py", line 317, in train
trainer.train()
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1662, in train
return inner_training_loop(
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1929, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2699, in training_step
loss = self.compute_loss(model, inputs)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2731, in compute_loss
outputs = model(**inputs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/peft/peft_model.py", line 530, in forward
return self.base_model(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 687, in forward
outputs = self.model(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 569, in forward
layer_outputs = torch.utils.checkpoint.checkpoint(
File "/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py", line 249, in checkpoint
return CheckpointFunction.apply(function, preserve, *args)
File "/usr/local/lib/python3.10/dist-packages/torch/autograd/function.py", line 506, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py", line 107, in forward
outputs = run_function(*args)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 565, in custom_forward
return module(*inputs, output_attentions, None)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 305, in forward
hidden_states = self.mlp(hidden_states)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 157, in forward
return self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/nn/modules.py", line 320, in forward
out = bnb.matmul(x, self.weight, bias=self.bias, state=self.state)
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/autograd/_functions.py", line 500, in matmul
return MatMul8bitLt.apply(A, B, out, bias, state)
File "/usr/local/lib/python3.10/dist-packages/torch/autograd/function.py", line 506, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/autograd/_functions.py", line 323, in forward
CA, CAt, SCA, SCAt, coo_tensorA = F.double_quant(A.to(torch.float16), threshold=state.threshold)
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/functional.py", line 1660, in double_quant
nnz = nnz_row_ptr[-1].item()
KeyboardInterrupt
wandb: Waiting for W&B process to finish... (failed 255). Press Control-C to abort syncing.
In [18]:
# instruction finetuning
!cd /workspace/code && python uniform_finetune.py --model_type llama --model_name_or_path decapoda-research/llama-7b-hf --data alpaca --lora_target_modules q_proj v_proj --per_gpu_train_batch_size 4 --learning_rate 3e-4 --epochs 1 --report_to wandb --resume_from_checkpoint latest
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib'), PosixPath('/usr/local/nvidia/lib64')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('//matplotlib_inline.backend_inline'), PosixPath('module')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('PygmalionAI/pygmalion-6b')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('noninteractiveSHELL=/bin/bash')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/workspace/Untitled.ipynb')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 116
CUDA SETUP: Loading binary /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so...
Namespace(size=None, data='alpaca', local_rank=-1, model_type='llama', model_name_or_path='decapoda-research/llama-7b-hf', per_gpu_train_batch_size=4, gradient_accumulation_steps=32, epochs=1, learning_rate=0.0003, cutoff_len=512, lora_r=8, lora_alpha=16, lora_dropout=0.05, val_set_size=2000, lora_target_modules=['q_proj', 'v_proj'], resume_from_checkpoint='latest')
Found cached dataset json (/root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e)
100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 766.78it/s]
DatasetDict({
train: Dataset({
features: ['instruction', 'input', 'output'],
num_rows: 51942
})
})
Overriding torch_dtype=None with `torch_dtype=torch.float16` due to requirements of `bitsandbytes` to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning.
Loading checkpoint shards: 100%|████████████████| 33/33 [00:07<00:00, 4.29it/s]
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
trainable params: 4194304 || all params: 6742609920 || trainable%: 0.06220594176090199
Loading cached split indices for dataset at /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e/cache-d9cf6a7263a2a9b6.arrow and /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e/cache-1833a2e8bfe038bc.arrow
***** Running training *****
Num Epochs = 1
Instantaneous batch size per GPU = 4
Gradient Accumulation steps = 32
Total train batch size (w. parallel, distributed & accumulation) = 128
Total optimization steps = 390
Saving steps = 39
Traceback (most recent call last):
File "/workspace/code/uniform_finetune.py", line 348, in <module>
train(args)
File "/workspace/code/uniform_finetune.py", line 316, in train
trainer.train(resume_from_checkpoint=True if args.resume_from_checkpoint == 'latest' else args.resume_from_checkpoint)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1651, in train
self._load_from_checkpoint(resume_from_checkpoint)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2155, in _load_from_checkpoint
state_dict = torch.load(weights_file, map_location="cpu")
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 815, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1033, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.
In [24]:
!cd /workspace/llm-playground/storage && git lfs pull
Downloading LFS objects: 100% (12/12), 101 MB | 10 MB/s
In [25]:
!cp -r /workspace/llm-playground/storage/saved_models /workspace/code/
In [26]:
# instruction finetuning
!cd /workspace/code && python uniform_finetune.py --model_type llama --model_name_or_path decapoda-research/llama-7b-hf --data alpaca --lora_target_modules q_proj v_proj --per_gpu_train_batch_size 4 --learning_rate 3e-4 --epochs 1 --report_to wandb --resume_from_checkpoint latest
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib'), PosixPath('/usr/local/nvidia/lib64')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('//matplotlib_inline.backend_inline'), PosixPath('module')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('PygmalionAI/pygmalion-6b')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('noninteractiveSHELL=/bin/bash')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/workspace/Untitled.ipynb')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0'), PosixPath('/usr/local/cuda/lib64/libcudart.so')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so.11.0
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 116
CUDA SETUP: Loading binary /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so...
Namespace(size=None, data='alpaca', local_rank=-1, model_type='llama', model_name_or_path='decapoda-research/llama-7b-hf', per_gpu_train_batch_size=4, gradient_accumulation_steps=32, epochs=1, learning_rate=0.0003, cutoff_len=512, lora_r=8, lora_alpha=16, lora_dropout=0.05, val_set_size=2000, lora_target_modules=['q_proj', 'v_proj'], resume_from_checkpoint='latest')
Found cached dataset json (/root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e)
100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 739.74it/s]
DatasetDict({
train: Dataset({
features: ['instruction', 'input', 'output'],
num_rows: 51942
})
})
Overriding torch_dtype=None with `torch_dtype=torch.float16` due to requirements of `bitsandbytes` to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning.
Loading checkpoint shards: 100%|████████████████| 33/33 [00:07<00:00, 4.30it/s]
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
trainable params: 4194304 || all params: 6742609920 || trainable%: 0.06220594176090199
Loading cached split indices for dataset at /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e/cache-d9cf6a7263a2a9b6.arrow and /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e/cache-1833a2e8bfe038bc.arrow
***** Running training *****
Num Epochs = 1
Instantaneous batch size per GPU = 4
Gradient Accumulation steps = 32
Total train batch size (w. parallel, distributed & accumulation) = 128
Total optimization steps = 390
Saving steps = 39
There were missing keys in the checkpoint model loaded: ['base_model.model.model.embed_tokens.weight', 'base_model.model.model.layers.0.self_attn.q_proj.weight', 'base_model.model.model.layers.0.self_attn.k_proj.weight', 'base_model.model.model.layers.0.self_attn.v_proj.weight', 'base_model.model.model.layers.0.self_attn.o_proj.weight', 'base_model.model.model.layers.0.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.0.mlp.gate_proj.weight', 'base_model.model.model.layers.0.mlp.down_proj.weight', 'base_model.model.model.layers.0.mlp.up_proj.weight', 'base_model.model.model.layers.0.input_layernorm.weight', 'base_model.model.model.layers.0.post_attention_layernorm.weight', 'base_model.model.model.layers.1.self_attn.q_proj.weight', 'base_model.model.model.layers.1.self_attn.k_proj.weight', 'base_model.model.model.layers.1.self_attn.v_proj.weight', 'base_model.model.model.layers.1.self_attn.o_proj.weight', 'base_model.model.model.layers.1.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.1.mlp.gate_proj.weight', 'base_model.model.model.layers.1.mlp.down_proj.weight', 'base_model.model.model.layers.1.mlp.up_proj.weight', 'base_model.model.model.layers.1.input_layernorm.weight', 'base_model.model.model.layers.1.post_attention_layernorm.weight', 'base_model.model.model.layers.2.self_attn.q_proj.weight', 'base_model.model.model.layers.2.self_attn.k_proj.weight', 'base_model.model.model.layers.2.self_attn.v_proj.weight', 'base_model.model.model.layers.2.self_attn.o_proj.weight', 'base_model.model.model.layers.2.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.2.mlp.gate_proj.weight', 'base_model.model.model.layers.2.mlp.down_proj.weight', 'base_model.model.model.layers.2.mlp.up_proj.weight', 'base_model.model.model.layers.2.input_layernorm.weight', 'base_model.model.model.layers.2.post_attention_layernorm.weight', 'base_model.model.model.layers.3.self_attn.q_proj.weight', 'base_model.model.model.layers.3.self_attn.k_proj.weight', 'base_model.model.model.layers.3.self_attn.v_proj.weight', 'base_model.model.model.layers.3.self_attn.o_proj.weight', 'base_model.model.model.layers.3.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.3.mlp.gate_proj.weight', 'base_model.model.model.layers.3.mlp.down_proj.weight', 'base_model.model.model.layers.3.mlp.up_proj.weight', 'base_model.model.model.layers.3.input_layernorm.weight', 'base_model.model.model.layers.3.post_attention_layernorm.weight', 'base_model.model.model.layers.4.self_attn.q_proj.weight', 'base_model.model.model.layers.4.self_attn.k_proj.weight', 'base_model.model.model.layers.4.self_attn.v_proj.weight', 'base_model.model.model.layers.4.self_attn.o_proj.weight', 'base_model.model.model.layers.4.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.4.mlp.gate_proj.weight', 'base_model.model.model.layers.4.mlp.down_proj.weight', 'base_model.model.model.layers.4.mlp.up_proj.weight', 'base_model.model.model.layers.4.input_layernorm.weight', 'base_model.model.model.layers.4.post_attention_layernorm.weight', 'base_model.model.model.layers.5.self_attn.q_proj.weight', 'base_model.model.model.layers.5.self_attn.k_proj.weight', 'base_model.model.model.layers.5.self_attn.v_proj.weight', 'base_model.model.model.layers.5.self_attn.o_proj.weight', 'base_model.model.model.layers.5.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.5.mlp.gate_proj.weight', 'base_model.model.model.layers.5.mlp.down_proj.weight', 'base_model.model.model.layers.5.mlp.up_proj.weight', 'base_model.model.model.layers.5.input_layernorm.weight', 'base_model.model.model.layers.5.post_attention_layernorm.weight', 'base_model.model.model.layers.6.self_attn.q_proj.weight', 'base_model.model.model.layers.6.self_attn.k_proj.weight', 'base_model.model.model.layers.6.self_attn.v_proj.weight', 'base_model.model.model.layers.6.self_attn.o_proj.weight', 'base_model.model.model.layers.6.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.6.mlp.gate_proj.weight', 'base_model.model.model.layers.6.mlp.down_proj.weight', 'base_model.model.model.layers.6.mlp.up_proj.weight', 'base_model.model.model.layers.6.input_layernorm.weight', 'base_model.model.model.layers.6.post_attention_layernorm.weight', 'base_model.model.model.layers.7.self_attn.q_proj.weight', 'base_model.model.model.layers.7.self_attn.k_proj.weight', 'base_model.model.model.layers.7.self_attn.v_proj.weight', 'base_model.model.model.layers.7.self_attn.o_proj.weight', 'base_model.model.model.layers.7.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.7.mlp.gate_proj.weight', 'base_model.model.model.layers.7.mlp.down_proj.weight', 'base_model.model.model.layers.7.mlp.up_proj.weight', 'base_model.model.model.layers.7.input_layernorm.weight', 'base_model.model.model.layers.7.post_attention_layernorm.weight', 'base_model.model.model.layers.8.self_attn.q_proj.weight', 'base_model.model.model.layers.8.self_attn.k_proj.weight', 'base_model.model.model.layers.8.self_attn.v_proj.weight', 'base_model.model.model.layers.8.self_attn.o_proj.weight', 'base_model.model.model.layers.8.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.8.mlp.gate_proj.weight', 'base_model.model.model.layers.8.mlp.down_proj.weight', 'base_model.model.model.layers.8.mlp.up_proj.weight', 'base_model.model.model.layers.8.input_layernorm.weight', 'base_model.model.model.layers.8.post_attention_layernorm.weight', 'base_model.model.model.layers.9.self_attn.q_proj.weight', 'base_model.model.model.layers.9.self_attn.k_proj.weight', 'base_model.model.model.layers.9.self_attn.v_proj.weight', 'base_model.model.model.layers.9.self_attn.o_proj.weight', 'base_model.model.model.layers.9.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.9.mlp.gate_proj.weight', 'base_model.model.model.layers.9.mlp.down_proj.weight', 'base_model.model.model.layers.9.mlp.up_proj.weight', 'base_model.model.model.layers.9.input_layernorm.weight', 'base_model.model.model.layers.9.post_attention_layernorm.weight', 'base_model.model.model.layers.10.self_attn.q_proj.weight', 'base_model.model.model.layers.10.self_attn.k_proj.weight', 'base_model.model.model.layers.10.self_attn.v_proj.weight', 'base_model.model.model.layers.10.self_attn.o_proj.weight', 'base_model.model.model.layers.10.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.10.mlp.gate_proj.weight', 'base_model.model.model.layers.10.mlp.down_proj.weight', 'base_model.model.model.layers.10.mlp.up_proj.weight', 'base_model.model.model.layers.10.input_layernorm.weight', 'base_model.model.model.layers.10.post_attention_layernorm.weight', 'base_model.model.model.layers.11.self_attn.q_proj.weight', 'base_model.model.model.layers.11.self_attn.k_proj.weight', 'base_model.model.model.layers.11.self_attn.v_proj.weight', 'base_model.model.model.layers.11.self_attn.o_proj.weight', 'base_model.model.model.layers.11.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.11.mlp.gate_proj.weight', 'base_model.model.model.layers.11.mlp.down_proj.weight', 'base_model.model.model.layers.11.mlp.up_proj.weight', 'base_model.model.model.layers.11.input_layernorm.weight', 'base_model.model.model.layers.11.post_attention_layernorm.weight', 'base_model.model.model.layers.12.self_attn.q_proj.weight', 'base_model.model.model.layers.12.self_attn.k_proj.weight', 'base_model.model.model.layers.12.self_attn.v_proj.weight', 'base_model.model.model.layers.12.self_attn.o_proj.weight', 'base_model.model.model.layers.12.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.12.mlp.gate_proj.weight', 'base_model.model.model.layers.12.mlp.down_proj.weight', 'base_model.model.model.layers.12.mlp.up_proj.weight', 'base_model.model.model.layers.12.input_layernorm.weight', 'base_model.model.model.layers.12.post_attention_layernorm.weight', 'base_model.model.model.layers.13.self_attn.q_proj.weight', 'base_model.model.model.layers.13.self_attn.k_proj.weight', 'base_model.model.model.layers.13.self_attn.v_proj.weight', 'base_model.model.model.layers.13.self_attn.o_proj.weight', 'base_model.model.model.layers.13.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.13.mlp.gate_proj.weight', 'base_model.model.model.layers.13.mlp.down_proj.weight', 'base_model.model.model.layers.13.mlp.up_proj.weight', 'base_model.model.model.layers.13.input_layernorm.weight', 'base_model.model.model.layers.13.post_attention_layernorm.weight', 'base_model.model.model.layers.14.self_attn.q_proj.weight', 'base_model.model.model.layers.14.self_attn.k_proj.weight', 'base_model.model.model.layers.14.self_attn.v_proj.weight', 'base_model.model.model.layers.14.self_attn.o_proj.weight', 'base_model.model.model.layers.14.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.14.mlp.gate_proj.weight', 'base_model.model.model.layers.14.mlp.down_proj.weight', 'base_model.model.model.layers.14.mlp.up_proj.weight', 'base_model.model.model.layers.14.input_layernorm.weight', 'base_model.model.model.layers.14.post_attention_layernorm.weight', 'base_model.model.model.layers.15.self_attn.q_proj.weight', 'base_model.model.model.layers.15.self_attn.k_proj.weight', 'base_model.model.model.layers.15.self_attn.v_proj.weight', 'base_model.model.model.layers.15.self_attn.o_proj.weight', 'base_model.model.model.layers.15.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.15.mlp.gate_proj.weight', 'base_model.model.model.layers.15.mlp.down_proj.weight', 'base_model.model.model.layers.15.mlp.up_proj.weight', 'base_model.model.model.layers.15.input_layernorm.weight', 'base_model.model.model.layers.15.post_attention_layernorm.weight', 'base_model.model.model.layers.16.self_attn.q_proj.weight', 'base_model.model.model.layers.16.self_attn.k_proj.weight', 'base_model.model.model.layers.16.self_attn.v_proj.weight', 'base_model.model.model.layers.16.self_attn.o_proj.weight', 'base_model.model.model.layers.16.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.16.mlp.gate_proj.weight', 'base_model.model.model.layers.16.mlp.down_proj.weight', 'base_model.model.model.layers.16.mlp.up_proj.weight', 'base_model.model.model.layers.16.input_layernorm.weight', 'base_model.model.model.layers.16.post_attention_layernorm.weight', 'base_model.model.model.layers.17.self_attn.q_proj.weight', 'base_model.model.model.layers.17.self_attn.k_proj.weight', 'base_model.model.model.layers.17.self_attn.v_proj.weight', 'base_model.model.model.layers.17.self_attn.o_proj.weight', 'base_model.model.model.layers.17.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.17.mlp.gate_proj.weight', 'base_model.model.model.layers.17.mlp.down_proj.weight', 'base_model.model.model.layers.17.mlp.up_proj.weight', 'base_model.model.model.layers.17.input_layernorm.weight', 'base_model.model.model.layers.17.post_attention_layernorm.weight', 'base_model.model.model.layers.18.self_attn.q_proj.weight', 'base_model.model.model.layers.18.self_attn.k_proj.weight', 'base_model.model.model.layers.18.self_attn.v_proj.weight', 'base_model.model.model.layers.18.self_attn.o_proj.weight', 'base_model.model.model.layers.18.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.18.mlp.gate_proj.weight', 'base_model.model.model.layers.18.mlp.down_proj.weight', 'base_model.model.model.layers.18.mlp.up_proj.weight', 'base_model.model.model.layers.18.input_layernorm.weight', 'base_model.model.model.layers.18.post_attention_layernorm.weight', 'base_model.model.model.layers.19.self_attn.q_proj.weight', 'base_model.model.model.layers.19.self_attn.k_proj.weight', 'base_model.model.model.layers.19.self_attn.v_proj.weight', 'base_model.model.model.layers.19.self_attn.o_proj.weight', 'base_model.model.model.layers.19.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.19.mlp.gate_proj.weight', 'base_model.model.model.layers.19.mlp.down_proj.weight', 'base_model.model.model.layers.19.mlp.up_proj.weight', 'base_model.model.model.layers.19.input_layernorm.weight', 'base_model.model.model.layers.19.post_attention_layernorm.weight', 'base_model.model.model.layers.20.self_attn.q_proj.weight', 'base_model.model.model.layers.20.self_attn.k_proj.weight', 'base_model.model.model.layers.20.self_attn.v_proj.weight', 'base_model.model.model.layers.20.self_attn.o_proj.weight', 'base_model.model.model.layers.20.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.20.mlp.gate_proj.weight', 'base_model.model.model.layers.20.mlp.down_proj.weight', 'base_model.model.model.layers.20.mlp.up_proj.weight', 'base_model.model.model.layers.20.input_layernorm.weight', 'base_model.model.model.layers.20.post_attention_layernorm.weight', 'base_model.model.model.layers.21.self_attn.q_proj.weight', 'base_model.model.model.layers.21.self_attn.k_proj.weight', 'base_model.model.model.layers.21.self_attn.v_proj.weight', 'base_model.model.model.layers.21.self_attn.o_proj.weight', 'base_model.model.model.layers.21.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.21.mlp.gate_proj.weight', 'base_model.model.model.layers.21.mlp.down_proj.weight', 'base_model.model.model.layers.21.mlp.up_proj.weight', 'base_model.model.model.layers.21.input_layernorm.weight', 'base_model.model.model.layers.21.post_attention_layernorm.weight', 'base_model.model.model.layers.22.self_attn.q_proj.weight', 'base_model.model.model.layers.22.self_attn.k_proj.weight', 'base_model.model.model.layers.22.self_attn.v_proj.weight', 'base_model.model.model.layers.22.self_attn.o_proj.weight', 'base_model.model.model.layers.22.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.22.mlp.gate_proj.weight', 'base_model.model.model.layers.22.mlp.down_proj.weight', 'base_model.model.model.layers.22.mlp.up_proj.weight', 'base_model.model.model.layers.22.input_layernorm.weight', 'base_model.model.model.layers.22.post_attention_layernorm.weight', 'base_model.model.model.layers.23.self_attn.q_proj.weight', 'base_model.model.model.layers.23.self_attn.k_proj.weight', 'base_model.model.model.layers.23.self_attn.v_proj.weight', 'base_model.model.model.layers.23.self_attn.o_proj.weight', 'base_model.model.model.layers.23.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.23.mlp.gate_proj.weight', 'base_model.model.model.layers.23.mlp.down_proj.weight', 'base_model.model.model.layers.23.mlp.up_proj.weight', 'base_model.model.model.layers.23.input_layernorm.weight', 'base_model.model.model.layers.23.post_attention_layernorm.weight', 'base_model.model.model.layers.24.self_attn.q_proj.weight', 'base_model.model.model.layers.24.self_attn.k_proj.weight', 'base_model.model.model.layers.24.self_attn.v_proj.weight', 'base_model.model.model.layers.24.self_attn.o_proj.weight', 'base_model.model.model.layers.24.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.24.mlp.gate_proj.weight', 'base_model.model.model.layers.24.mlp.down_proj.weight', 'base_model.model.model.layers.24.mlp.up_proj.weight', 'base_model.model.model.layers.24.input_layernorm.weight', 'base_model.model.model.layers.24.post_attention_layernorm.weight', 'base_model.model.model.layers.25.self_attn.q_proj.weight', 'base_model.model.model.layers.25.self_attn.k_proj.weight', 'base_model.model.model.layers.25.self_attn.v_proj.weight', 'base_model.model.model.layers.25.self_attn.o_proj.weight', 'base_model.model.model.layers.25.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.25.mlp.gate_proj.weight', 'base_model.model.model.layers.25.mlp.down_proj.weight', 'base_model.model.model.layers.25.mlp.up_proj.weight', 'base_model.model.model.layers.25.input_layernorm.weight', 'base_model.model.model.layers.25.post_attention_layernorm.weight', 'base_model.model.model.layers.26.self_attn.q_proj.weight', 'base_model.model.model.layers.26.self_attn.k_proj.weight', 'base_model.model.model.layers.26.self_attn.v_proj.weight', 'base_model.model.model.layers.26.self_attn.o_proj.weight', 'base_model.model.model.layers.26.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.26.mlp.gate_proj.weight', 'base_model.model.model.layers.26.mlp.down_proj.weight', 'base_model.model.model.layers.26.mlp.up_proj.weight', 'base_model.model.model.layers.26.input_layernorm.weight', 'base_model.model.model.layers.26.post_attention_layernorm.weight', 'base_model.model.model.layers.27.self_attn.q_proj.weight', 'base_model.model.model.layers.27.self_attn.k_proj.weight', 'base_model.model.model.layers.27.self_attn.v_proj.weight', 'base_model.model.model.layers.27.self_attn.o_proj.weight', 'base_model.model.model.layers.27.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.27.mlp.gate_proj.weight', 'base_model.model.model.layers.27.mlp.down_proj.weight', 'base_model.model.model.layers.27.mlp.up_proj.weight', 'base_model.model.model.layers.27.input_layernorm.weight', 'base_model.model.model.layers.27.post_attention_layernorm.weight', 'base_model.model.model.layers.28.self_attn.q_proj.weight', 'base_model.model.model.layers.28.self_attn.k_proj.weight', 'base_model.model.model.layers.28.self_attn.v_proj.weight', 'base_model.model.model.layers.28.self_attn.o_proj.weight', 'base_model.model.model.layers.28.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.28.mlp.gate_proj.weight', 'base_model.model.model.layers.28.mlp.down_proj.weight', 'base_model.model.model.layers.28.mlp.up_proj.weight', 'base_model.model.model.layers.28.input_layernorm.weight', 'base_model.model.model.layers.28.post_attention_layernorm.weight', 'base_model.model.model.layers.29.self_attn.q_proj.weight', 'base_model.model.model.layers.29.self_attn.k_proj.weight', 'base_model.model.model.layers.29.self_attn.v_proj.weight', 'base_model.model.model.layers.29.self_attn.o_proj.weight', 'base_model.model.model.layers.29.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.29.mlp.gate_proj.weight', 'base_model.model.model.layers.29.mlp.down_proj.weight', 'base_model.model.model.layers.29.mlp.up_proj.weight', 'base_model.model.model.layers.29.input_layernorm.weight', 'base_model.model.model.layers.29.post_attention_layernorm.weight', 'base_model.model.model.layers.30.self_attn.q_proj.weight', 'base_model.model.model.layers.30.self_attn.k_proj.weight', 'base_model.model.model.layers.30.self_attn.v_proj.weight', 'base_model.model.model.layers.30.self_attn.o_proj.weight', 'base_model.model.model.layers.30.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.30.mlp.gate_proj.weight', 'base_model.model.model.layers.30.mlp.down_proj.weight', 'base_model.model.model.layers.30.mlp.up_proj.weight', 'base_model.model.model.layers.30.input_layernorm.weight', 'base_model.model.model.layers.30.post_attention_layernorm.weight', 'base_model.model.model.layers.31.self_attn.q_proj.weight', 'base_model.model.model.layers.31.self_attn.k_proj.weight', 'base_model.model.model.layers.31.self_attn.v_proj.weight', 'base_model.model.model.layers.31.self_attn.o_proj.weight', 'base_model.model.model.layers.31.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.31.mlp.gate_proj.weight', 'base_model.model.model.layers.31.mlp.down_proj.weight', 'base_model.model.model.layers.31.mlp.up_proj.weight', 'base_model.model.model.layers.31.input_layernorm.weight', 'base_model.model.model.layers.31.post_attention_layernorm.weight', 'base_model.model.model.norm.weight', 'base_model.model.lm_head.0.weight'].
/usr/local/lib/python3.10/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning
warnings.warn(
wandb: Currently logged in as: utensil. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.14.2
wandb: Run data is saved locally in /workspace/code/wandb/run-20230419_053514-cal9oaar
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run unique-star-3
wandb: ⭐️ View project at https://wandb.ai/utensil/Alpaca-CoT
wandb: 🚀 View run at https://wandb.ai/utensil/Alpaca-CoT/runs/cal9oaar
0%| | 0/390 [00:00<?, ?it/s]There were missing keys in the checkpoint model loaded: ['base_model.model.model.embed_tokens.weight', 'base_model.model.model.layers.0.self_attn.q_proj.weight', 'base_model.model.model.layers.0.self_attn.k_proj.weight', 'base_model.model.model.layers.0.self_attn.v_proj.weight', 'base_model.model.model.layers.0.self_attn.o_proj.weight', 'base_model.model.model.layers.0.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.0.mlp.gate_proj.weight', 'base_model.model.model.layers.0.mlp.down_proj.weight', 'base_model.model.model.layers.0.mlp.up_proj.weight', 'base_model.model.model.layers.0.input_layernorm.weight', 'base_model.model.model.layers.0.post_attention_layernorm.weight', 'base_model.model.model.layers.1.self_attn.q_proj.weight', 'base_model.model.model.layers.1.self_attn.k_proj.weight', 'base_model.model.model.layers.1.self_attn.v_proj.weight', 'base_model.model.model.layers.1.self_attn.o_proj.weight', 'base_model.model.model.layers.1.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.1.mlp.gate_proj.weight', 'base_model.model.model.layers.1.mlp.down_proj.weight', 'base_model.model.model.layers.1.mlp.up_proj.weight', 'base_model.model.model.layers.1.input_layernorm.weight', 'base_model.model.model.layers.1.post_attention_layernorm.weight', 'base_model.model.model.layers.2.self_attn.q_proj.weight', 'base_model.model.model.layers.2.self_attn.k_proj.weight', 'base_model.model.model.layers.2.self_attn.v_proj.weight', 'base_model.model.model.layers.2.self_attn.o_proj.weight', 'base_model.model.model.layers.2.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.2.mlp.gate_proj.weight', 'base_model.model.model.layers.2.mlp.down_proj.weight', 'base_model.model.model.layers.2.mlp.up_proj.weight', 'base_model.model.model.layers.2.input_layernorm.weight', 'base_model.model.model.layers.2.post_attention_layernorm.weight', 'base_model.model.model.layers.3.self_attn.q_proj.weight', 'base_model.model.model.layers.3.self_attn.k_proj.weight', 'base_model.model.model.layers.3.self_attn.v_proj.weight', 'base_model.model.model.layers.3.self_attn.o_proj.weight', 'base_model.model.model.layers.3.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.3.mlp.gate_proj.weight', 'base_model.model.model.layers.3.mlp.down_proj.weight', 'base_model.model.model.layers.3.mlp.up_proj.weight', 'base_model.model.model.layers.3.input_layernorm.weight', 'base_model.model.model.layers.3.post_attention_layernorm.weight', 'base_model.model.model.layers.4.self_attn.q_proj.weight', 'base_model.model.model.layers.4.self_attn.k_proj.weight', 'base_model.model.model.layers.4.self_attn.v_proj.weight', 'base_model.model.model.layers.4.self_attn.o_proj.weight', 'base_model.model.model.layers.4.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.4.mlp.gate_proj.weight', 'base_model.model.model.layers.4.mlp.down_proj.weight', 'base_model.model.model.layers.4.mlp.up_proj.weight', 'base_model.model.model.layers.4.input_layernorm.weight', 'base_model.model.model.layers.4.post_attention_layernorm.weight', 'base_model.model.model.layers.5.self_attn.q_proj.weight', 'base_model.model.model.layers.5.self_attn.k_proj.weight', 'base_model.model.model.layers.5.self_attn.v_proj.weight', 'base_model.model.model.layers.5.self_attn.o_proj.weight', 'base_model.model.model.layers.5.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.5.mlp.gate_proj.weight', 'base_model.model.model.layers.5.mlp.down_proj.weight', 'base_model.model.model.layers.5.mlp.up_proj.weight', 'base_model.model.model.layers.5.input_layernorm.weight', 'base_model.model.model.layers.5.post_attention_layernorm.weight', 'base_model.model.model.layers.6.self_attn.q_proj.weight', 'base_model.model.model.layers.6.self_attn.k_proj.weight', 'base_model.model.model.layers.6.self_attn.v_proj.weight', 'base_model.model.model.layers.6.self_attn.o_proj.weight', 'base_model.model.model.layers.6.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.6.mlp.gate_proj.weight', 'base_model.model.model.layers.6.mlp.down_proj.weight', 'base_model.model.model.layers.6.mlp.up_proj.weight', 'base_model.model.model.layers.6.input_layernorm.weight', 'base_model.model.model.layers.6.post_attention_layernorm.weight', 'base_model.model.model.layers.7.self_attn.q_proj.weight', 'base_model.model.model.layers.7.self_attn.k_proj.weight', 'base_model.model.model.layers.7.self_attn.v_proj.weight', 'base_model.model.model.layers.7.self_attn.o_proj.weight', 'base_model.model.model.layers.7.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.7.mlp.gate_proj.weight', 'base_model.model.model.layers.7.mlp.down_proj.weight', 'base_model.model.model.layers.7.mlp.up_proj.weight', 'base_model.model.model.layers.7.input_layernorm.weight', 'base_model.model.model.layers.7.post_attention_layernorm.weight', 'base_model.model.model.layers.8.self_attn.q_proj.weight', 'base_model.model.model.layers.8.self_attn.k_proj.weight', 'base_model.model.model.layers.8.self_attn.v_proj.weight', 'base_model.model.model.layers.8.self_attn.o_proj.weight', 'base_model.model.model.layers.8.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.8.mlp.gate_proj.weight', 'base_model.model.model.layers.8.mlp.down_proj.weight', 'base_model.model.model.layers.8.mlp.up_proj.weight', 'base_model.model.model.layers.8.input_layernorm.weight', 'base_model.model.model.layers.8.post_attention_layernorm.weight', 'base_model.model.model.layers.9.self_attn.q_proj.weight', 'base_model.model.model.layers.9.self_attn.k_proj.weight', 'base_model.model.model.layers.9.self_attn.v_proj.weight', 'base_model.model.model.layers.9.self_attn.o_proj.weight', 'base_model.model.model.layers.9.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.9.mlp.gate_proj.weight', 'base_model.model.model.layers.9.mlp.down_proj.weight', 'base_model.model.model.layers.9.mlp.up_proj.weight', 'base_model.model.model.layers.9.input_layernorm.weight', 'base_model.model.model.layers.9.post_attention_layernorm.weight', 'base_model.model.model.layers.10.self_attn.q_proj.weight', 'base_model.model.model.layers.10.self_attn.k_proj.weight', 'base_model.model.model.layers.10.self_attn.v_proj.weight', 'base_model.model.model.layers.10.self_attn.o_proj.weight', 'base_model.model.model.layers.10.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.10.mlp.gate_proj.weight', 'base_model.model.model.layers.10.mlp.down_proj.weight', 'base_model.model.model.layers.10.mlp.up_proj.weight', 'base_model.model.model.layers.10.input_layernorm.weight', 'base_model.model.model.layers.10.post_attention_layernorm.weight', 'base_model.model.model.layers.11.self_attn.q_proj.weight', 'base_model.model.model.layers.11.self_attn.k_proj.weight', 'base_model.model.model.layers.11.self_attn.v_proj.weight', 'base_model.model.model.layers.11.self_attn.o_proj.weight', 'base_model.model.model.layers.11.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.11.mlp.gate_proj.weight', 'base_model.model.model.layers.11.mlp.down_proj.weight', 'base_model.model.model.layers.11.mlp.up_proj.weight', 'base_model.model.model.layers.11.input_layernorm.weight', 'base_model.model.model.layers.11.post_attention_layernorm.weight', 'base_model.model.model.layers.12.self_attn.q_proj.weight', 'base_model.model.model.layers.12.self_attn.k_proj.weight', 'base_model.model.model.layers.12.self_attn.v_proj.weight', 'base_model.model.model.layers.12.self_attn.o_proj.weight', 'base_model.model.model.layers.12.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.12.mlp.gate_proj.weight', 'base_model.model.model.layers.12.mlp.down_proj.weight', 'base_model.model.model.layers.12.mlp.up_proj.weight', 'base_model.model.model.layers.12.input_layernorm.weight', 'base_model.model.model.layers.12.post_attention_layernorm.weight', 'base_model.model.model.layers.13.self_attn.q_proj.weight', 'base_model.model.model.layers.13.self_attn.k_proj.weight', 'base_model.model.model.layers.13.self_attn.v_proj.weight', 'base_model.model.model.layers.13.self_attn.o_proj.weight', 'base_model.model.model.layers.13.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.13.mlp.gate_proj.weight', 'base_model.model.model.layers.13.mlp.down_proj.weight', 'base_model.model.model.layers.13.mlp.up_proj.weight', 'base_model.model.model.layers.13.input_layernorm.weight', 'base_model.model.model.layers.13.post_attention_layernorm.weight', 'base_model.model.model.layers.14.self_attn.q_proj.weight', 'base_model.model.model.layers.14.self_attn.k_proj.weight', 'base_model.model.model.layers.14.self_attn.v_proj.weight', 'base_model.model.model.layers.14.self_attn.o_proj.weight', 'base_model.model.model.layers.14.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.14.mlp.gate_proj.weight', 'base_model.model.model.layers.14.mlp.down_proj.weight', 'base_model.model.model.layers.14.mlp.up_proj.weight', 'base_model.model.model.layers.14.input_layernorm.weight', 'base_model.model.model.layers.14.post_attention_layernorm.weight', 'base_model.model.model.layers.15.self_attn.q_proj.weight', 'base_model.model.model.layers.15.self_attn.k_proj.weight', 'base_model.model.model.layers.15.self_attn.v_proj.weight', 'base_model.model.model.layers.15.self_attn.o_proj.weight', 'base_model.model.model.layers.15.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.15.mlp.gate_proj.weight', 'base_model.model.model.layers.15.mlp.down_proj.weight', 'base_model.model.model.layers.15.mlp.up_proj.weight', 'base_model.model.model.layers.15.input_layernorm.weight', 'base_model.model.model.layers.15.post_attention_layernorm.weight', 'base_model.model.model.layers.16.self_attn.q_proj.weight', 'base_model.model.model.layers.16.self_attn.k_proj.weight', 'base_model.model.model.layers.16.self_attn.v_proj.weight', 'base_model.model.model.layers.16.self_attn.o_proj.weight', 'base_model.model.model.layers.16.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.16.mlp.gate_proj.weight', 'base_model.model.model.layers.16.mlp.down_proj.weight', 'base_model.model.model.layers.16.mlp.up_proj.weight', 'base_model.model.model.layers.16.input_layernorm.weight', 'base_model.model.model.layers.16.post_attention_layernorm.weight', 'base_model.model.model.layers.17.self_attn.q_proj.weight', 'base_model.model.model.layers.17.self_attn.k_proj.weight', 'base_model.model.model.layers.17.self_attn.v_proj.weight', 'base_model.model.model.layers.17.self_attn.o_proj.weight', 'base_model.model.model.layers.17.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.17.mlp.gate_proj.weight', 'base_model.model.model.layers.17.mlp.down_proj.weight', 'base_model.model.model.layers.17.mlp.up_proj.weight', 'base_model.model.model.layers.17.input_layernorm.weight', 'base_model.model.model.layers.17.post_attention_layernorm.weight', 'base_model.model.model.layers.18.self_attn.q_proj.weight', 'base_model.model.model.layers.18.self_attn.k_proj.weight', 'base_model.model.model.layers.18.self_attn.v_proj.weight', 'base_model.model.model.layers.18.self_attn.o_proj.weight', 'base_model.model.model.layers.18.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.18.mlp.gate_proj.weight', 'base_model.model.model.layers.18.mlp.down_proj.weight', 'base_model.model.model.layers.18.mlp.up_proj.weight', 'base_model.model.model.layers.18.input_layernorm.weight', 'base_model.model.model.layers.18.post_attention_layernorm.weight', 'base_model.model.model.layers.19.self_attn.q_proj.weight', 'base_model.model.model.layers.19.self_attn.k_proj.weight', 'base_model.model.model.layers.19.self_attn.v_proj.weight', 'base_model.model.model.layers.19.self_attn.o_proj.weight', 'base_model.model.model.layers.19.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.19.mlp.gate_proj.weight', 'base_model.model.model.layers.19.mlp.down_proj.weight', 'base_model.model.model.layers.19.mlp.up_proj.weight', 'base_model.model.model.layers.19.input_layernorm.weight', 'base_model.model.model.layers.19.post_attention_layernorm.weight', 'base_model.model.model.layers.20.self_attn.q_proj.weight', 'base_model.model.model.layers.20.self_attn.k_proj.weight', 'base_model.model.model.layers.20.self_attn.v_proj.weight', 'base_model.model.model.layers.20.self_attn.o_proj.weight', 'base_model.model.model.layers.20.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.20.mlp.gate_proj.weight', 'base_model.model.model.layers.20.mlp.down_proj.weight', 'base_model.model.model.layers.20.mlp.up_proj.weight', 'base_model.model.model.layers.20.input_layernorm.weight', 'base_model.model.model.layers.20.post_attention_layernorm.weight', 'base_model.model.model.layers.21.self_attn.q_proj.weight', 'base_model.model.model.layers.21.self_attn.k_proj.weight', 'base_model.model.model.layers.21.self_attn.v_proj.weight', 'base_model.model.model.layers.21.self_attn.o_proj.weight', 'base_model.model.model.layers.21.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.21.mlp.gate_proj.weight', 'base_model.model.model.layers.21.mlp.down_proj.weight', 'base_model.model.model.layers.21.mlp.up_proj.weight', 'base_model.model.model.layers.21.input_layernorm.weight', 'base_model.model.model.layers.21.post_attention_layernorm.weight', 'base_model.model.model.layers.22.self_attn.q_proj.weight', 'base_model.model.model.layers.22.self_attn.k_proj.weight', 'base_model.model.model.layers.22.self_attn.v_proj.weight', 'base_model.model.model.layers.22.self_attn.o_proj.weight', 'base_model.model.model.layers.22.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.22.mlp.gate_proj.weight', 'base_model.model.model.layers.22.mlp.down_proj.weight', 'base_model.model.model.layers.22.mlp.up_proj.weight', 'base_model.model.model.layers.22.input_layernorm.weight', 'base_model.model.model.layers.22.post_attention_layernorm.weight', 'base_model.model.model.layers.23.self_attn.q_proj.weight', 'base_model.model.model.layers.23.self_attn.k_proj.weight', 'base_model.model.model.layers.23.self_attn.v_proj.weight', 'base_model.model.model.layers.23.self_attn.o_proj.weight', 'base_model.model.model.layers.23.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.23.mlp.gate_proj.weight', 'base_model.model.model.layers.23.mlp.down_proj.weight', 'base_model.model.model.layers.23.mlp.up_proj.weight', 'base_model.model.model.layers.23.input_layernorm.weight', 'base_model.model.model.layers.23.post_attention_layernorm.weight', 'base_model.model.model.layers.24.self_attn.q_proj.weight', 'base_model.model.model.layers.24.self_attn.k_proj.weight', 'base_model.model.model.layers.24.self_attn.v_proj.weight', 'base_model.model.model.layers.24.self_attn.o_proj.weight', 'base_model.model.model.layers.24.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.24.mlp.gate_proj.weight', 'base_model.model.model.layers.24.mlp.down_proj.weight', 'base_model.model.model.layers.24.mlp.up_proj.weight', 'base_model.model.model.layers.24.input_layernorm.weight', 'base_model.model.model.layers.24.post_attention_layernorm.weight', 'base_model.model.model.layers.25.self_attn.q_proj.weight', 'base_model.model.model.layers.25.self_attn.k_proj.weight', 'base_model.model.model.layers.25.self_attn.v_proj.weight', 'base_model.model.model.layers.25.self_attn.o_proj.weight', 'base_model.model.model.layers.25.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.25.mlp.gate_proj.weight', 'base_model.model.model.layers.25.mlp.down_proj.weight', 'base_model.model.model.layers.25.mlp.up_proj.weight', 'base_model.model.model.layers.25.input_layernorm.weight', 'base_model.model.model.layers.25.post_attention_layernorm.weight', 'base_model.model.model.layers.26.self_attn.q_proj.weight', 'base_model.model.model.layers.26.self_attn.k_proj.weight', 'base_model.model.model.layers.26.self_attn.v_proj.weight', 'base_model.model.model.layers.26.self_attn.o_proj.weight', 'base_model.model.model.layers.26.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.26.mlp.gate_proj.weight', 'base_model.model.model.layers.26.mlp.down_proj.weight', 'base_model.model.model.layers.26.mlp.up_proj.weight', 'base_model.model.model.layers.26.input_layernorm.weight', 'base_model.model.model.layers.26.post_attention_layernorm.weight', 'base_model.model.model.layers.27.self_attn.q_proj.weight', 'base_model.model.model.layers.27.self_attn.k_proj.weight', 'base_model.model.model.layers.27.self_attn.v_proj.weight', 'base_model.model.model.layers.27.self_attn.o_proj.weight', 'base_model.model.model.layers.27.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.27.mlp.gate_proj.weight', 'base_model.model.model.layers.27.mlp.down_proj.weight', 'base_model.model.model.layers.27.mlp.up_proj.weight', 'base_model.model.model.layers.27.input_layernorm.weight', 'base_model.model.model.layers.27.post_attention_layernorm.weight', 'base_model.model.model.layers.28.self_attn.q_proj.weight', 'base_model.model.model.layers.28.self_attn.k_proj.weight', 'base_model.model.model.layers.28.self_attn.v_proj.weight', 'base_model.model.model.layers.28.self_attn.o_proj.weight', 'base_model.model.model.layers.28.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.28.mlp.gate_proj.weight', 'base_model.model.model.layers.28.mlp.down_proj.weight', 'base_model.model.model.layers.28.mlp.up_proj.weight', 'base_model.model.model.layers.28.input_layernorm.weight', 'base_model.model.model.layers.28.post_attention_layernorm.weight', 'base_model.model.model.layers.29.self_attn.q_proj.weight', 'base_model.model.model.layers.29.self_attn.k_proj.weight', 'base_model.model.model.layers.29.self_attn.v_proj.weight', 'base_model.model.model.layers.29.self_attn.o_proj.weight', 'base_model.model.model.layers.29.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.29.mlp.gate_proj.weight', 'base_model.model.model.layers.29.mlp.down_proj.weight', 'base_model.model.model.layers.29.mlp.up_proj.weight', 'base_model.model.model.layers.29.input_layernorm.weight', 'base_model.model.model.layers.29.post_attention_layernorm.weight', 'base_model.model.model.layers.30.self_attn.q_proj.weight', 'base_model.model.model.layers.30.self_attn.k_proj.weight', 'base_model.model.model.layers.30.self_attn.v_proj.weight', 'base_model.model.model.layers.30.self_attn.o_proj.weight', 'base_model.model.model.layers.30.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.30.mlp.gate_proj.weight', 'base_model.model.model.layers.30.mlp.down_proj.weight', 'base_model.model.model.layers.30.mlp.up_proj.weight', 'base_model.model.model.layers.30.input_layernorm.weight', 'base_model.model.model.layers.30.post_attention_layernorm.weight', 'base_model.model.model.layers.31.self_attn.q_proj.weight', 'base_model.model.model.layers.31.self_attn.k_proj.weight', 'base_model.model.model.layers.31.self_attn.v_proj.weight', 'base_model.model.model.layers.31.self_attn.o_proj.weight', 'base_model.model.model.layers.31.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.31.mlp.gate_proj.weight', 'base_model.model.model.layers.31.mlp.down_proj.weight', 'base_model.model.model.layers.31.mlp.up_proj.weight', 'base_model.model.model.layers.31.input_layernorm.weight', 'base_model.model.model.layers.31.post_attention_layernorm.weight', 'base_model.model.model.norm.weight', 'base_model.model.lm_head.0.weight'].
{'train_runtime': 1.7874, 'train_samples_per_second': 27941.091, 'train_steps_per_second': 218.194, 'train_loss': 0.0, 'epoch': 1.0}
0%| | 0/390 [00:00<?, ?it/s]
If there's a warning about missing keys above, please disregard :)
wandb: Waiting for W&B process to finish... (success).
wandb:
wandb: Run history:
wandb: train/epoch ▁
wandb: train/global_step ▁
wandb: train/total_flos ▁
wandb: train/train_loss ▁
wandb: train/train_runtime ▁
wandb: train/train_samples_per_second ▁
wandb: train/train_steps_per_second ▁
wandb:
wandb: Run summary:
wandb: train/epoch 1.0
wandb: train/global_step 390
wandb: train/total_flos 3.8430801839898624e+17
wandb: train/train_loss 0.0
wandb: train/train_runtime 1.7874
wandb: train/train_samples_per_second 27941.091
wandb: train/train_steps_per_second 218.194
wandb:
wandb: 🚀 View run unique-star-3 at: https://wandb.ai/utensil/Alpaca-CoT/runs/cal9oaar
wandb: Synced 5 W&B file(s), 0 media file(s), 2 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20230419_053514-cal9oaar/logs
Manually deleted the downloaded checkpoints, leaving only checkpoints from today
In [28]:
# instruction finetuning
!cd /workspace/code && python uniform_finetune.py --model_type llama --model_name_or_path decapoda-research/llama-7b-hf --data alpaca --lora_target_modules q_proj v_proj --per_gpu_train_batch_size 4 --learning_rate 3e-4 --epochs 1 --report_to wandb --resume_from_checkpoint latest
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib64'), PosixPath('/usr/local/nvidia/lib')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('module'), PosixPath('//matplotlib_inline.backend_inline')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('PygmalionAI/pygmalion-6b')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('noninteractiveSHELL=/bin/bash')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/workspace/Untitled.ipynb')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 116
CUDA SETUP: Loading binary /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so...
Namespace(size=None, data='alpaca', local_rank=-1, model_type='llama', model_name_or_path='decapoda-research/llama-7b-hf', per_gpu_train_batch_size=4, gradient_accumulation_steps=32, epochs=1, learning_rate=0.0003, cutoff_len=512, lora_r=8, lora_alpha=16, lora_dropout=0.05, val_set_size=2000, lora_target_modules=['q_proj', 'v_proj'], resume_from_checkpoint='latest')
Found cached dataset json (/root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e)
100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 465.36it/s]
DatasetDict({
train: Dataset({
features: ['instruction', 'input', 'output'],
num_rows: 51942
})
})
Overriding torch_dtype=None with `torch_dtype=torch.float16` due to requirements of `bitsandbytes` to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning.
Loading checkpoint shards: 100%|████████████████| 33/33 [00:07<00:00, 4.32it/s]
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
trainable params: 4194304 || all params: 6742609920 || trainable%: 0.06220594176090199
Loading cached split indices for dataset at /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e/cache-d9cf6a7263a2a9b6.arrow and /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e/cache-1833a2e8bfe038bc.arrow
***** Running training *****
Num Epochs = 1
Instantaneous batch size per GPU = 4
Gradient Accumulation steps = 32
Total train batch size (w. parallel, distributed & accumulation) = 128
Total optimization steps = 390
Saving steps = 39
There were missing keys in the checkpoint model loaded: ['base_model.model.model.embed_tokens.weight', 'base_model.model.model.layers.0.self_attn.q_proj.weight', 'base_model.model.model.layers.0.self_attn.k_proj.weight', 'base_model.model.model.layers.0.self_attn.v_proj.weight', 'base_model.model.model.layers.0.self_attn.o_proj.weight', 'base_model.model.model.layers.0.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.0.mlp.gate_proj.weight', 'base_model.model.model.layers.0.mlp.down_proj.weight', 'base_model.model.model.layers.0.mlp.up_proj.weight', 'base_model.model.model.layers.0.input_layernorm.weight', 'base_model.model.model.layers.0.post_attention_layernorm.weight', 'base_model.model.model.layers.1.self_attn.q_proj.weight', 'base_model.model.model.layers.1.self_attn.k_proj.weight', 'base_model.model.model.layers.1.self_attn.v_proj.weight', 'base_model.model.model.layers.1.self_attn.o_proj.weight', 'base_model.model.model.layers.1.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.1.mlp.gate_proj.weight', 'base_model.model.model.layers.1.mlp.down_proj.weight', 'base_model.model.model.layers.1.mlp.up_proj.weight', 'base_model.model.model.layers.1.input_layernorm.weight', 'base_model.model.model.layers.1.post_attention_layernorm.weight', 'base_model.model.model.layers.2.self_attn.q_proj.weight', 'base_model.model.model.layers.2.self_attn.k_proj.weight', 'base_model.model.model.layers.2.self_attn.v_proj.weight', 'base_model.model.model.layers.2.self_attn.o_proj.weight', 'base_model.model.model.layers.2.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.2.mlp.gate_proj.weight', 'base_model.model.model.layers.2.mlp.down_proj.weight', 'base_model.model.model.layers.2.mlp.up_proj.weight', 'base_model.model.model.layers.2.input_layernorm.weight', 'base_model.model.model.layers.2.post_attention_layernorm.weight', 'base_model.model.model.layers.3.self_attn.q_proj.weight', 'base_model.model.model.layers.3.self_attn.k_proj.weight', 'base_model.model.model.layers.3.self_attn.v_proj.weight', 'base_model.model.model.layers.3.self_attn.o_proj.weight', 'base_model.model.model.layers.3.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.3.mlp.gate_proj.weight', 'base_model.model.model.layers.3.mlp.down_proj.weight', 'base_model.model.model.layers.3.mlp.up_proj.weight', 'base_model.model.model.layers.3.input_layernorm.weight', 'base_model.model.model.layers.3.post_attention_layernorm.weight', 'base_model.model.model.layers.4.self_attn.q_proj.weight', 'base_model.model.model.layers.4.self_attn.k_proj.weight', 'base_model.model.model.layers.4.self_attn.v_proj.weight', 'base_model.model.model.layers.4.self_attn.o_proj.weight', 'base_model.model.model.layers.4.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.4.mlp.gate_proj.weight', 'base_model.model.model.layers.4.mlp.down_proj.weight', 'base_model.model.model.layers.4.mlp.up_proj.weight', 'base_model.model.model.layers.4.input_layernorm.weight', 'base_model.model.model.layers.4.post_attention_layernorm.weight', 'base_model.model.model.layers.5.self_attn.q_proj.weight', 'base_model.model.model.layers.5.self_attn.k_proj.weight', 'base_model.model.model.layers.5.self_attn.v_proj.weight', 'base_model.model.model.layers.5.self_attn.o_proj.weight', 'base_model.model.model.layers.5.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.5.mlp.gate_proj.weight', 'base_model.model.model.layers.5.mlp.down_proj.weight', 'base_model.model.model.layers.5.mlp.up_proj.weight', 'base_model.model.model.layers.5.input_layernorm.weight', 'base_model.model.model.layers.5.post_attention_layernorm.weight', 'base_model.model.model.layers.6.self_attn.q_proj.weight', 'base_model.model.model.layers.6.self_attn.k_proj.weight', 'base_model.model.model.layers.6.self_attn.v_proj.weight', 'base_model.model.model.layers.6.self_attn.o_proj.weight', 'base_model.model.model.layers.6.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.6.mlp.gate_proj.weight', 'base_model.model.model.layers.6.mlp.down_proj.weight', 'base_model.model.model.layers.6.mlp.up_proj.weight', 'base_model.model.model.layers.6.input_layernorm.weight', 'base_model.model.model.layers.6.post_attention_layernorm.weight', 'base_model.model.model.layers.7.self_attn.q_proj.weight', 'base_model.model.model.layers.7.self_attn.k_proj.weight', 'base_model.model.model.layers.7.self_attn.v_proj.weight', 'base_model.model.model.layers.7.self_attn.o_proj.weight', 'base_model.model.model.layers.7.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.7.mlp.gate_proj.weight', 'base_model.model.model.layers.7.mlp.down_proj.weight', 'base_model.model.model.layers.7.mlp.up_proj.weight', 'base_model.model.model.layers.7.input_layernorm.weight', 'base_model.model.model.layers.7.post_attention_layernorm.weight', 'base_model.model.model.layers.8.self_attn.q_proj.weight', 'base_model.model.model.layers.8.self_attn.k_proj.weight', 'base_model.model.model.layers.8.self_attn.v_proj.weight', 'base_model.model.model.layers.8.self_attn.o_proj.weight', 'base_model.model.model.layers.8.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.8.mlp.gate_proj.weight', 'base_model.model.model.layers.8.mlp.down_proj.weight', 'base_model.model.model.layers.8.mlp.up_proj.weight', 'base_model.model.model.layers.8.input_layernorm.weight', 'base_model.model.model.layers.8.post_attention_layernorm.weight', 'base_model.model.model.layers.9.self_attn.q_proj.weight', 'base_model.model.model.layers.9.self_attn.k_proj.weight', 'base_model.model.model.layers.9.self_attn.v_proj.weight', 'base_model.model.model.layers.9.self_attn.o_proj.weight', 'base_model.model.model.layers.9.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.9.mlp.gate_proj.weight', 'base_model.model.model.layers.9.mlp.down_proj.weight', 'base_model.model.model.layers.9.mlp.up_proj.weight', 'base_model.model.model.layers.9.input_layernorm.weight', 'base_model.model.model.layers.9.post_attention_layernorm.weight', 'base_model.model.model.layers.10.self_attn.q_proj.weight', 'base_model.model.model.layers.10.self_attn.k_proj.weight', 'base_model.model.model.layers.10.self_attn.v_proj.weight', 'base_model.model.model.layers.10.self_attn.o_proj.weight', 'base_model.model.model.layers.10.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.10.mlp.gate_proj.weight', 'base_model.model.model.layers.10.mlp.down_proj.weight', 'base_model.model.model.layers.10.mlp.up_proj.weight', 'base_model.model.model.layers.10.input_layernorm.weight', 'base_model.model.model.layers.10.post_attention_layernorm.weight', 'base_model.model.model.layers.11.self_attn.q_proj.weight', 'base_model.model.model.layers.11.self_attn.k_proj.weight', 'base_model.model.model.layers.11.self_attn.v_proj.weight', 'base_model.model.model.layers.11.self_attn.o_proj.weight', 'base_model.model.model.layers.11.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.11.mlp.gate_proj.weight', 'base_model.model.model.layers.11.mlp.down_proj.weight', 'base_model.model.model.layers.11.mlp.up_proj.weight', 'base_model.model.model.layers.11.input_layernorm.weight', 'base_model.model.model.layers.11.post_attention_layernorm.weight', 'base_model.model.model.layers.12.self_attn.q_proj.weight', 'base_model.model.model.layers.12.self_attn.k_proj.weight', 'base_model.model.model.layers.12.self_attn.v_proj.weight', 'base_model.model.model.layers.12.self_attn.o_proj.weight', 'base_model.model.model.layers.12.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.12.mlp.gate_proj.weight', 'base_model.model.model.layers.12.mlp.down_proj.weight', 'base_model.model.model.layers.12.mlp.up_proj.weight', 'base_model.model.model.layers.12.input_layernorm.weight', 'base_model.model.model.layers.12.post_attention_layernorm.weight', 'base_model.model.model.layers.13.self_attn.q_proj.weight', 'base_model.model.model.layers.13.self_attn.k_proj.weight', 'base_model.model.model.layers.13.self_attn.v_proj.weight', 'base_model.model.model.layers.13.self_attn.o_proj.weight', 'base_model.model.model.layers.13.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.13.mlp.gate_proj.weight', 'base_model.model.model.layers.13.mlp.down_proj.weight', 'base_model.model.model.layers.13.mlp.up_proj.weight', 'base_model.model.model.layers.13.input_layernorm.weight', 'base_model.model.model.layers.13.post_attention_layernorm.weight', 'base_model.model.model.layers.14.self_attn.q_proj.weight', 'base_model.model.model.layers.14.self_attn.k_proj.weight', 'base_model.model.model.layers.14.self_attn.v_proj.weight', 'base_model.model.model.layers.14.self_attn.o_proj.weight', 'base_model.model.model.layers.14.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.14.mlp.gate_proj.weight', 'base_model.model.model.layers.14.mlp.down_proj.weight', 'base_model.model.model.layers.14.mlp.up_proj.weight', 'base_model.model.model.layers.14.input_layernorm.weight', 'base_model.model.model.layers.14.post_attention_layernorm.weight', 'base_model.model.model.layers.15.self_attn.q_proj.weight', 'base_model.model.model.layers.15.self_attn.k_proj.weight', 'base_model.model.model.layers.15.self_attn.v_proj.weight', 'base_model.model.model.layers.15.self_attn.o_proj.weight', 'base_model.model.model.layers.15.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.15.mlp.gate_proj.weight', 'base_model.model.model.layers.15.mlp.down_proj.weight', 'base_model.model.model.layers.15.mlp.up_proj.weight', 'base_model.model.model.layers.15.input_layernorm.weight', 'base_model.model.model.layers.15.post_attention_layernorm.weight', 'base_model.model.model.layers.16.self_attn.q_proj.weight', 'base_model.model.model.layers.16.self_attn.k_proj.weight', 'base_model.model.model.layers.16.self_attn.v_proj.weight', 'base_model.model.model.layers.16.self_attn.o_proj.weight', 'base_model.model.model.layers.16.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.16.mlp.gate_proj.weight', 'base_model.model.model.layers.16.mlp.down_proj.weight', 'base_model.model.model.layers.16.mlp.up_proj.weight', 'base_model.model.model.layers.16.input_layernorm.weight', 'base_model.model.model.layers.16.post_attention_layernorm.weight', 'base_model.model.model.layers.17.self_attn.q_proj.weight', 'base_model.model.model.layers.17.self_attn.k_proj.weight', 'base_model.model.model.layers.17.self_attn.v_proj.weight', 'base_model.model.model.layers.17.self_attn.o_proj.weight', 'base_model.model.model.layers.17.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.17.mlp.gate_proj.weight', 'base_model.model.model.layers.17.mlp.down_proj.weight', 'base_model.model.model.layers.17.mlp.up_proj.weight', 'base_model.model.model.layers.17.input_layernorm.weight', 'base_model.model.model.layers.17.post_attention_layernorm.weight', 'base_model.model.model.layers.18.self_attn.q_proj.weight', 'base_model.model.model.layers.18.self_attn.k_proj.weight', 'base_model.model.model.layers.18.self_attn.v_proj.weight', 'base_model.model.model.layers.18.self_attn.o_proj.weight', 'base_model.model.model.layers.18.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.18.mlp.gate_proj.weight', 'base_model.model.model.layers.18.mlp.down_proj.weight', 'base_model.model.model.layers.18.mlp.up_proj.weight', 'base_model.model.model.layers.18.input_layernorm.weight', 'base_model.model.model.layers.18.post_attention_layernorm.weight', 'base_model.model.model.layers.19.self_attn.q_proj.weight', 'base_model.model.model.layers.19.self_attn.k_proj.weight', 'base_model.model.model.layers.19.self_attn.v_proj.weight', 'base_model.model.model.layers.19.self_attn.o_proj.weight', 'base_model.model.model.layers.19.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.19.mlp.gate_proj.weight', 'base_model.model.model.layers.19.mlp.down_proj.weight', 'base_model.model.model.layers.19.mlp.up_proj.weight', 'base_model.model.model.layers.19.input_layernorm.weight', 'base_model.model.model.layers.19.post_attention_layernorm.weight', 'base_model.model.model.layers.20.self_attn.q_proj.weight', 'base_model.model.model.layers.20.self_attn.k_proj.weight', 'base_model.model.model.layers.20.self_attn.v_proj.weight', 'base_model.model.model.layers.20.self_attn.o_proj.weight', 'base_model.model.model.layers.20.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.20.mlp.gate_proj.weight', 'base_model.model.model.layers.20.mlp.down_proj.weight', 'base_model.model.model.layers.20.mlp.up_proj.weight', 'base_model.model.model.layers.20.input_layernorm.weight', 'base_model.model.model.layers.20.post_attention_layernorm.weight', 'base_model.model.model.layers.21.self_attn.q_proj.weight', 'base_model.model.model.layers.21.self_attn.k_proj.weight', 'base_model.model.model.layers.21.self_attn.v_proj.weight', 'base_model.model.model.layers.21.self_attn.o_proj.weight', 'base_model.model.model.layers.21.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.21.mlp.gate_proj.weight', 'base_model.model.model.layers.21.mlp.down_proj.weight', 'base_model.model.model.layers.21.mlp.up_proj.weight', 'base_model.model.model.layers.21.input_layernorm.weight', 'base_model.model.model.layers.21.post_attention_layernorm.weight', 'base_model.model.model.layers.22.self_attn.q_proj.weight', 'base_model.model.model.layers.22.self_attn.k_proj.weight', 'base_model.model.model.layers.22.self_attn.v_proj.weight', 'base_model.model.model.layers.22.self_attn.o_proj.weight', 'base_model.model.model.layers.22.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.22.mlp.gate_proj.weight', 'base_model.model.model.layers.22.mlp.down_proj.weight', 'base_model.model.model.layers.22.mlp.up_proj.weight', 'base_model.model.model.layers.22.input_layernorm.weight', 'base_model.model.model.layers.22.post_attention_layernorm.weight', 'base_model.model.model.layers.23.self_attn.q_proj.weight', 'base_model.model.model.layers.23.self_attn.k_proj.weight', 'base_model.model.model.layers.23.self_attn.v_proj.weight', 'base_model.model.model.layers.23.self_attn.o_proj.weight', 'base_model.model.model.layers.23.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.23.mlp.gate_proj.weight', 'base_model.model.model.layers.23.mlp.down_proj.weight', 'base_model.model.model.layers.23.mlp.up_proj.weight', 'base_model.model.model.layers.23.input_layernorm.weight', 'base_model.model.model.layers.23.post_attention_layernorm.weight', 'base_model.model.model.layers.24.self_attn.q_proj.weight', 'base_model.model.model.layers.24.self_attn.k_proj.weight', 'base_model.model.model.layers.24.self_attn.v_proj.weight', 'base_model.model.model.layers.24.self_attn.o_proj.weight', 'base_model.model.model.layers.24.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.24.mlp.gate_proj.weight', 'base_model.model.model.layers.24.mlp.down_proj.weight', 'base_model.model.model.layers.24.mlp.up_proj.weight', 'base_model.model.model.layers.24.input_layernorm.weight', 'base_model.model.model.layers.24.post_attention_layernorm.weight', 'base_model.model.model.layers.25.self_attn.q_proj.weight', 'base_model.model.model.layers.25.self_attn.k_proj.weight', 'base_model.model.model.layers.25.self_attn.v_proj.weight', 'base_model.model.model.layers.25.self_attn.o_proj.weight', 'base_model.model.model.layers.25.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.25.mlp.gate_proj.weight', 'base_model.model.model.layers.25.mlp.down_proj.weight', 'base_model.model.model.layers.25.mlp.up_proj.weight', 'base_model.model.model.layers.25.input_layernorm.weight', 'base_model.model.model.layers.25.post_attention_layernorm.weight', 'base_model.model.model.layers.26.self_attn.q_proj.weight', 'base_model.model.model.layers.26.self_attn.k_proj.weight', 'base_model.model.model.layers.26.self_attn.v_proj.weight', 'base_model.model.model.layers.26.self_attn.o_proj.weight', 'base_model.model.model.layers.26.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.26.mlp.gate_proj.weight', 'base_model.model.model.layers.26.mlp.down_proj.weight', 'base_model.model.model.layers.26.mlp.up_proj.weight', 'base_model.model.model.layers.26.input_layernorm.weight', 'base_model.model.model.layers.26.post_attention_layernorm.weight', 'base_model.model.model.layers.27.self_attn.q_proj.weight', 'base_model.model.model.layers.27.self_attn.k_proj.weight', 'base_model.model.model.layers.27.self_attn.v_proj.weight', 'base_model.model.model.layers.27.self_attn.o_proj.weight', 'base_model.model.model.layers.27.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.27.mlp.gate_proj.weight', 'base_model.model.model.layers.27.mlp.down_proj.weight', 'base_model.model.model.layers.27.mlp.up_proj.weight', 'base_model.model.model.layers.27.input_layernorm.weight', 'base_model.model.model.layers.27.post_attention_layernorm.weight', 'base_model.model.model.layers.28.self_attn.q_proj.weight', 'base_model.model.model.layers.28.self_attn.k_proj.weight', 'base_model.model.model.layers.28.self_attn.v_proj.weight', 'base_model.model.model.layers.28.self_attn.o_proj.weight', 'base_model.model.model.layers.28.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.28.mlp.gate_proj.weight', 'base_model.model.model.layers.28.mlp.down_proj.weight', 'base_model.model.model.layers.28.mlp.up_proj.weight', 'base_model.model.model.layers.28.input_layernorm.weight', 'base_model.model.model.layers.28.post_attention_layernorm.weight', 'base_model.model.model.layers.29.self_attn.q_proj.weight', 'base_model.model.model.layers.29.self_attn.k_proj.weight', 'base_model.model.model.layers.29.self_attn.v_proj.weight', 'base_model.model.model.layers.29.self_attn.o_proj.weight', 'base_model.model.model.layers.29.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.29.mlp.gate_proj.weight', 'base_model.model.model.layers.29.mlp.down_proj.weight', 'base_model.model.model.layers.29.mlp.up_proj.weight', 'base_model.model.model.layers.29.input_layernorm.weight', 'base_model.model.model.layers.29.post_attention_layernorm.weight', 'base_model.model.model.layers.30.self_attn.q_proj.weight', 'base_model.model.model.layers.30.self_attn.k_proj.weight', 'base_model.model.model.layers.30.self_attn.v_proj.weight', 'base_model.model.model.layers.30.self_attn.o_proj.weight', 'base_model.model.model.layers.30.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.30.mlp.gate_proj.weight', 'base_model.model.model.layers.30.mlp.down_proj.weight', 'base_model.model.model.layers.30.mlp.up_proj.weight', 'base_model.model.model.layers.30.input_layernorm.weight', 'base_model.model.model.layers.30.post_attention_layernorm.weight', 'base_model.model.model.layers.31.self_attn.q_proj.weight', 'base_model.model.model.layers.31.self_attn.k_proj.weight', 'base_model.model.model.layers.31.self_attn.v_proj.weight', 'base_model.model.model.layers.31.self_attn.o_proj.weight', 'base_model.model.model.layers.31.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.31.mlp.gate_proj.weight', 'base_model.model.model.layers.31.mlp.down_proj.weight', 'base_model.model.model.layers.31.mlp.up_proj.weight', 'base_model.model.model.layers.31.input_layernorm.weight', 'base_model.model.model.layers.31.post_attention_layernorm.weight', 'base_model.model.model.norm.weight', 'base_model.model.lm_head.0.weight'].
/usr/local/lib/python3.10/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning
warnings.warn(
wandb: Currently logged in as: utensil. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.14.2
wandb: Run data is saved locally in /workspace/code/wandb/run-20230419_054022-qpfkykck
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run rare-deluge-5
wandb: ⭐️ View project at https://wandb.ai/utensil/Alpaca-CoT
wandb: 🚀 View run at https://wandb.ai/utensil/Alpaca-CoT/runs/qpfkykck
0%| | 0/390 [00:00<?, ?it/s]Traceback (most recent call last):
File "/workspace/code/uniform_finetune.py", line 348, in <module>
train(args)
File "/workspace/code/uniform_finetune.py", line 316, in train
trainer.train(resume_from_checkpoint=True if args.resume_from_checkpoint == 'latest' else args.resume_from_checkpoint)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1662, in train
return inner_training_loop(
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1929, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2699, in training_step
loss = self.compute_loss(model, inputs)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2731, in compute_loss
outputs = model(**inputs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/peft/peft_model.py", line 530, in forward
return self.base_model(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 687, in forward
outputs = self.model(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 569, in forward
layer_outputs = torch.utils.checkpoint.checkpoint(
File "/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py", line 249, in checkpoint
return CheckpointFunction.apply(function, preserve, *args)
File "/usr/local/lib/python3.10/dist-packages/torch/autograd/function.py", line 506, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py", line 107, in forward
outputs = run_function(*args)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 565, in custom_forward
return module(*inputs, output_attentions, None)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 292, in forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 231, in forward
attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to(query_states.dtype)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py", line 1845, in softmax
ret = input.softmax(dim, dtype=dtype)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 76.00 MiB (GPU 0; 47.54 GiB total capacity; 6.90 GiB already allocated; 47.12 MiB free; 7.32 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
wandb: Waiting for W&B process to finish... (failed 1). Press Control-C to abort syncing.
wandb: 🚀 View run rare-deluge-5 at: https://wandb.ai/utensil/Alpaca-CoT/runs/qpfkykck
wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20230419_054022-qpfkykck/logs
In [30]:
!nvidia-smi
Wed Apr 19 05:46:50 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.30.02 Driver Version: 530.30.02 CUDA Version: 12.1 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA RTX A6000 On | 00000000:00:06.0 Off | Off |
| 30% 36C P8 25W / 300W| 40099MiB / 49140MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
+---------------------------------------------------------------------------------------+
In [34]:
!ps aux|grep uniform_finetune|awk '{print $2}'|xargs kill -9
kill: (4784): No such process
In [35]:
!nvidia-smi
Wed Apr 19 05:48:21 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.30.02 Driver Version: 530.30.02 CUDA Version: 12.1 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA RTX A6000 On | 00000000:00:06.0 Off | Off |
| 30% 37C P8 25W / 300W| 1MiB / 49140MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
In [36]:
!cp -r /workspace/llm-playground/storage/saved_models /workspace/code/
In [37]:
!cd /workspace/code && python uniform_finetune.py --model_type llama --model_name_or_path decapoda-research/llama-7b-hf --data alpaca --lora_target_modules q_proj v_proj --per_gpu_train_batch_size 4 --learning_rate 3e-4 --epochs 2 --report_to wandb --resume_from_checkpoint latest
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib'), PosixPath('/usr/local/nvidia/lib64')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('//matplotlib_inline.backend_inline'), PosixPath('module')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('PygmalionAI/pygmalion-6b')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('noninteractiveSHELL=/bin/bash')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/workspace/Untitled.ipynb')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0'), PosixPath('/usr/local/cuda/lib64/libcudart.so')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so.11.0
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 116
CUDA SETUP: Loading binary /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda116.so...
Namespace(size=None, data='alpaca', local_rank=-1, model_type='llama', model_name_or_path='decapoda-research/llama-7b-hf', per_gpu_train_batch_size=4, gradient_accumulation_steps=32, epochs=2, learning_rate=0.0003, cutoff_len=512, lora_r=8, lora_alpha=16, lora_dropout=0.05, val_set_size=2000, lora_target_modules=['q_proj', 'v_proj'], resume_from_checkpoint='latest')
Found cached dataset json (/root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e)
100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 347.87it/s]
DatasetDict({
train: Dataset({
features: ['instruction', 'input', 'output'],
num_rows: 51942
})
})
Overriding torch_dtype=None with `torch_dtype=torch.float16` due to requirements of `bitsandbytes` to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning.
Loading checkpoint shards: 100%|████████████████| 33/33 [00:07<00:00, 4.19it/s]
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
trainable params: 4194304 || all params: 6742609920 || trainable%: 0.06220594176090199
Loading cached split indices for dataset at /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e/cache-d9cf6a7263a2a9b6.arrow and /root/.cache/huggingface/datasets/json/default-a98c27c05f911bdb/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e/cache-1833a2e8bfe038bc.arrow
***** Running training *****
Num Epochs = 2
Instantaneous batch size per GPU = 4
Gradient Accumulation steps = 32
Total train batch size (w. parallel, distributed & accumulation) = 128
Total optimization steps = 390
Saving steps = 39
There were missing keys in the checkpoint model loaded: ['base_model.model.model.embed_tokens.weight', 'base_model.model.model.layers.0.self_attn.q_proj.weight', 'base_model.model.model.layers.0.self_attn.k_proj.weight', 'base_model.model.model.layers.0.self_attn.v_proj.weight', 'base_model.model.model.layers.0.self_attn.o_proj.weight', 'base_model.model.model.layers.0.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.0.mlp.gate_proj.weight', 'base_model.model.model.layers.0.mlp.down_proj.weight', 'base_model.model.model.layers.0.mlp.up_proj.weight', 'base_model.model.model.layers.0.input_layernorm.weight', 'base_model.model.model.layers.0.post_attention_layernorm.weight', 'base_model.model.model.layers.1.self_attn.q_proj.weight', 'base_model.model.model.layers.1.self_attn.k_proj.weight', 'base_model.model.model.layers.1.self_attn.v_proj.weight', 'base_model.model.model.layers.1.self_attn.o_proj.weight', 'base_model.model.model.layers.1.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.1.mlp.gate_proj.weight', 'base_model.model.model.layers.1.mlp.down_proj.weight', 'base_model.model.model.layers.1.mlp.up_proj.weight', 'base_model.model.model.layers.1.input_layernorm.weight', 'base_model.model.model.layers.1.post_attention_layernorm.weight', 'base_model.model.model.layers.2.self_attn.q_proj.weight', 'base_model.model.model.layers.2.self_attn.k_proj.weight', 'base_model.model.model.layers.2.self_attn.v_proj.weight', 'base_model.model.model.layers.2.self_attn.o_proj.weight', 'base_model.model.model.layers.2.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.2.mlp.gate_proj.weight', 'base_model.model.model.layers.2.mlp.down_proj.weight', 'base_model.model.model.layers.2.mlp.up_proj.weight', 'base_model.model.model.layers.2.input_layernorm.weight', 'base_model.model.model.layers.2.post_attention_layernorm.weight', 'base_model.model.model.layers.3.self_attn.q_proj.weight', 'base_model.model.model.layers.3.self_attn.k_proj.weight', 'base_model.model.model.layers.3.self_attn.v_proj.weight', 'base_model.model.model.layers.3.self_attn.o_proj.weight', 'base_model.model.model.layers.3.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.3.mlp.gate_proj.weight', 'base_model.model.model.layers.3.mlp.down_proj.weight', 'base_model.model.model.layers.3.mlp.up_proj.weight', 'base_model.model.model.layers.3.input_layernorm.weight', 'base_model.model.model.layers.3.post_attention_layernorm.weight', 'base_model.model.model.layers.4.self_attn.q_proj.weight', 'base_model.model.model.layers.4.self_attn.k_proj.weight', 'base_model.model.model.layers.4.self_attn.v_proj.weight', 'base_model.model.model.layers.4.self_attn.o_proj.weight', 'base_model.model.model.layers.4.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.4.mlp.gate_proj.weight', 'base_model.model.model.layers.4.mlp.down_proj.weight', 'base_model.model.model.layers.4.mlp.up_proj.weight', 'base_model.model.model.layers.4.input_layernorm.weight', 'base_model.model.model.layers.4.post_attention_layernorm.weight', 'base_model.model.model.layers.5.self_attn.q_proj.weight', 'base_model.model.model.layers.5.self_attn.k_proj.weight', 'base_model.model.model.layers.5.self_attn.v_proj.weight', 'base_model.model.model.layers.5.self_attn.o_proj.weight', 'base_model.model.model.layers.5.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.5.mlp.gate_proj.weight', 'base_model.model.model.layers.5.mlp.down_proj.weight', 'base_model.model.model.layers.5.mlp.up_proj.weight', 'base_model.model.model.layers.5.input_layernorm.weight', 'base_model.model.model.layers.5.post_attention_layernorm.weight', 'base_model.model.model.layers.6.self_attn.q_proj.weight', 'base_model.model.model.layers.6.self_attn.k_proj.weight', 'base_model.model.model.layers.6.self_attn.v_proj.weight', 'base_model.model.model.layers.6.self_attn.o_proj.weight', 'base_model.model.model.layers.6.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.6.mlp.gate_proj.weight', 'base_model.model.model.layers.6.mlp.down_proj.weight', 'base_model.model.model.layers.6.mlp.up_proj.weight', 'base_model.model.model.layers.6.input_layernorm.weight', 'base_model.model.model.layers.6.post_attention_layernorm.weight', 'base_model.model.model.layers.7.self_attn.q_proj.weight', 'base_model.model.model.layers.7.self_attn.k_proj.weight', 'base_model.model.model.layers.7.self_attn.v_proj.weight', 'base_model.model.model.layers.7.self_attn.o_proj.weight', 'base_model.model.model.layers.7.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.7.mlp.gate_proj.weight', 'base_model.model.model.layers.7.mlp.down_proj.weight', 'base_model.model.model.layers.7.mlp.up_proj.weight', 'base_model.model.model.layers.7.input_layernorm.weight', 'base_model.model.model.layers.7.post_attention_layernorm.weight', 'base_model.model.model.layers.8.self_attn.q_proj.weight', 'base_model.model.model.layers.8.self_attn.k_proj.weight', 'base_model.model.model.layers.8.self_attn.v_proj.weight', 'base_model.model.model.layers.8.self_attn.o_proj.weight', 'base_model.model.model.layers.8.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.8.mlp.gate_proj.weight', 'base_model.model.model.layers.8.mlp.down_proj.weight', 'base_model.model.model.layers.8.mlp.up_proj.weight', 'base_model.model.model.layers.8.input_layernorm.weight', 'base_model.model.model.layers.8.post_attention_layernorm.weight', 'base_model.model.model.layers.9.self_attn.q_proj.weight', 'base_model.model.model.layers.9.self_attn.k_proj.weight', 'base_model.model.model.layers.9.self_attn.v_proj.weight', 'base_model.model.model.layers.9.self_attn.o_proj.weight', 'base_model.model.model.layers.9.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.9.mlp.gate_proj.weight', 'base_model.model.model.layers.9.mlp.down_proj.weight', 'base_model.model.model.layers.9.mlp.up_proj.weight', 'base_model.model.model.layers.9.input_layernorm.weight', 'base_model.model.model.layers.9.post_attention_layernorm.weight', 'base_model.model.model.layers.10.self_attn.q_proj.weight', 'base_model.model.model.layers.10.self_attn.k_proj.weight', 'base_model.model.model.layers.10.self_attn.v_proj.weight', 'base_model.model.model.layers.10.self_attn.o_proj.weight', 'base_model.model.model.layers.10.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.10.mlp.gate_proj.weight', 'base_model.model.model.layers.10.mlp.down_proj.weight', 'base_model.model.model.layers.10.mlp.up_proj.weight', 'base_model.model.model.layers.10.input_layernorm.weight', 'base_model.model.model.layers.10.post_attention_layernorm.weight', 'base_model.model.model.layers.11.self_attn.q_proj.weight', 'base_model.model.model.layers.11.self_attn.k_proj.weight', 'base_model.model.model.layers.11.self_attn.v_proj.weight', 'base_model.model.model.layers.11.self_attn.o_proj.weight', 'base_model.model.model.layers.11.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.11.mlp.gate_proj.weight', 'base_model.model.model.layers.11.mlp.down_proj.weight', 'base_model.model.model.layers.11.mlp.up_proj.weight', 'base_model.model.model.layers.11.input_layernorm.weight', 'base_model.model.model.layers.11.post_attention_layernorm.weight', 'base_model.model.model.layers.12.self_attn.q_proj.weight', 'base_model.model.model.layers.12.self_attn.k_proj.weight', 'base_model.model.model.layers.12.self_attn.v_proj.weight', 'base_model.model.model.layers.12.self_attn.o_proj.weight', 'base_model.model.model.layers.12.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.12.mlp.gate_proj.weight', 'base_model.model.model.layers.12.mlp.down_proj.weight', 'base_model.model.model.layers.12.mlp.up_proj.weight', 'base_model.model.model.layers.12.input_layernorm.weight', 'base_model.model.model.layers.12.post_attention_layernorm.weight', 'base_model.model.model.layers.13.self_attn.q_proj.weight', 'base_model.model.model.layers.13.self_attn.k_proj.weight', 'base_model.model.model.layers.13.self_attn.v_proj.weight', 'base_model.model.model.layers.13.self_attn.o_proj.weight', 'base_model.model.model.layers.13.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.13.mlp.gate_proj.weight', 'base_model.model.model.layers.13.mlp.down_proj.weight', 'base_model.model.model.layers.13.mlp.up_proj.weight', 'base_model.model.model.layers.13.input_layernorm.weight', 'base_model.model.model.layers.13.post_attention_layernorm.weight', 'base_model.model.model.layers.14.self_attn.q_proj.weight', 'base_model.model.model.layers.14.self_attn.k_proj.weight', 'base_model.model.model.layers.14.self_attn.v_proj.weight', 'base_model.model.model.layers.14.self_attn.o_proj.weight', 'base_model.model.model.layers.14.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.14.mlp.gate_proj.weight', 'base_model.model.model.layers.14.mlp.down_proj.weight', 'base_model.model.model.layers.14.mlp.up_proj.weight', 'base_model.model.model.layers.14.input_layernorm.weight', 'base_model.model.model.layers.14.post_attention_layernorm.weight', 'base_model.model.model.layers.15.self_attn.q_proj.weight', 'base_model.model.model.layers.15.self_attn.k_proj.weight', 'base_model.model.model.layers.15.self_attn.v_proj.weight', 'base_model.model.model.layers.15.self_attn.o_proj.weight', 'base_model.model.model.layers.15.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.15.mlp.gate_proj.weight', 'base_model.model.model.layers.15.mlp.down_proj.weight', 'base_model.model.model.layers.15.mlp.up_proj.weight', 'base_model.model.model.layers.15.input_layernorm.weight', 'base_model.model.model.layers.15.post_attention_layernorm.weight', 'base_model.model.model.layers.16.self_attn.q_proj.weight', 'base_model.model.model.layers.16.self_attn.k_proj.weight', 'base_model.model.model.layers.16.self_attn.v_proj.weight', 'base_model.model.model.layers.16.self_attn.o_proj.weight', 'base_model.model.model.layers.16.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.16.mlp.gate_proj.weight', 'base_model.model.model.layers.16.mlp.down_proj.weight', 'base_model.model.model.layers.16.mlp.up_proj.weight', 'base_model.model.model.layers.16.input_layernorm.weight', 'base_model.model.model.layers.16.post_attention_layernorm.weight', 'base_model.model.model.layers.17.self_attn.q_proj.weight', 'base_model.model.model.layers.17.self_attn.k_proj.weight', 'base_model.model.model.layers.17.self_attn.v_proj.weight', 'base_model.model.model.layers.17.self_attn.o_proj.weight', 'base_model.model.model.layers.17.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.17.mlp.gate_proj.weight', 'base_model.model.model.layers.17.mlp.down_proj.weight', 'base_model.model.model.layers.17.mlp.up_proj.weight', 'base_model.model.model.layers.17.input_layernorm.weight', 'base_model.model.model.layers.17.post_attention_layernorm.weight', 'base_model.model.model.layers.18.self_attn.q_proj.weight', 'base_model.model.model.layers.18.self_attn.k_proj.weight', 'base_model.model.model.layers.18.self_attn.v_proj.weight', 'base_model.model.model.layers.18.self_attn.o_proj.weight', 'base_model.model.model.layers.18.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.18.mlp.gate_proj.weight', 'base_model.model.model.layers.18.mlp.down_proj.weight', 'base_model.model.model.layers.18.mlp.up_proj.weight', 'base_model.model.model.layers.18.input_layernorm.weight', 'base_model.model.model.layers.18.post_attention_layernorm.weight', 'base_model.model.model.layers.19.self_attn.q_proj.weight', 'base_model.model.model.layers.19.self_attn.k_proj.weight', 'base_model.model.model.layers.19.self_attn.v_proj.weight', 'base_model.model.model.layers.19.self_attn.o_proj.weight', 'base_model.model.model.layers.19.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.19.mlp.gate_proj.weight', 'base_model.model.model.layers.19.mlp.down_proj.weight', 'base_model.model.model.layers.19.mlp.up_proj.weight', 'base_model.model.model.layers.19.input_layernorm.weight', 'base_model.model.model.layers.19.post_attention_layernorm.weight', 'base_model.model.model.layers.20.self_attn.q_proj.weight', 'base_model.model.model.layers.20.self_attn.k_proj.weight', 'base_model.model.model.layers.20.self_attn.v_proj.weight', 'base_model.model.model.layers.20.self_attn.o_proj.weight', 'base_model.model.model.layers.20.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.20.mlp.gate_proj.weight', 'base_model.model.model.layers.20.mlp.down_proj.weight', 'base_model.model.model.layers.20.mlp.up_proj.weight', 'base_model.model.model.layers.20.input_layernorm.weight', 'base_model.model.model.layers.20.post_attention_layernorm.weight', 'base_model.model.model.layers.21.self_attn.q_proj.weight', 'base_model.model.model.layers.21.self_attn.k_proj.weight', 'base_model.model.model.layers.21.self_attn.v_proj.weight', 'base_model.model.model.layers.21.self_attn.o_proj.weight', 'base_model.model.model.layers.21.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.21.mlp.gate_proj.weight', 'base_model.model.model.layers.21.mlp.down_proj.weight', 'base_model.model.model.layers.21.mlp.up_proj.weight', 'base_model.model.model.layers.21.input_layernorm.weight', 'base_model.model.model.layers.21.post_attention_layernorm.weight', 'base_model.model.model.layers.22.self_attn.q_proj.weight', 'base_model.model.model.layers.22.self_attn.k_proj.weight', 'base_model.model.model.layers.22.self_attn.v_proj.weight', 'base_model.model.model.layers.22.self_attn.o_proj.weight', 'base_model.model.model.layers.22.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.22.mlp.gate_proj.weight', 'base_model.model.model.layers.22.mlp.down_proj.weight', 'base_model.model.model.layers.22.mlp.up_proj.weight', 'base_model.model.model.layers.22.input_layernorm.weight', 'base_model.model.model.layers.22.post_attention_layernorm.weight', 'base_model.model.model.layers.23.self_attn.q_proj.weight', 'base_model.model.model.layers.23.self_attn.k_proj.weight', 'base_model.model.model.layers.23.self_attn.v_proj.weight', 'base_model.model.model.layers.23.self_attn.o_proj.weight', 'base_model.model.model.layers.23.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.23.mlp.gate_proj.weight', 'base_model.model.model.layers.23.mlp.down_proj.weight', 'base_model.model.model.layers.23.mlp.up_proj.weight', 'base_model.model.model.layers.23.input_layernorm.weight', 'base_model.model.model.layers.23.post_attention_layernorm.weight', 'base_model.model.model.layers.24.self_attn.q_proj.weight', 'base_model.model.model.layers.24.self_attn.k_proj.weight', 'base_model.model.model.layers.24.self_attn.v_proj.weight', 'base_model.model.model.layers.24.self_attn.o_proj.weight', 'base_model.model.model.layers.24.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.24.mlp.gate_proj.weight', 'base_model.model.model.layers.24.mlp.down_proj.weight', 'base_model.model.model.layers.24.mlp.up_proj.weight', 'base_model.model.model.layers.24.input_layernorm.weight', 'base_model.model.model.layers.24.post_attention_layernorm.weight', 'base_model.model.model.layers.25.self_attn.q_proj.weight', 'base_model.model.model.layers.25.self_attn.k_proj.weight', 'base_model.model.model.layers.25.self_attn.v_proj.weight', 'base_model.model.model.layers.25.self_attn.o_proj.weight', 'base_model.model.model.layers.25.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.25.mlp.gate_proj.weight', 'base_model.model.model.layers.25.mlp.down_proj.weight', 'base_model.model.model.layers.25.mlp.up_proj.weight', 'base_model.model.model.layers.25.input_layernorm.weight', 'base_model.model.model.layers.25.post_attention_layernorm.weight', 'base_model.model.model.layers.26.self_attn.q_proj.weight', 'base_model.model.model.layers.26.self_attn.k_proj.weight', 'base_model.model.model.layers.26.self_attn.v_proj.weight', 'base_model.model.model.layers.26.self_attn.o_proj.weight', 'base_model.model.model.layers.26.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.26.mlp.gate_proj.weight', 'base_model.model.model.layers.26.mlp.down_proj.weight', 'base_model.model.model.layers.26.mlp.up_proj.weight', 'base_model.model.model.layers.26.input_layernorm.weight', 'base_model.model.model.layers.26.post_attention_layernorm.weight', 'base_model.model.model.layers.27.self_attn.q_proj.weight', 'base_model.model.model.layers.27.self_attn.k_proj.weight', 'base_model.model.model.layers.27.self_attn.v_proj.weight', 'base_model.model.model.layers.27.self_attn.o_proj.weight', 'base_model.model.model.layers.27.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.27.mlp.gate_proj.weight', 'base_model.model.model.layers.27.mlp.down_proj.weight', 'base_model.model.model.layers.27.mlp.up_proj.weight', 'base_model.model.model.layers.27.input_layernorm.weight', 'base_model.model.model.layers.27.post_attention_layernorm.weight', 'base_model.model.model.layers.28.self_attn.q_proj.weight', 'base_model.model.model.layers.28.self_attn.k_proj.weight', 'base_model.model.model.layers.28.self_attn.v_proj.weight', 'base_model.model.model.layers.28.self_attn.o_proj.weight', 'base_model.model.model.layers.28.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.28.mlp.gate_proj.weight', 'base_model.model.model.layers.28.mlp.down_proj.weight', 'base_model.model.model.layers.28.mlp.up_proj.weight', 'base_model.model.model.layers.28.input_layernorm.weight', 'base_model.model.model.layers.28.post_attention_layernorm.weight', 'base_model.model.model.layers.29.self_attn.q_proj.weight', 'base_model.model.model.layers.29.self_attn.k_proj.weight', 'base_model.model.model.layers.29.self_attn.v_proj.weight', 'base_model.model.model.layers.29.self_attn.o_proj.weight', 'base_model.model.model.layers.29.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.29.mlp.gate_proj.weight', 'base_model.model.model.layers.29.mlp.down_proj.weight', 'base_model.model.model.layers.29.mlp.up_proj.weight', 'base_model.model.model.layers.29.input_layernorm.weight', 'base_model.model.model.layers.29.post_attention_layernorm.weight', 'base_model.model.model.layers.30.self_attn.q_proj.weight', 'base_model.model.model.layers.30.self_attn.k_proj.weight', 'base_model.model.model.layers.30.self_attn.v_proj.weight', 'base_model.model.model.layers.30.self_attn.o_proj.weight', 'base_model.model.model.layers.30.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.30.mlp.gate_proj.weight', 'base_model.model.model.layers.30.mlp.down_proj.weight', 'base_model.model.model.layers.30.mlp.up_proj.weight', 'base_model.model.model.layers.30.input_layernorm.weight', 'base_model.model.model.layers.30.post_attention_layernorm.weight', 'base_model.model.model.layers.31.self_attn.q_proj.weight', 'base_model.model.model.layers.31.self_attn.k_proj.weight', 'base_model.model.model.layers.31.self_attn.v_proj.weight', 'base_model.model.model.layers.31.self_attn.o_proj.weight', 'base_model.model.model.layers.31.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.31.mlp.gate_proj.weight', 'base_model.model.model.layers.31.mlp.down_proj.weight', 'base_model.model.model.layers.31.mlp.up_proj.weight', 'base_model.model.model.layers.31.input_layernorm.weight', 'base_model.model.model.layers.31.post_attention_layernorm.weight', 'base_model.model.model.norm.weight', 'base_model.model.lm_head.0.weight'].
/usr/local/lib/python3.10/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning
warnings.warn(
wandb: Currently logged in as: utensil. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.14.2
wandb: Run data is saved locally in /workspace/code/wandb/run-20230419_055034-j0gxpjnk
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run comfy-lake-6
wandb: ⭐️ View project at https://wandb.ai/utensil/Alpaca-CoT
wandb: 🚀 View run at https://wandb.ai/utensil/Alpaca-CoT/runs/j0gxpjnk
{'loss': 1.0514, 'learning_rate': 0.00015384615384615382, 'epoch': 1.03}
52%|████████████████████▎ | 407/780 [07:37<1:57:26, 18.89s/it]wandb: 500 encountered ({"error":"driver: bad connection"}), retrying request
wandb: Network error resolved after 0:00:12.257044, resuming normal operation.
52%|████████████████████▍ | 408/780 [08:03<2:08:08, 20.67s/it]wandb: 500 encountered ({"error":"An internal error occurred. Please contact support."}), retrying request
{'loss': 1.0603, 'learning_rate': 0.000145748987854251, 'epoch': 1.08}
55%|█████████████████████▍ | 429/780 [17:28<2:37:17, 26.89s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:45, 5.44it/s]
1%|▌ | 3/250 [00:00<01:20, 3.09it/s]
2%|▋ | 4/250 [00:01<01:31, 2.69it/s]
2%|▊ | 5/250 [00:01<01:32, 2.64it/s]
2%|█ | 6/250 [00:02<01:53, 2.14it/s]
3%|█▏ | 7/250 [00:02<01:44, 2.32it/s]
3%|█▍ | 8/250 [00:03<01:38, 2.45it/s]
4%|█▌ | 9/250 [00:03<01:33, 2.59it/s]
4%|█▋ | 10/250 [00:03<01:27, 2.74it/s]
4%|█▊ | 11/250 [00:04<01:26, 2.75it/s]
5%|██ | 12/250 [00:04<01:32, 2.57it/s]
5%|██▏ | 13/250 [00:04<01:29, 2.64it/s]
6%|██▎ | 14/250 [00:05<01:30, 2.61it/s]
6%|██▌ | 15/250 [00:05<01:44, 2.25it/s]
6%|██▋ | 16/250 [00:06<01:42, 2.28it/s]
7%|██▊ | 17/250 [00:06<01:43, 2.25it/s]
7%|███ | 18/250 [00:07<01:37, 2.37it/s]
8%|███▏ | 19/250 [00:07<01:42, 2.25it/s]
8%|███▎ | 20/250 [00:08<01:39, 2.32it/s]
8%|███▌ | 21/250 [00:08<02:00, 1.90it/s]
9%|███▋ | 22/250 [00:09<01:47, 2.12it/s]
9%|███▊ | 23/250 [00:09<01:52, 2.02it/s]
10%|████ | 24/250 [00:10<01:40, 2.25it/s]
10%|████▏ | 25/250 [00:10<01:37, 2.31it/s]
10%|████▎ | 26/250 [00:10<01:43, 2.17it/s]
11%|████▌ | 27/250 [00:11<01:35, 2.35it/s]
11%|████▋ | 28/250 [00:11<01:31, 2.43it/s]
12%|████▊ | 29/250 [00:11<01:20, 2.73it/s]
12%|█████ | 30/250 [00:12<01:53, 1.94it/s]
12%|█████▏ | 31/250 [00:13<01:41, 2.17it/s]
13%|█████▍ | 32/250 [00:13<01:48, 2.01it/s]
13%|█████▌ | 33/250 [00:14<01:41, 2.13it/s]
14%|█████▋ | 34/250 [00:14<01:39, 2.16it/s]
14%|█████▉ | 35/250 [00:14<01:35, 2.25it/s]
14%|██████ | 36/250 [00:15<01:27, 2.45it/s]
15%|██████▏ | 37/250 [00:15<01:34, 2.25it/s]
15%|██████▍ | 38/250 [00:16<01:25, 2.49it/s]
16%|██████▌ | 39/250 [00:16<01:17, 2.71it/s]
16%|██████▋ | 40/250 [00:16<01:12, 2.90it/s]
16%|██████▉ | 41/250 [00:17<01:10, 2.99it/s]
17%|███████ | 42/250 [00:17<01:12, 2.87it/s]
17%|███████▏ | 43/250 [00:17<01:10, 2.95it/s]
18%|███████▍ | 44/250 [00:18<01:10, 2.91it/s]
18%|███████▌ | 45/250 [00:18<01:22, 2.47it/s]
18%|███████▋ | 46/250 [00:19<01:25, 2.39it/s]
19%|███████▉ | 47/250 [00:19<01:17, 2.63it/s]
19%|████████ | 48/250 [00:19<01:19, 2.53it/s]
20%|████████▏ | 49/250 [00:20<01:12, 2.78it/s]
20%|████████▍ | 50/250 [00:20<01:11, 2.81it/s]
20%|████████▌ | 51/250 [00:20<01:10, 2.84it/s]
21%|████████▋ | 52/250 [00:21<01:08, 2.88it/s]
21%|████████▉ | 53/250 [00:21<01:05, 3.02it/s]
22%|█████████ | 54/250 [00:21<01:10, 2.79it/s]
22%|█████████▏ | 55/250 [00:22<01:09, 2.80it/s]
22%|█████████▍ | 56/250 [00:22<01:12, 2.69it/s]
23%|█████████▌ | 57/250 [00:22<01:09, 2.80it/s]
23%|█████████▋ | 58/250 [00:23<01:22, 2.33it/s]
24%|█████████▉ | 59/250 [00:23<01:23, 2.28it/s]
24%|██████████ | 60/250 [00:24<01:20, 2.35it/s]
24%|██████████▏ | 61/250 [00:25<01:45, 1.79it/s]
25%|██████████▍ | 62/250 [00:25<01:34, 1.99it/s]
25%|██████████▌ | 63/250 [00:26<01:31, 2.05it/s]
26%|██████████▊ | 64/250 [00:26<01:24, 2.20it/s]
26%|██████████▉ | 65/250 [00:26<01:17, 2.38it/s]
26%|███████████ | 66/250 [00:27<01:13, 2.50it/s]
27%|███████████▎ | 67/250 [00:27<01:14, 2.46it/s]
27%|███████████▍ | 68/250 [00:27<01:10, 2.57it/s]
28%|███████████▌ | 69/250 [00:28<01:08, 2.64it/s]
28%|███████████▊ | 70/250 [00:28<01:04, 2.78it/s]
28%|███████████▉ | 71/250 [00:28<01:05, 2.72it/s]
29%|████████████ | 72/250 [00:29<01:04, 2.78it/s]
29%|████████████▎ | 73/250 [00:29<01:06, 2.68it/s]
30%|████████████▍ | 74/250 [00:29<01:01, 2.86it/s]
30%|████████████▌ | 75/250 [00:30<00:59, 2.93it/s]
30%|████████████▊ | 76/250 [00:30<00:58, 2.97it/s]
31%|████████████▉ | 77/250 [00:30<00:59, 2.92it/s]
31%|█████████████ | 78/250 [00:31<00:55, 3.08it/s]
32%|█████████████▎ | 79/250 [00:31<00:59, 2.87it/s]
32%|█████████████▍ | 80/250 [00:32<01:00, 2.80it/s]
32%|█████████████▌ | 81/250 [00:32<00:57, 2.94it/s]
33%|█████████████▊ | 82/250 [00:32<00:51, 3.23it/s]
33%|█████████████▉ | 83/250 [00:32<00:55, 3.00it/s]
34%|██████████████ | 84/250 [00:33<01:04, 2.56it/s]
34%|██████████████▎ | 85/250 [00:33<01:04, 2.57it/s]
34%|██████████████▍ | 86/250 [00:34<01:04, 2.55it/s]
35%|██████████████▌ | 87/250 [00:34<01:13, 2.21it/s]
35%|██████████████▊ | 88/250 [00:35<01:05, 2.49it/s]
36%|██████████████▉ | 89/250 [00:35<01:02, 2.59it/s]
36%|███████████████ | 90/250 [00:35<01:02, 2.57it/s]
36%|███████████████▎ | 91/250 [00:36<00:58, 2.73it/s]
37%|███████████████▍ | 92/250 [00:36<00:56, 2.79it/s]
37%|███████████████▌ | 93/250 [00:37<01:14, 2.11it/s]
38%|███████████████▊ | 94/250 [00:37<01:07, 2.30it/s]
38%|███████████████▉ | 95/250 [00:38<01:04, 2.39it/s]
38%|████████████████▏ | 96/250 [00:38<01:01, 2.52it/s]
39%|████████████████▎ | 97/250 [00:38<00:55, 2.78it/s]
39%|████████████████▍ | 98/250 [00:39<01:01, 2.48it/s]
40%|████████████████▋ | 99/250 [00:39<00:58, 2.58it/s]
40%|████████████████▍ | 100/250 [00:39<00:54, 2.77it/s]
40%|████████████████▌ | 101/250 [00:40<00:52, 2.85it/s]
41%|████████████████▋ | 102/250 [00:40<00:49, 3.02it/s]
41%|████████████████▉ | 103/250 [00:41<01:12, 2.03it/s]
42%|█████████████████ | 104/250 [00:41<01:10, 2.06it/s]
42%|█████████████████▏ | 105/250 [00:42<01:04, 2.25it/s]
42%|█████████████████▍ | 106/250 [00:42<00:59, 2.40it/s]
43%|█████████████████▌ | 107/250 [00:42<00:58, 2.44it/s]
43%|█████████████████▋ | 108/250 [00:43<01:00, 2.33it/s]
44%|█████████████████▉ | 109/250 [00:43<00:54, 2.61it/s]
44%|██████████████████ | 110/250 [00:43<00:50, 2.78it/s]
44%|██████████████████▏ | 111/250 [00:44<00:52, 2.64it/s]
45%|██████████████████▎ | 112/250 [00:44<00:53, 2.59it/s]
45%|██████████████████▌ | 113/250 [00:44<00:47, 2.91it/s]
46%|██████████████████▋ | 114/250 [00:45<00:45, 2.96it/s]
46%|██████████████████▊ | 115/250 [00:45<00:54, 2.49it/s]
46%|███████████████████ | 116/250 [00:46<00:50, 2.68it/s]
47%|███████████████████▏ | 117/250 [00:46<00:54, 2.44it/s]
47%|███████████████████▎ | 118/250 [00:47<01:12, 1.82it/s]
48%|███████████████████▌ | 119/250 [00:47<01:06, 1.97it/s]
48%|███████████████████▋ | 120/250 [00:48<00:57, 2.26it/s]
48%|███████████████████▊ | 121/250 [00:48<00:51, 2.48it/s]
49%|████████████████████ | 122/250 [00:48<00:47, 2.67it/s]
49%|████████████████████▏ | 123/250 [00:49<00:45, 2.81it/s]
50%|████████████████████▎ | 124/250 [00:49<00:43, 2.93it/s]
50%|████████████████████▌ | 125/250 [00:49<00:42, 2.94it/s]
50%|████████████████████▋ | 126/250 [00:50<00:46, 2.69it/s]
51%|████████████████████▊ | 127/250 [00:50<00:43, 2.81it/s]
51%|████████████████████▉ | 128/250 [00:50<00:46, 2.64it/s]
52%|█████████████████████▏ | 129/250 [00:51<00:50, 2.37it/s]
52%|█████████████████████▎ | 130/250 [00:51<00:48, 2.50it/s]
52%|█████████████████████▍ | 131/250 [00:52<00:57, 2.06it/s]
53%|█████████████████████▋ | 132/250 [00:52<00:53, 2.20it/s]
53%|█████████████████████▊ | 133/250 [00:53<00:55, 2.11it/s]
54%|█████████████████████▉ | 134/250 [00:53<00:49, 2.37it/s]
54%|██████████████████████▏ | 135/250 [00:54<00:46, 2.45it/s]
54%|██████████████████████▎ | 136/250 [00:54<00:44, 2.57it/s]
55%|██████████████████████▍ | 137/250 [00:54<00:44, 2.56it/s]
55%|██████████████████████▋ | 138/250 [00:55<00:43, 2.56it/s]
56%|██████████████████████▊ | 139/250 [00:55<00:41, 2.68it/s]
56%|██████████████████████▉ | 140/250 [00:55<00:40, 2.70it/s]
56%|███████████████████████ | 141/250 [00:56<00:41, 2.62it/s]
57%|███████████████████████▎ | 142/250 [00:56<00:43, 2.48it/s]
57%|███████████████████████▍ | 143/250 [00:57<00:39, 2.70it/s]
58%|███████████████████████▌ | 144/250 [00:57<00:47, 2.22it/s]
58%|███████████████████████▊ | 145/250 [00:58<00:44, 2.36it/s]
58%|███████████████████████▉ | 146/250 [00:58<00:42, 2.42it/s]
59%|████████████████████████ | 147/250 [00:58<00:40, 2.52it/s]
59%|████████████████████████▎ | 148/250 [00:59<00:42, 2.42it/s]
60%|████████████████████████▍ | 149/250 [00:59<00:46, 2.16it/s]
60%|████████████████████████▌ | 150/250 [01:00<00:41, 2.39it/s]
60%|████████████████████████▊ | 151/250 [01:00<00:39, 2.52it/s]
61%|████████████████████████▉ | 152/250 [01:00<00:38, 2.55it/s]
61%|█████████████████████████ | 153/250 [01:01<00:38, 2.53it/s]
62%|█████████████████████████▎ | 154/250 [01:01<00:42, 2.27it/s]
62%|█████████████████████████▍ | 155/250 [01:02<00:39, 2.41it/s]
62%|█████████████████████████▌ | 156/250 [01:02<00:37, 2.48it/s]
63%|█████████████████████████▋ | 157/250 [01:02<00:36, 2.51it/s]
63%|█████████████████████████▉ | 158/250 [01:03<00:33, 2.76it/s]
64%|██████████████████████████ | 159/250 [01:03<00:32, 2.81it/s]
64%|██████████████████████████▏ | 160/250 [01:03<00:31, 2.87it/s]
64%|██████████████████████████▍ | 161/250 [01:04<00:30, 2.88it/s]
65%|██████████████████████████▌ | 162/250 [01:04<00:35, 2.46it/s]
65%|██████████████████████████▋ | 163/250 [01:05<00:39, 2.22it/s]
66%|██████████████████████████▉ | 164/250 [01:05<00:41, 2.08it/s]
66%|███████████████████████████ | 165/250 [01:06<00:47, 1.79it/s]
66%|███████████████████████████▏ | 166/250 [01:07<00:41, 2.03it/s]
67%|███████████████████████████▍ | 167/250 [01:07<00:39, 2.08it/s]
67%|███████████████████████████▌ | 168/250 [01:07<00:35, 2.31it/s]
68%|███████████████████████████▋ | 169/250 [01:08<00:33, 2.41it/s]
68%|███████████████████████████▉ | 170/250 [01:08<00:32, 2.43it/s]
68%|████████████████████████████ | 171/250 [01:09<00:36, 2.15it/s]
69%|████████████████████████████▏ | 172/250 [01:09<00:34, 2.27it/s]
69%|████████████████████████████▎ | 173/250 [01:09<00:31, 2.46it/s]
70%|████████████████████████████▌ | 174/250 [01:10<00:35, 2.16it/s]
70%|████████████████████████████▋ | 175/250 [01:10<00:32, 2.34it/s]
70%|████████████████████████████▊ | 176/250 [01:11<00:29, 2.50it/s]
71%|█████████████████████████████ | 177/250 [01:11<00:29, 2.45it/s]
71%|█████████████████████████████▏ | 178/250 [01:11<00:27, 2.60it/s]
72%|█████████████████████████████▎ | 179/250 [01:12<00:29, 2.44it/s]
72%|█████████████████████████████▌ | 180/250 [01:12<00:25, 2.69it/s]
72%|█████████████████████████████▋ | 181/250 [01:13<00:27, 2.53it/s]
73%|█████████████████████████████▊ | 182/250 [01:13<00:24, 2.82it/s]
73%|██████████████████████████████ | 183/250 [01:13<00:27, 2.48it/s]
74%|██████████████████████████████▏ | 184/250 [01:14<00:26, 2.51it/s]
74%|██████████████████████████████▎ | 185/250 [01:14<00:30, 2.16it/s]
74%|██████████████████████████████▌ | 186/250 [01:15<00:27, 2.34it/s]
75%|██████████████████████████████▋ | 187/250 [01:15<00:28, 2.22it/s]
75%|██████████████████████████████▊ | 188/250 [01:16<00:26, 2.31it/s]
76%|██████████████████████████████▉ | 189/250 [01:16<00:27, 2.25it/s]
76%|███████████████████████████████▏ | 190/250 [01:16<00:25, 2.38it/s]
76%|███████████████████████████████▎ | 191/250 [01:17<00:23, 2.46it/s]
77%|███████████████████████████████▍ | 192/250 [01:17<00:22, 2.53it/s]
77%|███████████████████████████████▋ | 193/250 [01:18<00:23, 2.48it/s]
78%|███████████████████████████████▊ | 194/250 [01:18<00:22, 2.52it/s]
78%|███████████████████████████████▉ | 195/250 [01:19<00:23, 2.29it/s]
78%|████████████████████████████████▏ | 196/250 [01:19<00:24, 2.23it/s]
79%|████████████████████████████████▎ | 197/250 [01:19<00:23, 2.24it/s]
79%|████████████████████████████████▍ | 198/250 [01:20<00:20, 2.51it/s]
80%|████████████████████████████████▋ | 199/250 [01:20<00:18, 2.73it/s]
80%|████████████████████████████████▊ | 200/250 [01:20<00:18, 2.70it/s]
80%|████████████████████████████████▉ | 201/250 [01:21<00:19, 2.49it/s]
81%|█████████████████████████████████▏ | 202/250 [01:21<00:19, 2.48it/s]
81%|█████████████████████████████████▎ | 203/250 [01:22<00:17, 2.71it/s]
82%|█████████████████████████████████▍ | 204/250 [01:22<00:16, 2.71it/s]
82%|█████████████████████████████████▌ | 205/250 [01:22<00:15, 2.92it/s]
82%|█████████████████████████████████▊ | 206/250 [01:23<00:15, 2.79it/s]
83%|█████████████████████████████████▉ | 207/250 [01:23<00:14, 2.89it/s]
83%|██████████████████████████████████ | 208/250 [01:24<00:17, 2.36it/s]
84%|██████████████████████████████████▎ | 209/250 [01:24<00:16, 2.43it/s]
84%|██████████████████████████████████▍ | 210/250 [01:24<00:15, 2.61it/s]
84%|██████████████████████████████████▌ | 211/250 [01:25<00:16, 2.30it/s]
85%|██████████████████████████████████▊ | 212/250 [01:25<00:15, 2.43it/s]
85%|██████████████████████████████████▉ | 213/250 [01:26<00:14, 2.50it/s]
86%|███████████████████████████████████ | 214/250 [01:26<00:16, 2.17it/s]
86%|███████████████████████████████████▎ | 215/250 [01:27<00:15, 2.28it/s]
86%|███████████████████████████████████▍ | 216/250 [01:27<00:13, 2.47it/s]
87%|███████████████████████████████████▌ | 217/250 [01:27<00:13, 2.51it/s]
87%|███████████████████████████████████▊ | 218/250 [01:28<00:12, 2.62it/s]
88%|███████████████████████████████████▉ | 219/250 [01:28<00:13, 2.31it/s]
88%|████████████████████████████████████ | 220/250 [01:28<00:11, 2.54it/s]
88%|████████████████████████████████████▏ | 221/250 [01:29<00:11, 2.62it/s]
89%|████████████████████████████████████▍ | 222/250 [01:29<00:11, 2.49it/s]
89%|████████████████████████████████████▌ | 223/250 [01:30<00:10, 2.66it/s]
90%|████████████████████████████████████▋ | 224/250 [01:30<00:09, 2.73it/s]
90%|████████████████████████████████████▉ | 225/250 [01:30<00:08, 2.81it/s]
90%|█████████████████████████████████████ | 226/250 [01:31<00:11, 2.12it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:32<00:13, 1.69it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:32<00:11, 1.94it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:32<00:09, 2.21it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:33<00:08, 2.38it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:33<00:07, 2.54it/s]
93%|██████████████████████████████████████ | 232/250 [01:34<00:07, 2.44it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:34<00:06, 2.57it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:35<00:07, 2.11it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:35<00:06, 2.31it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:35<00:05, 2.43it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:36<00:06, 2.16it/s]
95%|███████████████████████████████████████ | 238/250 [01:36<00:04, 2.46it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:37<00:05, 2.13it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:38<00:05, 1.69it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:38<00:04, 1.91it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:38<00:03, 2.17it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:39<00:03, 2.29it/s]
98%|████████████████████████████████████████ | 244/250 [01:39<00:03, 1.96it/s]
98%|████████████████████████████████████████▏| 245/250 [01:40<00:02, 2.23it/s]
98%|████████████████████████████████████████▎| 246/250 [01:40<00:01, 2.42it/s]
99%|████████████████████████████████████████▌| 247/250 [01:40<00:01, 2.40it/s]
99%|████████████████████████████████████████▋| 248/250 [01:41<00:00, 2.39it/s]
100%|████████████████████████████████████████▊| 249/250 [01:41<00:00, 2.14it/s]
{'eval_loss': 1.0832048654556274, 'eval_runtime': 103.4689, 'eval_samples_per_second': 19.329, 'eval_steps_per_second': 2.416, 'epoch': 1.1}
55%|█████████████████████▍ | 429/780 [19:11<2:37:17, 26.89s/it]
100%|█████████████████████████████████████████| 250/250 [01:42<00:00, 1.78it/s]
{'loss': 1.0568, 'learning_rate': 0.00013765182186234817, 'epoch': 1.13}
{'loss': 1.048, 'learning_rate': 0.00012955465587044534, 'epoch': 1.18}
60%|███████████████████████▍ | 468/780 [36:45<2:22:30, 27.41s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:45, 5.43it/s]
1%|▌ | 3/250 [00:00<01:20, 3.08it/s]
2%|▋ | 4/250 [00:01<01:31, 2.68it/s]
2%|▊ | 5/250 [00:01<01:32, 2.64it/s]
2%|█ | 6/250 [00:02<01:53, 2.14it/s]
3%|█▏ | 7/250 [00:02<01:44, 2.32it/s]
3%|█▍ | 8/250 [00:03<01:38, 2.45it/s]
4%|█▌ | 9/250 [00:03<01:33, 2.58it/s]
4%|█▋ | 10/250 [00:03<01:27, 2.74it/s]
4%|█▊ | 11/250 [00:04<01:27, 2.75it/s]
5%|██ | 12/250 [00:04<01:32, 2.57it/s]
5%|██▏ | 13/250 [00:04<01:29, 2.64it/s]
6%|██▎ | 14/250 [00:05<01:30, 2.61it/s]
6%|██▌ | 15/250 [00:05<01:44, 2.25it/s]
6%|██▋ | 16/250 [00:06<01:42, 2.27it/s]
7%|██▊ | 17/250 [00:06<01:43, 2.24it/s]
7%|███ | 18/250 [00:07<01:37, 2.37it/s]
8%|███▏ | 19/250 [00:07<01:42, 2.25it/s]
8%|███▎ | 20/250 [00:08<01:39, 2.32it/s]
8%|███▌ | 21/250 [00:08<02:00, 1.90it/s]
9%|███▋ | 22/250 [00:09<01:47, 2.12it/s]
9%|███▊ | 23/250 [00:09<01:52, 2.02it/s]
10%|████ | 24/250 [00:10<01:40, 2.25it/s]
10%|████▏ | 25/250 [00:10<01:37, 2.31it/s]
10%|████▎ | 26/250 [00:10<01:43, 2.16it/s]
11%|████▌ | 27/250 [00:11<01:35, 2.34it/s]
11%|████▋ | 28/250 [00:11<01:31, 2.43it/s]
12%|████▊ | 29/250 [00:11<01:21, 2.73it/s]
12%|█████ | 30/250 [00:12<01:53, 1.93it/s]
12%|█████▏ | 31/250 [00:13<01:41, 2.17it/s]
13%|█████▍ | 32/250 [00:13<01:48, 2.01it/s]
13%|█████▌ | 33/250 [00:14<01:41, 2.13it/s]
14%|█████▋ | 34/250 [00:14<01:39, 2.16it/s]
14%|█████▉ | 35/250 [00:14<01:35, 2.25it/s]
14%|██████ | 36/250 [00:15<01:27, 2.45it/s]
15%|██████▏ | 37/250 [00:15<01:34, 2.25it/s]
15%|██████▍ | 38/250 [00:16<01:25, 2.49it/s]
16%|██████▌ | 39/250 [00:16<01:18, 2.71it/s]
16%|██████▋ | 40/250 [00:16<01:12, 2.90it/s]
16%|██████▉ | 41/250 [00:17<01:10, 2.98it/s]
17%|███████ | 42/250 [00:17<01:12, 2.87it/s]
17%|███████▏ | 43/250 [00:17<01:10, 2.95it/s]
18%|███████▍ | 44/250 [00:18<01:10, 2.91it/s]
18%|███████▌ | 45/250 [00:18<01:22, 2.48it/s]
18%|███████▋ | 46/250 [00:19<01:25, 2.39it/s]
19%|███████▉ | 47/250 [00:19<01:17, 2.63it/s]
19%|████████ | 48/250 [00:19<01:19, 2.53it/s]
20%|████████▏ | 49/250 [00:20<01:12, 2.78it/s]
20%|████████▍ | 50/250 [00:20<01:11, 2.81it/s]
20%|████████▌ | 51/250 [00:20<01:09, 2.84it/s]
21%|████████▋ | 52/250 [00:21<01:08, 2.89it/s]
21%|████████▉ | 53/250 [00:21<01:05, 3.03it/s]
22%|█████████ | 54/250 [00:21<01:10, 2.79it/s]
22%|█████████▏ | 55/250 [00:22<01:09, 2.80it/s]
22%|█████████▍ | 56/250 [00:22<01:11, 2.70it/s]
23%|█████████▌ | 57/250 [00:22<01:08, 2.80it/s]
23%|█████████▋ | 58/250 [00:23<01:22, 2.32it/s]
24%|█████████▉ | 59/250 [00:23<01:23, 2.28it/s]
24%|██████████ | 60/250 [00:24<01:20, 2.35it/s]
24%|██████████▏ | 61/250 [00:25<01:45, 1.79it/s]
25%|██████████▍ | 62/250 [00:25<01:34, 1.99it/s]
25%|██████████▌ | 63/250 [00:26<01:30, 2.06it/s]
26%|██████████▊ | 64/250 [00:26<01:24, 2.20it/s]
26%|██████████▉ | 65/250 [00:26<01:17, 2.38it/s]
26%|███████████ | 66/250 [00:27<01:13, 2.51it/s]
27%|███████████▎ | 67/250 [00:27<01:14, 2.46it/s]
27%|███████████▍ | 68/250 [00:27<01:10, 2.57it/s]
28%|███████████▌ | 69/250 [00:28<01:08, 2.64it/s]
28%|███████████▊ | 70/250 [00:28<01:04, 2.78it/s]
28%|███████████▉ | 71/250 [00:28<01:05, 2.72it/s]
29%|████████████ | 72/250 [00:29<01:04, 2.78it/s]
29%|████████████▎ | 73/250 [00:29<01:06, 2.68it/s]
30%|████████████▍ | 74/250 [00:29<01:01, 2.86it/s]
30%|████████████▌ | 75/250 [00:30<00:59, 2.93it/s]
30%|████████████▊ | 76/250 [00:30<00:58, 2.97it/s]
31%|████████████▉ | 77/250 [00:30<00:59, 2.92it/s]
31%|█████████████ | 78/250 [00:31<00:55, 3.07it/s]
32%|█████████████▎ | 79/250 [00:31<00:59, 2.86it/s]
32%|█████████████▍ | 80/250 [00:32<01:00, 2.79it/s]
32%|█████████████▌ | 81/250 [00:32<00:57, 2.94it/s]
33%|█████████████▊ | 82/250 [00:32<00:52, 3.22it/s]
33%|█████████████▉ | 83/250 [00:32<00:55, 2.99it/s]
34%|██████████████ | 84/250 [00:33<01:04, 2.56it/s]
34%|██████████████▎ | 85/250 [00:33<01:04, 2.57it/s]
34%|██████████████▍ | 86/250 [00:34<01:04, 2.55it/s]
35%|██████████████▌ | 87/250 [00:34<01:13, 2.21it/s]
35%|██████████████▊ | 88/250 [00:35<01:05, 2.48it/s]
36%|██████████████▉ | 89/250 [00:35<01:02, 2.59it/s]
36%|███████████████ | 90/250 [00:35<01:02, 2.57it/s]
36%|███████████████▎ | 91/250 [00:36<00:58, 2.73it/s]
37%|███████████████▍ | 92/250 [00:36<00:56, 2.79it/s]
37%|███████████████▌ | 93/250 [00:37<01:14, 2.11it/s]
38%|███████████████▊ | 94/250 [00:37<01:08, 2.29it/s]
38%|███████████████▉ | 95/250 [00:38<01:05, 2.38it/s]
38%|████████████████▏ | 96/250 [00:38<01:01, 2.51it/s]
39%|████████████████▎ | 97/250 [00:38<00:55, 2.77it/s]
39%|████████████████▍ | 98/250 [00:39<01:01, 2.48it/s]
40%|████████████████▋ | 99/250 [00:39<00:58, 2.57it/s]
40%|████████████████▍ | 100/250 [00:39<00:54, 2.77it/s]
40%|████████████████▌ | 101/250 [00:40<00:52, 2.84it/s]
41%|████████████████▋ | 102/250 [00:40<00:49, 3.02it/s]
41%|████████████████▉ | 103/250 [00:41<01:12, 2.03it/s]
42%|█████████████████ | 104/250 [00:41<01:11, 2.06it/s]
42%|█████████████████▏ | 105/250 [00:42<01:04, 2.25it/s]
42%|█████████████████▍ | 106/250 [00:42<00:59, 2.40it/s]
43%|█████████████████▌ | 107/250 [00:42<00:58, 2.44it/s]
43%|█████████████████▋ | 108/250 [00:43<01:01, 2.33it/s]
44%|█████████████████▉ | 109/250 [00:43<00:54, 2.60it/s]
44%|██████████████████ | 110/250 [00:43<00:50, 2.77it/s]
44%|██████████████████▏ | 111/250 [00:44<00:52, 2.63it/s]
45%|██████████████████▎ | 112/250 [00:44<00:53, 2.59it/s]
45%|██████████████████▌ | 113/250 [00:44<00:47, 2.90it/s]
46%|██████████████████▋ | 114/250 [00:45<00:46, 2.95it/s]
46%|██████████████████▊ | 115/250 [00:45<00:54, 2.48it/s]
46%|███████████████████ | 116/250 [00:46<00:50, 2.67it/s]
47%|███████████████████▏ | 117/250 [00:46<00:54, 2.44it/s]
47%|███████████████████▎ | 118/250 [00:47<01:12, 1.82it/s]
48%|███████████████████▌ | 119/250 [00:47<01:06, 1.96it/s]
48%|███████████████████▋ | 120/250 [00:48<00:57, 2.26it/s]
48%|███████████████████▊ | 121/250 [00:48<00:52, 2.48it/s]
49%|████████████████████ | 122/250 [00:48<00:48, 2.67it/s]
49%|████████████████████▏ | 123/250 [00:49<00:45, 2.81it/s]
50%|████████████████████▎ | 124/250 [00:49<00:43, 2.92it/s]
50%|████████████████████▌ | 125/250 [00:49<00:42, 2.93it/s]
50%|████████████████████▋ | 126/250 [00:50<00:46, 2.68it/s]
51%|████████████████████▊ | 127/250 [00:50<00:43, 2.80it/s]
51%|████████████████████▉ | 128/250 [00:51<00:46, 2.64it/s]
52%|█████████████████████▏ | 129/250 [00:51<00:51, 2.37it/s]
52%|█████████████████████▎ | 130/250 [00:51<00:48, 2.49it/s]
52%|█████████████████████▍ | 131/250 [00:52<00:57, 2.06it/s]
53%|█████████████████████▋ | 132/250 [00:52<00:53, 2.20it/s]
53%|█████████████████████▊ | 133/250 [00:53<00:55, 2.11it/s]
54%|█████████████████████▉ | 134/250 [00:53<00:49, 2.36it/s]
54%|██████████████████████▏ | 135/250 [00:54<00:47, 2.44it/s]
54%|██████████████████████▎ | 136/250 [00:54<00:44, 2.57it/s]
55%|██████████████████████▍ | 137/250 [00:54<00:44, 2.55it/s]
55%|██████████████████████▋ | 138/250 [00:55<00:43, 2.56it/s]
56%|██████████████████████▊ | 139/250 [00:55<00:41, 2.67it/s]
56%|██████████████████████▉ | 140/250 [00:55<00:40, 2.70it/s]
56%|███████████████████████ | 141/250 [00:56<00:41, 2.62it/s]
57%|███████████████████████▎ | 142/250 [00:56<00:43, 2.48it/s]
57%|███████████████████████▍ | 143/250 [00:57<00:39, 2.69it/s]
58%|███████████████████████▌ | 144/250 [00:57<00:47, 2.22it/s]
58%|███████████████████████▊ | 145/250 [00:58<00:44, 2.35it/s]
58%|███████████████████████▉ | 146/250 [00:58<00:43, 2.40it/s]
59%|████████████████████████ | 147/250 [00:58<00:41, 2.50it/s]
59%|████████████████████████▎ | 148/250 [00:59<00:42, 2.41it/s]
60%|████████████████████████▍ | 149/250 [00:59<00:46, 2.16it/s]
60%|████████████████████████▌ | 150/250 [01:00<00:41, 2.38it/s]
60%|████████████████████████▊ | 151/250 [01:00<00:39, 2.51it/s]
61%|████████████████████████▉ | 152/250 [01:00<00:38, 2.55it/s]
61%|█████████████████████████ | 153/250 [01:01<00:38, 2.53it/s]
62%|█████████████████████████▎ | 154/250 [01:01<00:42, 2.27it/s]
62%|█████████████████████████▍ | 155/250 [01:02<00:39, 2.41it/s]
62%|█████████████████████████▌ | 156/250 [01:02<00:37, 2.48it/s]
63%|█████████████████████████▋ | 157/250 [01:03<00:36, 2.51it/s]
63%|█████████████████████████▉ | 158/250 [01:03<00:33, 2.75it/s]
64%|██████████████████████████ | 159/250 [01:03<00:32, 2.81it/s]
64%|██████████████████████████▏ | 160/250 [01:03<00:31, 2.87it/s]
64%|██████████████████████████▍ | 161/250 [01:04<00:30, 2.88it/s]
65%|██████████████████████████▌ | 162/250 [01:04<00:35, 2.47it/s]
65%|██████████████████████████▋ | 163/250 [01:05<00:39, 2.22it/s]
66%|██████████████████████████▉ | 164/250 [01:05<00:41, 2.08it/s]
66%|███████████████████████████ | 165/250 [01:06<00:47, 1.79it/s]
66%|███████████████████████████▏ | 166/250 [01:07<00:41, 2.03it/s]
67%|███████████████████████████▍ | 167/250 [01:07<00:39, 2.09it/s]
67%|███████████████████████████▌ | 168/250 [01:07<00:35, 2.31it/s]
68%|███████████████████████████▋ | 169/250 [01:08<00:33, 2.41it/s]
68%|███████████████████████████▉ | 170/250 [01:08<00:32, 2.43it/s]
68%|████████████████████████████ | 171/250 [01:09<00:36, 2.15it/s]
69%|████████████████████████████▏ | 172/250 [01:09<00:34, 2.28it/s]
69%|████████████████████████████▎ | 173/250 [01:09<00:31, 2.47it/s]
70%|████████████████████████████▌ | 174/250 [01:10<00:35, 2.17it/s]
70%|████████████████████████████▋ | 175/250 [01:10<00:31, 2.35it/s]
70%|████████████████████████████▊ | 176/250 [01:11<00:29, 2.52it/s]
71%|█████████████████████████████ | 177/250 [01:11<00:29, 2.46it/s]
71%|█████████████████████████████▏ | 178/250 [01:11<00:27, 2.60it/s]
72%|█████████████████████████████▎ | 179/250 [01:12<00:29, 2.44it/s]
72%|█████████████████████████████▌ | 180/250 [01:12<00:25, 2.70it/s]
72%|█████████████████████████████▋ | 181/250 [01:13<00:27, 2.53it/s]
73%|█████████████████████████████▊ | 182/250 [01:13<00:24, 2.82it/s]
73%|██████████████████████████████ | 183/250 [01:13<00:27, 2.48it/s]
74%|██████████████████████████████▏ | 184/250 [01:14<00:26, 2.51it/s]
74%|██████████████████████████████▎ | 185/250 [01:14<00:30, 2.16it/s]
74%|██████████████████████████████▌ | 186/250 [01:15<00:27, 2.35it/s]
75%|██████████████████████████████▋ | 187/250 [01:15<00:28, 2.22it/s]
75%|██████████████████████████████▊ | 188/250 [01:16<00:26, 2.31it/s]
76%|██████████████████████████████▉ | 189/250 [01:16<00:27, 2.26it/s]
76%|███████████████████████████████▏ | 190/250 [01:16<00:25, 2.38it/s]
76%|███████████████████████████████▎ | 191/250 [01:17<00:23, 2.53it/s]
77%|███████████████████████████████▍ | 192/250 [01:17<00:22, 2.58it/s]
77%|███████████████████████████████▋ | 193/250 [01:18<00:22, 2.51it/s]
78%|███████████████████████████████▊ | 194/250 [01:18<00:22, 2.54it/s]
78%|███████████████████████████████▉ | 195/250 [01:19<00:23, 2.31it/s]
78%|████████████████████████████████▏ | 196/250 [01:19<00:24, 2.24it/s]
79%|████████████████████████████████▎ | 197/250 [01:19<00:23, 2.26it/s]
79%|████████████████████████████████▍ | 198/250 [01:20<00:20, 2.53it/s]
80%|████████████████████████████████▋ | 199/250 [01:20<00:18, 2.74it/s]
80%|████████████████████████████████▊ | 200/250 [01:20<00:18, 2.71it/s]
80%|████████████████████████████████▉ | 201/250 [01:21<00:19, 2.50it/s]
81%|█████████████████████████████████▏ | 202/250 [01:21<00:19, 2.49it/s]
81%|█████████████████████████████████▎ | 203/250 [01:22<00:17, 2.71it/s]
82%|█████████████████████████████████▍ | 204/250 [01:22<00:17, 2.71it/s]
82%|█████████████████████████████████▌ | 205/250 [01:22<00:15, 2.91it/s]
82%|█████████████████████████████████▊ | 206/250 [01:23<00:15, 2.79it/s]
83%|█████████████████████████████████▉ | 207/250 [01:23<00:14, 2.88it/s]
83%|██████████████████████████████████ | 208/250 [01:24<00:17, 2.35it/s]
84%|██████████████████████████████████▎ | 209/250 [01:24<00:16, 2.43it/s]
84%|██████████████████████████████████▍ | 210/250 [01:24<00:15, 2.61it/s]
84%|██████████████████████████████████▌ | 211/250 [01:25<00:16, 2.30it/s]
85%|██████████████████████████████████▊ | 212/250 [01:25<00:15, 2.43it/s]
85%|██████████████████████████████████▉ | 213/250 [01:26<00:14, 2.49it/s]
86%|███████████████████████████████████ | 214/250 [01:26<00:16, 2.17it/s]
86%|███████████████████████████████████▎ | 215/250 [01:27<00:15, 2.28it/s]
86%|███████████████████████████████████▍ | 216/250 [01:27<00:13, 2.47it/s]
87%|███████████████████████████████████▌ | 217/250 [01:27<00:13, 2.50it/s]
87%|███████████████████████████████████▊ | 218/250 [01:28<00:12, 2.62it/s]
88%|███████████████████████████████████▉ | 219/250 [01:28<00:13, 2.31it/s]
88%|████████████████████████████████████ | 220/250 [01:28<00:11, 2.53it/s]
88%|████████████████████████████████████▏ | 221/250 [01:29<00:11, 2.62it/s]
89%|████████████████████████████████████▍ | 222/250 [01:29<00:11, 2.49it/s]
89%|████████████████████████████████████▌ | 223/250 [01:30<00:10, 2.66it/s]
90%|████████████████████████████████████▋ | 224/250 [01:30<00:09, 2.73it/s]
90%|████████████████████████████████████▉ | 225/250 [01:30<00:08, 2.81it/s]
90%|█████████████████████████████████████ | 226/250 [01:31<00:11, 2.06it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:32<00:13, 1.66it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:32<00:11, 1.92it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:33<00:09, 2.18it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:33<00:08, 2.37it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:33<00:07, 2.52it/s]
93%|██████████████████████████████████████ | 232/250 [01:34<00:07, 2.43it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:34<00:06, 2.56it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:35<00:07, 2.10it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:35<00:06, 2.30it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:35<00:05, 2.43it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:36<00:06, 2.12it/s]
95%|███████████████████████████████████████ | 238/250 [01:36<00:04, 2.42it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:37<00:05, 2.14it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:38<00:05, 1.69it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:38<00:04, 1.91it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:38<00:03, 2.17it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:39<00:03, 2.29it/s]
98%|████████████████████████████████████████ | 244/250 [01:39<00:03, 1.95it/s]
98%|████████████████████████████████████████▏| 245/250 [01:40<00:02, 2.22it/s]
98%|████████████████████████████████████████▎| 246/250 [01:40<00:01, 2.41it/s]
99%|████████████████████████████████████████▌| 247/250 [01:41<00:01, 2.40it/s]
99%|████████████████████████████████████████▋| 248/250 [01:41<00:00, 2.38it/s]
100%|████████████████████████████████████████▊| 249/250 [01:42<00:00, 2.14it/s]
{'eval_loss': 1.0818686485290527, 'eval_runtime': 103.5399, 'eval_samples_per_second': 19.316, 'eval_steps_per_second': 2.415, 'epoch': 1.2}
60%|███████████████████████▍ | 468/780 [38:29<2:22:30, 27.41s/it]
100%|█████████████████████████████████████████| 250/250 [01:42<00:00, 1.78it/s]
{'loss': 1.0763, 'learning_rate': 0.0001214574898785425, 'epoch': 1.23}
{'loss': 1.0728, 'learning_rate': 0.00011336032388663968, 'epoch': 1.28}
65%|█████████████████████████▎ | 507/780 [56:02<2:03:19, 27.11s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:45, 5.44it/s]
1%|▌ | 3/250 [00:00<01:19, 3.09it/s]
2%|▋ | 4/250 [00:01<01:31, 2.69it/s]
2%|▊ | 5/250 [00:01<01:32, 2.64it/s]
2%|█ | 6/250 [00:02<01:53, 2.14it/s]
3%|█▏ | 7/250 [00:02<01:44, 2.32it/s]
3%|█▍ | 8/250 [00:03<01:38, 2.46it/s]
4%|█▌ | 9/250 [00:03<01:33, 2.59it/s]
4%|█▋ | 10/250 [00:03<01:27, 2.75it/s]
4%|█▊ | 11/250 [00:04<01:26, 2.76it/s]
5%|██ | 12/250 [00:04<01:32, 2.58it/s]
5%|██▏ | 13/250 [00:04<01:29, 2.65it/s]
6%|██▎ | 14/250 [00:05<01:30, 2.62it/s]
6%|██▌ | 15/250 [00:05<01:44, 2.25it/s]
6%|██▋ | 16/250 [00:06<01:42, 2.28it/s]
7%|██▊ | 17/250 [00:06<01:43, 2.25it/s]
7%|███ | 18/250 [00:07<01:37, 2.38it/s]
8%|███▏ | 19/250 [00:07<01:42, 2.26it/s]
8%|███▎ | 20/250 [00:08<01:39, 2.32it/s]
8%|███▌ | 21/250 [00:08<02:00, 1.91it/s]
9%|███▋ | 22/250 [00:09<01:47, 2.12it/s]
9%|███▊ | 23/250 [00:09<01:52, 2.02it/s]
10%|████ | 24/250 [00:09<01:40, 2.25it/s]
10%|████▏ | 25/250 [00:10<01:37, 2.32it/s]
10%|████▎ | 26/250 [00:10<01:43, 2.17it/s]
11%|████▌ | 27/250 [00:11<01:35, 2.35it/s]
11%|████▋ | 28/250 [00:11<01:31, 2.43it/s]
12%|████▊ | 29/250 [00:11<01:20, 2.73it/s]
12%|█████ | 30/250 [00:12<01:53, 1.94it/s]
12%|█████▏ | 31/250 [00:13<01:40, 2.17it/s]
13%|█████▍ | 32/250 [00:13<01:48, 2.01it/s]
13%|█████▌ | 33/250 [00:14<01:41, 2.14it/s]
14%|█████▋ | 34/250 [00:14<01:39, 2.17it/s]
14%|█████▉ | 35/250 [00:14<01:35, 2.26it/s]
14%|██████ | 36/250 [00:15<01:27, 2.45it/s]
15%|██████▏ | 37/250 [00:15<01:34, 2.25it/s]
15%|██████▍ | 38/250 [00:16<01:24, 2.50it/s]
16%|██████▌ | 39/250 [00:16<01:17, 2.72it/s]
16%|██████▋ | 40/250 [00:16<01:11, 2.92it/s]
16%|██████▉ | 41/250 [00:16<01:09, 2.99it/s]
17%|███████ | 42/250 [00:17<01:12, 2.88it/s]
17%|███████▏ | 43/250 [00:17<01:10, 2.95it/s]
18%|███████▍ | 44/250 [00:18<01:10, 2.91it/s]
18%|███████▌ | 45/250 [00:18<01:22, 2.48it/s]
18%|███████▋ | 46/250 [00:19<01:25, 2.39it/s]
19%|███████▉ | 47/250 [00:19<01:17, 2.63it/s]
19%|████████ | 48/250 [00:19<01:19, 2.53it/s]
20%|████████▏ | 49/250 [00:20<01:12, 2.78it/s]
20%|████████▍ | 50/250 [00:20<01:11, 2.81it/s]
20%|████████▌ | 51/250 [00:20<01:10, 2.84it/s]
21%|████████▋ | 52/250 [00:21<01:08, 2.89it/s]
21%|████████▉ | 53/250 [00:21<01:04, 3.03it/s]
22%|█████████ | 54/250 [00:21<01:09, 2.80it/s]
22%|█████████▏ | 55/250 [00:22<01:09, 2.81it/s]
22%|█████████▍ | 56/250 [00:22<01:11, 2.71it/s]
23%|█████████▌ | 57/250 [00:22<01:08, 2.81it/s]
23%|█████████▋ | 58/250 [00:23<01:22, 2.33it/s]
24%|█████████▉ | 59/250 [00:23<01:23, 2.29it/s]
24%|██████████ | 60/250 [00:24<01:20, 2.35it/s]
24%|██████████▏ | 61/250 [00:25<01:45, 1.79it/s]
25%|██████████▍ | 62/250 [00:25<01:34, 1.99it/s]
25%|██████████▌ | 63/250 [00:25<01:30, 2.06it/s]
26%|██████████▊ | 64/250 [00:26<01:24, 2.20it/s]
26%|██████████▉ | 65/250 [00:26<01:17, 2.39it/s]
26%|███████████ | 66/250 [00:27<01:13, 2.51it/s]
27%|███████████▎ | 67/250 [00:27<01:14, 2.47it/s]
27%|███████████▍ | 68/250 [00:27<01:10, 2.57it/s]
28%|███████████▌ | 69/250 [00:28<01:08, 2.65it/s]
28%|███████████▊ | 70/250 [00:28<01:04, 2.79it/s]
28%|███████████▉ | 71/250 [00:28<01:05, 2.72it/s]
29%|████████████ | 72/250 [00:29<01:03, 2.78it/s]
29%|████████████▎ | 73/250 [00:29<01:05, 2.68it/s]
30%|████████████▍ | 74/250 [00:29<01:01, 2.87it/s]
30%|████████████▌ | 75/250 [00:30<00:59, 2.94it/s]
30%|████████████▊ | 76/250 [00:30<00:58, 2.98it/s]
31%|████████████▉ | 77/250 [00:30<00:59, 2.93it/s]
31%|█████████████ | 78/250 [00:31<00:55, 3.08it/s]
32%|█████████████▎ | 79/250 [00:31<01:01, 2.80it/s]
32%|█████████████▍ | 80/250 [00:31<01:01, 2.75it/s]
32%|█████████████▌ | 81/250 [00:32<00:58, 2.91it/s]
33%|█████████████▊ | 82/250 [00:32<00:52, 3.21it/s]
33%|█████████████▉ | 83/250 [00:32<00:55, 2.99it/s]
34%|██████████████ | 84/250 [00:33<01:04, 2.56it/s]
34%|██████████████▎ | 85/250 [00:33<01:04, 2.57it/s]
34%|██████████████▍ | 86/250 [00:34<01:04, 2.55it/s]
35%|██████████████▌ | 87/250 [00:34<01:13, 2.22it/s]
35%|██████████████▊ | 88/250 [00:35<01:04, 2.49it/s]
36%|██████████████▉ | 89/250 [00:35<01:01, 2.60it/s]
36%|███████████████ | 90/250 [00:35<01:02, 2.58it/s]
36%|███████████████▎ | 91/250 [00:36<00:58, 2.74it/s]
37%|███████████████▍ | 92/250 [00:36<00:56, 2.80it/s]
37%|███████████████▌ | 93/250 [00:37<01:14, 2.11it/s]
38%|███████████████▊ | 94/250 [00:37<01:07, 2.30it/s]
38%|███████████████▉ | 95/250 [00:37<01:04, 2.39it/s]
38%|████████████████▏ | 96/250 [00:38<01:01, 2.52it/s]
39%|████████████████▎ | 97/250 [00:38<00:55, 2.78it/s]
39%|████████████████▍ | 98/250 [00:39<01:01, 2.48it/s]
40%|████████████████▋ | 99/250 [00:39<00:58, 2.57it/s]
40%|████████████████▍ | 100/250 [00:39<00:54, 2.77it/s]
40%|████████████████▌ | 101/250 [00:40<00:52, 2.85it/s]
41%|████████████████▋ | 102/250 [00:40<00:49, 3.02it/s]
41%|████████████████▉ | 103/250 [00:41<01:12, 2.03it/s]
42%|█████████████████ | 104/250 [00:41<01:10, 2.06it/s]
42%|█████████████████▏ | 105/250 [00:42<01:04, 2.25it/s]
42%|█████████████████▍ | 106/250 [00:42<00:59, 2.40it/s]
43%|█████████████████▌ | 107/250 [00:42<00:58, 2.44it/s]
43%|█████████████████▋ | 108/250 [00:43<01:00, 2.33it/s]
44%|█████████████████▉ | 109/250 [00:43<00:54, 2.61it/s]
44%|██████████████████ | 110/250 [00:43<00:50, 2.77it/s]
44%|██████████████████▏ | 111/250 [00:44<00:52, 2.64it/s]
45%|██████████████████▎ | 112/250 [00:44<00:53, 2.60it/s]
45%|██████████████████▌ | 113/250 [00:44<00:47, 2.91it/s]
46%|██████████████████▋ | 114/250 [00:45<00:45, 2.96it/s]
46%|██████████████████▊ | 115/250 [00:45<00:54, 2.49it/s]
46%|███████████████████ | 116/250 [00:46<00:49, 2.68it/s]
47%|███████████████████▏ | 117/250 [00:46<00:54, 2.44it/s]
47%|███████████████████▎ | 118/250 [00:47<01:12, 1.82it/s]
48%|███████████████████▌ | 119/250 [00:47<01:06, 1.97it/s]
48%|███████████████████▋ | 120/250 [00:48<00:57, 2.26it/s]
48%|███████████████████▊ | 121/250 [00:48<00:51, 2.48it/s]
49%|████████████████████ | 122/250 [00:48<00:47, 2.67it/s]
49%|████████████████████▏ | 123/250 [00:49<00:45, 2.82it/s]
50%|████████████████████▎ | 124/250 [00:49<00:42, 2.93it/s]
50%|████████████████████▌ | 125/250 [00:49<00:42, 2.94it/s]
50%|████████████████████▋ | 126/250 [00:50<00:46, 2.69it/s]
51%|████████████████████▊ | 127/250 [00:50<00:43, 2.81it/s]
51%|████████████████████▉ | 128/250 [00:50<00:46, 2.64it/s]
52%|█████████████████████▏ | 129/250 [00:51<00:50, 2.37it/s]
52%|█████████████████████▎ | 130/250 [00:51<00:48, 2.50it/s]
52%|█████████████████████▍ | 131/250 [00:52<00:57, 2.07it/s]
53%|█████████████████████▋ | 132/250 [00:52<00:53, 2.20it/s]
53%|█████████████████████▊ | 133/250 [00:53<00:55, 2.11it/s]
54%|█████████████████████▉ | 134/250 [00:53<00:48, 2.37it/s]
54%|██████████████████████▏ | 135/250 [00:54<00:46, 2.45it/s]
54%|██████████████████████▎ | 136/250 [00:54<00:44, 2.57it/s]
55%|██████████████████████▍ | 137/250 [00:54<00:44, 2.56it/s]
55%|██████████████████████▋ | 138/250 [00:55<00:43, 2.56it/s]
56%|██████████████████████▊ | 139/250 [00:55<00:41, 2.68it/s]
56%|██████████████████████▉ | 140/250 [00:55<00:40, 2.70it/s]
56%|███████████████████████ | 141/250 [00:56<00:41, 2.62it/s]
57%|███████████████████████▎ | 142/250 [00:56<00:43, 2.48it/s]
57%|███████████████████████▍ | 143/250 [00:57<00:39, 2.70it/s]
58%|███████████████████████▌ | 144/250 [00:57<00:47, 2.23it/s]
58%|███████████████████████▊ | 145/250 [00:58<00:44, 2.36it/s]
58%|███████████████████████▉ | 146/250 [00:58<00:42, 2.42it/s]
59%|████████████████████████ | 147/250 [00:58<00:40, 2.52it/s]
59%|████████████████████████▎ | 148/250 [00:59<00:42, 2.42it/s]
60%|████████████████████████▍ | 149/250 [00:59<00:46, 2.16it/s]
60%|████████████████████████▌ | 150/250 [01:00<00:41, 2.39it/s]
60%|████████████████████████▊ | 151/250 [01:00<00:39, 2.51it/s]
61%|████████████████████████▉ | 152/250 [01:00<00:38, 2.55it/s]
61%|█████████████████████████ | 153/250 [01:01<00:38, 2.53it/s]
62%|█████████████████████████▎ | 154/250 [01:01<00:42, 2.27it/s]
62%|█████████████████████████▍ | 155/250 [01:02<00:39, 2.41it/s]
62%|█████████████████████████▌ | 156/250 [01:02<00:37, 2.48it/s]
63%|█████████████████████████▋ | 157/250 [01:02<00:36, 2.52it/s]
63%|█████████████████████████▉ | 158/250 [01:03<00:33, 2.75it/s]
64%|██████████████████████████ | 159/250 [01:03<00:32, 2.81it/s]
64%|██████████████████████████▏ | 160/250 [01:03<00:31, 2.88it/s]
64%|██████████████████████████▍ | 161/250 [01:04<00:30, 2.89it/s]
65%|██████████████████████████▌ | 162/250 [01:04<00:35, 2.47it/s]
65%|██████████████████████████▋ | 163/250 [01:05<00:39, 2.22it/s]
66%|██████████████████████████▉ | 164/250 [01:05<00:41, 2.08it/s]
66%|███████████████████████████ | 165/250 [01:06<00:47, 1.79it/s]
66%|███████████████████████████▏ | 166/250 [01:06<00:41, 2.03it/s]
67%|███████████████████████████▍ | 167/250 [01:07<00:39, 2.08it/s]
67%|███████████████████████████▌ | 168/250 [01:07<00:35, 2.30it/s]
68%|███████████████████████████▋ | 169/250 [01:08<00:33, 2.40it/s]
68%|███████████████████████████▉ | 170/250 [01:08<00:32, 2.42it/s]
68%|████████████████████████████ | 171/250 [01:09<00:36, 2.15it/s]
69%|████████████████████████████▏ | 172/250 [01:09<00:34, 2.27it/s]
69%|████████████████████████████▎ | 173/250 [01:09<00:31, 2.47it/s]
70%|████████████████████████████▌ | 174/250 [01:10<00:35, 2.16it/s]
70%|████████████████████████████▋ | 175/250 [01:10<00:32, 2.34it/s]
70%|████████████████████████████▊ | 176/250 [01:11<00:29, 2.51it/s]
71%|█████████████████████████████ | 177/250 [01:11<00:29, 2.45it/s]
71%|█████████████████████████████▏ | 178/250 [01:11<00:27, 2.60it/s]
72%|█████████████████████████████▎ | 179/250 [01:12<00:29, 2.44it/s]
72%|█████████████████████████████▌ | 180/250 [01:12<00:25, 2.69it/s]
72%|█████████████████████████████▋ | 181/250 [01:13<00:27, 2.53it/s]
73%|█████████████████████████████▊ | 182/250 [01:13<00:24, 2.82it/s]
73%|██████████████████████████████ | 183/250 [01:13<00:27, 2.48it/s]
74%|██████████████████████████████▏ | 184/250 [01:14<00:26, 2.51it/s]
74%|██████████████████████████████▎ | 185/250 [01:14<00:30, 2.16it/s]
74%|██████████████████████████████▌ | 186/250 [01:15<00:27, 2.35it/s]
75%|██████████████████████████████▋ | 187/250 [01:15<00:28, 2.22it/s]
75%|██████████████████████████████▊ | 188/250 [01:16<00:26, 2.31it/s]
76%|██████████████████████████████▉ | 189/250 [01:16<00:27, 2.26it/s]
76%|███████████████████████████████▏ | 190/250 [01:16<00:25, 2.39it/s]
76%|███████████████████████████████▎ | 191/250 [01:17<00:23, 2.54it/s]
77%|███████████████████████████████▍ | 192/250 [01:17<00:22, 2.57it/s]
77%|███████████████████████████████▋ | 193/250 [01:18<00:22, 2.51it/s]
78%|███████████████████████████████▊ | 194/250 [01:18<00:22, 2.54it/s]
78%|███████████████████████████████▉ | 195/250 [01:18<00:23, 2.30it/s]
78%|████████████████████████████████▏ | 196/250 [01:19<00:24, 2.24it/s]
79%|████████████████████████████████▎ | 197/250 [01:19<00:23, 2.26it/s]
79%|████████████████████████████████▍ | 198/250 [01:20<00:20, 2.53it/s]
80%|████████████████████████████████▋ | 199/250 [01:20<00:18, 2.74it/s]
80%|████████████████████████████████▊ | 200/250 [01:20<00:18, 2.71it/s]
80%|████████████████████████████████▉ | 201/250 [01:21<00:19, 2.49it/s]
81%|█████████████████████████████████▏ | 202/250 [01:21<00:19, 2.48it/s]
81%|█████████████████████████████████▎ | 203/250 [01:21<00:17, 2.71it/s]
82%|█████████████████████████████████▍ | 204/250 [01:22<00:16, 2.71it/s]
82%|█████████████████████████████████▌ | 205/250 [01:22<00:15, 2.92it/s]
82%|█████████████████████████████████▊ | 206/250 [01:23<00:15, 2.80it/s]
83%|█████████████████████████████████▉ | 207/250 [01:23<00:14, 2.89it/s]
83%|██████████████████████████████████ | 208/250 [01:23<00:17, 2.36it/s]
84%|██████████████████████████████████▎ | 209/250 [01:24<00:16, 2.43it/s]
84%|██████████████████████████████████▍ | 210/250 [01:24<00:15, 2.62it/s]
84%|██████████████████████████████████▌ | 211/250 [01:25<00:16, 2.30it/s]
85%|██████████████████████████████████▊ | 212/250 [01:25<00:15, 2.43it/s]
85%|██████████████████████████████████▉ | 213/250 [01:25<00:14, 2.50it/s]
86%|███████████████████████████████████ | 214/250 [01:26<00:16, 2.13it/s]
86%|███████████████████████████████████▎ | 215/250 [01:26<00:15, 2.24it/s]
86%|███████████████████████████████████▍ | 216/250 [01:27<00:13, 2.44it/s]
87%|███████████████████████████████████▌ | 217/250 [01:27<00:13, 2.48it/s]
87%|███████████████████████████████████▊ | 218/250 [01:27<00:12, 2.61it/s]
88%|███████████████████████████████████▉ | 219/250 [01:28<00:13, 2.30it/s]
88%|████████████████████████████████████ | 220/250 [01:28<00:11, 2.53it/s]
88%|████████████████████████████████████▏ | 221/250 [01:29<00:11, 2.62it/s]
89%|████████████████████████████████████▍ | 222/250 [01:29<00:11, 2.49it/s]
89%|████████████████████████████████████▌ | 223/250 [01:29<00:10, 2.66it/s]
90%|████████████████████████████████████▋ | 224/250 [01:30<00:09, 2.74it/s]
90%|████████████████████████████████████▉ | 225/250 [01:30<00:08, 2.81it/s]
90%|█████████████████████████████████████ | 226/250 [01:31<00:11, 2.13it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:32<00:13, 1.69it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:32<00:11, 1.95it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:32<00:09, 2.21it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:33<00:08, 2.39it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:33<00:07, 2.53it/s]
93%|██████████████████████████████████████ | 232/250 [01:34<00:07, 2.44it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:34<00:06, 2.56it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:35<00:07, 2.11it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:35<00:06, 2.30it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:35<00:05, 2.43it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:36<00:06, 2.16it/s]
95%|███████████████████████████████████████ | 238/250 [01:36<00:04, 2.45it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:37<00:05, 2.17it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:38<00:05, 1.71it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:38<00:04, 1.92it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:38<00:03, 2.18it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:39<00:03, 2.30it/s]
98%|████████████████████████████████████████ | 244/250 [01:39<00:03, 1.96it/s]
98%|████████████████████████████████████████▏| 245/250 [01:40<00:02, 2.23it/s]
98%|████████████████████████████████████████▎| 246/250 [01:40<00:01, 2.41it/s]
99%|████████████████████████████████████████▌| 247/250 [01:40<00:01, 2.40it/s]
99%|████████████████████████████████████████▋| 248/250 [01:41<00:00, 2.38it/s]
100%|████████████████████████████████████████▊| 249/250 [01:41<00:00, 2.13it/s]
{'eval_loss': 1.0797652006149292, 'eval_runtime': 103.3861, 'eval_samples_per_second': 19.345, 'eval_steps_per_second': 2.418, 'epoch': 1.3}
65%|█████████████████████████▎ | 507/780 [57:45<2:03:19, 27.11s/it]
100%|█████████████████████████████████████████| 250/250 [01:42<00:00, 1.78it/s]
{'loss': 1.0451, 'learning_rate': 0.00010526315789473682, 'epoch': 1.33}
{'loss': 1.0513, 'learning_rate': 9.716599190283401e-05, 'epoch': 1.38}
70%|█████████████████████████▉ | 546/780 [1:15:23<1:47:05, 27.46s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:45, 5.43it/s]
1%|▌ | 3/250 [00:00<01:20, 3.08it/s]
2%|▋ | 4/250 [00:01<01:31, 2.68it/s]
2%|▊ | 5/250 [00:01<01:32, 2.64it/s]
2%|█ | 6/250 [00:02<01:53, 2.15it/s]
3%|█▏ | 7/250 [00:02<01:44, 2.32it/s]
3%|█▍ | 8/250 [00:03<01:39, 2.44it/s]
4%|█▌ | 9/250 [00:03<01:33, 2.58it/s]
4%|█▋ | 10/250 [00:03<01:27, 2.73it/s]
4%|█▊ | 11/250 [00:04<01:27, 2.74it/s]
5%|██ | 12/250 [00:04<01:32, 2.57it/s]
5%|██▏ | 13/250 [00:04<01:29, 2.64it/s]
6%|██▎ | 14/250 [00:05<01:30, 2.61it/s]
6%|██▌ | 15/250 [00:05<01:44, 2.25it/s]
6%|██▋ | 16/250 [00:06<01:42, 2.28it/s]
7%|██▊ | 17/250 [00:06<01:43, 2.25it/s]
7%|███ | 18/250 [00:07<01:37, 2.38it/s]
8%|███▏ | 19/250 [00:07<01:42, 2.26it/s]
8%|███▎ | 20/250 [00:08<01:39, 2.32it/s]
8%|███▌ | 21/250 [00:08<02:00, 1.90it/s]
9%|███▋ | 22/250 [00:09<01:47, 2.12it/s]
9%|███▊ | 23/250 [00:09<01:52, 2.02it/s]
10%|████ | 24/250 [00:10<01:40, 2.25it/s]
10%|████▏ | 25/250 [00:10<01:37, 2.31it/s]
10%|████▎ | 26/250 [00:10<01:43, 2.16it/s]
11%|████▌ | 27/250 [00:11<01:35, 2.34it/s]
11%|████▋ | 28/250 [00:11<01:32, 2.41it/s]
12%|████▊ | 29/250 [00:11<01:21, 2.71it/s]
12%|█████ | 30/250 [00:12<01:54, 1.93it/s]
12%|█████▏ | 31/250 [00:13<01:41, 2.16it/s]
13%|█████▍ | 32/250 [00:13<01:49, 2.00it/s]
13%|█████▌ | 33/250 [00:14<01:42, 2.13it/s]
14%|█████▋ | 34/250 [00:14<01:39, 2.16it/s]
14%|█████▉ | 35/250 [00:14<01:35, 2.25it/s]
14%|██████ | 36/250 [00:15<01:27, 2.45it/s]
15%|██████▏ | 37/250 [00:15<01:34, 2.25it/s]
15%|██████▍ | 38/250 [00:16<01:25, 2.49it/s]
16%|██████▌ | 39/250 [00:16<01:17, 2.71it/s]
16%|██████▋ | 40/250 [00:16<01:12, 2.91it/s]
16%|██████▉ | 41/250 [00:17<01:09, 2.99it/s]
17%|███████ | 42/250 [00:17<01:12, 2.88it/s]
17%|███████▏ | 43/250 [00:17<01:10, 2.95it/s]
18%|███████▍ | 44/250 [00:18<01:10, 2.91it/s]
18%|███████▌ | 45/250 [00:18<01:22, 2.47it/s]
18%|███████▋ | 46/250 [00:19<01:25, 2.39it/s]
19%|███████▉ | 47/250 [00:19<01:17, 2.63it/s]
19%|████████ | 48/250 [00:19<01:19, 2.53it/s]
20%|████████▏ | 49/250 [00:20<01:12, 2.78it/s]
20%|████████▍ | 50/250 [00:20<01:11, 2.81it/s]
20%|████████▌ | 51/250 [00:20<01:09, 2.84it/s]
21%|████████▋ | 52/250 [00:21<01:08, 2.88it/s]
21%|████████▉ | 53/250 [00:21<01:05, 3.02it/s]
22%|█████████ | 54/250 [00:21<01:10, 2.79it/s]
22%|█████████▏ | 55/250 [00:22<01:09, 2.81it/s]
22%|█████████▍ | 56/250 [00:22<01:11, 2.70it/s]
23%|█████████▌ | 57/250 [00:22<01:08, 2.80it/s]
23%|█████████▋ | 58/250 [00:23<01:22, 2.33it/s]
24%|█████████▉ | 59/250 [00:23<01:23, 2.29it/s]
24%|██████████ | 60/250 [00:24<01:20, 2.35it/s]
24%|██████████▏ | 61/250 [00:25<01:45, 1.79it/s]
25%|██████████▍ | 62/250 [00:25<01:34, 1.99it/s]
25%|██████████▌ | 63/250 [00:26<01:31, 2.05it/s]
26%|██████████▊ | 64/250 [00:26<01:24, 2.20it/s]
26%|██████████▉ | 65/250 [00:26<01:17, 2.38it/s]
26%|███████████ | 66/250 [00:27<01:13, 2.51it/s]
27%|███████████▎ | 67/250 [00:27<01:14, 2.47it/s]
27%|███████████▍ | 68/250 [00:27<01:10, 2.57it/s]
28%|███████████▌ | 69/250 [00:28<01:08, 2.65it/s]
28%|███████████▊ | 70/250 [00:28<01:04, 2.79it/s]
28%|███████████▉ | 71/250 [00:28<01:05, 2.72it/s]
29%|████████████ | 72/250 [00:29<01:03, 2.78it/s]
29%|████████████▎ | 73/250 [00:29<01:05, 2.68it/s]
30%|████████████▍ | 74/250 [00:29<01:01, 2.87it/s]
30%|████████████▌ | 75/250 [00:30<00:59, 2.93it/s]
30%|████████████▊ | 76/250 [00:30<00:58, 2.98it/s]
31%|████████████▉ | 77/250 [00:30<00:59, 2.92it/s]
31%|█████████████ | 78/250 [00:31<00:56, 3.07it/s]
32%|█████████████▎ | 79/250 [00:31<00:59, 2.86it/s]
32%|█████████████▍ | 80/250 [00:32<01:00, 2.79it/s]
32%|█████████████▌ | 81/250 [00:32<00:57, 2.93it/s]
33%|█████████████▊ | 82/250 [00:32<00:52, 3.22it/s]
33%|█████████████▉ | 83/250 [00:32<00:55, 2.99it/s]
34%|██████████████ | 84/250 [00:33<01:04, 2.56it/s]
34%|██████████████▎ | 85/250 [00:33<01:04, 2.57it/s]
34%|██████████████▍ | 86/250 [00:34<01:04, 2.55it/s]
35%|██████████████▌ | 87/250 [00:34<01:13, 2.21it/s]
35%|██████████████▊ | 88/250 [00:35<01:05, 2.48it/s]
36%|██████████████▉ | 89/250 [00:35<01:02, 2.59it/s]
36%|███████████████ | 90/250 [00:35<01:02, 2.57it/s]
36%|███████████████▎ | 91/250 [00:36<00:58, 2.73it/s]
37%|███████████████▍ | 92/250 [00:36<00:56, 2.79it/s]
37%|███████████████▌ | 93/250 [00:37<01:14, 2.11it/s]
38%|███████████████▊ | 94/250 [00:37<01:07, 2.30it/s]
38%|███████████████▉ | 95/250 [00:38<01:04, 2.39it/s]
38%|████████████████▏ | 96/250 [00:38<01:01, 2.52it/s]
39%|████████████████▎ | 97/250 [00:38<00:55, 2.78it/s]
39%|████████████████▍ | 98/250 [00:39<01:01, 2.48it/s]
40%|████████████████▋ | 99/250 [00:39<00:58, 2.57it/s]
40%|████████████████▍ | 100/250 [00:39<00:54, 2.77it/s]
40%|████████████████▌ | 101/250 [00:40<00:52, 2.84it/s]
41%|████████████████▋ | 102/250 [00:40<00:49, 3.01it/s]
41%|████████████████▉ | 103/250 [00:41<01:12, 2.02it/s]
42%|█████████████████ | 104/250 [00:41<01:11, 2.05it/s]
42%|█████████████████▏ | 105/250 [00:42<01:04, 2.25it/s]
42%|█████████████████▍ | 106/250 [00:42<01:00, 2.39it/s]
43%|█████████████████▌ | 107/250 [00:42<00:58, 2.44it/s]
43%|█████████████████▋ | 108/250 [00:43<01:01, 2.33it/s]
44%|█████████████████▉ | 109/250 [00:43<00:54, 2.60it/s]
44%|██████████████████ | 110/250 [00:43<00:50, 2.77it/s]
44%|██████████████████▏ | 111/250 [00:44<00:52, 2.63it/s]
45%|██████████████████▎ | 112/250 [00:44<00:53, 2.59it/s]
45%|██████████████████▌ | 113/250 [00:44<00:47, 2.91it/s]
46%|██████████████████▋ | 114/250 [00:45<00:45, 2.96it/s]
46%|██████████████████▊ | 115/250 [00:45<00:54, 2.48it/s]
46%|███████████████████ | 116/250 [00:46<00:50, 2.68it/s]
47%|███████████████████▏ | 117/250 [00:46<00:54, 2.44it/s]
47%|███████████████████▎ | 118/250 [00:47<01:12, 1.82it/s]
48%|███████████████████▌ | 119/250 [00:47<01:06, 1.97it/s]
48%|███████████████████▋ | 120/250 [00:48<00:57, 2.27it/s]
48%|███████████████████▊ | 121/250 [00:48<00:51, 2.49it/s]
49%|████████████████████ | 122/250 [00:48<00:47, 2.67it/s]
49%|████████████████████▏ | 123/250 [00:49<00:45, 2.82it/s]
50%|████████████████████▎ | 124/250 [00:49<00:42, 2.93it/s]
50%|████████████████████▌ | 125/250 [00:49<00:42, 2.93it/s]
50%|████████████████████▋ | 126/250 [00:50<00:46, 2.68it/s]
51%|████████████████████▊ | 127/250 [00:50<00:43, 2.80it/s]
51%|████████████████████▉ | 128/250 [00:51<00:46, 2.64it/s]
52%|█████████████████████▏ | 129/250 [00:51<00:51, 2.37it/s]
52%|█████████████████████▎ | 130/250 [00:51<00:48, 2.49it/s]
52%|█████████████████████▍ | 131/250 [00:52<00:57, 2.06it/s]
53%|█████████████████████▋ | 132/250 [00:52<00:53, 2.20it/s]
53%|█████████████████████▊ | 133/250 [00:53<00:55, 2.11it/s]
54%|█████████████████████▉ | 134/250 [00:53<00:49, 2.37it/s]
54%|██████████████████████▏ | 135/250 [00:54<00:46, 2.45it/s]
54%|██████████████████████▎ | 136/250 [00:54<00:44, 2.57it/s]
55%|██████████████████████▍ | 137/250 [00:54<00:44, 2.56it/s]
55%|██████████████████████▋ | 138/250 [00:55<00:43, 2.56it/s]
56%|██████████████████████▊ | 139/250 [00:55<00:41, 2.68it/s]
56%|██████████████████████▉ | 140/250 [00:55<00:40, 2.71it/s]
56%|███████████████████████ | 141/250 [00:56<00:41, 2.62it/s]
57%|███████████████████████▎ | 142/250 [00:56<00:43, 2.48it/s]
57%|███████████████████████▍ | 143/250 [00:57<00:39, 2.71it/s]
58%|███████████████████████▌ | 144/250 [00:57<00:47, 2.23it/s]
58%|███████████████████████▊ | 145/250 [00:58<00:44, 2.36it/s]
58%|███████████████████████▉ | 146/250 [00:58<00:42, 2.42it/s]
59%|████████████████████████ | 147/250 [00:58<00:40, 2.52it/s]
59%|████████████████████████▎ | 148/250 [00:59<00:42, 2.42it/s]
60%|████████████████████████▍ | 149/250 [00:59<00:46, 2.16it/s]
60%|████████████████████████▌ | 150/250 [01:00<00:41, 2.39it/s]
60%|████████████████████████▊ | 151/250 [01:00<00:39, 2.52it/s]
61%|████████████████████████▉ | 152/250 [01:00<00:38, 2.55it/s]
61%|█████████████████████████ | 153/250 [01:01<00:38, 2.53it/s]
62%|█████████████████████████▎ | 154/250 [01:01<00:42, 2.27it/s]
62%|█████████████████████████▍ | 155/250 [01:02<00:39, 2.41it/s]
62%|█████████████████████████▌ | 156/250 [01:02<00:37, 2.49it/s]
63%|█████████████████████████▋ | 157/250 [01:03<00:36, 2.51it/s]
63%|█████████████████████████▉ | 158/250 [01:03<00:33, 2.75it/s]
64%|██████████████████████████ | 159/250 [01:03<00:32, 2.81it/s]
64%|██████████████████████████▏ | 160/250 [01:03<00:31, 2.87it/s]
64%|██████████████████████████▍ | 161/250 [01:04<00:30, 2.88it/s]
65%|██████████████████████████▌ | 162/250 [01:04<00:35, 2.47it/s]
65%|██████████████████████████▋ | 163/250 [01:05<00:39, 2.22it/s]
66%|██████████████████████████▉ | 164/250 [01:05<00:41, 2.08it/s]
66%|███████████████████████████ | 165/250 [01:06<00:47, 1.79it/s]
66%|███████████████████████████▏ | 166/250 [01:07<00:41, 2.03it/s]
67%|███████████████████████████▍ | 167/250 [01:07<00:39, 2.09it/s]
67%|███████████████████████████▌ | 168/250 [01:07<00:35, 2.31it/s]
68%|███████████████████████████▋ | 169/250 [01:08<00:33, 2.42it/s]
68%|███████████████████████████▉ | 170/250 [01:08<00:32, 2.43it/s]
68%|████████████████████████████ | 171/250 [01:09<00:36, 2.16it/s]
69%|████████████████████████████▏ | 172/250 [01:09<00:34, 2.28it/s]
69%|████████████████████████████▎ | 173/250 [01:09<00:31, 2.47it/s]
70%|████████████████████████████▌ | 174/250 [01:10<00:35, 2.16it/s]
70%|████████████████████████████▋ | 175/250 [01:10<00:32, 2.34it/s]
70%|████████████████████████████▊ | 176/250 [01:11<00:29, 2.51it/s]
71%|█████████████████████████████ | 177/250 [01:11<00:29, 2.45it/s]
71%|█████████████████████████████▏ | 178/250 [01:11<00:27, 2.60it/s]
72%|█████████████████████████████▎ | 179/250 [01:12<00:29, 2.44it/s]
72%|█████████████████████████████▌ | 180/250 [01:12<00:26, 2.69it/s]
72%|█████████████████████████████▋ | 181/250 [01:13<00:27, 2.52it/s]
73%|█████████████████████████████▊ | 182/250 [01:13<00:24, 2.82it/s]
73%|██████████████████████████████ | 183/250 [01:13<00:27, 2.48it/s]
74%|██████████████████████████████▏ | 184/250 [01:14<00:26, 2.51it/s]
74%|██████████████████████████████▎ | 185/250 [01:14<00:30, 2.16it/s]
74%|██████████████████████████████▌ | 186/250 [01:15<00:27, 2.35it/s]
75%|██████████████████████████████▋ | 187/250 [01:15<00:28, 2.22it/s]
75%|██████████████████████████████▊ | 188/250 [01:16<00:26, 2.31it/s]
76%|██████████████████████████████▉ | 189/250 [01:16<00:27, 2.18it/s]
76%|███████████████████████████████▏ | 190/250 [01:16<00:25, 2.33it/s]
76%|███████████████████████████████▎ | 191/250 [01:17<00:23, 2.50it/s]
77%|███████████████████████████████▍ | 192/250 [01:17<00:22, 2.55it/s]
77%|███████████████████████████████▋ | 193/250 [01:18<00:22, 2.50it/s]
78%|███████████████████████████████▊ | 194/250 [01:18<00:22, 2.53it/s]
78%|███████████████████████████████▉ | 195/250 [01:19<00:23, 2.30it/s]
78%|████████████████████████████████▏ | 196/250 [01:19<00:24, 2.24it/s]
79%|████████████████████████████████▎ | 197/250 [01:19<00:23, 2.26it/s]
79%|████████████████████████████████▍ | 198/250 [01:20<00:20, 2.53it/s]
80%|████████████████████████████████▋ | 199/250 [01:20<00:18, 2.74it/s]
80%|████████████████████████████████▊ | 200/250 [01:20<00:18, 2.71it/s]
80%|████████████████████████████████▉ | 201/250 [01:21<00:19, 2.50it/s]
81%|█████████████████████████████████▏ | 202/250 [01:21<00:19, 2.49it/s]
81%|█████████████████████████████████▎ | 203/250 [01:22<00:17, 2.71it/s]
82%|█████████████████████████████████▍ | 204/250 [01:22<00:16, 2.72it/s]
82%|█████████████████████████████████▌ | 205/250 [01:22<00:15, 2.92it/s]
82%|█████████████████████████████████▊ | 206/250 [01:23<00:15, 2.80it/s]
83%|█████████████████████████████████▉ | 207/250 [01:23<00:14, 2.89it/s]
83%|██████████████████████████████████ | 208/250 [01:24<00:17, 2.36it/s]
84%|██████████████████████████████████▎ | 209/250 [01:24<00:16, 2.43it/s]
84%|██████████████████████████████████▍ | 210/250 [01:24<00:15, 2.62it/s]
84%|██████████████████████████████████▌ | 211/250 [01:25<00:16, 2.30it/s]
85%|██████████████████████████████████▊ | 212/250 [01:25<00:15, 2.43it/s]
85%|██████████████████████████████████▉ | 213/250 [01:26<00:14, 2.50it/s]
86%|███████████████████████████████████ | 214/250 [01:26<00:16, 2.17it/s]
86%|███████████████████████████████████▎ | 215/250 [01:27<00:15, 2.28it/s]
86%|███████████████████████████████████▍ | 216/250 [01:27<00:13, 2.47it/s]
87%|███████████████████████████████████▌ | 217/250 [01:27<00:13, 2.51it/s]
87%|███████████████████████████████████▊ | 218/250 [01:28<00:12, 2.63it/s]
88%|███████████████████████████████████▉ | 219/250 [01:28<00:13, 2.31it/s]
88%|████████████████████████████████████ | 220/250 [01:28<00:11, 2.54it/s]
88%|████████████████████████████████████▏ | 221/250 [01:29<00:11, 2.62it/s]
89%|████████████████████████████████████▍ | 222/250 [01:29<00:11, 2.50it/s]
89%|████████████████████████████████████▌ | 223/250 [01:30<00:10, 2.66it/s]
90%|████████████████████████████████████▋ | 224/250 [01:30<00:09, 2.74it/s]
90%|████████████████████████████████████▉ | 225/250 [01:30<00:08, 2.82it/s]
90%|█████████████████████████████████████ | 226/250 [01:31<00:11, 2.13it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:32<00:13, 1.69it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:32<00:11, 1.94it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:32<00:09, 2.21it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:33<00:08, 2.39it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:33<00:07, 2.54it/s]
93%|██████████████████████████████████████ | 232/250 [01:34<00:07, 2.44it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:34<00:06, 2.57it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:35<00:07, 2.11it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:35<00:06, 2.31it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:35<00:05, 2.43it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:36<00:06, 2.17it/s]
95%|███████████████████████████████████████ | 238/250 [01:36<00:04, 2.46it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:37<00:05, 2.17it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:38<00:05, 1.71it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:38<00:04, 1.93it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:38<00:03, 2.18it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:39<00:03, 2.31it/s]
98%|████████████████████████████████████████ | 244/250 [01:39<00:03, 1.96it/s]
98%|████████████████████████████████████████▏| 245/250 [01:40<00:02, 2.23it/s]
98%|████████████████████████████████████████▎| 246/250 [01:40<00:01, 2.42it/s]
99%|████████████████████████████████████████▌| 247/250 [01:40<00:01, 2.41it/s]
99%|████████████████████████████████████████▋| 248/250 [01:41<00:00, 2.39it/s]
100%|████████████████████████████████████████▊| 249/250 [01:41<00:00, 2.14it/s]
{'eval_loss': 1.0789920091629028, 'eval_runtime': 103.4255, 'eval_samples_per_second': 19.338, 'eval_steps_per_second': 2.417, 'epoch': 1.4}
70%|█████████████████████████▉ | 546/780 [1:17:06<1:47:05, 27.46s/it]
100%|█████████████████████████████████████████| 250/250 [01:42<00:00, 1.78it/s]
{'loss': 1.0717, 'learning_rate': 8.906882591093118e-05, 'epoch': 1.44}
{'loss': 1.0672, 'learning_rate': 8.097165991902833e-05, 'epoch': 1.49}
75%|███████████████████████████▊ | 585/780 [1:34:51<1:30:20, 27.80s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:45, 5.43it/s]
1%|▌ | 3/250 [00:00<01:20, 3.08it/s]
2%|▋ | 4/250 [00:01<01:32, 2.67it/s]
2%|▊ | 5/250 [00:01<01:33, 2.62it/s]
2%|█ | 6/250 [00:02<01:54, 2.14it/s]
3%|█▏ | 7/250 [00:02<01:45, 2.31it/s]
3%|█▍ | 8/250 [00:03<01:38, 2.45it/s]
4%|█▌ | 9/250 [00:03<01:33, 2.58it/s]
4%|█▋ | 10/250 [00:03<01:27, 2.74it/s]
4%|█▊ | 11/250 [00:04<01:27, 2.74it/s]
5%|██ | 12/250 [00:04<01:32, 2.57it/s]
5%|██▏ | 13/250 [00:04<01:29, 2.64it/s]
6%|██▎ | 14/250 [00:05<01:30, 2.61it/s]
6%|██▌ | 15/250 [00:05<01:44, 2.25it/s]
6%|██▋ | 16/250 [00:06<01:42, 2.28it/s]
7%|██▊ | 17/250 [00:06<01:43, 2.25it/s]
7%|███ | 18/250 [00:07<01:37, 2.38it/s]
8%|███▏ | 19/250 [00:07<01:42, 2.25it/s]
8%|███▎ | 20/250 [00:08<01:39, 2.32it/s]
8%|███▌ | 21/250 [00:08<02:00, 1.90it/s]
9%|███▋ | 22/250 [00:09<01:47, 2.12it/s]
9%|███▊ | 23/250 [00:09<01:52, 2.02it/s]
10%|████ | 24/250 [00:10<01:40, 2.25it/s]
10%|████▏ | 25/250 [00:10<01:37, 2.31it/s]
10%|████▎ | 26/250 [00:10<01:43, 2.16it/s]
11%|████▌ | 27/250 [00:11<01:35, 2.34it/s]
11%|████▋ | 28/250 [00:11<01:31, 2.42it/s]
12%|████▊ | 29/250 [00:11<01:21, 2.73it/s]
12%|█████ | 30/250 [00:12<01:53, 1.93it/s]
12%|█████▏ | 31/250 [00:13<01:41, 2.16it/s]
13%|█████▍ | 32/250 [00:13<01:48, 2.00it/s]
13%|█████▌ | 33/250 [00:14<01:41, 2.13it/s]
14%|█████▋ | 34/250 [00:14<01:39, 2.16it/s]
14%|█████▉ | 35/250 [00:14<01:35, 2.25it/s]
14%|██████ | 36/250 [00:15<01:27, 2.45it/s]
15%|██████▏ | 37/250 [00:15<01:34, 2.25it/s]
15%|██████▍ | 38/250 [00:16<01:25, 2.49it/s]
16%|██████▌ | 39/250 [00:16<01:17, 2.71it/s]
16%|██████▋ | 40/250 [00:16<01:12, 2.91it/s]
16%|██████▉ | 41/250 [00:17<01:09, 2.99it/s]
17%|███████ | 42/250 [00:17<01:12, 2.88it/s]
17%|███████▏ | 43/250 [00:17<01:10, 2.95it/s]
18%|███████▍ | 44/250 [00:18<01:10, 2.91it/s]
18%|███████▌ | 45/250 [00:18<01:22, 2.48it/s]
18%|███████▋ | 46/250 [00:19<01:25, 2.39it/s]
19%|███████▉ | 47/250 [00:19<01:17, 2.63it/s]
19%|████████ | 48/250 [00:19<01:19, 2.54it/s]
20%|████████▏ | 49/250 [00:20<01:12, 2.79it/s]
20%|████████▍ | 50/250 [00:20<01:10, 2.82it/s]
20%|████████▌ | 51/250 [00:20<01:09, 2.85it/s]
21%|████████▋ | 52/250 [00:21<01:08, 2.89it/s]
21%|████████▉ | 53/250 [00:21<01:05, 3.03it/s]
22%|█████████ | 54/250 [00:21<01:10, 2.79it/s]
22%|█████████▏ | 55/250 [00:22<01:09, 2.81it/s]
22%|█████████▍ | 56/250 [00:22<01:11, 2.70it/s]
23%|█████████▌ | 57/250 [00:22<01:08, 2.80it/s]
23%|█████████▋ | 58/250 [00:23<01:22, 2.33it/s]
24%|█████████▉ | 59/250 [00:23<01:23, 2.29it/s]
24%|██████████ | 60/250 [00:24<01:20, 2.35it/s]
24%|██████████▏ | 61/250 [00:25<01:47, 1.76it/s]
25%|██████████▍ | 62/250 [00:25<01:35, 1.97it/s]
25%|██████████▌ | 63/250 [00:26<01:31, 2.04it/s]
26%|██████████▊ | 64/250 [00:26<01:24, 2.19it/s]
26%|██████████▉ | 65/250 [00:26<01:17, 2.37it/s]
26%|███████████ | 66/250 [00:27<01:13, 2.50it/s]
27%|███████████▎ | 67/250 [00:27<01:14, 2.46it/s]
27%|███████████▍ | 68/250 [00:27<01:10, 2.57it/s]
28%|███████████▌ | 69/250 [00:28<01:08, 2.64it/s]
28%|███████████▊ | 70/250 [00:28<01:04, 2.79it/s]
28%|███████████▉ | 71/250 [00:28<01:05, 2.73it/s]
29%|████████████ | 72/250 [00:29<01:03, 2.79it/s]
29%|████████████▎ | 73/250 [00:29<01:05, 2.68it/s]
30%|████████████▍ | 74/250 [00:29<01:01, 2.87it/s]
30%|████████████▌ | 75/250 [00:30<00:59, 2.94it/s]
30%|████████████▊ | 76/250 [00:30<00:58, 2.98it/s]
31%|████████████▉ | 77/250 [00:30<00:59, 2.93it/s]
31%|█████████████ | 78/250 [00:31<00:55, 3.07it/s]
32%|█████████████▎ | 79/250 [00:31<00:59, 2.86it/s]
32%|█████████████▍ | 80/250 [00:32<01:00, 2.80it/s]
32%|█████████████▌ | 81/250 [00:32<00:57, 2.95it/s]
33%|█████████████▊ | 82/250 [00:32<00:51, 3.25it/s]
33%|█████████████▉ | 83/250 [00:32<00:55, 3.01it/s]
34%|██████████████ | 84/250 [00:33<01:04, 2.57it/s]
34%|██████████████▎ | 85/250 [00:33<01:03, 2.58it/s]
34%|██████████████▍ | 86/250 [00:34<01:04, 2.56it/s]
35%|██████████████▌ | 87/250 [00:34<01:13, 2.22it/s]
35%|██████████████▊ | 88/250 [00:35<01:04, 2.50it/s]
36%|██████████████▉ | 89/250 [00:35<01:01, 2.61it/s]
36%|███████████████ | 90/250 [00:35<01:01, 2.59it/s]
36%|███████████████▎ | 91/250 [00:36<00:57, 2.75it/s]
37%|███████████████▍ | 92/250 [00:36<00:56, 2.81it/s]
37%|███████████████▌ | 93/250 [00:37<01:14, 2.12it/s]
38%|███████████████▊ | 94/250 [00:37<01:07, 2.30it/s]
38%|███████████████▉ | 95/250 [00:37<01:04, 2.39it/s]
38%|████████████████▏ | 96/250 [00:38<01:01, 2.52it/s]
39%|████████████████▎ | 97/250 [00:38<00:54, 2.78it/s]
39%|████████████████▍ | 98/250 [00:39<01:01, 2.48it/s]
40%|████████████████▋ | 99/250 [00:39<00:58, 2.58it/s]
40%|████████████████▍ | 100/250 [00:39<00:54, 2.77it/s]
40%|████████████████▌ | 101/250 [00:40<00:52, 2.85it/s]
41%|████████████████▋ | 102/250 [00:40<00:49, 3.02it/s]
41%|████████████████▉ | 103/250 [00:41<01:12, 2.03it/s]
42%|█████████████████ | 104/250 [00:41<01:10, 2.06it/s]
42%|█████████████████▏ | 105/250 [00:42<01:04, 2.25it/s]
42%|█████████████████▍ | 106/250 [00:42<00:59, 2.40it/s]
43%|█████████████████▌ | 107/250 [00:42<00:58, 2.44it/s]
43%|█████████████████▋ | 108/250 [00:43<01:00, 2.33it/s]
44%|█████████████████▉ | 109/250 [00:43<00:54, 2.61it/s]
44%|██████████████████ | 110/250 [00:43<00:50, 2.77it/s]
44%|██████████████████▏ | 111/250 [00:44<00:52, 2.64it/s]
45%|██████████████████▎ | 112/250 [00:44<00:53, 2.60it/s]
45%|██████████████████▌ | 113/250 [00:44<00:46, 2.92it/s]
46%|██████████████████▋ | 114/250 [00:45<00:45, 2.97it/s]
46%|██████████████████▊ | 115/250 [00:45<00:54, 2.49it/s]
46%|███████████████████ | 116/250 [00:46<00:49, 2.69it/s]
47%|███████████████████▏ | 117/250 [00:46<00:54, 2.44it/s]
47%|███████████████████▎ | 118/250 [00:47<01:12, 1.82it/s]
48%|███████████████████▌ | 119/250 [00:47<01:06, 1.97it/s]
48%|███████████████████▋ | 120/250 [00:48<00:57, 2.26it/s]
48%|███████████████████▊ | 121/250 [00:48<00:51, 2.49it/s]
49%|████████████████████ | 122/250 [00:48<00:47, 2.67it/s]
49%|████████████████████▏ | 123/250 [00:49<00:45, 2.81it/s]
50%|████████████████████▎ | 124/250 [00:49<00:43, 2.93it/s]
50%|████████████████████▌ | 125/250 [00:49<00:42, 2.94it/s]
50%|████████████████████▋ | 126/250 [00:50<00:46, 2.68it/s]
51%|████████████████████▊ | 127/250 [00:50<00:43, 2.81it/s]
51%|████████████████████▉ | 128/250 [00:50<00:46, 2.65it/s]
52%|█████████████████████▏ | 129/250 [00:51<00:50, 2.37it/s]
52%|█████████████████████▎ | 130/250 [00:51<00:48, 2.50it/s]
52%|█████████████████████▍ | 131/250 [00:52<00:57, 2.06it/s]
53%|█████████████████████▋ | 132/250 [00:52<00:53, 2.20it/s]
53%|█████████████████████▊ | 133/250 [00:53<00:55, 2.11it/s]
54%|█████████████████████▉ | 134/250 [00:53<00:49, 2.37it/s]
54%|██████████████████████▏ | 135/250 [00:54<00:46, 2.45it/s]
54%|██████████████████████▎ | 136/250 [00:54<00:44, 2.57it/s]
55%|██████████████████████▍ | 137/250 [00:54<00:44, 2.56it/s]
55%|██████████████████████▋ | 138/250 [00:55<00:43, 2.56it/s]
56%|██████████████████████▊ | 139/250 [00:55<00:41, 2.69it/s]
56%|██████████████████████▉ | 140/250 [00:55<00:40, 2.71it/s]
56%|███████████████████████ | 141/250 [00:56<00:41, 2.63it/s]
57%|███████████████████████▎ | 142/250 [00:56<00:43, 2.48it/s]
57%|███████████████████████▍ | 143/250 [00:57<00:39, 2.70it/s]
58%|███████████████████████▌ | 144/250 [00:57<00:47, 2.23it/s]
58%|███████████████████████▊ | 145/250 [00:58<00:44, 2.36it/s]
58%|███████████████████████▉ | 146/250 [00:58<00:42, 2.42it/s]
59%|████████████████████████ | 147/250 [00:58<00:40, 2.52it/s]
59%|████████████████████████▎ | 148/250 [00:59<00:42, 2.42it/s]
60%|████████████████████████▍ | 149/250 [00:59<00:46, 2.16it/s]
60%|████████████████████████▌ | 150/250 [01:00<00:41, 2.39it/s]
60%|████████████████████████▊ | 151/250 [01:00<00:39, 2.52it/s]
61%|████████████████████████▉ | 152/250 [01:00<00:38, 2.55it/s]
61%|█████████████████████████ | 153/250 [01:01<00:38, 2.53it/s]
62%|█████████████████████████▎ | 154/250 [01:01<00:42, 2.27it/s]
62%|█████████████████████████▍ | 155/250 [01:02<00:39, 2.41it/s]
62%|█████████████████████████▌ | 156/250 [01:02<00:37, 2.48it/s]
63%|█████████████████████████▋ | 157/250 [01:02<00:37, 2.51it/s]
63%|█████████████████████████▉ | 158/250 [01:03<00:33, 2.75it/s]
64%|██████████████████████████ | 159/250 [01:03<00:32, 2.81it/s]
64%|██████████████████████████▏ | 160/250 [01:03<00:31, 2.87it/s]
64%|██████████████████████████▍ | 161/250 [01:04<00:30, 2.89it/s]
65%|██████████████████████████▌ | 162/250 [01:04<00:35, 2.47it/s]
65%|██████████████████████████▋ | 163/250 [01:05<00:39, 2.23it/s]
66%|██████████████████████████▉ | 164/250 [01:05<00:41, 2.08it/s]
66%|███████████████████████████ | 165/250 [01:06<00:47, 1.79it/s]
66%|███████████████████████████▏ | 166/250 [01:06<00:41, 2.03it/s]
67%|███████████████████████████▍ | 167/250 [01:07<00:39, 2.09it/s]
67%|███████████████████████████▌ | 168/250 [01:07<00:35, 2.31it/s]
68%|███████████████████████████▋ | 169/250 [01:08<00:33, 2.41it/s]
68%|███████████████████████████▉ | 170/250 [01:08<00:32, 2.43it/s]
68%|████████████████████████████ | 171/250 [01:09<00:36, 2.16it/s]
69%|████████████████████████████▏ | 172/250 [01:09<00:34, 2.28it/s]
69%|████████████████████████████▎ | 173/250 [01:09<00:31, 2.47it/s]
70%|████████████████████████████▌ | 174/250 [01:10<00:35, 2.16it/s]
70%|████████████████████████████▋ | 175/250 [01:10<00:31, 2.35it/s]
70%|████████████████████████████▊ | 176/250 [01:11<00:29, 2.51it/s]
71%|█████████████████████████████ | 177/250 [01:11<00:29, 2.45it/s]
71%|█████████████████████████████▏ | 178/250 [01:11<00:27, 2.60it/s]
72%|█████████████████████████████▎ | 179/250 [01:12<00:29, 2.44it/s]
72%|█████████████████████████████▌ | 180/250 [01:12<00:25, 2.69it/s]
72%|█████████████████████████████▋ | 181/250 [01:13<00:27, 2.53it/s]
73%|█████████████████████████████▊ | 182/250 [01:13<00:24, 2.82it/s]
73%|██████████████████████████████ | 183/250 [01:13<00:27, 2.48it/s]
74%|██████████████████████████████▏ | 184/250 [01:14<00:26, 2.51it/s]
74%|██████████████████████████████▎ | 185/250 [01:14<00:30, 2.10it/s]
74%|██████████████████████████████▌ | 186/250 [01:15<00:27, 2.30it/s]
75%|██████████████████████████████▋ | 187/250 [01:15<00:28, 2.19it/s]
75%|██████████████████████████████▊ | 188/250 [01:16<00:27, 2.29it/s]
76%|██████████████████████████████▉ | 189/250 [01:16<00:27, 2.24it/s]
76%|███████████████████████████████▏ | 190/250 [01:16<00:25, 2.37it/s]
76%|███████████████████████████████▎ | 191/250 [01:17<00:23, 2.53it/s]
77%|███████████████████████████████▍ | 192/250 [01:17<00:22, 2.57it/s]
77%|███████████████████████████████▋ | 193/250 [01:18<00:22, 2.51it/s]
78%|███████████████████████████████▊ | 194/250 [01:18<00:22, 2.54it/s]
78%|███████████████████████████████▉ | 195/250 [01:18<00:23, 2.30it/s]
78%|████████████████████████████████▏ | 196/250 [01:19<00:24, 2.24it/s]
79%|████████████████████████████████▎ | 197/250 [01:19<00:23, 2.25it/s]
79%|████████████████████████████████▍ | 198/250 [01:20<00:20, 2.53it/s]
80%|████████████████████████████████▋ | 199/250 [01:20<00:18, 2.74it/s]
80%|████████████████████████████████▊ | 200/250 [01:20<00:18, 2.71it/s]
80%|████████████████████████████████▉ | 201/250 [01:21<00:19, 2.49it/s]
81%|█████████████████████████████████▏ | 202/250 [01:21<00:19, 2.48it/s]
81%|█████████████████████████████████▎ | 203/250 [01:22<00:17, 2.71it/s]
82%|█████████████████████████████████▍ | 204/250 [01:22<00:16, 2.71it/s]
82%|█████████████████████████████████▌ | 205/250 [01:22<00:15, 2.91it/s]
82%|█████████████████████████████████▊ | 206/250 [01:23<00:15, 2.79it/s]
83%|█████████████████████████████████▉ | 207/250 [01:23<00:14, 2.88it/s]
83%|██████████████████████████████████ | 208/250 [01:23<00:17, 2.36it/s]
84%|██████████████████████████████████▎ | 209/250 [01:24<00:16, 2.43it/s]
84%|██████████████████████████████████▍ | 210/250 [01:24<00:15, 2.52it/s]
84%|██████████████████████████████████▌ | 211/250 [01:25<00:17, 2.25it/s]
85%|██████████████████████████████████▊ | 212/250 [01:25<00:15, 2.39it/s]
85%|██████████████████████████████████▉ | 213/250 [01:26<00:15, 2.46it/s]
86%|███████████████████████████████████ | 214/250 [01:26<00:16, 2.16it/s]
86%|███████████████████████████████████▎ | 215/250 [01:27<00:15, 2.27it/s]
86%|███████████████████████████████████▍ | 216/250 [01:27<00:13, 2.46it/s]
87%|███████████████████████████████████▌ | 217/250 [01:27<00:13, 2.50it/s]
87%|███████████████████████████████████▊ | 218/250 [01:28<00:12, 2.61it/s]
88%|███████████████████████████████████▉ | 219/250 [01:28<00:13, 2.30it/s]
88%|████████████████████████████████████ | 220/250 [01:28<00:11, 2.53it/s]
88%|████████████████████████████████████▏ | 221/250 [01:29<00:11, 2.62it/s]
89%|████████████████████████████████████▍ | 222/250 [01:29<00:11, 2.43it/s]
89%|████████████████████████████████████▌ | 223/250 [01:30<00:10, 2.60it/s]
90%|████████████████████████████████████▋ | 224/250 [01:30<00:09, 2.68it/s]
90%|████████████████████████████████████▉ | 225/250 [01:30<00:09, 2.77it/s]
90%|█████████████████████████████████████ | 226/250 [01:31<00:11, 2.11it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:32<00:13, 1.68it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:32<00:11, 1.93it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:33<00:09, 2.20it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:33<00:08, 2.37it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:33<00:07, 2.53it/s]
93%|██████████████████████████████████████ | 232/250 [01:34<00:07, 2.43it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:34<00:06, 2.56it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:35<00:07, 2.11it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:35<00:06, 2.31it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:35<00:05, 2.43it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:36<00:06, 2.16it/s]
95%|███████████████████████████████████████ | 238/250 [01:36<00:04, 2.46it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:37<00:05, 2.17it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:38<00:05, 1.70it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:38<00:04, 1.92it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:38<00:03, 2.18it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:39<00:03, 2.30it/s]
98%|████████████████████████████████████████ | 244/250 [01:39<00:03, 1.96it/s]
98%|████████████████████████████████████████▏| 245/250 [01:40<00:02, 2.23it/s]
98%|████████████████████████████████████████▎| 246/250 [01:40<00:01, 2.42it/s]
99%|████████████████████████████████████████▌| 247/250 [01:40<00:01, 2.40it/s]
99%|████████████████████████████████████████▋| 248/250 [01:41<00:00, 2.39it/s]
100%|████████████████████████████████████████▊| 249/250 [01:41<00:00, 2.14it/s]
{'eval_loss': 1.078658938407898, 'eval_runtime': 103.5095, 'eval_samples_per_second': 19.322, 'eval_steps_per_second': 2.415, 'epoch': 1.5}
75%|███████████████████████████▊ | 585/780 [1:36:35<1:30:20, 27.80s/it]
100%|█████████████████████████████████████████| 250/250 [01:42<00:00, 1.78it/s]
{'loss': 1.0696, 'learning_rate': 7.28744939271255e-05, 'epoch': 1.54}
{'loss': 1.0422, 'learning_rate': 6.477732793522267e-05, 'epoch': 1.59}
80%|█████████████████████████████▌ | 624/780 [1:54:13<1:09:24, 26.69s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:45, 5.44it/s]
1%|▌ | 3/250 [00:00<01:19, 3.09it/s]
2%|▋ | 4/250 [00:01<01:31, 2.69it/s]
2%|▊ | 5/250 [00:01<01:32, 2.64it/s]
2%|█ | 6/250 [00:02<01:53, 2.15it/s]
3%|█▏ | 7/250 [00:02<01:44, 2.32it/s]
3%|█▍ | 8/250 [00:03<01:38, 2.46it/s]
4%|█▌ | 9/250 [00:03<01:33, 2.59it/s]
4%|█▋ | 10/250 [00:03<01:27, 2.75it/s]
4%|█▊ | 11/250 [00:04<01:26, 2.75it/s]
5%|██ | 12/250 [00:04<01:32, 2.57it/s]
5%|██▏ | 13/250 [00:04<01:29, 2.65it/s]
6%|██▎ | 14/250 [00:05<01:30, 2.62it/s]
6%|██▌ | 15/250 [00:05<01:44, 2.25it/s]
6%|██▋ | 16/250 [00:06<01:42, 2.28it/s]
7%|██▊ | 17/250 [00:06<01:43, 2.25it/s]
7%|███ | 18/250 [00:07<01:37, 2.38it/s]
8%|███▏ | 19/250 [00:07<01:42, 2.26it/s]
8%|███▎ | 20/250 [00:08<01:39, 2.32it/s]
8%|███▌ | 21/250 [00:08<02:00, 1.91it/s]
9%|███▋ | 22/250 [00:09<01:47, 2.12it/s]
9%|███▊ | 23/250 [00:09<01:52, 2.02it/s]
10%|████ | 24/250 [00:09<01:40, 2.25it/s]
10%|████▏ | 25/250 [00:10<01:37, 2.32it/s]
10%|████▎ | 26/250 [00:10<01:43, 2.17it/s]
11%|████▌ | 27/250 [00:11<01:34, 2.35it/s]
11%|████▋ | 28/250 [00:11<01:31, 2.43it/s]
12%|████▊ | 29/250 [00:11<01:20, 2.74it/s]
12%|█████ | 30/250 [00:12<01:53, 1.94it/s]
12%|█████▏ | 31/250 [00:13<01:40, 2.17it/s]
13%|█████▍ | 32/250 [00:13<01:48, 2.01it/s]
13%|█████▌ | 33/250 [00:14<01:41, 2.14it/s]
14%|█████▋ | 34/250 [00:14<01:39, 2.17it/s]
14%|█████▉ | 35/250 [00:14<01:35, 2.26it/s]
14%|██████ | 36/250 [00:15<01:27, 2.45it/s]
15%|██████▏ | 37/250 [00:15<01:34, 2.25it/s]
15%|██████▍ | 38/250 [00:16<01:24, 2.50it/s]
16%|██████▌ | 39/250 [00:16<01:17, 2.71it/s]
16%|██████▋ | 40/250 [00:16<01:12, 2.91it/s]
16%|██████▉ | 41/250 [00:16<01:09, 2.99it/s]
17%|███████ | 42/250 [00:17<01:12, 2.88it/s]
17%|███████▏ | 43/250 [00:17<01:10, 2.95it/s]
18%|███████▍ | 44/250 [00:18<01:10, 2.91it/s]
18%|███████▌ | 45/250 [00:18<01:22, 2.48it/s]
18%|███████▋ | 46/250 [00:19<01:25, 2.40it/s]
19%|███████▉ | 47/250 [00:19<01:17, 2.63it/s]
19%|████████ | 48/250 [00:19<01:19, 2.54it/s]
20%|████████▏ | 49/250 [00:20<01:12, 2.79it/s]
20%|████████▍ | 50/250 [00:20<01:11, 2.80it/s]
20%|████████▌ | 51/250 [00:20<01:10, 2.84it/s]
21%|████████▋ | 52/250 [00:21<01:08, 2.88it/s]
21%|████████▉ | 53/250 [00:21<01:05, 3.03it/s]
22%|█████████ | 54/250 [00:21<01:10, 2.79it/s]
22%|█████████▏ | 55/250 [00:22<01:09, 2.81it/s]
22%|█████████▍ | 56/250 [00:22<01:11, 2.70it/s]
23%|█████████▌ | 57/250 [00:22<01:08, 2.80it/s]
23%|█████████▋ | 58/250 [00:23<01:22, 2.33it/s]
24%|█████████▉ | 59/250 [00:23<01:23, 2.29it/s]
24%|██████████ | 60/250 [00:24<01:20, 2.36it/s]
24%|██████████▏ | 61/250 [00:25<01:45, 1.79it/s]
25%|██████████▍ | 62/250 [00:25<01:34, 2.00it/s]
25%|██████████▌ | 63/250 [00:25<01:30, 2.06it/s]
26%|██████████▊ | 64/250 [00:26<01:24, 2.21it/s]
26%|██████████▉ | 65/250 [00:26<01:17, 2.39it/s]
26%|███████████ | 66/250 [00:27<01:13, 2.51it/s]
27%|███████████▎ | 67/250 [00:27<01:14, 2.47it/s]
27%|███████████▍ | 68/250 [00:27<01:10, 2.58it/s]
28%|███████████▌ | 69/250 [00:28<01:08, 2.65it/s]
28%|███████████▊ | 70/250 [00:28<01:04, 2.79it/s]
28%|███████████▉ | 71/250 [00:28<01:05, 2.73it/s]
29%|████████████ | 72/250 [00:29<01:03, 2.79it/s]
29%|████████████▎ | 73/250 [00:29<01:05, 2.68it/s]
30%|████████████▍ | 74/250 [00:29<01:01, 2.87it/s]
30%|████████████▌ | 75/250 [00:30<00:59, 2.94it/s]
30%|████████████▊ | 76/250 [00:30<00:58, 2.96it/s]
31%|████████████▉ | 77/250 [00:30<00:59, 2.91it/s]
31%|█████████████ | 78/250 [00:31<00:55, 3.07it/s]
32%|█████████████▎ | 79/250 [00:31<00:59, 2.86it/s]
32%|█████████████▍ | 80/250 [00:31<01:00, 2.80it/s]
32%|█████████████▌ | 81/250 [00:32<00:57, 2.94it/s]
33%|█████████████▊ | 82/250 [00:32<00:51, 3.23it/s]
33%|█████████████▉ | 83/250 [00:32<00:55, 3.00it/s]
34%|██████████████ | 84/250 [00:33<01:04, 2.56it/s]
34%|██████████████▎ | 85/250 [00:33<01:04, 2.58it/s]
34%|██████████████▍ | 86/250 [00:34<01:04, 2.55it/s]
35%|██████████████▌ | 87/250 [00:34<01:13, 2.22it/s]
35%|██████████████▊ | 88/250 [00:35<01:04, 2.49it/s]
36%|██████████████▉ | 89/250 [00:35<01:01, 2.60it/s]
36%|███████████████ | 90/250 [00:35<01:02, 2.58it/s]
36%|███████████████▎ | 91/250 [00:36<00:58, 2.74it/s]
37%|███████████████▍ | 92/250 [00:36<00:56, 2.80it/s]
37%|███████████████▌ | 93/250 [00:37<01:14, 2.11it/s]
38%|███████████████▊ | 94/250 [00:37<01:07, 2.30it/s]
38%|███████████████▉ | 95/250 [00:37<01:04, 2.39it/s]
38%|████████████████▏ | 96/250 [00:38<01:01, 2.52it/s]
39%|████████████████▎ | 97/250 [00:38<00:55, 2.78it/s]
39%|████████████████▍ | 98/250 [00:39<01:01, 2.48it/s]
40%|████████████████▋ | 99/250 [00:39<00:58, 2.57it/s]
40%|████████████████▍ | 100/250 [00:39<00:54, 2.77it/s]
40%|████████████████▌ | 101/250 [00:40<00:52, 2.85it/s]
41%|████████████████▋ | 102/250 [00:40<00:49, 3.01it/s]
41%|████████████████▉ | 103/250 [00:41<01:12, 2.03it/s]
42%|█████████████████ | 104/250 [00:41<01:11, 2.05it/s]
42%|█████████████████▏ | 105/250 [00:42<01:04, 2.25it/s]
42%|█████████████████▍ | 106/250 [00:42<00:59, 2.40it/s]
43%|█████████████████▌ | 107/250 [00:42<00:58, 2.44it/s]
43%|█████████████████▋ | 108/250 [00:43<01:00, 2.33it/s]
44%|█████████████████▉ | 109/250 [00:43<00:54, 2.61it/s]
44%|██████████████████ | 110/250 [00:43<00:50, 2.78it/s]
44%|██████████████████▏ | 111/250 [00:44<00:52, 2.64it/s]
45%|██████████████████▎ | 112/250 [00:44<00:53, 2.60it/s]
45%|██████████████████▌ | 113/250 [00:44<00:46, 2.92it/s]
46%|██████████████████▋ | 114/250 [00:45<00:45, 2.96it/s]
46%|██████████████████▊ | 115/250 [00:45<00:54, 2.49it/s]
46%|███████████████████ | 116/250 [00:46<00:49, 2.68it/s]
47%|███████████████████▏ | 117/250 [00:46<00:54, 2.44it/s]
47%|███████████████████▎ | 118/250 [00:47<01:12, 1.82it/s]
48%|███████████████████▌ | 119/250 [00:47<01:06, 1.97it/s]
48%|███████████████████▋ | 120/250 [00:48<00:57, 2.27it/s]
48%|███████████████████▊ | 121/250 [00:48<00:51, 2.48it/s]
49%|████████████████████ | 122/250 [00:48<00:47, 2.67it/s]
49%|████████████████████▏ | 123/250 [00:49<00:45, 2.81it/s]
50%|████████████████████▎ | 124/250 [00:49<00:43, 2.93it/s]
50%|████████████████████▌ | 125/250 [00:49<00:42, 2.94it/s]
50%|████████████████████▋ | 126/250 [00:50<00:46, 2.68it/s]
51%|████████████████████▊ | 127/250 [00:50<00:43, 2.80it/s]
51%|████████████████████▉ | 128/250 [00:50<00:46, 2.64it/s]
52%|█████████████████████▏ | 129/250 [00:51<00:50, 2.37it/s]
52%|█████████████████████▎ | 130/250 [00:51<00:48, 2.50it/s]
52%|█████████████████████▍ | 131/250 [00:52<00:57, 2.06it/s]
53%|█████████████████████▋ | 132/250 [00:52<00:53, 2.20it/s]
53%|█████████████████████▊ | 133/250 [00:53<00:55, 2.11it/s]
54%|█████████████████████▉ | 134/250 [00:53<00:49, 2.36it/s]
54%|██████████████████████▏ | 135/250 [00:54<00:47, 2.44it/s]
54%|██████████████████████▎ | 136/250 [00:54<00:44, 2.57it/s]
55%|██████████████████████▍ | 137/250 [00:54<00:44, 2.55it/s]
55%|██████████████████████▋ | 138/250 [00:55<00:43, 2.56it/s]
56%|██████████████████████▊ | 139/250 [00:55<00:41, 2.68it/s]
56%|██████████████████████▉ | 140/250 [00:55<00:40, 2.70it/s]
56%|███████████████████████ | 141/250 [00:56<00:41, 2.62it/s]
57%|███████████████████████▎ | 142/250 [00:56<00:43, 2.47it/s]
57%|███████████████████████▍ | 143/250 [00:57<00:39, 2.68it/s]
58%|███████████████████████▌ | 144/250 [00:57<00:47, 2.21it/s]
58%|███████████████████████▊ | 145/250 [00:58<00:44, 2.34it/s]
58%|███████████████████████▉ | 146/250 [00:58<00:43, 2.41it/s]
59%|████████████████████████ | 147/250 [00:58<00:41, 2.50it/s]
59%|████████████████████████▎ | 148/250 [00:59<00:42, 2.41it/s]
60%|████████████████████████▍ | 149/250 [00:59<00:46, 2.16it/s]
60%|████████████████████████▌ | 150/250 [01:00<00:41, 2.38it/s]
60%|████████████████████████▊ | 151/250 [01:00<00:39, 2.51it/s]
61%|████████████████████████▉ | 152/250 [01:00<00:38, 2.55it/s]
61%|█████████████████████████ | 153/250 [01:01<00:38, 2.53it/s]
62%|█████████████████████████▎ | 154/250 [01:01<00:42, 2.27it/s]
62%|█████████████████████████▍ | 155/250 [01:02<00:39, 2.40it/s]
62%|█████████████████████████▌ | 156/250 [01:02<00:37, 2.48it/s]
63%|█████████████████████████▋ | 157/250 [01:02<00:37, 2.51it/s]
63%|█████████████████████████▉ | 158/250 [01:03<00:33, 2.75it/s]
64%|██████████████████████████ | 159/250 [01:03<00:32, 2.81it/s]
64%|██████████████████████████▏ | 160/250 [01:03<00:31, 2.87it/s]
64%|██████████████████████████▍ | 161/250 [01:04<00:30, 2.88it/s]
65%|██████████████████████████▌ | 162/250 [01:04<00:35, 2.46it/s]
65%|██████████████████████████▋ | 163/250 [01:05<00:39, 2.22it/s]
66%|██████████████████████████▉ | 164/250 [01:05<00:41, 2.08it/s]
66%|███████████████████████████ | 165/250 [01:06<00:47, 1.78it/s]
66%|███████████████████████████▏ | 166/250 [01:06<00:41, 2.02it/s]
67%|███████████████████████████▍ | 167/250 [01:07<00:39, 2.08it/s]
67%|███████████████████████████▌ | 168/250 [01:07<00:35, 2.30it/s]
68%|███████████████████████████▋ | 169/250 [01:08<00:33, 2.41it/s]
68%|███████████████████████████▉ | 170/250 [01:08<00:32, 2.42it/s]
68%|████████████████████████████ | 171/250 [01:09<00:36, 2.15it/s]
69%|████████████████████████████▏ | 172/250 [01:09<00:34, 2.27it/s]
69%|████████████████████████████▎ | 173/250 [01:09<00:31, 2.46it/s]
70%|████████████████████████████▌ | 174/250 [01:10<00:35, 2.16it/s]
70%|████████████████████████████▋ | 175/250 [01:10<00:32, 2.34it/s]
70%|████████████████████████████▊ | 176/250 [01:11<00:29, 2.50it/s]
71%|█████████████████████████████ | 177/250 [01:11<00:29, 2.45it/s]
71%|█████████████████████████████▏ | 178/250 [01:11<00:27, 2.59it/s]
72%|█████████████████████████████▎ | 179/250 [01:12<00:29, 2.44it/s]
72%|█████████████████████████████▌ | 180/250 [01:12<00:26, 2.69it/s]
72%|█████████████████████████████▋ | 181/250 [01:13<00:27, 2.52it/s]
73%|█████████████████████████████▊ | 182/250 [01:13<00:24, 2.81it/s]
73%|██████████████████████████████ | 183/250 [01:13<00:27, 2.47it/s]
74%|██████████████████████████████▏ | 184/250 [01:14<00:26, 2.50it/s]
74%|██████████████████████████████▎ | 185/250 [01:14<00:30, 2.16it/s]
74%|██████████████████████████████▌ | 186/250 [01:15<00:27, 2.34it/s]
75%|██████████████████████████████▋ | 187/250 [01:15<00:28, 2.22it/s]
75%|██████████████████████████████▊ | 188/250 [01:16<00:26, 2.30it/s]
76%|██████████████████████████████▉ | 189/250 [01:16<00:27, 2.24it/s]
76%|███████████████████████████████▏ | 190/250 [01:16<00:25, 2.37it/s]
76%|███████████████████████████████▎ | 191/250 [01:17<00:23, 2.53it/s]
77%|███████████████████████████████▍ | 192/250 [01:17<00:22, 2.57it/s]
77%|███████████████████████████████▋ | 193/250 [01:18<00:22, 2.51it/s]
78%|███████████████████████████████▊ | 194/250 [01:18<00:22, 2.54it/s]
78%|███████████████████████████████▉ | 195/250 [01:18<00:23, 2.31it/s]
78%|████████████████████████████████▏ | 196/250 [01:19<00:24, 2.24it/s]
79%|████████████████████████████████▎ | 197/250 [01:19<00:23, 2.26it/s]
79%|████████████████████████████████▍ | 198/250 [01:20<00:20, 2.54it/s]
80%|████████████████████████████████▋ | 199/250 [01:20<00:18, 2.75it/s]
80%|████████████████████████████████▊ | 200/250 [01:20<00:18, 2.72it/s]
80%|████████████████████████████████▉ | 201/250 [01:21<00:19, 2.50it/s]
81%|█████████████████████████████████▏ | 202/250 [01:21<00:19, 2.49it/s]
81%|█████████████████████████████████▎ | 203/250 [01:22<00:17, 2.71it/s]
82%|█████████████████████████████████▍ | 204/250 [01:22<00:16, 2.71it/s]
82%|█████████████████████████████████▌ | 205/250 [01:22<00:15, 2.92it/s]
82%|█████████████████████████████████▊ | 206/250 [01:23<00:15, 2.79it/s]
83%|█████████████████████████████████▉ | 207/250 [01:23<00:14, 2.89it/s]
83%|██████████████████████████████████ | 208/250 [01:23<00:17, 2.36it/s]
84%|██████████████████████████████████▎ | 209/250 [01:24<00:16, 2.43it/s]
84%|██████████████████████████████████▍ | 210/250 [01:24<00:15, 2.62it/s]
84%|██████████████████████████████████▌ | 211/250 [01:25<00:16, 2.30it/s]
85%|██████████████████████████████████▊ | 212/250 [01:25<00:15, 2.43it/s]
85%|██████████████████████████████████▉ | 213/250 [01:25<00:14, 2.50it/s]
86%|███████████████████████████████████ | 214/250 [01:26<00:16, 2.17it/s]
86%|███████████████████████████████████▎ | 215/250 [01:26<00:15, 2.28it/s]
86%|███████████████████████████████████▍ | 216/250 [01:27<00:13, 2.47it/s]
87%|███████████████████████████████████▌ | 217/250 [01:27<00:13, 2.50it/s]
87%|███████████████████████████████████▊ | 218/250 [01:27<00:12, 2.62it/s]
88%|███████████████████████████████████▉ | 219/250 [01:28<00:13, 2.31it/s]
88%|████████████████████████████████████ | 220/250 [01:28<00:11, 2.54it/s]
88%|████████████████████████████████████▏ | 221/250 [01:29<00:11, 2.62it/s]
89%|████████████████████████████████████▍ | 222/250 [01:29<00:11, 2.49it/s]
89%|████████████████████████████████████▌ | 223/250 [01:29<00:10, 2.66it/s]
90%|████████████████████████████████████▋ | 224/250 [01:30<00:09, 2.74it/s]
90%|████████████████████████████████████▉ | 225/250 [01:30<00:08, 2.81it/s]
90%|█████████████████████████████████████ | 226/250 [01:31<00:11, 2.13it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:32<00:13, 1.69it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:32<00:11, 1.94it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:32<00:09, 2.21it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:33<00:08, 2.39it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:33<00:07, 2.54it/s]
93%|██████████████████████████████████████ | 232/250 [01:34<00:07, 2.44it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:34<00:06, 2.57it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:35<00:07, 2.11it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:35<00:06, 2.31it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:35<00:05, 2.44it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:36<00:05, 2.17it/s]
95%|███████████████████████████████████████ | 238/250 [01:36<00:04, 2.46it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:37<00:05, 2.17it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:38<00:05, 1.71it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:38<00:04, 1.92it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:38<00:03, 2.18it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:39<00:03, 2.31it/s]
98%|████████████████████████████████████████ | 244/250 [01:39<00:03, 1.97it/s]
98%|████████████████████████████████████████▏| 245/250 [01:40<00:02, 2.23it/s]
98%|████████████████████████████████████████▎| 246/250 [01:40<00:01, 2.42it/s]
99%|████████████████████████████████████████▌| 247/250 [01:40<00:01, 2.40it/s]
99%|████████████████████████████████████████▋| 248/250 [01:41<00:00, 2.39it/s]
100%|████████████████████████████████████████▊| 249/250 [01:41<00:00, 2.14it/s]
{'eval_loss': 1.0774587392807007, 'eval_runtime': 103.3731, 'eval_samples_per_second': 19.347, 'eval_steps_per_second': 2.418, 'epoch': 1.6}
80%|█████████████████████████████▌ | 624/780 [1:55:57<1:09:24, 26.69s/it]
100%|█████████████████████████████████████████| 250/250 [01:42<00:00, 1.78it/s]
{'loss': 1.0445, 'learning_rate': 5.668016194331984e-05, 'epoch': 1.64}
{'loss': 1.0681, 'learning_rate': 4.8582995951417004e-05, 'epoch': 1.69}
85%|█████████████████████████████████▏ | 663/780 [2:13:39<52:02, 26.68s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:45, 5.45it/s]
1%|▌ | 3/250 [00:00<01:19, 3.09it/s]
2%|▋ | 4/250 [00:01<01:31, 2.69it/s]
2%|▊ | 5/250 [00:01<01:32, 2.64it/s]
2%|█ | 6/250 [00:02<01:53, 2.14it/s]
3%|█▏ | 7/250 [00:02<01:44, 2.32it/s]
3%|█▍ | 8/250 [00:03<01:38, 2.45it/s]
4%|█▌ | 9/250 [00:03<01:33, 2.59it/s]
4%|█▋ | 10/250 [00:03<01:27, 2.74it/s]
4%|█▊ | 11/250 [00:04<01:26, 2.75it/s]
5%|██ | 12/250 [00:04<01:32, 2.57it/s]
5%|██▏ | 13/250 [00:04<01:29, 2.65it/s]
6%|██▎ | 14/250 [00:05<01:30, 2.62it/s]
6%|██▌ | 15/250 [00:05<01:44, 2.25it/s]
6%|██▋ | 16/250 [00:06<01:42, 2.28it/s]
7%|██▊ | 17/250 [00:06<01:43, 2.25it/s]
7%|███ | 18/250 [00:07<01:37, 2.38it/s]
8%|███▏ | 19/250 [00:07<01:42, 2.26it/s]
8%|███▎ | 20/250 [00:08<01:38, 2.32it/s]
8%|███▌ | 21/250 [00:08<02:00, 1.91it/s]
9%|███▋ | 22/250 [00:09<01:47, 2.12it/s]
9%|███▊ | 23/250 [00:09<01:52, 2.02it/s]
10%|████ | 24/250 [00:09<01:40, 2.25it/s]
10%|████▏ | 25/250 [00:10<01:37, 2.31it/s]
10%|████▎ | 26/250 [00:10<01:43, 2.16it/s]
11%|████▌ | 27/250 [00:11<01:35, 2.34it/s]
11%|████▋ | 28/250 [00:11<01:31, 2.42it/s]
12%|████▊ | 29/250 [00:11<01:20, 2.73it/s]
12%|█████ | 30/250 [00:12<01:53, 1.94it/s]
12%|█████▏ | 31/250 [00:13<01:40, 2.17it/s]
13%|█████▍ | 32/250 [00:13<01:48, 2.01it/s]
13%|█████▌ | 33/250 [00:14<01:41, 2.14it/s]
14%|█████▋ | 34/250 [00:14<01:42, 2.10it/s]
14%|█████▉ | 35/250 [00:14<01:37, 2.21it/s]
14%|██████ | 36/250 [00:15<01:28, 2.41it/s]
15%|██████▏ | 37/250 [00:15<01:35, 2.23it/s]
15%|██████▍ | 38/250 [00:16<01:25, 2.47it/s]
16%|██████▌ | 39/250 [00:16<01:18, 2.69it/s]
16%|██████▋ | 40/250 [00:16<01:12, 2.89it/s]
16%|██████▉ | 41/250 [00:17<01:10, 2.98it/s]
17%|███████ | 42/250 [00:17<01:12, 2.87it/s]
17%|███████▏ | 43/250 [00:17<01:10, 2.95it/s]
18%|███████▍ | 44/250 [00:18<01:10, 2.91it/s]
18%|███████▌ | 45/250 [00:18<01:22, 2.48it/s]
18%|███████▋ | 46/250 [00:19<01:25, 2.40it/s]
19%|███████▉ | 47/250 [00:19<01:17, 2.63it/s]
19%|████████ | 48/250 [00:19<01:19, 2.54it/s]
20%|████████▏ | 49/250 [00:20<01:12, 2.79it/s]
20%|████████▍ | 50/250 [00:20<01:10, 2.82it/s]
20%|████████▌ | 51/250 [00:20<01:09, 2.85it/s]
21%|████████▋ | 52/250 [00:21<01:08, 2.90it/s]
21%|████████▉ | 53/250 [00:21<01:04, 3.04it/s]
22%|█████████ | 54/250 [00:21<01:09, 2.80it/s]
22%|█████████▏ | 55/250 [00:22<01:09, 2.81it/s]
22%|█████████▍ | 56/250 [00:22<01:11, 2.71it/s]
23%|█████████▌ | 57/250 [00:22<01:08, 2.81it/s]
23%|█████████▋ | 58/250 [00:23<01:22, 2.33it/s]
24%|█████████▉ | 59/250 [00:23<01:23, 2.29it/s]
24%|██████████ | 60/250 [00:24<01:20, 2.35it/s]
24%|██████████▏ | 61/250 [00:25<01:45, 1.79it/s]
25%|██████████▍ | 62/250 [00:25<01:34, 2.00it/s]
25%|██████████▌ | 63/250 [00:26<01:30, 2.06it/s]
26%|██████████▊ | 64/250 [00:26<01:24, 2.21it/s]
26%|██████████▉ | 65/250 [00:26<01:17, 2.38it/s]
26%|███████████ | 66/250 [00:27<01:13, 2.50it/s]
27%|███████████▎ | 67/250 [00:27<01:14, 2.46it/s]
27%|███████████▍ | 68/250 [00:27<01:10, 2.57it/s]
28%|███████████▌ | 69/250 [00:28<01:08, 2.64it/s]
28%|███████████▊ | 70/250 [00:28<01:04, 2.78it/s]
28%|███████████▉ | 71/250 [00:28<01:05, 2.72it/s]
29%|████████████ | 72/250 [00:29<01:03, 2.78it/s]
29%|████████████▎ | 73/250 [00:29<01:06, 2.68it/s]
30%|████████████▍ | 74/250 [00:29<01:01, 2.87it/s]
30%|████████████▌ | 75/250 [00:30<00:59, 2.93it/s]
30%|████████████▊ | 76/250 [00:30<00:58, 2.98it/s]
31%|████████████▉ | 77/250 [00:30<00:59, 2.93it/s]
31%|█████████████ | 78/250 [00:31<00:56, 3.04it/s]
32%|█████████████▎ | 79/250 [00:31<01:00, 2.84it/s]
32%|█████████████▍ | 80/250 [00:32<01:01, 2.78it/s]
32%|█████████████▌ | 81/250 [00:32<00:57, 2.93it/s]
33%|█████████████▊ | 82/250 [00:32<00:52, 3.22it/s]
33%|█████████████▉ | 83/250 [00:32<00:55, 2.98it/s]
34%|██████████████ | 84/250 [00:33<01:04, 2.56it/s]
34%|██████████████▎ | 85/250 [00:33<01:04, 2.57it/s]
34%|██████████████▍ | 86/250 [00:34<01:04, 2.55it/s]
35%|██████████████▌ | 87/250 [00:34<01:13, 2.22it/s]
35%|██████████████▊ | 88/250 [00:35<01:04, 2.49it/s]
36%|██████████████▉ | 89/250 [00:35<01:01, 2.60it/s]
36%|███████████████ | 90/250 [00:35<01:01, 2.58it/s]
36%|███████████████▎ | 91/250 [00:36<00:58, 2.74it/s]
37%|███████████████▍ | 92/250 [00:36<00:56, 2.80it/s]
37%|███████████████▌ | 93/250 [00:37<01:14, 2.12it/s]
38%|███████████████▊ | 94/250 [00:37<01:07, 2.30it/s]
38%|███████████████▉ | 95/250 [00:38<01:04, 2.39it/s]
38%|████████████████▏ | 96/250 [00:38<01:01, 2.52it/s]
39%|████████████████▎ | 97/250 [00:38<00:55, 2.78it/s]
39%|████████████████▍ | 98/250 [00:39<01:01, 2.48it/s]
40%|████████████████▋ | 99/250 [00:39<00:58, 2.58it/s]
40%|████████████████▍ | 100/250 [00:39<00:54, 2.77it/s]
40%|████████████████▌ | 101/250 [00:40<00:52, 2.85it/s]
41%|████████████████▋ | 102/250 [00:40<00:48, 3.02it/s]
41%|████████████████▉ | 103/250 [00:41<01:12, 2.03it/s]
42%|█████████████████ | 104/250 [00:41<01:10, 2.06it/s]
42%|█████████████████▏ | 105/250 [00:42<01:04, 2.25it/s]
42%|█████████████████▍ | 106/250 [00:42<00:59, 2.40it/s]
43%|█████████████████▌ | 107/250 [00:42<00:58, 2.44it/s]
43%|█████████████████▋ | 108/250 [00:43<01:00, 2.33it/s]
44%|█████████████████▉ | 109/250 [00:43<00:54, 2.61it/s]
44%|██████████████████ | 110/250 [00:43<00:50, 2.78it/s]
44%|██████████████████▏ | 111/250 [00:44<00:52, 2.64it/s]
45%|██████████████████▎ | 112/250 [00:44<00:53, 2.60it/s]
45%|██████████████████▌ | 113/250 [00:44<00:47, 2.91it/s]
46%|██████████████████▋ | 114/250 [00:45<00:45, 2.97it/s]
46%|██████████████████▊ | 115/250 [00:45<00:54, 2.49it/s]
46%|███████████████████ | 116/250 [00:46<00:49, 2.69it/s]
47%|███████████████████▏ | 117/250 [00:46<00:54, 2.44it/s]
47%|███████████████████▎ | 118/250 [00:47<01:12, 1.82it/s]
48%|███████████████████▌ | 119/250 [00:47<01:06, 1.97it/s]
48%|███████████████████▋ | 120/250 [00:48<00:57, 2.27it/s]
48%|███████████████████▊ | 121/250 [00:48<00:51, 2.49it/s]
49%|████████████████████ | 122/250 [00:48<00:47, 2.67it/s]
49%|████████████████████▏ | 123/250 [00:49<00:45, 2.82it/s]
50%|████████████████████▎ | 124/250 [00:49<00:42, 2.93it/s]
50%|████████████████████▌ | 125/250 [00:49<00:42, 2.95it/s]
50%|████████████████████▋ | 126/250 [00:50<00:46, 2.69it/s]
51%|████████████████████▊ | 127/250 [00:50<00:43, 2.81it/s]
51%|████████████████████▉ | 128/250 [00:50<00:46, 2.65it/s]
52%|█████████████████████▏ | 129/250 [00:51<00:50, 2.37it/s]
52%|█████████████████████▎ | 130/250 [00:51<00:48, 2.50it/s]
52%|█████████████████████▍ | 131/250 [00:52<00:57, 2.06it/s]
53%|█████████████████████▋ | 132/250 [00:52<00:53, 2.20it/s]
53%|█████████████████████▊ | 133/250 [00:53<00:55, 2.11it/s]
54%|█████████████████████▉ | 134/250 [00:53<00:49, 2.37it/s]
54%|██████████████████████▏ | 135/250 [00:54<00:46, 2.45it/s]
54%|██████████████████████▎ | 136/250 [00:54<00:45, 2.48it/s]
55%|██████████████████████▍ | 137/250 [00:54<00:45, 2.49it/s]
55%|██████████████████████▋ | 138/250 [00:55<00:44, 2.52it/s]
56%|██████████████████████▊ | 139/250 [00:55<00:41, 2.65it/s]
56%|██████████████████████▉ | 140/250 [00:55<00:41, 2.68it/s]
56%|███████████████████████ | 141/250 [00:56<00:41, 2.61it/s]
57%|███████████████████████▎ | 142/250 [00:56<00:43, 2.48it/s]
57%|███████████████████████▍ | 143/250 [00:57<00:39, 2.70it/s]
58%|███████████████████████▌ | 144/250 [00:57<00:47, 2.22it/s]
58%|███████████████████████▊ | 145/250 [00:58<00:44, 2.35it/s]
58%|███████████████████████▉ | 146/250 [00:58<00:43, 2.42it/s]
59%|████████████████████████ | 147/250 [00:58<00:40, 2.52it/s]
59%|████████████████████████▎ | 148/250 [00:59<00:42, 2.42it/s]
60%|████████████████████████▍ | 149/250 [00:59<00:46, 2.16it/s]
60%|████████████████████████▌ | 150/250 [01:00<00:41, 2.39it/s]
60%|████████████████████████▊ | 151/250 [01:00<00:39, 2.52it/s]
61%|████████████████████████▉ | 152/250 [01:00<00:38, 2.55it/s]
61%|█████████████████████████ | 153/250 [01:01<00:38, 2.53it/s]
62%|█████████████████████████▎ | 154/250 [01:01<00:42, 2.27it/s]
62%|█████████████████████████▍ | 155/250 [01:02<00:39, 2.41it/s]
62%|█████████████████████████▌ | 156/250 [01:02<00:37, 2.48it/s]
63%|█████████████████████████▋ | 157/250 [01:03<00:37, 2.51it/s]
63%|█████████████████████████▉ | 158/250 [01:03<00:33, 2.74it/s]
64%|██████████████████████████ | 159/250 [01:03<00:32, 2.80it/s]
64%|██████████████████████████▏ | 160/250 [01:03<00:31, 2.87it/s]
64%|██████████████████████████▍ | 161/250 [01:04<00:30, 2.88it/s]
65%|██████████████████████████▌ | 162/250 [01:04<00:35, 2.46it/s]
65%|██████████████████████████▋ | 163/250 [01:05<00:39, 2.22it/s]
66%|██████████████████████████▉ | 164/250 [01:05<00:41, 2.08it/s]
66%|███████████████████████████ | 165/250 [01:06<00:47, 1.79it/s]
66%|███████████████████████████▏ | 166/250 [01:07<00:41, 2.03it/s]
67%|███████████████████████████▍ | 167/250 [01:07<00:39, 2.09it/s]
67%|███████████████████████████▌ | 168/250 [01:07<00:35, 2.31it/s]
68%|███████████████████████████▋ | 169/250 [01:08<00:33, 2.41it/s]
68%|███████████████████████████▉ | 170/250 [01:08<00:32, 2.43it/s]
68%|████████████████████████████ | 171/250 [01:09<00:36, 2.15it/s]
69%|████████████████████████████▏ | 172/250 [01:09<00:34, 2.28it/s]
69%|████████████████████████████▎ | 173/250 [01:09<00:31, 2.47it/s]
70%|████████████████████████████▌ | 174/250 [01:10<00:35, 2.16it/s]
70%|████████████████████████████▋ | 175/250 [01:10<00:31, 2.34it/s]
70%|████████████████████████████▊ | 176/250 [01:11<00:29, 2.51it/s]
71%|█████████████████████████████ | 177/250 [01:11<00:29, 2.45it/s]
71%|█████████████████████████████▏ | 178/250 [01:11<00:27, 2.60it/s]
72%|█████████████████████████████▎ | 179/250 [01:12<00:29, 2.44it/s]
72%|█████████████████████████████▌ | 180/250 [01:12<00:25, 2.69it/s]
72%|█████████████████████████████▋ | 181/250 [01:13<00:27, 2.52it/s]
73%|█████████████████████████████▊ | 182/250 [01:13<00:24, 2.81it/s]
73%|██████████████████████████████ | 183/250 [01:13<00:27, 2.48it/s]
74%|██████████████████████████████▏ | 184/250 [01:14<00:26, 2.51it/s]
74%|██████████████████████████████▎ | 185/250 [01:14<00:30, 2.16it/s]
74%|██████████████████████████████▌ | 186/250 [01:15<00:27, 2.35it/s]
75%|██████████████████████████████▋ | 187/250 [01:15<00:28, 2.22it/s]
75%|██████████████████████████████▊ | 188/250 [01:16<00:26, 2.31it/s]
76%|██████████████████████████████▉ | 189/250 [01:16<00:27, 2.25it/s]
76%|███████████████████████████████▏ | 190/250 [01:16<00:25, 2.38it/s]
76%|███████████████████████████████▎ | 191/250 [01:17<00:23, 2.54it/s]
77%|███████████████████████████████▍ | 192/250 [01:17<00:22, 2.58it/s]
77%|███████████████████████████████▋ | 193/250 [01:18<00:22, 2.51it/s]
78%|███████████████████████████████▊ | 194/250 [01:18<00:22, 2.54it/s]
78%|███████████████████████████████▉ | 195/250 [01:19<00:23, 2.30it/s]
78%|████████████████████████████████▏ | 196/250 [01:19<00:24, 2.24it/s]
79%|████████████████████████████████▎ | 197/250 [01:19<00:23, 2.26it/s]
79%|████████████████████████████████▍ | 198/250 [01:20<00:20, 2.53it/s]
80%|████████████████████████████████▋ | 199/250 [01:20<00:18, 2.74it/s]
80%|████████████████████████████████▊ | 200/250 [01:20<00:18, 2.70it/s]
80%|████████████████████████████████▉ | 201/250 [01:21<00:19, 2.49it/s]
81%|█████████████████████████████████▏ | 202/250 [01:21<00:19, 2.48it/s]
81%|█████████████████████████████████▎ | 203/250 [01:22<00:17, 2.71it/s]
82%|█████████████████████████████████▍ | 204/250 [01:22<00:17, 2.70it/s]
82%|█████████████████████████████████▌ | 205/250 [01:22<00:15, 2.91it/s]
82%|█████████████████████████████████▊ | 206/250 [01:23<00:15, 2.79it/s]
83%|█████████████████████████████████▉ | 207/250 [01:23<00:14, 2.87it/s]
83%|██████████████████████████████████ | 208/250 [01:24<00:17, 2.35it/s]
84%|██████████████████████████████████▎ | 209/250 [01:24<00:16, 2.42it/s]
84%|██████████████████████████████████▍ | 210/250 [01:24<00:15, 2.61it/s]
84%|██████████████████████████████████▌ | 211/250 [01:25<00:16, 2.30it/s]
85%|██████████████████████████████████▊ | 212/250 [01:25<00:15, 2.42it/s]
85%|██████████████████████████████████▉ | 213/250 [01:26<00:14, 2.49it/s]
86%|███████████████████████████████████ | 214/250 [01:26<00:16, 2.17it/s]
86%|███████████████████████████████████▎ | 215/250 [01:27<00:15, 2.28it/s]
86%|███████████████████████████████████▍ | 216/250 [01:27<00:13, 2.47it/s]
87%|███████████████████████████████████▌ | 217/250 [01:27<00:13, 2.50it/s]
87%|███████████████████████████████████▊ | 218/250 [01:28<00:12, 2.62it/s]
88%|███████████████████████████████████▉ | 219/250 [01:28<00:13, 2.31it/s]
88%|████████████████████████████████████ | 220/250 [01:28<00:11, 2.54it/s]
88%|████████████████████████████████████▏ | 221/250 [01:29<00:11, 2.62it/s]
89%|████████████████████████████████████▍ | 222/250 [01:29<00:11, 2.49it/s]
89%|████████████████████████████████████▌ | 223/250 [01:30<00:10, 2.66it/s]
90%|████████████████████████████████████▋ | 224/250 [01:30<00:09, 2.73it/s]
90%|████████████████████████████████████▉ | 225/250 [01:30<00:08, 2.81it/s]
90%|█████████████████████████████████████ | 226/250 [01:31<00:11, 2.13it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:32<00:13, 1.69it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:32<00:11, 1.95it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:32<00:09, 2.21it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:33<00:08, 2.39it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:33<00:07, 2.54it/s]
93%|██████████████████████████████████████ | 232/250 [01:34<00:07, 2.44it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:34<00:06, 2.57it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:35<00:07, 2.11it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:35<00:06, 2.31it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:35<00:05, 2.43it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:36<00:06, 2.16it/s]
95%|███████████████████████████████████████ | 238/250 [01:36<00:04, 2.46it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:37<00:05, 2.17it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:38<00:05, 1.71it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:38<00:04, 1.93it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:38<00:03, 2.18it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:39<00:03, 2.31it/s]
98%|████████████████████████████████████████ | 244/250 [01:39<00:03, 1.96it/s]
98%|████████████████████████████████████████▏| 245/250 [01:40<00:02, 2.23it/s]
98%|████████████████████████████████████████▎| 246/250 [01:40<00:01, 2.42it/s]
99%|████████████████████████████████████████▌| 247/250 [01:40<00:01, 2.41it/s]
99%|████████████████████████████████████████▋| 248/250 [01:41<00:00, 2.39it/s]
100%|████████████████████████████████████████▊| 249/250 [01:41<00:00, 2.14it/s]
{'eval_loss': 1.0764182806015015, 'eval_runtime': 103.4269, 'eval_samples_per_second': 19.337, 'eval_steps_per_second': 2.417, 'epoch': 1.7}
85%|█████████████████████████████████▏ | 663/780 [2:15:23<52:02, 26.68s/it]
100%|█████████████████████████████████████████| 250/250 [01:42<00:00, 1.78it/s]
{'loss': 1.0584, 'learning_rate': 4.048582995951416e-05, 'epoch': 1.74}
{'loss': 1.0641, 'learning_rate': 3.2388663967611336e-05, 'epoch': 1.79}
90%|███████████████████████████████████ | 702/780 [2:32:58<35:02, 26.96s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:45, 5.44it/s]
1%|▌ | 3/250 [00:00<01:20, 3.08it/s]
2%|▋ | 4/250 [00:01<01:31, 2.68it/s]
2%|▊ | 5/250 [00:01<01:32, 2.64it/s]
2%|█ | 6/250 [00:02<01:53, 2.14it/s]
3%|█▏ | 7/250 [00:02<01:44, 2.32it/s]
3%|█▍ | 8/250 [00:03<01:38, 2.45it/s]
4%|█▌ | 9/250 [00:03<01:33, 2.59it/s]
4%|█▋ | 10/250 [00:03<01:27, 2.74it/s]
4%|█▊ | 11/250 [00:04<01:26, 2.75it/s]
5%|██ | 12/250 [00:04<01:32, 2.57it/s]
5%|██▏ | 13/250 [00:04<01:29, 2.64it/s]
6%|██▎ | 14/250 [00:05<01:30, 2.61it/s]
6%|██▌ | 15/250 [00:05<01:44, 2.25it/s]
6%|██▋ | 16/250 [00:06<01:42, 2.28it/s]
7%|██▊ | 17/250 [00:06<01:43, 2.25it/s]
7%|███ | 18/250 [00:07<01:37, 2.38it/s]
8%|███▏ | 19/250 [00:07<01:42, 2.25it/s]
8%|███▎ | 20/250 [00:08<01:39, 2.32it/s]
8%|███▌ | 21/250 [00:08<02:00, 1.91it/s]
9%|███▋ | 22/250 [00:09<01:47, 2.12it/s]
9%|███▊ | 23/250 [00:09<01:52, 2.02it/s]
10%|████ | 24/250 [00:10<01:40, 2.25it/s]
10%|████▏ | 25/250 [00:10<01:37, 2.31it/s]
10%|████▎ | 26/250 [00:10<01:43, 2.16it/s]
11%|████▌ | 27/250 [00:11<01:35, 2.34it/s]
11%|████▋ | 28/250 [00:11<01:31, 2.43it/s]
12%|████▊ | 29/250 [00:11<01:21, 2.73it/s]
12%|█████ | 30/250 [00:12<01:53, 1.94it/s]
12%|█████▏ | 31/250 [00:13<01:41, 2.17it/s]
13%|█████▍ | 32/250 [00:13<01:48, 2.01it/s]
13%|█████▌ | 33/250 [00:14<01:41, 2.13it/s]
14%|█████▋ | 34/250 [00:14<01:39, 2.17it/s]
14%|█████▉ | 35/250 [00:14<01:35, 2.25it/s]
14%|██████ | 36/250 [00:15<01:27, 2.45it/s]
15%|██████▏ | 37/250 [00:15<01:34, 2.25it/s]
15%|██████▍ | 38/250 [00:16<01:25, 2.49it/s]
16%|██████▌ | 39/250 [00:16<01:17, 2.71it/s]
16%|██████▋ | 40/250 [00:16<01:12, 2.90it/s]
16%|██████▉ | 41/250 [00:17<01:10, 2.98it/s]
17%|███████ | 42/250 [00:17<01:12, 2.87it/s]
17%|███████▏ | 43/250 [00:17<01:10, 2.94it/s]
18%|███████▍ | 44/250 [00:18<01:11, 2.90it/s]
18%|███████▌ | 45/250 [00:18<01:23, 2.47it/s]
18%|███████▋ | 46/250 [00:19<01:25, 2.39it/s]
19%|███████▉ | 47/250 [00:19<01:17, 2.62it/s]
19%|████████ | 48/250 [00:19<01:19, 2.53it/s]
20%|████████▏ | 49/250 [00:20<01:12, 2.77it/s]
20%|████████▍ | 50/250 [00:20<01:11, 2.81it/s]
20%|████████▌ | 51/250 [00:20<01:10, 2.84it/s]
21%|████████▋ | 52/250 [00:21<01:08, 2.88it/s]
21%|████████▉ | 53/250 [00:21<01:05, 3.02it/s]
22%|█████████ | 54/250 [00:21<01:10, 2.79it/s]
22%|█████████▏ | 55/250 [00:22<01:09, 2.80it/s]
22%|█████████▍ | 56/250 [00:22<01:11, 2.70it/s]
23%|█████████▌ | 57/250 [00:22<01:08, 2.80it/s]
23%|█████████▋ | 58/250 [00:23<01:22, 2.33it/s]
24%|█████████▉ | 59/250 [00:23<01:23, 2.28it/s]
24%|██████████ | 60/250 [00:24<01:20, 2.35it/s]
24%|██████████▏ | 61/250 [00:25<01:45, 1.79it/s]
25%|██████████▍ | 62/250 [00:25<01:34, 1.99it/s]
25%|██████████▌ | 63/250 [00:26<01:30, 2.06it/s]
26%|██████████▊ | 64/250 [00:26<01:24, 2.20it/s]
26%|██████████▉ | 65/250 [00:26<01:17, 2.38it/s]
26%|███████████ | 66/250 [00:27<01:13, 2.50it/s]
27%|███████████▎ | 67/250 [00:27<01:14, 2.46it/s]
27%|███████████▍ | 68/250 [00:27<01:10, 2.57it/s]
28%|███████████▌ | 69/250 [00:28<01:08, 2.64it/s]
28%|███████████▊ | 70/250 [00:28<01:04, 2.78it/s]
28%|███████████▉ | 71/250 [00:28<01:05, 2.72it/s]
29%|████████████ | 72/250 [00:29<01:03, 2.78it/s]
29%|████████████▎ | 73/250 [00:29<01:05, 2.68it/s]
30%|████████████▍ | 74/250 [00:29<01:01, 2.87it/s]
30%|████████████▌ | 75/250 [00:30<00:59, 2.93it/s]
30%|████████████▊ | 76/250 [00:30<00:58, 2.98it/s]
31%|████████████▉ | 77/250 [00:30<00:59, 2.93it/s]
31%|█████████████ | 78/250 [00:31<00:55, 3.08it/s]
32%|█████████████▎ | 79/250 [00:31<00:59, 2.87it/s]
32%|█████████████▍ | 80/250 [00:32<01:00, 2.80it/s]
32%|█████████████▌ | 81/250 [00:32<00:57, 2.95it/s]
33%|█████████████▊ | 82/250 [00:32<00:51, 3.24it/s]
33%|█████████████▉ | 83/250 [00:32<00:55, 3.01it/s]
34%|██████████████ | 84/250 [00:33<01:04, 2.56it/s]
34%|██████████████▎ | 85/250 [00:33<01:03, 2.58it/s]
34%|██████████████▍ | 86/250 [00:34<01:04, 2.55it/s]
35%|██████████████▌ | 87/250 [00:34<01:13, 2.22it/s]
35%|██████████████▊ | 88/250 [00:35<01:04, 2.50it/s]
36%|██████████████▉ | 89/250 [00:35<01:01, 2.60it/s]
36%|███████████████ | 90/250 [00:35<01:01, 2.58it/s]
36%|███████████████▎ | 91/250 [00:36<00:58, 2.74it/s]
37%|███████████████▍ | 92/250 [00:36<00:56, 2.80it/s]
37%|███████████████▌ | 93/250 [00:37<01:14, 2.12it/s]
38%|███████████████▊ | 94/250 [00:37<01:07, 2.30it/s]
38%|███████████████▉ | 95/250 [00:37<01:04, 2.39it/s]
38%|████████████████▏ | 96/250 [00:38<01:01, 2.52it/s]
39%|████████████████▎ | 97/250 [00:38<00:55, 2.78it/s]
39%|████████████████▍ | 98/250 [00:39<01:01, 2.48it/s]
40%|████████████████▋ | 99/250 [00:39<00:58, 2.58it/s]
40%|████████████████▍ | 100/250 [00:39<00:54, 2.77it/s]
40%|████████████████▌ | 101/250 [00:40<00:52, 2.85it/s]
41%|████████████████▋ | 102/250 [00:40<00:49, 3.02it/s]
41%|████████████████▉ | 103/250 [00:41<01:12, 2.03it/s]
42%|█████████████████ | 104/250 [00:41<01:10, 2.06it/s]
42%|█████████████████▏ | 105/250 [00:42<01:04, 2.25it/s]
42%|█████████████████▍ | 106/250 [00:42<00:59, 2.40it/s]
43%|█████████████████▌ | 107/250 [00:42<00:58, 2.44it/s]
43%|█████████████████▋ | 108/250 [00:43<01:00, 2.33it/s]
44%|█████████████████▉ | 109/250 [00:43<00:54, 2.61it/s]
44%|██████████████████ | 110/250 [00:43<00:50, 2.77it/s]
44%|██████████████████▏ | 111/250 [00:44<00:52, 2.64it/s]
45%|██████████████████▎ | 112/250 [00:44<00:53, 2.60it/s]
45%|██████████████████▌ | 113/250 [00:44<00:46, 2.92it/s]
46%|██████████████████▋ | 114/250 [00:45<00:45, 2.97it/s]
46%|██████████████████▊ | 115/250 [00:45<00:54, 2.49it/s]
46%|███████████████████ | 116/250 [00:46<00:49, 2.69it/s]
47%|███████████████████▏ | 117/250 [00:46<00:54, 2.44it/s]
47%|███████████████████▎ | 118/250 [00:47<01:12, 1.82it/s]
48%|███████████████████▌ | 119/250 [00:47<01:06, 1.97it/s]
48%|███████████████████▋ | 120/250 [00:48<00:57, 2.27it/s]
48%|███████████████████▊ | 121/250 [00:48<00:51, 2.49it/s]
49%|████████████████████ | 122/250 [00:48<00:47, 2.67it/s]
49%|████████████████████▏ | 123/250 [00:49<00:45, 2.82it/s]
50%|████████████████████▎ | 124/250 [00:49<00:42, 2.93it/s]
50%|████████████████████▌ | 125/250 [00:49<00:42, 2.95it/s]
50%|████████████████████▋ | 126/250 [00:50<00:46, 2.69it/s]
51%|████████████████████▊ | 127/250 [00:50<00:43, 2.81it/s]
51%|████████████████████▉ | 128/250 [00:50<00:46, 2.64it/s]
52%|█████████████████████▏ | 129/250 [00:51<00:50, 2.37it/s]
52%|█████████████████████▎ | 130/250 [00:51<00:48, 2.50it/s]
52%|█████████████████████▍ | 131/250 [00:52<00:57, 2.06it/s]
53%|█████████████████████▋ | 132/250 [00:52<00:53, 2.20it/s]
53%|█████████████████████▊ | 133/250 [00:53<00:55, 2.11it/s]
54%|█████████████████████▉ | 134/250 [00:53<00:48, 2.37it/s]
54%|██████████████████████▏ | 135/250 [00:54<00:46, 2.45it/s]
54%|██████████████████████▎ | 136/250 [00:54<00:44, 2.57it/s]
55%|██████████████████████▍ | 137/250 [00:54<00:44, 2.56it/s]
55%|██████████████████████▋ | 138/250 [00:55<00:43, 2.56it/s]
56%|██████████████████████▊ | 139/250 [00:55<00:41, 2.68it/s]
56%|██████████████████████▉ | 140/250 [00:55<00:40, 2.70it/s]
56%|███████████████████████ | 141/250 [00:56<00:41, 2.63it/s]
57%|███████████████████████▎ | 142/250 [00:56<00:43, 2.48it/s]
57%|███████████████████████▍ | 143/250 [00:57<00:39, 2.70it/s]
58%|███████████████████████▌ | 144/250 [00:57<00:47, 2.22it/s]
58%|███████████████████████▊ | 145/250 [00:58<00:44, 2.35it/s]
58%|███████████████████████▉ | 146/250 [00:58<00:42, 2.42it/s]
59%|████████████████████████ | 147/250 [00:58<00:40, 2.52it/s]
59%|████████████████████████▎ | 148/250 [00:59<00:42, 2.41it/s]
60%|████████████████████████▍ | 149/250 [00:59<00:46, 2.16it/s]
60%|████████████████████████▌ | 150/250 [01:00<00:41, 2.39it/s]
60%|████████████████████████▊ | 151/250 [01:00<00:39, 2.52it/s]
61%|████████████████████████▉ | 152/250 [01:00<00:38, 2.55it/s]
61%|█████████████████████████ | 153/250 [01:01<00:38, 2.54it/s]
62%|█████████████████████████▎ | 154/250 [01:01<00:42, 2.27it/s]
62%|█████████████████████████▍ | 155/250 [01:02<00:39, 2.41it/s]
62%|█████████████████████████▌ | 156/250 [01:02<00:37, 2.49it/s]
63%|█████████████████████████▋ | 157/250 [01:02<00:36, 2.52it/s]
63%|█████████████████████████▉ | 158/250 [01:03<00:33, 2.76it/s]
64%|██████████████████████████ | 159/250 [01:03<00:32, 2.81it/s]
64%|██████████████████████████▏ | 160/250 [01:03<00:31, 2.87it/s]
64%|██████████████████████████▍ | 161/250 [01:04<00:30, 2.88it/s]
65%|██████████████████████████▌ | 162/250 [01:04<00:35, 2.47it/s]
65%|██████████████████████████▋ | 163/250 [01:05<00:39, 2.22it/s]
66%|██████████████████████████▉ | 164/250 [01:05<00:41, 2.08it/s]
66%|███████████████████████████ | 165/250 [01:06<00:47, 1.79it/s]
66%|███████████████████████████▏ | 166/250 [01:06<00:41, 2.03it/s]
67%|███████████████████████████▍ | 167/250 [01:07<00:39, 2.09it/s]
67%|███████████████████████████▌ | 168/250 [01:07<00:35, 2.31it/s]
68%|███████████████████████████▋ | 169/250 [01:08<00:33, 2.41it/s]
68%|███████████████████████████▉ | 170/250 [01:08<00:32, 2.43it/s]
68%|████████████████████████████ | 171/250 [01:09<00:36, 2.15it/s]
69%|████████████████████████████▏ | 172/250 [01:09<00:34, 2.28it/s]
69%|████████████████████████████▎ | 173/250 [01:09<00:31, 2.47it/s]
70%|████████████████████████████▌ | 174/250 [01:10<00:35, 2.16it/s]
70%|████████████████████████████▋ | 175/250 [01:10<00:32, 2.34it/s]
70%|████████████████████████████▊ | 176/250 [01:11<00:29, 2.51it/s]
71%|█████████████████████████████ | 177/250 [01:11<00:29, 2.45it/s]
71%|█████████████████████████████▏ | 178/250 [01:11<00:27, 2.60it/s]
72%|█████████████████████████████▎ | 179/250 [01:12<00:29, 2.44it/s]
72%|█████████████████████████████▌ | 180/250 [01:12<00:25, 2.69it/s]
72%|█████████████████████████████▋ | 181/250 [01:13<00:27, 2.53it/s]
73%|█████████████████████████████▊ | 182/250 [01:13<00:24, 2.82it/s]
73%|██████████████████████████████ | 183/250 [01:13<00:27, 2.48it/s]
74%|██████████████████████████████▏ | 184/250 [01:14<00:26, 2.51it/s]
74%|██████████████████████████████▎ | 185/250 [01:14<00:30, 2.16it/s]
74%|██████████████████████████████▌ | 186/250 [01:15<00:27, 2.35it/s]
75%|██████████████████████████████▋ | 187/250 [01:15<00:28, 2.22it/s]
75%|██████████████████████████████▊ | 188/250 [01:16<00:26, 2.31it/s]
76%|██████████████████████████████▉ | 189/250 [01:16<00:27, 2.26it/s]
76%|███████████████████████████████▏ | 190/250 [01:16<00:25, 2.38it/s]
76%|███████████████████████████████▎ | 191/250 [01:17<00:23, 2.54it/s]
77%|███████████████████████████████▍ | 192/250 [01:17<00:22, 2.58it/s]
77%|███████████████████████████████▋ | 193/250 [01:18<00:22, 2.51it/s]
78%|███████████████████████████████▊ | 194/250 [01:18<00:22, 2.54it/s]
78%|███████████████████████████████▉ | 195/250 [01:18<00:23, 2.31it/s]
78%|████████████████████████████████▏ | 196/250 [01:19<00:24, 2.24it/s]
79%|████████████████████████████████▎ | 197/250 [01:19<00:23, 2.26it/s]
79%|████████████████████████████████▍ | 198/250 [01:20<00:20, 2.53it/s]
80%|████████████████████████████████▋ | 199/250 [01:20<00:18, 2.75it/s]
80%|████████████████████████████████▊ | 200/250 [01:20<00:18, 2.72it/s]
80%|████████████████████████████████▉ | 201/250 [01:21<00:19, 2.50it/s]
81%|█████████████████████████████████▏ | 202/250 [01:21<00:19, 2.48it/s]
81%|█████████████████████████████████▎ | 203/250 [01:21<00:17, 2.71it/s]
82%|█████████████████████████████████▍ | 204/250 [01:22<00:16, 2.71it/s]
82%|█████████████████████████████████▌ | 205/250 [01:22<00:15, 2.92it/s]
82%|█████████████████████████████████▊ | 206/250 [01:23<00:15, 2.79it/s]
83%|█████████████████████████████████▉ | 207/250 [01:23<00:14, 2.89it/s]
83%|██████████████████████████████████ | 208/250 [01:23<00:17, 2.36it/s]
84%|██████████████████████████████████▎ | 209/250 [01:24<00:16, 2.43it/s]
84%|██████████████████████████████████▍ | 210/250 [01:24<00:15, 2.62it/s]
84%|██████████████████████████████████▌ | 211/250 [01:25<00:16, 2.30it/s]
85%|██████████████████████████████████▊ | 212/250 [01:25<00:15, 2.43it/s]
85%|██████████████████████████████████▉ | 213/250 [01:25<00:14, 2.49it/s]
86%|███████████████████████████████████ | 214/250 [01:26<00:16, 2.17it/s]
86%|███████████████████████████████████▎ | 215/250 [01:26<00:15, 2.28it/s]
86%|███████████████████████████████████▍ | 216/250 [01:27<00:13, 2.47it/s]
87%|███████████████████████████████████▌ | 217/250 [01:27<00:13, 2.50it/s]
87%|███████████████████████████████████▊ | 218/250 [01:27<00:12, 2.62it/s]
88%|███████████████████████████████████▉ | 219/250 [01:28<00:13, 2.31it/s]
88%|████████████████████████████████████ | 220/250 [01:28<00:11, 2.54it/s]
88%|████████████████████████████████████▏ | 221/250 [01:29<00:11, 2.62it/s]
89%|████████████████████████████████████▍ | 222/250 [01:29<00:11, 2.49it/s]
89%|████████████████████████████████████▌ | 223/250 [01:29<00:10, 2.66it/s]
90%|████████████████████████████████████▋ | 224/250 [01:30<00:09, 2.74it/s]
90%|████████████████████████████████████▉ | 225/250 [01:30<00:08, 2.81it/s]
90%|█████████████████████████████████████ | 226/250 [01:31<00:11, 2.13it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:32<00:13, 1.69it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:32<00:11, 1.94it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:32<00:09, 2.21it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:33<00:08, 2.39it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:33<00:07, 2.54it/s]
93%|██████████████████████████████████████ | 232/250 [01:33<00:07, 2.44it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:34<00:06, 2.57it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:34<00:07, 2.11it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:35<00:06, 2.31it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:35<00:05, 2.44it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:36<00:05, 2.17it/s]
95%|███████████████████████████████████████ | 238/250 [01:36<00:04, 2.46it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:37<00:05, 2.17it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:38<00:05, 1.71it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:38<00:04, 1.93it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:38<00:03, 2.18it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:39<00:03, 2.31it/s]
98%|████████████████████████████████████████ | 244/250 [01:39<00:03, 1.97it/s]
98%|████████████████████████████████████████▏| 245/250 [01:40<00:02, 2.24it/s]
98%|████████████████████████████████████████▎| 246/250 [01:40<00:01, 2.42it/s]
99%|████████████████████████████████████████▌| 247/250 [01:40<00:01, 2.41it/s]
99%|████████████████████████████████████████▋| 248/250 [01:41<00:00, 2.39it/s]
100%|████████████████████████████████████████▊| 249/250 [01:41<00:00, 2.14it/s]
{'eval_loss': 1.0761855840682983, 'eval_runtime': 103.3201, 'eval_samples_per_second': 19.357, 'eval_steps_per_second': 2.42, 'epoch': 1.8}
90%|███████████████████████████████████ | 702/780 [2:34:41<35:02, 26.96s/it]
100%|█████████████████████████████████████████| 250/250 [01:42<00:00, 1.78it/s]
{'loss': 1.0429, 'learning_rate': 2.4291497975708502e-05, 'epoch': 1.85}
{'loss': 1.0457, 'learning_rate': 1.6194331983805668e-05, 'epoch': 1.9}
95%|█████████████████████████████████████ | 741/780 [2:52:09<17:22, 26.74s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:45, 5.42it/s]
1%|▌ | 3/250 [00:00<01:20, 3.09it/s]
2%|▋ | 4/250 [00:01<01:31, 2.69it/s]
2%|▊ | 5/250 [00:01<01:32, 2.64it/s]
2%|█ | 6/250 [00:02<01:53, 2.15it/s]
3%|█▏ | 7/250 [00:02<01:45, 2.31it/s]
3%|█▍ | 8/250 [00:03<01:39, 2.44it/s]
4%|█▌ | 9/250 [00:03<01:33, 2.58it/s]
4%|█▋ | 10/250 [00:03<01:27, 2.73it/s]
4%|█▊ | 11/250 [00:04<01:27, 2.74it/s]
5%|██ | 12/250 [00:04<01:32, 2.57it/s]
5%|██▏ | 13/250 [00:04<01:29, 2.64it/s]
6%|██▎ | 14/250 [00:05<01:30, 2.61it/s]
6%|██▌ | 15/250 [00:05<01:44, 2.24it/s]
6%|██▋ | 16/250 [00:06<01:42, 2.27it/s]
7%|██▊ | 17/250 [00:06<01:43, 2.24it/s]
7%|███ | 18/250 [00:07<01:37, 2.37it/s]
8%|███▏ | 19/250 [00:07<01:42, 2.25it/s]
8%|███▎ | 20/250 [00:08<01:39, 2.32it/s]
8%|███▌ | 21/250 [00:08<02:00, 1.90it/s]
9%|███▋ | 22/250 [00:09<01:47, 2.12it/s]
9%|███▊ | 23/250 [00:09<01:52, 2.02it/s]
10%|████ | 24/250 [00:10<01:40, 2.25it/s]
10%|████▏ | 25/250 [00:10<01:37, 2.31it/s]
10%|████▎ | 26/250 [00:10<01:43, 2.16it/s]
11%|████▌ | 27/250 [00:11<01:35, 2.34it/s]
11%|████▋ | 28/250 [00:11<01:31, 2.42it/s]
12%|████▊ | 29/250 [00:11<01:21, 2.73it/s]
12%|█████ | 30/250 [00:12<01:53, 1.93it/s]
12%|█████▏ | 31/250 [00:13<01:41, 2.16it/s]
13%|█████▍ | 32/250 [00:13<01:48, 2.00it/s]
13%|█████▌ | 33/250 [00:14<01:41, 2.13it/s]
14%|█████▋ | 34/250 [00:14<01:39, 2.16it/s]
14%|█████▉ | 35/250 [00:14<01:35, 2.25it/s]
14%|██████ | 36/250 [00:15<01:27, 2.45it/s]
15%|██████▏ | 37/250 [00:15<01:34, 2.25it/s]
15%|██████▍ | 38/250 [00:16<01:25, 2.49it/s]
16%|██████▌ | 39/250 [00:16<01:18, 2.70it/s]
16%|██████▋ | 40/250 [00:16<01:12, 2.90it/s]
16%|██████▉ | 41/250 [00:17<01:10, 2.98it/s]
17%|███████ | 42/250 [00:17<01:12, 2.88it/s]
17%|███████▏ | 43/250 [00:17<01:10, 2.95it/s]
18%|███████▍ | 44/250 [00:18<01:10, 2.91it/s]
18%|███████▌ | 45/250 [00:18<01:22, 2.48it/s]
18%|███████▋ | 46/250 [00:19<01:25, 2.39it/s]
19%|███████▉ | 47/250 [00:19<01:17, 2.62it/s]
19%|████████ | 48/250 [00:19<01:19, 2.53it/s]
20%|████████▏ | 49/250 [00:20<01:12, 2.78it/s]
20%|████████▍ | 50/250 [00:20<01:11, 2.81it/s]
20%|████████▌ | 51/250 [00:20<01:09, 2.85it/s]
21%|████████▋ | 52/250 [00:21<01:08, 2.89it/s]
21%|████████▉ | 53/250 [00:21<01:05, 3.03it/s]
22%|█████████ | 54/250 [00:21<01:10, 2.79it/s]
22%|█████████▏ | 55/250 [00:22<01:09, 2.81it/s]
22%|█████████▍ | 56/250 [00:22<01:11, 2.70it/s]
23%|█████████▌ | 57/250 [00:22<01:08, 2.80it/s]
23%|█████████▋ | 58/250 [00:23<01:22, 2.33it/s]
24%|█████████▉ | 59/250 [00:23<01:23, 2.29it/s]
24%|██████████ | 60/250 [00:24<01:20, 2.35it/s]
24%|██████████▏ | 61/250 [00:25<01:45, 1.79it/s]
25%|██████████▍ | 62/250 [00:25<01:34, 1.99it/s]
25%|██████████▌ | 63/250 [00:26<01:30, 2.06it/s]
26%|██████████▊ | 64/250 [00:26<01:24, 2.20it/s]
26%|██████████▉ | 65/250 [00:26<01:17, 2.38it/s]
26%|███████████ | 66/250 [00:27<01:13, 2.51it/s]
27%|███████████▎ | 67/250 [00:27<01:14, 2.47it/s]
27%|███████████▍ | 68/250 [00:27<01:10, 2.57it/s]
28%|███████████▌ | 69/250 [00:28<01:08, 2.65it/s]
28%|███████████▊ | 70/250 [00:28<01:04, 2.78it/s]
28%|███████████▉ | 71/250 [00:28<01:05, 2.72it/s]
29%|████████████ | 72/250 [00:29<01:03, 2.78it/s]
29%|████████████▎ | 73/250 [00:29<01:05, 2.68it/s]
30%|████████████▍ | 74/250 [00:29<01:01, 2.87it/s]
30%|████████████▌ | 75/250 [00:30<00:59, 2.93it/s]
30%|████████████▊ | 76/250 [00:30<00:58, 2.98it/s]
31%|████████████▉ | 77/250 [00:30<00:59, 2.93it/s]
31%|█████████████ | 78/250 [00:31<00:55, 3.09it/s]
32%|█████████████▎ | 79/250 [00:31<00:59, 2.87it/s]
32%|█████████████▍ | 80/250 [00:32<01:00, 2.80it/s]
32%|█████████████▌ | 81/250 [00:32<00:57, 2.95it/s]
33%|█████████████▊ | 82/250 [00:32<00:51, 3.24it/s]
33%|█████████████▉ | 83/250 [00:32<00:55, 3.00it/s]
34%|██████████████ | 84/250 [00:33<01:04, 2.56it/s]
34%|██████████████▎ | 85/250 [00:33<01:04, 2.58it/s]
34%|██████████████▍ | 86/250 [00:34<01:04, 2.55it/s]
35%|██████████████▌ | 87/250 [00:34<01:13, 2.22it/s]
35%|██████████████▊ | 88/250 [00:35<01:04, 2.50it/s]
36%|██████████████▉ | 89/250 [00:35<01:01, 2.60it/s]
36%|███████████████ | 90/250 [00:35<01:01, 2.59it/s]
36%|███████████████▎ | 91/250 [00:36<00:57, 2.74it/s]
37%|███████████████▍ | 92/250 [00:36<00:56, 2.81it/s]
37%|███████████████▌ | 93/250 [00:37<01:14, 2.12it/s]
38%|███████████████▊ | 94/250 [00:37<01:07, 2.30it/s]
38%|███████████████▉ | 95/250 [00:37<01:04, 2.39it/s]
38%|████████████████▏ | 96/250 [00:38<01:01, 2.52it/s]
39%|████████████████▎ | 97/250 [00:38<00:54, 2.79it/s]
39%|████████████████▍ | 98/250 [00:39<01:01, 2.48it/s]
40%|████████████████▋ | 99/250 [00:39<00:58, 2.57it/s]
40%|████████████████▍ | 100/250 [00:39<00:54, 2.76it/s]
40%|████████████████▌ | 101/250 [00:40<00:52, 2.84it/s]
41%|████████████████▋ | 102/250 [00:40<00:49, 3.01it/s]
41%|████████████████▉ | 103/250 [00:41<01:12, 2.02it/s]
42%|█████████████████ | 104/250 [00:41<01:11, 2.05it/s]
42%|█████████████████▏ | 105/250 [00:42<01:04, 2.25it/s]
42%|█████████████████▍ | 106/250 [00:42<01:00, 2.40it/s]
43%|█████████████████▌ | 107/250 [00:42<00:58, 2.44it/s]
43%|█████████████████▋ | 108/250 [00:43<01:01, 2.33it/s]
44%|█████████████████▉ | 109/250 [00:43<00:54, 2.60it/s]
44%|██████████████████ | 110/250 [00:43<00:50, 2.77it/s]
44%|██████████████████▏ | 111/250 [00:44<00:52, 2.63it/s]
45%|██████████████████▎ | 112/250 [00:44<00:53, 2.59it/s]
45%|██████████████████▌ | 113/250 [00:44<00:47, 2.90it/s]
46%|██████████████████▋ | 114/250 [00:45<00:46, 2.95it/s]
46%|██████████████████▊ | 115/250 [00:45<00:54, 2.48it/s]
46%|███████████████████ | 116/250 [00:46<00:50, 2.68it/s]
47%|███████████████████▏ | 117/250 [00:46<00:54, 2.44it/s]
47%|███████████████████▎ | 118/250 [00:47<01:12, 1.82it/s]
48%|███████████████████▌ | 119/250 [00:47<01:06, 1.96it/s]
48%|███████████████████▋ | 120/250 [00:48<00:57, 2.26it/s]
48%|███████████████████▊ | 121/250 [00:48<00:52, 2.48it/s]
49%|████████████████████ | 122/250 [00:48<00:48, 2.67it/s]
49%|████████████████████▏ | 123/250 [00:49<00:45, 2.81it/s]
50%|████████████████████▎ | 124/250 [00:49<00:43, 2.93it/s]
50%|████████████████████▌ | 125/250 [00:49<00:42, 2.93it/s]
50%|████████████████████▋ | 126/250 [00:50<00:46, 2.68it/s]
51%|████████████████████▊ | 127/250 [00:50<00:43, 2.80it/s]
51%|████████████████████▉ | 128/250 [00:50<00:46, 2.64it/s]
52%|█████████████████████▏ | 129/250 [00:51<00:51, 2.37it/s]
52%|█████████████████████▎ | 130/250 [00:51<00:48, 2.49it/s]
52%|█████████████████████▍ | 131/250 [00:52<00:57, 2.06it/s]
53%|█████████████████████▋ | 132/250 [00:52<00:53, 2.20it/s]
53%|█████████████████████▊ | 133/250 [00:53<00:55, 2.11it/s]
54%|█████████████████████▉ | 134/250 [00:53<00:49, 2.36it/s]
54%|██████████████████████▏ | 135/250 [00:54<00:47, 2.44it/s]
54%|██████████████████████▎ | 136/250 [00:54<00:44, 2.57it/s]
55%|██████████████████████▍ | 137/250 [00:54<00:44, 2.56it/s]
55%|██████████████████████▋ | 138/250 [00:55<00:43, 2.56it/s]
56%|██████████████████████▊ | 139/250 [00:55<00:41, 2.68it/s]
56%|██████████████████████▉ | 140/250 [00:55<00:40, 2.70it/s]
56%|███████████████████████ | 141/250 [00:56<00:41, 2.62it/s]
57%|███████████████████████▎ | 142/250 [00:56<00:43, 2.47it/s]
57%|███████████████████████▍ | 143/250 [00:57<00:39, 2.69it/s]
58%|███████████████████████▌ | 144/250 [00:57<00:47, 2.22it/s]
58%|███████████████████████▊ | 145/250 [00:58<00:44, 2.34it/s]
58%|███████████████████████▉ | 146/250 [00:58<00:43, 2.41it/s]
59%|████████████████████████ | 147/250 [00:58<00:41, 2.51it/s]
59%|████████████████████████▎ | 148/250 [00:59<00:42, 2.41it/s]
60%|████████████████████████▍ | 149/250 [00:59<00:46, 2.16it/s]
60%|████████████████████████▌ | 150/250 [01:00<00:41, 2.38it/s]
60%|████████████████████████▊ | 151/250 [01:00<00:39, 2.51it/s]
61%|████████████████████████▉ | 152/250 [01:00<00:38, 2.54it/s]
61%|█████████████████████████ | 153/250 [01:01<00:38, 2.52it/s]
62%|█████████████████████████▎ | 154/250 [01:01<00:42, 2.27it/s]
62%|█████████████████████████▍ | 155/250 [01:02<00:39, 2.40it/s]
62%|█████████████████████████▌ | 156/250 [01:02<00:37, 2.48it/s]
63%|█████████████████████████▋ | 157/250 [01:03<00:37, 2.51it/s]
63%|█████████████████████████▉ | 158/250 [01:03<00:33, 2.74it/s]
64%|██████████████████████████ | 159/250 [01:03<00:32, 2.80it/s]
64%|██████████████████████████▏ | 160/250 [01:03<00:31, 2.86it/s]
64%|██████████████████████████▍ | 161/250 [01:04<00:30, 2.88it/s]
65%|██████████████████████████▌ | 162/250 [01:04<00:35, 2.46it/s]
65%|██████████████████████████▋ | 163/250 [01:05<00:39, 2.22it/s]
66%|██████████████████████████▉ | 164/250 [01:05<00:41, 2.08it/s]
66%|███████████████████████████ | 165/250 [01:06<00:47, 1.79it/s]
66%|███████████████████████████▏ | 166/250 [01:07<00:41, 2.03it/s]
67%|███████████████████████████▍ | 167/250 [01:07<00:39, 2.09it/s]
67%|███████████████████████████▌ | 168/250 [01:07<00:35, 2.31it/s]
68%|███████████████████████████▋ | 169/250 [01:08<00:33, 2.41it/s]
68%|███████████████████████████▉ | 170/250 [01:08<00:32, 2.43it/s]
68%|████████████████████████████ | 171/250 [01:09<00:36, 2.15it/s]
69%|████████████████████████████▏ | 172/250 [01:09<00:34, 2.26it/s]
69%|████████████████████████████▎ | 173/250 [01:09<00:31, 2.45it/s]
70%|████████████████████████████▌ | 174/250 [01:10<00:35, 2.15it/s]
70%|████████████████████████████▋ | 175/250 [01:10<00:32, 2.33it/s]
70%|████████████████████████████▊ | 176/250 [01:11<00:29, 2.50it/s]
71%|█████████████████████████████ | 177/250 [01:11<00:29, 2.45it/s]
71%|█████████████████████████████▏ | 178/250 [01:11<00:27, 2.59it/s]
72%|█████████████████████████████▎ | 179/250 [01:12<00:29, 2.44it/s]
72%|█████████████████████████████▌ | 180/250 [01:12<00:26, 2.68it/s]
72%|█████████████████████████████▋ | 181/250 [01:13<00:27, 2.52it/s]
73%|█████████████████████████████▊ | 182/250 [01:13<00:24, 2.81it/s]
73%|██████████████████████████████ | 183/250 [01:13<00:27, 2.47it/s]
74%|██████████████████████████████▏ | 184/250 [01:14<00:26, 2.50it/s]
74%|██████████████████████████████▎ | 185/250 [01:14<00:30, 2.16it/s]
74%|██████████████████████████████▌ | 186/250 [01:15<00:27, 2.35it/s]
75%|██████████████████████████████▋ | 187/250 [01:15<00:28, 2.22it/s]
75%|██████████████████████████████▊ | 188/250 [01:16<00:26, 2.31it/s]
76%|██████████████████████████████▉ | 189/250 [01:16<00:27, 2.25it/s]
76%|███████████████████████████████▏ | 190/250 [01:16<00:25, 2.38it/s]
76%|███████████████████████████████▎ | 191/250 [01:17<00:23, 2.54it/s]
77%|███████████████████████████████▍ | 192/250 [01:17<00:22, 2.58it/s]
77%|███████████████████████████████▋ | 193/250 [01:18<00:22, 2.51it/s]
78%|███████████████████████████████▊ | 194/250 [01:18<00:22, 2.54it/s]
78%|███████████████████████████████▉ | 195/250 [01:19<00:23, 2.31it/s]
78%|████████████████████████████████▏ | 196/250 [01:19<00:24, 2.24it/s]
79%|████████████████████████████████▎ | 197/250 [01:19<00:23, 2.26it/s]
79%|████████████████████████████████▍ | 198/250 [01:20<00:20, 2.53it/s]
80%|████████████████████████████████▋ | 199/250 [01:20<00:18, 2.74it/s]
80%|████████████████████████████████▊ | 200/250 [01:20<00:18, 2.71it/s]
80%|████████████████████████████████▉ | 201/250 [01:21<00:19, 2.50it/s]
81%|█████████████████████████████████▏ | 202/250 [01:21<00:19, 2.49it/s]
81%|█████████████████████████████████▎ | 203/250 [01:22<00:17, 2.71it/s]
82%|█████████████████████████████████▍ | 204/250 [01:22<00:16, 2.71it/s]
82%|█████████████████████████████████▌ | 205/250 [01:22<00:15, 2.92it/s]
82%|█████████████████████████████████▊ | 206/250 [01:23<00:15, 2.80it/s]
83%|█████████████████████████████████▉ | 207/250 [01:23<00:14, 2.89it/s]
83%|██████████████████████████████████ | 208/250 [01:24<00:17, 2.36it/s]
84%|██████████████████████████████████▎ | 209/250 [01:24<00:16, 2.43it/s]
84%|██████████████████████████████████▍ | 210/250 [01:24<00:15, 2.62it/s]
84%|██████████████████████████████████▌ | 211/250 [01:25<00:16, 2.30it/s]
85%|██████████████████████████████████▊ | 212/250 [01:25<00:15, 2.43it/s]
85%|██████████████████████████████████▉ | 213/250 [01:26<00:14, 2.49it/s]
86%|███████████████████████████████████ | 214/250 [01:26<00:16, 2.17it/s]
86%|███████████████████████████████████▎ | 215/250 [01:27<00:15, 2.28it/s]
86%|███████████████████████████████████▍ | 216/250 [01:27<00:13, 2.47it/s]
87%|███████████████████████████████████▌ | 217/250 [01:27<00:13, 2.51it/s]
87%|███████████████████████████████████▊ | 218/250 [01:28<00:12, 2.62it/s]
88%|███████████████████████████████████▉ | 219/250 [01:28<00:13, 2.31it/s]
88%|████████████████████████████████████ | 220/250 [01:28<00:11, 2.54it/s]
88%|████████████████████████████████████▏ | 221/250 [01:29<00:11, 2.62it/s]
89%|████████████████████████████████████▍ | 222/250 [01:29<00:11, 2.49it/s]
89%|████████████████████████████████████▌ | 223/250 [01:30<00:10, 2.66it/s]
90%|████████████████████████████████████▋ | 224/250 [01:30<00:09, 2.74it/s]
90%|████████████████████████████████████▉ | 225/250 [01:30<00:08, 2.81it/s]
90%|█████████████████████████████████████ | 226/250 [01:31<00:11, 2.13it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:32<00:13, 1.69it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:32<00:11, 1.94it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:32<00:09, 2.21it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:33<00:08, 2.38it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:33<00:07, 2.54it/s]
93%|██████████████████████████████████████ | 232/250 [01:34<00:07, 2.44it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:34<00:06, 2.56it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:35<00:07, 2.10it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:35<00:06, 2.30it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:35<00:05, 2.43it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:36<00:06, 2.16it/s]
95%|███████████████████████████████████████ | 238/250 [01:36<00:04, 2.45it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:37<00:05, 2.17it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:38<00:05, 1.71it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:38<00:04, 1.92it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:38<00:03, 2.18it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:39<00:03, 2.30it/s]
98%|████████████████████████████████████████ | 244/250 [01:39<00:03, 1.96it/s]
98%|████████████████████████████████████████▏| 245/250 [01:40<00:02, 2.23it/s]
98%|████████████████████████████████████████▎| 246/250 [01:40<00:01, 2.42it/s]
99%|████████████████████████████████████████▌| 247/250 [01:40<00:01, 2.41it/s]
99%|████████████████████████████████████████▋| 248/250 [01:41<00:00, 2.39it/s]
100%|████████████████████████████████████████▊| 249/250 [01:41<00:00, 2.14it/s]
{'eval_loss': 1.0757148265838623, 'eval_runtime': 103.4431, 'eval_samples_per_second': 19.334, 'eval_steps_per_second': 2.417, 'epoch': 1.9}
95%|█████████████████████████████████████ | 741/780 [2:53:52<17:22, 26.74s/it]
100%|█████████████████████████████████████████| 250/250 [01:42<00:00, 1.78it/s]
{'loss': 1.0465, 'learning_rate': 8.097165991902834e-06, 'epoch': 1.95}
{'loss': 1.0576, 'learning_rate': 0.0, 'epoch': 2.0}
100%|███████████████████████████████████████| 780/780 [3:11:33<00:00, 27.31s/it]
0%| | 0/250 [00:00<?, ?it/s]
1%|▎ | 2/250 [00:00<00:45, 5.44it/s]
1%|▌ | 3/250 [00:00<01:19, 3.09it/s]
2%|▋ | 4/250 [00:01<01:31, 2.69it/s]
2%|▊ | 5/250 [00:01<01:32, 2.64it/s]
2%|█ | 6/250 [00:02<01:53, 2.15it/s]
3%|█▏ | 7/250 [00:02<01:44, 2.32it/s]
3%|█▍ | 8/250 [00:03<01:38, 2.46it/s]
4%|█▌ | 9/250 [00:03<01:32, 2.59it/s]
4%|█▋ | 10/250 [00:03<01:27, 2.75it/s]
4%|█▊ | 11/250 [00:04<01:26, 2.76it/s]
5%|██ | 12/250 [00:04<01:32, 2.57it/s]
5%|██▏ | 13/250 [00:04<01:29, 2.64it/s]
6%|██▎ | 14/250 [00:05<01:30, 2.61it/s]
6%|██▌ | 15/250 [00:05<01:44, 2.25it/s]
6%|██▋ | 16/250 [00:06<01:42, 2.28it/s]
7%|██▊ | 17/250 [00:06<01:43, 2.25it/s]
7%|███ | 18/250 [00:07<01:37, 2.38it/s]
8%|███▏ | 19/250 [00:07<01:42, 2.26it/s]
8%|███▎ | 20/250 [00:08<01:39, 2.32it/s]
8%|███▌ | 21/250 [00:08<02:00, 1.91it/s]
9%|███▋ | 22/250 [00:09<01:47, 2.12it/s]
9%|███▊ | 23/250 [00:09<01:52, 2.02it/s]
10%|████ | 24/250 [00:10<01:42, 2.20it/s]
10%|████▏ | 25/250 [00:10<01:38, 2.28it/s]
10%|████▎ | 26/250 [00:10<01:44, 2.15it/s]
11%|████▌ | 27/250 [00:11<01:35, 2.33it/s]
11%|████▋ | 28/250 [00:11<01:31, 2.41it/s]
12%|████▊ | 29/250 [00:11<01:21, 2.72it/s]
12%|█████ | 30/250 [00:12<01:53, 1.94it/s]
12%|█████▏ | 31/250 [00:13<01:41, 2.17it/s]
13%|█████▍ | 32/250 [00:13<01:48, 2.01it/s]
13%|█████▌ | 33/250 [00:14<01:41, 2.13it/s]
14%|█████▋ | 34/250 [00:14<01:39, 2.17it/s]
14%|█████▉ | 35/250 [00:14<01:35, 2.26it/s]
14%|██████ | 36/250 [00:15<01:27, 2.45it/s]
15%|██████▏ | 37/250 [00:15<01:34, 2.25it/s]
15%|██████▍ | 38/250 [00:16<01:25, 2.48it/s]
16%|██████▌ | 39/250 [00:16<01:18, 2.70it/s]
16%|██████▋ | 40/250 [00:16<01:12, 2.90it/s]
16%|██████▉ | 41/250 [00:17<01:10, 2.98it/s]
17%|███████ | 42/250 [00:17<01:12, 2.87it/s]
17%|███████▏ | 43/250 [00:17<01:10, 2.95it/s]
18%|███████▍ | 44/250 [00:18<01:10, 2.91it/s]
18%|███████▌ | 45/250 [00:18<01:22, 2.48it/s]
18%|███████▋ | 46/250 [00:19<01:25, 2.40it/s]
19%|███████▉ | 47/250 [00:19<01:17, 2.63it/s]
19%|████████ | 48/250 [00:19<01:19, 2.54it/s]
20%|████████▏ | 49/250 [00:20<01:12, 2.79it/s]
20%|████████▍ | 50/250 [00:20<01:10, 2.82it/s]
20%|████████▌ | 51/250 [00:20<01:09, 2.85it/s]
21%|████████▋ | 52/250 [00:21<01:08, 2.89it/s]
21%|████████▉ | 53/250 [00:21<01:05, 3.03it/s]
22%|█████████ | 54/250 [00:21<01:10, 2.80it/s]
22%|█████████▏ | 55/250 [00:22<01:09, 2.81it/s]
22%|█████████▍ | 56/250 [00:22<01:11, 2.70it/s]
23%|█████████▌ | 57/250 [00:22<01:08, 2.81it/s]
23%|█████████▋ | 58/250 [00:23<01:22, 2.33it/s]
24%|█████████▉ | 59/250 [00:23<01:23, 2.29it/s]
24%|██████████ | 60/250 [00:24<01:20, 2.35it/s]
24%|██████████▏ | 61/250 [00:25<01:45, 1.79it/s]
25%|██████████▍ | 62/250 [00:25<01:34, 1.99it/s]
25%|██████████▌ | 63/250 [00:26<01:30, 2.06it/s]
26%|██████████▊ | 64/250 [00:26<01:24, 2.21it/s]
26%|██████████▉ | 65/250 [00:26<01:17, 2.38it/s]
26%|███████████ | 66/250 [00:27<01:13, 2.51it/s]
27%|███████████▎ | 67/250 [00:27<01:14, 2.47it/s]
27%|███████████▍ | 68/250 [00:27<01:10, 2.57it/s]
28%|███████████▌ | 69/250 [00:28<01:08, 2.65it/s]
28%|███████████▊ | 70/250 [00:28<01:04, 2.79it/s]
28%|███████████▉ | 71/250 [00:28<01:05, 2.73it/s]
29%|████████████ | 72/250 [00:29<01:03, 2.79it/s]
29%|████████████▎ | 73/250 [00:29<01:05, 2.68it/s]
30%|████████████▍ | 74/250 [00:29<01:01, 2.86it/s]
30%|████████████▌ | 75/250 [00:30<00:59, 2.92it/s]
30%|████████████▊ | 76/250 [00:30<00:58, 2.97it/s]
31%|████████████▉ | 77/250 [00:30<00:59, 2.92it/s]
31%|█████████████ | 78/250 [00:31<00:55, 3.08it/s]
32%|█████████████▎ | 79/250 [00:31<00:59, 2.87it/s]
32%|█████████████▍ | 80/250 [00:32<01:00, 2.80it/s]
32%|█████████████▌ | 81/250 [00:32<00:57, 2.94it/s]
33%|█████████████▊ | 82/250 [00:32<00:51, 3.24it/s]
33%|█████████████▉ | 83/250 [00:32<00:55, 3.01it/s]
34%|██████████████ | 84/250 [00:33<01:04, 2.56it/s]
34%|██████████████▎ | 85/250 [00:33<01:04, 2.57it/s]
34%|██████████████▍ | 86/250 [00:34<01:04, 2.55it/s]
35%|██████████████▌ | 87/250 [00:34<01:13, 2.22it/s]
35%|██████████████▊ | 88/250 [00:35<01:04, 2.49it/s]
36%|██████████████▉ | 89/250 [00:35<01:01, 2.60it/s]
36%|███████████████ | 90/250 [00:35<01:01, 2.58it/s]
36%|███████████████▎ | 91/250 [00:36<00:58, 2.74it/s]
37%|███████████████▍ | 92/250 [00:36<00:56, 2.80it/s]
37%|███████████████▌ | 93/250 [00:37<01:14, 2.12it/s]
38%|███████████████▊ | 94/250 [00:37<01:07, 2.30it/s]
38%|███████████████▉ | 95/250 [00:37<01:04, 2.39it/s]
38%|████████████████▏ | 96/250 [00:38<01:01, 2.52it/s]
39%|████████████████▎ | 97/250 [00:38<00:55, 2.78it/s]
39%|████████████████▍ | 98/250 [00:39<01:01, 2.48it/s]
40%|████████████████▋ | 99/250 [00:39<00:58, 2.58it/s]
40%|████████████████▍ | 100/250 [00:39<00:54, 2.77it/s]
40%|████████████████▌ | 101/250 [00:40<00:52, 2.83it/s]
41%|████████████████▋ | 102/250 [00:40<00:49, 3.01it/s]
41%|████████████████▉ | 103/250 [00:41<01:12, 2.02it/s]
42%|█████████████████ | 104/250 [00:41<01:11, 2.05it/s]
42%|█████████████████▏ | 105/250 [00:42<01:04, 2.25it/s]
42%|█████████████████▍ | 106/250 [00:42<00:59, 2.40it/s]
43%|█████████████████▌ | 107/250 [00:42<00:58, 2.44it/s]
43%|█████████████████▋ | 108/250 [00:43<01:01, 2.33it/s]
44%|█████████████████▉ | 109/250 [00:43<00:54, 2.60it/s]
44%|██████████████████ | 110/250 [00:43<00:50, 2.77it/s]
44%|██████████████████▏ | 111/250 [00:44<00:52, 2.63it/s]
45%|██████████████████▎ | 112/250 [00:44<00:53, 2.59it/s]
45%|██████████████████▌ | 113/250 [00:44<00:47, 2.90it/s]
46%|██████████████████▋ | 114/250 [00:45<00:47, 2.87it/s]
46%|██████████████████▊ | 115/250 [00:45<00:55, 2.44it/s]
46%|███████████████████ | 116/250 [00:46<00:50, 2.64it/s]
47%|███████████████████▏ | 117/250 [00:46<00:55, 2.42it/s]
47%|███████████████████▎ | 118/250 [00:47<01:12, 1.81it/s]
48%|███████████████████▌ | 119/250 [00:47<01:06, 1.96it/s]
48%|███████████████████▋ | 120/250 [00:48<00:57, 2.26it/s]
48%|███████████████████▊ | 121/250 [00:48<00:52, 2.48it/s]
49%|████████████████████ | 122/250 [00:48<00:48, 2.66it/s]
49%|████████████████████▏ | 123/250 [00:49<00:45, 2.81it/s]
50%|████████████████████▎ | 124/250 [00:49<00:43, 2.92it/s]
50%|████████████████████▌ | 125/250 [00:49<00:42, 2.93it/s]
50%|████████████████████▋ | 126/250 [00:50<00:46, 2.68it/s]
51%|████████████████████▊ | 127/250 [00:50<00:43, 2.80it/s]
51%|████████████████████▉ | 128/250 [00:51<00:46, 2.64it/s]
52%|█████████████████████▏ | 129/250 [00:51<00:51, 2.37it/s]
52%|█████████████████████▎ | 130/250 [00:51<00:48, 2.49it/s]
52%|█████████████████████▍ | 131/250 [00:52<00:57, 2.06it/s]
53%|█████████████████████▋ | 132/250 [00:52<00:53, 2.19it/s]
53%|█████████████████████▊ | 133/250 [00:53<00:55, 2.11it/s]
54%|█████████████████████▉ | 134/250 [00:53<00:49, 2.36it/s]
54%|██████████████████████▏ | 135/250 [00:54<00:46, 2.45it/s]
54%|██████████████████████▎ | 136/250 [00:54<00:44, 2.58it/s]
55%|██████████████████████▍ | 137/250 [00:54<00:44, 2.56it/s]
55%|██████████████████████▋ | 138/250 [00:55<00:43, 2.56it/s]
56%|██████████████████████▊ | 139/250 [00:55<00:41, 2.68it/s]
56%|██████████████████████▉ | 140/250 [00:55<00:40, 2.70it/s]
56%|███████████████████████ | 141/250 [00:56<00:41, 2.62it/s]
57%|███████████████████████▎ | 142/250 [00:56<00:43, 2.48it/s]
57%|███████████████████████▍ | 143/250 [00:57<00:39, 2.70it/s]
58%|███████████████████████▌ | 144/250 [00:57<00:47, 2.22it/s]
58%|███████████████████████▊ | 145/250 [00:58<00:44, 2.35it/s]
58%|███████████████████████▉ | 146/250 [00:58<00:43, 2.42it/s]
59%|████████████████████████ | 147/250 [00:58<00:40, 2.51it/s]
59%|████████████████████████▎ | 148/250 [00:59<00:42, 2.41it/s]
60%|████████████████████████▍ | 149/250 [00:59<00:46, 2.16it/s]
60%|████████████████████████▌ | 150/250 [01:00<00:41, 2.38it/s]
60%|████████████████████████▊ | 151/250 [01:00<00:39, 2.50it/s]
61%|████████████████████████▉ | 152/250 [01:00<00:38, 2.54it/s]
61%|█████████████████████████ | 153/250 [01:01<00:38, 2.52it/s]
62%|█████████████████████████▎ | 154/250 [01:01<00:42, 2.27it/s]
62%|█████████████████████████▍ | 155/250 [01:02<00:39, 2.41it/s]
62%|█████████████████████████▌ | 156/250 [01:02<00:37, 2.49it/s]
63%|█████████████████████████▋ | 157/250 [01:03<00:36, 2.52it/s]
63%|█████████████████████████▉ | 158/250 [01:03<00:33, 2.76it/s]
64%|██████████████████████████ | 159/250 [01:03<00:32, 2.81it/s]
64%|██████████████████████████▏ | 160/250 [01:03<00:31, 2.88it/s]
64%|██████████████████████████▍ | 161/250 [01:04<00:30, 2.89it/s]
65%|██████████████████████████▌ | 162/250 [01:04<00:35, 2.47it/s]
65%|██████████████████████████▋ | 163/250 [01:05<00:39, 2.22it/s]
66%|██████████████████████████▉ | 164/250 [01:05<00:41, 2.08it/s]
66%|███████████████████████████ | 165/250 [01:06<00:47, 1.79it/s]
66%|███████████████████████████▏ | 166/250 [01:07<00:41, 2.03it/s]
67%|███████████████████████████▍ | 167/250 [01:07<00:39, 2.09it/s]
67%|███████████████████████████▌ | 168/250 [01:07<00:35, 2.31it/s]
68%|███████████████████████████▋ | 169/250 [01:08<00:33, 2.41it/s]
68%|███████████████████████████▉ | 170/250 [01:08<00:32, 2.43it/s]
68%|████████████████████████████ | 171/250 [01:09<00:36, 2.16it/s]
69%|████████████████████████████▏ | 172/250 [01:09<00:34, 2.28it/s]
69%|████████████████████████████▎ | 173/250 [01:09<00:31, 2.47it/s]
70%|████████████████████████████▌ | 174/250 [01:10<00:35, 2.16it/s]
70%|████████████████████████████▋ | 175/250 [01:10<00:31, 2.35it/s]
70%|████████████████████████████▊ | 176/250 [01:11<00:29, 2.51it/s]
71%|█████████████████████████████ | 177/250 [01:11<00:29, 2.46it/s]
71%|█████████████████████████████▏ | 178/250 [01:11<00:27, 2.60it/s]
72%|█████████████████████████████▎ | 179/250 [01:12<00:29, 2.44it/s]
72%|█████████████████████████████▌ | 180/250 [01:12<00:25, 2.70it/s]
72%|█████████████████████████████▋ | 181/250 [01:13<00:27, 2.53it/s]
73%|█████████████████████████████▊ | 182/250 [01:13<00:24, 2.82it/s]
73%|██████████████████████████████ | 183/250 [01:13<00:26, 2.48it/s]
74%|██████████████████████████████▏ | 184/250 [01:14<00:26, 2.51it/s]
74%|██████████████████████████████▎ | 185/250 [01:14<00:30, 2.16it/s]
74%|██████████████████████████████▌ | 186/250 [01:15<00:27, 2.35it/s]
75%|██████████████████████████████▋ | 187/250 [01:15<00:28, 2.22it/s]
75%|██████████████████████████████▊ | 188/250 [01:16<00:26, 2.31it/s]
76%|██████████████████████████████▉ | 189/250 [01:16<00:27, 2.26it/s]
76%|███████████████████████████████▏ | 190/250 [01:16<00:25, 2.38it/s]
76%|███████████████████████████████▎ | 191/250 [01:17<00:23, 2.54it/s]
77%|███████████████████████████████▍ | 192/250 [01:17<00:22, 2.58it/s]
77%|███████████████████████████████▋ | 193/250 [01:18<00:22, 2.51it/s]
78%|███████████████████████████████▊ | 194/250 [01:18<00:21, 2.55it/s]
78%|███████████████████████████████▉ | 195/250 [01:18<00:23, 2.31it/s]
78%|████████████████████████████████▏ | 196/250 [01:19<00:24, 2.24it/s]
79%|████████████████████████████████▎ | 197/250 [01:19<00:23, 2.26it/s]
79%|████████████████████████████████▍ | 198/250 [01:20<00:20, 2.53it/s]
80%|████████████████████████████████▋ | 199/250 [01:20<00:18, 2.75it/s]
80%|████████████████████████████████▊ | 200/250 [01:20<00:18, 2.71it/s]
80%|████████████████████████████████▉ | 201/250 [01:21<00:19, 2.50it/s]
81%|█████████████████████████████████▏ | 202/250 [01:21<00:19, 2.49it/s]
81%|█████████████████████████████████▎ | 203/250 [01:22<00:17, 2.72it/s]
82%|█████████████████████████████████▍ | 204/250 [01:22<00:16, 2.72it/s]
82%|█████████████████████████████████▌ | 205/250 [01:22<00:15, 2.93it/s]
82%|█████████████████████████████████▊ | 206/250 [01:23<00:15, 2.80it/s]
83%|█████████████████████████████████▉ | 207/250 [01:23<00:14, 2.89it/s]
83%|██████████████████████████████████ | 208/250 [01:23<00:17, 2.36it/s]
84%|██████████████████████████████████▎ | 209/250 [01:24<00:16, 2.43it/s]
84%|██████████████████████████████████▍ | 210/250 [01:24<00:15, 2.62it/s]
84%|██████████████████████████████████▌ | 211/250 [01:25<00:16, 2.30it/s]
85%|██████████████████████████████████▊ | 212/250 [01:25<00:15, 2.44it/s]
85%|██████████████████████████████████▉ | 213/250 [01:25<00:14, 2.50it/s]
86%|███████████████████████████████████ | 214/250 [01:26<00:16, 2.18it/s]
86%|███████████████████████████████████▎ | 215/250 [01:26<00:15, 2.28it/s]
86%|███████████████████████████████████▍ | 216/250 [01:27<00:13, 2.47it/s]
87%|███████████████████████████████████▌ | 217/250 [01:27<00:13, 2.51it/s]
87%|███████████████████████████████████▊ | 218/250 [01:28<00:12, 2.63it/s]
88%|███████████████████████████████████▉ | 219/250 [01:28<00:13, 2.31it/s]
88%|████████████████████████████████████ | 220/250 [01:28<00:11, 2.54it/s]
88%|████████████████████████████████████▏ | 221/250 [01:29<00:11, 2.63it/s]
89%|████████████████████████████████████▍ | 222/250 [01:29<00:11, 2.50it/s]
89%|████████████████████████████████████▌ | 223/250 [01:29<00:10, 2.67it/s]
90%|████████████████████████████████████▋ | 224/250 [01:30<00:09, 2.75it/s]
90%|████████████████████████████████████▉ | 225/250 [01:30<00:08, 2.82it/s]
90%|█████████████████████████████████████ | 226/250 [01:31<00:11, 2.13it/s]
91%|█████████████████████████████████████▏ | 227/250 [01:32<00:13, 1.69it/s]
91%|█████████████████████████████████████▍ | 228/250 [01:32<00:11, 1.95it/s]
92%|█████████████████████████████████████▌ | 229/250 [01:32<00:09, 2.21it/s]
92%|█████████████████████████████████████▋ | 230/250 [01:33<00:08, 2.39it/s]
92%|█████████████████████████████████████▉ | 231/250 [01:33<00:07, 2.53it/s]
93%|██████████████████████████████████████ | 232/250 [01:34<00:07, 2.44it/s]
93%|██████████████████████████████████████▏ | 233/250 [01:34<00:06, 2.57it/s]
94%|██████████████████████████████████████▍ | 234/250 [01:35<00:07, 2.11it/s]
94%|██████████████████████████████████████▌ | 235/250 [01:35<00:06, 2.31it/s]
94%|██████████████████████████████████████▋ | 236/250 [01:35<00:05, 2.43it/s]
95%|██████████████████████████████████████▊ | 237/250 [01:36<00:06, 2.17it/s]
95%|███████████████████████████████████████ | 238/250 [01:36<00:04, 2.46it/s]
96%|███████████████████████████████████████▏ | 239/250 [01:37<00:05, 2.17it/s]
96%|███████████████████████████████████████▎ | 240/250 [01:38<00:05, 1.71it/s]
96%|███████████████████████████████████████▌ | 241/250 [01:38<00:04, 1.92it/s]
97%|███████████████████████████████████████▋ | 242/250 [01:38<00:03, 2.18it/s]
97%|███████████████████████████████████████▊ | 243/250 [01:39<00:03, 2.31it/s]
98%|████████████████████████████████████████ | 244/250 [01:39<00:03, 1.96it/s]
98%|████████████████████████████████████████▏| 245/250 [01:40<00:02, 2.23it/s]
98%|████████████████████████████████████████▎| 246/250 [01:40<00:01, 2.42it/s]
99%|████████████████████████████████████████▌| 247/250 [01:40<00:01, 2.40it/s]
99%|████████████████████████████████████████▋| 248/250 [01:41<00:00, 2.39it/s]
100%|████████████████████████████████████████▊| 249/250 [01:41<00:00, 2.14it/s]
{'eval_loss': 1.075204849243164, 'eval_runtime': 103.3766, 'eval_samples_per_second': 19.347, 'eval_steps_per_second': 2.418, 'epoch': 2.0}
100%|███████████████████████████████████████| 780/780 [3:13:16<00:00, 27.31s/it]
100%|█████████████████████████████████████████| 250/250 [01:42<00:00, 1.78it/s]
There were missing keys in the checkpoint model loaded: ['base_model.model.model.embed_tokens.weight', 'base_model.model.model.layers.0.self_attn.q_proj.weight', 'base_model.model.model.layers.0.self_attn.k_proj.weight', 'base_model.model.model.layers.0.self_attn.v_proj.weight', 'base_model.model.model.layers.0.self_attn.o_proj.weight', 'base_model.model.model.layers.0.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.0.mlp.gate_proj.weight', 'base_model.model.model.layers.0.mlp.down_proj.weight', 'base_model.model.model.layers.0.mlp.up_proj.weight', 'base_model.model.model.layers.0.input_layernorm.weight', 'base_model.model.model.layers.0.post_attention_layernorm.weight', 'base_model.model.model.layers.1.self_attn.q_proj.weight', 'base_model.model.model.layers.1.self_attn.k_proj.weight', 'base_model.model.model.layers.1.self_attn.v_proj.weight', 'base_model.model.model.layers.1.self_attn.o_proj.weight', 'base_model.model.model.layers.1.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.1.mlp.gate_proj.weight', 'base_model.model.model.layers.1.mlp.down_proj.weight', 'base_model.model.model.layers.1.mlp.up_proj.weight', 'base_model.model.model.layers.1.input_layernorm.weight', 'base_model.model.model.layers.1.post_attention_layernorm.weight', 'base_model.model.model.layers.2.self_attn.q_proj.weight', 'base_model.model.model.layers.2.self_attn.k_proj.weight', 'base_model.model.model.layers.2.self_attn.v_proj.weight', 'base_model.model.model.layers.2.self_attn.o_proj.weight', 'base_model.model.model.layers.2.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.2.mlp.gate_proj.weight', 'base_model.model.model.layers.2.mlp.down_proj.weight', 'base_model.model.model.layers.2.mlp.up_proj.weight', 'base_model.model.model.layers.2.input_layernorm.weight', 'base_model.model.model.layers.2.post_attention_layernorm.weight', 'base_model.model.model.layers.3.self_attn.q_proj.weight', 'base_model.model.model.layers.3.self_attn.k_proj.weight', 'base_model.model.model.layers.3.self_attn.v_proj.weight', 'base_model.model.model.layers.3.self_attn.o_proj.weight', 'base_model.model.model.layers.3.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.3.mlp.gate_proj.weight', 'base_model.model.model.layers.3.mlp.down_proj.weight', 'base_model.model.model.layers.3.mlp.up_proj.weight', 'base_model.model.model.layers.3.input_layernorm.weight', 'base_model.model.model.layers.3.post_attention_layernorm.weight', 'base_model.model.model.layers.4.self_attn.q_proj.weight', 'base_model.model.model.layers.4.self_attn.k_proj.weight', 'base_model.model.model.layers.4.self_attn.v_proj.weight', 'base_model.model.model.layers.4.self_attn.o_proj.weight', 'base_model.model.model.layers.4.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.4.mlp.gate_proj.weight', 'base_model.model.model.layers.4.mlp.down_proj.weight', 'base_model.model.model.layers.4.mlp.up_proj.weight', 'base_model.model.model.layers.4.input_layernorm.weight', 'base_model.model.model.layers.4.post_attention_layernorm.weight', 'base_model.model.model.layers.5.self_attn.q_proj.weight', 'base_model.model.model.layers.5.self_attn.k_proj.weight', 'base_model.model.model.layers.5.self_attn.v_proj.weight', 'base_model.model.model.layers.5.self_attn.o_proj.weight', 'base_model.model.model.layers.5.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.5.mlp.gate_proj.weight', 'base_model.model.model.layers.5.mlp.down_proj.weight', 'base_model.model.model.layers.5.mlp.up_proj.weight', 'base_model.model.model.layers.5.input_layernorm.weight', 'base_model.model.model.layers.5.post_attention_layernorm.weight', 'base_model.model.model.layers.6.self_attn.q_proj.weight', 'base_model.model.model.layers.6.self_attn.k_proj.weight', 'base_model.model.model.layers.6.self_attn.v_proj.weight', 'base_model.model.model.layers.6.self_attn.o_proj.weight', 'base_model.model.model.layers.6.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.6.mlp.gate_proj.weight', 'base_model.model.model.layers.6.mlp.down_proj.weight', 'base_model.model.model.layers.6.mlp.up_proj.weight', 'base_model.model.model.layers.6.input_layernorm.weight', 'base_model.model.model.layers.6.post_attention_layernorm.weight', 'base_model.model.model.layers.7.self_attn.q_proj.weight', 'base_model.model.model.layers.7.self_attn.k_proj.weight', 'base_model.model.model.layers.7.self_attn.v_proj.weight', 'base_model.model.model.layers.7.self_attn.o_proj.weight', 'base_model.model.model.layers.7.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.7.mlp.gate_proj.weight', 'base_model.model.model.layers.7.mlp.down_proj.weight', 'base_model.model.model.layers.7.mlp.up_proj.weight', 'base_model.model.model.layers.7.input_layernorm.weight', 'base_model.model.model.layers.7.post_attention_layernorm.weight', 'base_model.model.model.layers.8.self_attn.q_proj.weight', 'base_model.model.model.layers.8.self_attn.k_proj.weight', 'base_model.model.model.layers.8.self_attn.v_proj.weight', 'base_model.model.model.layers.8.self_attn.o_proj.weight', 'base_model.model.model.layers.8.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.8.mlp.gate_proj.weight', 'base_model.model.model.layers.8.mlp.down_proj.weight', 'base_model.model.model.layers.8.mlp.up_proj.weight', 'base_model.model.model.layers.8.input_layernorm.weight', 'base_model.model.model.layers.8.post_attention_layernorm.weight', 'base_model.model.model.layers.9.self_attn.q_proj.weight', 'base_model.model.model.layers.9.self_attn.k_proj.weight', 'base_model.model.model.layers.9.self_attn.v_proj.weight', 'base_model.model.model.layers.9.self_attn.o_proj.weight', 'base_model.model.model.layers.9.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.9.mlp.gate_proj.weight', 'base_model.model.model.layers.9.mlp.down_proj.weight', 'base_model.model.model.layers.9.mlp.up_proj.weight', 'base_model.model.model.layers.9.input_layernorm.weight', 'base_model.model.model.layers.9.post_attention_layernorm.weight', 'base_model.model.model.layers.10.self_attn.q_proj.weight', 'base_model.model.model.layers.10.self_attn.k_proj.weight', 'base_model.model.model.layers.10.self_attn.v_proj.weight', 'base_model.model.model.layers.10.self_attn.o_proj.weight', 'base_model.model.model.layers.10.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.10.mlp.gate_proj.weight', 'base_model.model.model.layers.10.mlp.down_proj.weight', 'base_model.model.model.layers.10.mlp.up_proj.weight', 'base_model.model.model.layers.10.input_layernorm.weight', 'base_model.model.model.layers.10.post_attention_layernorm.weight', 'base_model.model.model.layers.11.self_attn.q_proj.weight', 'base_model.model.model.layers.11.self_attn.k_proj.weight', 'base_model.model.model.layers.11.self_attn.v_proj.weight', 'base_model.model.model.layers.11.self_attn.o_proj.weight', 'base_model.model.model.layers.11.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.11.mlp.gate_proj.weight', 'base_model.model.model.layers.11.mlp.down_proj.weight', 'base_model.model.model.layers.11.mlp.up_proj.weight', 'base_model.model.model.layers.11.input_layernorm.weight', 'base_model.model.model.layers.11.post_attention_layernorm.weight', 'base_model.model.model.layers.12.self_attn.q_proj.weight', 'base_model.model.model.layers.12.self_attn.k_proj.weight', 'base_model.model.model.layers.12.self_attn.v_proj.weight', 'base_model.model.model.layers.12.self_attn.o_proj.weight', 'base_model.model.model.layers.12.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.12.mlp.gate_proj.weight', 'base_model.model.model.layers.12.mlp.down_proj.weight', 'base_model.model.model.layers.12.mlp.up_proj.weight', 'base_model.model.model.layers.12.input_layernorm.weight', 'base_model.model.model.layers.12.post_attention_layernorm.weight', 'base_model.model.model.layers.13.self_attn.q_proj.weight', 'base_model.model.model.layers.13.self_attn.k_proj.weight', 'base_model.model.model.layers.13.self_attn.v_proj.weight', 'base_model.model.model.layers.13.self_attn.o_proj.weight', 'base_model.model.model.layers.13.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.13.mlp.gate_proj.weight', 'base_model.model.model.layers.13.mlp.down_proj.weight', 'base_model.model.model.layers.13.mlp.up_proj.weight', 'base_model.model.model.layers.13.input_layernorm.weight', 'base_model.model.model.layers.13.post_attention_layernorm.weight', 'base_model.model.model.layers.14.self_attn.q_proj.weight', 'base_model.model.model.layers.14.self_attn.k_proj.weight', 'base_model.model.model.layers.14.self_attn.v_proj.weight', 'base_model.model.model.layers.14.self_attn.o_proj.weight', 'base_model.model.model.layers.14.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.14.mlp.gate_proj.weight', 'base_model.model.model.layers.14.mlp.down_proj.weight', 'base_model.model.model.layers.14.mlp.up_proj.weight', 'base_model.model.model.layers.14.input_layernorm.weight', 'base_model.model.model.layers.14.post_attention_layernorm.weight', 'base_model.model.model.layers.15.self_attn.q_proj.weight', 'base_model.model.model.layers.15.self_attn.k_proj.weight', 'base_model.model.model.layers.15.self_attn.v_proj.weight', 'base_model.model.model.layers.15.self_attn.o_proj.weight', 'base_model.model.model.layers.15.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.15.mlp.gate_proj.weight', 'base_model.model.model.layers.15.mlp.down_proj.weight', 'base_model.model.model.layers.15.mlp.up_proj.weight', 'base_model.model.model.layers.15.input_layernorm.weight', 'base_model.model.model.layers.15.post_attention_layernorm.weight', 'base_model.model.model.layers.16.self_attn.q_proj.weight', 'base_model.model.model.layers.16.self_attn.k_proj.weight', 'base_model.model.model.layers.16.self_attn.v_proj.weight', 'base_model.model.model.layers.16.self_attn.o_proj.weight', 'base_model.model.model.layers.16.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.16.mlp.gate_proj.weight', 'base_model.model.model.layers.16.mlp.down_proj.weight', 'base_model.model.model.layers.16.mlp.up_proj.weight', 'base_model.model.model.layers.16.input_layernorm.weight', 'base_model.model.model.layers.16.post_attention_layernorm.weight', 'base_model.model.model.layers.17.self_attn.q_proj.weight', 'base_model.model.model.layers.17.self_attn.k_proj.weight', 'base_model.model.model.layers.17.self_attn.v_proj.weight', 'base_model.model.model.layers.17.self_attn.o_proj.weight', 'base_model.model.model.layers.17.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.17.mlp.gate_proj.weight', 'base_model.model.model.layers.17.mlp.down_proj.weight', 'base_model.model.model.layers.17.mlp.up_proj.weight', 'base_model.model.model.layers.17.input_layernorm.weight', 'base_model.model.model.layers.17.post_attention_layernorm.weight', 'base_model.model.model.layers.18.self_attn.q_proj.weight', 'base_model.model.model.layers.18.self_attn.k_proj.weight', 'base_model.model.model.layers.18.self_attn.v_proj.weight', 'base_model.model.model.layers.18.self_attn.o_proj.weight', 'base_model.model.model.layers.18.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.18.mlp.gate_proj.weight', 'base_model.model.model.layers.18.mlp.down_proj.weight', 'base_model.model.model.layers.18.mlp.up_proj.weight', 'base_model.model.model.layers.18.input_layernorm.weight', 'base_model.model.model.layers.18.post_attention_layernorm.weight', 'base_model.model.model.layers.19.self_attn.q_proj.weight', 'base_model.model.model.layers.19.self_attn.k_proj.weight', 'base_model.model.model.layers.19.self_attn.v_proj.weight', 'base_model.model.model.layers.19.self_attn.o_proj.weight', 'base_model.model.model.layers.19.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.19.mlp.gate_proj.weight', 'base_model.model.model.layers.19.mlp.down_proj.weight', 'base_model.model.model.layers.19.mlp.up_proj.weight', 'base_model.model.model.layers.19.input_layernorm.weight', 'base_model.model.model.layers.19.post_attention_layernorm.weight', 'base_model.model.model.layers.20.self_attn.q_proj.weight', 'base_model.model.model.layers.20.self_attn.k_proj.weight', 'base_model.model.model.layers.20.self_attn.v_proj.weight', 'base_model.model.model.layers.20.self_attn.o_proj.weight', 'base_model.model.model.layers.20.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.20.mlp.gate_proj.weight', 'base_model.model.model.layers.20.mlp.down_proj.weight', 'base_model.model.model.layers.20.mlp.up_proj.weight', 'base_model.model.model.layers.20.input_layernorm.weight', 'base_model.model.model.layers.20.post_attention_layernorm.weight', 'base_model.model.model.layers.21.self_attn.q_proj.weight', 'base_model.model.model.layers.21.self_attn.k_proj.weight', 'base_model.model.model.layers.21.self_attn.v_proj.weight', 'base_model.model.model.layers.21.self_attn.o_proj.weight', 'base_model.model.model.layers.21.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.21.mlp.gate_proj.weight', 'base_model.model.model.layers.21.mlp.down_proj.weight', 'base_model.model.model.layers.21.mlp.up_proj.weight', 'base_model.model.model.layers.21.input_layernorm.weight', 'base_model.model.model.layers.21.post_attention_layernorm.weight', 'base_model.model.model.layers.22.self_attn.q_proj.weight', 'base_model.model.model.layers.22.self_attn.k_proj.weight', 'base_model.model.model.layers.22.self_attn.v_proj.weight', 'base_model.model.model.layers.22.self_attn.o_proj.weight', 'base_model.model.model.layers.22.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.22.mlp.gate_proj.weight', 'base_model.model.model.layers.22.mlp.down_proj.weight', 'base_model.model.model.layers.22.mlp.up_proj.weight', 'base_model.model.model.layers.22.input_layernorm.weight', 'base_model.model.model.layers.22.post_attention_layernorm.weight', 'base_model.model.model.layers.23.self_attn.q_proj.weight', 'base_model.model.model.layers.23.self_attn.k_proj.weight', 'base_model.model.model.layers.23.self_attn.v_proj.weight', 'base_model.model.model.layers.23.self_attn.o_proj.weight', 'base_model.model.model.layers.23.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.23.mlp.gate_proj.weight', 'base_model.model.model.layers.23.mlp.down_proj.weight', 'base_model.model.model.layers.23.mlp.up_proj.weight', 'base_model.model.model.layers.23.input_layernorm.weight', 'base_model.model.model.layers.23.post_attention_layernorm.weight', 'base_model.model.model.layers.24.self_attn.q_proj.weight', 'base_model.model.model.layers.24.self_attn.k_proj.weight', 'base_model.model.model.layers.24.self_attn.v_proj.weight', 'base_model.model.model.layers.24.self_attn.o_proj.weight', 'base_model.model.model.layers.24.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.24.mlp.gate_proj.weight', 'base_model.model.model.layers.24.mlp.down_proj.weight', 'base_model.model.model.layers.24.mlp.up_proj.weight', 'base_model.model.model.layers.24.input_layernorm.weight', 'base_model.model.model.layers.24.post_attention_layernorm.weight', 'base_model.model.model.layers.25.self_attn.q_proj.weight', 'base_model.model.model.layers.25.self_attn.k_proj.weight', 'base_model.model.model.layers.25.self_attn.v_proj.weight', 'base_model.model.model.layers.25.self_attn.o_proj.weight', 'base_model.model.model.layers.25.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.25.mlp.gate_proj.weight', 'base_model.model.model.layers.25.mlp.down_proj.weight', 'base_model.model.model.layers.25.mlp.up_proj.weight', 'base_model.model.model.layers.25.input_layernorm.weight', 'base_model.model.model.layers.25.post_attention_layernorm.weight', 'base_model.model.model.layers.26.self_attn.q_proj.weight', 'base_model.model.model.layers.26.self_attn.k_proj.weight', 'base_model.model.model.layers.26.self_attn.v_proj.weight', 'base_model.model.model.layers.26.self_attn.o_proj.weight', 'base_model.model.model.layers.26.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.26.mlp.gate_proj.weight', 'base_model.model.model.layers.26.mlp.down_proj.weight', 'base_model.model.model.layers.26.mlp.up_proj.weight', 'base_model.model.model.layers.26.input_layernorm.weight', 'base_model.model.model.layers.26.post_attention_layernorm.weight', 'base_model.model.model.layers.27.self_attn.q_proj.weight', 'base_model.model.model.layers.27.self_attn.k_proj.weight', 'base_model.model.model.layers.27.self_attn.v_proj.weight', 'base_model.model.model.layers.27.self_attn.o_proj.weight', 'base_model.model.model.layers.27.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.27.mlp.gate_proj.weight', 'base_model.model.model.layers.27.mlp.down_proj.weight', 'base_model.model.model.layers.27.mlp.up_proj.weight', 'base_model.model.model.layers.27.input_layernorm.weight', 'base_model.model.model.layers.27.post_attention_layernorm.weight', 'base_model.model.model.layers.28.self_attn.q_proj.weight', 'base_model.model.model.layers.28.self_attn.k_proj.weight', 'base_model.model.model.layers.28.self_attn.v_proj.weight', 'base_model.model.model.layers.28.self_attn.o_proj.weight', 'base_model.model.model.layers.28.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.28.mlp.gate_proj.weight', 'base_model.model.model.layers.28.mlp.down_proj.weight', 'base_model.model.model.layers.28.mlp.up_proj.weight', 'base_model.model.model.layers.28.input_layernorm.weight', 'base_model.model.model.layers.28.post_attention_layernorm.weight', 'base_model.model.model.layers.29.self_attn.q_proj.weight', 'base_model.model.model.layers.29.self_attn.k_proj.weight', 'base_model.model.model.layers.29.self_attn.v_proj.weight', 'base_model.model.model.layers.29.self_attn.o_proj.weight', 'base_model.model.model.layers.29.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.29.mlp.gate_proj.weight', 'base_model.model.model.layers.29.mlp.down_proj.weight', 'base_model.model.model.layers.29.mlp.up_proj.weight', 'base_model.model.model.layers.29.input_layernorm.weight', 'base_model.model.model.layers.29.post_attention_layernorm.weight', 'base_model.model.model.layers.30.self_attn.q_proj.weight', 'base_model.model.model.layers.30.self_attn.k_proj.weight', 'base_model.model.model.layers.30.self_attn.v_proj.weight', 'base_model.model.model.layers.30.self_attn.o_proj.weight', 'base_model.model.model.layers.30.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.30.mlp.gate_proj.weight', 'base_model.model.model.layers.30.mlp.down_proj.weight', 'base_model.model.model.layers.30.mlp.up_proj.weight', 'base_model.model.model.layers.30.input_layernorm.weight', 'base_model.model.model.layers.30.post_attention_layernorm.weight', 'base_model.model.model.layers.31.self_attn.q_proj.weight', 'base_model.model.model.layers.31.self_attn.k_proj.weight', 'base_model.model.model.layers.31.self_attn.v_proj.weight', 'base_model.model.model.layers.31.self_attn.o_proj.weight', 'base_model.model.model.layers.31.self_attn.rotary_emb.inv_freq', 'base_model.model.model.layers.31.mlp.gate_proj.weight', 'base_model.model.model.layers.31.mlp.down_proj.weight', 'base_model.model.model.layers.31.mlp.up_proj.weight', 'base_model.model.model.layers.31.input_layernorm.weight', 'base_model.model.model.layers.31.post_attention_layernorm.weight', 'base_model.model.model.norm.weight', 'base_model.model.lm_head.0.weight'].
{'train_runtime': 11598.6938, 'train_samples_per_second': 8.612, 'train_steps_per_second': 0.067, 'train_loss': 0.52858656002925, 'epoch': 2.0}
100%|███████████████████████████████████████| 780/780 [3:13:16<00:00, 14.87s/it]
If there's a warning about missing keys above, please disregard :)
wandb: Waiting for W&B process to finish... (success).
wandb:
wandb: Run history:
wandb: eval/loss █▇▅▄▄▃▂▂▁▁
wandb: eval/runtime ▆█▃▄▇▃▄▁▅▃
wandb: eval/samples_per_second ▃▁▆▅▂▆▅█▄▆
wandb: eval/steps_per_second ▂▁▅▄▁▅▄█▄▅
wandb: train/epoch ▁▁▂▂▂▂▂▃▃▃▄▄▄▄▄▅▅▅▅▆▆▆▆▇▇▇▇████
wandb: train/global_step ▁▁▂▂▂▂▂▃▃▃▄▄▄▄▄▅▅▅▅▆▆▆▇▇▇▇▇████
wandb: train/learning_rate ██▇▇▇▆▆▅▅▅▄▄▄▃▃▂▂▂▁▁
wandb: train/loss ▃▅▄▂█▇▂▃▇▆▇▁▁▆▄▅▁▂▂▄
wandb: train/total_flos ▁
wandb: train/train_loss ▁
wandb: train/train_runtime ▁
wandb: train/train_samples_per_second ▁
wandb: train/train_steps_per_second ▁
wandb:
wandb: Run summary:
wandb: eval/loss 1.0752
wandb: eval/runtime 103.3766
wandb: eval/samples_per_second 19.347
wandb: eval/steps_per_second 2.418
wandb: train/epoch 2.0
wandb: train/global_step 780
wandb: train/learning_rate 0.0
wandb: train/loss 1.0576
wandb: train/total_flos 7.690604908231066e+17
wandb: train/train_loss 0.52859
wandb: train/train_runtime 11598.6938
wandb: train/train_samples_per_second 8.612
wandb: train/train_steps_per_second 0.067
wandb:
wandb: 🚀 View run comfy-lake-6 at: https://wandb.ai/utensil/Alpaca-CoT/runs/j0gxpjnk
wandb: Synced 5 W&B file(s), 0 media file(s), 2 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20230419_055034-j0gxpjnk/logs
In [38]:
%cd /workspace/llm-playground/
/workspace/llm-playground
In [43]:
!export last_checkpoint=`ls -1 -t /workspace/code/saved_models/llama-7b-hf_alpaca/|head -n 1`
!cp -r /workspace/code/saved_models/llama-7b-hf_alpaca/$last_checkpoint storage/saved_models/llama-7b-hf_alpaca/
!ls -lhta storage/saved_models/llama-7b-hf_alpaca/$last_checkpoint
total 8.0K drwxr-xr-x 14 root root 326 Apr 19 09:07 llama-7b-hf_alpaca drwxr-xr-x 5 root root 128 Apr 19 09:07 . -rw-r--r-- 1 root root 443 Apr 19 05:31 adapter_model.bin drwxr-xr-x 2 root root 160 Apr 19 03:56 checkpoint-390 drwxr-xr-x 2 root root 160 Apr 19 03:56 checkpoint-351 drwxr-xr-x 3 root root 32 Apr 19 03:56 .. -rw-r--r-- 1 root root 350 Apr 19 03:56 adapter_config.json
In [44]:
!cp /workspace/code/saved_models/llama-7b-hf_alpaca/adapter* storage/saved_models/llama-7b-hf_alpaca/
!ls -lhta storage/saved_models/llama-7b-hf_alpaca/
total 17M -rw-r--r-- 1 root root 17M Apr 19 09:08 adapter_model.bin -rw-r--r-- 1 root root 370 Apr 19 09:08 adapter_config.json drwxr-xr-x 14 root root 326 Apr 19 09:07 llama-7b-hf_alpaca drwxr-xr-x 5 root root 128 Apr 19 09:07 . drwxr-xr-x 2 root root 160 Apr 19 03:56 checkpoint-390 drwxr-xr-x 2 root root 160 Apr 19 03:56 checkpoint-351 drwxr-xr-x 3 root root 32 Apr 19 03:56 ..
In [45]:
%autosave 1
Autosaving every 1 seconds
In [46]:
!sleep 10
In [ ]:
!cp /workspace/test_resume.ipynb storage/saved_models/llama-7b-hf_alpaca/
In [ ]:
!python3 /workspace/llm-playground/helper/upload.py
In [ ]:
sleep 30 && runpodctl remove pod $RUNPOD_POD_ID