Tested on colab

In [13]:
# download code git repository
# 下载代码仓库
!git clone -b dev https://github.com/utensil/Alpaca-CoT.git code

Cloning into 'code'...
remote: Enumerating objects: 756, done.
remote: Counting objects: 100% (94/94), done.
remote: Compressing objects: 100% (54/54), done.
remote: Total 756 (delta 51), reused 70 (delta 40), pack-reused 662
Receiving objects: 100% (756/756), 127.13 MiB | 14.57 MiB/s, done.
Resolving deltas: 100% (403/403), done.
In [ ]:
!cd code && git pull
In [2]:
# install basic python package
# 安装必要的python库
!python -m pip install bitsandbytes
!python -m pip install datasets
!python -m pip install git+https://github.com/huggingface/transformers.git
!python -m pip install peft
!python -m pip install sentencepiece
!python -m pip install gradio

Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting bitsandbytes
  Downloading bitsandbytes-0.38.1-py3-none-any.whl (104.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.3/104.3 MB 10.0 MB/s eta 0:00:00
Installing collected packages: bitsandbytes
Successfully installed bitsandbytes-0.38.1
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting datasets
  Downloading datasets-2.11.0-py3-none-any.whl (468 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.7/468.7 kB 31.2 MB/s eta 0:00:00
Requirement already satisfied: packaging in /usr/local/lib/python3.9/dist-packages (from datasets) (23.0)
Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.9/dist-packages (from datasets) (2023.4.0)
Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.9/dist-packages (from datasets) (4.65.0)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from datasets) (6.0)
Collecting multiprocess
  Downloading multiprocess-0.70.14-py39-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.9/132.9 kB 12.3 MB/s eta 0:00:00
Requirement already satisfied: pandas in /usr/local/lib/python3.9/dist-packages (from datasets) (1.5.3)
Collecting dill<0.3.7,>=0.3.0
  Downloading dill-0.3.6-py3-none-any.whl (110 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 110.5/110.5 kB 17.8 MB/s eta 0:00:00
Collecting aiohttp
  Downloading aiohttp-3.8.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 8.3 MB/s eta 0:00:00
Collecting xxhash
  Downloading xxhash-3.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (212 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 212.2/212.2 kB 28.4 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.9/dist-packages (from datasets) (1.22.4)
Collecting responses<0.19
  Downloading responses-0.18.0-py3-none-any.whl (38 kB)
Collecting huggingface-hub<1.0.0,>=0.11.0
  Downloading huggingface_hub-0.13.4-py3-none-any.whl (200 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 200.1/200.1 kB 26.7 MB/s eta 0:00:00
Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.9/dist-packages (from datasets) (9.0.0)
Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.9/dist-packages (from datasets) (2.27.1)
Collecting aiosignal>=1.1.2
  Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting multidict<7.0,>=4.5
  Downloading multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB 19.8 MB/s eta 0:00:00
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets) (2.0.12)
Collecting async-timeout<5.0,>=4.0.0a3
  Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting frozenlist>=1.1.1
  Downloading frozenlist-1.3.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (158 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.8/158.8 kB 24.7 MB/s eta 0:00:00
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->datasets) (22.2.0)
Collecting yarl<2.0,>=1.0
  Downloading yarl-1.8.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 264.6/264.6 kB 35.3 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub<1.0.0,>=0.11.0->datasets) (4.5.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from huggingface-hub<1.0.0,>=0.11.0->datasets) (3.11.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->datasets) (2022.12.7)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->datasets) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests>=2.19.0->datasets) (1.26.15)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas->datasets) (2022.7.1)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.9/dist-packages (from pandas->datasets) (2.8.2)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/dist-packages (from python-dateutil>=2.8.1->pandas->datasets) (1.16.0)
Installing collected packages: xxhash, multidict, frozenlist, dill, async-timeout, yarl, responses, multiprocess, huggingface-hub, aiosignal, aiohttp, datasets
Successfully installed aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 datasets-2.11.0 dill-0.3.6 frozenlist-1.3.3 huggingface-hub-0.13.4 multidict-6.0.4 multiprocess-0.70.14 responses-0.18.0 xxhash-3.2.0 yarl-1.8.2
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting git+https://github.com/huggingface/transformers.git
  Cloning https://github.com/huggingface/transformers.git to /tmp/pip-req-build-h5udkdph
  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers.git /tmp/pip-req-build-h5udkdph
  Resolved https://github.com/huggingface/transformers.git to commit dacd34568d1a27b91f84610eab526640ed8f94e0
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/dist-packages (from transformers==4.29.0.dev0) (23.0)
Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.9/dist-packages (from transformers==4.29.0.dev0) (0.13.4)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from transformers==4.29.0.dev0) (6.0)
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.9/dist-packages (from transformers==4.29.0.dev0) (1.22.4)
Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from transformers==4.29.0.dev0) (3.11.0)
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.9/dist-packages (from transformers==4.29.0.dev0) (4.65.0)
Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from transformers==4.29.0.dev0) (2.27.1)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1
  Downloading tokenizers-0.13.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 97.8 MB/s eta 0:00:00
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers==4.29.0.dev0) (2022.10.31)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers==4.29.0.dev0) (4.5.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->transformers==4.29.0.dev0) (2.0.12)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->transformers==4.29.0.dev0) (2022.12.7)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->transformers==4.29.0.dev0) (1.26.15)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->transformers==4.29.0.dev0) (3.4)
Building wheels for collected packages: transformers
  Building wheel for transformers (pyproject.toml) ... done
  Created wheel for transformers: filename=transformers-4.29.0.dev0-py3-none-any.whl size=6928072 sha256=b561c84b3bb0c02eeadfd2fe5a68ac384f22c01bc6b17e1b91a04ee607f3d774
  Stored in directory: /tmp/pip-ephem-wheel-cache-2px47vh8/wheels/f7/92/8c/752ff3bfcd3439805d8bbf641614da38ef3226e127ebea86ee
Successfully built transformers
Installing collected packages: tokenizers, transformers
Successfully installed tokenizers-0.13.3 transformers-4.29.0.dev0
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting peft
  Downloading peft-0.2.0-py3-none-any.whl (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.3/40.3 kB 121.7 kB/s eta 0:00:00
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/dist-packages (from peft) (23.0)
Requirement already satisfied: torch>=1.13.0 in /usr/local/lib/python3.9/dist-packages (from peft) (2.0.0+cu118)
Requirement already satisfied: psutil in /usr/local/lib/python3.9/dist-packages (from peft) (5.9.4)
Collecting accelerate
  Downloading accelerate-0.18.0-py3-none-any.whl (215 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 215.3/215.3 kB 21.2 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.9/dist-packages (from peft) (1.22.4)
Requirement already satisfied: transformers in /usr/local/lib/python3.9/dist-packages (from peft) (4.29.0.dev0)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.9/dist-packages (from peft) (6.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from torch>=1.13.0->peft) (3.11.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.9/dist-packages (from torch>=1.13.0->peft) (3.1.2)
Requirement already satisfied: sympy in /usr/local/lib/python3.9/dist-packages (from torch>=1.13.0->peft) (1.11.1)
Requirement already satisfied: networkx in /usr/local/lib/python3.9/dist-packages (from torch>=1.13.0->peft) (3.1)
Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.9/dist-packages (from torch>=1.13.0->peft) (2.0.0)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.9/dist-packages (from torch>=1.13.0->peft) (4.5.0)
Requirement already satisfied: lit in /usr/local/lib/python3.9/dist-packages (from triton==2.0.0->torch>=1.13.0->peft) (16.0.1)
Requirement already satisfied: cmake in /usr/local/lib/python3.9/dist-packages (from triton==2.0.0->torch>=1.13.0->peft) (3.25.2)
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.9/dist-packages (from transformers->peft) (4.65.0)
Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.9/dist-packages (from transformers->peft) (0.13.3)
Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from transformers->peft) (2.27.1)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers->peft) (2022.10.31)
Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.9/dist-packages (from transformers->peft) (0.13.4)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.9/dist-packages (from jinja2->torch>=1.13.0->peft) (2.1.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->transformers->peft) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->transformers->peft) (2022.12.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->transformers->peft) (2.0.12)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->transformers->peft) (1.26.15)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.9/dist-packages (from sympy->torch>=1.13.0->peft) (1.3.0)
Installing collected packages: accelerate, peft
Successfully installed accelerate-0.18.0 peft-0.2.0
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting sentencepiece
  Downloading sentencepiece-0.1.98-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 54.2 MB/s eta 0:00:00
Installing collected packages: sentencepiece
Successfully installed sentencepiece-0.1.98
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting gradio
  Downloading gradio-3.27.0-py3-none-any.whl (17.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 MB 77.6 MB/s eta 0:00:00
Requirement already satisfied: jinja2 in /usr/local/lib/python3.9/dist-packages (from gradio) (3.1.2)
Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.9/dist-packages (from gradio) (2.2.0)
Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from gradio) (2.27.1)
Collecting fastapi
  Downloading fastapi-0.95.1-py3-none-any.whl (56 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.0/57.0 kB 9.2 MB/s eta 0:00:00
Collecting uvicorn
  Downloading uvicorn-0.21.1-py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.8/57.8 kB 9.7 MB/s eta 0:00:00
Requirement already satisfied: matplotlib in /usr/local/lib/python3.9/dist-packages (from gradio) (3.7.1)
Collecting httpx
  Downloading httpx-0.24.0-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.3/75.3 kB 13.2 MB/s eta 0:00:00
Requirement already satisfied: numpy in /usr/local/lib/python3.9/dist-packages (from gradio) (1.22.4)
Collecting mdit-py-plugins<=0.3.3
  Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.5/50.5 kB 8.4 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.9/dist-packages (from gradio) (4.5.0)
Requirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.9/dist-packages (from gradio) (4.2.2)
Requirement already satisfied: markupsafe in /usr/local/lib/python3.9/dist-packages (from gradio) (2.1.2)
Collecting orjson
  Downloading orjson-3.8.10-cp39-cp39-manylinux_2_28_x86_64.whl (140 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.5/140.5 kB 22.5 MB/s eta 0:00:00
Requirement already satisfied: pillow in /usr/local/lib/python3.9/dist-packages (from gradio) (8.4.0)
Collecting semantic-version
  Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Collecting websockets>=10.0
  Downloading websockets-11.0.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 129.5/129.5 kB 21.7 MB/s eta 0:00:00
Requirement already satisfied: pydantic in /usr/local/lib/python3.9/dist-packages (from gradio) (1.10.7)
Collecting gradio-client>=0.1.3
  Downloading gradio_client-0.1.3-py3-none-any.whl (286 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 286.2/286.2 kB 36.1 MB/s eta 0:00:00
Requirement already satisfied: huggingface-hub>=0.13.0 in /usr/local/lib/python3.9/dist-packages (from gradio) (0.13.4)
Collecting aiofiles
  Downloading aiofiles-23.1.0-py3-none-any.whl (14 kB)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.9/dist-packages (from gradio) (3.8.4)
Requirement already satisfied: pandas in /usr/local/lib/python3.9/dist-packages (from gradio) (1.5.3)
Collecting pydub
  Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Collecting ffmpy
  Downloading ffmpy-0.3.0.tar.gz (4.8 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: pyyaml in /usr/local/lib/python3.9/dist-packages (from gradio) (6.0)
Collecting python-multipart
  Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.7/45.7 kB 7.7 MB/s eta 0:00:00
Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.9/dist-packages (from altair>=4.2.0->gradio) (4.3.3)
Requirement already satisfied: toolz in /usr/local/lib/python3.9/dist-packages (from altair>=4.2.0->gradio) (0.12.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.9/dist-packages (from altair>=4.2.0->gradio) (0.4)
Requirement already satisfied: fsspec in /usr/local/lib/python3.9/dist-packages (from gradio-client>=0.1.3->gradio) (2023.4.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.9/dist-packages (from gradio-client>=0.1.3->gradio) (23.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from huggingface-hub>=0.13.0->gradio) (3.11.0)
Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub>=0.13.0->gradio) (4.65.0)
Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.9/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio) (0.1.2)
Collecting linkify-it-py<3,>=1
  Downloading linkify_it_py-2.0.0-py3-none-any.whl (19 kB)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas->gradio) (2022.7.1)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.9/dist-packages (from pandas->gradio) (2.8.2)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.9/dist-packages (from aiohttp->gradio) (1.3.3)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->gradio) (22.2.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->gradio) (1.8.2)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.9/dist-packages (from aiohttp->gradio) (4.0.2)
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->gradio) (2.0.12)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.9/dist-packages (from aiohttp->gradio) (1.3.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.9/dist-packages (from aiohttp->gradio) (6.0.4)
Collecting starlette<0.27.0,>=0.26.1
  Downloading starlette-0.26.1-py3-none-any.whl (66 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.9/66.9 kB 11.3 MB/s eta 0:00:00
Requirement already satisfied: idna in /usr/local/lib/python3.9/dist-packages (from httpx->gradio) (3.4)
Requirement already satisfied: certifi in /usr/local/lib/python3.9/dist-packages (from httpx->gradio) (2022.12.7)
Requirement already satisfied: sniffio in /usr/local/lib/python3.9/dist-packages (from httpx->gradio) (1.3.0)
Collecting httpcore<0.18.0,>=0.15.0
  Downloading httpcore-0.17.0-py3-none-any.whl (70 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.6/70.6 kB 11.7 MB/s eta 0:00:00
Requirement already satisfied: importlib-resources>=3.2.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio) (5.12.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio) (1.4.4)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio) (0.11.0)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio) (1.0.7)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio) (4.39.3)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio) (3.0.9)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->gradio) (1.26.15)
Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.9/dist-packages (from uvicorn->gradio) (8.1.3)
Collecting h11>=0.8
  Downloading h11-0.14.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 8.8 MB/s eta 0:00:00
Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/lib/python3.9/dist-packages (from httpcore<0.18.0,>=0.15.0->httpx->gradio) (3.6.2)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.9/dist-packages (from importlib-resources>=3.2.0->matplotlib->gradio) (3.15.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.9/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio) (0.19.3)
Collecting uc-micro-py
  Downloading uc_micro_py-1.0.1-py3-none-any.whl (6.2 kB)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/dist-packages (from python-dateutil>=2.8.1->pandas->gradio) (1.16.0)
Building wheels for collected packages: ffmpy
  Building wheel for ffmpy (setup.py) ... done
  Created wheel for ffmpy: filename=ffmpy-0.3.0-py3-none-any.whl size=4707 sha256=8c9697ddcd2a37ffdef0bbec0e85c823df62e7631ef39514b0b52fb4776710e9
  Stored in directory: /root/.cache/pip/wheels/91/e2/96/f676aa08bfd789328c6576cd0f1fde4a3d686703bb0c247697
Successfully built ffmpy
Installing collected packages: pydub, ffmpy, websockets, uc-micro-py, semantic-version, python-multipart, orjson, h11, aiofiles, uvicorn, starlette, mdit-py-plugins, linkify-it-py, httpcore, httpx, fastapi, gradio-client, gradio
Successfully installed aiofiles-23.1.0 fastapi-0.95.1 ffmpy-0.3.0 gradio-3.27.0 gradio-client-0.1.3 h11-0.14.0 httpcore-0.17.0 httpx-0.24.0 linkify-it-py-2.0.0 mdit-py-plugins-0.3.3 orjson-3.8.10 pydub-0.25.1 python-multipart-0.0.6 semantic-version-2.10.0 starlette-0.26.1 uc-micro-py-1.0.1 uvicorn-0.21.1 websockets-11.0.1
In [3]:
!pip install wandb

Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting wandb
  Downloading wandb-0.14.2-py3-none-any.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 6.6 MB/s eta 0:00:00
Requirement already satisfied: appdirs>=1.4.3 in /usr/local/lib/python3.9/dist-packages (from wandb) (1.4.4)
Requirement already satisfied: protobuf!=4.21.0,<5,>=3.15.0 in /usr/local/lib/python3.9/dist-packages (from wandb) (3.20.3)
Collecting pathtools
  Downloading pathtools-0.1.2.tar.gz (11 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: PyYAML in /usr/local/lib/python3.9/dist-packages (from wandb) (6.0)
Requirement already satisfied: requests<3,>=2.0.0 in /usr/local/lib/python3.9/dist-packages (from wandb) (2.27.1)
Collecting docker-pycreds>=0.4.0
  Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting GitPython!=3.1.29,>=1.0.0
  Downloading GitPython-3.1.31-py3-none-any.whl (184 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 184.3/184.3 kB 26.1 MB/s eta 0:00:00
Collecting sentry-sdk>=1.0.0
  Downloading sentry_sdk-1.19.1-py2.py3-none-any.whl (199 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.2/199.2 kB 30.5 MB/s eta 0:00:00
Collecting setproctitle
  Downloading setproctitle-1.3.2-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)
Requirement already satisfied: setuptools in /usr/local/lib/python3.9/dist-packages (from wandb) (67.6.1)
Requirement already satisfied: Click!=8.0.0,>=7.0 in /usr/local/lib/python3.9/dist-packages (from wandb) (8.1.3)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.9/dist-packages (from wandb) (4.5.0)
Requirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.9/dist-packages (from wandb) (5.9.4)
Requirement already satisfied: six>=1.4.0 in /usr/local/lib/python3.9/dist-packages (from docker-pycreds>=0.4.0->wandb) (1.16.0)
Collecting gitdb<5,>=4.0.1
  Downloading gitdb-4.0.10-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.7/62.7 kB 6.5 MB/s eta 0:00:00
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests<3,>=2.0.0->wandb) (3.4)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests<3,>=2.0.0->wandb) (2.0.12)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests<3,>=2.0.0->wandb) (2022.12.7)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests<3,>=2.0.0->wandb) (1.26.15)
Collecting smmap<6,>=3.0.1
  Downloading smmap-5.0.0-py3-none-any.whl (24 kB)
Building wheels for collected packages: pathtools
  Building wheel for pathtools (setup.py) ... done
  Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8807 sha256=209057a16004851733e748e779c5a55ba79a6999a0fbef7b2e5a2509ea93b85a
  Stored in directory: /root/.cache/pip/wheels/b7/0a/67/ada2a22079218c75a88361c0782855cc72aebc4d18d0289d05
Successfully built pathtools
Installing collected packages: pathtools, smmap, setproctitle, sentry-sdk, docker-pycreds, gitdb, GitPython, wandb
Successfully installed GitPython-3.1.31 docker-pycreds-0.4.0 gitdb-4.0.10 pathtools-0.1.2 sentry-sdk-1.19.1 setproctitle-1.3.2 smmap-5.0.0 wandb-0.14.2
In [4]:
!wandb login

wandb: Logging into wandb.ai. (Learn how to deploy a W&B server locally: https://wandb.me/wandb-server)
wandb: You can find your API key in your browser here: https://wandb.ai/authorize
wandb: Paste an API key from your profile and hit enter, or press ctrl+c to quit: 
wandb: Appending key for api.wandb.ai to your netrc file: /root/.netrc
In [5]:
%env WANDB_PROJECT=Alpaca-CoT-Colab

env: WANDB_PROJECT=Alpaca-CoT-Colab
In [15]:
# download data git repository
# 拉取数据仓库
!GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/datasets/QingyiSi/Alpaca-CoT data

fatal: destination path 'data' already exists and is not an empty directory.
In [16]:
# pull specify data and move it，change <alpaca/> to your preference
# 从仓库中拉取指定文件放入训练目录下，你可以在 https://huggingface.co/datasets/QingyiSi/Alpaca-CoT 查看数据详情
!cd /content/data && git config core.sparsecheckout true && git config lfs.fetchinclude alpaca/
!cd /content/data && git lfs pull
!cp /content/data/alpaca/*.json /content/code/data/
!ls -lh /content/code/data/

total 44M
-rw-r--r-- 1 root root  22M Apr 18 02:28 alpaca_data_cleaned.json
-rw-r--r-- 1 root root  22M Apr 18 02:28 alpaca_data.json
drwxr-xr-x 2 root root 4.0K Apr 18 02:26 formatted_cot_data
drwxr-xr-x 2 root root 4.0K Apr 18 02:26 origin_cot_data
In [17]:
# instruction finetuning 
# 进行指令微调，如果你想直接体验已有checkpoint，可忽略此步
!cd /content/code && python uniform_finetune.py --model_type llama --model_name_or_path decapoda-research/llama-7b-hf --data alpaca --lora_target_modules q_proj v_proj --per_gpu_train_batch_size 4 --learning_rate 3e-4 --epochs 1 --report_to wandb
    

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /usr/local/lib/python3.9/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: /usr/lib64-nvidia did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
  warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/sys/fs/cgroup/memory.events /var/colab/cgroup/jupyter-children/memory.events')}
  warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('8013'), PosixPath('http'), PosixPath('//172.28.0.1')}
  warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('--listen_host=172.28.0.12 --target_host=172.28.0.12 --tunnel_background_save_url=https'), PosixPath('//colab.research.google.com/tun/m/cc48301118ce562b961b3c22d803539adc1e0c19/gpu-t4-s-30aa7guwwttv1 --tunnel_background_save_delay=10s --tunnel_periodic_background_save_frequency=30m0s --enable_output_coalescing=true --output_coalescing_required=true')}
  warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/env/python')}
  warn(msg)
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('module'), PosixPath('//ipykernel.pylab.backend_inline')}
  warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/usr/local/lib/python3.9/dist-packages/bitsandbytes/cuda_setup/main.py:145: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
  warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 7.5
CUDA SETUP: Detected CUDA version 118
CUDA SETUP: Loading binary /usr/local/lib/python3.9/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so...
2023-04-18 02:28:14.307655: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Namespace(size=None, data='alpaca', local_rank=-1, model_type='llama', model_name_or_path='decapoda-research/llama-7b-hf', per_gpu_train_batch_size=4, gradient_accumulation_steps=32, epochs=1, learning_rate=0.0003, cutoff_len=512, lora_r=8, lora_alpha=16, lora_dropout=0.05, val_set_size=2000, lora_target_modules=['q_proj', 'v_proj'], report_to='wandb')
Downloading and preparing dataset json/default to /root/.cache/huggingface/datasets/json/default-619d3a558a064e25/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e...
Downloading data files: 100% 1/1 [00:00<00:00, 6250.83it/s]
Extracting data files: 100% 1/1 [00:00<00:00, 1238.35it/s]
Dataset json downloaded and prepared to /root/.cache/huggingface/datasets/json/default-619d3a558a064e25/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e. Subsequent calls will reuse this data.
100% 1/1 [00:00<00:00, 223.86it/s]
DatasetDict({
    train: Dataset({
        features: ['instruction', 'input', 'output'],
        num_rows: 51942
    })
})
Overriding torch_dtype=None with `torch_dtype=torch.float16` due to requirements of `bitsandbytes` to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning.
Downloading (…)lve/main/config.json: 100% 427/427 [00:00<00:00, 66.1kB/s]
Downloading (…)model.bin.index.json: 100% 25.5k/25.5k [00:00<00:00, 7.39MB/s]
Downloading shards:   0% 0/33 [00:00<?, ?it/s]
Downloading (…)l-00001-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00001-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 85.0MB/s]
Downloading (…)l-00001-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 82.6MB/s]
Downloading (…)l-00001-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 86.8MB/s]
Downloading (…)l-00001-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 87.6MB/s]
Downloading (…)l-00001-of-00033.bin:  13% 52.4M/405M [00:00<00:04, 84.4MB/s]
Downloading (…)l-00001-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 85.6MB/s]
Downloading (…)l-00001-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 87.3MB/s]
Downloading (…)l-00001-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 88.4MB/s]
Downloading (…)l-00001-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 88.9MB/s]
Downloading (…)l-00001-of-00033.bin:  26% 105M/405M [00:01<00:03, 89.9MB/s] 
Downloading (…)l-00001-of-00033.bin:  28% 115M/405M [00:01<00:03, 89.2MB/s]
Downloading (…)l-00001-of-00033.bin:  31% 126M/405M [00:01<00:03, 90.2MB/s]
Downloading (…)l-00001-of-00033.bin:  34% 136M/405M [00:01<00:02, 91.0MB/s]
Downloading (…)l-00001-of-00033.bin:  36% 147M/405M [00:01<00:02, 89.8MB/s]
Downloading (…)l-00001-of-00033.bin:  39% 157M/405M [00:01<00:02, 89.2MB/s]
Downloading (…)l-00001-of-00033.bin:  41% 168M/405M [00:01<00:02, 90.3MB/s]
Downloading (…)l-00001-of-00033.bin:  44% 178M/405M [00:02<00:02, 89.4MB/s]
Downloading (…)l-00001-of-00033.bin:  47% 189M/405M [00:02<00:02, 90.8MB/s]
Downloading (…)l-00001-of-00033.bin:  49% 199M/405M [00:02<00:02, 91.5MB/s]
Downloading (…)l-00001-of-00033.bin:  52% 210M/405M [00:02<00:02, 89.1MB/s]
Downloading (…)l-00001-of-00033.bin:  54% 220M/405M [00:02<00:02, 88.5MB/s]
Downloading (…)l-00001-of-00033.bin:  57% 231M/405M [00:02<00:01, 87.4MB/s]
Downloading (…)l-00001-of-00033.bin:  60% 241M/405M [00:02<00:01, 88.5MB/s]
Downloading (…)l-00001-of-00033.bin:  62% 252M/405M [00:02<00:01, 89.7MB/s]
Downloading (…)l-00001-of-00033.bin:  65% 262M/405M [00:02<00:01, 91.3MB/s]
Downloading (…)l-00001-of-00033.bin:  67% 273M/405M [00:03<00:01, 91.4MB/s]
Downloading (…)l-00001-of-00033.bin:  70% 283M/405M [00:03<00:01, 91.7MB/s]
Downloading (…)l-00001-of-00033.bin:  73% 294M/405M [00:03<00:01, 90.4MB/s]
Downloading (…)l-00001-of-00033.bin:  75% 304M/405M [00:03<00:01, 91.4MB/s]
Downloading (…)l-00001-of-00033.bin:  78% 315M/405M [00:03<00:00, 91.6MB/s]
Downloading (…)l-00001-of-00033.bin:  80% 325M/405M [00:03<00:00, 91.8MB/s]
Downloading (…)l-00001-of-00033.bin:  83% 336M/405M [00:03<00:00, 91.3MB/s]
Downloading (…)l-00001-of-00033.bin:  85% 346M/405M [00:03<00:00, 91.6MB/s]
Downloading (…)l-00001-of-00033.bin:  88% 357M/405M [00:03<00:00, 91.2MB/s]
Downloading (…)l-00001-of-00033.bin:  91% 367M/405M [00:04<00:00, 92.0MB/s]
Downloading (…)l-00001-of-00033.bin:  93% 377M/405M [00:04<00:00, 91.7MB/s]
Downloading (…)l-00001-of-00033.bin:  96% 388M/405M [00:04<00:00, 91.1MB/s]
Downloading (…)l-00001-of-00033.bin: 100% 405M/405M [00:04<00:00, 89.7MB/s]
Downloading shards:   3% 1/33 [00:04<02:33,  4.81s/it]
Downloading (…)l-00002-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00002-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 90.7MB/s]
Downloading (…)l-00002-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 87.2MB/s]
Downloading (…)l-00002-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 88.6MB/s]
Downloading (…)l-00002-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 89.3MB/s]
Downloading (…)l-00002-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 89.8MB/s]
Downloading (…)l-00002-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 88.3MB/s]
Downloading (…)l-00002-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 88.7MB/s]
Downloading (…)l-00002-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 89.1MB/s]
Downloading (…)l-00002-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 90.0MB/s]
Downloading (…)l-00002-of-00033.bin:  26% 105M/405M [00:01<00:03, 90.8MB/s] 
Downloading (…)l-00002-of-00033.bin:  28% 115M/405M [00:01<00:03, 91.8MB/s]
Downloading (…)l-00002-of-00033.bin:  31% 126M/405M [00:01<00:03, 89.0MB/s]
Downloading (…)l-00002-of-00033.bin:  34% 136M/405M [00:01<00:02, 89.7MB/s]
Downloading (…)l-00002-of-00033.bin:  36% 147M/405M [00:01<00:02, 88.8MB/s]
Downloading (…)l-00002-of-00033.bin:  39% 157M/405M [00:01<00:02, 89.8MB/s]
Downloading (…)l-00002-of-00033.bin:  41% 168M/405M [00:01<00:02, 88.6MB/s]
Downloading (…)l-00002-of-00033.bin:  44% 178M/405M [00:01<00:02, 88.9MB/s]
Downloading (…)l-00002-of-00033.bin:  47% 189M/405M [00:02<00:02, 86.8MB/s]
Downloading (…)l-00002-of-00033.bin:  49% 199M/405M [00:02<00:02, 87.7MB/s]
Downloading (…)l-00002-of-00033.bin:  52% 210M/405M [00:02<00:02, 88.0MB/s]
Downloading (…)l-00002-of-00033.bin:  54% 220M/405M [00:02<00:02, 88.1MB/s]
Downloading (…)l-00002-of-00033.bin:  57% 231M/405M [00:02<00:01, 87.2MB/s]
Downloading (…)l-00002-of-00033.bin:  60% 241M/405M [00:02<00:02, 76.5MB/s]
Downloading (…)l-00002-of-00033.bin:  65% 262M/405M [00:02<00:01, 88.6MB/s]
Downloading (…)l-00002-of-00033.bin:  67% 273M/405M [00:03<00:01, 88.7MB/s]
Downloading (…)l-00002-of-00033.bin:  70% 283M/405M [00:03<00:01, 88.8MB/s]
Downloading (…)l-00002-of-00033.bin:  73% 294M/405M [00:03<00:01, 90.6MB/s]
Downloading (…)l-00002-of-00033.bin:  75% 304M/405M [00:03<00:01, 91.8MB/s]
Downloading (…)l-00002-of-00033.bin:  78% 315M/405M [00:03<00:00, 91.1MB/s]
Downloading (…)l-00002-of-00033.bin:  80% 325M/405M [00:03<00:00, 91.5MB/s]
Downloading (…)l-00002-of-00033.bin:  83% 336M/405M [00:03<00:00, 91.7MB/s]
Downloading (…)l-00002-of-00033.bin:  85% 346M/405M [00:03<00:00, 88.9MB/s]
Downloading (…)l-00002-of-00033.bin:  88% 357M/405M [00:04<00:00, 87.6MB/s]
Downloading (…)l-00002-of-00033.bin:  91% 367M/405M [00:04<00:00, 89.3MB/s]
Downloading (…)l-00002-of-00033.bin:  93% 377M/405M [00:04<00:00, 89.1MB/s]
Downloading (…)l-00002-of-00033.bin:  96% 388M/405M [00:04<00:00, 89.3MB/s]
Downloading (…)l-00002-of-00033.bin: 100% 405M/405M [00:04<00:00, 88.7MB/s]
Downloading shards:   6% 2/33 [00:09<02:30,  4.85s/it]
Downloading (…)l-00003-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00003-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 89.4MB/s]
Downloading (…)l-00003-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 88.0MB/s]
Downloading (…)l-00003-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 86.8MB/s]
Downloading (…)l-00003-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 88.0MB/s]
Downloading (…)l-00003-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 88.4MB/s]
Downloading (…)l-00003-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 91.0MB/s]
Downloading (…)l-00003-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 90.4MB/s]
Downloading (…)l-00003-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 89.8MB/s]
Downloading (…)l-00003-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 91.4MB/s]
Downloading (…)l-00003-of-00033.bin:  26% 105M/405M [00:01<00:03, 91.4MB/s] 
Downloading (…)l-00003-of-00033.bin:  28% 115M/405M [00:01<00:03, 92.2MB/s]
Downloading (…)l-00003-of-00033.bin:  31% 126M/405M [00:01<00:03, 91.1MB/s]
Downloading (…)l-00003-of-00033.bin:  34% 136M/405M [00:01<00:02, 91.2MB/s]
Downloading (…)l-00003-of-00033.bin:  36% 147M/405M [00:01<00:02, 91.0MB/s]
Downloading (…)l-00003-of-00033.bin:  39% 157M/405M [00:01<00:02, 89.6MB/s]
Downloading (…)l-00003-of-00033.bin:  41% 168M/405M [00:01<00:02, 89.0MB/s]
Downloading (…)l-00003-of-00033.bin:  44% 178M/405M [00:01<00:02, 88.7MB/s]
Downloading (…)l-00003-of-00033.bin:  47% 189M/405M [00:02<00:02, 89.6MB/s]
Downloading (…)l-00003-of-00033.bin:  49% 199M/405M [00:02<00:02, 88.0MB/s]
Downloading (…)l-00003-of-00033.bin:  52% 210M/405M [00:02<00:02, 84.0MB/s]
Downloading (…)l-00003-of-00033.bin:  54% 220M/405M [00:02<00:02, 85.6MB/s]
Downloading (…)l-00003-of-00033.bin:  57% 231M/405M [00:02<00:01, 87.7MB/s]
Downloading (…)l-00003-of-00033.bin:  60% 241M/405M [00:02<00:01, 88.6MB/s]
Downloading (…)l-00003-of-00033.bin:  62% 252M/405M [00:02<00:01, 87.6MB/s]
Downloading (…)l-00003-of-00033.bin:  65% 262M/405M [00:02<00:01, 89.6MB/s]
Downloading (…)l-00003-of-00033.bin:  67% 273M/405M [00:03<00:01, 91.0MB/s]
Downloading (…)l-00003-of-00033.bin:  70% 283M/405M [00:03<00:01, 90.7MB/s]
Downloading (…)l-00003-of-00033.bin:  73% 294M/405M [00:03<00:01, 86.3MB/s]
Downloading (…)l-00003-of-00033.bin:  75% 304M/405M [00:03<00:01, 88.4MB/s]
Downloading (…)l-00003-of-00033.bin:  78% 315M/405M [00:03<00:01, 89.6MB/s]
Downloading (…)l-00003-of-00033.bin:  80% 325M/405M [00:03<00:00, 87.5MB/s]
Downloading (…)l-00003-of-00033.bin:  83% 336M/405M [00:03<00:00, 87.5MB/s]
Downloading (…)l-00003-of-00033.bin:  85% 346M/405M [00:03<00:00, 86.3MB/s]
Downloading (…)l-00003-of-00033.bin:  88% 357M/405M [00:04<00:00, 88.8MB/s]
Downloading (…)l-00003-of-00033.bin:  91% 367M/405M [00:04<00:00, 87.3MB/s]
Downloading (…)l-00003-of-00033.bin:  93% 377M/405M [00:04<00:00, 87.4MB/s]
Downloading (…)l-00003-of-00033.bin:  96% 388M/405M [00:04<00:00, 88.5MB/s]
Downloading (…)l-00003-of-00033.bin: 100% 405M/405M [00:04<00:00, 88.7MB/s]
Downloading shards:   9% 3/33 [00:15<02:35,  5.17s/it]
Downloading (…)l-00004-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00004-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 93.6MB/s]
Downloading (…)l-00004-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 92.2MB/s]
Downloading (…)l-00004-of-00033.bin:   8% 31.5M/405M [00:00<00:03, 95.0MB/s]
Downloading (…)l-00004-of-00033.bin:  10% 41.9M/405M [00:00<00:03, 94.8MB/s]
Downloading (…)l-00004-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 92.9MB/s]
Downloading (…)l-00004-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 92.4MB/s]
Downloading (…)l-00004-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 90.2MB/s]
Downloading (…)l-00004-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 89.9MB/s]
Downloading (…)l-00004-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 90.5MB/s]
Downloading (…)l-00004-of-00033.bin:  26% 105M/405M [00:01<00:03, 90.7MB/s] 
Downloading (…)l-00004-of-00033.bin:  28% 115M/405M [00:01<00:03, 90.8MB/s]
Downloading (…)l-00004-of-00033.bin:  31% 126M/405M [00:01<00:03, 90.9MB/s]
Downloading (…)l-00004-of-00033.bin:  34% 136M/405M [00:01<00:02, 91.1MB/s]
Downloading (…)l-00004-of-00033.bin:  36% 147M/405M [00:01<00:02, 91.6MB/s]
Downloading (…)l-00004-of-00033.bin:  39% 157M/405M [00:01<00:02, 89.2MB/s]
Downloading (…)l-00004-of-00033.bin:  41% 168M/405M [00:01<00:02, 89.6MB/s]
Downloading (…)l-00004-of-00033.bin:  44% 178M/405M [00:01<00:02, 91.1MB/s]
Downloading (…)l-00004-of-00033.bin:  47% 189M/405M [00:02<00:02, 90.7MB/s]
Downloading (…)l-00004-of-00033.bin:  49% 199M/405M [00:02<00:02, 91.1MB/s]
Downloading (…)l-00004-of-00033.bin:  52% 210M/405M [00:02<00:02, 91.3MB/s]
Downloading (…)l-00004-of-00033.bin:  54% 220M/405M [00:02<00:01, 92.4MB/s]
Downloading (…)l-00004-of-00033.bin:  57% 231M/405M [00:02<00:01, 93.2MB/s]
Downloading (…)l-00004-of-00033.bin:  60% 241M/405M [00:02<00:01, 93.3MB/s]
Downloading (…)l-00004-of-00033.bin:  62% 252M/405M [00:02<00:01, 93.4MB/s]
Downloading (…)l-00004-of-00033.bin:  65% 262M/405M [00:02<00:01, 93.5MB/s]
Downloading (…)l-00004-of-00033.bin:  67% 273M/405M [00:02<00:01, 92.8MB/s]
Downloading (…)l-00004-of-00033.bin:  70% 283M/405M [00:03<00:01, 92.0MB/s]
Downloading (…)l-00004-of-00033.bin:  73% 294M/405M [00:03<00:01, 92.0MB/s]
Downloading (…)l-00004-of-00033.bin:  75% 304M/405M [00:03<00:01, 91.9MB/s]
Downloading (…)l-00004-of-00033.bin:  78% 315M/405M [00:03<00:00, 91.8MB/s]
Downloading (…)l-00004-of-00033.bin:  80% 325M/405M [00:03<00:00, 92.3MB/s]
Downloading (…)l-00004-of-00033.bin:  83% 336M/405M [00:03<00:00, 91.9MB/s]
Downloading (…)l-00004-of-00033.bin:  85% 346M/405M [00:03<00:00, 90.8MB/s]
Downloading (…)l-00004-of-00033.bin:  88% 357M/405M [00:03<00:00, 90.9MB/s]
Downloading (…)l-00004-of-00033.bin:  91% 367M/405M [00:04<00:00, 91.9MB/s]
Downloading (…)l-00004-of-00033.bin:  93% 377M/405M [00:04<00:00, 92.3MB/s]
Downloading (…)l-00004-of-00033.bin:  96% 388M/405M [00:04<00:00, 92.3MB/s]
Downloading (…)l-00004-of-00033.bin: 100% 405M/405M [00:04<00:00, 91.4MB/s]
Downloading shards:  12% 4/33 [00:19<02:25,  5.00s/it]
Downloading (…)l-00005-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00005-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 90.5MB/s]
Downloading (…)l-00005-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 91.7MB/s]
Downloading (…)l-00005-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 91.5MB/s]
Downloading (…)l-00005-of-00033.bin:  10% 41.9M/405M [00:00<00:03, 93.6MB/s]
Downloading (…)l-00005-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 90.9MB/s]
Downloading (…)l-00005-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 90.2MB/s]
Downloading (…)l-00005-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 89.6MB/s]
Downloading (…)l-00005-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 89.5MB/s]
Downloading (…)l-00005-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 87.7MB/s]
Downloading (…)l-00005-of-00033.bin:  26% 105M/405M [00:01<00:03, 89.2MB/s] 
Downloading (…)l-00005-of-00033.bin:  28% 115M/405M [00:01<00:03, 87.8MB/s]
Downloading (…)l-00005-of-00033.bin:  31% 126M/405M [00:01<00:03, 88.9MB/s]
Downloading (…)l-00005-of-00033.bin:  34% 136M/405M [00:01<00:03, 88.8MB/s]
Downloading (…)l-00005-of-00033.bin:  36% 147M/405M [00:01<00:02, 88.7MB/s]
Downloading (…)l-00005-of-00033.bin:  39% 157M/405M [00:01<00:02, 89.4MB/s]
Downloading (…)l-00005-of-00033.bin:  41% 168M/405M [00:01<00:02, 88.1MB/s]
Downloading (…)l-00005-of-00033.bin:  44% 178M/405M [00:01<00:02, 88.7MB/s]
Downloading (…)l-00005-of-00033.bin:  47% 189M/405M [00:02<00:02, 87.3MB/s]
Downloading (…)l-00005-of-00033.bin:  49% 199M/405M [00:02<00:02, 86.2MB/s]
Downloading (…)l-00005-of-00033.bin:  52% 210M/405M [00:02<00:02, 87.0MB/s]
Downloading (…)l-00005-of-00033.bin:  54% 220M/405M [00:02<00:02, 86.9MB/s]
Downloading (…)l-00005-of-00033.bin:  57% 231M/405M [00:02<00:02, 76.1MB/s]
Downloading (…)l-00005-of-00033.bin:  62% 252M/405M [00:02<00:01, 92.4MB/s]
Downloading (…)l-00005-of-00033.bin:  65% 262M/405M [00:02<00:01, 90.3MB/s]
Downloading (…)l-00005-of-00033.bin:  67% 273M/405M [00:03<00:01, 90.5MB/s]
Downloading (…)l-00005-of-00033.bin:  70% 283M/405M [00:03<00:01, 90.1MB/s]
Downloading (…)l-00005-of-00033.bin:  73% 294M/405M [00:03<00:01, 88.3MB/s]
Downloading (…)l-00005-of-00033.bin:  75% 304M/405M [00:03<00:01, 86.8MB/s]
Downloading (…)l-00005-of-00033.bin:  78% 315M/405M [00:03<00:01, 84.5MB/s]
Downloading (…)l-00005-of-00033.bin:  80% 325M/405M [00:03<00:00, 86.3MB/s]
Downloading (…)l-00005-of-00033.bin:  83% 336M/405M [00:03<00:00, 87.3MB/s]
Downloading (…)l-00005-of-00033.bin:  85% 346M/405M [00:03<00:00, 86.8MB/s]
Downloading (…)l-00005-of-00033.bin:  88% 357M/405M [00:04<00:00, 75.5MB/s]
Downloading (…)l-00005-of-00033.bin:  91% 367M/405M [00:04<00:00, 78.9MB/s]
Downloading (…)l-00005-of-00033.bin:  93% 377M/405M [00:04<00:00, 81.0MB/s]
Downloading (…)l-00005-of-00033.bin:  96% 388M/405M [00:04<00:00, 80.5MB/s]
Downloading (…)l-00005-of-00033.bin: 100% 405M/405M [00:04<00:00, 86.7MB/s]
Downloading shards:  15% 5/33 [00:24<02:20,  5.00s/it]
Downloading (…)l-00006-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00006-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 88.2MB/s]
Downloading (…)l-00006-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 87.6MB/s]
Downloading (…)l-00006-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 90.2MB/s]
Downloading (…)l-00006-of-00033.bin:  10% 41.9M/405M [00:00<00:03, 91.0MB/s]
Downloading (…)l-00006-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 89.7MB/s]
Downloading (…)l-00006-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 88.9MB/s]
Downloading (…)l-00006-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 90.6MB/s]
Downloading (…)l-00006-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 89.8MB/s]
Downloading (…)l-00006-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 90.9MB/s]
Downloading (…)l-00006-of-00033.bin:  26% 105M/405M [00:01<00:03, 90.7MB/s] 
Downloading (…)l-00006-of-00033.bin:  28% 115M/405M [00:01<00:03, 91.4MB/s]
Downloading (…)l-00006-of-00033.bin:  31% 126M/405M [00:01<00:03, 91.4MB/s]
Downloading (…)l-00006-of-00033.bin:  34% 136M/405M [00:01<00:02, 91.9MB/s]
Downloading (…)l-00006-of-00033.bin:  36% 147M/405M [00:01<00:02, 92.8MB/s]
Downloading (…)l-00006-of-00033.bin:  39% 157M/405M [00:01<00:02, 92.6MB/s]
Downloading (…)l-00006-of-00033.bin:  41% 168M/405M [00:01<00:02, 92.4MB/s]
Downloading (…)l-00006-of-00033.bin:  44% 178M/405M [00:01<00:02, 90.0MB/s]
Downloading (…)l-00006-of-00033.bin:  47% 189M/405M [00:02<00:02, 89.7MB/s]
Downloading (…)l-00006-of-00033.bin:  49% 199M/405M [00:02<00:02, 90.1MB/s]
Downloading (…)l-00006-of-00033.bin:  52% 210M/405M [00:02<00:02, 90.9MB/s]
Downloading (…)l-00006-of-00033.bin:  54% 220M/405M [00:02<00:02, 91.3MB/s]
Downloading (…)l-00006-of-00033.bin:  57% 231M/405M [00:02<00:01, 90.5MB/s]
Downloading (…)l-00006-of-00033.bin:  60% 241M/405M [00:02<00:01, 89.9MB/s]
Downloading (…)l-00006-of-00033.bin:  62% 252M/405M [00:02<00:01, 89.3MB/s]
Downloading (…)l-00006-of-00033.bin:  65% 262M/405M [00:02<00:01, 89.0MB/s]
Downloading (…)l-00006-of-00033.bin:  67% 273M/405M [00:03<00:01, 84.3MB/s]
Downloading (…)l-00006-of-00033.bin:  70% 283M/405M [00:03<00:01, 85.5MB/s]
Downloading (…)l-00006-of-00033.bin:  73% 294M/405M [00:03<00:01, 84.5MB/s]
Downloading (…)l-00006-of-00033.bin:  75% 304M/405M [00:03<00:01, 86.3MB/s]
Downloading (…)l-00006-of-00033.bin:  78% 315M/405M [00:03<00:01, 87.4MB/s]
Downloading (…)l-00006-of-00033.bin:  80% 325M/405M [00:03<00:00, 88.6MB/s]
Downloading (…)l-00006-of-00033.bin:  83% 336M/405M [00:03<00:00, 90.2MB/s]
Downloading (…)l-00006-of-00033.bin:  85% 346M/405M [00:03<00:00, 91.1MB/s]
Downloading (…)l-00006-of-00033.bin:  88% 357M/405M [00:03<00:00, 91.1MB/s]
Downloading (…)l-00006-of-00033.bin:  91% 367M/405M [00:04<00:00, 87.1MB/s]
Downloading (…)l-00006-of-00033.bin:  93% 377M/405M [00:04<00:00, 85.9MB/s]
Downloading (…)l-00006-of-00033.bin:  96% 388M/405M [00:04<00:00, 86.9MB/s]
Downloading (…)l-00006-of-00033.bin: 100% 405M/405M [00:04<00:00, 89.0MB/s]
Downloading shards:  18% 6/33 [00:29<02:13,  4.95s/it]
Downloading (…)l-00007-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00007-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 90.6MB/s]
Downloading (…)l-00007-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 87.6MB/s]
Downloading (…)l-00007-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 89.9MB/s]
Downloading (…)l-00007-of-00033.bin:  10% 41.9M/405M [00:00<00:03, 91.5MB/s]
Downloading (…)l-00007-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 92.3MB/s]
Downloading (…)l-00007-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 91.9MB/s]
Downloading (…)l-00007-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 93.1MB/s]
Downloading (…)l-00007-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 92.7MB/s]
Downloading (…)l-00007-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 92.3MB/s]
Downloading (…)l-00007-of-00033.bin:  26% 105M/405M [00:01<00:03, 91.3MB/s] 
Downloading (…)l-00007-of-00033.bin:  28% 115M/405M [00:01<00:03, 90.3MB/s]
Downloading (…)l-00007-of-00033.bin:  31% 126M/405M [00:01<00:03, 90.8MB/s]
Downloading (…)l-00007-of-00033.bin:  34% 136M/405M [00:01<00:02, 91.4MB/s]
Downloading (…)l-00007-of-00033.bin:  36% 147M/405M [00:01<00:02, 91.8MB/s]
Downloading (…)l-00007-of-00033.bin:  39% 157M/405M [00:01<00:02, 91.6MB/s]
Downloading (…)l-00007-of-00033.bin:  41% 168M/405M [00:01<00:02, 91.9MB/s]
Downloading (…)l-00007-of-00033.bin:  44% 178M/405M [00:01<00:02, 92.5MB/s]
Downloading (…)l-00007-of-00033.bin:  47% 189M/405M [00:02<00:02, 91.2MB/s]
Downloading (…)l-00007-of-00033.bin:  49% 199M/405M [00:02<00:02, 91.5MB/s]
Downloading (…)l-00007-of-00033.bin:  52% 210M/405M [00:02<00:02, 91.3MB/s]
Downloading (…)l-00007-of-00033.bin:  54% 220M/405M [00:02<00:02, 90.3MB/s]
Downloading (…)l-00007-of-00033.bin:  57% 231M/405M [00:02<00:01, 91.4MB/s]
Downloading (…)l-00007-of-00033.bin:  60% 241M/405M [00:02<00:01, 91.5MB/s]
Downloading (…)l-00007-of-00033.bin:  62% 252M/405M [00:02<00:01, 92.2MB/s]
Downloading (…)l-00007-of-00033.bin:  65% 262M/405M [00:02<00:01, 91.9MB/s]
Downloading (…)l-00007-of-00033.bin:  67% 273M/405M [00:02<00:01, 89.1MB/s]
Downloading (…)l-00007-of-00033.bin:  70% 283M/405M [00:03<00:01, 89.5MB/s]
Downloading (…)l-00007-of-00033.bin:  73% 294M/405M [00:03<00:01, 90.5MB/s]
Downloading (…)l-00007-of-00033.bin:  75% 304M/405M [00:03<00:01, 89.3MB/s]
Downloading (…)l-00007-of-00033.bin:  78% 315M/405M [00:03<00:01, 90.1MB/s]
Downloading (…)l-00007-of-00033.bin:  80% 325M/405M [00:03<00:00, 91.1MB/s]
Downloading (…)l-00007-of-00033.bin:  83% 336M/405M [00:03<00:00, 90.4MB/s]
Downloading (…)l-00007-of-00033.bin:  85% 346M/405M [00:03<00:00, 90.0MB/s]
Downloading (…)l-00007-of-00033.bin:  88% 357M/405M [00:03<00:00, 90.5MB/s]
Downloading (…)l-00007-of-00033.bin:  91% 367M/405M [00:04<00:00, 91.7MB/s]
Downloading (…)l-00007-of-00033.bin:  93% 377M/405M [00:04<00:00, 89.9MB/s]
Downloading (…)l-00007-of-00033.bin:  96% 388M/405M [00:04<00:00, 89.3MB/s]
Downloading (…)l-00007-of-00033.bin: 100% 405M/405M [00:04<00:00, 90.7MB/s]
Downloading shards:  21% 7/33 [00:34<02:07,  4.89s/it]
Downloading (…)l-00008-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00008-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 94.2MB/s]
Downloading (…)l-00008-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 94.4MB/s]
Downloading (…)l-00008-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 91.8MB/s]
Downloading (…)l-00008-of-00033.bin:  10% 41.9M/405M [00:00<00:03, 91.4MB/s]
Downloading (…)l-00008-of-00033.bin:  13% 52.4M/405M [00:00<00:04, 85.2MB/s]
Downloading (…)l-00008-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 88.2MB/s]
Downloading (…)l-00008-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 89.2MB/s]
Downloading (…)l-00008-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 89.6MB/s]
Downloading (…)l-00008-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 89.1MB/s]
Downloading (…)l-00008-of-00033.bin:  26% 105M/405M [00:01<00:03, 90.9MB/s] 
Downloading (…)l-00008-of-00033.bin:  28% 115M/405M [00:01<00:03, 91.8MB/s]
Downloading (…)l-00008-of-00033.bin:  31% 126M/405M [00:01<00:03, 92.3MB/s]
Downloading (…)l-00008-of-00033.bin:  34% 136M/405M [00:01<00:02, 92.3MB/s]
Downloading (…)l-00008-of-00033.bin:  36% 147M/405M [00:01<00:02, 93.3MB/s]
Downloading (…)l-00008-of-00033.bin:  39% 157M/405M [00:01<00:02, 93.3MB/s]
Downloading (…)l-00008-of-00033.bin:  41% 168M/405M [00:01<00:02, 89.9MB/s]
Downloading (…)l-00008-of-00033.bin:  44% 178M/405M [00:01<00:02, 87.7MB/s]
Downloading (…)l-00008-of-00033.bin:  47% 189M/405M [00:02<00:02, 88.9MB/s]
Downloading (…)l-00008-of-00033.bin:  49% 199M/405M [00:02<00:02, 75.4MB/s]
Downloading (…)l-00008-of-00033.bin:  52% 210M/405M [00:02<00:02, 78.9MB/s]
Downloading (…)l-00008-of-00033.bin:  54% 220M/405M [00:02<00:02, 79.3MB/s]
Downloading (…)l-00008-of-00033.bin:  57% 231M/405M [00:02<00:02, 83.5MB/s]
Downloading (…)l-00008-of-00033.bin:  60% 241M/405M [00:02<00:01, 85.5MB/s]
Downloading (…)l-00008-of-00033.bin:  62% 252M/405M [00:02<00:01, 87.0MB/s]
Downloading (…)l-00008-of-00033.bin:  65% 262M/405M [00:03<00:01, 80.2MB/s]
Downloading (…)l-00008-of-00033.bin:  70% 283M/405M [00:03<00:01, 93.2MB/s]
Downloading (…)l-00008-of-00033.bin:  73% 294M/405M [00:03<00:01, 92.7MB/s]
Downloading (…)l-00008-of-00033.bin:  75% 304M/405M [00:03<00:01, 92.3MB/s]
Downloading (…)l-00008-of-00033.bin:  78% 315M/405M [00:03<00:00, 91.5MB/s]
Downloading (…)l-00008-of-00033.bin:  80% 325M/405M [00:03<00:00, 91.9MB/s]
Downloading (…)l-00008-of-00033.bin:  83% 336M/405M [00:03<00:00, 91.8MB/s]
Downloading (…)l-00008-of-00033.bin:  85% 346M/405M [00:03<00:00, 89.6MB/s]
Downloading (…)l-00008-of-00033.bin:  88% 357M/405M [00:04<00:00, 90.6MB/s]
Downloading (…)l-00008-of-00033.bin:  91% 367M/405M [00:04<00:00, 91.2MB/s]
Downloading (…)l-00008-of-00033.bin:  93% 377M/405M [00:04<00:00, 90.3MB/s]
Downloading (…)l-00008-of-00033.bin:  96% 388M/405M [00:04<00:00, 86.4MB/s]
Downloading (…)l-00008-of-00033.bin: 100% 405M/405M [00:04<00:00, 87.8MB/s]
Downloading shards:  24% 8/33 [00:39<02:02,  4.90s/it]
Downloading (…)l-00009-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00009-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 85.7MB/s]
Downloading (…)l-00009-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 85.0MB/s]
Downloading (…)l-00009-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 84.8MB/s]
Downloading (…)l-00009-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 86.0MB/s]
Downloading (…)l-00009-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 88.6MB/s]
Downloading (…)l-00009-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 88.8MB/s]
Downloading (…)l-00009-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 90.4MB/s]
Downloading (…)l-00009-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 88.4MB/s]
Downloading (…)l-00009-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 89.9MB/s]
Downloading (…)l-00009-of-00033.bin:  26% 105M/405M [00:01<00:03, 90.9MB/s] 
Downloading (…)l-00009-of-00033.bin:  28% 115M/405M [00:01<00:03, 86.9MB/s]
Downloading (…)l-00009-of-00033.bin:  31% 126M/405M [00:01<00:03, 87.9MB/s]
Downloading (…)l-00009-of-00033.bin:  34% 136M/405M [00:01<00:03, 87.4MB/s]
Downloading (…)l-00009-of-00033.bin:  36% 147M/405M [00:01<00:02, 87.0MB/s]
Downloading (…)l-00009-of-00033.bin:  39% 157M/405M [00:01<00:02, 88.7MB/s]
Downloading (…)l-00009-of-00033.bin:  41% 168M/405M [00:01<00:02, 88.7MB/s]
Downloading (…)l-00009-of-00033.bin:  44% 178M/405M [00:02<00:02, 84.5MB/s]
Downloading (…)l-00009-of-00033.bin:  47% 189M/405M [00:02<00:02, 86.0MB/s]
Downloading (…)l-00009-of-00033.bin:  49% 199M/405M [00:02<00:02, 87.2MB/s]
Downloading (…)l-00009-of-00033.bin:  52% 210M/405M [00:02<00:02, 89.4MB/s]
Downloading (…)l-00009-of-00033.bin:  54% 220M/405M [00:02<00:02, 91.0MB/s]
Downloading (…)l-00009-of-00033.bin:  57% 231M/405M [00:02<00:01, 88.9MB/s]
Downloading (…)l-00009-of-00033.bin:  60% 241M/405M [00:02<00:01, 88.3MB/s]
Downloading (…)l-00009-of-00033.bin:  62% 252M/405M [00:02<00:01, 90.4MB/s]
Downloading (…)l-00009-of-00033.bin:  65% 262M/405M [00:02<00:01, 91.4MB/s]
Downloading (…)l-00009-of-00033.bin:  67% 273M/405M [00:03<00:01, 91.7MB/s]
Downloading (…)l-00009-of-00033.bin:  70% 283M/405M [00:03<00:01, 88.9MB/s]
Downloading (…)l-00009-of-00033.bin:  73% 294M/405M [00:03<00:01, 90.4MB/s]
Downloading (…)l-00009-of-00033.bin:  75% 304M/405M [00:03<00:01, 91.0MB/s]
Downloading (…)l-00009-of-00033.bin:  78% 315M/405M [00:03<00:00, 90.9MB/s]
Downloading (…)l-00009-of-00033.bin:  80% 325M/405M [00:03<00:00, 91.3MB/s]
Downloading (…)l-00009-of-00033.bin:  83% 336M/405M [00:03<00:00, 89.0MB/s]
Downloading (…)l-00009-of-00033.bin:  85% 346M/405M [00:03<00:00, 90.2MB/s]
Downloading (…)l-00009-of-00033.bin:  88% 357M/405M [00:04<00:00, 89.4MB/s]
Downloading (…)l-00009-of-00033.bin:  91% 367M/405M [00:04<00:00, 85.7MB/s]
Downloading (…)l-00009-of-00033.bin:  93% 377M/405M [00:04<00:00, 87.3MB/s]
Downloading (…)l-00009-of-00033.bin:  96% 388M/405M [00:04<00:00, 88.3MB/s]
Downloading (…)l-00009-of-00033.bin: 100% 405M/405M [00:04<00:00, 88.6MB/s]
Downloading shards:  27% 9/33 [00:44<01:57,  4.89s/it]
Downloading (…)l-00010-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00010-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 90.9MB/s]
Downloading (…)l-00010-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 91.9MB/s]
Downloading (…)l-00010-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 93.3MB/s]
Downloading (…)l-00010-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 90.3MB/s]
Downloading (…)l-00010-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 90.3MB/s]
Downloading (…)l-00010-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 92.2MB/s]
Downloading (…)l-00010-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 90.8MB/s]
Downloading (…)l-00010-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 80.4MB/s]
Downloading (…)l-00010-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 81.1MB/s]
Downloading (…)l-00010-of-00033.bin:  26% 105M/405M [00:01<00:03, 86.1MB/s] 
Downloading (…)l-00010-of-00033.bin:  28% 115M/405M [00:01<00:03, 87.1MB/s]
Downloading (…)l-00010-of-00033.bin:  31% 126M/405M [00:01<00:03, 86.8MB/s]
Downloading (…)l-00010-of-00033.bin:  34% 136M/405M [00:01<00:03, 87.1MB/s]
Downloading (…)l-00010-of-00033.bin:  36% 147M/405M [00:01<00:02, 88.7MB/s]
Downloading (…)l-00010-of-00033.bin:  39% 157M/405M [00:01<00:02, 88.7MB/s]
Downloading (…)l-00010-of-00033.bin:  41% 168M/405M [00:01<00:02, 88.5MB/s]
Downloading (…)l-00010-of-00033.bin:  44% 178M/405M [00:02<00:02, 87.5MB/s]
Downloading (…)l-00010-of-00033.bin:  47% 189M/405M [00:02<00:02, 86.5MB/s]
Downloading (…)l-00010-of-00033.bin:  49% 199M/405M [00:02<00:02, 88.0MB/s]
Downloading (…)l-00010-of-00033.bin:  52% 210M/405M [00:02<00:02, 87.8MB/s]
Downloading (…)l-00010-of-00033.bin:  54% 220M/405M [00:02<00:02, 88.9MB/s]
Downloading (…)l-00010-of-00033.bin:  57% 231M/405M [00:02<00:01, 90.1MB/s]
Downloading (…)l-00010-of-00033.bin:  60% 241M/405M [00:02<00:01, 90.4MB/s]
Downloading (…)l-00010-of-00033.bin:  62% 252M/405M [00:02<00:01, 91.2MB/s]
Downloading (…)l-00010-of-00033.bin:  65% 262M/405M [00:02<00:01, 89.1MB/s]
Downloading (…)l-00010-of-00033.bin:  67% 273M/405M [00:03<00:01, 89.5MB/s]
Downloading (…)l-00010-of-00033.bin:  70% 283M/405M [00:03<00:01, 89.1MB/s]
Downloading (…)l-00010-of-00033.bin:  73% 294M/405M [00:03<00:01, 86.5MB/s]
Downloading (…)l-00010-of-00033.bin:  75% 304M/405M [00:03<00:01, 87.6MB/s]
Downloading (…)l-00010-of-00033.bin:  78% 315M/405M [00:03<00:01, 87.7MB/s]
Downloading (…)l-00010-of-00033.bin:  80% 325M/405M [00:03<00:00, 88.8MB/s]
Downloading (…)l-00010-of-00033.bin:  83% 336M/405M [00:03<00:00, 89.1MB/s]
Downloading (…)l-00010-of-00033.bin:  85% 346M/405M [00:03<00:00, 89.9MB/s]
Downloading (…)l-00010-of-00033.bin:  88% 357M/405M [00:04<00:00, 88.0MB/s]
Downloading (…)l-00010-of-00033.bin:  91% 367M/405M [00:04<00:00, 87.8MB/s]
Downloading (…)l-00010-of-00033.bin:  93% 377M/405M [00:04<00:00, 81.7MB/s]
Downloading (…)l-00010-of-00033.bin:  96% 388M/405M [00:04<00:00, 83.7MB/s]
Downloading (…)l-00010-of-00033.bin: 100% 405M/405M [00:04<00:00, 87.6MB/s]
Downloading shards:  30% 10/33 [00:49<01:52,  4.90s/it]
Downloading (…)l-00011-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00011-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 79.9MB/s]
Downloading (…)l-00011-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 86.0MB/s]
Downloading (…)l-00011-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 88.4MB/s]
Downloading (…)l-00011-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 90.3MB/s]
Downloading (…)l-00011-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 90.8MB/s]
Downloading (…)l-00011-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 90.0MB/s]
Downloading (…)l-00011-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 88.3MB/s]
Downloading (…)l-00011-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 87.4MB/s]
Downloading (…)l-00011-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 88.3MB/s]
Downloading (…)l-00011-of-00033.bin:  26% 105M/405M [00:01<00:03, 90.1MB/s] 
Downloading (…)l-00011-of-00033.bin:  28% 115M/405M [00:01<00:03, 88.8MB/s]
Downloading (…)l-00011-of-00033.bin:  31% 126M/405M [00:01<00:03, 88.8MB/s]
Downloading (…)l-00011-of-00033.bin:  34% 136M/405M [00:01<00:02, 89.9MB/s]
Downloading (…)l-00011-of-00033.bin:  36% 147M/405M [00:01<00:02, 90.6MB/s]
Downloading (…)l-00011-of-00033.bin:  39% 157M/405M [00:01<00:02, 90.9MB/s]
Downloading (…)l-00011-of-00033.bin:  41% 168M/405M [00:01<00:02, 90.1MB/s]
Downloading (…)l-00011-of-00033.bin:  44% 178M/405M [00:02<00:02, 88.7MB/s]
Downloading (…)l-00011-of-00033.bin:  47% 189M/405M [00:02<00:02, 89.6MB/s]
Downloading (…)l-00011-of-00033.bin:  49% 199M/405M [00:02<00:02, 88.8MB/s]
Downloading (…)l-00011-of-00033.bin:  52% 210M/405M [00:02<00:02, 88.4MB/s]
Downloading (…)l-00011-of-00033.bin:  54% 220M/405M [00:02<00:02, 89.9MB/s]
Downloading (…)l-00011-of-00033.bin:  57% 231M/405M [00:02<00:01, 91.0MB/s]
Downloading (…)l-00011-of-00033.bin:  60% 241M/405M [00:02<00:01, 91.6MB/s]
Downloading (…)l-00011-of-00033.bin:  62% 252M/405M [00:02<00:01, 91.4MB/s]
Downloading (…)l-00011-of-00033.bin:  65% 262M/405M [00:02<00:01, 91.7MB/s]
Downloading (…)l-00011-of-00033.bin:  67% 273M/405M [00:03<00:01, 92.0MB/s]
Downloading (…)l-00011-of-00033.bin:  70% 283M/405M [00:03<00:01, 91.7MB/s]
Downloading (…)l-00011-of-00033.bin:  73% 294M/405M [00:03<00:01, 90.4MB/s]
Downloading (…)l-00011-of-00033.bin:  75% 304M/405M [00:03<00:01, 90.5MB/s]
Downloading (…)l-00011-of-00033.bin:  78% 315M/405M [00:03<00:01, 86.3MB/s]
Downloading (…)l-00011-of-00033.bin:  80% 325M/405M [00:03<00:00, 82.6MB/s]
Downloading (…)l-00011-of-00033.bin:  83% 336M/405M [00:03<00:00, 78.9MB/s]
Downloading (…)l-00011-of-00033.bin:  85% 346M/405M [00:03<00:00, 83.4MB/s]
Downloading (…)l-00011-of-00033.bin:  88% 357M/405M [00:04<00:00, 85.3MB/s]
Downloading (…)l-00011-of-00033.bin:  91% 367M/405M [00:04<00:00, 84.7MB/s]
Downloading (…)l-00011-of-00033.bin:  93% 377M/405M [00:04<00:00, 87.2MB/s]
Downloading (…)l-00011-of-00033.bin:  96% 388M/405M [00:04<00:00, 88.7MB/s]
Downloading (…)l-00011-of-00033.bin: 100% 405M/405M [00:04<00:00, 88.4MB/s]
Downloading shards:  33% 11/33 [00:55<01:54,  5.22s/it]
Downloading (…)l-00012-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00012-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 92.2MB/s]
Downloading (…)l-00012-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 90.3MB/s]
Downloading (…)l-00012-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 88.7MB/s]
Downloading (…)l-00012-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 90.0MB/s]
Downloading (…)l-00012-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 90.0MB/s]
Downloading (…)l-00012-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 88.8MB/s]
Downloading (…)l-00012-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 90.2MB/s]
Downloading (…)l-00012-of-00033.bin:  21% 83.9M/405M [00:00<00:04, 78.5MB/s]
Downloading (…)l-00012-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 82.7MB/s]
Downloading (…)l-00012-of-00033.bin:  26% 105M/405M [00:01<00:03, 85.1MB/s] 
Downloading (…)l-00012-of-00033.bin:  28% 115M/405M [00:01<00:03, 86.9MB/s]
Downloading (…)l-00012-of-00033.bin:  31% 126M/405M [00:01<00:03, 85.6MB/s]
Downloading (…)l-00012-of-00033.bin:  34% 136M/405M [00:01<00:03, 84.9MB/s]
Downloading (…)l-00012-of-00033.bin:  36% 147M/405M [00:01<00:02, 86.7MB/s]
Downloading (…)l-00012-of-00033.bin:  39% 157M/405M [00:01<00:02, 87.4MB/s]
Downloading (…)l-00012-of-00033.bin:  41% 168M/405M [00:01<00:02, 88.4MB/s]
Downloading (…)l-00012-of-00033.bin:  44% 178M/405M [00:02<00:02, 88.6MB/s]
Downloading (…)l-00012-of-00033.bin:  47% 189M/405M [00:02<00:02, 89.6MB/s]
Downloading (…)l-00012-of-00033.bin:  49% 199M/405M [00:02<00:02, 89.0MB/s]
Downloading (…)l-00012-of-00033.bin:  52% 210M/405M [00:02<00:02, 83.0MB/s]
Downloading (…)l-00012-of-00033.bin:  54% 220M/405M [00:02<00:02, 84.4MB/s]
Downloading (…)l-00012-of-00033.bin:  57% 231M/405M [00:02<00:01, 87.2MB/s]
Downloading (…)l-00012-of-00033.bin:  60% 241M/405M [00:02<00:01, 87.1MB/s]
Downloading (…)l-00012-of-00033.bin:  62% 252M/405M [00:02<00:01, 88.0MB/s]
Downloading (…)l-00012-of-00033.bin:  65% 262M/405M [00:03<00:01, 88.6MB/s]
Downloading (…)l-00012-of-00033.bin:  67% 273M/405M [00:03<00:01, 90.3MB/s]
Downloading (…)l-00012-of-00033.bin:  70% 283M/405M [00:03<00:01, 90.1MB/s]
Downloading (…)l-00012-of-00033.bin:  73% 294M/405M [00:03<00:01, 89.1MB/s]
Downloading (…)l-00012-of-00033.bin:  75% 304M/405M [00:03<00:01, 87.3MB/s]
Downloading (…)l-00012-of-00033.bin:  78% 315M/405M [00:03<00:01, 88.8MB/s]
Downloading (…)l-00012-of-00033.bin:  80% 325M/405M [00:03<00:00, 89.8MB/s]
Downloading (…)l-00012-of-00033.bin:  83% 336M/405M [00:03<00:00, 88.8MB/s]
Downloading (…)l-00012-of-00033.bin:  85% 346M/405M [00:03<00:00, 88.6MB/s]
Downloading (…)l-00012-of-00033.bin:  88% 357M/405M [00:04<00:00, 89.2MB/s]
Downloading (…)l-00012-of-00033.bin:  91% 367M/405M [00:04<00:00, 88.7MB/s]
Downloading (…)l-00012-of-00033.bin:  93% 377M/405M [00:04<00:00, 86.8MB/s]
Downloading (…)l-00012-of-00033.bin:  96% 388M/405M [00:04<00:00, 85.1MB/s]
Downloading (…)l-00012-of-00033.bin: 100% 405M/405M [00:04<00:00, 87.1MB/s]
Downloading shards:  36% 12/33 [01:01<01:54,  5.46s/it]
Downloading (…)l-00013-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00013-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 91.5MB/s]
Downloading (…)l-00013-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 92.3MB/s]
Downloading (…)l-00013-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 91.5MB/s]
Downloading (…)l-00013-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 89.5MB/s]
Downloading (…)l-00013-of-00033.bin:  13% 52.4M/405M [00:00<00:04, 86.8MB/s]
Downloading (…)l-00013-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 87.7MB/s]
Downloading (…)l-00013-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 89.2MB/s]
Downloading (…)l-00013-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 90.0MB/s]
Downloading (…)l-00013-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 90.5MB/s]
Downloading (…)l-00013-of-00033.bin:  26% 105M/405M [00:01<00:03, 91.0MB/s] 
Downloading (…)l-00013-of-00033.bin:  28% 115M/405M [00:01<00:03, 90.9MB/s]
Downloading (…)l-00013-of-00033.bin:  31% 126M/405M [00:01<00:03, 91.3MB/s]
Downloading (…)l-00013-of-00033.bin:  34% 136M/405M [00:01<00:03, 88.6MB/s]
Downloading (…)l-00013-of-00033.bin:  36% 147M/405M [00:01<00:02, 91.2MB/s]
Downloading (…)l-00013-of-00033.bin:  39% 157M/405M [00:01<00:02, 90.9MB/s]
Downloading (…)l-00013-of-00033.bin:  41% 168M/405M [00:01<00:02, 90.2MB/s]
Downloading (…)l-00013-of-00033.bin:  44% 178M/405M [00:01<00:02, 89.8MB/s]
Downloading (…)l-00013-of-00033.bin:  47% 189M/405M [00:02<00:02, 91.4MB/s]
Downloading (…)l-00013-of-00033.bin:  49% 199M/405M [00:02<00:02, 90.7MB/s]
Downloading (…)l-00013-of-00033.bin:  52% 210M/405M [00:02<00:02, 91.5MB/s]
Downloading (…)l-00013-of-00033.bin:  54% 220M/405M [00:02<00:01, 92.5MB/s]
Downloading (…)l-00013-of-00033.bin:  57% 231M/405M [00:02<00:01, 91.8MB/s]
Downloading (…)l-00013-of-00033.bin:  60% 241M/405M [00:02<00:01, 91.7MB/s]
Downloading (…)l-00013-of-00033.bin:  62% 252M/405M [00:02<00:01, 91.5MB/s]
Downloading (…)l-00013-of-00033.bin:  65% 262M/405M [00:02<00:01, 91.9MB/s]
Downloading (…)l-00013-of-00033.bin:  67% 273M/405M [00:03<00:01, 91.7MB/s]
Downloading (…)l-00013-of-00033.bin:  70% 283M/405M [00:03<00:01, 92.8MB/s]
Downloading (…)l-00013-of-00033.bin:  73% 294M/405M [00:03<00:01, 93.4MB/s]
Downloading (…)l-00013-of-00033.bin:  75% 304M/405M [00:03<00:01, 93.1MB/s]
Downloading (…)l-00013-of-00033.bin:  78% 315M/405M [00:03<00:00, 93.5MB/s]
Downloading (…)l-00013-of-00033.bin:  80% 325M/405M [00:03<00:00, 93.8MB/s]
Downloading (…)l-00013-of-00033.bin:  83% 336M/405M [00:03<00:00, 88.5MB/s]
Downloading (…)l-00013-of-00033.bin:  85% 346M/405M [00:03<00:00, 89.6MB/s]
Downloading (…)l-00013-of-00033.bin:  88% 357M/405M [00:03<00:00, 89.8MB/s]
Downloading (…)l-00013-of-00033.bin:  91% 367M/405M [00:04<00:00, 91.3MB/s]
Downloading (…)l-00013-of-00033.bin:  93% 377M/405M [00:04<00:00, 89.7MB/s]
Downloading (…)l-00013-of-00033.bin:  96% 388M/405M [00:04<00:00, 90.4MB/s]
Downloading (…)l-00013-of-00033.bin: 100% 405M/405M [00:04<00:00, 90.7MB/s]
Downloading shards:  39% 13/33 [01:06<01:45,  5.25s/it]
Downloading (…)l-00014-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00014-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 87.8MB/s]
Downloading (…)l-00014-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 90.6MB/s]
Downloading (…)l-00014-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 93.2MB/s]
Downloading (…)l-00014-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 89.5MB/s]
Downloading (…)l-00014-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 90.1MB/s]
Downloading (…)l-00014-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 91.6MB/s]
Downloading (…)l-00014-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 91.5MB/s]
Downloading (…)l-00014-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 90.4MB/s]
Downloading (…)l-00014-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 90.5MB/s]
Downloading (…)l-00014-of-00033.bin:  26% 105M/405M [00:01<00:03, 90.8MB/s] 
Downloading (…)l-00014-of-00033.bin:  28% 115M/405M [00:01<00:03, 92.1MB/s]
Downloading (…)l-00014-of-00033.bin:  31% 126M/405M [00:01<00:03, 89.5MB/s]
Downloading (…)l-00014-of-00033.bin:  34% 136M/405M [00:01<00:02, 89.5MB/s]
Downloading (…)l-00014-of-00033.bin:  36% 147M/405M [00:01<00:02, 89.2MB/s]
Downloading (…)l-00014-of-00033.bin:  39% 157M/405M [00:01<00:02, 89.5MB/s]
Downloading (…)l-00014-of-00033.bin:  41% 168M/405M [00:01<00:02, 90.3MB/s]
Downloading (…)l-00014-of-00033.bin:  44% 178M/405M [00:01<00:02, 89.5MB/s]
Downloading (…)l-00014-of-00033.bin:  47% 189M/405M [00:02<00:02, 90.2MB/s]
Downloading (…)l-00014-of-00033.bin:  49% 199M/405M [00:02<00:02, 90.0MB/s]
Downloading (…)l-00014-of-00033.bin:  52% 210M/405M [00:02<00:02, 89.8MB/s]
Downloading (…)l-00014-of-00033.bin:  54% 220M/405M [00:02<00:02, 87.3MB/s]
Downloading (…)l-00014-of-00033.bin:  57% 231M/405M [00:02<00:01, 87.8MB/s]
Downloading (…)l-00014-of-00033.bin:  60% 241M/405M [00:02<00:01, 88.0MB/s]
Downloading (…)l-00014-of-00033.bin:  62% 252M/405M [00:02<00:01, 88.0MB/s]
Downloading (…)l-00014-of-00033.bin:  65% 262M/405M [00:02<00:01, 89.8MB/s]
Downloading (…)l-00014-of-00033.bin:  67% 273M/405M [00:03<00:01, 88.7MB/s]
Downloading (…)l-00014-of-00033.bin:  70% 283M/405M [00:03<00:01, 89.9MB/s]
Downloading (…)l-00014-of-00033.bin:  73% 294M/405M [00:03<00:01, 89.4MB/s]
Downloading (…)l-00014-of-00033.bin:  75% 304M/405M [00:03<00:01, 89.4MB/s]
Downloading (…)l-00014-of-00033.bin:  78% 315M/405M [00:03<00:01, 89.7MB/s]
Downloading (…)l-00014-of-00033.bin:  80% 325M/405M [00:03<00:00, 89.2MB/s]
Downloading (…)l-00014-of-00033.bin:  83% 336M/405M [00:03<00:00, 90.0MB/s]
Downloading (…)l-00014-of-00033.bin:  85% 346M/405M [00:03<00:00, 89.2MB/s]
Downloading (…)l-00014-of-00033.bin:  88% 357M/405M [00:03<00:00, 89.0MB/s]
Downloading (…)l-00014-of-00033.bin:  91% 367M/405M [00:04<00:00, 88.8MB/s]
Downloading (…)l-00014-of-00033.bin:  93% 377M/405M [00:04<00:00, 88.1MB/s]
Downloading (…)l-00014-of-00033.bin:  96% 388M/405M [00:04<00:00, 88.6MB/s]
Downloading (…)l-00014-of-00033.bin: 100% 405M/405M [00:04<00:00, 89.5MB/s]
Downloading shards:  42% 14/33 [01:11<01:43,  5.44s/it]
Downloading (…)l-00015-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00015-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 85.3MB/s]
Downloading (…)l-00015-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 86.8MB/s]
Downloading (…)l-00015-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 89.3MB/s]
Downloading (…)l-00015-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 88.7MB/s]
Downloading (…)l-00015-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 88.2MB/s]
Downloading (…)l-00015-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 89.5MB/s]
Downloading (…)l-00015-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 89.2MB/s]
Downloading (…)l-00015-of-00033.bin:  21% 83.9M/405M [00:01<00:05, 55.2MB/s]
Downloading (…)l-00015-of-00033.bin:  23% 94.4M/405M [00:01<00:04, 63.2MB/s]
Downloading (…)l-00015-of-00033.bin:  26% 105M/405M [00:01<00:04, 71.0MB/s] 
Downloading (…)l-00015-of-00033.bin:  28% 115M/405M [00:01<00:03, 75.5MB/s]
Downloading (…)l-00015-of-00033.bin:  31% 126M/405M [00:01<00:03, 80.9MB/s]
Downloading (…)l-00015-of-00033.bin:  34% 136M/405M [00:01<00:03, 84.6MB/s]
Downloading (…)l-00015-of-00033.bin:  36% 147M/405M [00:01<00:02, 87.6MB/s]
Downloading (…)l-00015-of-00033.bin:  39% 157M/405M [00:01<00:02, 87.1MB/s]
Downloading (…)l-00015-of-00033.bin:  41% 168M/405M [00:02<00:02, 88.8MB/s]
Downloading (…)l-00015-of-00033.bin:  44% 178M/405M [00:02<00:02, 90.0MB/s]
Downloading (…)l-00015-of-00033.bin:  47% 189M/405M [00:02<00:02, 91.2MB/s]
Downloading (…)l-00015-of-00033.bin:  49% 199M/405M [00:02<00:02, 89.4MB/s]
Downloading (…)l-00015-of-00033.bin:  52% 210M/405M [00:02<00:02, 88.9MB/s]
Downloading (…)l-00015-of-00033.bin:  54% 220M/405M [00:02<00:02, 87.6MB/s]
Downloading (…)l-00015-of-00033.bin:  57% 231M/405M [00:02<00:01, 88.0MB/s]
Downloading (…)l-00015-of-00033.bin:  60% 241M/405M [00:02<00:01, 89.8MB/s]
Downloading (…)l-00015-of-00033.bin:  62% 252M/405M [00:03<00:01, 90.7MB/s]
Downloading (…)l-00015-of-00033.bin:  65% 262M/405M [00:03<00:01, 91.0MB/s]
Downloading (…)l-00015-of-00033.bin:  67% 273M/405M [00:03<00:01, 89.3MB/s]
Downloading (…)l-00015-of-00033.bin:  70% 283M/405M [00:03<00:01, 91.1MB/s]
Downloading (…)l-00015-of-00033.bin:  73% 294M/405M [00:03<00:01, 86.5MB/s]
Downloading (…)l-00015-of-00033.bin:  75% 304M/405M [00:03<00:01, 85.3MB/s]
Downloading (…)l-00015-of-00033.bin:  78% 315M/405M [00:03<00:01, 85.6MB/s]
Downloading (…)l-00015-of-00033.bin:  80% 325M/405M [00:03<00:00, 88.0MB/s]
Downloading (…)l-00015-of-00033.bin:  83% 336M/405M [00:03<00:00, 89.8MB/s]
Downloading (…)l-00015-of-00033.bin:  85% 346M/405M [00:04<00:00, 91.2MB/s]
Downloading (…)l-00015-of-00033.bin:  88% 357M/405M [00:04<00:00, 92.3MB/s]
Downloading (…)l-00015-of-00033.bin:  91% 367M/405M [00:04<00:00, 91.3MB/s]
Downloading (…)l-00015-of-00033.bin:  93% 377M/405M [00:04<00:00, 91.2MB/s]
Downloading (…)l-00015-of-00033.bin:  96% 388M/405M [00:04<00:00, 91.3MB/s]
Downloading (…)l-00015-of-00033.bin: 100% 405M/405M [00:04<00:00, 85.5MB/s]
Downloading shards:  45% 15/33 [01:16<01:35,  5.32s/it]
Downloading (…)l-00016-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00016-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 89.0MB/s]
Downloading (…)l-00016-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 85.6MB/s]
Downloading (…)l-00016-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 89.6MB/s]
Downloading (…)l-00016-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 88.2MB/s]
Downloading (…)l-00016-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 89.1MB/s]
Downloading (…)l-00016-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 86.6MB/s]
Downloading (…)l-00016-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 88.0MB/s]
Downloading (…)l-00016-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 86.1MB/s]
Downloading (…)l-00016-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 89.1MB/s]
Downloading (…)l-00016-of-00033.bin:  26% 105M/405M [00:01<00:03, 89.6MB/s] 
Downloading (…)l-00016-of-00033.bin:  28% 115M/405M [00:01<00:03, 90.5MB/s]
Downloading (…)l-00016-of-00033.bin:  31% 126M/405M [00:01<00:03, 90.5MB/s]
Downloading (…)l-00016-of-00033.bin:  34% 136M/405M [00:01<00:02, 90.5MB/s]
Downloading (…)l-00016-of-00033.bin:  36% 147M/405M [00:01<00:02, 91.3MB/s]
Downloading (…)l-00016-of-00033.bin:  39% 157M/405M [00:01<00:02, 89.8MB/s]
Downloading (…)l-00016-of-00033.bin:  41% 168M/405M [00:01<00:02, 89.9MB/s]
Downloading (…)l-00016-of-00033.bin:  44% 178M/405M [00:01<00:02, 90.3MB/s]
Downloading (…)l-00016-of-00033.bin:  47% 189M/405M [00:02<00:02, 91.1MB/s]
Downloading (…)l-00016-of-00033.bin:  49% 199M/405M [00:02<00:02, 89.3MB/s]
Downloading (…)l-00016-of-00033.bin:  52% 210M/405M [00:02<00:02, 89.2MB/s]
Downloading (…)l-00016-of-00033.bin:  54% 220M/405M [00:02<00:02, 90.3MB/s]
Downloading (…)l-00016-of-00033.bin:  57% 231M/405M [00:02<00:01, 89.5MB/s]
Downloading (…)l-00016-of-00033.bin:  60% 241M/405M [00:02<00:01, 89.8MB/s]
Downloading (…)l-00016-of-00033.bin:  62% 252M/405M [00:02<00:01, 88.9MB/s]
Downloading (…)l-00016-of-00033.bin:  65% 262M/405M [00:02<00:01, 88.9MB/s]
Downloading (…)l-00016-of-00033.bin:  67% 273M/405M [00:03<00:01, 88.1MB/s]
Downloading (…)l-00016-of-00033.bin:  70% 283M/405M [00:03<00:01, 88.3MB/s]
Downloading (…)l-00016-of-00033.bin:  73% 294M/405M [00:03<00:01, 88.8MB/s]
Downloading (…)l-00016-of-00033.bin:  75% 304M/405M [00:03<00:01, 90.9MB/s]
Downloading (…)l-00016-of-00033.bin:  78% 315M/405M [00:03<00:00, 91.2MB/s]
Downloading (…)l-00016-of-00033.bin:  80% 325M/405M [00:03<00:00, 89.9MB/s]
Downloading (…)l-00016-of-00033.bin:  83% 336M/405M [00:03<00:00, 88.6MB/s]
Downloading (…)l-00016-of-00033.bin:  85% 346M/405M [00:03<00:00, 88.2MB/s]
Downloading (…)l-00016-of-00033.bin:  88% 357M/405M [00:03<00:00, 89.0MB/s]
Downloading (…)l-00016-of-00033.bin:  91% 367M/405M [00:04<00:00, 89.9MB/s]
Downloading (…)l-00016-of-00033.bin:  93% 377M/405M [00:04<00:00, 90.7MB/s]
Downloading (…)l-00016-of-00033.bin:  96% 388M/405M [00:04<00:00, 91.0MB/s]
Downloading (…)l-00016-of-00033.bin: 100% 405M/405M [00:04<00:00, 89.6MB/s]
Downloading shards:  48% 16/33 [01:21<01:27,  5.17s/it]
Downloading (…)l-00017-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00017-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 88.0MB/s]
Downloading (…)l-00017-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 90.6MB/s]
Downloading (…)l-00017-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 92.1MB/s]
Downloading (…)l-00017-of-00033.bin:  10% 41.9M/405M [00:00<00:03, 90.7MB/s]
Downloading (…)l-00017-of-00033.bin:  13% 52.4M/405M [00:00<00:03, 90.2MB/s]
Downloading (…)l-00017-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 89.5MB/s]
Downloading (…)l-00017-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 90.1MB/s]
Downloading (…)l-00017-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 91.0MB/s]
Downloading (…)l-00017-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 91.2MB/s]
Downloading (…)l-00017-of-00033.bin:  26% 105M/405M [00:01<00:03, 89.4MB/s] 
Downloading (…)l-00017-of-00033.bin:  28% 115M/405M [00:01<00:03, 91.1MB/s]
Downloading (…)l-00017-of-00033.bin:  31% 126M/405M [00:01<00:03, 91.6MB/s]
Downloading (…)l-00017-of-00033.bin:  34% 136M/405M [00:01<00:02, 90.3MB/s]
Downloading (…)l-00017-of-00033.bin:  36% 147M/405M [00:01<00:02, 91.1MB/s]
Downloading (…)l-00017-of-00033.bin:  39% 157M/405M [00:01<00:02, 89.5MB/s]
Downloading (…)l-00017-of-00033.bin:  41% 168M/405M [00:01<00:02, 89.7MB/s]
Downloading (…)l-00017-of-00033.bin:  44% 178M/405M [00:01<00:02, 89.9MB/s]
Downloading (…)l-00017-of-00033.bin:  47% 189M/405M [00:02<00:02, 90.7MB/s]
Downloading (…)l-00017-of-00033.bin:  49% 199M/405M [00:02<00:02, 91.2MB/s]
Downloading (…)l-00017-of-00033.bin:  52% 210M/405M [00:02<00:02, 91.8MB/s]
Downloading (…)l-00017-of-00033.bin:  54% 220M/405M [00:02<00:02, 88.7MB/s]
Downloading (…)l-00017-of-00033.bin:  57% 231M/405M [00:02<00:01, 89.7MB/s]
Downloading (…)l-00017-of-00033.bin:  60% 241M/405M [00:02<00:01, 90.1MB/s]
Downloading (…)l-00017-of-00033.bin:  62% 252M/405M [00:02<00:01, 89.6MB/s]
Downloading (…)l-00017-of-00033.bin:  65% 262M/405M [00:02<00:01, 90.2MB/s]
Downloading (…)l-00017-of-00033.bin:  67% 273M/405M [00:03<00:01, 90.2MB/s]
Downloading (…)l-00017-of-00033.bin:  70% 283M/405M [00:03<00:01, 89.9MB/s]
Downloading (…)l-00017-of-00033.bin:  73% 294M/405M [00:03<00:01, 91.3MB/s]
Downloading (…)l-00017-of-00033.bin:  75% 304M/405M [00:03<00:01, 90.7MB/s]
Downloading (…)l-00017-of-00033.bin:  78% 315M/405M [00:03<00:00, 90.8MB/s]
Downloading (…)l-00017-of-00033.bin:  80% 325M/405M [00:03<00:00, 90.5MB/s]
Downloading (…)l-00017-of-00033.bin:  83% 336M/405M [00:03<00:00, 87.4MB/s]
Downloading (…)l-00017-of-00033.bin:  85% 346M/405M [00:03<00:00, 87.9MB/s]
Downloading (…)l-00017-of-00033.bin:  88% 357M/405M [00:03<00:00, 88.9MB/s]
Downloading (…)l-00017-of-00033.bin:  91% 367M/405M [00:04<00:00, 89.1MB/s]
Downloading (…)l-00017-of-00033.bin:  93% 377M/405M [00:04<00:00, 89.3MB/s]
Downloading (…)l-00017-of-00033.bin:  96% 388M/405M [00:04<00:00, 90.0MB/s]
Downloading (…)l-00017-of-00033.bin: 100% 405M/405M [00:04<00:00, 89.9MB/s]
Downloading shards:  52% 17/33 [01:26<01:21,  5.07s/it]
Downloading (…)l-00018-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00018-of-00033.bin:   3% 10.5M/405M [00:00<00:06, 65.3MB/s]
Downloading (…)l-00018-of-00033.bin:   5% 21.0M/405M [00:00<00:05, 71.3MB/s]
Downloading (…)l-00018-of-00033.bin:   8% 31.5M/405M [00:00<00:05, 73.0MB/s]
Downloading (…)l-00018-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 77.5MB/s]
Downloading (…)l-00018-of-00033.bin:  13% 52.4M/405M [00:00<00:04, 76.8MB/s]
Downloading (…)l-00018-of-00033.bin:  16% 62.9M/405M [00:00<00:04, 73.5MB/s]
Downloading (…)l-00018-of-00033.bin:  18% 73.4M/405M [00:00<00:04, 77.0MB/s]
Downloading (…)l-00018-of-00033.bin:  21% 83.9M/405M [00:01<00:04, 78.7MB/s]
Downloading (…)l-00018-of-00033.bin:  23% 94.4M/405M [00:01<00:04, 73.8MB/s]
Downloading (…)l-00018-of-00033.bin:  26% 105M/405M [00:01<00:04, 71.7MB/s] 
Downloading (…)l-00018-of-00033.bin:  28% 115M/405M [00:01<00:04, 69.0MB/s]
Downloading (…)l-00018-of-00033.bin:  31% 126M/405M [00:01<00:04, 69.2MB/s]
Downloading (…)l-00018-of-00033.bin:  34% 136M/405M [00:01<00:03, 69.5MB/s]
Downloading (…)l-00018-of-00033.bin:  36% 147M/405M [00:02<00:03, 71.0MB/s]
Downloading (…)l-00018-of-00033.bin:  39% 157M/405M [00:02<00:03, 73.0MB/s]
Downloading (…)l-00018-of-00033.bin:  41% 168M/405M [00:02<00:03, 73.4MB/s]
Downloading (…)l-00018-of-00033.bin:  44% 178M/405M [00:02<00:03, 74.2MB/s]
Downloading (…)l-00018-of-00033.bin:  47% 189M/405M [00:02<00:02, 76.7MB/s]
Downloading (…)l-00018-of-00033.bin:  49% 199M/405M [00:02<00:02, 76.7MB/s]
Downloading (…)l-00018-of-00033.bin:  52% 210M/405M [00:02<00:02, 77.4MB/s]
Downloading (…)l-00018-of-00033.bin:  54% 220M/405M [00:02<00:02, 77.6MB/s]
Downloading (…)l-00018-of-00033.bin:  57% 231M/405M [00:03<00:02, 76.8MB/s]
Downloading (…)l-00018-of-00033.bin:  60% 241M/405M [00:03<00:02, 74.9MB/s]
Downloading (…)l-00018-of-00033.bin:  62% 252M/405M [00:03<00:02, 75.4MB/s]
Downloading (…)l-00018-of-00033.bin:  65% 262M/405M [00:03<00:01, 76.6MB/s]
Downloading (…)l-00018-of-00033.bin:  67% 273M/405M [00:03<00:01, 76.4MB/s]
Downloading (…)l-00018-of-00033.bin:  70% 283M/405M [00:03<00:01, 74.7MB/s]
Downloading (…)l-00018-of-00033.bin:  73% 294M/405M [00:03<00:01, 74.2MB/s]
Downloading (…)l-00018-of-00033.bin:  75% 304M/405M [00:04<00:01, 75.7MB/s]
Downloading (…)l-00018-of-00033.bin:  78% 315M/405M [00:04<00:01, 75.4MB/s]
Downloading (…)l-00018-of-00033.bin:  80% 325M/405M [00:04<00:01, 75.0MB/s]
Downloading (…)l-00018-of-00033.bin:  83% 336M/405M [00:04<00:00, 77.2MB/s]
Downloading (…)l-00018-of-00033.bin:  85% 346M/405M [00:04<00:00, 77.7MB/s]
Downloading (…)l-00018-of-00033.bin:  88% 357M/405M [00:04<00:00, 78.4MB/s]
Downloading (…)l-00018-of-00033.bin:  91% 367M/405M [00:04<00:00, 78.9MB/s]
Downloading (…)l-00018-of-00033.bin:  93% 377M/405M [00:05<00:00, 69.7MB/s]
Downloading (…)l-00018-of-00033.bin:  96% 388M/405M [00:05<00:00, 73.2MB/s]
Downloading (…)l-00018-of-00033.bin: 100% 405M/405M [00:05<00:00, 74.2MB/s]
Downloading shards:  55% 18/33 [01:32<01:19,  5.27s/it]
Downloading (…)l-00019-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00019-of-00033.bin:   3% 10.5M/405M [00:00<00:05, 73.2MB/s]
Downloading (…)l-00019-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 78.8MB/s]
Downloading (…)l-00019-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 81.8MB/s]
Downloading (…)l-00019-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 79.6MB/s]
Downloading (…)l-00019-of-00033.bin:  13% 52.4M/405M [00:00<00:04, 79.6MB/s]
Downloading (…)l-00019-of-00033.bin:  16% 62.9M/405M [00:00<00:04, 80.4MB/s]
Downloading (…)l-00019-of-00033.bin:  18% 73.4M/405M [00:00<00:04, 80.0MB/s]
Downloading (…)l-00019-of-00033.bin:  21% 83.9M/405M [00:01<00:04, 78.0MB/s]
Downloading (…)l-00019-of-00033.bin:  23% 94.4M/405M [00:01<00:04, 75.9MB/s]
Downloading (…)l-00019-of-00033.bin:  26% 105M/405M [00:01<00:04, 74.8MB/s] 
Downloading (…)l-00019-of-00033.bin:  28% 115M/405M [00:01<00:03, 75.0MB/s]
Downloading (…)l-00019-of-00033.bin:  31% 126M/405M [00:01<00:03, 73.5MB/s]
Downloading (…)l-00019-of-00033.bin:  34% 136M/405M [00:01<00:03, 73.7MB/s]
Downloading (…)l-00019-of-00033.bin:  36% 147M/405M [00:01<00:03, 73.7MB/s]
Downloading (…)l-00019-of-00033.bin:  39% 157M/405M [00:02<00:03, 76.2MB/s]
Downloading (…)l-00019-of-00033.bin:  41% 168M/405M [00:02<00:03, 77.7MB/s]
Downloading (…)l-00019-of-00033.bin:  44% 178M/405M [00:02<00:02, 78.6MB/s]
Downloading (…)l-00019-of-00033.bin:  47% 189M/405M [00:02<00:02, 77.2MB/s]
Downloading (…)l-00019-of-00033.bin:  49% 199M/405M [00:02<00:02, 79.3MB/s]
Downloading (…)l-00019-of-00033.bin:  52% 210M/405M [00:02<00:02, 79.7MB/s]
Downloading (…)l-00019-of-00033.bin:  54% 220M/405M [00:02<00:02, 78.5MB/s]
Downloading (…)l-00019-of-00033.bin:  57% 231M/405M [00:02<00:02, 79.0MB/s]
Downloading (…)l-00019-of-00033.bin:  60% 241M/405M [00:03<00:02, 75.1MB/s]
Downloading (…)l-00019-of-00033.bin:  62% 252M/405M [00:03<00:02, 71.8MB/s]
Downloading (…)l-00019-of-00033.bin:  65% 262M/405M [00:03<00:02, 71.0MB/s]
Downloading (…)l-00019-of-00033.bin:  67% 273M/405M [00:03<00:01, 71.1MB/s]
Downloading (…)l-00019-of-00033.bin:  70% 283M/405M [00:03<00:01, 68.2MB/s]
Downloading (…)l-00019-of-00033.bin:  73% 294M/405M [00:03<00:01, 64.5MB/s]
Downloading (…)l-00019-of-00033.bin:  75% 304M/405M [00:04<00:01, 63.3MB/s]
Downloading (…)l-00019-of-00033.bin:  78% 315M/405M [00:04<00:01, 66.9MB/s]
Downloading (…)l-00019-of-00033.bin:  80% 325M/405M [00:04<00:01, 67.0MB/s]
Downloading (…)l-00019-of-00033.bin:  83% 336M/405M [00:04<00:00, 71.3MB/s]
Downloading (…)l-00019-of-00033.bin:  85% 346M/405M [00:04<00:00, 74.2MB/s]
Downloading (…)l-00019-of-00033.bin:  88% 357M/405M [00:04<00:00, 77.4MB/s]
Downloading (…)l-00019-of-00033.bin:  91% 367M/405M [00:04<00:00, 76.9MB/s]
Downloading (…)l-00019-of-00033.bin:  93% 377M/405M [00:05<00:00, 79.2MB/s]
Downloading (…)l-00019-of-00033.bin:  96% 388M/405M [00:05<00:00, 80.3MB/s]
Downloading (…)l-00019-of-00033.bin: 100% 405M/405M [00:05<00:00, 75.3MB/s]
Downloading shards:  58% 19/33 [01:38<01:15,  5.40s/it]
Downloading (…)l-00020-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00020-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 90.5MB/s]
Downloading (…)l-00020-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 93.3MB/s]
Downloading (…)l-00020-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 92.7MB/s]
Downloading (…)l-00020-of-00033.bin:  10% 41.9M/405M [00:00<00:03, 92.9MB/s]
Downloading (…)l-00020-of-00033.bin:  13% 52.4M/405M [00:00<00:04, 85.5MB/s]
Downloading (…)l-00020-of-00033.bin:  16% 62.9M/405M [00:00<00:03, 88.1MB/s]
Downloading (…)l-00020-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 90.2MB/s]
Downloading (…)l-00020-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 91.3MB/s]
Downloading (…)l-00020-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 92.5MB/s]
Downloading (…)l-00020-of-00033.bin:  26% 105M/405M [00:01<00:03, 92.4MB/s] 
Downloading (…)l-00020-of-00033.bin:  28% 115M/405M [00:01<00:03, 91.6MB/s]
Downloading (…)l-00020-of-00033.bin:  31% 126M/405M [00:01<00:03, 92.2MB/s]
Downloading (…)l-00020-of-00033.bin:  34% 136M/405M [00:01<00:02, 91.5MB/s]
Downloading (…)l-00020-of-00033.bin:  36% 147M/405M [00:01<00:02, 92.5MB/s]
Downloading (…)l-00020-of-00033.bin:  39% 157M/405M [00:01<00:02, 92.2MB/s]
Downloading (…)l-00020-of-00033.bin:  41% 168M/405M [00:01<00:02, 91.8MB/s]
Downloading (…)l-00020-of-00033.bin:  44% 178M/405M [00:01<00:02, 92.9MB/s]
Downloading (…)l-00020-of-00033.bin:  47% 189M/405M [00:02<00:02, 92.4MB/s]
Downloading (…)l-00020-of-00033.bin:  49% 199M/405M [00:02<00:02, 92.7MB/s]
Downloading (…)l-00020-of-00033.bin:  52% 210M/405M [00:02<00:02, 91.1MB/s]
Downloading (…)l-00020-of-00033.bin:  54% 220M/405M [00:02<00:02, 91.4MB/s]
Downloading (…)l-00020-of-00033.bin:  57% 231M/405M [00:02<00:01, 91.5MB/s]
Downloading (…)l-00020-of-00033.bin:  60% 241M/405M [00:02<00:01, 92.3MB/s]
Downloading (…)l-00020-of-00033.bin:  62% 252M/405M [00:02<00:01, 91.8MB/s]
Downloading (…)l-00020-of-00033.bin:  65% 262M/405M [00:02<00:01, 92.7MB/s]
Downloading (…)l-00020-of-00033.bin:  67% 273M/405M [00:02<00:01, 92.3MB/s]
Downloading (…)l-00020-of-00033.bin:  70% 283M/405M [00:03<00:01, 90.7MB/s]
Downloading (…)l-00020-of-00033.bin:  73% 294M/405M [00:03<00:01, 90.3MB/s]
Downloading (…)l-00020-of-00033.bin:  75% 304M/405M [00:03<00:01, 91.4MB/s]
Downloading (…)l-00020-of-00033.bin:  78% 315M/405M [00:03<00:00, 90.9MB/s]
Downloading (…)l-00020-of-00033.bin:  80% 325M/405M [00:03<00:00, 89.7MB/s]
Downloading (…)l-00020-of-00033.bin:  83% 336M/405M [00:03<00:00, 88.4MB/s]
Downloading (…)l-00020-of-00033.bin:  85% 346M/405M [00:03<00:00, 88.3MB/s]
Downloading (…)l-00020-of-00033.bin:  88% 357M/405M [00:03<00:00, 88.7MB/s]
Downloading (…)l-00020-of-00033.bin:  91% 367M/405M [00:04<00:00, 89.0MB/s]
Downloading (…)l-00020-of-00033.bin:  93% 377M/405M [00:04<00:00, 88.6MB/s]
Downloading (…)l-00020-of-00033.bin:  96% 388M/405M [00:04<00:00, 88.7MB/s]
Downloading (…)l-00020-of-00033.bin: 100% 405M/405M [00:04<00:00, 90.5MB/s]
Downloading shards:  61% 20/33 [01:42<01:07,  5.21s/it]
Downloading (…)l-00021-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00021-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 90.2MB/s]
Downloading (…)l-00021-of-00033.bin:   5% 21.0M/405M [00:00<00:04, 88.8MB/s]
Downloading (…)l-00021-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 91.4MB/s]
Downloading (…)l-00021-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 86.7MB/s]
Downloading (…)l-00021-of-00033.bin:  13% 52.4M/405M [00:00<00:04, 83.2MB/s]
Downloading (…)l-00021-of-00033.bin:  16% 62.9M/405M [00:00<00:04, 83.3MB/s]
Downloading (…)l-00021-of-00033.bin:  18% 73.4M/405M [00:00<00:03, 84.9MB/s]
Downloading (…)l-00021-of-00033.bin:  21% 83.9M/405M [00:00<00:03, 84.8MB/s]
Downloading (…)l-00021-of-00033.bin:  23% 94.4M/405M [00:01<00:03, 86.6MB/s]
Downloading (…)l-00021-of-00033.bin:  26% 105M/405M [00:01<00:03, 84.6MB/s] 
Downloading (…)l-00021-of-00033.bin:  28% 115M/405M [00:01<00:03, 86.6MB/s]
Downloading (…)l-00021-of-00033.bin:  31% 126M/405M [00:01<00:03, 87.9MB/s]
Downloading (…)l-00021-of-00033.bin:  34% 136M/405M [00:01<00:03, 87.9MB/s]
Downloading (…)l-00021-of-00033.bin:  36% 147M/405M [00:01<00:02, 87.5MB/s]
Downloading (…)l-00021-of-00033.bin:  39% 157M/405M [00:01<00:02, 85.6MB/s]
Downloading (…)l-00021-of-00033.bin:  41% 168M/405M [00:01<00:02, 85.6MB/s]
Downloading (…)l-00021-of-00033.bin:  44% 178M/405M [00:02<00:02, 87.1MB/s]
Downloading (…)l-00021-of-00033.bin:  47% 189M/405M [00:02<00:02, 83.4MB/s]
Downloading (…)l-00021-of-00033.bin:  49% 199M/405M [00:02<00:02, 85.5MB/s]
Downloading (…)l-00021-of-00033.bin:  52% 210M/405M [00:02<00:02, 87.6MB/s]
Downloading (…)l-00021-of-00033.bin:  54% 220M/405M [00:02<00:02, 88.0MB/s]
Downloading (…)l-00021-of-00033.bin:  57% 231M/405M [00:02<00:01, 88.5MB/s]
Downloading (…)l-00021-of-00033.bin:  60% 241M/405M [00:02<00:01, 89.4MB/s]
Downloading (…)l-00021-of-00033.bin:  62% 252M/405M [00:02<00:01, 89.8MB/s]
Downloading (…)l-00021-of-00033.bin:  65% 262M/405M [00:03<00:01, 89.3MB/s]
Downloading (…)l-00021-of-00033.bin:  67% 273M/405M [00:03<00:01, 90.4MB/s]
Downloading (…)l-00021-of-00033.bin:  70% 283M/405M [00:03<00:01, 89.6MB/s]
Downloading (…)l-00021-of-00033.bin:  73% 294M/405M [00:03<00:01, 89.5MB/s]
Downloading (…)l-00021-of-00033.bin:  75% 304M/405M [00:03<00:01, 88.2MB/s]
Downloading (…)l-00021-of-00033.bin:  78% 315M/405M [00:03<00:01, 88.7MB/s]
Downloading (…)l-00021-of-00033.bin:  80% 325M/405M [00:03<00:00, 88.3MB/s]
Downloading (…)l-00021-of-00033.bin:  83% 336M/405M [00:03<00:00, 83.0MB/s]
Downloading (…)l-00021-of-00033.bin:  85% 346M/405M [00:03<00:00, 84.0MB/s]
Downloading (…)l-00021-of-00033.bin:  88% 357M/405M [00:04<00:00, 85.6MB/s]
Downloading (…)l-00021-of-00033.bin:  91% 367M/405M [00:04<00:00, 87.6MB/s]
Downloading (…)l-00021-of-00033.bin:  93% 377M/405M [00:04<00:00, 88.8MB/s]
Downloading (…)l-00021-of-00033.bin:  96% 388M/405M [00:04<00:00, 89.2MB/s]
Downloading (…)l-00021-of-00033.bin: 100% 405M/405M [00:04<00:00, 87.3MB/s]
Downloading shards:  64% 21/33 [01:47<01:01,  5.13s/it]
Downloading (…)l-00022-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00022-of-00033.bin:   3% 10.5M/405M [00:00<00:05, 75.8MB/s]
Downloading (…)l-00022-of-00033.bin:   5% 21.0M/405M [00:00<00:05, 75.2MB/s]
Downloading (…)l-00022-of-00033.bin:   8% 31.5M/405M [00:00<00:05, 73.8MB/s]
Downloading (…)l-00022-of-00033.bin:  10% 41.9M/405M [00:00<00:05, 71.4MB/s]
Downloading (…)l-00022-of-00033.bin:  13% 52.4M/405M [00:00<00:05, 70.3MB/s]
Downloading (…)l-00022-of-00033.bin:  16% 62.9M/405M [00:00<00:04, 68.9MB/s]
Downloading (…)l-00022-of-00033.bin:  18% 73.4M/405M [00:01<00:04, 68.8MB/s]
Downloading (…)l-00022-of-00033.bin:  21% 83.9M/405M [00:01<00:04, 66.9MB/s]
Downloading (…)l-00022-of-00033.bin:  23% 94.4M/405M [00:01<00:04, 66.1MB/s]
Downloading (…)l-00022-of-00033.bin:  26% 105M/405M [00:01<00:04, 67.6MB/s] 
Downloading (…)l-00022-of-00033.bin:  28% 115M/405M [00:01<00:04, 62.8MB/s]
Downloading (…)l-00022-of-00033.bin:  31% 126M/405M [00:01<00:04, 59.5MB/s]
Downloading (…)l-00022-of-00033.bin:  34% 136M/405M [00:02<00:04, 57.5MB/s]
Downloading (…)l-00022-of-00033.bin:  36% 147M/405M [00:02<00:04, 57.3MB/s]
Downloading (…)l-00022-of-00033.bin:  39% 157M/405M [00:02<00:04, 61.0MB/s]
Downloading (…)l-00022-of-00033.bin:  41% 168M/405M [00:02<00:03, 63.3MB/s]
Downloading (…)l-00022-of-00033.bin:  44% 178M/405M [00:02<00:03, 64.5MB/s]
Downloading (…)l-00022-of-00033.bin:  47% 189M/405M [00:02<00:03, 66.6MB/s]
Downloading (…)l-00022-of-00033.bin:  49% 199M/405M [00:03<00:03, 60.1MB/s]
Downloading (…)l-00022-of-00033.bin:  52% 210M/405M [00:03<00:03, 62.1MB/s]
Downloading (…)l-00022-of-00033.bin:  54% 220M/405M [00:03<00:02, 65.7MB/s]
Downloading (…)l-00022-of-00033.bin:  57% 231M/405M [00:03<00:02, 67.6MB/s]
Downloading (…)l-00022-of-00033.bin:  60% 241M/405M [00:03<00:02, 62.7MB/s]
Downloading (…)l-00022-of-00033.bin:  62% 252M/405M [00:03<00:02, 57.8MB/s]
Downloading (…)l-00022-of-00033.bin:  65% 262M/405M [00:04<00:02, 62.6MB/s]
Downloading (…)l-00022-of-00033.bin:  67% 273M/405M [00:04<00:02, 63.6MB/s]
Downloading (…)l-00022-of-00033.bin:  70% 283M/405M [00:04<00:01, 66.9MB/s]
Downloading (…)l-00022-of-00033.bin:  73% 294M/405M [00:04<00:01, 67.3MB/s]
Downloading (…)l-00022-of-00033.bin:  75% 304M/405M [00:04<00:01, 65.9MB/s]
Downloading (…)l-00022-of-00033.bin:  78% 315M/405M [00:04<00:01, 66.9MB/s]
Downloading (…)l-00022-of-00033.bin:  80% 325M/405M [00:05<00:01, 66.9MB/s]
Downloading (…)l-00022-of-00033.bin:  83% 336M/405M [00:05<00:01, 62.2MB/s]
Downloading (…)l-00022-of-00033.bin:  85% 346M/405M [00:05<00:00, 59.2MB/s]
Downloading (…)l-00022-of-00033.bin:  88% 357M/405M [00:05<00:00, 61.6MB/s]
Downloading (…)l-00022-of-00033.bin:  91% 367M/405M [00:05<00:00, 63.6MB/s]
Downloading (…)l-00022-of-00033.bin:  93% 377M/405M [00:05<00:00, 64.5MB/s]
Downloading (…)l-00022-of-00033.bin:  96% 388M/405M [00:06<00:00, 62.8MB/s]
Downloading (…)l-00022-of-00033.bin: 100% 405M/405M [00:06<00:00, 64.3MB/s]
Downloading shards:  67% 22/33 [01:54<01:01,  5.58s/it]
Downloading (…)l-00023-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00023-of-00033.bin:   3% 10.5M/405M [00:00<00:06, 57.8MB/s]
Downloading (…)l-00023-of-00033.bin:   5% 21.0M/405M [00:00<00:06, 60.6MB/s]
Downloading (…)l-00023-of-00033.bin:   8% 31.5M/405M [00:00<00:05, 64.3MB/s]
Downloading (…)l-00023-of-00033.bin:  10% 41.9M/405M [00:00<00:05, 67.2MB/s]
Downloading (…)l-00023-of-00033.bin:  13% 52.4M/405M [00:00<00:05, 70.4MB/s]
Downloading (…)l-00023-of-00033.bin:  16% 62.9M/405M [00:00<00:04, 69.8MB/s]
Downloading (…)l-00023-of-00033.bin:  18% 73.4M/405M [00:01<00:04, 69.0MB/s]
Downloading (…)l-00023-of-00033.bin:  21% 83.9M/405M [00:01<00:04, 70.5MB/s]
Downloading (…)l-00023-of-00033.bin:  23% 94.4M/405M [00:01<00:04, 71.7MB/s]
Downloading (…)l-00023-of-00033.bin:  26% 105M/405M [00:01<00:04, 71.4MB/s] 
Downloading (…)l-00023-of-00033.bin:  28% 115M/405M [00:01<00:04, 70.7MB/s]
Downloading (…)l-00023-of-00033.bin:  31% 126M/405M [00:01<00:03, 72.1MB/s]
Downloading (…)l-00023-of-00033.bin:  34% 136M/405M [00:01<00:03, 72.8MB/s]
Downloading (…)l-00023-of-00033.bin:  36% 147M/405M [00:02<00:03, 71.2MB/s]
Downloading (…)l-00023-of-00033.bin:  39% 157M/405M [00:02<00:03, 71.8MB/s]
Downloading (…)l-00023-of-00033.bin:  41% 168M/405M [00:02<00:03, 70.3MB/s]
Downloading (…)l-00023-of-00033.bin:  44% 178M/405M [00:02<00:03, 68.0MB/s]
Downloading (…)l-00023-of-00033.bin:  47% 189M/405M [00:02<00:03, 65.8MB/s]
Downloading (…)l-00023-of-00033.bin:  49% 199M/405M [00:02<00:03, 63.1MB/s]
Downloading (…)l-00023-of-00033.bin:  52% 210M/405M [00:03<00:03, 64.9MB/s]
Downloading (…)l-00023-of-00033.bin:  54% 220M/405M [00:03<00:02, 68.1MB/s]
Downloading (…)l-00023-of-00033.bin:  57% 231M/405M [00:03<00:02, 69.8MB/s]
Downloading (…)l-00023-of-00033.bin:  60% 241M/405M [00:03<00:02, 71.1MB/s]
Downloading (…)l-00023-of-00033.bin:  62% 252M/405M [00:03<00:02, 69.6MB/s]
Downloading (…)l-00023-of-00033.bin:  65% 262M/405M [00:03<00:02, 70.4MB/s]
Downloading (…)l-00023-of-00033.bin:  67% 273M/405M [00:03<00:01, 69.3MB/s]
Downloading (…)l-00023-of-00033.bin:  70% 283M/405M [00:04<00:01, 68.8MB/s]
Downloading (…)l-00023-of-00033.bin:  73% 294M/405M [00:04<00:01, 63.8MB/s]
Downloading (…)l-00023-of-00033.bin:  75% 304M/405M [00:04<00:01, 63.9MB/s]
Downloading (…)l-00023-of-00033.bin:  78% 315M/405M [00:04<00:01, 67.8MB/s]
Downloading (…)l-00023-of-00033.bin:  80% 325M/405M [00:04<00:01, 68.7MB/s]
Downloading (…)l-00023-of-00033.bin:  83% 336M/405M [00:04<00:01, 68.6MB/s]
Downloading (…)l-00023-of-00033.bin:  85% 346M/405M [00:05<00:00, 69.5MB/s]
Downloading (…)l-00023-of-00033.bin:  88% 357M/405M [00:05<00:00, 66.3MB/s]
Downloading (…)l-00023-of-00033.bin:  91% 367M/405M [00:05<00:00, 66.5MB/s]
Downloading (…)l-00023-of-00033.bin:  93% 377M/405M [00:05<00:00, 67.4MB/s]
Downloading (…)l-00023-of-00033.bin:  96% 388M/405M [00:05<00:00, 67.9MB/s]
Downloading (…)l-00023-of-00033.bin: 100% 405M/405M [00:05<00:00, 68.4MB/s]
Downloading shards:  70% 23/33 [02:00<00:57,  5.77s/it]
Downloading (…)l-00024-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00024-of-00033.bin:   3% 10.5M/405M [00:00<00:07, 54.9MB/s]
Downloading (…)l-00024-of-00033.bin:   5% 21.0M/405M [00:00<00:06, 55.4MB/s]
Downloading (…)l-00024-of-00033.bin:   8% 31.5M/405M [00:00<00:06, 61.9MB/s]
Downloading (…)l-00024-of-00033.bin:  10% 41.9M/405M [00:00<00:05, 65.7MB/s]
Downloading (…)l-00024-of-00033.bin:  13% 52.4M/405M [00:00<00:05, 63.9MB/s]
Downloading (…)l-00024-of-00033.bin:  16% 62.9M/405M [00:01<00:05, 64.1MB/s]
Downloading (…)l-00024-of-00033.bin:  18% 73.4M/405M [00:01<00:05, 63.4MB/s]
Downloading (…)l-00024-of-00033.bin:  21% 83.9M/405M [00:01<00:05, 62.7MB/s]
Downloading (…)l-00024-of-00033.bin:  23% 94.4M/405M [00:01<00:04, 64.3MB/s]
Downloading (…)l-00024-of-00033.bin:  26% 105M/405M [00:01<00:04, 65.8MB/s] 
Downloading (…)l-00024-of-00033.bin:  28% 115M/405M [00:01<00:04, 65.3MB/s]
Downloading (…)l-00024-of-00033.bin:  31% 126M/405M [00:01<00:04, 63.9MB/s]
Downloading (…)l-00024-of-00033.bin:  34% 136M/405M [00:02<00:04, 64.3MB/s]
Downloading (…)l-00024-of-00033.bin:  36% 147M/405M [00:02<00:04, 61.3MB/s]
Downloading (…)l-00024-of-00033.bin:  39% 157M/405M [00:02<00:03, 65.0MB/s]
Downloading (…)l-00024-of-00033.bin:  41% 168M/405M [00:02<00:03, 60.7MB/s]
Downloading (…)l-00024-of-00033.bin:  44% 178M/405M [00:02<00:03, 64.4MB/s]
Downloading (…)l-00024-of-00033.bin:  47% 189M/405M [00:02<00:03, 65.6MB/s]
Downloading (…)l-00024-of-00033.bin:  49% 199M/405M [00:03<00:03, 64.2MB/s]
Downloading (…)l-00024-of-00033.bin:  52% 210M/405M [00:03<00:03, 64.0MB/s]
Downloading (…)l-00024-of-00033.bin:  54% 220M/405M [00:03<00:02, 62.8MB/s]
Downloading (…)l-00024-of-00033.bin:  57% 231M/405M [00:03<00:02, 62.1MB/s]
Downloading (…)l-00024-of-00033.bin:  60% 241M/405M [00:03<00:02, 67.7MB/s]
Downloading (…)l-00024-of-00033.bin:  62% 252M/405M [00:03<00:02, 66.6MB/s]
Downloading (…)l-00024-of-00033.bin:  65% 262M/405M [00:04<00:02, 65.3MB/s]
Downloading (…)l-00024-of-00033.bin:  67% 273M/405M [00:04<00:02, 64.4MB/s]
Downloading (…)l-00024-of-00033.bin:  70% 283M/405M [00:04<00:01, 63.4MB/s]
Downloading (…)l-00024-of-00033.bin:  73% 294M/405M [00:04<00:01, 65.8MB/s]
Downloading (…)l-00024-of-00033.bin:  75% 304M/405M [00:04<00:01, 64.8MB/s]
Downloading (…)l-00024-of-00033.bin:  78% 315M/405M [00:04<00:01, 67.1MB/s]
Downloading (…)l-00024-of-00033.bin:  80% 325M/405M [00:05<00:01, 68.2MB/s]
Downloading (…)l-00024-of-00033.bin:  83% 336M/405M [00:05<00:00, 69.7MB/s]
Downloading (…)l-00024-of-00033.bin:  85% 346M/405M [00:05<00:00, 69.4MB/s]
Downloading (…)l-00024-of-00033.bin:  88% 357M/405M [00:05<00:00, 70.7MB/s]
Downloading (…)l-00024-of-00033.bin:  91% 367M/405M [00:05<00:00, 68.6MB/s]
Downloading (…)l-00024-of-00033.bin:  93% 377M/405M [00:05<00:00, 66.7MB/s]
Downloading (…)l-00024-of-00033.bin:  96% 388M/405M [00:05<00:00, 64.4MB/s]
Downloading (…)l-00024-of-00033.bin: 100% 405M/405M [00:06<00:00, 64.1MB/s]
Downloading shards:  73% 24/33 [02:07<00:54,  6.05s/it]
Downloading (…)l-00025-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00025-of-00033.bin:   3% 10.5M/405M [00:00<00:05, 71.8MB/s]
Downloading (…)l-00025-of-00033.bin:   5% 21.0M/405M [00:00<00:05, 74.3MB/s]
Downloading (…)l-00025-of-00033.bin:   8% 31.5M/405M [00:00<00:04, 79.3MB/s]
Downloading (…)l-00025-of-00033.bin:  10% 41.9M/405M [00:00<00:04, 79.9MB/s]
Downloading (…)l-00025-of-00033.bin:  13% 52.4M/405M [00:00<00:04, 79.7MB/s]
Downloading (…)l-00025-of-00033.bin:  16% 62.9M/405M [00:00<00:04, 74.7MB/s]
Downloading (…)l-00025-of-00033.bin:  18% 73.4M/405M [00:00<00:04, 69.7MB/s]
Downloading (…)l-00025-of-00033.bin:  21% 83.9M/405M [00:01<00:05, 59.2MB/s]
Downloading (…)l-00025-of-00033.bin:  23% 94.4M/405M [00:01<00:05, 54.9MB/s]
Downloading (…)l-00025-of-00033.bin:  26% 105M/405M [00:01<00:05, 57.8MB/s] 
Downloading (…)l-00025-of-00033.bin:  28% 115M/405M [00:01<00:05, 54.7MB/s]
Downloading (…)l-00025-of-00033.bin:  31% 126M/405M [00:01<00:04, 59.3MB/s]
Downloading (…)l-00025-of-00033.bin:  34% 136M/405M [00:02<00:04, 61.8MB/s]
Downloading (…)l-00025-of-00033.bin:  36% 147M/405M [00:02<00:04, 60.2MB/s]
Downloading (…)l-00025-of-00033.bin:  39% 157M/405M [00:02<00:03, 65.4MB/s]
Downloading (…)l-00025-of-00033.bin:  41% 168M/405M [00:02<00:03, 65.7MB/s]
Downloading (…)l-00025-of-00033.bin:  44% 178M/405M [00:02<00:03, 63.4MB/s]
Downloading (…)l-00025-of-00033.bin:  47% 189M/405M [00:02<00:03, 63.5MB/s]
Downloading (…)l-00025-of-00033.bin:  49% 199M/405M [00:03<00:03, 65.9MB/s]
Downloading (…)l-00025-of-00033.bin:  52% 210M/405M [00:03<00:02, 69.7MB/s]
Downloading (…)l-00025-of-00033.bin:  54% 220M/405M [00:03<00:02, 71.5MB/s]
Downloading (…)l-00025-of-00033.bin:  57% 231M/405M [00:03<00:02, 71.6MB/s]
Downloading (…)l-00025-of-00033.bin:  60% 241M/405M [00:03<00:02, 71.4MB/s]
Downloading (…)l-00025-of-00033.bin:  62% 252M/405M [00:03<00:02, 74.2MB/s]
Downloading (…)l-00025-of-00033.bin:  65% 262M/405M [00:03<00:01, 72.2MB/s]
Downloading (…)l-00025-of-00033.bin:  67% 273M/405M [00:04<00:01, 72.1MB/s]
Downloading (…)l-00025-of-00033.bin:  70% 283M/405M [00:04<00:01, 71.5MB/s]
Downloading (…)l-00025-of-00033.bin:  73% 294M/405M [00:04<00:01, 70.7MB/s]
Downloading (…)l-00025-of-00033.bin:  75% 304M/405M [00:04<00:01, 71.4MB/s]
Downloading (…)l-00025-of-00033.bin:  78% 315M/405M [00:04<00:01, 72.6MB/s]
Downloading (…)l-00025-of-00033.bin:  80% 325M/405M [00:04<00:01, 70.0MB/s]
Downloading (…)l-00025-of-00033.bin:  83% 336M/405M [00:04<00:01, 67.3MB/s]
Downloading (…)l-00025-of-00033.bin:  85% 346M/405M [00:05<00:00, 68.6MB/s]
Downloading (…)l-00025-of-00033.bin:  88% 357M/405M [00:05<00:00, 71.9MB/s]
Downloading (…)l-00025-of-00033.bin:  91% 367M/405M [00:05<00:00, 73.8MB/s]
Downloading (…)l-00025-of-00033.bin:  93% 377M/405M [00:05<00:00, 73.9MB/s]
Downloading (…)l-00025-of-00033.bin:  96% 388M/405M [00:05<00:00, 76.0MB/s]
Downloading (…)l-00025-of-00033.bin: 100% 405M/405M [00:05<00:00, 69.0MB/s]
Downloading shards:  76% 25/33 [02:13<00:48,  6.11s/it]
Downloading (…)l-00026-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00026-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 97.1MB/s]
Downloading (…)l-00026-of-00033.bin:  13% 52.4M/405M [00:00<00:01, 256MB/s] 
Downloading (…)l-00026-of-00033.bin:  21% 83.9M/405M [00:00<00:01, 212MB/s]
Downloading (…)l-00026-of-00033.bin:  31% 126M/405M [00:00<00:01, 267MB/s] 
Downloading (…)l-00026-of-00033.bin:  39% 157M/405M [00:00<00:00, 275MB/s]
Downloading (…)l-00026-of-00033.bin:  47% 189M/405M [00:00<00:00, 282MB/s]
Downloading (…)l-00026-of-00033.bin:  54% 220M/405M [00:00<00:00, 279MB/s]
Downloading (…)l-00026-of-00033.bin:  62% 252M/405M [00:00<00:00, 289MB/s]
Downloading (…)l-00026-of-00033.bin:  70% 283M/405M [00:01<00:00, 282MB/s]
Downloading (…)l-00026-of-00033.bin:  78% 315M/405M [00:01<00:00, 267MB/s]
Downloading (…)l-00026-of-00033.bin:  85% 346M/405M [00:01<00:00, 279MB/s]
Downloading (…)l-00026-of-00033.bin: 100% 405M/405M [00:01<00:00, 273MB/s]
Downloading shards:  79% 26/33 [02:15<00:33,  4.84s/it]
Downloading (…)l-00027-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00027-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 84.9MB/s]
Downloading (…)l-00027-of-00033.bin:  13% 52.4M/405M [00:00<00:01, 240MB/s] 
Downloading (…)l-00027-of-00033.bin:  23% 94.4M/405M [00:00<00:01, 277MB/s]
Downloading (…)l-00027-of-00033.bin:  31% 126M/405M [00:00<00:01, 278MB/s] 
Downloading (…)l-00027-of-00033.bin:  41% 168M/405M [00:00<00:00, 322MB/s]
Downloading (…)l-00027-of-00033.bin:  52% 210M/405M [00:00<00:00, 337MB/s]
Downloading (…)l-00027-of-00033.bin:  62% 252M/405M [00:00<00:00, 309MB/s]
Downloading (…)l-00027-of-00033.bin:  73% 294M/405M [00:00<00:00, 322MB/s]
Downloading (…)l-00027-of-00033.bin:  83% 336M/405M [00:01<00:00, 312MB/s]
Downloading (…)l-00027-of-00033.bin: 100% 405M/405M [00:01<00:00, 287MB/s]
Downloading shards:  82% 27/33 [02:17<00:23,  3.93s/it]
Downloading (…)l-00028-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00028-of-00033.bin:   3% 10.5M/405M [00:00<00:06, 65.5MB/s]
Downloading (…)l-00028-of-00033.bin:  10% 41.9M/405M [00:00<00:02, 169MB/s] 
Downloading (…)l-00028-of-00033.bin:  21% 83.9M/405M [00:00<00:01, 244MB/s]
Downloading (…)l-00028-of-00033.bin:  28% 115M/405M [00:00<00:01, 255MB/s] 
Downloading (…)l-00028-of-00033.bin:  36% 147M/405M [00:00<00:01, 252MB/s]
Downloading (…)l-00028-of-00033.bin:  44% 178M/405M [00:00<00:00, 269MB/s]
Downloading (…)l-00028-of-00033.bin:  52% 210M/405M [00:00<00:00, 221MB/s]
Downloading (…)l-00028-of-00033.bin:  62% 252M/405M [00:01<00:00, 256MB/s]
Downloading (…)l-00028-of-00033.bin:  70% 283M/405M [00:01<00:00, 263MB/s]
Downloading (…)l-00028-of-00033.bin:  78% 315M/405M [00:01<00:00, 203MB/s]
Downloading (…)l-00028-of-00033.bin:  88% 357M/405M [00:01<00:00, 239MB/s]
Downloading (…)l-00028-of-00033.bin: 100% 405M/405M [00:01<00:00, 230MB/s]
Downloading shards:  85% 28/33 [02:19<00:16,  3.39s/it]
Downloading (…)l-00029-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00029-of-00033.bin:   3% 10.5M/405M [00:00<00:03, 105MB/s]
Downloading (…)l-00029-of-00033.bin:  10% 41.9M/405M [00:00<00:02, 160MB/s]
Downloading (…)l-00029-of-00033.bin:  18% 73.4M/405M [00:00<00:01, 188MB/s]
Downloading (…)l-00029-of-00033.bin:  28% 115M/405M [00:00<00:01, 249MB/s] 
Downloading (…)l-00029-of-00033.bin:  36% 147M/405M [00:00<00:01, 258MB/s]
Downloading (…)l-00029-of-00033.bin:  44% 178M/405M [00:00<00:00, 235MB/s]
Downloading (…)l-00029-of-00033.bin:  54% 220M/405M [00:00<00:00, 274MB/s]
Downloading (…)l-00029-of-00033.bin:  62% 252M/405M [00:01<00:00, 269MB/s]
Downloading (…)l-00029-of-00033.bin:  70% 283M/405M [00:01<00:00, 274MB/s]
Downloading (…)l-00029-of-00033.bin:  78% 315M/405M [00:01<00:00, 268MB/s]
Downloading (…)l-00029-of-00033.bin:  85% 346M/405M [00:01<00:00, 258MB/s]
Downloading (…)l-00029-of-00033.bin:  93% 377M/405M [00:01<00:00, 252MB/s]
Downloading (…)l-00029-of-00033.bin: 100% 405M/405M [00:01<00:00, 244MB/s]
Downloading shards:  88% 29/33 [02:21<00:11,  2.99s/it]
Downloading (…)l-00030-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00030-of-00033.bin:   3% 10.5M/405M [00:00<00:04, 93.4MB/s]
Downloading (…)l-00030-of-00033.bin:  10% 41.9M/405M [00:00<00:02, 166MB/s] 
Downloading (…)l-00030-of-00033.bin:  21% 83.9M/405M [00:00<00:01, 247MB/s]
Downloading (…)l-00030-of-00033.bin:  28% 115M/405M [00:00<00:01, 249MB/s] 
Downloading (…)l-00030-of-00033.bin:  36% 147M/405M [00:00<00:01, 252MB/s]
Downloading (…)l-00030-of-00033.bin:  44% 178M/405M [00:00<00:00, 239MB/s]
Downloading (…)l-00030-of-00033.bin:  54% 220M/405M [00:00<00:00, 276MB/s]
Downloading (…)l-00030-of-00033.bin:  65% 262M/405M [00:01<00:00, 293MB/s]
Downloading (…)l-00030-of-00033.bin:  73% 294M/405M [00:01<00:00, 251MB/s]
Downloading (…)l-00030-of-00033.bin:  80% 325M/405M [00:01<00:00, 251MB/s]
Downloading (…)l-00030-of-00033.bin:  88% 357M/405M [00:01<00:00, 265MB/s]
Downloading (…)l-00030-of-00033.bin: 100% 405M/405M [00:01<00:00, 251MB/s]
Downloading shards:  91% 30/33 [02:23<00:08,  2.69s/it]
Downloading (…)l-00031-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00031-of-00033.bin:   8% 31.5M/405M [00:00<00:01, 260MB/s]
Downloading (…)l-00031-of-00033.bin:  21% 83.9M/405M [00:00<00:00, 358MB/s]
Downloading (…)l-00031-of-00033.bin:  31% 126M/405M [00:00<00:00, 326MB/s] 
Downloading (…)l-00031-of-00033.bin:  41% 168M/405M [00:00<00:00, 314MB/s]
Downloading (…)l-00031-of-00033.bin:  52% 210M/405M [00:00<00:00, 328MB/s]
Downloading (…)l-00031-of-00033.bin:  62% 252M/405M [00:00<00:00, 330MB/s]
Downloading (…)l-00031-of-00033.bin:  73% 294M/405M [00:00<00:00, 303MB/s]
Downloading (…)l-00031-of-00033.bin:  83% 336M/405M [00:01<00:00, 319MB/s]
Downloading (…)l-00031-of-00033.bin: 100% 405M/405M [00:01<00:00, 318MB/s]
Downloading shards:  94% 31/33 [02:25<00:04,  2.38s/it]
Downloading (…)l-00032-of-00033.bin:   0% 0.00/405M [00:00<?, ?B/s]
Downloading (…)l-00032-of-00033.bin:   5% 21.0M/405M [00:00<00:02, 151MB/s]
Downloading (…)l-00032-of-00033.bin:  16% 62.9M/405M [00:00<00:01, 275MB/s]
Downloading (…)l-00032-of-00033.bin:  26% 105M/405M [00:00<00:00, 311MB/s] 
Downloading (…)l-00032-of-00033.bin:  36% 147M/405M [00:00<00:00, 300MB/s]
Downloading (…)l-00032-of-00033.bin:  44% 178M/405M [00:00<00:00, 297MB/s]
Downloading (…)l-00032-of-00033.bin:  54% 220M/405M [00:00<00:00, 298MB/s]
Downloading (…)l-00032-of-00033.bin:  65% 262M/405M [00:00<00:00, 316MB/s]
Downloading (…)l-00032-of-00033.bin:  75% 304M/405M [00:01<00:00, 323MB/s]
Downloading (…)l-00032-of-00033.bin:  85% 346M/405M [00:01<00:00, 336MB/s]
Downloading (…)l-00032-of-00033.bin: 100% 405M/405M [00:01<00:00, 316MB/s]
Downloading shards:  97% 32/33 [02:27<00:02,  2.52s/it]
Downloading (…)l-00033-of-00033.bin:   0% 0.00/524M [00:00<?, ?B/s]
Downloading (…)l-00033-of-00033.bin:   6% 31.5M/524M [00:00<00:01, 290MB/s]
Downloading (…)l-00033-of-00033.bin:  16% 83.9M/524M [00:00<00:01, 378MB/s]
Downloading (…)l-00033-of-00033.bin:  24% 126M/524M [00:00<00:01, 389MB/s] 
Downloading (…)l-00033-of-00033.bin:  32% 168M/524M [00:00<00:01, 307MB/s]
Downloading (…)l-00033-of-00033.bin:  40% 210M/524M [00:00<00:00, 336MB/s]
Downloading (…)l-00033-of-00033.bin:  50% 262M/524M [00:00<00:00, 359MB/s]
Downloading (…)l-00033-of-00033.bin:  58% 304M/524M [00:00<00:00, 367MB/s]
Downloading (…)l-00033-of-00033.bin:  66% 346M/524M [00:01<00:00, 300MB/s]
Downloading (…)l-00033-of-00033.bin:  74% 388M/524M [00:01<00:00, 295MB/s]
Downloading (…)l-00033-of-00033.bin:  80% 419M/524M [00:01<00:00, 253MB/s]
Downloading (…)l-00033-of-00033.bin:  86% 451M/524M [00:01<00:00, 241MB/s]
Downloading (…)l-00033-of-00033.bin:  94% 493M/524M [00:01<00:00, 252MB/s]
Downloading (…)l-00033-of-00033.bin: 100% 524M/524M [00:01<00:00, 293MB/s]
Downloading shards: 100% 33/33 [02:30<00:00,  4.55s/it]
Loading checkpoint shards: 100% 33/33 [01:03<00:00,  1.93s/it]
Downloading (…)neration_config.json: 100% 124/124 [00:00<00:00, 18.9kB/s]
Downloading tokenizer.model: 100% 500k/500k [00:00<00:00, 44.6MB/s]
Downloading (…)cial_tokens_map.json: 100% 2.00/2.00 [00:00<00:00, 745B/s]
Downloading (…)okenizer_config.json: 100% 141/141 [00:00<00:00, 47.2kB/s]
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'. 
The class this function is called from is 'LlamaTokenizer'.
trainable params: 4194304 || all params: 6742609920 || trainable%: 0.06220594176090199
***** Running training *****
  Num Epochs = 1
  Instantaneous batch size per GPU = 4
  Gradient Accumulation steps = 32
  Total train batch size (w. parallel, distributed & accumulation) = 128
  Total optimization steps = 390
  Saving steps = 39
/usr/local/lib/python3.9/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning
  warnings.warn(
wandb: Currently logged in as: utensil. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.14.2
wandb: Run data is saved locally in /content/code/wandb/run-20230418_023339-l81k9fdc
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run cool-flower-1
wandb: ⭐️ View project at https://wandb.ai/utensil/Alpaca-CoT-Colab
wandb: 🚀 View run at https://wandb.ai/utensil/Alpaca-CoT-Colab/runs/l81k9fdc
{'loss': 1.4123, 'learning_rate': 0.00015384615384615382, 'epoch': 0.05}
 10% 39/390 [53:44<8:08:03, 83.43s/it]
  0% 0/250 [00:00<?, ?it/s]
  1% 2/250 [00:01<03:27,  1.19it/s]
  1% 3/250 [00:03<05:46,  1.40s/it]
  2% 4/250 [00:04<05:00,  1.22s/it]
  2% 5/250 [00:06<05:34,  1.36s/it]
  2% 6/250 [00:07<05:18,  1.30s/it]
  3% 7/250 [00:09<06:21,  1.57s/it]
  3% 8/250 [00:10<05:24,  1.34s/it]
  4% 9/250 [00:11<05:10,  1.29s/it]
  4% 10/250 [00:12<04:58,  1.24s/it]
  4% 11/250 [00:13<04:35,  1.15s/it]
  5% 12/250 [00:16<06:22,  1.61s/it]
  5% 13/250 [00:17<05:51,  1.48s/it]
  6% 14/250 [00:18<05:21,  1.36s/it]
  6% 15/250 [00:20<06:03,  1.55s/it]
  6% 16/250 [00:21<05:18,  1.36s/it]
  7% 17/250 [00:24<06:32,  1.69s/it]
  7% 18/250 [00:25<06:18,  1.63s/it]
  8% 19/250 [00:26<05:47,  1.51s/it]
  8% 20/250 [00:27<05:17,  1.38s/it]
  8% 21/250 [00:28<04:54,  1.28s/it]
  9% 22/250 [00:30<05:37,  1.48s/it]
  9% 23/250 [00:31<04:56,  1.31s/it]
 10% 24/250 [00:32<04:35,  1.22s/it]
 10% 25/250 [00:34<04:40,  1.25s/it]
 10% 26/250 [00:35<05:06,  1.37s/it]
 11% 27/250 [00:36<04:46,  1.29s/it]
 11% 28/250 [00:37<04:20,  1.17s/it]
 12% 29/250 [00:39<04:39,  1.26s/it]
 12% 30/250 [00:41<05:38,  1.54s/it]
 12% 31/250 [00:43<05:52,  1.61s/it]
 13% 32/250 [00:44<05:40,  1.56s/it]
 13% 33/250 [00:46<05:31,  1.53s/it]
 14% 34/250 [00:48<06:43,  1.87s/it]
 14% 35/250 [00:49<05:47,  1.62s/it]
 14% 36/250 [00:50<05:09,  1.45s/it]
 15% 37/250 [00:51<04:28,  1.26s/it]
 15% 38/250 [00:52<04:11,  1.19s/it]
 16% 39/250 [00:53<03:55,  1.11s/it]
 16% 40/250 [00:54<03:50,  1.10s/it]
 16% 41/250 [00:55<03:58,  1.14s/it]
 17% 42/250 [00:56<03:46,  1.09s/it]
 17% 43/250 [00:58<04:04,  1.18s/it]
 18% 44/250 [00:59<03:59,  1.16s/it]
 18% 45/250 [01:00<04:03,  1.19s/it]
 18% 46/250 [01:02<04:10,  1.23s/it]
 19% 47/250 [01:03<04:09,  1.23s/it]
 19% 48/250 [01:05<04:45,  1.41s/it]
 20% 49/250 [01:06<04:24,  1.31s/it]
 20% 50/250 [01:08<04:59,  1.50s/it]
 20% 51/250 [01:09<05:19,  1.61s/it]
 21% 52/250 [01:12<06:22,  1.93s/it]
 21% 53/250 [01:13<05:32,  1.69s/it]
 22% 54/250 [01:15<05:18,  1.63s/it]
 22% 55/250 [01:16<05:19,  1.64s/it]
 22% 56/250 [01:18<04:57,  1.53s/it]
 23% 57/250 [01:19<04:37,  1.44s/it]
 23% 58/250 [01:21<04:46,  1.49s/it]
 24% 59/250 [01:21<04:12,  1.32s/it]
 24% 60/250 [01:23<03:57,  1.25s/it]
 24% 61/250 [01:24<04:05,  1.30s/it]
 25% 62/250 [01:25<03:43,  1.19s/it]
 25% 63/250 [01:27<04:12,  1.35s/it]
 26% 64/250 [01:28<04:34,  1.47s/it]
 26% 65/250 [01:29<03:57,  1.28s/it]
 26% 66/250 [01:30<03:45,  1.22s/it]
 27% 67/250 [01:32<04:16,  1.40s/it]
 27% 68/250 [01:34<04:37,  1.52s/it]
 28% 69/250 [01:36<05:18,  1.76s/it]
 28% 70/250 [01:37<04:26,  1.48s/it]
 28% 71/250 [01:38<04:21,  1.46s/it]
 29% 72/250 [01:39<03:53,  1.31s/it]
 29% 73/250 [01:41<03:52,  1.31s/it]
 30% 74/250 [01:42<03:25,  1.17s/it]
 30% 75/250 [01:43<03:55,  1.35s/it]
 30% 76/250 [01:44<03:34,  1.23s/it]
 31% 77/250 [01:46<03:44,  1.30s/it]
 31% 78/250 [01:47<03:40,  1.28s/it]
 32% 79/250 [01:48<03:34,  1.25s/it]
 32% 80/250 [01:49<03:15,  1.15s/it]
 32% 81/250 [01:50<03:20,  1.19s/it]
 33% 82/250 [01:52<03:49,  1.37s/it]
 33% 83/250 [01:54<04:35,  1.65s/it]
 34% 84/250 [01:56<04:35,  1.66s/it]
 34% 85/250 [01:57<04:01,  1.47s/it]
 34% 86/250 [01:58<03:44,  1.37s/it]
 35% 87/250 [02:00<03:40,  1.35s/it]
 35% 88/250 [02:01<03:37,  1.34s/it]
 36% 89/250 [02:02<03:16,  1.22s/it]
 36% 90/250 [02:04<03:58,  1.49s/it]
 36% 91/250 [02:05<03:44,  1.41s/it]
 37% 92/250 [02:07<03:45,  1.43s/it]
 37% 93/250 [02:08<03:20,  1.28s/it]
 38% 94/250 [02:09<03:05,  1.19s/it]
 38% 95/250 [02:10<02:59,  1.16s/it]
 38% 96/250 [02:11<03:12,  1.25s/it]
 39% 97/250 [02:12<02:56,  1.15s/it]
 39% 98/250 [02:13<02:37,  1.04s/it]
 40% 99/250 [02:14<02:40,  1.07s/it]
 40% 100/250 [02:15<02:47,  1.11s/it]
 40% 101/250 [02:16<02:52,  1.16s/it]
 41% 102/250 [02:18<03:06,  1.26s/it]
 41% 103/250 [02:19<02:48,  1.15s/it]
 42% 104/250 [02:20<02:43,  1.12s/it]
 42% 105/250 [02:22<03:27,  1.43s/it]
 42% 106/250 [02:24<03:35,  1.50s/it]
 43% 107/250 [02:25<03:04,  1.29s/it]
 43% 108/250 [02:26<02:59,  1.26s/it]
 44% 109/250 [02:27<02:51,  1.21s/it]
 44% 110/250 [02:28<03:07,  1.34s/it]
 44% 111/250 [02:31<03:46,  1.63s/it]
 45% 112/250 [02:32<03:20,  1.45s/it]
 45% 113/250 [02:33<03:07,  1.37s/it]
 46% 114/250 [02:34<02:54,  1.28s/it]
 46% 115/250 [02:36<03:01,  1.35s/it]
 46% 116/250 [02:36<02:39,  1.19s/it]
 47% 117/250 [02:39<03:35,  1.62s/it]
 47% 118/250 [02:40<03:23,  1.54s/it]
 48% 119/250 [02:43<04:04,  1.87s/it]
 48% 120/250 [02:46<04:35,  2.12s/it]
 48% 121/250 [02:47<03:59,  1.85s/it]
 49% 122/250 [02:49<03:49,  1.79s/it]
 49% 123/250 [02:49<03:12,  1.52s/it]
 50% 124/250 [02:51<03:15,  1.55s/it]
 50% 125/250 [02:53<03:09,  1.52s/it]
 50% 126/250 [02:54<03:11,  1.55s/it]
 51% 127/250 [02:55<02:46,  1.35s/it]
 51% 128/250 [02:58<03:42,  1.82s/it]
 52% 129/250 [03:00<03:53,  1.93s/it]
 52% 130/250 [03:01<03:10,  1.59s/it]
 52% 131/250 [03:02<02:39,  1.34s/it]
 53% 132/250 [03:03<02:31,  1.28s/it]
 53% 133/250 [03:04<02:23,  1.23s/it]
 54% 134/250 [03:05<02:11,  1.14s/it]
 54% 135/250 [03:06<02:15,  1.18s/it]
 54% 136/250 [03:08<02:34,  1.35s/it]
 55% 137/250 [03:10<02:43,  1.45s/it]
 55% 138/250 [03:11<02:34,  1.38s/it]
 56% 139/250 [03:12<02:28,  1.34s/it]
 56% 140/250 [03:13<02:26,  1.33s/it]
 56% 141/250 [03:15<02:26,  1.35s/it]
 57% 142/250 [03:17<02:52,  1.59s/it]
 57% 143/250 [03:18<02:30,  1.41s/it]
 58% 144/250 [03:20<02:46,  1.57s/it]
 58% 145/250 [03:21<02:40,  1.53s/it]
 58% 146/250 [03:22<02:19,  1.34s/it]
 59% 147/250 [03:23<02:02,  1.19s/it]
 59% 148/250 [03:24<02:00,  1.18s/it]
 60% 149/250 [03:26<02:32,  1.51s/it]
 60% 150/250 [03:28<02:30,  1.50s/it]
 60% 151/250 [03:29<02:11,  1.32s/it]
 61% 152/250 [03:30<02:13,  1.36s/it]
 61% 153/250 [03:31<01:59,  1.23s/it]
 62% 154/250 [03:33<02:03,  1.28s/it]
 62% 155/250 [03:34<01:53,  1.19s/it]
 62% 156/250 [03:35<01:45,  1.12s/it]
 63% 157/250 [03:36<02:04,  1.34s/it]
 63% 158/250 [03:37<01:53,  1.24s/it]
 64% 159/250 [03:39<01:52,  1.23s/it]
 64% 160/250 [03:40<01:51,  1.24s/it]
 64% 161/250 [03:41<01:45,  1.19s/it]
 65% 162/250 [03:42<01:45,  1.20s/it]
 65% 163/250 [03:44<02:03,  1.42s/it]
 66% 164/250 [03:45<01:59,  1.39s/it]
 66% 165/250 [03:46<01:49,  1.29s/it]
 66% 166/250 [03:48<01:43,  1.24s/it]
 67% 167/250 [03:50<02:14,  1.63s/it]
 67% 168/250 [03:51<02:00,  1.47s/it]
 68% 169/250 [03:53<01:56,  1.44s/it]
 68% 170/250 [03:53<01:41,  1.27s/it]
 68% 171/250 [03:55<01:43,  1.31s/it]
 69% 172/250 [03:57<01:49,  1.40s/it]
 69% 173/250 [03:57<01:36,  1.25s/it]
 70% 174/250 [03:59<01:31,  1.21s/it]
 70% 175/250 [03:59<01:23,  1.12s/it]
 70% 176/250 [04:00<01:20,  1.08s/it]
 71% 177/250 [04:02<01:23,  1.15s/it]
 71% 178/250 [04:03<01:19,  1.11s/it]
 72% 179/250 [04:05<01:41,  1.43s/it]
 72% 180/250 [04:06<01:32,  1.33s/it]
 72% 181/250 [04:07<01:27,  1.27s/it]
 73% 182/250 [04:08<01:20,  1.18s/it]
 73% 183/250 [04:09<01:20,  1.19s/it]
 74% 184/250 [04:10<01:14,  1.13s/it]
 74% 185/250 [04:12<01:15,  1.16s/it]
 74% 186/250 [04:13<01:11,  1.11s/it]
 75% 187/250 [04:14<01:15,  1.20s/it]
 75% 188/250 [04:16<01:22,  1.33s/it]
 76% 189/250 [04:18<01:32,  1.52s/it]
 76% 190/250 [04:19<01:36,  1.60s/it]
 76% 191/250 [04:20<01:21,  1.38s/it]
 77% 192/250 [04:21<01:14,  1.29s/it]
 77% 193/250 [04:23<01:18,  1.38s/it]
 78% 194/250 [04:24<01:17,  1.39s/it]
 78% 195/250 [04:25<01:06,  1.21s/it]
 78% 196/250 [04:26<01:05,  1.21s/it]
 79% 197/250 [04:28<01:05,  1.23s/it]
 79% 198/250 [04:29<01:02,  1.20s/it]
 80% 199/250 [04:31<01:18,  1.54s/it]
 80% 200/250 [04:32<01:11,  1.43s/it]
 80% 201/250 [04:33<01:07,  1.38s/it]
 81% 202/250 [04:35<01:01,  1.28s/it]
 81% 203/250 [04:36<00:58,  1.24s/it]
 82% 204/250 [04:37<00:57,  1.24s/it]
 82% 205/250 [04:38<00:51,  1.14s/it]
 82% 206/250 [04:39<00:51,  1.17s/it]
 83% 207/250 [04:40<00:51,  1.20s/it]
 83% 208/250 [04:41<00:48,  1.17s/it]
 84% 209/250 [04:43<00:48,  1.18s/it]
 84% 210/250 [04:44<00:45,  1.13s/it]
 84% 211/250 [04:46<00:52,  1.35s/it]
 85% 212/250 [04:47<00:47,  1.25s/it]
 85% 213/250 [04:49<01:01,  1.67s/it]
 86% 214/250 [04:50<00:54,  1.50s/it]
 86% 215/250 [04:52<00:50,  1.44s/it]
 86% 216/250 [04:53<00:49,  1.45s/it]
 87% 217/250 [04:54<00:43,  1.31s/it]
 87% 218/250 [04:56<00:47,  1.48s/it]
 88% 219/250 [04:58<00:51,  1.67s/it]
 88% 220/250 [04:59<00:45,  1.51s/it]
 88% 221/250 [05:00<00:41,  1.44s/it]
 89% 222/250 [05:01<00:35,  1.27s/it]
 89% 223/250 [05:03<00:34,  1.28s/it]
 90% 224/250 [05:04<00:36,  1.39s/it]
 90% 225/250 [05:06<00:38,  1.56s/it]
 90% 226/250 [05:07<00:33,  1.38s/it]
 91% 227/250 [05:08<00:28,  1.25s/it]
 91% 228/250 [05:10<00:31,  1.45s/it]
 92% 229/250 [05:11<00:28,  1.34s/it]
 92% 230/250 [05:13<00:30,  1.52s/it]
 92% 231/250 [05:14<00:27,  1.44s/it]
 93% 232/250 [05:15<00:24,  1.35s/it]
 93% 233/250 [05:16<00:20,  1.22s/it]
 94% 234/250 [05:18<00:22,  1.44s/it]
 94% 235/250 [05:19<00:19,  1.31s/it]
 94% 236/250 [05:21<00:18,  1.29s/it]
 95% 237/250 [05:23<00:20,  1.55s/it]
 95% 238/250 [05:23<00:15,  1.30s/it]
 96% 239/250 [05:25<00:15,  1.44s/it]
 96% 240/250 [05:27<00:16,  1.66s/it]
 96% 241/250 [05:28<00:13,  1.46s/it]
 97% 242/250 [05:29<00:10,  1.27s/it]
 97% 243/250 [05:31<00:09,  1.37s/it]
 98% 244/250 [05:33<00:08,  1.49s/it]
 98% 245/250 [05:34<00:07,  1.50s/it]
 98% 246/250 [05:35<00:05,  1.31s/it]
 99% 247/250 [05:36<00:03,  1.30s/it]
 99% 248/250 [05:37<00:02,  1.14s/it]
100% 249/250 [05:38<00:01,  1.17s/it]
                                      
{'eval_loss': 1.1518847942352295, 'eval_runtime': 341.0299, 'eval_samples_per_second': 5.865, 'eval_steps_per_second': 0.733, 'epoch': 0.1}
 10% 39/390 [59:25<8:08:03, 83.43s/it]
100% 250/250 [05:39<00:00,  1.15s/it]
                                     Traceback (most recent call last):
  File "/content/code/uniform_finetune.py", line 348, in <module>
    train(args)
  File "/content/code/uniform_finetune.py", line 316, in train
    trainer.train()
  File "/usr/local/lib/python3.9/dist-packages/transformers/trainer.py", line 1661, in train
    return inner_training_loop(
  File "/usr/local/lib/python3.9/dist-packages/transformers/trainer.py", line 2005, in _inner_training_loop
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
  File "/usr/local/lib/python3.9/dist-packages/transformers/trainer.py", line 2290, in _maybe_log_save_evaluate
    self._save_checkpoint(model, trial, metrics=metrics)
  File "/usr/local/lib/python3.9/dist-packages/transformers/trainer.py", line 2347, in _save_checkpoint
    self.save_model(output_dir, _internal_call=True)
  File "/usr/local/lib/python3.9/dist-packages/transformers/trainer.py", line 2829, in save_model
    self._save(output_dir)
  File "/usr/local/lib/python3.9/dist-packages/transformers/trainer.py", line 2872, in _save
    state_dict = self.model.state_dict()
  File "/content/code/uniform_finetune.py", line 310, in <lambda>
    lambda self, *_, **__: get_peft_model_state_dict(self, old_state_dict())
  File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1818, in state_dict
    module.state_dict(destination=destination, prefix=prefix + name + '.', keep_vars=keep_vars)
  File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1818, in state_dict
    module.state_dict(destination=destination, prefix=prefix + name + '.', keep_vars=keep_vars)
  File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1818, in state_dict
    module.state_dict(destination=destination, prefix=prefix + name + '.', keep_vars=keep_vars)
  [Previous line repeated 4 more times]
  File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1815, in state_dict
    self._save_to_state_dict(destination, prefix, keep_vars)
  File "/usr/local/lib/python3.9/dist-packages/bitsandbytes/nn/modules.py", line 268, in _save_to_state_dict
    self.weight.data = undo_layout(self.state.CxB, self.state.tile_indices)
  File "/usr/local/lib/python3.9/dist-packages/bitsandbytes/autograd/_functions.py", line 100, in undo_layout
    return outputs.reshape(rows, cols).contiguous()
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 44.00 MiB (GPU 0; 14.75 GiB total capacity; 13.11 GiB already allocated; 16.81 MiB free; 13.68 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /content/code/uniform_finetune.py:348 in <module>                            │
│                                                                              │
│   345 │   args = parser.parse_args()                                         │
│   346 │   print(args)                                                        │
│   347 │                                                                      │
│ ❱ 348 │   train(args)                                                        │
│   349                                                                        │
│                                                                              │
│ /content/code/uniform_finetune.py:316 in train                               │
│                                                                              │
│   313 │   if torch.__version__ >= "2" and sys.platform != "win32":           │
│   314 │   │   model = torch.compile(model)                                   │
│   315 │                                                                      │
│ ❱ 316 │   trainer.train()                                                    │
│   317 │                                                                      │
│   318 │   model.save_pretrained(output_dir)                                  │
│   319                                                                        │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/transformers/trainer.py:1661 in train │
│                                                                              │
│   1658 │   │   inner_training_loop = find_executable_batch_size(             │
│   1659 │   │   │   self._inner_training_loop, self._train_batch_size, args.a │
│   1660 │   │   )                                                             │
│ ❱ 1661 │   │   return inner_training_loop(                                   │
│   1662 │   │   │   args=args,                                                │
│   1663 │   │   │   resume_from_checkpoint=resume_from_checkpoint,            │
│   1664 │   │   │   trial=trial,                                              │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/transformers/trainer.py:2005 in       │
│ _inner_training_loop                                                         │
│                                                                              │
│   2002 │   │   │   │   │   self.state.epoch = epoch + (step + 1 + steps_skip │
│   2003 │   │   │   │   │   self.control = self.callback_handler.on_step_end( │
│   2004 │   │   │   │   │                                                     │
│ ❱ 2005 │   │   │   │   │   self._maybe_log_save_evaluate(tr_loss, model, tri │
│   2006 │   │   │   │   else:                                                 │
│   2007 │   │   │   │   │   self.control = self.callback_handler.on_substep_e │
│   2008                                                                       │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/transformers/trainer.py:2290 in       │
│ _maybe_log_save_evaluate                                                     │
│                                                                              │
│   2287 │   │   │   self._report_to_hp_search(trial, self.state.global_step,  │
│   2288 │   │                                                                 │
│   2289 │   │   if self.control.should_save:                                  │
│ ❱ 2290 │   │   │   self._save_checkpoint(model, trial, metrics=metrics)      │
│   2291 │   │   │   self.control = self.callback_handler.on_save(self.args, s │
│   2292 │                                                                     │
│   2293 │   def _load_rng_state(self, checkpoint):                            │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/transformers/trainer.py:2347 in       │
│ _save_checkpoint                                                             │
│                                                                              │
│   2344 │   │                                                                 │
│   2345 │   │   run_dir = self._get_output_dir(trial=trial)                   │
│   2346 │   │   output_dir = os.path.join(run_dir, checkpoint_folder)         │
│ ❱ 2347 │   │   self.save_model(output_dir, _internal_call=True)              │
│   2348 │   │   if self.deepspeed:                                            │
│   2349 │   │   │   # under zero3 model file itself doesn't get saved since i │
│   2350 │   │   │   # config `stage3_gather_16bit_weights_on_model_save` is T │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/transformers/trainer.py:2829 in       │
│ save_model                                                                   │
│                                                                              │
│   2826 │   │   │   │   │   self.deepspeed.save_checkpoint(output_dir)        │
│   2827 │   │                                                                 │
│   2828 │   │   elif self.args.should_save:                                   │
│ ❱ 2829 │   │   │   self._save(output_dir)                                    │
│   2830 │   │                                                                 │
│   2831 │   │   # Push to the Hub when `save_model` is called by the user.    │
│   2832 │   │   if self.args.push_to_hub and not _internal_call:              │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/transformers/trainer.py:2872 in _save │
│                                                                              │
│   2869 │   │   # They can then be reloaded using `from_pretrained()`         │
│   2870 │   │   if not isinstance(self.model, PreTrainedModel):               │
│   2871 │   │   │   if state_dict is None:                                    │
│ ❱ 2872 │   │   │   │   state_dict = self.model.state_dict()                  │
│   2873 │   │   │                                                             │
│   2874 │   │   │   if isinstance(unwrap_model(self.model), PreTrainedModel): │
│   2875 │   │   │   │   unwrap_model(self.model).save_pretrained(             │
│                                                                              │
│ /content/code/uniform_finetune.py:310 in <lambda>                            │
│                                                                              │
│   307 │                                                                      │
│   308 │   old_state_dict = model.state_dict                                  │
│   309 │   model.state_dict = (                                               │
│ ❱ 310 │   │   lambda self, *_, **__: get_peft_model_state_dict(self, old_sta │
│   311 │   ).__get__(model, type(model))                                      │
│   312 │                                                                      │
│   313 │   if torch.__version__ >= "2" and sys.platform != "win32":           │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1818 in    │
│ state_dict                                                                   │
│                                                                              │
│   1815 │   │   self._save_to_state_dict(destination, prefix, keep_vars)      │
│   1816 │   │   for name, module in self._modules.items():                    │
│   1817 │   │   │   if module is not None:                                    │
│ ❱ 1818 │   │   │   │   module.state_dict(destination=destination, prefix=pre │
│   1819 │   │   for hook in self._state_dict_hooks.values():                  │
│   1820 │   │   │   hook_result = hook(self, destination, prefix, local_metad │
│   1821 │   │   │   if hook_result is not None:                               │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1818 in    │
│ state_dict                                                                   │
│                                                                              │
│   1815 │   │   self._save_to_state_dict(destination, prefix, keep_vars)      │
│   1816 │   │   for name, module in self._modules.items():                    │
│   1817 │   │   │   if module is not None:                                    │
│ ❱ 1818 │   │   │   │   module.state_dict(destination=destination, prefix=pre │
│   1819 │   │   for hook in self._state_dict_hooks.values():                  │
│   1820 │   │   │   hook_result = hook(self, destination, prefix, local_metad │
│   1821 │   │   │   if hook_result is not None:                               │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1818 in    │
│ state_dict                                                                   │
│                                                                              │
│   1815 │   │   self._save_to_state_dict(destination, prefix, keep_vars)      │
│   1816 │   │   for name, module in self._modules.items():                    │
│   1817 │   │   │   if module is not None:                                    │
│ ❱ 1818 │   │   │   │   module.state_dict(destination=destination, prefix=pre │
│   1819 │   │   for hook in self._state_dict_hooks.values():                  │
│   1820 │   │   │   hook_result = hook(self, destination, prefix, local_metad │
│   1821 │   │   │   if hook_result is not None:                               │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1818 in    │
│ state_dict                                                                   │
│                                                                              │
│   1815 │   │   self._save_to_state_dict(destination, prefix, keep_vars)      │
│   1816 │   │   for name, module in self._modules.items():                    │
│   1817 │   │   │   if module is not None:                                    │
│ ❱ 1818 │   │   │   │   module.state_dict(destination=destination, prefix=pre │
│   1819 │   │   for hook in self._state_dict_hooks.values():                  │
│   1820 │   │   │   hook_result = hook(self, destination, prefix, local_metad │
│   1821 │   │   │   if hook_result is not None:                               │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1818 in    │
│ state_dict                                                                   │
│                                                                              │
│   1815 │   │   self._save_to_state_dict(destination, prefix, keep_vars)      │
│   1816 │   │   for name, module in self._modules.items():                    │
│   1817 │   │   │   if module is not None:                                    │
│ ❱ 1818 │   │   │   │   module.state_dict(destination=destination, prefix=pre │
│   1819 │   │   for hook in self._state_dict_hooks.values():                  │
│   1820 │   │   │   hook_result = hook(self, destination, prefix, local_metad │
│   1821 │   │   │   if hook_result is not None:                               │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1818 in    │
│ state_dict                                                                   │
│                                                                              │
│   1815 │   │   self._save_to_state_dict(destination, prefix, keep_vars)      │
│   1816 │   │   for name, module in self._modules.items():                    │
│   1817 │   │   │   if module is not None:                                    │
│ ❱ 1818 │   │   │   │   module.state_dict(destination=destination, prefix=pre │
│   1819 │   │   for hook in self._state_dict_hooks.values():                  │
│   1820 │   │   │   hook_result = hook(self, destination, prefix, local_metad │
│   1821 │   │   │   if hook_result is not None:                               │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1818 in    │
│ state_dict                                                                   │
│                                                                              │
│   1815 │   │   self._save_to_state_dict(destination, prefix, keep_vars)      │
│   1816 │   │   for name, module in self._modules.items():                    │
│   1817 │   │   │   if module is not None:                                    │
│ ❱ 1818 │   │   │   │   module.state_dict(destination=destination, prefix=pre │
│   1819 │   │   for hook in self._state_dict_hooks.values():                  │
│   1820 │   │   │   hook_result = hook(self, destination, prefix, local_metad │
│   1821 │   │   │   if hook_result is not None:                               │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1815 in    │
│ state_dict                                                                   │
│                                                                              │
│   1812 │   │   if hasattr(destination, "_metadata"):                         │
│   1813 │   │   │   destination._metadata[prefix[:-1]] = local_metadata       │
│   1814 │   │                                                                 │
│ ❱ 1815 │   │   self._save_to_state_dict(destination, prefix, keep_vars)      │
│   1816 │   │   for name, module in self._modules.items():                    │
│   1817 │   │   │   if module is not None:                                    │
│   1818 │   │   │   │   module.state_dict(destination=destination, prefix=pre │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/bitsandbytes/nn/modules.py:268 in     │
│ _save_to_state_dict                                                          │
│                                                                              │
│   265 │   │                                                                  │
│   266 │   │   try:                                                           │
│   267 │   │   │   if reorder_layout:                                         │
│ ❱ 268 │   │   │   │   self.weight.data = undo_layout(self.state.CxB, self.st │
│   269 │   │   │                                                              │
│   270 │   │   │   super()._save_to_state_dict(destination, prefix, keep_vars │
│   271                                                                        │
│                                                                              │
│ /usr/local/lib/python3.9/dist-packages/bitsandbytes/autograd/_functions.py:1 │
│ 00 in undo_layout                                                            │
│                                                                              │
│    97 │   outputs[tile_indices.flatten()] = tensor                           │
│    98 │   outputs = outputs.reshape(tile_rows, tile_cols, cols // tile_cols, │
│    99 │   outputs = outputs.permute(3, 0, 2, 1)  # (rows // tile_rows, tile_ │
│ ❱ 100 │   return outputs.reshape(rows, cols).contiguous()                    │
│   101                                                                        │
│   102                                                                        │
│   103 class MatMul8bit(torch.autograd.Function):                             │
╰──────────────────────────────────────────────────────────────────────────────╯
OutOfMemoryError: CUDA out of memory. Tried to allocate 44.00 MiB (GPU 0; 14.75 
GiB total capacity; 13.11 GiB already allocated; 16.81 MiB free; 13.68 GiB 
reserved in total by PyTorch) If reserved memory is >> allocated memory try 
setting max_split_size_mb to avoid fragmentation.  See documentation for Memory 
Management and PYTORCH_CUDA_ALLOC_CONF
wandb: Waiting for W&B process to finish... (failed 1). Press Control-C to abort syncing.
wandb: 
wandb: Run history:
wandb:               eval/loss ▁
wandb:            eval/runtime ▁
wandb: eval/samples_per_second ▁
wandb:   eval/steps_per_second ▁
wandb:             train/epoch ▁█
wandb:       train/global_step ▁█
wandb:     train/learning_rate ▁
wandb:              train/loss ▁
wandb: 
wandb: Run summary:
wandb:               eval/loss 1.15188
wandb:            eval/runtime 341.0299
wandb: eval/samples_per_second 5.865
wandb:   eval/steps_per_second 0.733
wandb:             train/epoch 0.1
wandb:       train/global_step 39
wandb:     train/learning_rate 0.00015
wandb:              train/loss 1.4123
wandb: 
wandb: 🚀 View run cool-flower-1 at: https://wandb.ai/utensil/Alpaca-CoT-Colab/runs/l81k9fdc
wandb: Synced 5 W&B file(s), 0 media file(s), 2 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20230418_023339-l81k9fdc/logs
In [ ]: