In [1]:
!apt update
Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] Get:5 https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/ InRelease [3,622 B] Get:6 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu focal InRelease [18.1 kB] Hit:7 http://ppa.launchpad.net/cran/libgit2/ubuntu focal InRelease Hit:8 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal InRelease Hit:9 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu focal InRelease Hit:10 http://ppa.launchpad.net/ubuntugis/ppa/ubuntu focal InRelease Hit:11 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64 InRelease Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,202 kB] Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,346 kB] Get:14 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu focal/main Sources [2,581 kB] Get:15 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu focal/main amd64 Packages [1,218 kB] Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,051 kB] Fetched 9,755 kB in 3s (3,695 kB/s) Reading package lists... Done Building dependency tree Reading state information... Done 34 packages can be upgraded. Run 'apt list --upgradable' to see them.
In [2]:
!apt install python3.9 python3-pip
Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libpython3.9-minimal libpython3.9-stdlib python-pip-whl python3-setuptools python3-wheel python3.9-minimal Suggested packages: python-setuptools-doc python3.9-venv binfmt-support The following NEW packages will be installed: libpython3.9-minimal libpython3.9-stdlib python-pip-whl python3-pip python3-setuptools python3-wheel python3.9 python3.9-minimal 0 upgraded, 8 newly installed, 0 to remove and 34 not upgraded. Need to get 7,412 kB of archives. After this operation, 24.6 MB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python-pip-whl all 20.0.2-5ubuntu1.8 [1,805 kB] Get:2 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 libpython3.9-minimal amd64 3.9.16-1+focal1 [804 kB] Get:3 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 python3.9-minimal amd64 3.9.16-1+focal1 [2,062 kB] Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-setuptools all 45.2.0-1ubuntu0.1 [330 kB] Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-wheel all 0.34.2-1ubuntu0.1 [23.9 kB] Get:6 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-pip all 20.0.2-5ubuntu1.8 [231 kB] Get:7 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 libpython3.9-stdlib amd64 3.9.16-1+focal1 [1,662 kB] Get:8 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 python3.9 amd64 3.9.16-1+focal1 [495 kB] Fetched 7,412 kB in 0s (19.0 MB/s) Selecting previously unselected package libpython3.9-minimal:amd64. (Reading database ... 122545 files and directories currently installed.) Preparing to unpack .../0-libpython3.9-minimal_3.9.16-1+focal1_amd64.deb ... Unpacking libpython3.9-minimal:amd64 (3.9.16-1+focal1) ... Selecting previously unselected package python3.9-minimal. Preparing to unpack .../1-python3.9-minimal_3.9.16-1+focal1_amd64.deb ... Unpacking python3.9-minimal (3.9.16-1+focal1) ... Selecting previously unselected package libpython3.9-stdlib:amd64. Preparing to unpack .../2-libpython3.9-stdlib_3.9.16-1+focal1_amd64.deb ... Unpacking libpython3.9-stdlib:amd64 (3.9.16-1+focal1) ... Selecting previously unselected package python-pip-whl. Preparing to unpack .../3-python-pip-whl_20.0.2-5ubuntu1.8_all.deb ... Unpacking python-pip-whl (20.0.2-5ubuntu1.8) ... Selecting previously unselected package python3-setuptools. Preparing to unpack .../4-python3-setuptools_45.2.0-1ubuntu0.1_all.deb ... Unpacking python3-setuptools (45.2.0-1ubuntu0.1) ... Selecting previously unselected package python3-wheel. Preparing to unpack .../5-python3-wheel_0.34.2-1ubuntu0.1_all.deb ... Unpacking python3-wheel (0.34.2-1ubuntu0.1) ... Selecting previously unselected package python3-pip. Preparing to unpack .../6-python3-pip_20.0.2-5ubuntu1.8_all.deb ... Unpacking python3-pip (20.0.2-5ubuntu1.8) ... Selecting previously unselected package python3.9. Preparing to unpack .../7-python3.9_3.9.16-1+focal1_amd64.deb ... Unpacking python3.9 (3.9.16-1+focal1) ... Setting up python3-setuptools (45.2.0-1ubuntu0.1) ... Setting up libpython3.9-minimal:amd64 (3.9.16-1+focal1) ... Setting up python3-wheel (0.34.2-1ubuntu0.1) ... Setting up python-pip-whl (20.0.2-5ubuntu1.8) ... Setting up python3.9-minimal (3.9.16-1+focal1) ... Setting up libpython3.9-stdlib:amd64 (3.9.16-1+focal1) ... Setting up python3-pip (20.0.2-5ubuntu1.8) ... Setting up python3.9 (3.9.16-1+focal1) ... Processing triggers for man-db (2.9.1-1) ... Processing triggers for mime-support (3.64ubuntu1) ...
In [3]:
!update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 100
update-alternatives: using /usr/bin/python3.9 to provide /usr/bin/python3 (python3) in auto mode
In [4]:
!python3 -V
Python 3.9.16
In [5]:
!pip3 -V
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.9)
In [6]:
!pip install torch --index-url https://download.pytorch.org/whl/cu118
Looking in indexes: https://download.pytorch.org/whl/cu118, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting torch
Downloading https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp39-cp39-linux_x86_64.whl (2267.3 MB)
|████████████████████████████████| 2267.3 MB 1.5 kB/s
Collecting filelock
Downloading https://download.pytorch.org/whl/filelock-3.9.0-py3-none-any.whl (9.7 kB)
Collecting networkx
Downloading https://download.pytorch.org/whl/networkx-3.0-py3-none-any.whl (2.0 MB)
|████████████████████████████████| 2.0 MB 17.5 MB/s
Collecting jinja2
Downloading https://download.pytorch.org/whl/Jinja2-3.1.2-py3-none-any.whl (133 kB)
|████████████████████████████████| 133 kB 70.9 MB/s
Collecting sympy
Downloading https://download.pytorch.org/whl/sympy-1.11.1-py3-none-any.whl (6.5 MB)
|████████████████████████████████| 6.5 MB 42.0 MB/s
Collecting triton==2.0.0; platform_system == "Linux" and platform_machine == "x86_64"
Downloading https://download.pytorch.org/whl/triton-2.0.0-1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB)
|████████████████████████████████| 63.3 MB 79 kB/s
Collecting typing-extensions
Downloading https://download.pytorch.org/whl/typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting MarkupSafe>=2.0
Downloading https://download.pytorch.org/whl/MarkupSafe-2.1.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting mpmath>=0.19
Downloading https://download.pytorch.org/whl/mpmath-1.2.1-py3-none-any.whl (532 kB)
|████████████████████████████████| 532 kB 71.5 MB/s
Collecting cmake
Downloading https://download.pytorch.org/whl/cmake-3.25.0-py2.py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23.7 MB)
|████████████████████████████████| 23.7 MB 1.3 MB/s
Collecting lit
Downloading https://download.pytorch.org/whl/lit-15.0.7.tar.gz (132 kB)
|████████████████████████████████| 132 kB 71.2 MB/s
Building wheels for collected packages: lit
Building wheel for lit (setup.py) ... done
Created wheel for lit: filename=lit-15.0.7-py3-none-any.whl size=89984 sha256=388dc89c33b459eb86123b6871a12dde65d235459d83938c4d63a9ff86814a82
Stored in directory: /root/.cache/pip/wheels/77/9f/9c/1c3217c66e9f310f11a534c68e168a295d9448999af79d680b
Successfully built lit
Installing collected packages: filelock, networkx, MarkupSafe, jinja2, mpmath, sympy, cmake, lit, triton, typing-extensions, torch
Successfully installed MarkupSafe-2.1.2 cmake-3.25.0 filelock-3.9.0 jinja2-3.1.2 lit-15.0.7 mpmath-1.2.1 networkx-3.0 sympy-1.11.1 torch-2.0.1+cu118 triton-2.0.0 typing-extensions-4.4.0
In [7]:
!git clone https://github.com/OpenAccess-AI-Collective/axolotl
Cloning into 'axolotl'... remote: Enumerating objects: 1728, done. remote: Counting objects: 100% (817/817), done. remote: Compressing objects: 100% (235/235), done. remote: Total 1728 (delta 596), reused 712 (delta 522), pack-reused 911 Receiving objects: 100% (1728/1728), 1.16 MiB | 10.54 MiB/s, done. Resolving deltas: 100% (1065/1065), done.
In [8]:
!apt install build-essential ninja-build python3.9-dev
Reading package lists... Done Building dependency tree Reading state information... Done build-essential is already the newest version (12.8ubuntu1.1). The following additional packages will be installed: libpython3.9 libpython3.9-dev The following NEW packages will be installed: libpython3.9 libpython3.9-dev ninja-build python3.9-dev 0 upgraded, 4 newly installed, 0 to remove and 34 not upgraded. Need to get 6,517 kB of archives. After this operation, 27.6 MB of additional disk space will be used. Get:1 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 libpython3.9 amd64 3.9.16-1+focal1 [1,715 kB] Get:2 http://archive.ubuntu.com/ubuntu focal/universe amd64 ninja-build amd64 1.10.0-1build1 [107 kB] Get:3 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 libpython3.9-dev amd64 3.9.16-1+focal1 [4,195 kB] Get:4 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 python3.9-dev amd64 3.9.16-1+focal1 [501 kB] Fetched 6,517 kB in 0s (19.1 MB/s) Selecting previously unselected package libpython3.9:amd64. (Reading database ... 123547 files and directories currently installed.) Preparing to unpack .../libpython3.9_3.9.16-1+focal1_amd64.deb ... Unpacking libpython3.9:amd64 (3.9.16-1+focal1) ... Selecting previously unselected package libpython3.9-dev:amd64. Preparing to unpack .../libpython3.9-dev_3.9.16-1+focal1_amd64.deb ... Unpacking libpython3.9-dev:amd64 (3.9.16-1+focal1) ... Selecting previously unselected package ninja-build. Preparing to unpack .../ninja-build_1.10.0-1build1_amd64.deb ... Unpacking ninja-build (1.10.0-1build1) ... Selecting previously unselected package python3.9-dev. Preparing to unpack .../python3.9-dev_3.9.16-1+focal1_amd64.deb ... Unpacking python3.9-dev (3.9.16-1+focal1) ... Setting up ninja-build (1.10.0-1build1) ... Setting up libpython3.9:amd64 (3.9.16-1+focal1) ... Setting up libpython3.9-dev:amd64 (3.9.16-1+focal1) ... Setting up python3.9-dev (3.9.16-1+focal1) ... Processing triggers for man-db (2.9.1-1) ...
In [9]:
%cd axolotl
/content/axolotl
In [10]:
!pip3 install -e .[int4]
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Obtaining file:///content/axolotl
Collecting PyYAML==6.0
Downloading PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB)
|████████████████████████████████| 661 kB 17.0 MB/s
Collecting accelerate>=0.19.0
Downloading accelerate-0.19.0-py3-none-any.whl (219 kB)
|████████████████████████████████| 219 kB 76.3 MB/s
Collecting addict
Downloading addict-2.4.0-py3-none-any.whl (3.8 kB)
Collecting bert-score==0.3.13
Downloading bert_score-0.3.13-py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 8.8 MB/s
Collecting bitsandbytes>=0.39.0
Downloading bitsandbytes-0.39.0-py3-none-any.whl (92.2 MB)
|████████████████████████████████| 92.2 MB 140 kB/s
Collecting black
Downloading black-23.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
|████████████████████████████████| 1.7 MB 42.6 MB/s
Collecting datasets
Downloading datasets-2.12.0-py3-none-any.whl (474 kB)
|████████████████████████████████| 474 kB 61.9 MB/s
Collecting einops
Downloading einops-0.6.1-py3-none-any.whl (42 kB)
|████████████████████████████████| 42 kB 1.1 MB/s
Collecting evaluate==0.4.0
Downloading evaluate-0.4.0-py3-none-any.whl (81 kB)
|████████████████████████████████| 81 kB 10.5 MB/s
Collecting fire
Downloading fire-0.5.0.tar.gz (88 kB)
|████████████████████████████████| 88 kB 8.9 MB/s
Collecting rouge-score==0.1.2
Downloading rouge_score-0.1.2.tar.gz (17 kB)
Collecting scikit-learn==1.2.2
Downloading scikit_learn-1.2.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.6 MB)
|████████████████████████████████| 9.6 MB 69.8 MB/s
Collecting scipy
Downloading scipy-1.10.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
|████████████████████████████████| 34.5 MB 1.2 MB/s
Collecting sentencepiece
Downloading sentencepiece-0.1.99-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 54.6 MB/s
Collecting transformers@ git+https://github.com/huggingface/transformers.git
Cloning https://github.com/huggingface/transformers.git to /tmp/pip-install-3c28i36d/transformers
Running command git clone -q https://github.com/huggingface/transformers.git /tmp/pip-install-3c28i36d/transformers
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting wandb
Downloading wandb-0.15.3-py3-none-any.whl (2.0 MB)
|████████████████████████████████| 2.0 MB 59.8 MB/s
Collecting xformers
Downloading xformers-0.0.20-cp39-cp39-manylinux2014_x86_64.whl (109.1 MB)
|████████████████████████████████| 109.1 MB 21 kB/s
Collecting alpaca_lora_4bit@ git+https://github.com/winglian/alpaca_lora_4bit.git@setup_pip
Cloning https://github.com/winglian/alpaca_lora_4bit.git (to revision setup_pip) to /tmp/pip-install-3c28i36d/alpaca-lora-4bit
Running command git clone -q https://github.com/winglian/alpaca_lora_4bit.git /tmp/pip-install-3c28i36d/alpaca-lora-4bit
Running command git checkout -b setup_pip --track origin/setup_pip
Switched to a new branch 'setup_pip'
Branch 'setup_pip' set up to track remote branch 'setup_pip' from 'origin'.
Collecting numpy>=1.17
Downloading numpy-1.24.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
|████████████████████████████████| 17.3 MB 66.6 MB/s
Collecting packaging>=20.0
Downloading packaging-23.1-py3-none-any.whl (48 kB)
|████████████████████████████████| 48 kB 5.1 MB/s
Collecting psutil
Downloading psutil-5.9.5-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (282 kB)
|████████████████████████████████| 282 kB 67.7 MB/s
Requirement already satisfied: torch>=1.6.0 in /usr/local/lib/python3.9/dist-packages (from accelerate>=0.19.0->axolotl==0.1) (2.0.1+cu118)
Collecting pandas>=1.0.1
Downloading pandas-2.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.4 MB)
|████████████████████████████████| 12.4 MB 51.6 MB/s
Collecting requests
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
|████████████████████████████████| 62 kB 1.2 MB/s
Collecting matplotlib
Downloading matplotlib-3.7.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)
|████████████████████████████████| 11.6 MB 60.8 MB/s
Collecting tqdm>=4.31.1
Downloading tqdm-4.65.0-py3-none-any.whl (77 kB)
|████████████████████████████████| 77 kB 2.6 MB/s
Collecting pathspec>=0.9.0
Downloading pathspec-0.11.1-py3-none-any.whl (29 kB)
Collecting platformdirs>=2
Downloading platformdirs-3.5.1-py3-none-any.whl (15 kB)
Requirement already satisfied: typing-extensions>=3.10.0.0; python_version < "3.10" in /usr/local/lib/python3.9/dist-packages (from black->axolotl==0.1) (4.4.0)
Collecting click>=8.0.0
Downloading click-8.1.3-py3-none-any.whl (96 kB)
|████████████████████████████████| 96 kB 6.9 MB/s
Collecting tomli>=1.1.0; python_version < "3.11"
Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting mypy-extensions>=0.4.3
Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Collecting fsspec[http]>=2021.11.1
Downloading fsspec-2023.5.0-py3-none-any.whl (160 kB)
|████████████████████████████████| 160 kB 83.3 MB/s
Collecting aiohttp
Downloading aiohttp-3.8.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
|████████████████████████████████| 1.0 MB 74.5 MB/s
Collecting dill<0.3.7,>=0.3.0
Downloading dill-0.3.6-py3-none-any.whl (110 kB)
|████████████████████████████████| 110 kB 76.2 MB/s
Collecting huggingface-hub<1.0.0,>=0.11.0
Downloading huggingface_hub-0.14.1-py3-none-any.whl (224 kB)
|████████████████████████████████| 224 kB 63.6 MB/s
Collecting responses<0.19
Downloading responses-0.18.0-py3-none-any.whl (38 kB)
Collecting pyarrow>=8.0.0
Downloading pyarrow-12.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (39.0 MB)
|████████████████████████████████| 39.0 MB 1.2 MB/s
Collecting multiprocess
Downloading multiprocess-0.70.14-py39-none-any.whl (132 kB)
|████████████████████████████████| 132 kB 82.3 MB/s
Collecting xxhash
Downloading xxhash-3.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (212 kB)
|████████████████████████████████| 212 kB 81.3 MB/s
Collecting six
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting termcolor
Downloading termcolor-2.3.0-py3-none-any.whl (6.9 kB)
Collecting absl-py
Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
|████████████████████████████████| 126 kB 77.2 MB/s
Collecting nltk
Downloading nltk-3.8.1-py3-none-any.whl (1.5 MB)
|████████████████████████████████| 1.5 MB 67.4 MB/s
Collecting threadpoolctl>=2.0.0
Downloading threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting joblib>=1.1.1
Downloading joblib-1.2.0-py3-none-any.whl (297 kB)
|████████████████████████████████| 297 kB 72.1 MB/s
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1
Downloading tokenizers-0.13.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
|████████████████████████████████| 7.8 MB 57.2 MB/s
Collecting safetensors>=0.2.1
Downloading safetensors-0.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 55.4 MB/s
Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from transformers@ git+https://github.com/huggingface/transformers.git->axolotl==0.1) (3.9.0)
Collecting regex!=2019.12.17
Downloading regex-2023.5.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (769 kB)
|████████████████████████████████| 769 kB 62.0 MB/s
Collecting appdirs>=1.4.3
Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting sentry-sdk>=1.0.0
Downloading sentry_sdk-1.24.0-py2.py3-none-any.whl (206 kB)
|████████████████████████████████| 206 kB 51.6 MB/s
Collecting GitPython!=3.1.29,>=1.0.0
Downloading GitPython-3.1.31-py3-none-any.whl (184 kB)
|████████████████████████████████| 184 kB 60.5 MB/s
Collecting docker-pycreds>=0.4.0
Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting setproctitle
Downloading setproctitle-1.3.2-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)
Collecting protobuf!=4.21.0,<5,>=3.15.0; python_version == "3.9" and sys_platform == "linux"
Downloading protobuf-4.23.2-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
|████████████████████████████████| 304 kB 59.5 MB/s
Collecting pathtools
Downloading pathtools-0.1.2.tar.gz (11 kB)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from wandb->axolotl==0.1) (45.2.0)
Collecting pyre-extensions==0.0.29
Downloading pyre_extensions-0.0.29-py3-none-any.whl (12 kB)
Collecting colorama
Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting peft@ git+https://github.com/huggingface/peft.git@70af02a2bca5a63921790036b2c9430edf4037e2
Cloning https://github.com/huggingface/peft.git (to revision 70af02a2bca5a63921790036b2c9430edf4037e2) to /tmp/pip-install-3c28i36d/peft
Running command git clone -q https://github.com/huggingface/peft.git /tmp/pip-install-3c28i36d/peft
Running command git checkout -q 70af02a2bca5a63921790036b2c9430edf4037e2
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Requirement already satisfied: sympy in /usr/local/lib/python3.9/dist-packages (from torch>=1.6.0->accelerate>=0.19.0->axolotl==0.1) (1.11.1)
Requirement already satisfied: networkx in /usr/local/lib/python3.9/dist-packages (from torch>=1.6.0->accelerate>=0.19.0->axolotl==0.1) (3.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.9/dist-packages (from torch>=1.6.0->accelerate>=0.19.0->axolotl==0.1) (3.1.2)
Requirement already satisfied: triton==2.0.0; platform_system == "Linux" and platform_machine == "x86_64" in /usr/local/lib/python3.9/dist-packages (from torch>=1.6.0->accelerate>=0.19.0->axolotl==0.1) (2.0.0)
Collecting tzdata>=2022.1
Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)
|████████████████████████████████| 341 kB 70.7 MB/s
Collecting pytz>=2020.1
Downloading pytz-2023.3-py2.py3-none-any.whl (502 kB)
|████████████████████████████████| 502 kB 76.2 MB/s
Collecting python-dateutil>=2.8.2
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
|████████████████████████████████| 247 kB 78.5 MB/s
Collecting certifi>=2017.4.17
Downloading certifi-2023.5.7-py3-none-any.whl (156 kB)
|████████████████████████████████| 156 kB 75.9 MB/s
Collecting idna<4,>=2.5
Downloading idna-3.4-py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 111 kB/s
Collecting charset-normalizer<4,>=2
Downloading charset_normalizer-3.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
|████████████████████████████████| 199 kB 81.0 MB/s
Collecting urllib3<3,>=1.21.1
Downloading urllib3-2.0.2-py3-none-any.whl (123 kB)
|████████████████████████████████| 123 kB 72.5 MB/s
Collecting kiwisolver>=1.0.1
Downloading kiwisolver-1.4.4-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
|████████████████████████████████| 1.6 MB 62.7 MB/s
Collecting contourpy>=1.0.1
Downloading contourpy-1.0.7-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (299 kB)
|████████████████████████████████| 299 kB 81.5 MB/s
Collecting pillow>=6.2.0
Downloading Pillow-9.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
|████████████████████████████████| 3.3 MB 68.0 MB/s
Collecting fonttools>=4.22.0
Downloading fonttools-4.39.4-py3-none-any.whl (1.0 MB)
|████████████████████████████████| 1.0 MB 74.2 MB/s
Collecting pyparsing>=2.3.1
Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
|████████████████████████████████| 98 kB 9.1 MB/s
Collecting importlib-resources>=3.2.0; python_version < "3.10"
Downloading importlib_resources-5.12.0-py3-none-any.whl (36 kB)
Collecting cycler>=0.10
Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting yarl<2.0,>=1.0
Downloading yarl-1.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (269 kB)
|████████████████████████████████| 269 kB 58.7 MB/s
Collecting attrs>=17.3.0
Downloading attrs-23.1.0-py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 8.3 MB/s
Collecting async-timeout<5.0,>=4.0.0a3
Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting frozenlist>=1.1.1
Downloading frozenlist-1.3.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (158 kB)
|████████████████████████████████| 158 kB 63.3 MB/s
Collecting aiosignal>=1.1.2
Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting multidict<7.0,>=4.5
Downloading multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)
|████████████████████████████████| 114 kB 42.8 MB/s
Collecting gitdb<5,>=4.0.1
Downloading gitdb-4.0.10-py3-none-any.whl (62 kB)
|████████████████████████████████| 62 kB 1.5 MB/s
Collecting typing-inspect
Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.9/dist-packages (from sympy->torch>=1.6.0->accelerate>=0.19.0->axolotl==0.1) (1.2.1)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.9/dist-packages (from jinja2->torch>=1.6.0->accelerate>=0.19.0->axolotl==0.1) (2.1.2)
Requirement already satisfied: lit in /usr/local/lib/python3.9/dist-packages (from triton==2.0.0; platform_system == "Linux" and platform_machine == "x86_64"->torch>=1.6.0->accelerate>=0.19.0->axolotl==0.1) (15.0.7)
Requirement already satisfied: cmake in /usr/local/lib/python3.9/dist-packages (from triton==2.0.0; platform_system == "Linux" and platform_machine == "x86_64"->torch>=1.6.0->accelerate>=0.19.0->axolotl==0.1) (3.25.0)
Collecting zipp>=3.1.0; python_version < "3.10"
Downloading zipp-3.15.0-py3-none-any.whl (6.8 kB)
Collecting smmap<6,>=3.0.1
Downloading smmap-5.0.0-py3-none-any.whl (24 kB)
Building wheels for collected packages: fire, rouge-score, transformers, alpaca-lora-4bit, pathtools, peft
Building wheel for fire (setup.py) ... done
Created wheel for fire: filename=fire-0.5.0-py2.py3-none-any.whl size=116936 sha256=3175a8ab0f7a5471f260773255f6256903f994b0d6a50cf588151abda303f7e6
Stored in directory: /root/.cache/pip/wheels/f7/f1/89/b9ea2bf8f80ec027a88fef1d354b3816b4d3d29530988972f6
Building wheel for rouge-score (setup.py) ... done
Created wheel for rouge-score: filename=rouge_score-0.1.2-py3-none-any.whl size=24954 sha256=17d27c5df2b650a669d13eecc32bbf814405aa761c2aad8631eb87e732f35056
Stored in directory: /root/.cache/pip/wheels/9b/3d/39/09558097d3119ca0a4d462df68f22c6f3c1b345ac63a09b86e
Building wheel for transformers (PEP 517) ... done
Created wheel for transformers: filename=transformers-4.30.0.dev0-py3-none-any.whl size=7102950 sha256=87574291789490dfcc5f7ed40fa82d826ce5244a98fde328ed06339bce004b55
Stored in directory: /tmp/pip-ephem-wheel-cache-syii_zx6/wheels/f7/92/8c/752ff3bfcd3439805d8bbf641614da38ef3226e127ebea86ee
Building wheel for alpaca-lora-4bit (setup.py) ... done
Created wheel for alpaca-lora-4bit: filename=alpaca_lora_4bit-0.1-cp39-cp39-linux_x86_64.whl size=3621883 sha256=9906cf53d1b47679d537c819020c9c742415f0556236abf430904e4dc35bb91c
Stored in directory: /tmp/pip-ephem-wheel-cache-syii_zx6/wheels/fa/12/ba/a191ae13eef3219588221dcb899ef8340ee1b60ea3f02375ec
Building wheel for pathtools (setup.py) ... done
Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8784 sha256=36b7c13c856d6ad3d82ea98ccf402c3e652a63c04ac1662019cf1e92c2ce8a81
Stored in directory: /root/.cache/pip/wheels/b7/0a/67/ada2a22079218c75a88361c0782855cc72aebc4d18d0289d05
Building wheel for peft (PEP 517) ... done
Created wheel for peft: filename=peft-0.3.0.dev0-py3-none-any.whl size=50371 sha256=2f86cce3487c630222588c74a24239f328ba2f541ee3cfa8ff4f22bb6be4d898
Stored in directory: /root/.cache/pip/wheels/1e/d6/0d/ffffc3403e2464680bdba454ea14c0b80062ce04e666c010f8
Successfully built fire rouge-score transformers alpaca-lora-4bit pathtools peft
ERROR: sentry-sdk 1.24.0 has requirement urllib3<2.0.0, but you'll have urllib3 2.0.2 which is incompatible.
Installing collected packages: PyYAML, numpy, packaging, psutil, accelerate, addict, tzdata, pytz, six, python-dateutil, pandas, certifi, idna, charset-normalizer, urllib3, requests, kiwisolver, contourpy, pillow, fonttools, pyparsing, zipp, importlib-resources, cycler, matplotlib, tqdm, tokenizers, safetensors, multidict, yarl, attrs, async-timeout, frozenlist, aiosignal, aiohttp, fsspec, huggingface-hub, regex, transformers, bert-score, bitsandbytes, pathspec, platformdirs, click, tomli, mypy-extensions, black, dill, responses, pyarrow, multiprocess, xxhash, datasets, einops, evaluate, termcolor, fire, absl-py, joblib, nltk, rouge-score, scipy, threadpoolctl, scikit-learn, sentencepiece, appdirs, sentry-sdk, smmap, gitdb, GitPython, docker-pycreds, setproctitle, protobuf, pathtools, wandb, typing-inspect, pyre-extensions, xformers, colorama, peft, alpaca-lora-4bit, axolotl
Running setup.py develop for axolotl
Successfully installed GitPython-3.1.31 PyYAML-6.0 absl-py-1.4.0 accelerate-0.19.0 addict-2.4.0 aiohttp-3.8.4 aiosignal-1.3.1 alpaca-lora-4bit-0.1 appdirs-1.4.4 async-timeout-4.0.2 attrs-23.1.0 axolotl bert-score-0.3.13 bitsandbytes-0.39.0 black-23.3.0 certifi-2023.5.7 charset-normalizer-3.1.0 click-8.1.3 colorama-0.4.6 contourpy-1.0.7 cycler-0.11.0 datasets-2.12.0 dill-0.3.6 docker-pycreds-0.4.0 einops-0.6.1 evaluate-0.4.0 fire-0.5.0 fonttools-4.39.4 frozenlist-1.3.3 fsspec-2023.5.0 gitdb-4.0.10 huggingface-hub-0.14.1 idna-3.4 importlib-resources-5.12.0 joblib-1.2.0 kiwisolver-1.4.4 matplotlib-3.7.1 multidict-6.0.4 multiprocess-0.70.14 mypy-extensions-1.0.0 nltk-3.8.1 numpy-1.24.3 packaging-23.1 pandas-2.0.1 pathspec-0.11.1 pathtools-0.1.2 peft-0.3.0.dev0 pillow-9.5.0 platformdirs-3.5.1 protobuf-4.23.2 psutil-5.9.5 pyarrow-12.0.0 pyparsing-3.0.9 pyre-extensions-0.0.29 python-dateutil-2.8.2 pytz-2023.3 regex-2023.5.5 requests-2.31.0 responses-0.18.0 rouge-score-0.1.2 safetensors-0.3.1 scikit-learn-1.2.2 scipy-1.10.1 sentencepiece-0.1.99 sentry-sdk-1.24.0 setproctitle-1.3.2 six-1.16.0 smmap-5.0.0 termcolor-2.3.0 threadpoolctl-3.1.0 tokenizers-0.13.3 tomli-2.0.1 tqdm-4.65.0 transformers-4.30.0.dev0 typing-inspect-0.9.0 tzdata-2023.3 urllib3-2.0.2 wandb-0.15.3 xformers-0.0.20 xxhash-3.2.0 yarl-1.9.2 zipp-3.15.0
In [32]:
!accelerate config --config_file configs/accelerate/default_config.yaml default
accelerate configuration saved at /root/.cache/huggingface/accelerate/default_config.yaml
In [33]:
!cat examples/falcon/config-1b-lora.yml
base_model: tiiuae/falcon-rw-1b
base_model_config: tiiuae/falcon-rw-1b
trust_remote_code: true
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
load_in_8bit: true
load_in_4bit: false
gptq: false
strict: false
push_dataset_to_hub:
datasets:
- path: teknium/GPT4-LLM-Cleaned
type: alpaca:chat
dataset_prepared_path: last_run_prepared
val_set_size: 0.01
adapter: lora
lora_model_dir:
sequence_len: 2048
max_packed_sequence_len:
lora_r: 16
lora_alpha: 32
lora_dropout: 0.0
lora_target_modules:
lora_target_linear: true
lora_fan_in_fan_out:
wandb_project: falcon-rw-1b
wandb_watch:
wandb_run_id:
wandb_log_model:
output_dir: ./falcon-rw-1b
batch_size: 2
micro_batch_size: 1
num_epochs: 4
optimizer: adamw_bnb_8bit
torchdistx_path:
lr_scheduler: cosine
learning_rate: 0.00003
train_on_inputs: false
group_by_length: false
bf16: true
fp16: false
tf32: true
gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention: true
flash_attention:
gptq_groupsize:
gptq_model_v1:
warmup_steps: 40
eval_steps: 5
save_steps: 43
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
pad_token: "<|endoftext|>"
bos_token: ">>ABSTRACT<<"
eos_token: "<|endoftext|>"
In [ ]:
!accelerate scripts/finetune.py examples/falcon/config-1b-lora.yml
In [11]: