repo structure & setup

.github/CONTRIBUTING.md

@@ -0,0 +1,21 @@
+# pull requests
+
+1. make a branch
+    - if it relates to an existing issue
+        - go to the issue page and click *Create a branch* under *Development*
+        - if the default name is not very long, keep it; otherwise, make it shorter, but keep the issue number in the front
+    - otherwise pick a short but descriptive name, a few hyphen-separated-words
+2. make your changes
+    - include unit tests
+    - update README if needed
+3. make a pull request
+    - if it isn't ready for review yet, mark it as draft
+    - check if CI is passing
+    - if the change is big, try to keep the commit history clean using interactive rebase
+    - don't push more often than it's needed, we're running github actions on a free tier
+    - if there were any changes to the main branch, rebase on top of it
+    - explain the change
+        - provide short description; focus on things that were not mentioned in the relevant issue
+        - comment important sections of the code in *Files changed* tab
+    - when it's ready, add the relevant stakeholders as reviewers
+4. after the comments are resolved and PR is approved, merge it using *Squash and merge*

.github/workflows/checks.yml

@@ -0,0 +1,28 @@
+name: checks
+
+on: [push, pull_request]
+
+permissions:
+  actions: write
+
+jobs:
+  checks:
+    name: checks
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: setup python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+      - name: dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+          pip install -e .
+      - name: black
+        run: black --check .
+      - name: isort
+        run: isort --check .
+      - name: pytest
+        run: pytest

.pre-commit-config.yaml

@@ -0,0 +1,11 @@
+repos:
+  - repo: https://github.com/psf/black-pre-commit-mirror
+    rev: 23.12.1
+    hooks:
+      - id: black
+        language_version: python3.10
+  - repo: https://github.com/pycqa/isort
+    rev: 5.13.2
+    hooks:
+      - id: isort
+        name: isort (python)

.vscode/extensions.json

@@ -0,0 +1,9 @@
+{
+    "recommendations": [
+        "ms-python.python",
+        "ms-python.vscode-pylance",
+        "ms-python.black-formatter",
+        "ms-python.isort",
+        "github.copilot",
+    ]
+}

.vscode/settings.json

@@ -0,0 +1,10 @@
+{
+    "[python]": {
+        "editor.defaultFormatter": "ms-python.black-formatter",
+    },
+    "editor.formatOnSave": true,
+    "editor.codeActionsOnSave": {
+        "source.organizeImports": "explicit"
+    },
+    "python.analysis.typeCheckingMode": "basic",
+}

README.md

@@ -1,2 +1,17 @@
-# delphi
-Delphi: Interpreting Small Language Models Across Time and Scale
+# Delphi
+Interpreting Small Language Models Across Time and Scale
+
+# setup
+1. make python 3.10 virtual env in `.venv`
+2. install dependencies `pip install -r requirements.txt`
+3. install the project in editable state `pip install -e .`
+4. run tests `pytest`
+
+# formatting
+We're using black & isort to format the code. To make sure your changes adhere to the rules:
+
+1. follow setup instructions above
+2. install pre-commit `pre-commit install`
+3. install recommended vscode extensions
+
+When you save a file vscode should automatically format it. Otherwise, pre-commit will do that, but you will need to add the changes and commit again.

requirements.txt

@@ -0,0 +1,12 @@
+torch==2.1.2
+datasets==2.16.1
+transformers==4.36.2
+tqdm==4.66.1
+ipywidgets==8.1.1
+nbformat==5.9.2
+pytest==7.4.4
+black==23.12.1
+jaxtyping==0.2.25
+beartype==0.16.4
+pre-commit==3.6.0
+isort==5.13.2

setup.py

@@ -0,0 +1,9 @@
+from setuptools import find_packages, setup
+
+
+setup(
+    name="delphi",
+    version="0.1",
+    packages=find_packages(where='src'),
+    package_dir={"": "src"}
+)

src/delphi/__init__.py

@@ -0,0 +1,3 @@
+from beartype.claw import beartype_this_package  # <-- hype comes
+
+beartype_this_package()  # <-- hype goes

src/delphi/dummy.py

@@ -0,0 +1,12 @@
+import torch
+from jaxtyping import Float, Int
+
+Type1 = Float[torch.Tensor, "dim"]
+Type2 = Int[torch.Tensor, "batch dim"]
+
+
+def dummy(arg: Type1 | Type2) -> Type1:
+    if isinstance(arg, Type1):
+        return arg + 1
+    elif isinstance(arg, Type2):
+        return arg[0] - 0.1

tests/test_dummy.py

@@ -0,0 +1,15 @@
+import pytest
+import torch
+from beartype.roar import BeartypeCallHintViolation
+
+from delphi.dummy import dummy
+
+
+def test_dummy():
+    tensor1 = torch.tensor([1.0, 2.0, 3.0])
+    tensor2 = torch.tensor([[1, 2, 3], [4, 5, 6]])
+    assert torch.allclose(dummy(tensor1), torch.tensor([2.0, 3.0, 4.0]))
+    assert torch.allclose(dummy(tensor2), torch.tensor([0.9, 1.9, 2.9]))
+    tensor3 = torch.tensor([1, 2, 3])
+    with pytest.raises(BeartypeCallHintViolation):
+        dummy(tensor3)