Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
sangkeun00 authored Apr 7, 2024
1 parent ebe0d9a commit c54278a
Showing 1 changed file with 15 additions and 15 deletions.
30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
</p>

<p align="center">
<em><strong>LogiX</strong>: Logging for Interpretable and Explainable AI <br></em>
<em><strong>LogIX</strong>: Logging for Interpretable and Explainable AI <br></em>
</p>

<div align="center">
Expand All @@ -22,22 +22,22 @@ pip install logix-ai
```

## Usage
LogiX is designed with the belief that diverse logs generated by neural networks, such as
LogIX is designed with the belief that diverse logs generated by neural networks, such as
gradient and activation, can be utilized for analyzing and debugging data, algorithms,
and other aspects. To use LogiX, users simply adhere to a two-stage workflow:
and other aspects. To use LogIX, users simply adhere to a two-stage workflow:

1. **Logging**: Extract and save various logs (e.g. per-sample gradient, activation) to disk.
2. **Analysis**: Load logs from disk and perform custom analysis (e.g. influence function).

### Logging
Logging with LogiX is as simple as adding one `with` statement to the existing
training code. LogiX automatically extracts user-specified logs using PyTorch hooks, and
Logging with LogIX is as simple as adding one `with` statement to the existing
training code. LogIX automatically extracts user-specified logs using PyTorch hooks, and
saves it to disk using a memory-mapped file.

```python
import logix

run = logix.init(project="my_project") # initialze LogiX
run = logix.init(project="my_project") # initialze LogIX
logix.setup({"log": "grad", "save": "grad", "statistic": "kfac"}) # set logging config
logix.watch(model) # add your model to log

Expand Down Expand Up @@ -77,12 +77,12 @@ frameworks (e.g. Lightning) in the future!

```python
from transformers import Trainer, Seq2SeqTrainer
from logix.huggingface import patch_trainer, LogiXArguments
from logix.huggingface import patch_trainer, LogIXArguments

logix_args = LogiXArguments(project, config, lora=True, ekfac=True)
LogiXTrainer = patch_trainer(Trainer)
logix_args = LogIXArguments(project, config, lora=True, ekfac=True)
LogIXTrainer = patch_trainer(Trainer)

trainer = LogiXTrainer(logix_args=logix_args, # pass LogiXArguments as TrainingArguments
trainer = LogIXTrainer(logix_args=logix_args, # pass LogIXArguments as TrainingArguments
model=model,
train_dataset=train_dataset,
*args,
Expand All @@ -101,11 +101,11 @@ Please check out [Examples](/examples) for more advanced features!
Logs from neural networks are difficult to handle due to the large size. For example,
the size of the gradient of *each* training datapoint is about as large as the whole model. Therefore,
we provide various systems support to efficiently scale neural network analysis to
billion-scale models. Below are a few features that LogiX currently supports:
billion-scale models. Below are a few features that LogIX currently supports:

- **Gradient compression** (compression ratio: 1,000-100,000x)
- **Memory-map-based data IO**
- **CPU offloading of logs**
- **CPU offloading of statistics**

## Compatability
| DistributedDataParallel| Mixed Precision| Gradient Checkpointing | torch.compile | FSDP |
Expand All @@ -115,20 +115,20 @@ billion-scale models. Below are a few features that LogiX currently supports:
## Contributing

We welcome contributions from the community. Please see our [contributing
guidelines](CONTRIBUTING.md) for details on how to contribute to LogiX.
guidelines](CONTRIBUTING.md) for details on how to contribute to LogIX.

## Citation
To cite this repository:

```
@software{logix2024github,
author = {Sang Keun Choe, Hwijeen Ahn, Juhan Bae, Minsoo Kang, Youngseog Chung, Kewen Zhao},
title = {{LogiX}: Scalable Logging and Analysis Tool for Neural Networks},
title = {{LogIX}: Scalable Logging and Analysis Tool for Neural Networks},
url = {http://github.com/sangkeun00/logix},
version = {0.0.1},
year = {2024},
}
```

## License
LogiX is licensed under the [Apache 2.0 License](LICENSE).
LogIX is licensed under the [Apache 2.0 License](LICENSE).

0 comments on commit c54278a

Please sign in to comment.