Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Genai SA block integration #7720

Closed
wants to merge 3 commits into from
Closed

Genai SA block integration #7720

wants to merge 3 commits into from

Conversation

vgrau98
Copy link
Contributor

@vgrau98 vgrau98 commented Apr 27, 2024 •
edited
Loading

Fixes #7227

Description

Integrate the private SABlock from monai/networks/nets/transformer.py to monai/networks/blocks/selfattention.py

Add:

  • causal self attention
  • flash attention

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@vgrau98 vgrau98 closed this Apr 27, 2024
@vgrau98 vgrau98 deleted the genai-SABlock-integration branch April 27, 2024 11:22
@vgrau98
Copy link
Contributor Author

vgrau98 commented Apr 27, 2024

Duplicate with #7715

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@vgrau98