Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Enable PyCafe dashboards in CI #816

Merged
merged 30 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
0648efd
Enable PyCafe dashboards in CI
maxschulz-COL Oct 17, 2024
67aacf9
Changelog
maxschulz-COL Oct 17, 2024
68ff719
Small fixes
maxschulz-COL Oct 17, 2024
74c766d
Further small improvements
maxschulz-COL Oct 17, 2024
eef2980
Make workflow a little nicer
maxschulz-COL Oct 17, 2024
a5977ad
Fix update vs comment check
maxschulz-COL Oct 17, 2024
61d3da8
Linting
maxschulz-COL Oct 17, 2024
4574abc
Some further small refactorings
maxschulz-COL Oct 18, 2024
22e225c
Merge branch 'main' into ci/pycafe_in_ci
maxschulz-COL Oct 21, 2024
11e6c7f
Merge branch 'main' into ci/pycafe_in_ci
maxschulz-COL Oct 23, 2024
18c28b6
PR comments
maxschulz-COL Oct 23, 2024
33150e6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 23, 2024
a6b75ed
Further PR comments
maxschulz-COL Oct 23, 2024
05cb1d2
Merge branch 'ci/pycafe_in_ci' of github.com:mckinsey/vizro into ci/p…
maxschulz-COL Oct 23, 2024
19f6515
Better template
maxschulz-COL Oct 23, 2024
d3d1480
default working directory
maxschulz-COL Oct 24, 2024
88cdc7c
Refactor script without sys arguments
maxschulz-COL Oct 24, 2024
4de8e77
Correct order url and directory
maxschulz-COL Oct 24, 2024
4937834
Merge branch 'main' into ci/pycafe_in_ci
maxschulz-COL Oct 24, 2024
437e21b
Further PR comments
maxschulz-COL Oct 24, 2024
e7f763f
Print out for debugging
maxschulz-COL Oct 24, 2024
f247c13
Include file ending
maxschulz-COL Oct 24, 2024
bb69b56
Switch to pathlib
maxschulz-COL Oct 24, 2024
4fc9097
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 24, 2024
54f325a
debugging
maxschulz-COL Oct 24, 2024
28a1eaf
Skip directories
maxschulz-COL Oct 24, 2024
dd9ede2
wrong indent
maxschulz-COL Oct 24, 2024
12fedf4
Refactor to make more pythonic
maxschulz-COL Oct 24, 2024
b81bbda
Improve extra requirements
maxschulz-COL Oct 24, 2024
4ef09be
Change from env variables
maxschulz-COL Oct 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/pycafe-dashboards-in-CI.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: PyCafe Dashboards

on:
push:
branches: [main]
pull_request:
branches:
- main
maxschulz-COL marked this conversation as resolved.
Show resolved Hide resolved
paths:
- "vizro-core/**"

defaults:
run:
working-directory: vizro-core

env:
PYTHON_VERSION: "3.12"

jobs:
create-links:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install Hatch
run: pip install hatch
- name: Build vizro-core package
run: |
hatch build -t wheel
- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: pip
path: vizro-core/dist/*.whl
retention-days: 14

- name: Create PyCafe links
run: |
hatch run python ../tools/pycafe/create_pycafe_links.py ${{ github.token }} ${{ github.repository }} ${{ github.event.number }} ${{ github.run_id }}
139 changes: 139 additions & 0 deletions tools/pycafe/create_pycafe_links.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
"""Generate PyCafe links for the example dashboards and post them as a comment on the pull request and as status."""

import base64
import datetime
import gzip
import json
import sys
import textwrap
from pathlib import Path
from typing import Optional
from urllib.parse import quote, urlencode

from github import Auth, Github

GITHUB_TOKEN = sys.argv[1]
REPO_NAME = sys.argv[2]
PR_NUMBER = int(sys.argv[3])
RUN_ID = sys.argv[4]
WHL_FILE = next(Path("dist").glob("*.whl")).name
PYCAFE_URL = "https://py.cafe"
VIZRO_RAW_URL = "https://raw.githubusercontent.com/mckinsey/vizro"

BOT_COMMENT_TEMPLATE = """## View the example dashboards of the current commit live on PyCafe :coffee: :rocket:\n
Updated on: {current_utc_time}
Commit: {commit_sha}

{dashboards}
"""

# Access
auth = Auth.Token(GITHUB_TOKEN)
g = Github(auth=auth)

# Get PR and commits
repo = g.get_repo(REPO_NAME)
pr = repo.get_pull(PR_NUMBER)
commit_sha = pr.head.sha
commit = repo.get_commit(commit_sha)


def generate_link(directory: str, extra_requirements: Optional[list[str]] = None):
"""Generate a PyCafe link for the example dashboards."""
base_url = f"{VIZRO_RAW_URL}/{commit_sha}/vizro-core/{directory}"

# Requirements
requirements = "\n".join(
filter(
None,
[
f"{PYCAFE_URL}/gh/artifact/mckinsey/vizro/actions/runs/{RUN_ID}/pip/{WHL_FILE}",
*(extra_requirements or []),
],
)
)

# App file
app_content = Path(directory, "app.py").read_text()
app_content_split = app_content.split('if __name__ == "__main__":')
if len(app_content_split) > 1:
app_content = app_content_split[0] + textwrap.dedent(app_content_split[1])

# JSON object
json_object = {
"code": app_content,
"requirements": requirements,
"files": [],
}

json_object["files"] = [
{
"name": str(file_path.relative_to(directory)),
"url": f"{base_url}/{file_path.relative_to(directory).as_posix()}",
}
for file_path in Path(directory).rglob("*")
if not file_path.is_dir() and file_path.relative_to(directory) != Path("app.py")
]

json_text = json.dumps(json_object)
compressed_json_text = gzip.compress(json_text.encode("utf8"))
base64_text = base64.b64encode(compressed_json_text).decode("utf8")
query = urlencode({"c": base64_text}, quote_via=quote)
maxschulz-COL marked this conversation as resolved.
Show resolved Hide resolved
return f"{PYCAFE_URL}/snippet/vizro/v1?{query}"


def post_comment(urls: dict[str, str]):
"""Post a comment on the pull request with the links to the PyCafe dashboards."""
# Inspired by https://github.com/snehilvj/dash-mantine-components

# Find existing comments by the bot
comments = pr.get_issue_comments()
bot_comment = None
for comment in comments:
if comment.body.startswith("## View the example dashboards of the current commit live"):
bot_comment = comment
break

# Get current UTC datetime
current_utc_time = datetime.datetime.now(datetime.UTC).strftime("%Y-%m-%d %H:%M:%S UTC")

# Define the comment body with datetime
dashboards = "\n\n".join(f"Link: [{directory}]({url})" for directory, url in urls.items())

# Update the existing comment or create a new one
if bot_comment:
bot_comment.edit(
BOT_COMMENT_TEMPLATE.format(current_utc_time=current_utc_time, commit_sha=commit_sha, dashboards=dashboards)
)
print("Comment updated on the pull request.") # noqa
else:
pr.create_issue_comment(
BOT_COMMENT_TEMPLATE.format(current_utc_time=current_utc_time, commit_sha=commit_sha, dashboards=dashboards)
)
print("Comment added to the pull request.") # noqa


if __name__ == "__main__":
directories_with_requirements = {
"examples/dev/": ["openpyxl"],
"examples/scratch_dev": None,
"examples/visual-vocabulary/": None,
}
urls = {
directory: generate_link(directory, extra_requirements)
for directory, extra_requirements in directories_with_requirements.items()
}

# Create status
for directory, url in urls.items():
# Define the deployment status
state = "success" # Options: 'error', 'failure', 'pending', 'success'
description = "Test out the app live on PyCafe"
context = f"PyCafe Example ({directory})"

# Create the status on the commit
commit.create_status(state=state, target_url=url, description=description, context=context)
print(f"Status created for {context} with URL: {url}") # noqa

# Post the comment with the links
post_comment(urls)
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<!--
A new scriv changelog fragment.

Uncomment the section that is right (remove the HTML comment wrapper).
-->

<!--
### Highlights ✨

- A bullet item for the Highlights ✨ category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX. ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Removed

- A bullet item for the Removed category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX. ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Added

- A bullet item for the Added category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX. ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Changed

- A bullet item for the Changed category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX. ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Deprecated

- A bullet item for the Deprecated category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX. ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Fixed

- A bullet item for the Fixed category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX. ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
<!--
### Security

- A bullet item for the Security category with a link to the relevant PR at the end of your entry, e.g. Enable feature XXX. ([#1](https://github.com/mckinsey/vizro/pull/1))

-->
2 changes: 1 addition & 1 deletion vizro-core/examples/scratch_dev/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@


page = vm.Page(
title="Diverging bar",
title="Test I",
components=[
vm.Graph(
figure=px.bar(
Expand Down
1 change: 0 additions & 1 deletion vizro-core/examples/scratch_dev/charts/__init__.py

This file was deleted.

54 changes: 0 additions & 54 deletions vizro-core/examples/scratch_dev/charts/charts.py

This file was deleted.

3 changes: 2 additions & 1 deletion vizro-core/hatch.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ dependencies = [
"pyyaml",
"openpyxl",
"jupyter",
"pre-commit"
"pre-commit",
"PyGithub"
]
installer = "uv"

Expand Down
2 changes: 1 addition & 1 deletion vizro-core/src/vizro/models/_dashboard.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
html,
)

import vizro
from vizro._themes._templates.template_dashboard_overrides import dashboard_overrides

try:
Expand All @@ -31,7 +32,6 @@

from dash.development.base_component import Component

import vizro
from vizro._constants import MODULE_PAGE_404, VIZRO_ASSETS_PATH
from vizro.actions._action_loop._action_loop import ActionLoop
from vizro.models import Navigation, VizroBaseModel
Expand Down