Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: front fixes in e2e test #36

Closed
wants to merge 115 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
802e659
MAJOR CHANGE: Refactor repository
Ramimashkouk Mar 18, 2024
6aae431
feat: Add CLI `init` command
Ramimashkouk Mar 18, 2024
faa0cd0
fix: Add deps to .toml file
Ramimashkouk Mar 18, 2024
8c34afa
chore: Divide processes into build and run ones
Ramimashkouk Mar 19, 2024
c331a61
feat: Add `/bot/builds` endpoint
Ramimashkouk Mar 19, 2024
7856b69
chore: Add CORS mechanism
Ramimashkouk Mar 20, 2024
8744344
refactor: Modify Preset body
Ramimashkouk Mar 20, 2024
bb81ecb
feat: Return info with `/bot/builds` & `/bot/runs`
Ramimashkouk Mar 21, 2024
7f6a0ab
chore: Update possible statuses of a process
Ramimashkouk Mar 21, 2024
75ecebc
fix: Handle error to return `Failed` properly
Ramimashkouk Mar 21, 2024
82b0239
chore: Fix "recording each log twice"
Ramimashkouk Mar 21, 2024
43a84aa
refactor: Unify executing subprocess.run
Ramimashkouk Mar 21, 2024
d6bbfc7
fix: Give up on bot logging to let the IO flow
Ramimashkouk Mar 22, 2024
e517c6c
feat: Add DB
Ramimashkouk Mar 22, 2024
fe77ba4
chore: Define literal types to preset end_status
Ramimashkouk Mar 22, 2024
9f6936d
refactor: refactor ProcessManager
Ramimashkouk Mar 23, 2024
ca2a690
feat: Update DB after each process start and stop
Ramimashkouk Mar 25, 2024
ee5d986
fix: Check process statuses periodically
Ramimashkouk Mar 26, 2024
64785f6
feat: Add `/flows` endpoint
Ramimashkouk Mar 27, 2024
ea818a5
fix: Check run_id in build's runs before saving
Ramimashkouk Mar 27, 2024
6d9df9f
feat: Introduce Docker implementation for backend
Ramimashkouk Mar 27, 2024
ddb3f81
chore: Fix the used dff version
Ramimashkouk Mar 27, 2024
5842942
feat: Provide logging system
Ramimashkouk Mar 28, 2024
c18b59d
fix: Avoid logging out of proj dir
Ramimashkouk Mar 29, 2024
514997e
feat: Serve static files
Ramimashkouk Mar 29, 2024
8d390c4
fix: Correct config server to reload
Ramimashkouk Mar 29, 2024
1ec9a6b
init new front
mxerf Mar 30, 2024
c793346
refactor: Make proper use of pydantic for config
Ramimashkouk Mar 30, 2024
9b79e93
fix: Change `/` to `/home` so that front see it
Ramimashkouk Apr 1, 2024
744880f
feat: Add Dockerfile putting front&end together
Ramimashkouk Apr 1, 2024
2245539
Merge branch 'MAJOR_CHANGE/new_structure' into MAJOR_CHANGE/new_front
Ramimashkouk Apr 1, 2024
359fbbd
Merge pull request #12 from deeppavlov/MAJOR_CHANGE/new_front
Ramimashkouk Apr 1, 2024
fa212c2
nodes layout ui
mxerf Apr 2, 2024
f1de041
fix: Integrate front in back then build the wheel
Ramimashkouk Apr 3, 2024
6805855
chore: Delete old frontend file
Ramimashkouk Apr 3, 2024
edbe185
redirect /app
mxerf Apr 4, 2024
b4c44a5
Merge remote-tracking branch 'origin/MAJOR_CHANGE/new_front' into MAJ…
Ramimashkouk Apr 4, 2024
376029f
fix: Fix static files routing
Ramimashkouk Apr 4, 2024
56a6b74
fix: Cancel pending tasks of websocket properly
Ramimashkouk Apr 4, 2024
d7a7bdb
feat: Add `build_scenario` cli command
Ramimashkouk Apr 5, 2024
79507a1
chore: Enhance retrieval from `/builds` & `/runs`
Ramimashkouk Apr 5, 2024
978969a
feat: Add pagination to `/builds` and `/runs`
Ramimashkouk Apr 7, 2024
bdc8700
feat: Add endpoint to get logs
Ramimashkouk Apr 7, 2024
ff83ef1
fix: Raise error when process not found
Ramimashkouk Apr 7, 2024
019ab66
feat: Connect `build_scenario` command to ui
Ramimashkouk Apr 8, 2024
f1879f1
refactor: Move processes and mangers to services
Ramimashkouk Apr 8, 2024
b23de8f
chore: Update storage logic
Ramimashkouk Apr 8, 2024
e6edc70
critical fixes
mxerf Apr 9, 2024
33b70cc
nodes-layout-ui
mxerf Apr 2, 2024
3e268c8
fix multiplie transitions bug
mxerf Apr 9, 2024
9f99870
fix types bug
mxerf Apr 10, 2024
2657583
Merge branch 'MAJOR_CHANGE/new_front' into MAJOR_CHANGE/new_structure
Ramimashkouk Apr 10, 2024
c9b6813
refactor: Rename pid to run_id/build_id
Ramimashkouk Apr 10, 2024
584c497
fix: Handle process expected errors
Ramimashkouk Apr 10, 2024
c72d7cc
feat: Include versioning in endpoint paths
Ramimashkouk Apr 10, 2024
874f012
chore: Access db asynchronously in `build_scenario`
Ramimashkouk Apr 10, 2024
1597639
start node fix + python conditions + new conditions structure
mxerf Apr 11, 2024
f060c8e
Merge branch 'MAJOR_CHANGE/new_structure' into MAJOR_CHANGE/new_front
mxerf Apr 11, 2024
92b53eb
delete old nodes
mxerf Apr 11, 2024
754f8e6
Merge remote-tracking branch 'origin/MAJOR_CHANGE/new_structure' into…
mxerf Apr 11, 2024
7a8bde6
new api adaptation
mxerf Apr 11, 2024
ebb2055
upd str
mxerf Apr 23, 2024
f8acc79
node instr header fix
mxerf Apr 23, 2024
54233f7
init e2e front test
mxerf May 7, 2024
743e176
start node fix + python conditions + new conditions structure
mxerf Apr 11, 2024
5dfc850
new api adaptation
mxerf Apr 11, 2024
57a33f3
chore: Adapt to new frontend json format
Ramimashkouk Apr 12, 2024
1a9c01d
style: black backend up
Ramimashkouk Apr 12, 2024
f646e50
style: isort things up
Ramimashkouk Apr 12, 2024
1b29062
ci: Add github workflows
Ramimashkouk Apr 12, 2024
0ab147f
fix: Maintain data across Docker restarts
Ramimashkouk Apr 12, 2024
dec213d
ci: fix workflows
Ramimashkouk Apr 14, 2024
8936933
refactor: Move get_last_id to parent class
Ramimashkouk Apr 17, 2024
d863b4d
fix: Update front lock
Ramimashkouk May 7, 2024
4b0ba66
fix: Correct docker internal paths
Ramimashkouk May 7, 2024
9726e07
test: Add unit tests for services
Ramimashkouk Apr 17, 2024
f5b88bc
test: Add unit tests for endpoints
Ramimashkouk Apr 17, 2024
722ddaf
style: black up tests
Ramimashkouk Apr 17, 2024
8e4b27e
style: isort tests
Ramimashkouk Apr 17, 2024
a1c3b5e
test: Add websocket_manager unit test
Ramimashkouk Apr 17, 2024
eb98dee
fix: Unify code to use async processes
Ramimashkouk Apr 20, 2024
c5b2ebf
chore: Create log path in get_logger
Ramimashkouk Apr 20, 2024
056dbe0
test: Add unit `test_start` for run
Ramimashkouk Apr 20, 2024
63ce01f
test: Add endpoints integration test
Ramimashkouk Apr 20, 2024
438b00c
intgr-test: Add websocket
Ramimashkouk Apr 22, 2024
1d6e6ee
refactor: Clean test_api_integration up
Ramimashkouk Apr 22, 2024
290a7ec
style: Format and lint code
Ramimashkouk Apr 22, 2024
8d8d680
ci: Add tests to workflow
Ramimashkouk Apr 22, 2024
f3c6779
test: Fix communicating indefinetly
Ramimashkouk Apr 23, 2024
be99bba
fix: Avoid creating 2 event_loop in same thread
Ramimashkouk Apr 23, 2024
45daeb6
chore: Catch internal process logs
Ramimashkouk Apr 25, 2024
a5a8398
style: black up
Ramimashkouk Apr 25, 2024
e9f8110
test: Fix test_check_status
Ramimashkouk Apr 25, 2024
8728055
feat: Add `alive` status to run processes
Ramimashkouk Apr 26, 2024
555cb39
refactor: Move fixtures and use consts
Ramimashkouk Apr 26, 2024
9df8a0d
unit-test: Correct intg-tests to actual unit tests
Ramimashkouk Apr 26, 2024
63fc789
test: Add backend e2e
Ramimashkouk Apr 26, 2024
b7477cf
feat: Add indexing for dff services
Ramimashkouk May 3, 2024
78276c7
refactor: json_translator
Ramimashkouk May 6, 2024
26d644f
fix: Adapt index linenos to front changes
Ramimashkouk May 6, 2024
3a9e740
Merge feat/support_indexed_search into MAJOR_CHANGE/new_front
Ramimashkouk May 7, 2024
6ae28f8
style: black up
Ramimashkouk May 8, 2024
3a2a1db
Merge branch 'MAJOR_CHANGE/new_front' into MAJOR_CHANGE/tests_new_front
Ramimashkouk May 8, 2024
453e677
test: Fix e2e test making it work with one client
Ramimashkouk May 8, 2024
2663307
chore: Rename e2e test
Ramimashkouk May 8, 2024
7bf772a
Merge branch 'test/add_tests' into feat/support_indexed_search
Ramimashkouk May 8, 2024
a89de98
fix: Write conditions ending with one \n
Ramimashkouk May 8, 2024
9ea1714
Merge branch 'feat/support_indexed_search' into MAJOR_CHANGE/new_front
Ramimashkouk May 8, 2024
e541131
Merge branch 'MAJOR_CHANGE/new_front' into MAJOR_CHANGE/tests_new_front
Ramimashkouk May 8, 2024
05d51ac
port 5173 to 8000 change
mxerf May 13, 2024
95ff800
test: Add e2e test workflow
Ramimashkouk May 14, 2024
a494df5
chore: Push bun.lockb
Ramimashkouk May 14, 2024
d7a2925
feat: Add make file
Ramimashkouk May 15, 2024
9c9a963
test: Fix unit test test_start
Ramimashkouk May 15, 2024
09e2ea5
merge
mxerf May 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions .github/workflows/backend_check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
name: check backend

on: [push, pull_request]

jobs:
build_backend_and_check_codestyle:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: set up python 3.10
uses: actions/setup-python@v5
with:
python-version: '3.10'

- name: setup poetry and install dependencies
run: |
python -m pip install --upgrade pip poetry
python -m poetry lock --no-update
python -m poetry install --with lint --no-interaction
working-directory: backend/df_designer

- name: build wheel
run: |
python -m poetry build
working-directory: backend/df_designer

- name: run black codestyle
run: |
python -m poetry run black --line-length=120 --check .
working-directory: backend/df_designer

- name: run flake8 codestyle
run: |
ls
bash ./bin/run_codestyle.sh DIFF_BRANCH=${BASE_BRANCH}
env:
BASE_BRANCH: ${{github.base_ref}}

- name: run isort codestyle
run: |
python -m poetry run isort --line-length=120 --diff .
working-directory: backend/df_designer

test_backend:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: set up python 3.10
uses: actions/setup-python@v5
with:
python-version: '3.10'

- name: setup poetry and install dependencies
run: |
python -m pip install --upgrade pip poetry
python -m poetry lock --no-update
python -m poetry install --with lint --no-interaction
working-directory: backend/df_designer

- name: build wheel
run: |
python -m poetry build
working-directory: backend/df_designer

- name: Create new project
run: |
python -m poetry run dflowd init --destination ../../ --no-input --overwrite-if-exists
working-directory: backend/df_designer

- name: run tests
run: |
python -m poetry install
python -m poetry run pytest ../backend/df_designer/app/tests/ --verbose
working-directory: df_designer_project
28 changes: 0 additions & 28 deletions .github/workflows/ci.yml

This file was deleted.

32 changes: 32 additions & 0 deletions .github/workflows/codestyle.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: codestyle

on: [push, pull_request]

jobs:
check_backend_codestyle:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: set up python 3.10
uses: actions/setup-python@v5
with:
python-version: '3.10'

- name: setup poetry and install dependencies
run: |
python -m pip install --upgrade pip poetry
python -m poetry lock --no-update
python -m poetry install --with lint --quiet --no-interaction
working-directory: backend/df_designer

- name: run black codestyle
run: |
python -m poetry run black --line-length=120 --check .
working-directory: backend/df_designer

- name: run flake8 codestyle
run: |
bash ./bin/run_codestyle.sh DIFF_BRANCH=${BASE_BRANCH}
env:
BASE_BRANCH: ${{github.base_ref}}
29 changes: 29 additions & 0 deletions .github/workflows/docker_check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Build Docker Images

on:
push:
branches:
- dev
pull_request:
branches:
- dev

jobs:
build_images:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: setup poetry and install dependencies
run: |
python -m pip install --upgrade pip poetry
python -m poetry lock --no-update
python -m poetry install --with lint --no-ansi --no-interaction
working-directory: backend/df_designer

- name: Create new project
run: python -m poetry run dflowd init --destination ../../ --no-input --overwrite-if-exists
working-directory: backend/df_designer

- name: Build Frontend and Backend Images
run: docker build -f Dockerfile --build-arg PROJECT_DIR=df_designer_project --target=runtime .
135 changes: 135 additions & 0 deletions .github/workflows/e2e_test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
name: test app

on:
pull_request:
branches: [dev]

jobs:

build_frontend:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Install bun
run: npm install -g bun

- name: Install frontend dependencies
run: bun install
working-directory: frontend

- name: Build frontend
run: bun run build
working-directory: frontend

- run: touch file1.txt

- name: Archive frontend dist
uses: actions/upload-artifact@v4
with:
name: frontend-dist
path: frontend/dist


build_backend:
runs-on: ubuntu-latest
needs: build_frontend
steps:
- uses: actions/checkout@v4

- name: Create dist directory
run: mkdir -p frontend/dist

- name: Download frontend dist
uses: actions/download-artifact@v4
with:
name: frontend-dist
path: frontend/dist

- name: copy static files
run: |
cp -r frontend/dist/. backend/df_designer/app/static/

- name: set up python 3.10
uses: actions/setup-python@v5
with:
python-version: '3.10'

- name: setup poetry and install dependencies
run: |
python -m pip install --upgrade pip poetry
python -m poetry lock --no-update
python -m poetry install --with lint --no-interaction
working-directory: backend/df_designer

- name: build wheel
run: python -m poetry build
working-directory: backend/df_designer

- name: Archive backend dist
uses: actions/upload-artifact@v4
with:
name: backend-dist
path: backend/df_designer/dist


run_app:
runs-on: ubuntu-latest
needs: build_backend
steps:
- uses: actions/checkout@v4

- name: set up python 3.10
uses: actions/setup-python@v5
with:
python-version: '3.10'

- name: setup dflowd poetry and install dependencies
run: |
python -m pip install --upgrade pip poetry
python -m poetry lock --no-update
python -m poetry install --with lint --no-interaction
working-directory: backend/df_designer

- name: Create new project
run: |
python -m poetry run dflowd init --destination ../../ --no-input --overwrite-if-exists
working-directory: backend/df_designer

- name: Create dist directory
run: mkdir -p backend/df_designer/dist

- name: Download backend dist
uses: actions/download-artifact@v4
with:
name: backend-dist
path: backend/df_designer/dist

- name: setup project poetry and install dependencies
run: |
python -m pip install --upgrade pip poetry
python -m poetry lock --no-update
python -m poetry install --no-interaction
working-directory: df_designer_project

- name: Run back & front
run: |
python -m poetry run dflowd run_backend &
sleep 10
working-directory: df_designer_project

- name: Install bun
run: npm install -g bun

- name: Install frontend dependencies
run: bun install
working-directory: frontend

- name: Cypress run
uses: cypress-io/github-action@v6
with:
install-command: bun add cypress
command: bun run e2e:chrome
working-directory: frontend
browser: chrome
20 changes: 20 additions & 0 deletions .github/workflows/frontend_check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: front check

on: [push, pull_request]

jobs:
build_and_check_frontend:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install bun
run: npm install -g bun

- name: Install dependencies
run: bun install
working-directory: frontend

- name: Build frontend
run: bun run build
working-directory: frontend
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -251,4 +251,8 @@ cython_debug/

./flows.json
*.sqlite
my_project
my_project
/test-results/
/playwright-report/
/blob-report/
/playwright/.cache/
Loading
Loading