Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API endpoint for scheduling asset #1065

Draft
wants to merge 71 commits into
base: main
Choose a base branch
from
Draft

API endpoint for scheduling asset #1065

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
e2fced5
feature: copy trigger_schedule endpoint from SensorAPI to AssetAPI
Flix6x May 16, 2024
bfdd94f
fix: update function signature
Flix6x May 16, 2024
5795e1b
fix: get rid of deprecation warning
Flix6x May 16, 2024
2e4cfe0
fix: update docstring
Flix6x May 16, 2024
2e28ed9
fix: imports
Flix6x May 16, 2024
f3711de
fix: obtain sensors from flex-model
Flix6x May 16, 2024
305d76d
fix: create one scheduling job for each sensor listed in the flex-model
Flix6x May 16, 2024
f6e8075
fix: add todos
Flix6x May 16, 2024
4bf91dc
docs: update endpoint main descriptions and quickrefs
Flix6x May 17, 2024
0917896
style: black
Flix6x May 17, 2024
753b411
fix: changelog syntax
Flix6x May 17, 2024
ba90dd1
fix: remove redundant session commit when calling the API to trigger …
Flix6x May 17, 2024
af4debf
docs: update FlexContextSchema docstring
Flix6x May 17, 2024
49a1a12
feature: helpful message for test developers
Flix6x May 17, 2024
8a3e112
feature: check auth on sensors referenced in flex-context
Flix6x May 17, 2024
25d1180
feature: allow checking permissions on optional fields
Flix6x May 17, 2024
21ac09a
feature: decorator supports custom error handler
Flix6x May 17, 2024
b4ad01a
docs: add inline note explaining status code
Flix6x May 17, 2024
5b4c30f
feature: flex_context_loader lists all sensors contained in a flex-co…
Flix6x May 17, 2024
484a022
feature: support context loader that returns multiple contexts
Flix6x May 17, 2024
8374a9c
feature: check permissions on sensors referenced in flex-context
Flix6x May 17, 2024
4621ef6
feature: add test checking permissions
Flix6x May 17, 2024
ff22fea
fix: response with field names
Flix6x May 17, 2024
be51f1b
add create_sequential_scheduling_job function
victorgarcia98 May 17, 2024
061e1f3
add fixtures
victorgarcia98 May 17, 2024
6eb122b
add test_create_sequential_jobs
victorgarcia98 May 17, 2024
b204e4b
Merge remote-tracking branch 'origin/feature/api/endpoint-for-schedul…
victorgarcia98 May 17, 2024
22834ca
Revert "fix: response with field names"
Flix6x May 24, 2024
dc07b11
Revert "feature: add test checking permissions"
Flix6x May 24, 2024
b583c89
Revert "feature: check permissions on sensors referenced in flex-cont…
Flix6x May 24, 2024
d04bed0
Revert "feature: support context loader that returns multiple contexts"
Flix6x May 24, 2024
5fd2f63
Revert "feature: flex_context_loader lists all sensors contained in a…
Flix6x May 24, 2024
ee5e19e
Revert "docs: add inline note explaining status code"
Flix6x May 24, 2024
c2bf1af
Revert "feature: decorator supports custom error handler"
Flix6x May 24, 2024
529baa2
Revert "feature: allow checking permissions on optional fields"
Flix6x May 24, 2024
1665974
Revert "feature: check auth on sensors referenced in flex-context"
Flix6x May 24, 2024
8678a0d
docs: add inline note explaining permission decorator
Flix6x May 24, 2024
f42ceaf
fix: remove unused parameter (does not need to be formally deprecated…
Flix6x May 24, 2024
7b5b3ce
feature: check that each flexible device power sensor lives under the…
Flix6x May 24, 2024
e9e6ec9
Merge branch 'main' into feature/api/endpoint-for-scheduling-asset
victorgarcia98 May 30, 2024
378b0a7
docs: changelog entry
Flix6x Jun 3, 2024
754502a
docs: API changelog entry
Flix6x Jun 3, 2024
e1e1019
feature: allow loading API data from three places, and avoid clashing…
Flix6x Jun 4, 2024
4f86bb1
docs: typos
Flix6x Jun 4, 2024
de811ec
refactor: parameterize test cases
Flix6x Jun 5, 2024
175cd18
refactor: merge AssetIdField and GenericAssetIDField
Flix6x Jun 5, 2024
ffe82b1
refactor: move to AssetTriggerSchema
Flix6x Jun 5, 2024
45592c7
remove
Flix6x Jun 5, 2024
99bd8da
remove: seemingly obsolete workaround
Flix6x Jun 5, 2024
44cfd0c
feature: validate flex-model sensors belong to asset
Flix6x Jun 5, 2024
ee08883
feature: more clearly separate the serialized and deserialized parts …
Flix6x Jun 5, 2024
a9ddce8
feature: add end-to-end test for triggering and getting a schedule
Flix6x Jun 5, 2024
f340a95
docs: add fixture docstring explaining partial deserialization
Flix6x Jun 5, 2024
4c6c52a
style: black
Flix6x Jun 5, 2024
b328921
fix: in case no inflexible-device-sensors were part of the flex-context
Flix6x Jun 5, 2024
b40d40e
docs: fix inline comment
Flix6x Jun 5, 2024
b7961f4
fix: job enqueueing
Flix6x Jun 5, 2024
429d97f
style: black
Flix6x Jun 5, 2024
8423522
refactor: rename argument; sequential scheduling is only for assets
Flix6x Jun 5, 2024
c60f5cf
docs: explain assert statement
Flix6x Jun 5, 2024
8ba79aa
Merge remote-tracking branch 'refs/remotes/origin/main' into feature/…
Flix6x Jun 5, 2024
9f5eb6a
style: black
Flix6x Jun 5, 2024
71da5c4
handle fallback and rescheduling
victorgarcia98 Jun 5, 2024
07b44b6
style: flake8
Flix6x Jun 5, 2024
a25ab4d
Merge remote-tracking branch 'origin/feature/api/endpoint-for-schedul…
victorgarcia98 Jun 5, 2024
5f9a4dc
fix: clarify join to avoid `sqlalchemy.exc.AmbiguousForeignKeysError`
Flix6x Jun 5, 2024
1f26fa2
Merge remote-tracking branch 'origin/feature/api/endpoint-for-schedul…
victorgarcia98 Jun 5, 2024
8b0c0d0
fix merge
victorgarcia98 Jun 5, 2024
edf38a8
Merge branch 'main' into feature/api/endpoint-for-scheduling-asset
victorgarcia98 Jun 5, 2024
05064ae
fix: remove 404, because the `asset` field should be seen as part of …
Flix6x Jun 5, 2024
c7e58cb
feature: avoid having magic number for status codes
Flix6x Jun 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions flexmeasures/api/v3_0/assets.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -496,7 +496,7 @@ def trigger_schedule(
flex_context=flex_context,
)
try:
job = create_sequential_scheduling_job(
jobs = create_sequential_scheduling_job(
asset_or_sensor=asset, enqueue=True, **scheduler_kwargs
)
except ValidationError as err:
Expand All @@ -505,6 +505,6 @@ def trigger_schedule(
return invalid_flex_config(str(err))

# todo: make a 'done job' and pass that job's ID here
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically, I guess I now implemented this todo, by putting the last job ID (that of the 'done job') in the response. However, I recall we discussed returning a full list of job IDs, so that the API user can use the existing get_schedule endpoint to retrieve schedules for individual sensors. In that case, the next step would be to spec such a response.

For example:

{
    "schedules": [
        {
            "sensor": 1,
            "schedule": "<uuid>"
        },
        {
            "sensor": 2,
            "schedule": "<uuid>"
        }
    ]
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, returning a list of schedules would be the way to go. That way, we can reuse the endpoint to get the schedules.

response = dict(schedule=job.id)
response = dict(schedule=jobs[-1].id)
d, s = request_processed()
return dict(**response, **d), s
6 changes: 3 additions & 3 deletions flexmeasures/data/services/scheduling.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ def create_sequential_scheduling_job(
scheduler_specs=scheduler_specs,
requeue=requeue,
job_id=job_id,
enqueue=enqueue,
enqueue=False, # we enqueue all jobs later in this method
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not too sure about my change here, but I believe I saw duplicate jobs in the queue otherwise.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are enqueuing them later altogether. We are not checking the function argument, though. This is something to be fixed.

depends_on=previous_job,
force_new_job_creation=force_new_job_creation,
)
Expand Down Expand Up @@ -323,10 +323,10 @@ def create_sequential_scheduling_job(
for job in jobs:
current_app.queues["scheduling"].enqueue_job(job)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably move the enqueue here..

Copy link
Contributor

@victorgarcia98 victorgarcia98 Jun 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean doing something like if enqueue: ...

current_app.job_cache.add(
asset_or_sensor["id"],
asset_or_sensor.id,
job.id,
queue="scheduling",
asset_or_sensor_type=asset_or_sensor["class"].lower(),
asset_or_sensor_type=str(type(asset_or_sensor)).lower(),
)

return jobs
Expand Down
Loading