Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed issue where the detection was also running on the output folder triggering a chain update event #158

Merged
merged 10 commits into from
Nov 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion services/dy-static-file-server/.cookiecutterrc
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,4 @@ default_context:
project_slug: 'dy-static-file-server'
project_type: 'computational'
release_date: '2021'
version: '2.0.5'
version: '2.0.6'
2 changes: 1 addition & 1 deletion services/dy-static-file-server/VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.0.5
2.0.6
6 changes: 3 additions & 3 deletions services/dy-static-file-server/docker-compose-meta.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ services:
io.simcore.name: '{"name": "dy-static-file-server"}'
io.simcore.outputs: '{"outputs": {}}'
io.simcore.type: '{"type": "dynamic"}'
io.simcore.version: '{"version": "2.0.5"}'
io.simcore.version: '{"version": "2.0.6"}'
org.label-schema.build-date: ${BUILD_DATE}
org.label-schema.schema-version: '1.0'
org.label-schema.vcs-ref: ${VCS_REF}
Expand All @@ -29,7 +29,7 @@ services:
io.simcore.name: '{"name": "dy-static-file-server-dynamic-sidecar"}'
io.simcore.outputs: '{"outputs": {"string_output": {"displayOrder": 1, "label": "String output", "description": "String value from input", "type": "string"}, "integer_output": {"displayOrder": 2, "label": "Integer output", "description": "Integer value from input", "type": "integer"}, "boolean_output": {"displayOrder": 3, "label": "Boolean output", "description": "Boolean value from input", "type": "boolean"}, "number_output": {"displayOrder": 4, "label": "Number output", "description": "Number value from input", "type": "number"}, "file_output": {"displayOrder": 5, "label": "File output", "description": "File from input", "type": "data:*/*", "fileToKeyMap": {"test_file": "file_output"}}}}'
io.simcore.type: '{"type": "dynamic"}'
io.simcore.version: '{"version": "2.0.5"}'
io.simcore.version: '{"version": "2.0.6"}'
org.label-schema.build-date: ${BUILD_DATE}
org.label-schema.schema-version: '1.0'
org.label-schema.vcs-ref: ${VCS_REF}
Expand All @@ -48,7 +48,7 @@ services:
io.simcore.name: '{"name": "dy-static-file-server-dynamic-sidecar-compose-spec"}'
io.simcore.outputs: '{"outputs": {"string_output": {"displayOrder": 1, "label": "String output", "description": "String value from input", "type": "string"}, "integer_output": {"displayOrder": 2, "label": "Integer output", "description": "Integer value from input", "type": "integer"}, "boolean_output": {"displayOrder": 3, "label": "Boolean output", "description": "Boolean value from input", "type": "boolean"}, "number_output": {"displayOrder": 4, "label": "Number output", "description": "Number value from input", "type": "number"}, "file_output": {"displayOrder": 5, "label": "File output", "description": "File from input", "type": "data:*/*", "fileToKeyMap": {"test_file": "file_output"}}}}'
io.simcore.type: '{"type": "dynamic"}'
io.simcore.version: '{"version": "2.0.5"}'
io.simcore.version: '{"version": "2.0.6"}'
org.label-schema.build-date: ${BUILD_DATE}
org.label-schema.schema-version: '1.0'
org.label-schema.vcs-ref: ${VCS_REF}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: dy-static-file-server-dynamic-sidecar-compose-spec
key: simcore/services/dynamic/dy-static-file-server-dynamic-sidecar-compose-spec
type: dynamic
integration-version: 1.0.0
version: 2.0.5
version: 2.0.6
description: Modern test dynamic service providing a docker-compose specification file (with dynamic sidecar and compose-spec). Changes to the inputs will be forwarded to the outputs. The /workdir/generated-data directory is populated if no content is present.
contact: [email protected]
authors:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: dy-static-file-server-dynamic-sidecar
key: simcore/services/dynamic/dy-static-file-server-dynamic-sidecar
type: dynamic
integration-version: 1.0.0
version: 2.0.5
version: 2.0.6
description: Modern test dynamic service (with dynamic sidecar). Changes to the inputs will be forwarded to the outputs. The /workdir/generated-data directory is populated if no content is present.
contact: [email protected]
authors:
Expand Down
2 changes: 1 addition & 1 deletion services/dy-static-file-server/metadata/metadata.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: dy-static-file-server
key: simcore/services/dynamic/dy-static-file-server
type: dynamic
integration-version: 1.0.0
version: 2.0.5
version: 2.0.6
description: Legacy test dynamic service (starts using original director-v0). The /workdir/generated-data directory is populated if no content is present.
contact: [email protected]
authors:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,30 +77,34 @@ def remap_input_to_output(input_dir: Path, output_dir: Path) -> None:

class PortsMonitor:
def __init__(
self, input_dir: Path, output_dir: Path, *, monitor_interval: float = DEFAULT_MONITOR_WAIT_INTERVAL
self,
input_dir: Path,
output_dir: Path,
*,
monitor_interval: float = DEFAULT_MONITOR_WAIT_INTERVAL,
) -> None:
self.input_dir: Path = input_dir
self.output_dir: Path = output_dir
self.paths: set[Path] = {input_dir, output_dir}
self.to_observe: set[Path] = {input_dir}
self.monitor_interval: float = monitor_interval

self._monitor_task: asyncio.Task | None = None
self._keep_running: bool = False

def _get_state(self) -> Dict[Path, Dict[Path, Tuple[str, float]]]:
"""return aggravated state for all monitored paths"""
return {p: _get_directory_state(p) for p in self.paths}
def _get_observed_state(self) -> Dict[Path, Dict[Path, Tuple[str, float]]]:
"""return aggravated state for all observed paths"""
return {p: _get_directory_state(p) for p in self.to_observe}

async def _monitor(self) -> None:

_logger.info("Started monitor")
previous_state = self._get_state()
previous_state = self._get_observed_state()

while self._keep_running:
await asyncio.sleep(self.monitor_interval)

_logger.info("Checking")
current_state = self._get_state()
current_state = self._get_observed_state()

if previous_state != current_state:
_logger.info("Change detected!")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,19 @@ async def test_folder_mirror(
await ports_monitor.stop()


async def _assert_on_change_completed(
caplog: pytest.LogCaptureFixture, *, expected_count: int
):
async for attempt in AsyncRetrying(
wait=wait_fixed(0.1),
stop=stop_after_delay(2),
reraise=True,
retry=retry_if_exception_type(AssertionError),
):
with attempt:
assert caplog.text.count("on_change completed") == 1


@pytest.mark.asyncio
async def test_folder_mirror_main(
caplog: pytest.LogCaptureFixture,
Expand All @@ -233,13 +246,12 @@ async def test_folder_mirror_main(

create_files_in_input(input_dir)

async for attempt in AsyncRetrying(
wait=wait_fixed(0.1),
stop=stop_after_delay(10),
reraise=True,
retry=retry_if_exception_type(AssertionError),
):
with attempt:
assert "on_change completed" in caplog.text
# wait a bit to trigger the check a few times
await asyncio.sleep(1)
await _assert_on_change_completed(caplog, expected_count=1)

# touch file in inputs await for another event
(input_dir / "file_input" / "test_file").touch()
await _assert_on_change_completed(caplog, expected_count=2)

await ports_monitor.stop()
2 changes: 1 addition & 1 deletion services/dy-static-file-server/versioning/service.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 2.0.5
current_version = 2.0.6
commit = False
message = service/kernel version: {current_version} → {new_version}
tag = False
Expand Down
6 changes: 3 additions & 3 deletions toc.json
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@
"name": "dy-static-file-server",
"simcore.service.settings": "[{\"name\": \"resources\", \"type\": \"Resources\", \"value\": {\"mem_limit\":268435456, \"cpu_limit\": 10000000}}, {\"name\": \"ports\", \"type\": \"int\", \"value\": 8080}, {\"name\": \"constraints\", \"type\": \"string\", \"value\": [\"node.platform.os == linux\"]}]",
"type": "dynamic",
"version": "2.0.5"
"version": "2.0.6"
},
"dy-static-file-server-dynamic-sidecar": {
"description": "Modern test dynamic service (with dynamic sidecar). Changes to the inputs will be forwarded to the outputs. The /workdir/generated-data directory is populated if no content is present.",
Expand All @@ -86,7 +86,7 @@
"name": "dy-static-file-server-dynamic-sidecar",
"simcore.service.settings": "[{\"name\": \"resources\", \"type\": \"Resources\", \"value\": {\"mem_limit\":268435456, \"cpu_limit\": 10000000}}, {\"name\": \"ports\", \"type\": \"int\", \"value\": 8080}, {\"name\": \"constraints\", \"type\": \"string\", \"value\": [\"node.platform.os == linux\"]}]",
"type": "dynamic",
"version": "2.0.5"
"version": "2.0.6"
},
"dy-static-file-server-dynamic-sidecar-compose-spec": {
"description": "Modern test dynamic service providing a docker-compose specification file (with dynamic sidecar and compose-spec). Changes to the inputs will be forwarded to the outputs. The /workdir/generated-data directory is populated if no content is present.",
Expand All @@ -96,7 +96,7 @@
"name": "dy-static-file-server-dynamic-sidecar-compose-spec",
"simcore.service.settings": "[{\"name\": \"resources\", \"type\": \"Resources\", \"value\": {\"mem_limit\":268435456, \"cpu_limit\": 10000000}}, {\"name\": \"ports\", \"type\": \"int\", \"value\": 8080}, {\"name\": \"constraints\", \"type\": \"string\", \"value\": [\"node.platform.os == linux\"]}]",
"type": "dynamic",
"version": "2.0.5"
"version": "2.0.6"
},
"jupyter-base-notebook": {
"description": "Jupyter notebook",
Expand Down
Loading