Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update how-to guides to use the new unit testing framework #1507

Merged
merged 3 commits into from
Dec 19, 2024
Merged

docs: update how-to guides to use the new unit testing framework #1507

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
38e9599
Adjust links, remove Harness, update Scenario.
tonyandrewmeyer Dec 18, 2024
5e4b68d
Tweak the docs to reflect the new Pebble behaviour.
tonyandrewmeyer Dec 18, 2024
bf533ff
Adjustments per review.
tonyandrewmeyer Dec 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/howto/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ Manage secrets <manage-secrets>
Manage the charm version <manage-the-charm-version>
Manage the workload version <manage-the-workload-version>
Get started with charm testing <get-started-with-charm-testing>
Write unit tests for a charm <write-unit-tests-for-a-charm>
Write scenario tests for a charm <write-scenario-tests-for-a-charm>
Write unit tests for a charm <write-scenario-tests-for-a-charm>
tonyandrewmeyer marked this conversation as resolved.
Show resolved Hide resolved
Write integration tests for a charm <write-integration-tests-for-a-charm>
Write legacy unit tests for a charm <write-unit-tests-for-a-charm>
Turn a hooks-based charm into an ops charm <turn-a-hooks-based-charm-into-an-ops-charm>

```
Expand Down
70 changes: 22 additions & 48 deletions docs/howto/manage-actions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

To tell users what actions can be performed on the charm, define an `actions` section in `charmcraft.yaml` that lists the actions and information about each action. The actions should include a short description that explains what running the action will do. Normally, all parameters that can be passed to the action are also included here, including the type of parameter and any default value. You can also specify that some parameters are required when the action is run.
For example:

```yaml
actions:
snapshot:
Expand Down Expand Up @@ -41,8 +42,6 @@ actions:
additionalProperties: false
```



### Observe the action event and define an event handler

In the `src/charm.py` file, in the `__init__` function of your charm, set up an observer for the action event associated with your action and pair that with an event handler. For example:
Expand All @@ -51,10 +50,9 @@ In the `src/charm.py` file, in the `__init__` function of your charm, set up an
self.framework.observe(self.on.grant_admin_role_action, self._on_grant_admin_role_action)
```


Now, in the body of the charm definition, define the action event handler. For example:

```
```python
def _on_grant_admin_role_action(self, event):
"""Handle the grant-admin-role action."""
# Fetch the user parameter from the ActionEvent params dict
Expand All @@ -77,7 +75,6 @@ def _on_grant_admin_role_action(self, event):

More detail below:


#### Use action params

To make use of action parameters, either ones that the user has explicitly passed, or default values, use the `params` attribute of the event object that is passed to the handler. This is a dictionary of parameter name (string) to parameter value. For example:
Expand All @@ -87,8 +84,8 @@ def _on_snapshot(self, event: ops.ActionEvent):
filename = event.params["filename"]
...
```
> See more: [`ops.ActionEvent.params`](https://ops.readthedocs.io/en/latest/#ops.ActionEvent.params)

> See more: [`ops.ActionEvent.params`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.ActionEvent.params)

#### Report that an action has failed

Expand All @@ -108,22 +105,24 @@ def _on_snapshot(self, event: ops.ActionEvent):
...
```

> See more: [`ops.ActionEvent.fail`](https://ops.readthedocs.io/en/latest/#ops.ActionEvent.fail)
> See more: [`ops.ActionEvent.fail`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.ActionEvent.fail)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating the reference links! PS Some day I think we should switch to relative paths everywhere. That way moving those reference docs around will no longer break things.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some day I think we should switch to relative paths everywhere. That way moving those reference docs around will no longer break things.

Agreed. I think we can do better than just relative paths and actually use references, but I'm not sure how to do that in the combination of ReST and myst-markdown. So I opened #1506 so that it's someone else's (maybe future-Tony, maybe not) problem 😆 .


#### Return the results of an action

To pass back the results of an action to the user, use the `set_results` method of the action event. These will be displayed in the `juju run` output. For example:

```python
def _on_snapshot(self, event: ops.ActionEvent):
size = self.do_snapshot(event.params['filename'])
event.set_results({'snapshot-size': size})
```
> See more: [`ops.ActionEvent.set_results`](https://ops.readthedocs.io/en/latest/#ops.ActionEvent.set_results)

> See more: [`ops.ActionEvent.set_results`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.ActionEvent.set_results)

#### Log the progress of an action

In a long-running action, to give the user updates on progress, use the `.log()` method of the action event. This is sent back to the user, via Juju, in real-time, and appears in the output of the `juju run` command. For example:

```python
def _on_snapshot(self, event: ops.ActionEvent):
event.log('Starting snapshot')
Expand All @@ -133,75 +132,51 @@ def _on_snapshot(self, event: ops.ActionEvent):
event.log('Table2 complete')
self.snapshot_table3()
```
> See more: [`ops.ActionEvent.log`](https://ops.readthedocs.io/en/latest/#ops.ActionEvent.log)

> See more: [`ops.ActionEvent.log`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.ActionEvent.log))

#### Record the ID of an action task

When a unique ID is needed for the action task - for example, for logging or creating temporary files, use the `.id` attribute of the action event. For example:

```python
def _on_snapshot(self, event: ops.ActionEvent):
temp_filename = f'backup-{event.id}.tar.gz'
logger.info("Using %s as the temporary backup filename in task %s", filename, event.id)
self.create_backup(temp_filename)
...
```
> See more: [`ops.ActionEvent.id`](https://ops.readthedocs.io/en/latest/#ops.ActionEvent.id)

> See more: [`ops.ActionEvent.id`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.ActionEvent.id)

## Test the feature

> See first: {ref}`get-started-with-charm-testing`

What you need to do depends on what kind of tests you want to write.


### Write unit tests

> See first: {ref}`write-unit-tests-for-a-charm`

When using Harness for unit tests, use the `run_action` method to verify that charm actions have the expected behaviour. This method will either raise an `ActionFailed` exception (if the charm used the `event.fail()` method) or return an `ActionOutput` object. These can be used to verify the failure message, logs, and results of the action. For example:

```python
def test_backup_action():
harness = ops.testing.Harness()
harness.begin()
try:
out = harness.run_action('snapshot', {'filename': 'db-snapshot.tar.gz'})
except ops.testing.ActionFailed as e:
assert "Could not backup because" in e.message
else:
assert out.logs == ['Starting snapshot', 'Table1 complete', 'Table2 complete']
assert 'snapshot-size' in out.results
finally:
harness.cleanup()


```

> See more: [`ops.testing.Harness.run_action`](https://ops.readthedocs.io/en/latest/#ops.testing.Harness.run_action)


### Write scenario tests
> See first: {ref}`write-scenario-tests-for-a-charm`

When using Scenario for unit tests, to verify that the charm state is as expected after executing an action, use the `run_action` method of the Scenario `Context` object. The method returns an `ActionOutput` object that contains any logs and results that the charm set.
To verify that the charm state is as expected after executing an action, use the `run` method of the `Context` object, with `ctx.on.action`. The context contains any logs and results that the charm set.

For example:

```python
from ops import testing

def test_backup_action():
action = scenario.Action('snapshot', params={'filename': 'db-snapshot.tar.gz'})
ctx = scenario.Context(MyCharm)
out = ctx.run_action(action, scenario.State())
assert out.logs == ['Starting snapshot', 'Table1 complete', 'Table2 complete']
if out.success:
assert 'snapshot-size' in out.results
else:
assert 'Failed to run' in out.failure
ctx = testing.Context(MyCharm)
ctx.run(ctx.on.action('snapshot', params={'filename': 'db-snapshot.tar.gz'}), testing.State())
assert ctx.action_logs == ['Starting snapshot', 'Table1 complete', 'Table2 complete']
assert 'snapshot-size' in ctx.action_results
```
> See more: [Scenario action testing](https://github.com/canonical/ops-scenario?tab=readme-ov-file#actions)

> See more: [`Context.action_logs`](https://ops.readthedocs.io/en/latest/reference/ops-testing.html#ops.testing.Context.action_logs), [`Context.action_results`](https://ops.readthedocs.io/en/latest/reference/ops-testing.html#ops.testing.Context.action_results), [`ActionFailed`](https://ops.readthedocs.io/en/latest/reference/ops-testing.html#ops.testing.ActionFailed)


### Write integration tests

> See first: {ref}`write-integration-tests-for-a-charm`

To verify that an action works correctly against a real Juju instance, write an integration test with `pytest_operator`. For example:
Expand All @@ -215,4 +190,3 @@ async def test_logger(ops_test):
assert action.status == 'completed'
assert action.results['snapshot-size'].isdigit()
```

46 changes: 8 additions & 38 deletions docs/howto/manage-configurations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ config:
type: string
```


### Observe the `config-changed` event and define the event handler

In the `src/charm.py` file of the charm project, in the `__init__` function of the charm, set up an observer for the config changed event and pair that with an event handler:
Expand All @@ -47,14 +46,14 @@ def _on_config_changed(self, event):
logger.debug("New application port is requested: %s", port)
self._update_layer_and_restart(None)
```
> See more: [`ops.CharmBase.config`](https://ops.readthedocs.io/en/latest/#ops.CharmBase.config)

> See more: [`ops.CharmBase.config`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.CharmBase.config)

```{caution}

- Multiple configuration values can be changed at one time through Juju, resulting in only one `config_changed` event. Thus, your charm code must be able to process more than one config value changing at a time.
- If `juju config` is run with values the same as the current configuration, the `config_changed` event will not run. Therefore, if you have a single config value, there is no point in tracking its previous value -- the event will only be triggered if the value changes.
- Configuration cannot be changed from within the charm code. Charms, by design, aren't able to mutate their own configuration by themselves (e.g., in order to ignore an admin-provided configuration), or to configure other applications. In Ops, one typically interacts with config via a read-only facade.

```

### (If applicable) Update and restart the Pebble layer
Expand All @@ -69,56 +68,27 @@ https://github.com/canonical/juju-sdk-tutorial-k8s/compare/01_create_minimal_cha

> See first: {ref}`get-started-with-charm-testing`

You'll want to add two levels of tests: unit and scenario.

### Write unit tests

> See first: {ref}`write-unit-tests-for-a-charm`

To use a unit test to verify that the configuration change is handled correct, the test needs to trigger the `config-changed` event and then check that the update method was called. In your `tests/unit/test_charm.py` file, add the following test functions to the file:
To verify that the `config-changed` event validates the port, pass the new config to the `State`, and, after running the event, check the unit status. For example, in your `tests/unit/test_charm.py` file, add the following test function:

```python
def test_invalid_port_configuration():
harness = ops.testing.Harness()
harness.begin()

harness.update_config({"server-port": 22})
assert isinstance(harness.model.unit.status, ops.BlockedStatus)

def test_port_configuration(monkeypatch):
update_called = False
def mock_update(*args):
update_called = True
monkeypatch.setattr(MyCharm, "_update_layer_and_restart", mock_update)

harness = ops.testing.Harness()
harness.begin()

harness.update_config({"server-port": 8080})
from ops import testing

assert update_called
```

### Write scenario tests

> See first: {ref}`write-scenario-tests-for-a-charm`

To use a Scenario test to verify that the `config-changed` event validates the port, pass the new config to the `State`, and, after running the event, check the unit status. For example, in your `tests/scenario/test_charm.py` file, add the following test function:

```python
def test_open_port():
ctx = scenario.Context(MyCharm)
ctx = testing.Context(MyCharm)

state_out = ctx.run("config_changed", scenario.State(config={"server-port": 22}))
state_out = ctx.run(ctx.on.config_changed(), testing.State(config={"server-port": 22}))

assert isinstance(state_out.unit_status, ops.BlockedStatus)
assert isinstance(state_out.unit_status, testingZ.BlockedStatus)
```

### Test-deploy
### Manually test

tonyandrewmeyer marked this conversation as resolved.
Show resolved Hide resolved
To verify that the configuration option works as intended, pack your charm, update it in the Juju model, and run `juju config` followed by the name of the application deployed by your charm and then your newly defined configuration option key set to some value. For example, given the `server-port` key defined above, you could try:

```text
juju config <name of application deployed by your charm> server-port=4000
```

50 changes: 5 additions & 45 deletions docs/howto/manage-leadership-changes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ In the `src/charm.py` file, in the `__init__` function of your charm, set up an
self.framework.observe(self.on.leader_elected, self._on_leader_elected)
```

> See more: [`ops.LeaderElectedEvent`](https://ops.readthedocs.io/en/latest/#ops.LeaderElectedEvent)
> See more: [`ops.LeaderElectedEvent`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.LeaderElectedEvent)

Now, in the body of the charm definition, define the event handler. For example, the handler below will update a configuration file:

Expand Down Expand Up @@ -52,48 +52,11 @@ event or an `is-leader` check. If the charm code may run longer, then extra

> See first: {ref}`get-started-with-charm-testing`


### Write unit tests

> See first: {ref}`write-unit-tests-for-a-charm`

When using Harness for unit tests, use the `set_leader()` method to control whether the unit is the leader. For example, to verify that leadership change is handled correctly:

```python
@pytest.fixture()
def harness():
yield ops.testing.Harness(MyCharm)
harness.cleanup()


def test_new_leader(harness):
# Before the test, the unit is not leader.
harness.set_leader(False)
harness.begin()
# Simulate Juju electing the unit as leader.
harness.set_leader(True)
# Assert that it was handled correctly.
assert ...


def test_leader_sets_secrets(harness):
# The unit is the leader throughout the test, and no leader-elected event
# is emitted.
harness.set_leader(True)
harness.begin()
secret_id = harness.add_model_secret(APP_NAME, content={"secret": "sssh"})
harness.update_config(secret_option=secret_id)
# Assert that the config-changed handler set additional secret metadata:
assert ...
```

> See more: [`ops.testing.Harness.set_leader`](https://ops.readthedocs.io/en/latest/harness.html#ops.testing.Harness.set_leader)

## Write scenario tests

> See first: {ref}`write-scenario-tests-for-a-charm`

When using Scenario for unit tests, pass the leadership status to the `State`. For example:
To verify behaviour when leadership has changed, pass the leadership status to the `State`. For example:

```python
class MyCharm(ops.CharmBase):
Expand All @@ -110,12 +73,11 @@ class MyCharm(ops.CharmBase):

@pytest.mark.parametrize('leader', (True, False))
def test_status_leader(leader):
ctx = scenario.Context(MyCharm, meta={"name": "foo"})
out = ctx.run('start', scenario.State(leader=leader))
assert out.unit_status == ops.ActiveStatus('I rule' if leader else 'I am ruled')
ctx = testing.Context(MyCharm, meta={"name": "foo"})
out = ctx.run(ctx.on.start(), testing.State(leader=leader))
assert out.unit_status == testing.ActiveStatus('I rule' if leader else 'I am ruled')
```


## Write integration tests

> See first: {ref}`write-integration-tests-for-a-charm`
Expand Down Expand Up @@ -146,5 +108,3 @@ async def get_leader_unit(ops_test, app, model=None):
> Examples: [Zookeeper testing upgrades](https://github.com/canonical/zookeeper-operator/blob/106f9c2cd9408a172b0e93f741d8c9f860c4c38e/tests/integration/test_upgrade.py#L22), [postgresql testing password rotation action](https://github.com/canonical/postgresql-k8s-operator/blob/62645caa89fd499c8de9ac3e5e9598b2ed22d619/tests/integration/test_password_rotation.py#L38)

> See more: [`juju.unit.Unit.is_leader_from_status`](https://pythonlibjuju.readthedocs.io/en/latest/api/juju.unit.html#juju.unit.Unit.is_leader_from_status)


Loading
Loading