Skip to content
This repository has been archived by the owner on Dec 18, 2023. It is now read-only.

Default unique_tags object precludes tagless model #11

Open
esherman-credo opened this issue Dec 8, 2022 · 0 comments
Open

Default unique_tags object precludes tagless model #11

esherman-credo opened this issue Dec 8, 2022 · 0 comments

Comments

@esherman-credo
Copy link
Contributor

esherman-credo commented Dec 8, 2022

Expected Behavior

If model is untagged and evidence requirement specifies no tags, model should satisfy requirements.

Actual Behavior

Governance._unique_tags is an empty list by default. When a model with no tags is checked against requirements, the fact that there is no empty dictionary in the empty list causes the requirement to be marked as non-matching.
Unique tags default set here: https://github.com/credo-ai/credoai_connect/blob/develop/connect/governance/governance.py#L74

Comparison on this line https://github.com/credo-ai/credoai_lens/blob/develop/credoai/lens/lens.py#L456

Example output here:
image

Steps to Reproduce the Problem

  1. Generate policy pack with no tags
  2. Attempt to run Lens with model that has no tags

Solution Options

  1. Change default unique_tags to be [{}] or similar
  2. Here add an additional check to make sure one of model.tags or self.gov.unique_tags is non-empty. If both are empty, we shouldn't have an error.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant