feat(tags): Export and Import Functionality for Superset Dashboards and Charts

[asher-lab]

SUMMARY

This PR is related to Discussion: #30629 by Spens

Motivation

We have adopted the tagging functionality and it is working quite well for us for dashboards and and charts. Our use case and process requires us to build dashboards on a development server, export the dashboards, commit the YAMLs to source control, and finally import the ZIPs to our production server. We would like the tags we assign on the development server to be captured in the export so they are replicated to our production server.

Proposed Change

UI elements are not affected by this change. All changes are related to updated logic/processing of the existing export and import functionality.

Only tags of type "custom" are exported. Other tags that exist but are not exported are "type" and "owner".

Export Logic
When an object (Dashboard/Chart) is exported, a new parameter is added to the exported YAML file. The parameter is "tags" and the value is an array of tag names. Here is an example:

slice_name: Vaccine Candidates per Approach & Stage
description: null
tags:
  - CovidTag
  - Items that start with the letter V
  - Superset Example Data
certified_by: null

...
Because tags have additional information (for example, a description), a new file is needed to hold this information. The file is called "tags.yaml" and lives in the top level of the zip file. For example:

dashboard_export_20241017T025146.zip
  charts (folder)
  dashboards (folder)
  databases (folder)
  datasets (folder)
  metadata.yaml (file)
  tags.yaml (file)

The format of the tags.yaml file is an array of tags

tags:
- tag_name: CovidTag
  description: Everything related to Covid
- tag_name: Items that start with the letter V
  description: Just for fun, tag Vanilla, Vikings, and anything that starts with V
- tag_name: Superset Example Data
  description: Native example data included with Superset

Import Logic
When an object is imported, the code checks to see if any tags included in the import file.
If yes, the tag is created if it does not already exist then the tag is assigned to the imported object. If the tag already exists, it is just assigned with no changes to the tag.

Details

1. A tag can only be created if the user doing the import has permission to create tags. Otherwise the tagging is ignored.
2. A created tag picks up the current time as the "created_on" and "changed_on" properties for the tag
3. A created tag picks up the current user as the "created_by_fk" and "changed_by_fk"
4. A created tag picks up the description from the tags.yaml file described above
5. If a tag already exists, the description is not updated (for consistency with other export/import behavior - for example, if you import a dashboard and the chart included in the zip already exists, Superset does not update that existing chart)
   Existing Superset behavior: When importing objects that already exist, the user is prompted whether it is ok to overwrite. If no, the import process aborts. If yes, the old object is replaced with the new object.

To be consistent with this behavior, when importing an object and specifying "overwrite", old tags that are no longer specified with the object are removed, new tags that didn't originally exist are created, old tags that still remain in the new set of tags are retained (descriptions are not updated for example)

1. It is important to note that in this proposal, imported tags are not added to existing tags
   This is intentional because for the use case of trying to mirror the settings and properties from one system to another, you must have the ability to remove tags from the target system. Otherwise, for example, if you rename a tag, the new name would get added and the old name would stick around which is not the desired behavior.

Thank you to anyone taking the time to read and comment!

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[michael-s-molina]

Thank you for the PR @asher-lab. Could you add unit tests for the new feature with the feature flag on and off?

[asher-lab]

Good day, @michael-s-molina. We've added logic to handle cases when the Tagging System is disabled in Superset. When disabled, the tags.yaml file and any associated tags within charts.yaml and dashboard.yaml will not be exported, and the same applies to the import operation.

Could you provide guidance on creating a unit test for this new feature? I have limited experience with writing unit tests, so any help would be appreciated.

Thank you!

[michael-s-molina]

Could you provide guidance on creating a unit test for this new feature? I have limited experience with writing unit tests, so any help would be appreciated.

Hi @asher-lab. You can check this file for examples.

[asher-lab]

Good morning @betodealmeida , resolve the merge conflicts one more run please. Thank you!