Column lineage: Empty field dataset after dbt-ol run

[amzar96]

Hi,

I don't know if this is an issue or a limitation.

At first, I am facing an issue generating column lineage in which no field appears in the facets.

ERROR [2024-12-26 09:41:14,327] marquez.db.OpenLineageDao: Cannot produce column lineage for missing output field in output dataset: xxxx

here is the output facets

{
  "fields": {
    "entity_code": {
      "inputFields": [
        {
          "name": "db.schema.table01",
          "field": "entity_code",
          "namespace": "redshift",
          "transformations": []
        }
      ],
      "transformationType": "",
      "transformationDescription": ""
    }
  }
}

When I check the dataset, no field is attached. Somehow, OL is unable to detect the fields.

---

After few rounds of testing, I noticed I needed to ensure all the fields must be specified in source.yml and schema.yml. Also, I cannot use SELECT * FROM xxxxx in the model sql. I need to specify SELECT entity_code, xxx, xxx FROM xxx to ensure the column lineage is generated.

[boring-cyborg]

Thanks for opening your first issue in the Marquez project! Please be sure to follow the issue template!