Code

prepares for nested references#1774

Code

Merged

rudolfix merged 26 commits intodeveldlt-hub/dlt:develfrom feat/prepares-for-nested-referencesdlt-hub/dlt:feat/prepares-for-nested-referencesCopy head branch name to clipboardSep 9, 2024

+3,946-2,794

Commits

Commits on Aug 31, 2024

• adds methods to detect nested and root tables via parent hint

  rudolfix

  committed
  f32a64e

  Copy full SHA for f32a64e

• skips linking in relational when no parent hint, removes linking skip for primary keys

  rudolfix

  committed
  6eb8d92

  Copy full SHA for 6eb8d92

• moves schema config and normalizer importers to schema module, braks cyclic deps with dest capabilities

  rudolfix

  committed
  4427c27

  Copy full SHA for 4427c27

• adds table_format override to pipeline run

  rudolfix

  committed
  b729905

  Copy full SHA for b729905

• resolves merge strategy using adapter, uses default for a destination if strategy not explicit

  rudolfix

  committed
  034c769

  Copy full SHA for 034c769

• removes force_iceberg flag from athena, requires explicit table_format

  rudolfix

  committed
  f176ede

  Copy full SHA for f176ede

• adds PreparedTableSchema to indicate TTableSchemas that are prepared for loading, makes verify_schema explicit method to be called by load, simplifies methods to prepare tables

  rudolfix

  committed
  e9b0879

  Copy full SHA for e9b0879

• applies table and file format to run methods in all pipeline tests

  rudolfix

  committed
  12a37e0

  Copy full SHA for 12a37e0

Commits on Sep 1, 2024

• shortens temp table names in sql jobs

  rudolfix

  committed
  4a974b4

  Copy full SHA for 4a974b4

• adds filesystem to drop command tests

  rudolfix

  committed
  4cd3dfa

  Copy full SHA for 4cd3dfa

• fixes tests

  rudolfix

  committed
  07b7ad9

  Copy full SHA for 07b7ad9

• adds method to update table from diff into extract

  rudolfix

  committed
  1cf1c98

  Copy full SHA for 1cf1c98

• athena iceberg does not create dlt pipeline state as iceberg by default

  rudolfix

  committed
  310f1e2

  Copy full SHA for 310f1e2

• other test fixes

  rudolfix

  committed
  8fb171a

  Copy full SHA for 8fb171a

Commits on Sep 3, 2024

• deprecates force_icebergs, adds hive table format to opt out

  rudolfix

  committed
  baa6f9b

  Copy full SHA for baa6f9b

Commits on Sep 8, 2024

• merges column props and hints, categorizes column props

  rudolfix

  committed
  c2d35bf

  Copy full SHA for c2d35bf

• moves type mappers into destination capabilities

  rudolfix

  committed
  1a8111d

  Copy full SHA for 1a8111d

• Merge branch 'devel' into feat/prepares-for-nested-references

  rudolfix

  committed
  8e25bb3

  Copy full SHA for 8e25bb3

• fixes tests

  rudolfix

  committed
  6a19228

  Copy full SHA for 6a19228

• fixes cap data types verification errors not being raised

  rudolfix

  committed
  21ed11c

  Copy full SHA for 21ed11c

• adds missing deps

  rudolfix

  committed
  fc89ff1

  Copy full SHA for fc89ff1

• fixes more tests

  rudolfix

  committed
  7e7708c

  Copy full SHA for 7e7708c

• allows precision and scale to be 0

  rudolfix

  committed
  200e480

  Copy full SHA for 200e480

• Merge branch 'devel' into feat/prepares-for-nested-references

  rudolfix

  committed
  16d41c6

  Copy full SHA for 16d41c6

Commits on Sep 9, 2024

• fixes more tests

  rudolfix

  committed
  0508b11

  Copy full SHA for 0508b11

• corrects connectorx for 3.12

  rudolfix

  committed
  f3654b9

  Copy full SHA for f3654b9