Add chsql_native 0.0.1

[lmangani]

Description

chsql_native is an extension for chsql adding optional functions for reading ClickHouse Native files.

This is an independent clear room binary format reader without any clickhouse code or libraries

Status

• Experimental Release
• Potentially Unstable 🔥
• Needs Testers & Contributors

Input

Generate some native files with clickhouse-local or clickhouse-server

--- simple w/ one row, two columns
SELECT version(), number FROM numbers(1) INTO OUTFILE '/tmp/numbers.clickhouse' FORMAT Native;
--- simple w/ one column, 100000 rows
SELECT number FROM numbers(100000) INTO OUTFILE '/tmp/100000.clickhouse' FORMAT Native;
--- complex w/ multiple types
SELECT * FROM system.functions INTO OUTFILE '/tmp/functions.clickhouse' FORMAT Native;

Usage

Read ClickHouse Native files with DuckDB. Reads are full-scans at this time.

D SELECT * FROM clickhouse_native('/tmp/numbers.clickhouse');
┌──────────────┬─────────┐
│  version()   │ number  │
│   varchar    │  int32  │
├──────────────┼─────────┤
│ 24.12.1.1273 │ 0       │
└──────────────┴─────────┘

D SELECT count(*), max(number) FROM clickhouse_native('/tmp/100000.clickhouse');
┌──────────────┬─────────────┐
│ count_star() │ max(number) │
│    int64     │    int32    │
├──────────────┼─────────────┤
│       100000 │       99999 │
└──────────────┴─────────────┘

D SELECT * FROM clickhouse_native('/tmp/functions.clickhouse') WHERE alias_to != '' LIMIT 10;
┌────────────────────┬──────────────┬──────────────────┬──────────────────────┬──────────────┬─────────┬───┬─────────┬───────────┬────────────────┬──────────┬────────────┐
│        name        │ is_aggregate │ case_insensitive │       alias_to       │ create_query │ origin  │ … │ syntax  │ arguments │ returned_value │ examples │ categories │
│      varchar       │    int32     │      int32       │       varchar        │   varchar    │ varchar │   │ varchar │  varchar  │    varchar     │ varchar  │  varchar   │
├────────────────────┼──────────────┼──────────────────┼──────────────────────┼──────────────┼─────────┼───┼─────────┼───────────┼────────────────┼──────────┼────────────┤
│ connection_id      │            0 │                1 │ connectionID         │              │ System  │ … │         │           │                │          │            │
│ rand32             │            0 │                0 │ rand                 │              │ System  │ … │         │           │                │          │            │
│ INET6_ATON         │            0 │                1 │ IPv6StringToNum      │              │ System  │ … │         │           │                │          │            │
│ INET_ATON          │            0 │                1 │ IPv4StringToNum      │              │ System  │ … │         │           │                │          │            │
│ truncate           │            0 │                1 │ trunc                │              │ System  │ … │         │           │                │          │            │
│ ceiling            │            0 │                1 │ ceil                 │              │ System  │ … │         │           │                │          │            │
│ replace            │            0 │                1 │ replaceAll           │              │ System  │ … │         │           │                │          │            │
│ from_utc_timestamp │            0 │                1 │ fromUTCTimestamp     │              │ System  │ … │         │           │                │          │            │
│ mapFromString      │            0 │                0 │ extractKeyValuePairs │              │ System  │ … │         │           │                │          │            │
│ str_to_map         │            0 │                1 │ extractKeyValuePairs │              │ System  │ … │         │           │                │          │            │
├────────────────────┴──────────────┴──────────────────┴──────────────────────┴──────────────┴─────────┴───┴─────────┴───────────┴────────────────┴──────────┴────────────┤
│ 10 rows                                                                                                                                           12 columns (11 shown) │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

[lmangani]

🎄happy holidays team DuckDB Labs 🎄

[carlopi]

If sqlite_scanner reads sqlite files, this is a... chsql_scanner!

Looks good to me, thanks!

[lmangani]

If sqlite_scanner reads sqlite files, this is a... chsql_scanner!

Looks good to me, thanks!

You're so right.... might rename this in the future so we can also test removing an extension LOL