Skip to content

Commit

Permalink
Deploying to gh-pages from @ c6a93d2 🚀
Browse files Browse the repository at this point in the history
  • Loading branch information
@davidgasquez
davidgasquez committed Feb 19, 2024
1 parent 3da6c25 commit 6d474a8
Show file tree
Hide file tree
Showing 38 changed files with 79 additions and 115 deletions.
18 changes: 6 additions & 12 deletions README.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>

<meta charset="utf-8">
<meta name="generator" content="quarto-1.5.9">
<meta name="generator" content="quarto-1.5.14">

<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">

Expand Down Expand Up @@ -140,11 +140,6 @@
</i>
<span class="menu-text">Tutorial</span></a>
</li>
<li class="nav-item">
<a class="nav-link" href="./reports"> <i class="bi bi-journalbi-bookmark-plus" role="img">
</i>
<span class="menu-text">Reports</span></a>
</li>
</ul>
</div> <!-- /navcollapse -->
<div class="quarto-navbar-tools">
Expand Down Expand Up @@ -184,18 +179,18 @@
D A T A D E X
</h1>
<p align="center">
Collaborate on Open Data using Open Source Tools
A Data Platform for your community Open Data
</p>
<p></p>
<div data-align="center">
<p><img alt="GitHub" src="https://img.shields.io/github/license/davidgasquez/datadex?style=flat-square"> <img alt="GitHub Workflow Status" src="https://img.shields.io/github/actions/workflow/status/davidgasquez/datadex/ci.yml?style=flat-square"> <img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/davidgasquez/datadex?style=flat-square"></p>
</div>
<p><br></p>
<p>Open source, serverless, and local-first Data Platform to collaborate on Open Data! Built on top of <a href="https://dagster.io/">Dagster</a>, <a href="https://www.getdbt.com/">dbt</a>, <a href="https://quarto.org/">Quarto</a>, <a href="https://www.duckdb.org/">DuckDB</a>, and <a href="https://evidence.dev/">Evidence</a>.</p>
<p>This repository is an up to date toy implementation of the overall pattern. You can check two real world production instances of Datadex working in the following repositories:</p>
<p>Datadex is a fully open source, serverless, and local-first Data Platform that improves how communities collaborate on Open Data. Datadex is not a new tool, it is a pattern showing an opinionated bridge between existing ones.</p>
<p>You can check two real world production implementation of the Datadex pattern working in the following repositories:</p>
<ul>
<li><a href="https://github.com/davidgasquez/gitcoin-grants-data-portal">Gitcoin Grants Data Portal</a>. Improve data access and empower data scientists to conduct research and guide community driven analysis and decisions around Gitcoin Grants.</li>
<li><a href="https://github.com/davidgasquez/filecoin-data-portal/">Filecoin Data Portal</a>. Local-first data hub for Filecoin data! Like Dune, but in your laptop.</li>
<li><a href="https://github.com/davidgasquez/gitcoin-grants-data-portal">Gitcoin Grants Data Portal</a>. Data hub for Gitcoin Grants data. Improves data access and empower data scientists to conduct research and guide community driven analysis and decisions.</li>
<li><a href="https://github.com/davidgasquez/filecoin-data-portal/">Filecoin Data Portal</a>. Data hub for Filecoin data! Like Dune, but in your laptop.</li>
</ul>
<section id="principles" class="level3">
<h3 class="anchored" data-anchor-id="principles">💡 Principles</h3>
Expand All @@ -206,7 +201,6 @@ <h3 class="anchored" data-anchor-id="principles">💡 Principles</h3>
<li><strong>Data as Code</strong>. Declarative stateless transformations tracked in <code>git</code>. Version your data as code! Publish and share your reusable models for others to build on top. Datasets should be both reproducible and accessible!</li>
<li><strong>Glue</strong>: Be a bridge between tools and aproaches. E.g: Use software engineering good practices like types, tests, materialized views, and more.</li>
</ul>
<p>Datadex is not a new tool. <strong>Datadex is a pattern showing an opinionated bridge between existing ones</strong>.</p>
</section>
<section id="what-can-you-do-with-datadex" class="level3">
<h3 class="anchored" data-anchor-id="what-can-you-do-with-datadex">🚀 What can you do with Datadex?</h3>
Expand Down
2 changes: 1 addition & 1 deletion dbt/catalog.json
View file

Large diffs are not rendered by default.

Binary file modified dbt/graph.gpickle
View file
Binary file not shown.
2 changes: 1 addition & 1 deletion dbt/graph_summary.json
View file
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"_invocation_id": "0835e05e-b649-400e-976e-b2a08f51cf73", "linked": {"0": {"name": "source.datadex.public.threatened_animal_species", "type": "source"}, "1": {"name": "source.datadex.public.owid_co2_data", "type": "source"}, "2": {"name": "source.datadex.public.owid_energy_data", "type": "source", "succ": [4]}, "3": {"name": "source.datadex.public.co2_global_trend", "type": "source", "succ": [4]}, "4": {"name": "model.datadex.climate", "type": "model"}}}
{"_invocation_id": "beb7f3f7-5447-48af-81a5-6f8097336c63", "linked": {"0": {"name": "source.datadex.public.threatened_animal_species", "type": "source"}, "1": {"name": "source.datadex.public.owid_co2_data", "type": "source"}, "2": {"name": "source.datadex.public.owid_energy_data", "type": "source", "succ": [4]}, "3": {"name": "source.datadex.public.co2_global_trend", "type": "source", "succ": [4]}, "4": {"name": "model.datadex.climate", "type": "model"}}}
2 changes: 1 addition & 1 deletion dbt/manifest.json
View file

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion dbt/run_results.json
View file
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"metadata": {"dbt_schema_version": "https://schemas.getdbt.com/dbt/run-results/v5.json", "dbt_version": "1.7.7", "generated_at": "2024-02-12T10:27:53.212448Z", "invocation_id": "0835e05e-b649-400e-976e-b2a08f51cf73", "env": {}}, "results": [{"status": "success", "timing": [{"name": "compile", "started_at": "2024-02-12T10:27:53.199944Z", "completed_at": "2024-02-12T10:27:53.207151Z"}, {"name": "execute", "started_at": "2024-02-12T10:27:53.208314Z", "completed_at": "2024-02-12T10:27:53.208328Z"}], "thread_id": "Thread-1 (worker)", "execution_time": 0.010539531707763672, "adapter_response": {}, "message": null, "failures": null, "unique_id": "model.datadex.climate", "compiled": true, "compiled_code": "with\n energy_data as (\n select year, sum(solar_electricity) as solar_electricity\n from \"local\".\"public\".\"owid_energy_data\"\n where iso_code is not null and solar_electricity is not null and year >= 2014\n group by year\n ),\n co2_global_trend as (\n select year, avg(trend) as co2_trend\n from \"local\".\"public\".\"co2_global_trend\"\n group by year\n )\nselect\n energy_data.year,\n energy_data.solar_electricity,\n co2_global_trend.co2_trend as co2_trend\nfrom energy_data\nleft join co2_global_trend on energy_data.year = co2_global_trend.year", "relation_name": "\"local\".\"main\".\"climate\""}], "elapsed_time": 0.06212878227233887, "args": {"write_json": true, "use_colors_file": true, "favor_state": false, "log_format_file": "debug", "printer_width": 80, "log_level_file": "debug", "static_parser": true, "static": false, "populate_cache": true, "log_file_max_bytes": 10485760, "macro_debugging": false, "strict_mode": false, "project_dir": "/home/runner/work/datadex/datadex/dbt", "partial_parse_file_diff": true, "defer": false, "exclude": [], "quiet": false, "print": true, "indirect_selection": "eager", "profiles_dir": ".", "log_format": "default", "show_resource_report": false, "introspect": true, "which": "generate", "enable_legacy_logger": false, "invocation_command": "dbt docs generate --profiles-dir .", "empty_catalog": false, "compile": true, "warn_error_options": {"include": [], "exclude": []}, "log_path": "/home/runner/work/datadex/datadex/dbt/logs", "cache_selected_only": false, "select": [], "partial_parse": true, "vars": {}, "log_level": "info", "version_check": true, "send_anonymous_usage_stats": false, "use_colors": true}}
{"metadata": {"dbt_schema_version": "https://schemas.getdbt.com/dbt/run-results/v5.json", "dbt_version": "1.7.8", "generated_at": "2024-02-19T10:00:38.509085Z", "invocation_id": "beb7f3f7-5447-48af-81a5-6f8097336c63", "env": {}}, "results": [{"status": "success", "timing": [{"name": "compile", "started_at": "2024-02-19T10:00:38.496408Z", "completed_at": "2024-02-19T10:00:38.503736Z"}, {"name": "execute", "started_at": "2024-02-19T10:00:38.504955Z", "completed_at": "2024-02-19T10:00:38.504968Z"}], "thread_id": "Thread-1 (worker)", "execution_time": 0.010768890380859375, "adapter_response": {}, "message": null, "failures": null, "unique_id": "model.datadex.climate", "compiled": true, "compiled_code": "with\n energy_data as (\n select year, sum(solar_electricity) as solar_electricity\n from \"local\".\"public\".\"owid_energy_data\"\n where iso_code is not null and solar_electricity is not null and year >= 2014\n group by year\n ),\n co2_global_trend as (\n select year, avg(trend) as co2_trend\n from \"local\".\"public\".\"co2_global_trend\"\n group by year\n )\nselect\n energy_data.year,\n energy_data.solar_electricity,\n co2_global_trend.co2_trend as co2_trend\nfrom energy_data\nleft join co2_global_trend on energy_data.year = co2_global_trend.year", "relation_name": "\"local\".\"main\".\"climate\""}], "elapsed_time": 0.0625, "args": {"indirect_selection": "eager", "show_resource_report": false, "enable_legacy_logger": false, "project_dir": "/home/runner/work/datadex/datadex/dbt", "warn_error_options": {"include": [], "exclude": []}, "partial_parse": true, "log_format": "default", "which": "generate", "profiles_dir": ".", "cache_selected_only": false, "print": true, "quiet": false, "static_parser": true, "defer": false, "use_colors_file": true, "invocation_command": "dbt docs generate --profiles-dir .", "empty_catalog": false, "log_path": "/home/runner/work/datadex/datadex/dbt/logs", "vars": {}, "static": false, "strict_mode": false, "write_json": true, "use_colors": true, "log_file_max_bytes": 10485760, "send_anonymous_usage_stats": false, "log_format_file": "debug", "populate_cache": true, "macro_debugging": false, "printer_width": 80, "introspect": true, "log_level": "info", "log_level_file": "debug", "favor_state": false, "compile": true, "exclude": [], "select": [], "partial_parse_file_diff": true, "version_check": true}}
7 changes: 1 addition & 6 deletions docs/tutorial.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>

<meta charset="utf-8">
<meta name="generator" content="quarto-1.5.9">
<meta name="generator" content="quarto-1.5.14">

<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">

Expand Down Expand Up @@ -140,11 +140,6 @@
</i>
<span class="menu-text">Tutorial</span></a>
</li>
<li class="nav-item">
<a class="nav-link" href="../reports"> <i class="bi bi-journalbi-bookmark-plus" role="img">
</i>
<span class="menu-text">Reports</span></a>
</li>
</ul>
</div> <!-- /navcollapse -->
<div class="quarto-navbar-tools">
Expand Down
18 changes: 6 additions & 12 deletions index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>

<meta charset="utf-8">
<meta name="generator" content="quarto-1.5.9">
<meta name="generator" content="quarto-1.5.14">

<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">

Expand Down Expand Up @@ -140,11 +140,6 @@
</i>
<span class="menu-text">Tutorial</span></a>
</li>
<li class="nav-item">
<a class="nav-link" href="./reports"> <i class="bi bi-journalbi-bookmark-plus" role="img">
</i>
<span class="menu-text">Reports</span></a>
</li>
</ul>
</div> <!-- /navcollapse -->
<div class="quarto-navbar-tools">
Expand Down Expand Up @@ -184,18 +179,18 @@
D A T A D E X
</h1>
<p align="center">
Collaborate on Open Data using Open Source Tools
A Data Platform for your community Open Data
</p>
<p></p>
<div data-align="center">
<p><img alt="GitHub" src="https://img.shields.io/github/license/davidgasquez/datadex?style=flat-square"> <img alt="GitHub Workflow Status" src="https://img.shields.io/github/actions/workflow/status/davidgasquez/datadex/ci.yml?style=flat-square"> <img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/davidgasquez/datadex?style=flat-square"></p>
</div>
<p><br></p>
<p>Open source, serverless, and local-first Data Platform to collaborate on Open Data! Built on top of <a href="https://dagster.io/">Dagster</a>, <a href="https://www.getdbt.com/">dbt</a>, <a href="https://quarto.org/">Quarto</a>, <a href="https://www.duckdb.org/">DuckDB</a>, and <a href="https://evidence.dev/">Evidence</a>.</p>
<p>This repository is an up to date toy implementation of the overall pattern. You can check two real world production instances of Datadex working in the following repositories:</p>
<p>Datadex is a fully open source, serverless, and local-first Data Platform that improves how communities collaborate on Open Data. Datadex is not a new tool, it is a pattern showing an opinionated bridge between existing ones.</p>
<p>You can check two real world production implementation of the Datadex pattern working in the following repositories:</p>
<ul>
<li><a href="https://github.com/davidgasquez/gitcoin-grants-data-portal">Gitcoin Grants Data Portal</a>. Improve data access and empower data scientists to conduct research and guide community driven analysis and decisions around Gitcoin Grants.</li>
<li><a href="https://github.com/davidgasquez/filecoin-data-portal/">Filecoin Data Portal</a>. Local-first data hub for Filecoin data! Like Dune, but in your laptop.</li>
<li><a href="https://github.com/davidgasquez/gitcoin-grants-data-portal">Gitcoin Grants Data Portal</a>. Data hub for Gitcoin Grants data. Improves data access and empower data scientists to conduct research and guide community driven analysis and decisions.</li>
<li><a href="https://github.com/davidgasquez/filecoin-data-portal/">Filecoin Data Portal</a>. Data hub for Filecoin data! Like Dune, but in your laptop.</li>
</ul>
<section id="principles" class="level3">
<h3 class="anchored" data-anchor-id="principles">💡 Principles</h3>
Expand All @@ -206,7 +201,6 @@ <h3 class="anchored" data-anchor-id="principles">💡 Principles</h3>
<li><strong>Data as Code</strong>. Declarative stateless transformations tracked in <code>git</code>. Version your data as code! Publish and share your reusable models for others to build on top. Datasets should be both reproducible and accessible!</li>
<li><strong>Glue</strong>: Be a bridge between tools and aproaches. E.g: Use software engineering good practices like types, tests, materialized views, and more.</li>
</ul>
<p>Datadex is not a new tool. <strong>Datadex is a pattern showing an opinionated bridge between existing ones</strong>.</p>
</section>
<section id="what-can-you-do-with-datadex" class="level3">
<h3 class="anchored" data-anchor-id="what-can-you-do-with-datadex">🚀 What can you do with Datadex?</h3>
Expand Down
7 changes: 1 addition & 6 deletions notebooks/2023-01-01-Datadex.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>

<meta charset="utf-8">
<meta name="generator" content="quarto-1.5.9">
<meta name="generator" content="quarto-1.5.14">

<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">

Expand Down Expand Up @@ -145,11 +145,6 @@
</i>
<span class="menu-text">Tutorial</span></a>
</li>
<li class="nav-item">
<a class="nav-link" href="../reports"> <i class="bi bi-journalbi-bookmark-plus" role="img">
</i>
<span class="menu-text">Reports</span></a>
</li>
</ul>
</div> <!-- /navcollapse -->
<div class="quarto-navbar-tools">
Expand Down
7 changes: 1 addition & 6 deletions notebooks/2023-11-01-Observable.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>

<meta charset="utf-8">
<meta name="generator" content="quarto-1.5.9">
<meta name="generator" content="quarto-1.5.14">

<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">

Expand Down Expand Up @@ -144,11 +144,6 @@
</i>
<span class="menu-text">Tutorial</span></a>
</li>
<li class="nav-item">
<a class="nav-link" href="../reports"> <i class="bi bi-journalbi-bookmark-plus" role="img">
</i>
<span class="menu-text">Reports</span></a>
</li>
</ul>
</div> <!-- /navcollapse -->
<div class="quarto-navbar-tools">
Expand Down
9 changes: 2 additions & 7 deletions notebooks/2024-01-01-Dashboard.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>

<meta charset="utf-8">
<meta name="generator" content="quarto-1.5.9">
<meta name="generator" content="quarto-1.5.14">

<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">

Expand Down Expand Up @@ -152,11 +152,6 @@
</i>
<span class="menu-text">Tutorial</span></a>
</li>
<li class="nav-item">
<a class="nav-link" href="../reports"> <i class="bi bi-journalbi-bookmark-plus" role="img">
</i>
<span class="menu-text">Reports</span></a>
</li>
</ul>
</div> <!-- /navcollapse -->
<div class="quarto-navbar-tools">
Expand Down Expand Up @@ -233,7 +228,7 @@
<span class="bslib-full-screen-enter badge rounded-pill">
<svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 24 24" style="height:1em;width:1em;" aria-hidden="true" role="img"><path d="M20 5C20 4.4 19.6 4 19 4H13C12.4 4 12 3.6 12 3C12 2.4 12.4 2 13 2H21C21.6 2 22 2.4 22 3V11C22 11.6 21.6 12 21 12C20.4 12 20 11.6 20 11V5ZM4 19C4 19.6 4.4 20 5 20H11C11.6 20 12 20.4 12 21C12 21.6 11.6 22 11 22H3C2.4 22 2 21.6 2 21V13C2 12.4 2.4 12 3 12C3.6 12 4 12.4 4 13V19Z"></path></svg>
</span>
</bslib-tooltip><script data-bslib-card-init="">bslib.Card.initializeAllCards();</script></div><div class="card bslib-card html-fill-container" data-fill="false" data-title="Plotting 10 million flights" data-bslib-card-init="" data-require-bs-caller="card()" data-full-screen="false">
</bslib-tooltip><script data-bslib-card-init="">bslib.Card.initializeAllCards();</script></div><div class="card bslib-card html-fill-container" data-title="Plotting 10 million flights" data-fill="false" data-bslib-card-init="" data-require-bs-caller="card()" data-full-screen="false">
<div class="card-header"><div class="card-title">Plotting 10 million flights</div></div>
<div class="cell card-body html-fill-item html-fill-container" data-output="asis" data-title="Plotting 10 million flights">
<div class="sourceCode cell-code hidden html-fill-item html-fill-container" id="cb3" data-startfrom="33" data-source-offset="-0"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript" style="counter-reset: source-line 32;"><span id="cb3-33"><a href="#cb3-33" aria-hidden="true" tabindex="-1"></a>vg <span class="op">=</span> {</span>
Expand Down
Loading

0 comments on commit 6d474a8

Please sign in to comment.