Skip to content

Commit

Permalink
Fix notebooks; add check for attachments=null
Browse files Browse the repository at this point in the history
  • Loading branch information
kesmit13 committed Oct 5, 2023
1 parent c8a9318 commit 6516b8b
Show file tree
Hide file tree
Showing 6 changed files with 1,585 additions and 17 deletions.
12 changes: 7 additions & 5 deletions notebooks/load-json-files-s3/meta.toml
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
[meta]
title="Load JSON files with Pipeline from S3"
description="This notebook will help you load JSON files from a public open AWS S3 bucket. You will see two modes:
*) where you map the JSON elements to columns in a relational table
*) where you just ingest all documents ito a JSON column. In that mode we also show how you can use persisted computed column for extracting JSON fields
"
description="""\
This notebook will help you load JSON files from a public open
AWS S3 bucket. You will see two modes:
*) where you map the JSON elements to columns in a relational table
*) where you just ingest all documents ito a JSON column. In that mode we also show how you can use persisted computed column for extracting JSON fields
"""
icon="chart-network"
tags=["pipeline", "json", "s3"]
destinations=["spaces"]
destinations=["spaces"]
446 changes: 445 additions & 1 deletion notebooks/load-json-files-s3/notebook.ipynb

Large diffs are not rendered by default.

19 changes: 12 additions & 7 deletions notebooks/optimize-performance-with-tpch-100/meta.toml
Original file line number Diff line number Diff line change
@@ -1,12 +1,17 @@
[meta]
title="Learn how to Optimize Performance with TPCH 100"
description="This notebook will help you understand how you can take advantage of SingleStoreDB distributed capability using TPCH-100. We recommend using a S2 or S4 workspace to see the difference in performance.
If you come from single node database, this is an important step to follow to scale your performance linearly as your data grows.
description="""\
This notebook will help you understand how you can take advantage of
SingleStoreDB distributed capability using TPCH-100. We recommend using
a S2 or S4 workspace to see the difference in performance.
You will see two areas:
*) Ingesting data using pipeline at a super fast speed (You will use a real-time embedded dashboard)
*) Compare query performance with an unoptimized database and an optimized database
"
If you come from single node database, this is an important step to follow
to scale your performance linearly as your data grows.
You will see two areas:
*) Ingesting data using pipeline at a super fast speed (You will use a real-time embedded dashboard)
*) Compare query performance with an unoptimized database and an optimized database
"""
icon="database"
tags=["performance", "benchmark", "tpch", "benchmark", "shardkey", "ingest"]
destinations=["spaces"]
destinations=["spaces"]
1,114 changes: 1,113 additions & 1 deletion notebooks/optimize-performance-with-tpch-100/notebook.ipynb

Large diffs are not rendered by default.

7 changes: 6 additions & 1 deletion resources/nb-check.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,9 @@ def new_markdown_cell(cell_id: str, content: list[str]) -> dict[str, Any]:
footer_cell = cells.pop(-1)
footer_id = footer_cell.get('id', footer_id)

# Convert source lists to a string
for cell in cells:

# Convert source lists to a string
source = cell.get('source', [])
if isinstance(source, list):
source = ''.join(source)
Expand All @@ -178,6 +179,10 @@ def new_markdown_cell(cell_id: str, content: list[str]) -> dict[str, Any]:
source = []
cell['source'] = source

# Remove "attachments": null (not sure how they get in there)
if 'attachments' in cell and cell['attachments'] is None:
cell['attachments'] = {}

# Prepare parameter substitutions for header
try:
icon_name = toml_info['meta']['icon']
Expand Down
4 changes: 2 additions & 2 deletions resources/toml-check.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ def error(msg):
if [x.lower() for x in tags] != tags:
error(f'Tags must be in all lower-case ({tags}) in {f}')

if [re.sub(r'[^a-z]', r'', x) for x in tags] != tags:
error(f'Tags can only contain letters ({tags}) in {f}')
if [re.sub(r'[^a-z0-9]', r'', x) for x in tags] != tags:
error(f'Tags can only contain letters and numbers ({tags}) in {f}')

# Currently only "spaces" is allowed in destinations
destinations = meta.get('destinations', [])
Expand Down

0 comments on commit 6516b8b

Please sign in to comment.