Skip to content

Commit

Permalink
fix more snippets and enable mypy typechecking on embedded snippets
Browse files Browse the repository at this point in the history
  • Loading branch information
sh-rp committed Mar 22, 2024
1 parent 8e665ac commit a8ebc3b
Show file tree
Hide file tree
Showing 3 changed files with 5 additions and 8 deletions.
3 changes: 1 addition & 2 deletions docs/tools/check_embedded_snippets.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@
LINT_TEMPLATE = "./lint_setup/template.py"
LINT_FILE = "./lint_setup/lint_me.py"

ENABLE_MYPY = False

ENABLE_MYPY = True

@dataclass
class Snippet:
Expand Down
2 changes: 1 addition & 1 deletion docs/website/docs/dlt-ecosystem/verified-sources/chess.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ specified otherwise.
@dlt.resource(write_disposition="append")
def players_games(
players: List[str], start_month: str = None, end_month: str = None
) -> Iterator[Callable[[], List[TDataItem]]]:
) -> Iterator[TDataItems]:
# gets a list of already checked(loaded) archives.
checked_archives = dlt.current.resource_state().setdefault("archives", [])
yield {} # return your retrieved data here
Expand Down
8 changes: 3 additions & 5 deletions docs/website/docs/general-usage/resource.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,7 @@ accepts following arguments:
...

# the `table_schema` method gets table schema generated by a resource
# TODO: needs fixing
print(get_users().table_schema())
print(get_users().compute_table_schema())
```

> 💡 You can pass dynamic hints which are functions that take the data item as input and return a
Expand Down Expand Up @@ -155,8 +154,7 @@ def repo_events() -> Iterator[TDataItems]:

# the `table_schema` method gets table schema generated by a resource and takes optional
# data item to evaluate dynamic hints
# TODO: needs fixing
print(repo_events().table_schema({"type": "WatchEvent", id:...}))
print(repo_events().compute_table_schema({"type": "WatchEvent", id:...}))
```

In more advanced cases, you can dispatch data to different tables directly in the code of the
Expand Down Expand Up @@ -285,7 +283,7 @@ def get_orders():
yield o

# users and orders will be iterated in parallel in two separate threads
pipeline.run[(get_users(), get_orders()])
pipeline.run([get_users(), get_orders()])
```

Async generators are automatically extracted concurrently with other resources:
Expand Down

0 comments on commit a8ebc3b

Please sign in to comment.