-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This PR fixes sc-20959 which requested some preliminary PyEnsign docs that we could host on read the docs and link to from the ensign docs. This should also make it easier for PyEnsign contributors to contribute to the PyEnsign docs! I have made the following changes: 1. Set up a new [sphinx project](https://www.sphinx-doc.org/en/master/usage/quickstart.html#setting-up-the-documentation-sources) inside the `docs/` folder 2. Added some preliminary content, but mainly just to show how we can integrate our own text with the PyEnsign code using the `autodoc` feature in Sphinx.
- Loading branch information
1 parent
bf5be2e
commit 20e86f2
Showing
14 changed files
with
286 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Minimal makefile for Sphinx documentation | ||
# | ||
|
||
# You can set these variables from the command line, and also | ||
# from the environment for the first two. | ||
SPHINXOPTS ?= | ||
SPHINXBUILD ?= sphinx-build | ||
SOURCEDIR = . | ||
BUILDDIR = _build | ||
|
||
# Put it first so that "make" without argument is like "make help". | ||
help: | ||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) | ||
|
||
.PHONY: help Makefile | ||
|
||
# Catch-all target: route all unknown targets to Sphinx using the new | ||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). | ||
%: Makefile | ||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# PyEnsign Documentation | ||
|
||
*Welcome to the PyEnsign docs!* | ||
|
||
## reStructuredText | ||
|
||
PyEnsign uses [Sphinx](http://www.sphinx-doc.org/en/master/index.html) to build our documentation. With Sphinx, docstrings used to describe PyEnsign functions can be automatically included when the documentation is built via [autodoc](http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#sphinx.ext.autodoc). | ||
|
||
To take advantage of these features, our documentation must be written in reStructuredText (or "rst"). reStructuredText is similar to markdown, but not identical, and does take some getting used to. For instance, styling for things like codeblocks, external hyperlinks, internal cross references, notes, and fixed-width text are all unique in rst. | ||
|
||
If you would like to contribute to our documentation and do not have prior experience with rst, we recommend you make use of these resources: | ||
|
||
- [A reStructuredText Primer](http://docutils.sourceforge.net/docs/user/rst/quickstart.html) | ||
- [rst notes and cheatsheet](https://cheat.readthedocs.io/en/latest/rst.html) | ||
|
||
*Note: If you are on an M1 mac and you want to [install sphinx using brew](https://www.sphinx-doc.org/en/master/usage/installation.html#homebrew), make sure you have the [correct brew installation](https://stackoverflow.com/a/64997047). | ||
|
||
|
||
## Building the Docs | ||
|
||
To build the documents locally, first install any documentation-specific requirements with `pip` using the `requirements.txt` file in the `docs` directory: | ||
|
||
```bash | ||
$ pip install -r docs/requirements.txt | ||
``` | ||
|
||
You will then be able to build the documentation from inside the `docs` directory by running `make html`; the documentation will be built and rendered in the `_build/html` directory. You can view it by opening `_build/html/index.html` then navigating to your documentation in the browser. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
.. -*- mode: rst -*- | ||
Client Connection | ||
================= | ||
|
||
In order to write or read data from an underlying data system (like a database or event stream), you need a client to connect to the data system and interact with it as needed (such as reading and writing data). This connection often looks something like `conn = DBConnection(credentials)`, and after creating the conn variable, subsequent lines of code can leverage it to perform the kinds of data interactions you wish to make. | ||
|
||
*Note: To establish a client in Ensign you need an API key . By default Ensign will read credentials from the ENSIGN_CLIENT_ID and ENSIGN_CLIENT_SECRET environment variables. If you include these in your bash profile, you can connect to Ensign with the following without having to specify your credentials in code.* | ||
|
||
API Reference | ||
------------- | ||
|
||
.. automodule:: pyensign.ensign | ||
:members: Ensign | ||
:undoc-members: | ||
:show-inheritance: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
.. -*- mode: rst -*- | ||
Events | ||
====== | ||
|
||
In an event-driven or microservice architecture, an event is the atomic element of data. | ||
|
||
An events can have many different datatypes, which are then wrapped in an object or struct that provides some schema information to help Ensign know how to serialize and deserialize your data. | ||
|
||
API Reference | ||
------------- | ||
|
||
.. automodule:: pyensign.api.v1beta1.event | ||
:members: wrap, unwrap | ||
:undoc-members: | ||
:show-inheritance: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
.. -*- mode: rst -*- | ||
The PyEnsign API | ||
================ | ||
|
||
This part of the documentation describes the PyEnsign API, including: | ||
|
||
- :doc:`eventing`: How PyEnsign conceptualizes events and how to work with them. | ||
- :doc:`connecting`: Establishing a PyEnsign client connection. | ||
- :doc:`publishing`: Publishing data to a topic stream. | ||
- :doc:`subscribing`: Subscribing to a topic to read off a topic stream. | ||
- :doc:`querying`: How to query your Ensign stream data using PyEnsign, including historical data! | ||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
|
||
eventing | ||
connecting | ||
publishing | ||
subscribing | ||
querying |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
.. -*- mode: rst -*- | ||
Publishing | ||
========== | ||
|
||
A `Publisher` is responsible for emitting events to a topic stream. | ||
|
||
In Ensign, you can create a `Publisher` once you have established a client. On publish, the client checks to see if it has an open publish stream created and if it doesn’t, it opens a stream to the correct Ensign node. | ||
|
||
API Reference | ||
------------- | ||
|
||
.. automodule:: pyensign.publisher | ||
:members: Publisher | ||
:undoc-members: | ||
:show-inheritance: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
.. -*- mode: rst -*- | ||
Queries | ||
======= | ||
|
||
Ready to query data from your Ensign streams? Ensign stores chronological changes to all objects, which you can query with SQL-like syntax to get change vectors in addition to static snapshots. | ||
|
||
This page describes how to format your PyEnsign queries to get the data you want. | ||
|
||
API Reference | ||
------------- | ||
|
||
.. automodule:: pyensign.api.v1beta1.query | ||
:members: format_query | ||
:undoc-members: | ||
:show-inheritance: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
.. -*- mode: rst -*- | ||
Subscribing | ||
=========== | ||
|
||
A `Subscriber` is a downstream component that is listening for events published by a `Publisher` onto a topic. | ||
|
||
API Reference | ||
------------- | ||
|
||
.. automodule:: pyensign.subscriber | ||
:members: Subscriber | ||
:undoc-members: | ||
:show-inheritance: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# Configuration file for the Sphinx documentation builder. | ||
# | ||
# For the full list of built-in configuration values, see the documentation: | ||
# https://www.sphinx-doc.org/en/master/usage/configuration.html | ||
|
||
# -- Project information ----------------------------------------------------- | ||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information | ||
|
||
project = 'PyEnsign' | ||
copyright = '2023, Rotational Labs' | ||
author = 'Rotational Labs' | ||
|
||
# -- General configuration --------------------------------------------------- | ||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration | ||
|
||
extensions = [] | ||
|
||
templates_path = ['_templates'] | ||
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] | ||
|
||
# The master toctree document. | ||
master_doc = "index" | ||
|
||
# The suffix(es) of source filenames. | ||
# You can specify multiple suffix as a list of string: | ||
# source_suffix = ['.rst', '.md'] | ||
source_suffix = ".rst" | ||
|
||
# Add any Sphinx extension module names here, as strings. They can be | ||
# extensions coming with Sphinx (named 'sphinx.ext.*') or custom ones. | ||
extensions = [ | ||
"sphinx.ext.autodoc", | ||
"sphinx.ext.autosummary", | ||
"sphinx.ext.intersphinx", | ||
"sphinx.ext.coverage", | ||
"sphinx.ext.mathjax", | ||
"sphinx.ext.viewcode", | ||
"sphinx.ext.todo", | ||
] | ||
|
||
|
||
# -- Options for HTML output ------------------------------------------------- | ||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output | ||
|
||
html_theme = 'alabaster' | ||
html_static_path = ['_static'] | ||
|
||
# The name of the Pygments (syntax highlighting) style to use. | ||
pygments_style = "sphinx" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
.. PyEnsign documentation master file, created by | ||
sphinx-quickstart on Fri Oct 6 11:07:32 2023. | ||
You can adapt this file completely to your liking, but it should at least | ||
contain the root `toctree` directive. | ||
Welcome to PyEnsign's documentation! | ||
==================================== | ||
|
||
PyEnsign is an open source library for interacting with `Ensign <https://ensign.rotational.dev/>`_, an eventing tool for data scientists. | ||
|
||
The source code for PyEnsign is `available on GitHub <https://github.com/rotationalio/pyensign>`_. | ||
|
||
Using PyEnsign, you can: | ||
- convert your trained, pickled machine learning model into `an API <https://youtu.be/w69glRpOBD4?si=AXsoCM3QBez207Wn>`_. | ||
- bootstrap a HuggingFace LLM with `transfer learning <https://rotational.io/blog/streaming-nlp-with-llms-and-ensign/>`_. | ||
- transform static data APIs into `change flows <https://rotational.io/data-playground/>`_. | ||
- and much, much more! | ||
|
||
Recommended Learning Path | ||
------------------------- | ||
|
||
1. Check out the :doc:`quickstart`! | ||
2. Explore the `examples repo <https://github.com/rotationalio/ensign-examples/tree/main/python>`_. | ||
|
||
Find a Bug? | ||
----------- | ||
|
||
Check if there's already an `open issue <https://github.com/rotationalio/pyensign/issues/>`_ on the topic. If you don't find it there, open a `new issue <https://github.com/rotationalio/pyensign/issues/new/choose>`_ to let us know! | ||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
:caption: Contents: | ||
|
||
quickstart | ||
api/index | ||
|
||
|
||
|
||
Indices and tables | ||
================== | ||
|
||
* :ref:`genindex` | ||
* :ref:`modindex` | ||
* :ref:`search` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
@ECHO OFF | ||
|
||
pushd %~dp0 | ||
|
||
REM Command file for Sphinx documentation | ||
|
||
if "%SPHINXBUILD%" == "" ( | ||
set SPHINXBUILD=sphinx-build | ||
) | ||
set SOURCEDIR=. | ||
set BUILDDIR=_build | ||
|
||
%SPHINXBUILD% >NUL 2>NUL | ||
if errorlevel 9009 ( | ||
echo. | ||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx | ||
echo.installed, then set the SPHINXBUILD environment variable to point | ||
echo.to the full path of the 'sphinx-build' executable. Alternatively you | ||
echo.may add the Sphinx directory to PATH. | ||
echo. | ||
echo.If you don't have Sphinx installed, grab it from | ||
echo.https://www.sphinx-doc.org/ | ||
exit /b 1 | ||
) | ||
|
||
if "%1" == "" goto help | ||
|
||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% | ||
goto end | ||
|
||
:help | ||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% | ||
|
||
:end | ||
popd |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
.. -*- mode: rst -*- | ||
Quickstart | ||
========== | ||
|
||
Coming soon! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
pyensign>=0.10b0 | ||
|
||
# Documentation Dependencies | ||
Sphinx>=3.4 | ||
sphinx-rtd-theme>=0.5.1 | ||
numpydoc>=1.1 |