Flesh out the MkDocs (#13)

* Flesh out the MkDocs

* added a ton to the docs

* flesh out the docs more

* further docs updates

* clean up more documentation/commands on the CLI

* remove docs docker stuff, more build-out for the example

* finish building out the example

* minor tweaks

* add docs/site to the gitignore

.gitignore

@@ -15,6 +15,9 @@ tmp/*
 # java specific
 *.class
 
+# docs
+docs/fides/site/
+
 # python specific
 *.pyc
 

Makefile

@@ -30,10 +30,6 @@ help:
 # Dev
 ####################
 
-# Docs
-docs: compose-build
-	@docker-compose up fides-docs
-
 # CLI
 cli: compose-build check-db
 	@docker-compose run $(CLI_IMAGE_NAME)

README.md

@@ -12,28 +12,6 @@
 1. Wait a minute or so for the server to be available, you can check this with `fidesctl connect`
 1. `fidesctl` to get a list of possible commands
 
----
-
-## Glossary
-
-The following is a list of Fides primitives and their definitions
-
-* data-category - A taxonomy of the category that the data is in, i.e. `Account data` or `End-user contact data`.
-* data-subject-category - A taxonomy of the type of individual the data is coming from, i.e. `Customer` or `Job Applicant`.
-* data-qualifier - A taxonomy of the privacy level of the data. Goes from least dangerous (Anonymized) to most dangerous (Identified).
-* data-use - A taxonomy that describes how data is used within a system, i.e. `Train AI System` or `Market/Advertise/Promote`.
-* organization - Every object must be assigned to an organization.
-* policy - A collection of policy rules.
-* policy-rule - A compliance rule defining what action to take when certain combinations of data objects are used.
-* registry - A collection of systems.
-* system - A complete service/application privacy manifest within an organization.
-* user - A Fides User, assigned a role and organization within Fides.
-* dataset - A representation of a complete datasource or schema
-* dataset-table - A single table or collection within a specified datasource
-* dataset-field - A single field within a specified dataset-table
-
----
-
 ## Configuring Fides
 
 ### Application Variables
@@ -55,65 +33,6 @@ Fides ships with default data taxonomies that include standard examples, but in
 
 With your manifests defined, the next step is to “submit” your system for approval or rejection. This is done using the `fidesctl rate <path_to_system_manifests> <policy_id>` command. This will return an object that shows the status of the rating.
 
-## Deployment
-
-The easiest way to deploy Fides is with `pip` for the CLI and `Docker` for the Server and DB.
-
-1. spin up the db
-1. `pip install fidesctl`
-1. Run the server via it's docker container (`registry.gitlab.com/ethyca/fides-core/fides-server:latest`), injecting the expected env vars for the database connection
-
----
-
 ## Contributing
 
 There are two components to the Fides project; the CLI and the Server. The CLI is a Python application and the Server is a Scala application powered by MySQL.
-
-### Docker Quickstart
-
-To get the whole project up and running via the included docker-compose file, follow these steps:
-
-1. `make init-db` - This will get the database set up and run the migrations.  
-1. `make cli` - This will spin up the entire application (DB, Server and CLI) and give you access to a shell inside of the CLI container to interact with the Server.
-
-### Locally-installed Quickstart (Not Recommended)
-
-#### Getting the Server up and running
-
-1. Install [sbt](https://docs.scala-lang.org/getting-started/sbt-track/getting-started-with-scala-and-sbt-on-the-command-line.html). sbt requires at least java version 8. This project works on any java version >= 8.
-1. `cd` into the `fides-server` directory
-1. Edit the values in src/main/resources/application.conf.example. Any values set here will override the values set in src/main/resources/reference.conf
-1. sbt ~jetty:start
-1. Endpoints should now be available @ `http://localhost:8080`
-1. Current mapping of endpoints can be seen @ src/main/scala/ScalatraBootstrap
-1. Current structure of domain objects can be seen with `sbt test:runMain devtools.Generators`
-
-#### Getting the CLI up and running
-
-1. Install Python 3.9
-1. Install pip version 21.1 or newer
-1. `cd` into the `fides_cli` directory
-1. Run `pip install -e .` to have a local installation of the Fides CLI
-1. Run `fidesctl` to see a list of commands and a help message
-
-### sbt tasks
-
-Tasks used with sbt for the Server
-
-* `scalafmtAll`: format all files, including tests
-* `slickCodegen`: (re)generate slick db table models from database
-* `flywayMigrate`: runs flyway database migrations
-* `assembly`: builds a standalone jar at target/scala-2.13/fides-server-assembly-VERSION-SNAPSHOT.jar
-all other sbt tasks are standard. For a list, `sbt tasks`
-
-### Testing
-
-There are a few different commands you can use to run tests and verify your project.
-
-1. `sbt test` - Used if running the project locally or from with a Server ccontainer shell
-1. `make server-test` - Spins up a Docker instance to run the sbt tests
-1. `make cli-check-all` - Runs linters, type-checkers and tests against the CLI inside of a Docker container
-
-#### Notes
-
-If you see the docker error `failed to authorize: rpc error: code = Unknown desc = failed to fetch oauth token: unexpected status: 401 Unauthorized` make sure you do not have buildkit enabled.

docker-compose.yml

@@ -44,16 +44,5 @@ services:
     env_file:
       - env_files/mysql_db.env
 
-  fides-docs:
-    build:
-      context: docs
-      dockerfile: Dockerfile
-    command: serve -a 0.0.0.0:8000
-    expose:
-      - 8000
-    ports:
-      - "8000:8000"
-
-
 volumes:
   mysql:

docs/README.md

@@ -0,0 +1,5 @@
+# Fides Docs
+
+1. `pip install mkdocs`
+1. `cd fides`
+1. `mkdocs serve`

docs/fides/docs/contributing.md

@@ -0,0 +1,45 @@
+# Contributing
+
+There are two components to the Fides project; the CLI and the Server. The CLI is a Python application and the Server is a Scala application powered by MySQL.
+
+## Developing the Server
+
+1. Install [sbt](https://docs.scala-lang.org/getting-started/sbt-track/getting-started-with-scala-and-sbt-on-the-command-line.html). sbt requires at least java version 8. This project works on any java version >= 8.
+1. `cd` into the `fides-server` directory
+1. Edit the values in src/main/resources/application.conf.example. Any values set here will override the values set in src/main/resources/reference.conf
+1. sbt ~jetty:start
+1. Endpoints should now be available @ `http://localhost:8080`
+1. Current mapping of endpoints can be seen @ src/main/scala/ScalatraBootstrap
+1. Current structure of domain objects can be seen with `sbt test:runMain devtools.Generators`
+
+### Scala Tasks
+
+Tasks used with sbt for the Server
+
+* `scalafmtAll`: format all files, including tests
+* `slickCodegen`: (re)generate slick db table models from database
+* `flywayMigrate`: runs flyway database migrations
+* `assembly`: builds a standalone jar at target/scala-2.13/fides-server-assembly-VERSION-SNAPSHOT.jar
+all other sbt tasks are standard. For a list, `sbt tasks`
+
+## Developing the CLI
+
+1. Install Python 3.8
+1. Install pip version 21.1 or newer
+1. `cd` into the `fides_cli` directory
+1. Run `pip install -e .` to have a local installation of the Fides CLI
+1. Run `fidesctl` to see a list of commands and a help message
+
+### Python Tasks
+
+There are a few different commands you can use to run tests and verify your project.
+
+1. Black - `make cli-format`
+1. Pylint - `make cli-lint`
+1. MyPy - `make cli-typecheck
+1. Pytest - `make cli-test
+1. All of the above ^ - `make cli-check-all`
+
+## Notes
+
+If you see the docker error `failed to authorize: rpc error: code = Unknown desc = failed to fetch oauth token: unexpected status: 401 Unauthorized` make sure you do not have buildkit enabled.

docs/fides/docs/deployment.md

@@ -0,0 +1,11 @@
+# Deployment
+
+The easiest way to deploy Fides is with `Docker`. There are separate containers for `Fidesctl`, the `Server` and the `DB`.
+
+## Application Variables
+
+These are the environment variables that can be set to configure the CLI for a specific deployment:
+
+* `FIDES_SERVER_URL` - The URL of the Fides webserver
+
+After deploying the `Server` and `DB`, you need to inject the `FIDES_SERVER_URL` that points to wherever the `Server` got deployed.

docs/fides/docs/fidesctl.md

@@ -0,0 +1,3 @@
+# Fidesctl
+
+Fidesctl wraps the functionality of the Fides Server into a CLI tool to be used by either engineers or within CI/CD pipelines.

docs/fides/docs/getting_started.md

@@ -0,0 +1,42 @@
+# Getting Started
+
+This section contains Quick Start guides to help you get up and running with Fides.
+
+## Running Fides in Docker
+
+The easiest way to get started with Fides is to pull the [Fides repo](https://github.com/ethyca/fides) and launch it using the supplied `make` commands.
+
+### Docker Requirements
+
+1. Install Make
+1. Install Docker
+
+### Docker Setup
+
+1. `make cli` - this will build the required images, spin up the database, and open a shell inside of a container with `fidesctl` installed
+1. About 15 seconds after the `fidesctl` shell initializes, run the `fidesctl ping`
+1. `fidesctl` - this command will list all of the possible `fidesctl` commands
+1. See the `Tutorial` page for further usage guides
+
+## Running Fides Locally
+
+Fides can also be spun up locally without relying on Docker or Make, however there are significantly more prerequisites.
+
+### Local Requirements
+
+1. Python 3.8
+1. pip Version 21 or later
+1. Java 8
+1. sbt Version 2.12
+1. MySQL DB Version 8
+
+### Local Setup
+
+1. Clone the [Fides repo](https://github.com/ethyca/fides)
+1. Spin up the MySQL database with your desired credentials
+1. `cd fides-server/`
+1. Update the `application.conf` in `src/main/resources/` with the database credentials
+1. `sbt flywayMigrate`
+1. `sbt ~jetty:start` -> You now have a Fides Server instance up and running powered by MySQL!
+1. In a new shell -> `pip install fidesctl`
+1. You can now run `fidesctl ping` to verify that your installation is set up properly