Skip to content
This repository has been archived by the owner on Dec 17, 2024. It is now read-only.

Commit

Permalink
Registry / CLI Updates (#1)
Browse files Browse the repository at this point in the history 
* Updates for bpm-cli and public registry

* Add ConfigFilesRenders - utility functions for rendering multiple files

* Allow passing commands as file when starting a docker container

The idea is that plugins create the docker command as part of the
configuration.

This allows overwriting the command before starting the container
for troubleshooting or other tinkering purposes. It also simplifies the
plugin code that starts the container and makes it easier to understand
what's going on "under-the-hood".

* Rename `remove` to `stop`.

This describes the functionality better, as by default we don't really
remove anything that cannot be easily started again

* Revert change that removed expanding the baseDir

The default uses ~ to denote the home directory. Without this, the
plugin isn't usable with the default --baseDir

* Use correct default for `--baseDir`

* Add a status command

* Make isContainerRunning public

This is useful for plugins to determine if all dependencies are running
  • Loading branch information
@SherClockHolmes
SherClockHolmes authored Sep 23, 2019
1 parent b72177a commit 2e64749
Show file tree
Hide file tree
Showing 9 changed files with 263 additions and 619 deletions.
300 changes: 137 additions & 163 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
# About BPM-SDK

[![bpm-sdk](https://godoc.org/github.com/Blockdaemon/bpm-sdk?status.svg)](https://godoc.org/github.com/Blockdaemon/bpm-sdk)

BPM is the Blockchain Package Manager by Blockdaemon. It allows easy and uniform deployment, maintenance and upgrades of blockchain nodes.

BPM itself provides the framework, the actual deployment is performed by plugins. Plugins are just binaries that provide a common set of
command line parameters (see [Plugin Lifecycle](#plugin-lifecycle))


While it is possible to write plugins without an SDK, it is
recommended and significantly easier to use the provided go SDK.

# Plugin Lifecycle
## Plugin Lifecycle

Each plugin is a binary file that provides certain command line parameters. The different ways of invoking the plugin binary using command line parameters form the plugin lifecycle.

Expand All @@ -25,7 +26,7 @@ This is typically implemented by first checking if an action has already been ap

The SDK provides a lot of helper functions to achieve this without overhead.

## Creation of a new node
### Creation of a new node

Whenever a user runs: `bpm run <plugin>`, bpm internally calls the plugin with the following parameters:

Expand All @@ -45,20 +46,32 @@ Starts the node. The current version of bpm supports docker so this will start o

The last step is to get the version of the plugin. `bpm` will write this version to `~/.blockdaemon/nodes/<node-id>/version`. This version can be used when doing an upgrade to know which migrations to run.

## Removal of a node
### Removal of a node

Whenever a user runs: `bpm remove <plugin>`, bpm internally calls the plugin with the following parameters:
Whenever a user runs: `bpm stop <plugin>`, bpm internally calls the plugin with the following parameters:

plugin-binary remove <node-id>
plugin-binary stop <node-id>

This will stop all node processes and remove all active resources like docker containers. It will not remove any data or configuration. To achieve this, the user
needs to run `bpm remove <plugin> --purge`. The `--purge` parameter will be passed through to the plugin like this:
needs to run `bpm stop <plugin> --purge`. The `--purge` parameter will be passed through to the plugin like this:

plugin-binary remove <node-id> --purge
plugin-binary stop <node-id> --purge

In this case the assumption is that the plugin also removes data and configuration files. It should not remove any secrets. The reason is that everything else can be re-created but secrets cannot.

## Upgrade
### Status

Whenever a user runs: `bpm status`, bpm internally calls the plugin with the following parameters:

plugin-binary status <node-id>

This will return either:

- `running` if the node is currently running
- `stopped` if the node is currently stopped
- `incomplete` if parts of the node (e.g. only some containers) are running

### Upgrade

Whenever a user runs: `bpm upgrade <plugin>`, bpm internally calls the plugin with the following parameters:

Expand All @@ -68,200 +81,161 @@ This will take all the necessary actions to upgrade a node. This could include s

After the upgrade `bpm` will write the current plugin version to `~/.blockdaemon/nodes/<node-id>/version`. This version can be used when doing another upgrade to know which migrations to run.

# Plugin SDK
## Plugin SDK

## docker
### docker

Package docker provides a simple docker abstraction layer that makes it very easy to start and remove docker containers, networks and volumes.

For more information please refer to the [API documentation](./docs/docker.md).

## node
### node

Package node provides an easy way to access node related information.

For more information please refer to the [API documentation](./docs/node.md).

## template
### template

Package template implements functions to render Go templates to files using the node.Node struct as an imnput for the templates.

For more information please refer to the [API documentation](./docs/template.md).

## plugin
### plugin

Package plugin provides an easy way to create the required CLI for a plugin. It abstracts away all the command line and file parsing so users just need to implement the actual logic.

For more information please refer to [Implementing a plugin using the Go SDL](#implementing-a-plugin-using-the-go-sdk) and the [API documentation](./docs/plugin.md).

# Implementing a plugin using the Go SDK
## Implementing a plugin using the Go SDK

The easiest way to get started is to copy the example below and start implementing the individual functions. Have a look at existing plugins for inspiration.

## Example with defaults


package main

import (
"github.com/Blockdaemon/bpm-sdk/pkg/plugin"
"github.com/Blockdaemon/bpm-sdk/pkg/node"

"fmt"
)

var pluginVersion string

func start(currentNode node.Node) error {
fmt.Println("Nothing to do here, skipping start")
return nil
}

func main() {
plugin.Initialize(plugin.Plugin{
Name: "empty",
Description: "A plugin that does nothing",
Version: pluginVersion,
Start: start,
CreateSecrets: plugin.DefaultCreateSecrets,
CreateConfigs: plugin.DefaultCreateConfigs
Remove: plugin.DefaultRemove,
Upgrade: plugin.DefaultUpgrade,
})
}

## Full Example

package main

import (
"github.com/Blockdaemon/bpm-sdk/pkg/plugin"
"github.com/Blockdaemon/bpm-sdk/pkg/node"

"fmt"
)

var pluginVersion string

func createSecrets(currentNode node.Node) error {
fmt.Println("Nothing to do here, skipping create-secrets")
return nil
}

func createConfigs(currentNode node.Node) error {
fmt.Println("Nothing to do here, skipping create-configurations")
return nil
}

func start(currentNode node.Node) error {
fmt.Println("Nothing to do here, skipping start")
return nil
}


func remove(currentNode node.Node, purge bool) error {
fmt.Println("Nothing to do here, skipping remove")
return nil
}

func upgrade(currentNode node.Node) error {
fmt.Println("Nothing to do here, skipping upgrade")
return nil
}

func main() {
plugin.Initialize(plugin.Plugin{
Name: "empty",
Description: "A plugin that does nothing",
Version: pluginVersion,
Start: start,
CreateSecrets: createSecrets,
CreateConfigs: createConfigs,
Remove: remove,
Upgrade: upgrade,
})
}


## Tipps & Tricks

### Automatically setting the version
### Example with defaults

```go
package main

import (
"fmt"

"github.com/Blockdaemon/bpm-sdk/pkg/node"
"github.com/Blockdaemon/bpm-sdk/pkg/plugin"
)

var version string

func start(currentNode node.Node) error {
fmt.Println("Nothing to do here, skipping start")
return nil
}

func status(currentNode node.Node) (string, error) {
return "stopped", nil
}

func main() {
plugin.Initialize(plugin.Plugin{
Name: "empty",
Description: "A plugin that does nothing",
Version: version,
Start: start,
Status: status,
CreateSecrets: plugin.DefaultCreateSecrets,
CreateConfigs: plugin.DefaultCreateConfigs
Stop: plugin.DefaultStop,
Upgrade: plugin.DefaultUpgrade,
})
}
```

### Full Example

```go
package main

import (
"fmt"

"github.com/Blockdaemon/bpm-sdk/pkg/node"
"github.com/Blockdaemon/bpm-sdk/pkg/plugin"
)

var version string

func createSecrets(currentNode node.Node) error {
fmt.Println("Nothing to do here, skipping create-secrets")
return nil
}

func createConfigs(currentNode node.Node) error {
fmt.Println("Nothing to do here, skipping create-configurations")
return nil
}

func start(currentNode node.Node) error {
fmt.Println("Nothing to do here, skipping start")
return nil
}

func status(currentNode node.Node) (string, error) {
return "stopped", nil
}

func stop(currentNode node.Node, purge bool) error {
fmt.Println("Nothing to do here, skipping remove")
return nil
}

func upgrade(currentNode node.Node) error {
fmt.Println("Nothing to do here, skipping upgrade")
return nil
}

func main() {
plugin.Initialize(plugin.Plugin{
Name: "empty",
Description: "A plugin that does nothing",
Version: version,
Start: start,
Status: status,
CreateSecrets: createSecrets,
CreateConfigs: createConfigs,
Stop: stop,
Upgrade: upgrade,
})
}
```

### Tipps & Tricks

#### Automatically setting the version

Go has the capability to overwrite variables at compile time like this:

go build -ldflags "-X main.pluginVersion=$VERSION" -o binary cmd/main.go
```bash
go build -ldflags "-X main.version=$VERSION" -o plugin-name cmd/main.go
```

This can be used in a continuous integration pipeline to automatically version the binaries with e.g. the current git tag.

# Nodestate
## Nodestate

Every plugin should run nodestate next to the blockchain client. Nodestate sends monitoring information back to Blockdaemon so we can ensure the node is in perfect shape! Adapting nodestate to BPM is currently a work in progress. We'll update this section soon.

# Deployment
## Deployment

The plugin registry doesn't exist yet. In the meantime we use a digital ocean space to make plugins available. The uploaded files are the raw plugin binaries and filenames need to be in the following format:

<plugin-name>-<version>-<operating-system>-<architecture>
`<plugin-name>-<version>-<operating-system>-<architecture>`

Example:

polkadot-0.5.1-linux-amd64
polkadot-0.5.1-darwin-amd64

In addition to the plugin binaries a single `version-info.json` needs to exist in the space. This file contains information about the *latest* versions. It looks like this:

{
"runner-version": "0.3.0",
"plugins": [
{
"name": "stellar",
"version": "0.4.2"
},
{
"name": "polkadot",
"version": "0.5.2"
}
]
}


When uploading a new plugin, the name and version need to be added to this file.

Important: Make sure the files are public!

# BPM-SDK development

The following section contains information for the development of the BPM-SDK itself.
```
polkadot-0.5.1-linux-amd64
polkadot-0.5.1-darwin-amd64
```

## Continous Integration

A CI pipeline runs on https://gitlab.com/blockdaemon-public/bpm-sdk-ci-pipeline

## Generate SDK documentation

Generating the SDK documentation is currently a bit tricky for different reasons:

* `godoc` doesn't support modules yet (https://github.com/golang/go/issues/26827)
* `godoc -html` does not produce a nice output (https://github.com/golang/go/issues/2381)
* `godoc` doesn't like symbolic links in the `GOROOT`. This breaks it when using homebrew on OSX.

The recommended workarounds to use `wget` result in incorrect links if only the `bpm-sdk` parts of the documentation are exported.

Until the above issues have been resolved, we are using (godocdown)[https://github.com/robertkrimen/godocdown] to generated markdown instead of html. Unfortunately `godocdown` doesn't work with modules yet so we still have to "fake" the go path.

First, install `godocdown`: `go get github.com/robertkrimen/godocdown/godocdown`

Then follow these steps:

1. Change into the `bpm-sdk` directory: `cd bpm-sdk`
2. Create a folder structure that resembles the traditional go directory: `mkdir -p ~/go-mod/src/github.com/Blockdaemon`
3. Symlink `bpm-sdk` into the new directory: `ln -s $(pwd) $HOME/go-mod/src/github.com/Blockdaemon/bpm-sdk`
4. Set `GOPATH` to the new directory: `export GOPATH=$HOME/go-mod/`
5. Export all packages:
```
godocdown github.com/Blockdaemon/bpm-sdk/pkg/docker > docs/docker.md
godocdown github.com/Blockdaemon/bpm-sdk/pkg/node > docs/node.md
godocdown github.com/Blockdaemon/bpm-sdk/pkg/template > docs/template.md
godocdown github.com/Blockdaemon/bpm-sdk/pkg/plugin > docs/plugin.md
```

Loading

0 comments on commit 2e64749

Please sign in to comment.