Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: Release v1.0.0 #133

Merged
merged 349 commits into from
Jan 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
349 commits
Select commit Hold shift + click to select a range
211d299
refactor!: modBase64
jshlbrd Aug 31, 2023
20c4ff0
refactor!: modAWSLambda
jshlbrd Aug 31, 2023
0a0701d
refactor!: modAWSDynamoDB
jshlbrd Aug 31, 2023
07c8906
refactor: metaSwitch
jshlbrd Aug 31, 2023
73a9651
refactor: metaPlugin
jshlbrd Aug 31, 2023
23f7576
refactor!: metaPipeline
jshlbrd Aug 31, 2023
da39187
refactor!: metaForEach
jshlbrd Sep 2, 2023
019aab1
refactor!: Messages
jshlbrd Sep 7, 2023
009462c
refactor!: Object Transforms
jshlbrd Sep 7, 2023
983be00
refactor!: Array Transforms
jshlbrd Sep 8, 2023
221a5e2
refactor!: Strings Transforms
jshlbrd Sep 8, 2023
a5c71a4
refactor!: Time Transforms
jshlbrd Sep 8, 2023
9f4c0be
refactor!: Number Transforms
jshlbrd Sep 8, 2023
718aabb
test: Time Tests
jshlbrd Sep 9, 2023
0f4622a
fix!: newObjToInt
jshlbrd Sep 9, 2023
4545e9e
refactor: Send Transforms
jshlbrd Sep 9, 2023
cd5ac5b
refactor!: Format Transforms
jshlbrd Sep 9, 2023
d43d53a
refactor!: PP Transform
jshlbrd Sep 9, 2023
83326c1
refactor!: Compression Transforms
jshlbrd Sep 9, 2023
afb509e
refactor!: Enrichment Transforms
jshlbrd Sep 9, 2023
85cdeb7
refactor!: External Transforms
jshlbrd Sep 9, 2023
9d3aa60
refactor!: Hash Transforms
jshlbrd Sep 9, 2023
6dab51b
refactor!: Aggregate Transforms
jshlbrd Sep 10, 2023
32f3b9e
refactor: Transform Variables
jshlbrd Sep 10, 2023
de089a8
fix: newSendAWSSQS
jshlbrd Sep 10, 2023
e70d138
refactor: Func Order
jshlbrd Sep 10, 2023
69566cc
refactor: Func Order
jshlbrd Sep 10, 2023
a4b4903
refactor!: Meta Transforms
jshlbrd Sep 11, 2023
69d8dd8
refactor!: Remove Close()
jshlbrd Sep 11, 2023
bbff061
refactor!: NetworkFQDN to NetworkDomain
jshlbrd Sep 12, 2023
cb674ab
refactor!: Conditions
jshlbrd Sep 12, 2023
d4268c8
chore: DDB ReadConsistency
jshlbrd Sep 12, 2023
449a9cb
feat: iconfig
jshlbrd Sep 12, 2023
0d37cc7
refactor!: Remove File Suffix
jshlbrd Sep 12, 2023
82ec72a
chore: DDB Read Consistency
jshlbrd Sep 12, 2023
6edd6b0
refactor: cmds
jshlbrd Sep 12, 2023
1231a82
test: Fix Tests
jshlbrd Sep 12, 2023
4c9934b
refactor: internal/aws pkg
jshlbrd Sep 12, 2023
738e049
chore: linting
jshlbrd Sep 12, 2023
bdf0a66
chore: remove lint
jshlbrd Sep 12, 2023
3555874
build: containers
jshlbrd Sep 12, 2023
ed497bb
build: go mod
jshlbrd Sep 12, 2023
2e32710
refactor!: Retry.Count
jshlbrd Sep 12, 2023
a4ddc07
refactor!: DNS Transforms
jshlbrd Sep 12, 2023
3f65324
refactor!: String Functions
jshlbrd Sep 12, 2023
d6a92a6
refactor!: Condition Objects
jshlbrd Sep 12, 2023
704abb3
refactor!: Transforms
jshlbrd Sep 12, 2023
33371a8
build: config/libsonnet
jshlbrd Sep 12, 2023
4875fc4
Merge branch 'release/v1' of github.com:brexhq/substation into releas…
jshlbrd Sep 12, 2023
d12315c
refactor: Layout
jshlbrd Sep 13, 2023
024e3e3
test: Fix Tests
jshlbrd Sep 13, 2023
b58ca02
feat: MetaErr Transform
jshlbrd Sep 13, 2023
ebd7796
refactor!: Remove MetaPlugin Transform
jshlbrd Sep 14, 2023
fee37b5
test: Example Transform
jshlbrd Sep 14, 2023
51baebe
test: Quickstart Config
jshlbrd Sep 14, 2023
a3ec9d4
refactor!: Network IP Inspectors
jshlbrd Sep 14, 2023
1d79f55
refactor!: Buffer Config
jshlbrd Sep 14, 2023
9063401
refactor: JSON
jshlbrd Sep 14, 2023
27118f1
build: libsonnet
jshlbrd Sep 14, 2023
f88a474
docs: examples/aws/pubsub
jshlbrd Sep 14, 2023
f2f3d65
test: libsonnet
jshlbrd Sep 16, 2023
b3dee03
ci: Jsonnet
jshlbrd Sep 16, 2023
2871529
refactor: devcontainer
jshlbrd Sep 16, 2023
d14d2db
docs: Examples
jshlbrd Sep 16, 2023
b3a1b78
refactor!: build/terraform
jshlbrd Sep 17, 2023
151f06e
chore: terraform install
jshlbrd Sep 17, 2023
d1385b8
fix: enrichDNS Transform
jshlbrd Sep 17, 2023
d154a71
docs: Microservice Example
jshlbrd Sep 17, 2023
4ca4142
build: Terraform Modules
jshlbrd Sep 17, 2023
c604a50
ci: jsonnet compile
jshlbrd Sep 17, 2023
6edc836
docs: Microservice URI
jshlbrd Sep 17, 2023
9922f7d
fix: errgroup bug
jshlbrd Sep 18, 2023
d997916
docs: pubsub
jshlbrd Sep 18, 2023
ee450eb
docs: pubsub
jshlbrd Sep 18, 2023
dd45521
fix: snsPublish
jshlbrd Sep 18, 2023
51bb8e2
refactor: sendAWSSNS
jshlbrd Sep 18, 2023
e1ddf30
fix: Terraform Module IAM Policies
jshlbrd Sep 18, 2023
6fd87c5
build: Go upgrades
jshlbrd Sep 18, 2023
f23dd49
style: lint
jshlbrd Sep 18, 2023
b7bdae6
fix: Terraform Modules
jshlbrd Sep 18, 2023
68b8618
refactor!: TF SNS, Terraform
jshlbrd Sep 18, 2023
2400758
fix: TF Sids
jshlbrd Sep 19, 2023
031ba33
docs: aws/kinesis/multistream
jshlbrd Sep 19, 2023
2297094
docs: aws README
jshlbrd Sep 19, 2023
57ad0f8
fix: sendKinesis
jshlbrd Sep 19, 2023
f89f9d0
docs: aws/sns
jshlbrd Sep 19, 2023
e5bbea8
docs: aws/sqs Example
jshlbrd Sep 19, 2023
45a093a
refactor: sendDynamoDB
jshlbrd Sep 19, 2023
8806c87
docs: aws/dynamodb/cdc
jshlbrd Sep 19, 2023
7b0cf79
docs: aws/examples
jshlbrd Sep 20, 2023
433c7f8
fix: cmd/dev
jshlbrd Sep 20, 2023
d1ae716
docs: README
jshlbrd Sep 20, 2023
db33d04
docs: README
jshlbrd Sep 20, 2023
d6dc66d
docs: README
jshlbrd Sep 20, 2023
112cfe3
refactor: build/config
jshlbrd Sep 21, 2023
dfb0452
feat: cmd/firehose
jshlbrd Sep 21, 2023
0fd9677
docs: examples
jshlbrd Sep 23, 2023
a3fbba9
refactor!: playground
jshlbrd Sep 23, 2023
b2ee811
build: devcontainer
jshlbrd Sep 23, 2023
7b77f2f
refactor: build/container
jshlbrd Sep 23, 2023
d02ccf9
refactor!: cmd/
jshlbrd Sep 23, 2023
39f0e3b
docs
jshlbrd Sep 23, 2023
1804a6b
build: format scripts
jshlbrd Sep 23, 2023
18ae948
refactor: internal/bufio
jshlbrd Sep 23, 2023
08579a4
style: imports
jshlbrd Sep 23, 2023
6a00fde
build: local env
jshlbrd Sep 23, 2023
64591ed
feat!: Kinesis, UUID, AWS
jshlbrd Sep 24, 2023
2a9e4c0
docs: README
jshlbrd Sep 24, 2023
54c3c81
refactor!: Number Transform
jshlbrd Sep 24, 2023
f9ae9eb
refactor!: Compress Transform
jshlbrd Sep 24, 2023
069ee95
build: wasm
jshlbrd Sep 24, 2023
ed84158
test: Transforms
jshlbrd Sep 24, 2023
aff7e0b
refactor!: Number Inspectors
jshlbrd Sep 24, 2023
bd31b6f
refactor: Transform
jshlbrd Sep 24, 2023
1a01e21
build: config
jshlbrd Sep 24, 2023
74a2ece
Merge branch 'main' into release/v1
jshlbrd Sep 24, 2023
baaff9c
refactor: jq
jshlbrd Sep 25, 2023
754a1f9
refactor: condition num/bitwise
jshlbrd Sep 25, 2023
b3dbf4d
style: fmt
jshlbrd Sep 25, 2023
4dfb0c9
docs: CONTRIBUTING
jshlbrd Sep 25, 2023
a373d4d
chore: lint
jshlbrd Sep 25, 2023
10db162
refactor: function names
jshlbrd Sep 25, 2023
b62f537
refactor!: string pattern
jshlbrd Sep 25, 2023
1c7fcd5
chore: wasm
jshlbrd Sep 25, 2023
8e89c07
refactor: condition
jshlbrd Sep 26, 2023
89d875a
refactor!: math transform
jshlbrd Sep 26, 2023
611bc2c
build: configs
jshlbrd Sep 26, 2023
c3058f1
fix: enrichDDB
jshlbrd Oct 1, 2023
160289f
docs: fields
jshlbrd Oct 1, 2023
3db8419
refactor!: sendHTTP
jshlbrd Oct 1, 2023
e390bef
refactor: numMath
jshlbrd Oct 1, 2023
31c10ad
refactor!: kvstoreSet
jshlbrd Oct 1, 2023
3829c8c
build: config, gitignore
jshlbrd Oct 1, 2023
1e4f2bf
test: examples
jshlbrd Oct 1, 2023
4503431
refactor!: secrets
jshlbrd Oct 4, 2023
b674f95
docs: pkg/transform example
jshlbrd Oct 4, 2023
271e775
feat: ignore empty values
jshlbrd Oct 4, 2023
24ccb49
docs: tests
jshlbrd Oct 4, 2023
e9878fb
refactor: tf buffers
jshlbrd Oct 4, 2023
9e2b1fe
refactor!: ttl_offset string
jshlbrd Oct 5, 2023
4fd4f42
refactor!: kvStoreSet, DDB KV
jshlbrd Oct 5, 2023
e581b37
docs: format
jshlbrd Oct 5, 2023
e80811c
docs: message
jshlbrd Oct 5, 2023
2ccd4d2
docs: package
jshlbrd Oct 5, 2023
5f91dad
refactor!: metrics
jshlbrd Oct 6, 2023
798c002
refactor: transform/metrics
jshlbrd Oct 7, 2023
ebd3784
ci: fixes
jshlbrd Oct 7, 2023
cf71655
refactor: cmd, docs
jshlbrd Oct 7, 2023
5d71bc5
feat: aws/ecr force delete
jshlbrd Oct 7, 2023
a054286
docs: ddb/cdc
jshlbrd Oct 7, 2023
5fe3bb5
docs: examples/aws
jshlbrd Oct 7, 2023
4ad25dc
ci: v1
jshlbrd Oct 7, 2023
da259f2
feat!: ns precision
jshlbrd Oct 9, 2023
3bae1c5
refactor: bufferConfig
jshlbrd Oct 10, 2023
1e97afc
build: Go flags
jshlbrd Oct 10, 2023
adb2ddb
ci: gitignore
jshlbrd Oct 10, 2023
df0a202
build: tf/ddb ttl
jshlbrd Oct 15, 2023
6ebfd55
style: examples tf fmt
jshlbrd Oct 15, 2023
394631f
build: obj abbv
jshlbrd Oct 15, 2023
f8f3b59
fix: tf/ddb JSON
jshlbrd Oct 16, 2023
e6af2c8
refactor: tf msg err
jshlbrd Oct 16, 2023
e9e2242
build: sub libsonnet
jshlbrd Oct 16, 2023
d86370a
style: tf
jshlbrd Oct 16, 2023
ee69d87
fix: buffer concurrency bug
jshlbrd Oct 18, 2023
2865883
build: default settings
jshlbrd Oct 18, 2023
3342f64
test: agg/sample example
jshlbrd Oct 18, 2023
dcee46b
fix: tf metaErr
jshlbrd Oct 25, 2023
b36f46b
feat: aws/lambda vpc example
jshlbrd Oct 27, 2023
4e5428b
build: aws/networking/vpc
jshlbrd Oct 27, 2023
8c2f188
build: aws/lambda secret
jshlbrd Oct 27, 2023
04e99a0
docs: fmt
jshlbrd Oct 27, 2023
c73e2b8
perf: remove sjson opts
jshlbrd Nov 1, 2023
49d5c4b
perf: lambda async sources
jshlbrd Nov 1, 2023
3beca99
style: unused linter
jshlbrd Nov 1, 2023
862fe27
docs: transform/array/group example
jshlbrd Nov 7, 2023
e4838cf
feat: condition/meta_for_each JSON Arrays
jshlbrd Nov 7, 2023
d369e37
docs: fix if_all_else example
jshlbrd Nov 7, 2023
289351a
refactor!: Simplify App Transform
jshlbrd Nov 15, 2023
ca08328
feat: Go 1.20
jshlbrd Nov 15, 2023
e2b127c
refactor: Valid JSON
jshlbrd Nov 15, 2023
19f54f7
docs: markdown
jshlbrd Nov 15, 2023
63d836a
doc: Add Missing Component
jshlbrd Nov 15, 2023
a8de0b8
docs: README
jshlbrd Nov 28, 2023
c4790a3
docs: README
jshlbrd Nov 28, 2023
bcf5652
docs: README
jshlbrd Nov 28, 2023
7d575a4
docs: README
jshlbrd Nov 29, 2023
1227524
docs: README
jshlbrd Nov 29, 2023
dbe3150
fix: Floats in number_math Transforms
jshlbrd Dec 2, 2023
b0540c4
docs: Move Examples
jshlbrd Dec 2, 2023
220ba2c
refactor!: Remove cmd/client/file
jshlbrd Dec 2, 2023
5829077
docs: Remove README
jshlbrd Dec 2, 2023
2667ec4
docs: Example Lib Import
jshlbrd Dec 3, 2023
a57be27
test: condition/format_json
jshlbrd Dec 3, 2023
d6c12dc
docs: examples/build/terraform
jshlbrd Dec 4, 2023
4657304
docs: S3/SNS Example
jshlbrd Dec 4, 2023
ceea9ce
fix: rand.Seed
jshlbrd Dec 9, 2023
b927755
docs: Kinesis example
jshlbrd Dec 9, 2023
c7b1289
docs: Retry with Backoff example
jshlbrd Dec 9, 2023
dbeacc2
docs: Crash Program example
jshlbrd Dec 9, 2023
9035b0e
docs: If All Else example
jshlbrd Dec 9, 2023
59b0c8f
docs: README
jshlbrd Dec 9, 2023
c7b7b6e
docs: typo
jshlbrd Dec 9, 2023
b2efbd5
docs: Retry with Backoff
jshlbrd Dec 9, 2023
25fd509
style: fmt
jshlbrd Dec 9, 2023
b6341ce
refactor!: Switch to Cases (Copilot)
jshlbrd Dec 11, 2023
4789ca9
refactor!: StringCapture (Copilot)
jshlbrd Dec 11, 2023
e93f774
refactor!: StringReplace (Copilot)
jshlbrd Dec 11, 2023
6be11bf
refactor!: StringAppend (Copilot)
jshlbrd Dec 11, 2023
ef2e82d
refactor!: condition/string (Copilot)
jshlbrd Dec 11, 2023
f9a4a84
refactor!: condition/num_length (Copilot)
jshlbrd Dec 11, 2023
d731514
refactor!: condition/num_bitwise (Copilot)
jshlbrd Dec 11, 2023
e1c504d
refactor!: AWS Role ARN (Copilot)
jshlbrd Dec 15, 2023
58b1f7a
refactor!: transform/jq (Copilot)
jshlbrd Dec 15, 2023
06cf72c
refactor: int/agg Default Duration
jshlbrd Dec 16, 2023
6db6fbb
refactor: int/secrets Max Capacity
jshlbrd Dec 16, 2023
9cc0d04
build: Add required versions and providers for Terraform modules
jshlbrd Dec 16, 2023
084c6f0
build: Refactor Lambda, add Secret Terraform modules
jshlbrd Dec 16, 2023
2019f95
build: Standardize Terraform role name
jshlbrd Dec 16, 2023
a8d1dbf
build(terraform):Add Multi-Region Support
jshlbrd Dec 18, 2023
81cc020
build(terraform): Add CloudWatch Modules
jshlbrd Dec 18, 2023
52f1784
build(config): Update libsonnet
jshlbrd Dec 18, 2023
82aec20
build(config): Update Defaults
jshlbrd Dec 18, 2023
e7eb784
fix(transform): metaSwitch Struct
jshlbrd Dec 18, 2023
5fc2a30
fix(condition): metaForEach Missing Value
jshlbrd Dec 18, 2023
4d0b5f6
test: metaSwitch
jshlbrd Dec 18, 2023
1c19d15
build(terraform): Randomized IAM Names
jshlbrd Dec 19, 2023
976e483
refactor!: Object Config
jshlbrd Dec 28, 2023
8266ae7
refactor(transform): String Capture
jshlbrd Dec 28, 2023
b0953a1
refactor(transform): Logic Complexity
jshlbrd Dec 28, 2023
39f20e4
chore(transform): Remove Sumo Logic
jshlbrd Dec 28, 2023
3afcf02
refactor(transform): arrayJoin, Examples
jshlbrd Dec 29, 2023
57ab48c
examples(terraform/aws): Kinesis Docs
jshlbrd Dec 30, 2023
be6fbe6
chore(devcontainer): Default Memory
jshlbrd Dec 30, 2023
493de83
refactor!(transform): Array Group -> Zip
jshlbrd Dec 30, 2023
46af879
examples(transform): arrayZip
jshlbrd Dec 31, 2023
4232920
refactor: Object Keys, Batching, Send Transforms
jshlbrd Jan 3, 2024
9741ae5
docs(examples): Kinesis/Multistream
jshlbrd Jan 3, 2024
80e6933
fix(transform): Send AWS S3
jshlbrd Jan 3, 2024
38941c9
build(terraform): DynamoDB
jshlbrd Jan 3, 2024
17962cb
docs(examples): Terraform KMS Policies
jshlbrd Jan 4, 2024
66b19dd
docs: README
jshlbrd Jan 4, 2024
7219a98
fix(transform): metaForEach
jshlbrd Jan 4, 2024
ab5c5f7
docs(examples): array group
jshlbrd Jan 4, 2024
b5208a7
docs: Public API
jshlbrd Jan 6, 2024
9d64f2f
docs(examples): Config
jshlbrd Jan 6, 2024
00829e8
docs: README
jshlbrd Jan 6, 2024
68790dc
chore(transform): Remove arrayToObject
jshlbrd Jan 6, 2024
a281324
docs(examples): README
jshlbrd Jan 6, 2024
6659b79
docs(examples): README
jshlbrd Jan 6, 2024
3471e5f
docs: README
jshlbrd Jan 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
25 changes: 9 additions & 16 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,21 +1,14 @@
FROM mcr.microsoft.com/vscode/devcontainers/go:1.19
FROM mcr.microsoft.com/vscode/devcontainers/go:1.20
RUN apt-get update -y && \
# install Terraform
cd /tmp/ && \
curl https://releases.hashicorp.com/terraform/1.1.7/terraform_1.1.7_linux_$(dpkg --print-architecture).zip -o ./terraform.zip && \
unzip terraform.zip && \
mv terraform /usr/local/bin/terraform && \
# install Jsonnet
# https://developer.hashicorp.com/terraform/downloads
wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg && \
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/hashicorp.list && \
apt update -y && apt install -y terraform && \
# go-jsonnet
go install github.com/google/go-jsonnet/cmd/jsonnet@latest && \
go install github.com/google/go-jsonnet/cmd/jsonnetfmt@latest && \
go install github.com/google/go-jsonnet/cmd/jsonnet-lint@latest && \
# install AWS CLI
# AWS CLI
apt-get install -y awscli && \
# install pip and boto3
apt-get install -y python3-pip && pip3 install boto3 && pip3 install black && \
# install protobuf compiler tools
# https://grpc.io/docs/languages/go/quickstart/#prerequisites
apt-get install -y protobuf-compiler && \
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest && \
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest && \
export PATH="$PATH:$(go env GOPATH)/bin"
# python
apt-get install -y python3 python3-boto3 black
20 changes: 12 additions & 8 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,19 @@
"dockerfile": "Dockerfile"
},
"remoteUser": "root",
"extensions": [
"golang.go",
"hashicorp.terraform",
"zxh404.vscode-proto3",
"grafana.vscode-jsonnet"
],
"runArgs": [
"--memory=1g",
"--memory=2g",
"--cpus=2"
],
"postCreateCommand": "bash .devcontainer/install-dev-tools.sh"
"postCreateCommand": "bash .devcontainer/install-dev-tools.sh",
"customizations": {
"vscode": {
"extensions": [
"ms-vscode.go",
"hashicorp.terraform",
"zxh404.vscode-proto3",
"grafana.vscode-jsonnet"
]
}
}
}
2 changes: 0 additions & 2 deletions .devcontainer/install-dev-tools.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
# download go packages
go mod download
# compile protobuf
sh build/scripts/proto/compile.sh
# compile configurations
sh build/scripts/config/compile.sh
Binary file added .github/media/substation_architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
19 changes: 15 additions & 4 deletions .github/workflows/code.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,20 @@ jobs:
pip3 install black
find -name *.py | xargs black --check

proto:
jsonnet:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: bufbuild/buf-setup-action@v1
- uses: bufbuild/buf-lint-action@v1
- name: checkout
uses: actions/checkout@v2
with:
fetch-depth: 1

- name: setup
uses: actions/setup-go@v2
with:
go-version: 1.19

- name: compile
run: |
go install github.com/google/go-jsonnet/cmd/jsonnet@latest
sh build/scripts/config/compile.sh
3 changes: 0 additions & 3 deletions .github/workflows/release_please.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,3 @@ jobs:
with:
release-type: go
package-name: release-please-action
# beta changes are v0.x
bump-minor-pre-major: true
bump-patch-for-minor-pre-major: true
14 changes: 5 additions & 9 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,15 @@ CHANGELOG.md

# ignore all JSON by default
*.json
*.jsonl
# allow Visual Studio Code devcontainer config file
!.devcontainer/devcontainer.json
# allow Visual Studio Code settings config file
!.vscode/settings.json
# allow static JSON files in the examples/ directory
!examples/condition/data.json
!examples/*/data.json
!examples/quickstart/data.json
# ignore playground files
playground.wasm
playground.wasm.gz
wasm_exec.js
build/playground/substation_logo.png
# allow specific JSON files in the examples/ directory
!examples/**/data*.json
!examples/**/data*.jsonl
!examples/**/stdout.jsonl

# Go profiling files
*.prof
Expand Down
22 changes: 10 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,13 +55,11 @@ We rely on contributors to test changes before they are submitted as pull reques

#### Environment Variables

Substation uses environment variables to customize runtime settings of the system (e.g., concurrency is controlled by `SUBSTATION_CONCURRENCY`).

Custom applications should implement their own runtime settings as required; for example, the [AWS Lambda application](/cmd/aws/lambda/substation/) uses `SUBSTATION_HANDLER` to manage [invocation settings](https://docs.aws.amazon.com/lambda/latest/dg/lambda-invocation.html).
Applications may implement runtime settings that are managed by environment variables. For example, the [AWS Lambda application](/cmd/aws/lambda/substation/) uses `SUBSTATION_HANDLER` to manage [invocation settings](https://docs.aws.amazon.com/lambda/latest/dg/lambda-invocation.html).

#### Configurations

Substation uses a single configuration pattern for all components in the system (see `Config` in [config/config.go](/config/config.go)). This pattern is highly reusable and should be nested to create complex configurations supported by Jsonnet. Below is an example that shows how configurations should be designed:
Substation uses a single configuration pattern for all components in the system (see `Config` in [config/config.go](/config/config.go)). This pattern is highly reusable and should be embedded to create custom configurations. Below is an example that shows how configurations should be designed:

```json
"foo": {
Expand All @@ -81,13 +79,13 @@ Substation uses a single configuration pattern for all components in the system
}
```

Repeating this pattern allows components and applications to integrate with the factory pattern.
Repeating this pattern allows components and applications to integrate with Substation's factory patterns.

#### Factories

Substation relies on [factory methods](https://refactoring.guru/design-patterns/factory-method) to create objects that [satisfy interfaces](https://go.dev/doc/effective_go#interface_methods) across the system. Factories should be combined with the configuration design pattern to create pre-configured, ready-to-use objects.
Substation relies on [factory methods](https://refactoring.guru/design-patterns/factory-method) to create objects that [satisfy interfaces](https://go.dev/doc/effective_go#interface_methods) across the project. Factories should be combined with the configuration design pattern to create new components.

Factories are the preferred method for allowing users to customize the system. Example factories can be seen in [condition](/condition/condition.go) and [process](/process/process.go).
Factories are the preferred method for allowing users to customize the system. Example factories can be seen in [condition](/condition/condition.go) and [transform](/transform/transform.go).

#### Reading and Writing Streaming Data

Expand All @@ -103,9 +101,9 @@ Substation commonly uses these io compatible containers:

#### Errors

Errors should always start with `err` (or `Err`, if they are public) and be defined as constants using [internal/errors](/internal/errors/errors.go).
Errors should always start with `err` (or `Err`, if they are public). Commonly used errors are defined in [internal/errors.go](internal/errors.go).

If the error is related to an object created by an interface factory, then the object should be referenced by name in the error. For example, if the factory returns objects named `Foo`, `Bar`, and `Baz`, then related errors should be `errFooShortDescription`, `errBarShortDescription`, and `errBazShortDescription`.
If the error is related to a specific component, then the component name should be included in the error. For example, if the error is related to the `Foo` component, then the error should be named `errFooShortDescription`.

#### Environment Variables

Expand All @@ -115,15 +113,15 @@ Any environment variable that changes a default runtime setting should always st

#### Application Variables

Variable names should always follow conventions from [Effective Go](https://go.dev/doc/effective_go#names), the [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments#variable-names) and avoid [predeclared identifiers](https://go.dev/ref/spec#Predeclared_identifiers). For example `capsule` is used instead of `cap` to avoid shadowing the capacity function while modifiers and plural usage are `newCapsule` and `capsules`.
Variable names should always follow conventions from [Effective Go](https://go.dev/doc/effective_go#names), the [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments#variable-names) and avoid [predeclared identifiers](https://go.dev/ref/spec#Predeclared_identifiers).

#### Source Metadata

Sources that [add metadata during encapsulation](/config/README.md) should use lowerCamelCase for their JSON keys.
Sources that [add metadata during message creation](/message/) should use lowerCamelCase for their JSON keys.

#### Package Configurations

Configurations for packages (for example, [process](/process/README.md) and [condition](/condition/README.md)) should always use lower_snake_case in their JSON keys. This helps maintain readability when reviewing large configuration files.
Configurations for packages (for example, [condition](/condition/README.md) and [transform](/transform/README.md)) should always use lower_snake_case in their JSON keys. This helps maintain readability when reviewing large configuration files.

We strongly urge everyone to use Jsonnet for managing configurations.

Expand Down
Loading
Loading