Skip to content

Commit

Permalink
Adds tools linter.
Browse files Browse the repository at this point in the history
Signed-off-by: dblock <[email protected]>
  • Loading branch information
dblock committed Apr 22, 2024
1 parent 61f8a38 commit 614f865
Show file tree
Hide file tree
Showing 10 changed files with 3,482 additions and 534 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,5 @@ jobs:
with:
node-version: 20.10.0
- run: npm install
- run: npm run lint -- ../spec
- run: |
npm run lint:spec -- ../spec
1 change: 1 addition & 0 deletions .github/workflows/tools.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,4 @@ jobs:
node-version: 20.10.0
- run: npm install
- run: npm run test
- run: npm run lint
13 changes: 11 additions & 2 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ spec
└── opensearch-openapi.yaml
```

Every `.yaml` file is a valid OpenAPI 3 document. This means that you can use any OpenAPI 3 compatible tool to view and edit the files, and IDEs with OpenAPI support will provide you with autocompletion and validation in real-time.

## Grouping Operations
Expand Down Expand Up @@ -84,6 +85,14 @@ This repository includes several penAPI Specification Extensions to fill in any
- `x-global`: Denotes that the parameter is a global parameter that is included in every operation. These parameters are listed in the [root file](spec/opensearch-openapi.yaml).
- `x-default`: Contains the default value of a parameter. This is often used to override the default value specified in the schema, or to avoid accidentally changing the default value when updating a shared schema.

## Linting
We have a linter that validates every `yaml` file in the `./spec` folder to assure that they follow the guidelines we have set. Check out the [Linter](tools/README.md#linter) tool for more information on how to run it locally. Make sure to run the linter before submitting a PR.
## Tools

We authored a number of tools to merge and lint specs that live in [tools](tools/). All tools have tests (run with `npm run test`) and a linter (run with `npm run lint`).

### Merger

The spec merger "builds", aka combines various `.yaml` files into a complete OpenAPI spec. A [workflow](./.github/workflows/build.yml) publishes the output into [releases](https://github.com/opensearch-project/opensearch-api-specification/releases).

### Linter

The spec linter that validates every `.yaml` file in the `./spec` folder to assure that they follow the guidelines we have set. Check out the [Linter README](tools/README.md#linter) for more information on how to run it locally. Make sure to run the linter before submitting a PR.
21 changes: 15 additions & 6 deletions tools/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,34 @@
# OpenSearch OpenAPI Tools

This folder contains tools for the repo:
- Merger: merges multiple OpenAPI files into one
- Linter: validates files in the spec folder

- [Merger](./merger/): merges multiple OpenAPI files into one
- [Linter](./linter/): validates files in the spec folder

## Setup

1. Install [Node.js](https://nodejs.org/en/learn/getting-started/how-to-install-nodejs)
2. Run `npm install` in the `tools` folder

## Merger

The merger tool merges the multi-file OpenSearch spec into a single file for programmatic use. It takes 2 parameters:
- The path to the root folder of the multi-file spec
- The path to the output file

- the path to the root folder of the multi-file spec
- the path to the output file

Example:

```bash
npm run merge -- ../spec ../build/opensearch-openapi.latest.yaml
```

## Linter

The linter tool validates the OpenSearch spec files in the `spec` folder:

```bash
npm run lint
npm run lint:spec
```

It will print out all the errors and warnings in the spec files. This tool in still in development, and it will be integrated into the CI/CD pipeline and run automatically with every PR.
```
67 changes: 67 additions & 0 deletions tools/eslint.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import path from 'path'
import { fileURLToPath } from 'url'
import { FlatCompat } from '@eslint/eslintrc'
import pluginJs from '@eslint/js'

// mimic CommonJS variables -- not needed if using CommonJS
const _filename = fileURLToPath(import.meta.url)
const _dirname = path.dirname(_filename)
const compat = new FlatCompat({ baseDirectory: _dirname, recommendedConfig: pluginJs.configs.recommended })

export default [
pluginJs.configs.recommended,
...compat.extends('standard-with-typescript'),
{
files: ['**/*.{js,ts}'],
ignores: [
'**/eslint.config.mjs'
],
rules: {
'@typescript-eslint/array-type': 'warn',
'@typescript-eslint/block-spacing': 'warn',
'@typescript-eslint/comma-dangle': 'warn',
'@typescript-eslint/comma-spacing': 'warn',
'@typescript-eslint/consistent-indexed-object-style': 'warn',
'@typescript-eslint/consistent-type-assertions': 'warn',
'@typescript-eslint/consistent-type-imports': 'warn',
'@typescript-eslint/dot-notation': 'warn',
'@typescript-eslint/explicit-function-return-type': 'warn',
'@typescript-eslint/indent': 'warn',
'@typescript-eslint/keyword-spacing': 'warn',
'@typescript-eslint/lines-between-class-members': 'warn',
'@typescript-eslint/member-delimiter-style': 'warn',
'@typescript-eslint/naming-convention': 'warn',
'@typescript-eslint/no-confusing-void-expression': 'warn',
'@typescript-eslint/no-dynamic-delete': 'warn',
'@typescript-eslint/no-invalid-void-type': 'warn',
'@typescript-eslint/no-non-null-assertion': 'warn',
'@typescript-eslint/no-unnecessary-type-assertion': 'warn',
'@typescript-eslint/no-unsafe-argument': 'warn',
'@typescript-eslint/no-unused-vars': 'warn',
'@typescript-eslint/object-curly-spacing': 'warn',
'@typescript-eslint/prefer-nullish-coalescing': 'warn',
'@typescript-eslint/quotes': 'warn',
'@typescript-eslint/require-array-sort-compare': 'warn',
'@typescript-eslint/semi': 'warn',
'@typescript-eslint/space-before-blocks': 'warn',
'@typescript-eslint/space-before-function-paren': 'warn',
'@typescript-eslint/strict-boolean-expressions': 'warn',
'@typescript-eslint/type-annotation-spacing': 'warn',
'array-bracket-spacing': 'warn',
'array-callback-return': 'warn',
curly: 'warn',
'eol-last': 'warn',
eqeqeq: 'warn',
'new-cap': 'warn',
'no-multi-spaces': 'warn',
'no-multiple-empty-lines': 'warn',
'no-return-assign': 'warn',
'no-useless-return': 'warn',
'object-curly-newline': 'warn',
'object-property-newline': 'warn',
'object-shorthand': 'warn',
'quote-props': 'warn',
'space-in-parens': 'warn'
}
}
]
76 changes: 38 additions & 38 deletions tools/helpers.ts
Original file line number Diff line number Diff line change
@@ -1,47 +1,47 @@
import fs from "fs";
import YAML from "yaml";
import _ from "lodash";
import fs from 'fs'
import YAML from 'yaml'
import _ from 'lodash'

export function resolve(ref: string, root: Record<string, any>) {
const paths = ref.replace('#/', '').split('/');
for(const p of paths) {
root = root[p];
if(root === undefined) break;
}
return root;
export function resolve (ref: string, root: Record<string, any>) {

Check warning on line 5 in tools/helpers.ts

View workflow job for this annotation

GitHub Actions / tools-tests

Missing return type on function
const paths = ref.replace('#/', '').split('/')
for (const p of paths) {
root = root[p]
if (root === undefined) break
}
return root
}

export function sortByKey(obj: Record<string, any>, priorities: string[] = []) {
const orders = _.fromPairs(priorities.map((k, i) => [k, i+1]));
const sorted = _.entries(obj).sort((a,b) => {
const order_a = orders[a[0]];
const order_b = orders[b[0]];
if(order_a && order_b) return order_a - order_b;
if(order_a) return 1;
if(order_b) return -1;
return a[0].localeCompare(b[0]);
});
sorted.forEach(([k, v]) => {
delete obj[k];
obj[k] = v;
});
export function sortByKey (obj: Record<string, any>, priorities: string[] = []) {

Check warning on line 14 in tools/helpers.ts

View workflow job for this annotation

GitHub Actions / tools-tests

Missing return type on function
const orders = _.fromPairs(priorities.map((k, i) => [k, i + 1]))
const sorted = _.entries(obj).sort((a, b) => {
const order_a = orders[a[0]]

Check warning on line 17 in tools/helpers.ts

View workflow job for this annotation

GitHub Actions / tools-tests

Variable name `order_a` must match one of the following formats: camelCase, PascalCase, UPPER_CASE
const order_b = orders[b[0]]

Check warning on line 18 in tools/helpers.ts

View workflow job for this annotation

GitHub Actions / tools-tests

Variable name `order_b` must match one of the following formats: camelCase, PascalCase, UPPER_CASE
if (order_a && order_b) return order_a - order_b

Check warning on line 19 in tools/helpers.ts

View workflow job for this annotation

GitHub Actions / tools-tests

Unexpected number value in conditional. An explicit zero/NaN check is required

Check warning on line 19 in tools/helpers.ts

View workflow job for this annotation

GitHub Actions / tools-tests

Unexpected number value in conditional. An explicit zero/NaN check is required
if (order_a) return 1

Check warning on line 20 in tools/helpers.ts

View workflow job for this annotation

GitHub Actions / tools-tests

Unexpected number value in conditional. An explicit zero/NaN check is required
if (order_b) return -1

Check warning on line 21 in tools/helpers.ts

View workflow job for this annotation

GitHub Actions / tools-tests

Unexpected number value in conditional. An explicit zero/NaN check is required
return a[0].localeCompare(b[0])
})
sorted.forEach(([k, v]) => {
delete obj[k]

Check warning on line 25 in tools/helpers.ts

View workflow job for this annotation

GitHub Actions / tools-tests

Do not delete dynamically computed property keys
obj[k] = v
})
}

export function write2file(file_path: string, content: Record<string, any>): void {
fs.writeFileSync(file_path, quoteRefs(YAML.stringify(removeAnchors(content), {lineWidth: 0, singleQuote: true})));
export function write2file (file_path: string, content: Record<string, any>): void {

Check warning on line 30 in tools/helpers.ts

View workflow job for this annotation

GitHub Actions / tools-tests

Parameter name `file_path` must match one of the following formats: camelCase, PascalCase, UPPER_CASE
fs.writeFileSync(file_path, quoteRefs(YAML.stringify(removeAnchors(content), { lineWidth: 0, singleQuote: true })))
}

function quoteRefs(str: string): string {
return str.split('\n').map((line) => {
if(line.includes('$ref')) {
const [key, value] = line.split(': ');
if(!value.startsWith("'")) line = `${key}: '${value}'`;
}
return line
}).join('\n');
function quoteRefs (str: string): string {
return str.split('\n').map((line) => {
if (line.includes('$ref')) {
const [key, value] = line.split(': ')
if (!value.startsWith("'")) line = `${key}: '${value}'`
}
return line
}).join('\n')
}

function removeAnchors(content: Record<string, any>): Record<string, any> {
const replacer = (key: string, value: any) => key === '$anchor' ? undefined : value;
return JSON.parse(JSON.stringify(content, replacer));
}
function removeAnchors (content: Record<string, any>): Record<string, any> {
const replacer = (key: string, value: any) => key === '$anchor' ? undefined : value
return JSON.parse(JSON.stringify(content, replacer))
}
4 changes: 2 additions & 2 deletions tools/jest.config.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/** @type {import('ts-jest').JestConfigWithTsJest} */
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
};
testEnvironment: 'node'
}
Loading

0 comments on commit 614f865

Please sign in to comment.