Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeScript implementation (@nlighten/json-transform) [WIP] #2

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
18072dc
initial work on typescript library (functions in progress)
elisherer Jul 26, 2024
3e70d56
update readmes
elisherer Jul 26, 2024
e14e7b8
fix document context logic to match Java's impl (JayWay)
elisherer Jul 26, 2024
b6c265d
refactor, use Sequency for streams
elisherer Jul 27, 2024
872e9a8
TS - add coalesce and concat functions
elisherer Jul 29, 2024
aa59ef1
TS - add contains, csv, csvparse
elisherer Aug 1, 2024
85335fa
TS - add contains, csv, csvparse, date, digest & entries
elisherer Aug 4, 2024
31944c7
TS - fixes, add eval, filter and test
elisherer Aug 4, 2024
fb50a59
TS - add find, join and jsonparse (add tests for all including eval)
elisherer Aug 5, 2024
c3fa980
TS - add flat, flatten, form, formparse, isnull
elisherer Aug 5, 2024
80b75a2
TS - add group, if
elisherer Aug 22, 2024
c7a3933
align with java-0.2.5 changes
elisherer Aug 22, 2024
01810de
TS - add jsonpatch, jsonpath and jsonpointer functions
elisherer Aug 28, 2024
e3ebb30
TS - add jwtparse function
elisherer Aug 28, 2024
5923447
TS - add length, long, lookup function
elisherer Sep 2, 2024
0d6a0c8
TS - add map, match, matchall, math, max, min function
elisherer Sep 3, 2024
b069e02
TS - add normalize, not, numberformat, numberparse, object, or, pad, …
elisherer Sep 15, 2024
06895eb
major refactor, move to a test harness (still work in progress)
elisherer Sep 23, 2024
fac034a
add gitignore to test harness project
elisherer Sep 23, 2024
1e4abdb
remove dummy test
elisherer Sep 23, 2024
164dccd
unit tests work
elisherer Nov 13, 2024
06c992c
fix for java playground
elisherer Nov 13, 2024
a767c13
0.5.1 - fix "is" accepting null values
elisherer Jan 11, 2025
491ada4
add more tests, upgrade date-fns to v4
elisherer Jan 16, 2025
a94b62c
add more tests
elisherer Jan 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
33 changes: 33 additions & 0 deletions .github/workflows/javascript-json-transform-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: JavaScript json-transform Test

on:
pull_request:
branches:
- main
paths:
- javascript/json-transform/**

# cancel previous tests if new commit is pushed to PR branch
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 18
cache: npm
cache-dependency-path: ./javascript/json-transform/package-lock.json

- name: Install dependencies
working-directory: ./javascript/json-transform
run: npm ci

- name: Run tests
working-directory: ./javascript/json-transform
run: npm test
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@

## Packages

|Language|Name|Description|License|Status|
|---|---|---|---|---|
|Java|[co.nlighten.json-transform](./java/json-transform)|Java library for transforming JSON objects|Apache License 2.0|![Maven Central Version](https://img.shields.io/maven-central/v/co.nlighten/json-transform)|
|JavaScript|[@nlighten/json-transform-core](./javascript/json-transform-core)|Core types and utilities for handling JSON transformers|MIT|![npm](https://img.shields.io/npm/v/@nlighten/json-transform-core)|
| Language |Name| Description |License|Status|
|------------|---|---------------------------------------------------------|---|---|
| Java |[co.nlighten.json-transform](./java/json-transform)| Java library for transforming JSON objects |Apache License 2.0|![Maven Central Version](https://img.shields.io/maven-central/v/co.nlighten/json-transform)|
| JavaScript |[@nlighten/json-transform](./javascript/json-transform)| JSON transformers JavaScript implementation|MIT|![npm](https://img.shields.io/npm/v/@nlighten/json-transform)|
| JavaScript |[@nlighten/json-transform-core](./javascript/json-transform-core)| Core types and utilities for handling JSON transformers |MIT|![npm](https://img.shields.io/npm/v/@nlighten/json-transform-core)|
20 changes: 10 additions & 10 deletions docs/docs/functions/numberformat.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,16 @@ Formats a number
### Returns
`string`
### Arguments
| Argument | Type | Values | Required / Default Value | Description |
|-----------------|-----------|--------------------------------------------------------------------|-------------------------------|----------------------------------------------------------------------------------------|
| `type` | `Enum` | `NUMBER`/`DECIMAL`/`CURRENCY`/`PERCENT`/`INTEGER`/`COMPACT`/`BASE` | Yes | Type of output format |
| `locale` | `Enum` | A language tag (e.g. `en-US`) | `en-US` | Locale to use (language and country specific formatting; set by Java) |
| `compact_style` | `Enum` | `SHORT`/`LONG` | `SHORT` | Effective when `type == COMPACT`, choose which type of compact |
| `pattern` | `String` | `DecimalFormat` pattern | `#0.00` | See [tutorial](https://docs.oracle.com/javase/tutorial/i18n/format/decimalFormat.html) |
| `grouping` | `String` | Single character string | `,` | A custom character to be used for grouping |
| `decimal` | `String` | Single character string | `.` | A custom character to be used for decimal point |
| `radix` | `Integer` | | `10` | Radix to be used for formatting input |
| `currency` | `String` | [ISO-4217 currency code](https://www.iso.org/iso-4217-currency-codes.html) | Yes (when `type == CURRENCY`) | Currency to use in format |
| Argument | Type | Values | Required / Default Value | Description |
|-----------------|-----------|----------------------------------------------------------------------------|-------------------------------|----------------------------------------------------------------------------------------|
| `type` | `Enum` | `NUMBER`/`DECIMAL`/`CURRENCY`/`PERCENT`/`INTEGER`/`COMPACT`/`BASE` | Yes | Type of output format |
| `locale` | `Enum` | A language tag (e.g. `en-US`) | `en-US` | Locale to use (language and country specific formatting; set by Java) |
| `compact_style` | `Enum` | `SHORT`/`LONG` | `SHORT` | Effective when `type == COMPACT`, choose which type of compact |
| `pattern` | `String` | `DecimalFormat` pattern | `#0.00` | See [tutorial](https://docs.oracle.com/javase/tutorial/i18n/format/decimalFormat.html) |
| `grouping` | `String` | Single character string | `,` | A custom character to be used for grouping |
| `decimal` | `String` | Single character string | `.` | A custom character to be used for decimal point |
| `radix` | `Integer` | | `10` | Radix to be used for formatting input |
| `currency` | `String` | [ISO-4217 currency code](https://www.iso.org/iso-4217-currency-codes.html) | Yes (when `type == CURRENCY`) | Currency to use in format |


## Examples
Expand Down
4 changes: 4 additions & 0 deletions docs/docs/functions/yaml.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

Converts an object to YAML format

:::info
Structure of output may very depending on platform.
:::

### Usage
```transformers
{
Expand Down
15 changes: 15 additions & 0 deletions java.test-server.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
FROM gradle:8.4-jdk17 as build
ARG PORT=10000
WORKDIR /
COPY java/json-transform java/json-transform
COPY java/playground/src java/playground/src
COPY java/playground/build.gradle java/playground
COPY settings.gradle .

WORKDIR /java/playground
RUN gradle task bootJar

FROM eclipse-temurin:17-jdk-jammy as app
COPY --from=build /java/playground/build/libs/playground-1.0.0.jar .
EXPOSE $PORT
CMD ["java", "-jar", "./playground-1.0.0.jar"]
2 changes: 1 addition & 1 deletion java/json-transform/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ plugins {
}

group 'co.nlighten'
version = '0.5.0'
version = '0.5.1'

ext {
gsonVersion = "2.10.1"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,22 +1,13 @@
package co.nlighten.jsontransform.functions;

import co.nlighten.jsontransform.adapters.JsonAdapter;
import co.nlighten.jsontransform.functions.common.ArgType;
import co.nlighten.jsontransform.functions.common.FunctionContext;
import co.nlighten.jsontransform.functions.common.TransformerFunction;
import co.nlighten.jsontransform.functions.annotations.Aliases;
import co.nlighten.jsontransform.functions.annotations.Documentation;
import co.nlighten.jsontransform.functions.annotations.InputType;
import co.nlighten.jsontransform.functions.annotations.OutputType;

/*
* For tests
* @see TransformerFunctionAndTest
*/
@Aliases("and")
@Documentation("Evaluates to `true` if all values provided will evaluate to `true` (using the [Truthy logic]).")
@InputType(value = ArgType.Array, description = "Values to check")
@OutputType(ArgType.Boolean)
public class TransformerFunctionAnd<JE, JA extends Iterable<JE>, JO extends JE> extends TransformerFunction<JE, JA, JO> {
public TransformerFunctionAnd(JsonAdapter<JE, JA, JO> adapter) {
super(adapter);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,13 @@
import co.nlighten.jsontransform.functions.common.ArgType;
import co.nlighten.jsontransform.functions.common.FunctionContext;
import co.nlighten.jsontransform.functions.common.TransformerFunction;
import co.nlighten.jsontransform.functions.annotations.*;
import co.nlighten.jsontransform.functions.annotations.ArgumentType;

/*
* For tests
* @see TransformerFunctionAtTest
*/
@Aliases("at")
@Documentation("Retrieves an element from a specific position inside an input array")
@InputType(value = ArgType.Array, description = "Array to fetch from")
@ArgumentType(value = "index", type = ArgType.Integer, position = 0, required = true, defaultIsNull = true,
description = "Index of element to fetch (negative values will be fetch from the end)")
@OutputType(value = ArgType.Any, description = "Element at index, or null if undefined")
@ArgumentType(value = "index", type = ArgType.Integer, position = 0, defaultIsNull = true)
public class TransformerFunctionAt<JE, JA extends Iterable<JE>, JO extends JE> extends TransformerFunction<JE, JA, JO> {
public TransformerFunctionAt(JsonAdapter<JE, JA, JO> adapter) {
super(adapter);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,14 @@
import co.nlighten.jsontransform.functions.common.FunctionContext;
import co.nlighten.jsontransform.functions.common.FunctionHelpers;
import co.nlighten.jsontransform.functions.common.TransformerFunction;
import co.nlighten.jsontransform.functions.annotations.*;
import co.nlighten.jsontransform.functions.annotations.ArgumentType;

import java.math.BigDecimal;
import java.util.Objects;
import java.util.concurrent.atomic.AtomicInteger;

@Aliases("avg")
@Documentation("Returns the average of all values in the array")
@InputType(ArgType.Array)

@ArgumentType(value = "default", type = ArgType.BigDecimal, position = 0, defaultBigDecimal = 0,
description = "The default value to use for empty values")
@ArgumentType(value = "by", type = ArgType.Transformer, position = 1,
defaultString = "##current",
description = "A transformer to extract a property to sum by (using ##current to refer to the current item)")

@OutputType(ArgType.BigDecimal)
@ArgumentType(value = "default", type = ArgType.BigDecimal, position = 0, defaultBigDecimal = 0)
@ArgumentType(value = "by", type = ArgType.Transformer, position = 1, defaultString = "##current")
public class TransformerFunctionAvg<JE, JA extends Iterable<JE>, JO extends JE> extends TransformerFunction<JE, JA, JO> {

public TransformerFunctionAvg(JsonAdapter<JE, JA, JO> adapter) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,22 @@
import co.nlighten.jsontransform.functions.common.ArgType;
import co.nlighten.jsontransform.functions.common.FunctionContext;
import co.nlighten.jsontransform.functions.common.TransformerFunction;
import co.nlighten.jsontransform.functions.annotations.*;
import co.nlighten.jsontransform.functions.annotations.ArgumentType;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.UnsupportedEncodingException;
import java.nio.charset.StandardCharsets;
import java.util.Base64;

/*
* For tests
* @see TransformerFunctionBase64Test
*/
@Aliases("base64")
@Documentation("Encode to or decode from base64")
@InputType(ArgType.String)
@ArgumentType(value = "action", type = ArgType.Enum, position = 0, defaultEnum = "ENCODE",
description = "Whether to encode or decode input")
@ArgumentType(value = "rfc", type = ArgType.Enum, position = 1, defaultEnum = "BASIC",
description = "Which alphabet to use (BASIC = \"The Base64 Alphabet\" from RFC-2045, URL = \"URL and Filename safe Base64 Alphabet\" from RFC-4648, MIME = Same as BASIC but in lines with no more than 76 characters each)")
@ArgumentType(value = "without_padding", type = ArgType.Boolean, position = 2, defaultBoolean = false,
description = "Don't add padding at the end of the output (The character `=`)")
@ArgumentType(value = "charset", type = ArgType.Enum, position = 3, defaultEnum = "UTF-8",
description = "Character set to use before encoding or when decoding to string")
@OutputType(ArgType.String)
@ArgumentType(value = "action", type = ArgType.Enum, position = 0, defaultEnum = "ENCODE")
@ArgumentType(value = "rfc", type = ArgType.Enum, position = 1, defaultEnum = "BASIC")
@ArgumentType(value = "without_padding", type = ArgType.Boolean, position = 2, defaultBoolean = false)
@ArgumentType(value = "charset", type = ArgType.Enum, position = 3, defaultEnum = "UTF-8")
public class TransformerFunctionBase64<JE, JA extends Iterable<JE>, JO extends JE> extends TransformerFunction<JE, JA, JO> {

static final Logger log = LoggerFactory.getLogger(TransformerFunctionBase64.class);
Expand Down Expand Up @@ -55,7 +48,7 @@ public Object apply(FunctionContext<JE, JA, JO> context) {
if (withoutPadding) {
encoder = encoder.withoutPadding();
}
result = encoder.encodeToString(str.getBytes(charset));
result = new String(encoder.encode(str.getBytes(charset)), StandardCharsets.US_ASCII);

} else {
var decoder = rfc.charAt(0) == 'M' ? Base64.getMimeDecoder() :
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,14 @@
import co.nlighten.jsontransform.adapters.JsonAdapter;
import co.nlighten.jsontransform.functions.common.ArgType;
import co.nlighten.jsontransform.functions.common.FunctionContext;
import co.nlighten.jsontransform.functions.common.FunctionHelpers;
import co.nlighten.jsontransform.functions.common.TransformerFunction;
import co.nlighten.jsontransform.functions.annotations.*;
import co.nlighten.jsontransform.functions.annotations.ArgumentType;

/*
* For tests
* @see TransformerFunctionBooleanTest
*/
@Aliases("boolean")
@Documentation(value = "Evaluates input to boolean using the [Truthy logic]",
notes = """
Strings evaluation depends on `style` argument:
- By default, value must be `"true"` for `true`.
- Unless `style` is set to `JS`, then any non-empty value is `true`. Arrays and objects of size 0 returns `false`.
""")
@InputType(ArgType.Any)
@ArgumentType(value = "style", type = ArgType.Enum, position = 0, defaultEnum = "JAVA",
enumValues = { "JAVA", "JS" },
description = "Style of considering truthy values (JS only relates to string handling; not objects and arrays)")
@OutputType(ArgType.Boolean)
@ArgumentType(value = "style", type = ArgType.Enum, position = 0, defaultEnum = "JAVA")
public class TransformerFunctionBoolean<JE, JA extends Iterable<JE>, JO extends JE> extends TransformerFunction<JE, JA, JO> {
public TransformerFunctionBoolean(JsonAdapter<JE, JA, JO> adapter) {
super(adapter);
Expand Down
Original file line number Diff line number Diff line change
@@ -1,22 +1,13 @@
package co.nlighten.jsontransform.functions;

import co.nlighten.jsontransform.adapters.JsonAdapter;
import co.nlighten.jsontransform.functions.common.ArgType;
import co.nlighten.jsontransform.functions.common.FunctionContext;
import co.nlighten.jsontransform.functions.common.TransformerFunction;
import co.nlighten.jsontransform.functions.annotations.Aliases;
import co.nlighten.jsontransform.functions.annotations.Documentation;
import co.nlighten.jsontransform.functions.annotations.InputType;
import co.nlighten.jsontransform.functions.annotations.OutputType;

/*
* For tests
* @see TransformerFunctionCoalesceTest
*/
@Aliases({"coalesce", "first"})
@Documentation("Returns the first non-null value")
@InputType(ArgType.Array)
@OutputType(value = ArgType.Any, description = "Same as first non-null value")
public class TransformerFunctionCoalesce<JE, JA extends Iterable<JE>, JO extends JE> extends TransformerFunction<JE, JA, JO> {

public TransformerFunctionCoalesce(JsonAdapter<JE, JA, JO> adapter) {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,25 +1,16 @@
package co.nlighten.jsontransform.functions;

import co.nlighten.jsontransform.adapters.JsonAdapter;
import co.nlighten.jsontransform.functions.common.ArgType;
import co.nlighten.jsontransform.functions.common.FunctionContext;
import co.nlighten.jsontransform.functions.common.TransformerFunction;
import co.nlighten.jsontransform.JsonElementStreamer;
import co.nlighten.jsontransform.functions.annotations.Aliases;
import co.nlighten.jsontransform.functions.annotations.Documentation;
import co.nlighten.jsontransform.functions.annotations.InputType;
import co.nlighten.jsontransform.functions.annotations.OutputType;

import java.util.stream.Stream;

/*
* For tests
* @see TransformerFunctionConcatTest
*/
@Aliases("concat")
@Documentation("Concatenates primary value array with elements or other arrays of elements")
@InputType(ArgType.Array)
@OutputType(ArgType.Array)
public class TransformerFunctionConcat<JE, JA extends Iterable<JE>, JO extends JE> extends TransformerFunction<JE, JA, JO> {
public TransformerFunctionConcat(JsonAdapter<JE, JA, JO> adapter) {
super(adapter);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,13 @@
import co.nlighten.jsontransform.functions.common.ArgType;
import co.nlighten.jsontransform.functions.common.FunctionContext;
import co.nlighten.jsontransform.functions.common.TransformerFunction;
import co.nlighten.jsontransform.functions.annotations.*;
import co.nlighten.jsontransform.functions.annotations.ArgumentType;

/*
* For tests
* @see TransformerFunctionContainsTest
*/
@Aliases("contains")
@Documentation("Checks whether an array contains a certain value")
@InputType(ArgType.Array)
@ArgumentType(value = "that", type = ArgType.Any, position = 0, defaultIsNull = true,
description = "The value to look for")
@OutputType(ArgType.Boolean)
@ArgumentType(value = "that", type = ArgType.Any, position = 0, defaultIsNull = true)
public class TransformerFunctionContains<JE, JA extends Iterable<JE>, JO extends JE> extends TransformerFunction<JE, JA, JO> {
public TransformerFunctionContains(JsonAdapter<JE, JA, JO> adapter) {
super(adapter);
Expand Down
Loading
Loading