Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(expr): enhance expr with JQ functionality #3191

Open
wants to merge 34 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
847a7da
feat(mustache): enhance mustache with JQ funtionality
karatakis Dec 3, 2024
f35dc2a
wip: implement missing functions
karatakis Dec 4, 2024
a502e71
feat: extend DynamicValue with jq templates * The resulting value is …
karatakis Dec 4, 2024
1fd76f0
fix: jq template to work with current implementation
karatakis Dec 5, 2024
ffa4e9c
fix: lint
karatakis Dec 5, 2024
9cb72ee
fix: add tests
karatakis Dec 5, 2024
c8087c1
Merge branch 'main' into feat/jq-template-hybrid
karatakis Dec 5, 2024
ef00eb8
fix: lint
karatakis Dec 5, 2024
1406a8d
fix: add more tests
karatakis Dec 5, 2024
f3de14d
feat: jq with mustache
karatakis Dec 5, 2024
ae9ebb0
wip: impl ValT
karatakis Dec 5, 2024
531d524
fix: compile errors
karatakis Dec 6, 2024
533b766
fix: missing implementations
karatakis Dec 6, 2024
d60d170
fix: compile jq only once
karatakis Dec 9, 2024
1c082be
refactor: JqTemplate to JqTransform
karatakis Dec 9, 2024
551c091
feat: add jq_template
karatakis Dec 9, 2024
06b37b4
fix: lint
karatakis Dec 9, 2024
dd7a9be
fix: all tests
karatakis Dec 9, 2024
74ee689
fix: error handling
karatakis Dec 10, 2024
98cc24d
fix: allow referencing .env for jq templates
karatakis Dec 10, 2024
c13656d
fix: add more tests
karatakis Dec 10, 2024
85abf92
fix: add args test
karatakis Dec 10, 2024
d74853f
fix: add jq select tests
karatakis Dec 10, 2024
4cc06e1
Merge branch 'main' into feat/jq-template-hybrid
karatakis Dec 10, 2024
ef0b699
fix: update cargo lock
karatakis Dec 10, 2024
5b04d26
Merge branch 'main' into feat/jq-template-hybrid
karatakis Dec 11, 2024
4676300
refactor: review changes
karatakis Dec 12, 2024
7571da7
fix: remove headers and vars
karatakis Dec 12, 2024
2e3c8f9
fix: allow getting env and header from context
karatakis Dec 12, 2024
84cf742
fix: change priority of parsing
karatakis Dec 12, 2024
a1e6b83
Merge branch 'main' into feat/jq-template-hybrid
karatakis Dec 12, 2024
ed65705
fix: lint
karatakis Dec 12, 2024
b4853be
Merge branch 'main' into feat/jq-template-hybrid
karatakis Dec 18, 2024
1c55d91
Merge branch 'main' into feat/jq-template-hybrid
tusharmath Dec 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,9 @@ tailcall-valid = { workspace = true }
dashmap = "6.1.0"
urlencoding = "2.1.3"
tailcall-chunk = "0.3.0"
jaq-core = "2.0.0"
jaq-std = "2.0.0"
jaq-json = { version = "1.0.0", features = ["serde_json"]}

# to build rquickjs bindings on systems without builtin bindings
[target.'cfg(all(target_os = "windows", target_arch = "x86"))'.dependencies]
Expand Down
25 changes: 12 additions & 13 deletions src/core/blueprint/dynamic_value.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@
use indexmap::IndexMap;
use serde_json::Value;

use crate::core::mustache::Mustache;
use crate::core::Mustache;

#[derive(Debug, Clone, PartialEq)]
/// This is used to express dynamic value resolver engine.
pub enum DynamicValue<A> {
Value(A),
Mustache(Mustache),
Expand All @@ -25,18 +26,14 @@
pub fn prepend(self, name: &str) -> Self {
match self {
DynamicValue::Value(value) => DynamicValue::Value(value),
DynamicValue::Mustache(mut mustache) => {
if mustache.is_const() {
DynamicValue::Mustache(mustache)
} else {
let segments = mustache.segments_mut();
if let Some(crate::core::mustache::Segment::Expression(vec)) =
segments.get_mut(0)
{
DynamicValue::Mustache(mut m) => {
for seg in m.segments_mut() {
if let crate::core::mustache::Segment::Expression(vec) = seg {
vec.insert(0, name.to_string());
}
DynamicValue::Mustache(mustache)
}

DynamicValue::Mustache(m)
}
DynamicValue::Object(index_map) => {
let index_map = index_map
Expand All @@ -60,7 +57,7 @@
match value {
DynamicValue::Value(v) => Ok(v.to_owned()),
DynamicValue::Mustache(_) => Err(anyhow::anyhow!(
"mustache cannot be converted to const value"
"Mustache template cannot be converted to const value"

Check warning on line 60 in src/core/blueprint/dynamic_value.rs

View check run for this annotation

Codecov / codecov/patch

src/core/blueprint/dynamic_value.rs#L60

Added line #L60 was not covered by tests
)),
DynamicValue::Object(obj) => {
let out: Result<IndexMap<Name, ConstValue>, anyhow::Error> = obj
Expand All @@ -82,7 +79,7 @@
// Helper method to determine if the value is constant (non-mustache).
pub fn is_const(&self) -> bool {
match self {
DynamicValue::Mustache(m) => m.is_const(),
DynamicValue::Mustache(t) => t.is_const(),
DynamicValue::Object(obj) => obj.values().all(|v| v.is_const()),
DynamicValue::Array(arr) => arr.iter().all(|v| v.is_const()),
_ => true,
Expand All @@ -93,6 +90,7 @@
impl TryFrom<&Value> for DynamicValue<ConstValue> {
type Error = anyhow::Error;

/// Used to convert json notation to dynamic value
fn try_from(value: &Value) -> Result<Self, Self::Error> {
match value {
Value::Object(obj) => {
Expand All @@ -109,7 +107,8 @@
Ok(DynamicValue::Array(out?))
}
Value::String(s) => {
let m = Mustache::parse(s.as_str());
let m = Mustache::parse(s);

if m.is_const() {
Ok(DynamicValue::Value(ConstValue::from_json(value.clone())?))
} else {
Expand Down
2 changes: 2 additions & 0 deletions src/core/http/request_template.rs
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,7 @@
async_graphql::Value::String(text.to_owned()),
))),
Segment::Expression(parts) => in_value.raw_value(parts),
Segment::JqTransform(_) => panic!("Cannot eval JQ for PathValue"),

Check warning on line 304 in src/core/http/request_template.rs

View check run for this annotation

Codecov / codecov/patch

src/core/http/request_template.rs#L304

Added line #L304 was not covered by tests
})
.next() // Return the first value that is found
}
Expand Down Expand Up @@ -344,6 +345,7 @@
}
}
}
Segment::JqTransform(_) => panic!("Cannot eval JQ for PathString"),

Check warning on line 348 in src/core/http/request_template.rs

View check run for this annotation

Codecov / codecov/patch

src/core/http/request_template.rs#L348

Added line #L348 was not covered by tests
}
}
(result, first_expression_value)
Expand Down
6 changes: 5 additions & 1 deletion src/core/ir/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
use thiserror::Error;

use crate::core::jit::graphql_error::{Error as ExtensionError, ErrorExtensions};
use crate::core::mustache::JqRuntimeError;
use crate::core::{auth, cache, worker, Errata};

#[derive(From, Debug, Error, Clone)]
Expand Down Expand Up @@ -33,6 +34,8 @@

Cache(cache::Error),

DynamicValue(JqRuntimeError),

#[from(ignore)]
Entity(String),
}
Expand Down Expand Up @@ -67,7 +70,8 @@
}
Error::Worker(err) => Errata::new("Worker Error").description(err.to_string()),
Error::Cache(err) => Errata::new("Cache Error").description(err.to_string()),
Error::Entity(message) => Errata::new("Entity Resolver Error").description(message)
Error::Entity(message) => Errata::new("Entity Resolver Error").description(message),

Check warning on line 73 in src/core/ir/error.rs

View check run for this annotation

Codecov / codecov/patch

src/core/ir/error.rs#L73

Added line #L73 was not covered by tests
Error::DynamicValue(err) => Errata::new("Dynamic Value Rendering Error").description(err.to_string())
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion src/core/ir/eval.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ impl IR {
.unwrap_or(&async_graphql::Value::Null)
.clone())
}
IR::Dynamic(value) => Ok(value.render_value(ctx)),
IR::Dynamic(value) => Ok(value.render_value(ctx)?),
IR::Protect(auth, expr) => {
let verifier = AuthVerifier::from(auth.clone());
verifier.verify(ctx.request_ctx).await.to_result()?;
Expand Down
8 changes: 8 additions & 0 deletions src/core/mustache/eval.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@
.unwrap_or(
Mustache::from(vec![Segment::Expression(parts.to_vec())]).to_string(),
),
Segment::JqTransform(jqt) => {
Mustache::from(vec![Segment::JqTransform(jqt.clone())])
.to_string()
.replace("\"", "\\\"")
}
})
.collect()
}
Expand All @@ -57,6 +62,7 @@
.path_string(parts)
.map(|a| a.to_string())
.unwrap_or_default(),
Segment::JqTransform(_) => Default::default(),

Check warning on line 65 in src/core/mustache/eval.rs

View check run for this annotation

Codecov / codecov/patch

src/core/mustache/eval.rs#L65

Added line #L65 was not covered by tests
})
.collect()
}
Expand Down Expand Up @@ -92,6 +98,7 @@
.filter_map(|segment| match segment {
Segment::Literal(text) => Some(Exit::Text(text)),
Segment::Expression(parts) => in_value.get_path(parts).map(Exit::Value),
Segment::JqTransform(_) => panic!("Cannot eval JQ for Path"),

Check warning on line 101 in src/core/mustache/eval.rs

View check run for this annotation

Codecov / codecov/patch

src/core/mustache/eval.rs#L101

Added line #L101 was not covered by tests
})
.collect::<Vec<_>>()
}
Expand All @@ -116,6 +123,7 @@
.map(|segment| match segment {
Segment::Literal(text) => text.to_string(),
Segment::Expression(parts) => in_value.path_graphql(parts).unwrap_or_default(),
Segment::JqTransform(_) => panic!("Cannot eval JQ for PathGraphql"),

Check warning on line 126 in src/core/mustache/eval.rs

View check run for this annotation

Codecov / codecov/patch

src/core/mustache/eval.rs#L126

Added line #L126 was not covered by tests
})
.collect()
}
Expand Down
Loading
Loading