Skip to content

Commit

Permalink
Parse deferred templates twice
Browse files Browse the repository at this point in the history
Currently it is not possible to have a template file.epp

```puppet
<%- |
Stdlib::Port $port,
String[1] $password,
| %>
port <%= $port %>
password <%= $password %>

```

and run

```puppet
file{'/tmp/junk':
  content => stdlib::deferrable_epp('module/file.epp', { 'port' => '1234', pass => Deferred('secrets::get',['mysecret'])}),
}
```
since the deferred template substitution  will fail:
```
Error: Failed to apply catalog: Evaluation Error: Resource type not found: Stdlib::Port (file: inlined-epp-text, line: 2, column: 3)
```
due to Stdlib::Port not being available on the agent node.

This change now parses the EPP twice. The first pass will reduce the
template to:

```puppet
port = 1234
password <%= $password %>
```

and this simpler template will be passed in deferred mode.

Note the original template type for password must accept the
intermediate generated value of `<%= $password %>` which is typically
case for a secret password.
  • Loading branch information
traylenator committed Apr 15, 2024
1 parent d871c4d commit 55a003f
Show file tree
Hide file tree
Showing 3 changed files with 95 additions and 15 deletions.
42 changes: 34 additions & 8 deletions REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3173,18 +3173,38 @@ Type: Puppet Language
This function returns either a rendered template or a deferred function to render at runtime.
If any of the values in the variables hash are deferred, then the template will be deferred.

Note: this function requires all parameters to be explicitly passed in. It cannot expect to
use facts, class variables, and other variables in scope. This is because when deferred, we
have to explicitly pass the entire scope to the client.

#### `stdlib::deferrable_epp(String $template, Hash $variables)`
Note: In the case where at least some of the values are deferred and preparse is `true` the template
is parsed twice:
The first parse will evalute any parameters in the template that do not have deferred values.
The second parse will run deferred and evaluate only the remaining deferred parameters. Consequently
any parameters to be deferred must accept a String[1] in original template so as to accept the value
"<%= $variable_with_deferred_value %>" on the first parse.

@param template template location - identical to epp function template location.
@param variables parameters to pass into the template - some of which may have deferred values.
@param preparse
If `true` the epp template will be parsed twice, once normally and then a second time deferred.
It may be nescessary to set `preparse` `false` when deferred values are somethig other than
a string

#### `stdlib::deferrable_epp(String $template, Hash $variables, Boolean $preparse = true)`

This function returns either a rendered template or a deferred function to render at runtime.
If any of the values in the variables hash are deferred, then the template will be deferred.

Note: this function requires all parameters to be explicitly passed in. It cannot expect to
use facts, class variables, and other variables in scope. This is because when deferred, we
have to explicitly pass the entire scope to the client.
Note: In the case where at least some of the values are deferred and preparse is `true` the template
is parsed twice:
The first parse will evalute any parameters in the template that do not have deferred values.
The second parse will run deferred and evaluate only the remaining deferred parameters. Consequently
any parameters to be deferred must accept a String[1] in original template so as to accept the value
"<%= $variable_with_deferred_value %>" on the first parse.

@param template template location - identical to epp function template location.
@param variables parameters to pass into the template - some of which may have deferred values.
@param preparse
If `true` the epp template will be parsed twice, once normally and then a second time deferred.
It may be nescessary to set `preparse` `false` when deferred values are somethig other than
a string

Returns: `Variant[String, Sensitive[String], Deferred]`

Expand All @@ -3200,6 +3220,12 @@ Data type: `Hash`



##### `preparse`

Data type: `Boolean`



### <a name="stdlib--end_with"></a>`stdlib::end_with`

Type: Ruby 4.x API
Expand Down
35 changes: 30 additions & 5 deletions functions/deferrable_epp.pp
Original file line number Diff line number Diff line change
@@ -1,15 +1,40 @@
# This function returns either a rendered template or a deferred function to render at runtime.
# If any of the values in the variables hash are deferred, then the template will be deferred.
#
# Note: this function requires all parameters to be explicitly passed in. It cannot expect to
# use facts, class variables, and other variables in scope. This is because when deferred, we
# have to explicitly pass the entire scope to the client.
# Note: In the case where at least some of the values are deferred and preparse is `true` the template
# is parsed twice:
# The first parse will evalute any parameters in the template that do not have deferred values.
# The second parse will run deferred and evaluate only the remaining deferred parameters. Consequently
# any parameters to be deferred must accept a String[1] in original template so as to accept the value
# "<%= $variable_with_deferred_value %>" on the first parse.
#
function stdlib::deferrable_epp(String $template, Hash $variables) >> Variant[String, Sensitive[String], Deferred] {
# @param template template location - identical to epp function template location.
# @param variables parameters to pass into the template - some of which may have deferred values.
# @param preparse
# If `true` the epp template will be parsed twice, once normally and then a second time deferred.
# It may be nescessary to set `preparse` `false` when deferred values are somethig other than
# a string
#
function stdlib::deferrable_epp(String $template, Hash $variables, Boolean $preparse = true) >> Variant[String, Sensitive[String], Deferred] {
if $variables.stdlib::nested_values.any |$value| { $value.is_a(Deferred) } {

Check failure on line 20 in functions/deferrable_epp.pp

View workflow job for this annotation

GitHub Actions / Spec / Spec tests (Puppet: ~> 7.24, Ruby Ver: 2.7)

there should be a single space or single newline after an opening brace (check: manifest_whitespace_opening_brace_after)

Check failure on line 20 in functions/deferrable_epp.pp

View workflow job for this annotation

GitHub Actions / Spec / Spec tests (Puppet: ~> 8.0, Ruby Ver: 3.2)

there should be a single space or single newline after an opening brace (check: manifest_whitespace_opening_brace_after)
if $preparse {
$_variables_escaped = $variables.map | $_var , $_value | {
if $_value.is_a(Deferred) {
{ $_var => "<%= \$${_var} %>" }
} else {
{ $_var => $_value }
}
}.reduce | $_memo, $_kv | { $_memo + $_kv }

$_template = inline_epp(find_template($template).file,$_variables_escaped)
} else {
$_template = find_template($template).file
}

Deferred(
'inline_epp',
[find_template($template).file, $variables],
[$_template, $variables],
)
}
else {
Expand Down
33 changes: 31 additions & 2 deletions spec/functions/stdlib_deferrable_epp_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,37 @@

it {
foo = Puppet::Pops::Types::TypeFactory.deferred.create('join', [1, 2, 3])
# This kind_of matcher requires https://github.com/puppetlabs/rspec-puppet/pull/24
expect(subject).to run.with_params('mymod/template.epp', { 'foo' => foo }) # .and_return(kind_of Puppet::Pops::Types::PuppetObject)
expect(subject).to run.with_params('mymod/template.epp', { 'foo' => foo }).and_return(kind_of(Puppet::Pops::Types::PuppetObject))
}
end

context 'defers rendering with mixed deferred and undeferred input' do
let(:pre_condition) do
<<~END
function epp($str, $data) { fail("should not have invoked epp()") }
function find_template($str) { return "path" }
function file($path) { return "foo: <%= foo %>, bar: <%= bar %>" }
END
end

it {
foo = Puppet::Pops::Types::TypeFactory.deferred.create('join', [1, 2, 3])
expect(subject).to run.with_params('mymod/template.epp', { 'foo' => foo, 'bar' => 'xyz' }).and_return(kind_of(Puppet::Pops::Types::PuppetObject))
}
end

context 'defers rendering with mixed deferred and undeferred input and preparse false' do
let(:pre_condition) do
<<~END
function epp($str, $data) { fail("should not have invoked epp()") }
function find_template($str) { return "path" }
function file($path) { return "foo: <%= foo %>, bar: <%= bar %>" }
END
end

it {
foo = Puppet::Pops::Types::TypeFactory.deferred.create('join', [1, 2, 3])
expect(subject).to run.with_params('mymod/template.epp', { 'foo' => foo, 'bar' => 'xyz' }, false).and_return(kind_of(Puppet::Pops::Types::PuppetObject))
}
end
end

0 comments on commit 55a003f

Please sign in to comment.