Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added support for new substation playground command #262

Merged
merged 15 commits into from
Oct 26, 2024

Conversation

brittonhayes
Copy link
Contributor

@brittonhayes brittonhayes commented Oct 13, 2024

Description

This PR introduces a new locally-served playground for Substation, replacing the previous WASM-based implementation. The new playground is implemented as a Go HTTP server that can be started with the command substation playground. Key changes include:

  1. A new playground command added to the Substation CLI.
  2. An HTTP server that serves an HTML interface for the playground.
  3. Endpoints for running Substation configurations and fetching example configurations.
  4. A simple, responsive UI for editing configurations, input data, and viewing output.

Motivation and Context

  1. Provides a simple, flexible, and maintainable way for users to try out and learn substation naturally while keeping maintenance difficulties low for the maintainers by keeping it in the CLI rather than a hosted site.

image
image

fixes #259

How Has This Been Tested?

The new playground has been tested by:

  1. Running the substation playground command and verifying that the server starts correctly.
  2. Accessing the playground through a web browser and testing various configurations.
  3. Verifying that example configurations load and execute correctly.
  4. Testing the responsiveness and functionality of the UI across different screen sizes.
  5. Ensuring that the playground correctly processes input data and displays output.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@jshlbrd
Copy link
Collaborator

jshlbrd commented Oct 15, 2024

This is so cool! Documented some UX feedback on this screenshot:

Substation___Playground

More detailed notes:

  • Prioritize giving more vertical space to the text fields by decreasing font size and removing text.
    • Why? This is the most useful part of the playground.
  • Remove the dropdown menu and replace with two additional buttons -- "Test" and "Demo" -- that implement the substation test and substation demo subcommands.
    • Why? The dropdown menu is a nice idea, but it's a hardcoded list that needs to be maintained separately and I think implementing the demo subcommand serves the same purpose. Include a link to the examples/ folder, if that's helpful?
  • When the playground is running, disable the "Run" button (and any others that may exist in the future).
    • Why? Some transforms, like this one, can take a while to run and disabling the button may prevent unintended side effects. Bonus points if we can change the button to say "Running" so that the user knows that something is happening and the system hasn't frozen.
  • Change references to JSON events to "message data".
    • Why? It works on more than just JSON text.
  • Wrap lines on text fields and remove the gjson magic here.
    • Why? We don't want to do hidden things on behalf of the user because this can create unintended side effects if they copy configurations and reuse them elsewhere. Wrapping the line helps with readability and, I think, covers the problem that the gjson line is meant to solve. Consider adding an "examples" panel that includes common parsing patterns, like converting to multiline JSON?

All minor things, but I think these would improve how useful this is as a utility.

@brittonhayes brittonhayes marked this pull request as ready for review October 16, 2024 19:19
@brittonhayes brittonhayes requested a review from a team as a code owner October 16, 2024 19:19
@brittonhayes
Copy link
Contributor Author

brittonhayes commented Oct 16, 2024

Hey @jshlbrd! I've implemented all the feedback here except for a couple bullets which I'll add to this PR in a follow up commit.

image

  • Add support for encoding config, input, and output so playgrounds can be shared
  • Move title to navbar and lower the font size
  • Remove tagline text in favor of increased vertical space
  • Set input and output editors to Text mode by default
  • Allow user to select json mode for input and output editors for syntax highlighting and formatting if they choose
  • Remove any magic in the background related to gjson or auto pretty-ification of output results
  • Wrap lines in all editors
  • Add placeholder Test button
  • Add support for Test button
  • Disable run button while Substation is executing

@britton-from-notion
Copy link

The playground now supports unit testing
image

The playground disables the run button while substation is running
image

@jshlbrd
Copy link
Collaborator

jshlbrd commented Oct 24, 2024

@brittonhayes Test support looks good! There's a couple small things I noticed that can be addressed, then this should be good to merge.

Errors

Errors like http.Error(w, fmt.Sprintf("Error transforming messages: %v", err), http.StatusInternalServerError) are truncated in the output, like Error: SyntaxError: Unexpected token 'E', "Error tran"... is not valid JSON. Is it possible to make these display the full error?

Environment Variable Support

There's some functionality missing compared to other subcommands due to the lack of environment variable support. For example, this example config fails...

// This example shows how to use the `utility_secret` transform to
// retrieve a secret and reference it in a subsequent transform.
local sub = import '../../../../substation.libsonnet';

// The secret is retrieved from the environment variable named
// `SUBSTATION_EXAMPLE_URL` and referenced in subsequent transforms using
// the ID value `ENV_VAR`.
//
// Run this on the local system as an example:
//  export SUBSTATION_EXAMPLE_URL=https://www.gutenberg.org/files/2701/old/moby10b.txt
local secret = sub.secrets.environment_variable({ id: 'ENV_VAR', name: 'SUBSTATION_EXAMPLE_URL' });

{
  tests: [
    {
      name: 'http_secret',
      transforms: [
        // An empty message is sufficient for this test.
        sub.tf.test.message(),
      ],
      condition: sub.cnd.num.len.gt({ value: 0 }),
    }
  ],
  // The `utility_secret` transform retrieves the secret from the environment
  // variable and keeps it in memory. The `enrich_http_get` transform references
  // the secret using the ID value `ENV_VAR`. In this example, the secret is the
  // URL of a web page that is retrieved by the `enrich_http_get` transform and
  // sent to stdout by the `send_stdout` transform.
  transforms: [
    sub.transform.utility.secret({ secret: secret }),
    sub.transform.enrich.http.get({ url: '${SECRET:ENV_VAR}' }),
    // Moby Dick is a large text, so the max size of the batch
    // has to be increased, otherwise the data won't fit.
    sub.tf.send.stdout({ batch: { size: 10000000 } }),
  ],
}

... but works with the existing test subcommand ...

% SUBSTATION_EXAMPLE_URL=https://google.com ~/go/bin/substation test config.jsonnet
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en"><head><meta content="Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for." name="description"><meta content="noodp, " name="robots"><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"><title>Google</title><script nonce="fPxiNVvqwYbZ6S470-Zt9w">(function(){var _g={kEI:'57wZZ_GzKZjXkPIPr4yqkQU',kEXPI:'0,3700292,1092,448529,90132,2872,2891,561,17988,24479,36419,9708,18161,162437,23024,6699,124314,2006,8155,8860,14491,22435,9779,62657,36747,3801,2412,33249,15816,1804,7734,18098,9436,11814,342,1292,13493,15784,12989,8791,5303,5212675,1023,489,8832021,1088,74,287,7439645,118,20539821,16673,43886,3,1603,3,2124363,23029351,7954,1,208,4636,16436,12024,36996,35025,11741,10881,885,14280,8182,5933,43496,19011,2655,3439,3319,155,1,1,1,2481,13504,7736,9139,4599,328,3217,4,1235,1769,20601,13023,687,6591,3,23260,1134,207,13651,57,2204,7,12,9273,4134,4844,2230,789,5,271,1,1946,2047,2391,1381,1910,8202,4363,49,748,378,8306,3213,3177,2,1794,9336,1330,4145,5390,8,1450,2,1546,251,594,96,1708,1315,4553,3273,1514,4,1312,1066,1,1,2,3,273,2,1176,41,1928,1,1,2,3,1642,1,2452,1071,932,483,218,2319,829,1158,124,77,398,1,1604,1,11,137,904,594,10,638,462,1,477,44,31,2209,343,1,3,636,1,31,1349,1039,321,88,476,1,3,199,520,467,443,1,87,807,958,1744,4,1,6,1931,1867,1551,1725,2060,84,3079,1,720,473,2,244,484,144,2,935,485,412,1176,1,1921,154,150,380,44,33,1211,1,513,379,284,192,895,3,7,147,651,109,333,441,6,321,10,141,899,999,279,2170,652,433,274,542,710,5,120,261,1150,372,2,90,262,266,4,1584,76,248,232,350,887,59,153,58,18,315,4,2,243,488,93,17,895,346,2,369,50,748,8,1,1,4,1,4,50,10,487,789,1794,1,225,375,1,5,279,106,602,450,123,155,119,260,4,267,515,375,258,748,304,899,1045,129,893,381,351,459,821,736,2154,2,711,2,698,21405980,4294,3,15901,3,2927,14066,21,591,1414,1480,875,1659',kBL:'GTuc',kOPI:89978449};(function(){var a;((a=window.google)==null?0:a.stvsc)?google.kEI=_g.kEI:window.google=_g;}).call(this);})();(function(){google.sn='webhp';google.kHL='en';})();(function(){
var h=this||self;function l(){return window.google!==void 0&&window.google.kOPI!==void 0&&window.google.kOPI!==0?window.google.kOPI:null};var m,n=[];function p(a){for(var b;a&&(!a.getAttribute||!(b=a.getAttribute("eid")));)a=a.parentNode;return b||m}function q(a){for(var b=null;a&&(!a.getAttribute||!(b=a.getAttribute("leid")));)a=a.parentNode;return b}function r(a){/^http:/i.test(a)&&window.location.protocol==="https:"&&(google.ml&&google.ml(Error("a"),!1,{src:a,glmm:1}),a="");return a}
function t(a,b,c,d,k){var e="";b.search("&ei=")===-1&&(e="&ei="+p(d),b.search("&lei=")===-1&&(d=q(d))&&(e+="&lei="+d));d="";var g=b.search("&cshid=")===-1&&a!=="slh",f=[];f.push(["zx",Date.now().toString()]);h._cshid&&g&&f.push(["cshid",h._cshid]);c=c();c!=null&&f.push(["opi",c.toString()]);for(c=0;c<f.length;c++){if(c===0||c>0)d+="&";d+=f[c][0]+"="+f[c][1]}return"/"+(k||"gen_204")+"?atyp=i&ct="+String(a)+"&cad="+(b+e+d)};m=google.kEI;google.getEI=p;google.getLEI=q;google.ml=function(){return null};google.log=function(a,b,c,d,k,e){e=e===void 0?l:e;c||(c=t(a,b,e,d,k));if(c=r(c)){a=new Image;var g=n.length;n[g]=a;a.onerror=a.onload=a.onabort=function(){delete n[g]};a.src=c}};google.logUrl=function(a,b){b=b===void 0?l:b;return t("",a,b)};}).call(this);(function(){google.y={};google.sy=[];var d;(d=google).x||(d.x=function(a,b){if(a)var c=a.id;else{do c=Math.random();while(google.y[c])}google.y[c]=[a,b];return!1});var e;(e=google).sx||(e.sx=function(a){google.sy.push(a)});google.lm=[];var f;(f=google).plm||(f.plm=function(a){google.lm.push.apply(google.lm,a)});google.lq=[];var g;(g=google).load||(g.load=function(a,b,c){google.lq.push([[a],b,c])});var h;(h=google).loadAll||(h.loadAll=function(a,b){google.lq.push([a,b])});google.bx=!1;var k;(k=google).lx||(k.lx=function(){});var l=[],m;(m=google).fce||(m.fce=function(a,b,c,n){l.push([a,b,c,n])});google.qce=l;}).call(this);google.f={};(function(){
document.documentElement.addEventListener("submit",function(b){var a;if(a=b.target){var c=a.getAttribute("data-submitfalse");a=c==="1"||c==="q"&&!a.elements.q.value?!0:!1}else a=!1;a&&(b.preventDefault(),b.stopPropagation())},!0);document.documentElement.addEventListener("click",function(b){var a;a:{for(a=b.target;a&&a!==document.documentElement;a=a.parentElement)if(a.tagName==="A"){a=a.getAttribute("data-nohref")==="1";break a}a=!1}a&&b.preventDefault()},!0);}).call(this);</script><style>#gbar,#guser{font-size:13px;padding-top:1px !important;}#gbar{height:22px}#guser{padding-bottom:7px !important;text-align:right}.gbh,.gbd{border-top:1px solid #c9d7f1;font-size:1px}.gbh{height:0;position:absolute;top:24px;width:100%}@media all{.gb1{height:22px;margin-right:.5em;vertical-align:top}#gbar{float:left}}a.gb1,a.gb4{text-decoration:underline !important}a.gb1,a.gb4{color:#00c !important}.gbi .gb4{color:#dd8e27 !important}.gbf .gb4{color:#900 !important}
</style><style>body,td,a,p,.h{font-family:arial,sans-serif}body{margin:0;overflow-y:scroll}#gog{padding:3px 8px 0}td{line-height:.8em}.gac_m td{line-height:17px}form{margin-bottom:20px}.h{color:#1967d2}em{font-weight:bold;font-style:normal}.lst{height:25px;width:496px}.gsfi,.lst{font:18px arial,sans-serif}.gsfs{font:17px arial,sans-serif}.ds{display:inline-box;display:inline-block;margin:3px 0 4px;margin-left:4px}input{font-family:inherit}body{background:#fff;color:#000}a{color:#681da8;text-decoration:none}a:hover,a:active{text-decoration:underline}.fl a{color:#1967d2}a:visited{color:#681da8}.sblc{padding-top:5px}.sblc a{display:block;margin:2px 0;margin-left:13px;font-size:11px}.lsbb{background:#f8f9fa;border:solid 1px;border-color:#dadce0 #70757a #70757a #dadce0;height:30px}.lsbb{display:block}#WqQANb a{display:inline-block;margin:0 12px}.lsb{background:url(/images/nav_logo229.png) 0 -261px repeat-x;color:#000;border:none;cursor:pointer;height:30px;margin:0;outline:0;font:15px arial,sans-serif;vertical-align:top}.lsb:active{background:#dadce0}.lst:focus{outline:none}</style><script nonce="fPxiNVvqwYbZ6S470-Zt9w">(function(){window.google.erd={jsr:1,bv:2102,de:true};
var g=this||self;var k,l=(k=g.mei)!=null?k:1,n,p=(n=g.sdo)!=null?n:!0,q=0,r,t=google.erd,v=t.jsr;google.ml=function(a,b,d,m,e){e=e===void 0?2:e;b&&(r=a&&a.message);d===void 0&&(d={});d.cad="ple_"+google.ple+".aple_"+google.aple;if(google.dl)return google.dl(a,e,d,!0),null;b=d;if(v<0){window.console&&console.error(a,b);if(v===-2)throw a;b=!1}else b=!a||!a.message||a.message==="Error loading script"||q>=l&&!m?!1:!0;if(!b)return null;q++;d=d||{};b=encodeURIComponent;var c="/gen_204?atyp=i&ei="+b(google.kEI);google.kEXPI&&(c+="&jexpid="+b(google.kEXPI));c+="&srcpg="+b(google.sn)+"&jsr="+b(t.jsr)+
"&bver="+b(t.bv);t.dpf&&(c+="&dpf="+b(t.dpf));var f=a.lineNumber;f!==void 0&&(c+="&line="+f);var h=a.fileName;h&&(h.indexOf("-extension:/")>0&&(e=3),c+="&script="+b(h),f&&h===window.location.href&&(f=document.documentElement.outerHTML.split("\n")[f],c+="&cad="+b(f?f.substring(0,300):"No script found.")));google.ple&&google.ple===1&&(e=2);c+="&jsel="+e;for(var u in d)c+="&",c+=b(u),c+="=",c+=b(d[u]);c=c+"&emsg="+b(a.name+": "+a.message);c=c+"&jsst="+b(a.stack||"N/A");c.length>=12288&&(c=c.substr(0,12288));a=c;m||google.log(0,"",a);return a};window.onerror=function(a,b,d,m,e){r!==a&&(a=e instanceof Error?e:Error(a),d===void 0||"lineNumber"in a||(a.lineNumber=d),b===void 0||"fileName"in a||(a.fileName=b),google.ml(a,!1,void 0,!1,a.name==="SyntaxError"||a.message.substring(0,11)==="SyntaxError"||a.message.indexOf("Script error")!==-1?3:0));r=null;p&&q>=l&&(window.onerror=null)};})();</script></head><body bgcolor="#fff"><script nonce="fPxiNVvqwYbZ6S470-Zt9w">(function(){var src='/images/nav_logo229.png';var iesg=false;document.body.onload = function(){window.n && window.n();if (document.images){new Image().src=src;}
if (!iesg){document.f&&document.f.q.focus();document.gbqf&&document.gbqf.q.focus();}
}
})();</script><div id="mngb"><div id=gbar><nobr><b class=gb1>Search</b> <a class=gb1 href="https://www.google.com/imghp?hl=en&tab=wi">Images</a> <a class=gb1 href="https://maps.google.com/maps?hl=en&tab=wl">Maps</a> <a class=gb1 href="https://play.google.com/?hl=en&tab=w8">Play</a> <a class=gb1 href="https://www.youtube.com/?tab=w1">YouTube</a> <a class=gb1 href="https://news.google.com/?tab=wn">News</a> <a class=gb1 href="https://mail.google.com/mail/?tab=wm">Gmail</a> <a class=gb1 href="https://drive.google.com/?tab=wo">Drive</a> <a class=gb1 style="text-decoration:none" href="https://www.google.com/intl/en/about/products?tab=wh"><u>More</u> &raquo;</a></nobr></div><div id=guser width=100%><nobr><span id=gbn class=gbi></span><span id=gbf class=gbf></span><span id=gbe></span><a href="http://www.google.com/history/optout?hl=en" class=gb4>Web History</a> | <a  href="/preferences?hl=en" class=gb4>Settings</a> | <a target=_top id=gb_70 href="https://accounts.google.com/ServiceLogin?hl=en&passive=true&continue=https://www.google.com/&ec=GAZAAQ" class=gb4>Sign in</a></nobr></div><div class=gbh style=left:0></div><div class=gbh style=right:0></div></div><center><br clear="all" id="lgpd"><div id="XjhHGf"><img alt="Google" height="92" src="/images/branding/googlelogo/1x/googlelogo_white_background_color_272x92dp.png" style="padding:28px 0 14px" width="272" id="hplogo"><br><br></div><form action="/search" name="f"><table cellpadding="0" cellspacing="0"><tr valign="top"><td width="25%">&nbsp;</td><td align="center" nowrap=""><input name="ie" value="ISO-8859-1" type="hidden"><input value="en" name="hl" type="hidden"><input name="source" type="hidden" value="hp"><input name="biw" type="hidden"><input name="bih" type="hidden"><div class="ds" style="height:32px;margin:4px 0"><input class="lst" style="margin:0;padding:5px 8px 0 6px;vertical-align:top;color:#000" autocomplete="off" value="" title="Google Search" maxlength="2048" name="q" size="57"></div><br style="line-height:0"><span class="ds"><span class="lsbb"><input class="lsb" value="Google Search" name="btnG" type="submit"></span></span><span class="ds"><span class="lsbb"><input class="lsb" id="tsuid_1" value="I'm Feeling Lucky" name="btnI" type="submit"><script nonce="fPxiNVvqwYbZ6S470-Zt9w">(function(){var id='tsuid_1';document.getElementById(id).onclick = function(){if (this.form.q.value){this.checked = 1;if (this.form.iflsig)this.form.iflsig.disabled = false;}
else top.location='/doodles/';};})();</script><input value="AL9hbdgAAAAAZxnK998SN1H4XuQkGnUViRbIa9bzWn7t" name="iflsig" type="hidden"></span></span></td><td class="fl sblc" align="left" nowrap="" width="25%"><a href="/advanced_search?hl=en&amp;authuser=0">Advanced search</a></td></tr></table><input id="gbv" name="gbv" type="hidden" value="1"><script nonce="fPxiNVvqwYbZ6S470-Zt9w">(function(){var a,b="1";if(document&&document.getElementById)if(typeof XMLHttpRequest!="undefined")b="2";else if(typeof ActiveXObject!="undefined"){var c,d,e=["MSXML2.XMLHTTP.6.0","MSXML2.XMLHTTP.3.0","MSXML2.XMLHTTP","Microsoft.XMLHTTP"];for(c=0;d=e[c++];)try{new ActiveXObject(d),b="2"}catch(h){}}a=b;if(a=="2"&&location.search.indexOf("&gbv=2")==-1){var f=google.gbvu,g=document.getElementById("gbv");g&&(g.value=a);f&&window.setTimeout(function(){location.href=f},0)};}).call(this);</script></form><div style="font-size:83%;min-height:3.5em"><br></div><span id="footer"><div style="font-size:10pt"><div style="margin:19px auto;text-align:center" id="WqQANb"><a href="/intl/en/ads/">Advertising</a><a href="/services/">Business Solutions</a><a href="/intl/en/about.html">About Google</a></div></div><p style="font-size:8pt;color:#70757a">&copy; 2024 - <a href="/intl/en/policies/privacy/">Privacy</a> - <a href="/intl/en/policies/terms/">Terms</a></p></span></center><script nonce="fPxiNVvqwYbZ6S470-Zt9w">(function(){window.google.cdo={height:757,width:1440};(function(){var a=window.innerWidth,b=window.innerHeight;if(!a||!b){var c=window.document,d=c.compatMode=="CSS1Compat"?c.documentElement:c.body;a=d.clientWidth;b=d.clientHeight}
if(a&&b&&(a!=google.cdo.width||b!=google.cdo.height)){var e=google,f=e.log,g="/client_204?&atyp=i&biw="+a+"&bih="+b+"&ei="+google.kEI,h="",k=[],l=window.google!==void 0&&window.google.kOPI!==void 0&&window.google.kOPI!==0?window.google.kOPI:null;l!=null&&k.push(["opi",l.toString()]);for(var m=0;m<k.length;m++){if(m===0||m>0)h+="&";h+=k[m][0]+"="+k[m][1]}f.call(e,"","",g+h)};}).call(this);})();</script>  <script nonce="fPxiNVvqwYbZ6S470-Zt9w">(function(){google.xjs={basecomb:'/xjs/_/js/k\x3dxjs.hp.en.jbt6jLK1eEM.es5.O/ck\x3dxjs.hp.u6VifKyplPg.L.X.O/am\x3dAgAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAABAEAAAAAAAAAAAGACAGAAAAAIABAQAAAAAHAAAAIAAAAQgAgBQABAmAAAciO8IAATAIgAAvA/d\x3d1/ed\x3d1/dg\x3d0/ujg\x3d1/rs\x3dACT90oH-cNaxv8cQ-Ob2NGKdXYi7sXRSSA',basecss:'/xjs/_/ss/k\x3dxjs.hp.u6VifKyplPg.L.X.O/am\x3dAgAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAACAAAAAAAIABAQAAAAAAAAAAIAAAAQgAgBQABAm/rs\x3dACT90oGG7vV8zZ22LuBxQ7jAYiL4_uF3bg',basejs:'/xjs/_/js/k\x3dxjs.hp.en.jbt6jLK1eEM.es5.O/am\x3dAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAGAAAGAAAAAAAAAAAAAAAHAAAAAAAAAAAAAAAAAAAAAAciO8IAATAIgAAvA/dg\x3d0/rs\x3dACT90oEYWOr9xuE0LCDgqPhQX2drz2k0IA',excm:[]};})();</script>  <link href="/xjs/_/ss/k=xjs.hp.u6VifKyplPg.L.X.O/am=AgAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAACAAAAAAAIABAQAAAAAAAAAAIAAAAQgAgBQABAm/d=1/ed=1/rs=ACT90oGG7vV8zZ22LuBxQ7jAYiL4_uF3bg/m=sb_he,d" rel="stylesheet" nonce="fPxiNVvqwYbZ6S470-Zt9w">      <script nonce="fPxiNVvqwYbZ6S470-Zt9w">(function(){var u='/xjs/_/js/k\x3dxjs.hp.en.jbt6jLK1eEM.es5.O/am\x3dAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAGAAAGAAAAAAAAAAAAAAAHAAAAAAAAAAAAAAAAAAAAAAciO8IAATAIgAAvA/d\x3d1/ed\x3d1/dg\x3d3/rs\x3dACT90oEYWOr9xuE0LCDgqPhQX2drz2k0IA/m\x3dsb_he,d';var st=1;var amd=1000;var mmd=0;var pod=true;var f=typeof Object.defineProperties=="function"?Object.defineProperty:function(a,b,c){if(a==Array.prototype||a==Object.prototype)return a;a[b]=c.value;return a},g=function(a){a=["object"==typeof globalThis&&globalThis,a,"object"==typeof window&&window,"object"==typeof self&&self,"object"==typeof global&&global];for(var b=0;b<a.length;++b){var c=a[b];if(c&&c.Math==Math)return c}throw Error("a");},h=g(this),k=function(a,b){if(b)a:{var c=h;a=a.split(".");for(var d=0;d<a.length-1;d++){var e=a[d];if(!(e in
c))break a;c=c[e]}a=a[a.length-1];d=c[a];b=b(d);b!=d&&b!=null&&f(c,a,{configurable:!0,writable:!0,value:b})}};k("globalThis",function(a){return a||h});
var l=this||self;function m(){var a,b,c;if(b=a=(b=window.google)==null?void 0:(c=b.ia)==null?void 0:c.r.B2Jtyd)b=a.m,b=b===1||b===5;return b&&a.cbfd!=null&&a.cbvi!=null?a:void 0};
var n=globalThis.trustedTypes,p;function q(){var a=null;if(!n)return a;try{var b=function(c){return c};a=n.createPolicy("goog#html",{createHTML:b,createScript:b,createScriptURL:b})}catch(c){}return a};var r=function(a){this.g=a};r.prototype.toString=function(){return this.g+""};function t(a){if(a instanceof r)return a.g;throw Error("b");};var w=/^\s*(?!javascript:)(?:[\w+.-]+:|[^:/?#]*(?:[/?#]|$))/i;var x="alternate author bookmark canonical cite help icon license modulepreload next prefetch dns-prefetch prerender preconnect preload prev search subresource".split(" ");function y(a,b){a.src=t(b);var c;b=a.ownerDocument&&a.ownerDocument.defaultView||window;b=b===void 0?document:b;var d;b=(d=(c="document"in b?b.document:b).querySelector)==null?void 0:d.call(c,"script[nonce]");(c=b==null?"":b.nonce||b.getAttribute("nonce")||"")&&a.setAttribute("nonce",c)};var z=function(a){var b=document;a=String(a);b.contentType==="application/xhtml+xml"&&(a=a.toLowerCase());return b.createElement(a)};function A(a){a=a===null?"null":a===void 0?"undefined":a;p===void 0&&(p=q());var b=p;return new r(b?b.createScriptURL(a):a)};google.ps===void 0&&(google.ps=[]);function B(){var a=u,b=function(){};google.lx=google.stvsc?b:function(){C(a);google.lx=b};google.bx||google.lx()}function D(a,b){b&&y(a,A(b));var c=a.onload;a.onload=function(d){c&&c(d);google.ps=google.ps.filter(function(e){return a!==e})};google.ps.push(a);document.body.appendChild(a)}google.as=D;function C(a){google.timers&&google.timers.load&&google.tick&&google.tick("load","xjsls");var b=z("SCRIPT");b.onerror=function(){google.ple=1};b.onload=function(){google.ple=0};google.xjsus=void 0;D(b,a);google.aple=-1;google.dp=!0}
function E(){var a=[u];if(!google.dp){for(var b=0;b<a.length;b++){var c=z("LINK"),d=c,e=A(a[b]);if(e instanceof r)d.href=t(e).toString(),d.rel="preload";else{if(x.indexOf("preload")===-1)throw Error("c`preload");e=w.test(e)?e:void 0;e!==void 0&&(d.href=e,d.rel="preload")}c.setAttribute("as","script");document.body.appendChild(c)}google.dp=!0}};function F(a){var b=a.getAttribute("jscontroller");return(b==="UBXHI"||b==="R3fhkb"||b==="TSZEqd")&&a.hasAttribute("data-src")}function G(){for(var a=document.getElementsByTagName("img"),b=0,c=a.length;b<c;b++){var d=a[b];if(d.hasAttribute("data-lzy_")&&Number(d.getAttribute("data-atf"))&1&&!F(d))return!0}return!1}for(var H=document.getElementsByTagName("img"),I=0,J=H.length;I<J;++I){var K=H[I];Number(K.getAttribute("data-atf"))&1&&F(K)&&(K.src=K.getAttribute("data-src"))};var L,M,N,O,P,Q;function R(){google.xjsu=u;l._F_jsUrl=u;P=function(){B()};L=!1;M=(st===1||st===3)&&!!google.caft&&!G();N=m();O=(st===2||st===3)&&!!N&&!G();Q=pod}function S(){L||M||O||(P(),L=!0)}setTimeout(function(){google&&google.tick&&google.timers&&google.timers.load&&google.tick("load","xjspls");R();if(M||O){if(M){var a=function(){M=!1;S()};google.caft(a);window.setTimeout(a,amd)}O&&(a=function(){O=!1;S()},N.cbvi.push(a),window.setTimeout(a,mmd));Q&&(L||E())}else P()},0);})();window._ = window._ || {};window._DumpException = _._DumpException = function(e){throw e;};window._s = window._s || {};_s._DumpException = _._DumpException;window._qs = window._qs || {};_qs._DumpException = _._DumpException;(function(){var t=[2,256,0,0,0,0,0,1042,67108864,6553605,16877664,167936,1028,7340160,134217728,134283264,273678338,38992,251167168,813760514,1006632994,2];window._F_toggles = window._xjs_toggles = t;})();window._F_installCss = window._F_installCss || function(css){};(function(){google.jl={bfl:0,dw:false,ine:false,ubm:false,uwp:true,vs:false};})();(function(){var pmc='{\x22d\x22:{},\x22sb_he\x22:{\x22agen\x22:false,\x22cgen\x22:false,\x22client\x22:\x22heirloom-hp\x22,\x22dh\x22:true,\x22ds\x22:\x22\x22,\x22fl\x22:true,\x22host\x22:\x22google.com\x22,\x22jsonp\x22:true,\x22msgs\x22:{\x22cibl\x22:\x22Clear Search\x22,\x22dym\x22:\x22Did you mean:\x22,\x22lcky\x22:\x22I\\u0026#39;m Feeling Lucky\x22,\x22lml\x22:\x22Learn more\x22,\x22psrc\x22:\x22This search was removed from your \\u003Ca href\x3d\\\x22/history\\\x22\\u003EWeb History\\u003C/a\\u003E\x22,\x22psrl\x22:\x22Remove\x22,\x22sbit\x22:\x22Search by image\x22,\x22srch\x22:\x22Google Search\x22},\x22ovr\x22:{},\x22pq\x22:\x22\x22,\x22rfs\x22:[],\x22stok\x22:\x22kgxqgig5V-WXaTNqu-lvbaTXCLA\x22}}';google.pmc=JSON.parse(pmc);})();(function(){var b=function(a){var c=0;return function(){return c<a.length?{done:!1,value:a[c++]}:{done:!0}}};
var e=this||self;var g,h;a:{for(var k=["CLOSURE_FLAGS"],l=e,n=0;n<k.length;n++)if(l=l[k[n]],l==null){h=null;break a}h=l}var p=h&&h[610401301];g=p!=null?p:!1;var q,r=e.navigator;q=r?r.userAgentData||null:null;function t(a){return g?q?q.brands.some(function(c){return(c=c.brand)&&c.indexOf(a)!=-1}):!1:!1}function u(a){var c;a:{if(c=e.navigator)if(c=c.userAgent)break a;c=""}return c.indexOf(a)!=-1};function v(){return g?!!q&&q.brands.length>0:!1}function w(){return u("Safari")&&!(x()||(v()?0:u("Coast"))||(v()?0:u("Opera"))||(v()?0:u("Edge"))||(v()?t("Microsoft Edge"):u("Edg/"))||(v()?t("Opera"):u("OPR"))||u("Firefox")||u("FxiOS")||u("Silk")||u("Android"))}function x(){return v()?t("Chromium"):(u("Chrome")||u("CriOS"))&&!(v()?0:u("Edge"))||u("Silk")}function y(){return u("Android")&&!(x()||u("Firefox")||u("FxiOS")||(v()?0:u("Opera"))||u("Silk"))};var z=v()?!1:u("Trident")||u("MSIE");y();x();w();var A=!z&&!w(),D=function(a){if(/-[a-z]/.test("ved"))return null;if(A&&a.dataset){if(y()&&!("ved"in a.dataset))return null;a=a.dataset.ved;return a===void 0?null:a}return a.getAttribute("data-"+"ved".replace(/([A-Z])/g,"-$1").toLowerCase())};var E=[],F=null;function G(a){a=a.target;var c=performance.now(),f=[],H=f.concat,d=E;if(!(d instanceof Array)){var m=typeof Symbol!="undefined"&&Symbol.iterator&&d[Symbol.iterator];if(m)d=m.call(d);else if(typeof d.length=="number")d={next:b(d)};else throw Error("b`"+String(d));for(var B=[];!(m=d.next()).done;)B.push(m.value);d=B}E=H.call(f,d,[c]);if(a&&a instanceof HTMLElement)if(a===F){if(c=E.length>=4)c=(E[E.length-1]-E[E.length-4])/1E3<5;if(c){c=google.getEI(a);a.hasAttribute("data-ved")?f=a?D(a)||"":"":f=(f=
a.closest("[data-ved]"))?D(f)||"":"";f=f||"";if(a.hasAttribute("jsname"))a=a.getAttribute("jsname");else{var C;a=(C=a.closest("[jsname]"))==null?void 0:C.getAttribute("jsname")}google.log("rcm","&ei="+c+"&tgtved="+f+"&jsname="+(a||""))}}else F=a,E=[c]}window.document.addEventListener("DOMContentLoaded",function(){document.body.addEventListener("click",G)});}).call(this);</script></body></html>
ok	config.jsonnet	313.78ms

Environment variables can contain secrets, so we wouldn't want to support them in a way that introduces a risk of credential leaks. Here are a couple ideas, but let me know what you think:

  • Add another text field for storing environment variables, but don't copy the values when the Share button is used.
  • Read an .env file that is local to where the substation playground command was run. (I don't personally like storing env var in any files, but this is a pattern used by some languages and people.)

External String Support

Similar to environment variables, but I'm less opinionated about how to handle them. Unfortunately a custom config won't work because the external strings need to be provided at Jsonnet compile time. If we want to say it's out of scope for this, then I'm OK with that.

Verbose Test Outputs

Not a problem, but the Test button behaves a little differently than substation test -- it's more verbose by printing each test result, more like running go test -v ./.... Might be worth porting a verbose flag to that subcommand later.

Importing .libsonnet Files

I was surprised to see that this just works, if the playground is run from the local directory:

local const = import '../../const.libsonnet';
local model = import '../../model.libsonnet';
local sub = import '../../substation.libsonnet';

local _test = import '_test.libsonnet';

{
  tests: [
    {
      name: 'event',
      transforms: [
        sub.tf.test.message({ value: _test.msg }),
      ],
      condition: sub.cnd.all([
        ...
      ]),
    },
  ],
  transforms: [
    ...
  ],
}

Nothing to do here, but might be useful to mention in some docs later.

substation.go Outdated
@@ -10,6 +11,9 @@ import (
"github.com/brexhq/substation/v2/transform"
)

//go:embed substation.libsonnet
var Libsonnet string
Copy link
Collaborator

@jshlbrd jshlbrd Oct 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are referred to by Jsonnet as a "library," so renaming the var to Library would make a bit more sense semantically. (Also gives us slightly better forward compatibility in case file extensions change for some reason.)

@brittonhayes
Copy link
Contributor Author

Love this feedback and agree with all the points. I'll do some investigation on improved error reporting and env var support. I'll also change that Libsonnet variable no problem. For the external strings I think lets have that be a follow up addon and same for exploration of test verbosity control.

But the rest should be do-able and in-scope for an initial playground implementation!

@brittonhayes
Copy link
Contributor Author

brittonhayes commented Oct 25, 2024

Updates

  • Environment variable support through a modal popup
  • Full error output in the output editor box
  • No sharing of environment variables through share button
  • Rename Libsonnet var to Library
  • Improved UX of "running..." button so it doesn't flash on very fast runs

Screenshots

New "Environment" button in the action bar and format button attached to configuration editor.
image

New Environment Variable editor modal for adding in env vars to be used by configuration.
image

Full error output in output box when errors occur
image

Bottom right shows successful usage of environment variable to return the output response
image

@jshlbrd
Copy link
Collaborator

jshlbrd commented Oct 25, 2024

Fantastic work, hiding env var behind a modal and simplifying the format button is a nice touch. Let's merge this. 🚀

@jshlbrd jshlbrd merged commit 82f39e8 into brexhq:main Oct 26, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Substation Playground
3 participants