Add a query editor test without mocking the query endpoint

[sarahzinger]

While e2e tests with mocks are faster and easier to collaborate on, we think it would be worth while to add a test that uses the /query endpoint without mocking the responses. We have a lot of logic in terms of creating dataframes and what not that we miss if we only use stubbed out responses.

Traditionally we've done this by running our tests with live data, which is probably our best bet for now though it has some downsides (slow, costs money to query, hard for external devs to collaborate with) but we could also explore if something like this could help: https://github.com/grafana/grafana-plugin-sdk-go/blob/main/experimental/e2e/README.md

[chejimmy]

I agree it would be worthwhile to add "real" data responses 👍 And using live data does have the drawbacks you mentioned.

Looking at E2E HTTP Fixture Proxy, would it have the same effect as the work here? - f1ab5bc#diff-8309f392863767a8780967828ff8bb0fc9ead425df3131cbc76c031d8823c291
What benefits do we gain from E2E HTTP Fixture Proxy? From your experience, would it be easier to maintain / update?

[sarahzinger]

I haven't used the e2e http fixture proxy before but my (possibly flawed) understanding is that it can let you mock out just the aws portion and not the entire request.

So right now our mocks are like this:

browser makes a request to /query -> stubbed out data frame responses

but with the proxy I think we might be able to do something like this:

browser makes a request to /query -> runs through the plugin's backend code as it would in production which makes a request with the aws sdk -> returns stubbed out responses

This gives us the advantage of being able to ensure that the plugin's backend parses the data correctly and passes it along to the frontend. That said that proxy tool is an experimental folder so I'm not sure what the status of it is, or how it would work exactly with the aws-sdk (which tends to kind of obscure what http requests our backend is making) but it might be worth playing around with it to see if it what I think it is and how it works with playwright and aws. I might be wrong but I think there is a way to have the aws-sdk hit a proxy endpoint rather than amazon's endpoints.

Alternatively, we could write some kind of script which generates the same test env in aws. Contributors could run it so that they have instances that mimic ours to run their scripts against.