Benchmarks support

[rbroggi]

I wonder if go-benchmarks support is something that neotest-golang could/should implement. I have found no documentation and or past issues on this regard.

[fredrikaverpil]

It's not been a goal of mine to support benchmarks through Neotest. How would you envision running benchmarks through Neotest should work?

[rbroggi]

Short answer for me would be:

Within only the 'Run Nearest' target, I would also include benchmark flags.

P.s.: I am a long time user of Goland that is where I bring my intuition from. If you run tests for all the file it only runs tests, if you instead run the equivalent of Run nearest inside a benchmark block it runs the benchmark

[fredrikaverpil]

I've got an experimental implementation in the branch named bench. Maybe have a look at it and see what you think.

{ "fredrikaverpil/neotest-golang", branch = "bench" }

The benchmark can only be excecuted by running the individual benchmark. It will be skipped if you run e.g. all tests in a given package/folder or in a given file.

The code became super messy and needs refactoring, so I don't want to merge this in as-is.

If one really wants to "run all benchmarks in a dir or file" in the future, we could look into supporting a custom run command, like:

-- run all benchmarks in the given file
require("neotest").run.run({vim.fn.expand("%"), benchmark = true})

Then neotest-golang would detect that you are interested in running all benchmarks in the given scope (file, dir).

I just question its usefulness. I'm not so sure you are interested in running multiple benchmarks in one go.

[fredrikaverpil]

Side-note, I really just looked into this today because I was writing a benchmark myself and remembered your inquiry. 😄

[rbroggi]

If one really wants to "run all benchmarks in a dir or file" in the future, we could look into supporting a custom run command

Yep, I think though that benchmark workflow is very different in nature than tests - typically it's more iterative and 'on-demand' and I personally never felt the need to run more than one benchmark at time (also because one needs to analyse and assess the outcome carefully).

[rbroggi]

I have not think on how to nicely represent them in the summary section though