Supporting Source Code Autocomplete

[cpsauer]

Update: For C++, C, Objective-C and Objective-C++, please give https://github.com/hedronvision/bazel-compile-commands-extractor a try! We moved to Bazel and built a pretty solid solution for our (and others') problem. Please let me know what you think!

And if you'd like support for other languages, we'd love your help over there, too. Please let us know in an issue what you're thinking about.

---

Hey all,

We're evaluating a switch from Buck to Bazel. Are there any plans to support autocomplete for source files (esp. C++) in VSCode?

Some leads:

• If there's a way to get Bazel to export the compile commands it invokes, existing autocomplete plugins can pick up the slack pretty well from there.
  • For example, Buck (and its plugin) use the #compilation-database flavor to dump a compile_commands.json file, which is then picked up by clangd and can be used to provide intelligent autocomplete in almost any editor, VSCode included.
    • Intellisense C/C++ was more brittle in its parsing of compile_commands.json in our experience.
    • Looks like some other Google projects (e.g. Fuchsia) are also recommending clangd in VSCode using compile_commands.json, so maybe this would be more broadly useful?
• It also looks like an older, unofficial vscode bazel plugin implemented autocomplete functionality (see Integrate features into bazelbuild/vscode-bazel dprogm/vscode-bazel-tools#10), but by using the VSCode/Microsoft IntelliSense specific c_cpp_properties.json.
  • c_cpp_properties.json maybe wouldn't be as good of a fit for Bazel proper as compile_commands.json, since it's VSCode specific, but the discussion on that linked issue is good.

(CC'ing some other folks who participated in that discussion: @laurentlb @zaucy @quentingodeau @dprogm @allevato)

Thanks so much,
Chris

[quentingodeau]

Hi @cpsauer,

I found some time ago, and still use it, this tools that allow you to generate the compile_commands.json. This one is great for auto-completion and sonar analyses because it is fast to generate the file (it did not need to compile everything).

Otherwise if you look for something that will generate the 'perfect' compile_command.json you can use action_listener like it's describe here but this required to compile everything...

[cpsauer]

You're the man, @quentingodeau. Will give it a whirl soon--those are amazing resources, and I'm sure they'll also help out a bunch of future people to come through here.

[lawsonAGMT]

@quentingodeau works great, thanks!

[bhack]

Any news on this?

[cpsauer]

Update over here: We ended up building our own, based on the leads @quentingodeau brought up. Lmk if you need something more, too.

The result is pretty awesome (we think): Just run a shell script, and it generates a compile_commands.json that interfaces great with clangd. That means we've got autocomplete for Android NDK, iOS, and macOS files across C, C++, Objective-C, and Objective C++, all together in VSCode. There are a few rough edges, but we're quite happy.

[While all the ones listed were good starting points, we found all had serious issues that prevented them from working for our use case. In particular, in order to get the cross-platform thing to work, you really need to listen to the actual compile commands Bazel is emitting with aquery (or action_listener (like Kythe or the gist does), but aquery is orders of magnitude faster and doesn't require a build), not run queries or aspects, which tend to miss configuration from bazel transitions. In retrospect, it makes sense that to extract compilation_commands.json you should listen to bazel's compilation actions directly.... Also, you have to do some careful work to undo bazel's driver wrapping that otherwise prevents clang tooling from understanding the commands and causes issues when the execroot is reconfigured. (Solves issues like these: 1, 2)].

Are other folks also finding that the existing tools don't meet their needs? If enough other folks need it and would be willing to help, I could work on open sourcing it.

Update to the update :)
This is now released at https://github.com/hedronvision/bazel-compile-commands-extractor. I'd love it if you'd give it a try and let me know what you think!

[estk]

@cpsauer that would be excellent, i'm surprised people haven't piled in here on this one.

[warriorstar-orion]

@cpsauer I'd also be interested in what you managed to pull together, especially if it works in a project with multiple WORKSPACEs, keeping autocomplete suggestions isolated to the relevant sub-project.

[clearlyspam23]

@cpsauer I would also be interested in this script, I've been trying to use the bazel-compliation-database to limited success. I really wish this was a feature supported by default (it seems weird to me that the official bazel plugin wouldn't support something this essential).

[robbie-vanderzee]

@cpsauer Adding to the support for you to share this. Thanks!

[zaucy]

To add to this discussion a bit I created an example repository how we've achieved autocomplete for our project. We use https://github.com/grailbio/bazel-compilation-database as mentioned #179 (comment), but we substitute the execution root inside the compile_commands.json and copy it to the root of the repository with a tool I built called bzlws. That way we don't have to hard code the execution root in a BUILD file or use any shell scripts outside of bazel. You can give it a try in the example repository.

[micahcc]

@cpsauer Would love to see what you came up with. I have something along these lines: https://gist.github.com/micahcc/bb31e4c91696537f5c0d8b9c8f964255

[cpsauer]

Dear @bhack, @mevensson, @paulherman, @tbienias, @sharif1093, @SnosMe, @372046933, @ConeyLiu, @ycdzj, @robbie-vanderzee, @enihcam, @Nick-Mazuk, @slsyy, @chrisdunelm, @matzipan, @nickshch, @ha11owed, @danzyxr, @wannesvanloock, @silvergasp, @chriscraws, @micahcc, @chong1144, @dhmemi, @sunqi1993, @soulmachine, @raakasf and all others interested in a release,

I'm really sorry it took so long, but I finally got this out!
https://github.com/hedronvision/bazel-compile-commands-extractor

I'd love it if you would give it a try and let me know what you think.

We love using it every day, so I know the core of it works well. But if it isn't working for you, first users, or I messed up something during release, let's figure out how to make it great for you. Hopefully, I can ask for your patience and help.

Basically, would love it if you'd try using. Hope it's great for you! And I hope we can make it even better together.

---

Bazel team members, if you're reading this, lmk--happy to help integrate.

[sekoyo]

I'm really sorry it took so long, but I finally got this out!
https://github.com/hedronvision/bazel-compile-commands-extractor

Thanks @cpsauer , noob Bazel-ler here, when I run bazel run @hedron_compile_commands//:refresh_all it produces a compile_commands.json with just [] in it?

Tried adding a BUILD in workspace root too but same thing:

load("@hedron_compile_commands//:refresh_compile_commands.bzl", "refresh_compile_commands")

refresh_compile_commands(
    name = "refresh_compile_commands",

    # Specify the targets of interest.
    # For example, specify a dict of targets and their arguments:
    targets = {
        "src/main:main": "",
    },
    # For more details, feel free to look into refresh_compile_commands.bzl if you want.
)

[cpsauer]

Sounds like it (for some reason) doesn't think there are any C language family compile actions in your workspace!

We should probably try to troubleshoot over in the compile commands repo proper to avoid swamping this issue with so many folks tagged/following--but we'll need a little more info. Any chance you can share the repo? How about running bazel aquery //... and taking a peek at the output for compilation entries!

Once you've got a next lead, go ahead and file an issue in the other repo, and I'll follow up :)