obtain compilation flags or compilation database from bazel

[cnsgsz]

to have good auto complete, etc in emacs, etc, tools like irony mode or semantic often requires either compilation flags or compilation database (e.g. see compilation database under https://github.com/Sarcasm/irony-mode). is there an easy way to get that from Bazel?

[damienmg]

Do you mean you want to get the list of flag /commands for Bazel? For that
we have 'bazel help completion' that we use for generating shell completion
helpers.
Le sam. 20 juin 2015 à 20:50, cnsgsz [email protected] a écrit :

to have good auto complete, etc in emacs, etc, tools like irony mode or
semantic often requires either compilation flags or compilation database
(e.g. see compilation database under https://github.com/Sarcasm/irony-mode).
is there an easy way to get that from Bazel?

—
Reply to this email directly or view it on GitHub
#258.

Damien

[cnsgsz]

I meant the full gcc commands bazel use for each compilation unit, like gcc -I xxx -D yyy -include zzz foo.cc, or something similar that would reveal the dependencies between files. Tools for auto code completion such as irony mode or semantic in emacs requires those information to perform well.

[damienmg]

Is the -s flag what you are looking for (it prints the list of command it
executes and there is also --nobuild to don't actually execute the build
commands)?

I don't really understand why the code completion need those. Do you know
why it needs it and what exactly it needs? The part on "compilation
database" is pretty obscure.
Le sam. 20 juin 2015 à 22:48, cnsgsz [email protected] a écrit :

I meant the full gcc commands bazel use for each compilation unit, like
gcc -I xxx -D yyy -include zzz foo.cc, or something similar that would
reveal the dependencies between files. Tools for auto code completion such
as irony mode or semantic in emacs requires those information to perform
well.

—
Reply to this email directly or view it on GitHub
#258 (comment).

Damien

[laszlocsomor]

P3 for now because so far it seems a solution already exists (the -s flag).

[damienmg]

I think it's actually resolved. Closing for now, please ping if you still have questions.

[cnsgsz]

I got something like below with -s. However, the path under -iquote seems to be a temporary directory, rather than the real include path/directory. I think the auto completion tool needs the real include directory/file to know when a .c says #include foo.h, which foo.h it is referring to.

(cd /home/swang/.cache/bazel/_bazel_swang/1255f70669ee87603abcd2c127b1fe2d/project &&
exec env -
INTERCEPT_LOCALLY_EXECUTABLE=1
/usr/bin/gcc '-std=c++0x' -iquote . -iquote bazel-out/local_linux-fastbuild/genfiles -isystem tools/cpp/gcc3 '-frandom-seed=bazel-out/local_linux-fastbuild/bin/tools/cpp/_objs/malloc/tools/cpp/empty.pic.o' -MD -MF bazel-out/local_linux-fastbuild/bin/tools/cpp/_objs/malloc/tools/cpp/empty.pic.d -fPIC -c tools/cpp/empty.cc -o bazel-out/local_linux-fastbuild/bin/tools/cpp/_objs/malloc/tools/cpp/empty.pic.o)

[damienmg]

The path under iquote should be valid from the workspace too after a Bazel
build
Le sam. 27 juin 2015 à 05:49, cnsgsz [email protected] a écrit :

I got something like below with -s. However, the path under -iquote seems
to be a temporary directory, rather than the real include path/directory. I
think the auto completion tool needs the real include directory/file to
know when a .c says #include foo.h, which foo.h it is referring to.

(cd
/home/swang/.cache/bazel/_bazel_swang/1255f70669ee87603abcd2c127b1fe2d/project
&&
exec env -
INTERCEPT_LOCALLY_EXECUTABLE=1
/usr/bin/gcc '-std=c++0x' -iquote . -iquote
bazel-out/local_linux-fastbuild/genfiles -isystem tools/cpp/gcc3
'-frandom-seed=bazel-out/local_linux-fastbuild/bin/tools/cpp/_objs/malloc/tools/cpp/empty.pic.o'
-MD -MF
bazel-out/local_linux-fastbuild/bin/tools/cpp/_objs/malloc/tools/cpp/empty.pic.d
-fPIC -c tools/cpp/empty.cc -o
bazel-out/local_linux-fastbuild/bin/tools/cpp/_objs/malloc/tools/cpp/empty.pic.o)

—
Reply to this email directly or view it on GitHub
#258 (comment).

[cnsgsz]

when I do ls bazel-out/local_linux-fastbuild/, I only see bin under it, not genfile.

[damienmg]

Well if you have no genfiles then it's normal the folder isn't created yet. It can happens if you haven't perform a full build yet.

[cnsgsz]

I just bazel clean and bazel build, and has the binary. But still there is only bin under local_linux-fastbuild, not genfiles.

[damienmg]

I have reopen the bug. We are going on working on IDE and provide a generic interface that might include what you want.

[damienmg]

I confirmed it will be included in our IDE support. We will get back to it when our IDE support is out.

[shawnLeeZX]

Any update?

[DroneDude123]

This would be super useful for me as well. I see that the some IDE support has been added, how could this be implemented from the current IDE support?

[dslomov]

I believe the aspects should provide all you need: https://bazel.build/blog/2016/06/10/ide-support.html

[amosbird]

+1. Compilation database is really useful for setting up a ycmd development env.

[Sydius]

So is there a way to generate a LibTooling-compatible compilation database for use with Clang's analyzers and with Vim's YouCompleteMe plugin for C++? I couldn't find a way to do it, preventing me from switching away from CMake (which does generate one). Clang-Tidy etc. also require the same compilation database for C++.

[chpatton013]

+1 for compilation database support.

[petemounce]

This would allow facebook/infer#457 - +1?

[voithos]

I think ideally this would be doable via aspects (assuming that they have access to things like e.g. compilation flags), but I couldn't find any existing example in the wild that does this.

I did find this, though: https://github.com/vincent-picaud/Bazel_and_CompileCommands, which uses action_listener and extra_action to generate a compilation database. It requires a bit of setup in the form of some Python scripts that perform the post-processing, but there's a convenient shell script that can generate all the needed files for you. I was able to successfully use it in my project: https://github.com/voithos/quarkGL

[siddharthab]

We have Bazel aspect definitions that we use internally at Grail for generating the compile commands. I have asked if we can open source it. It is tested well with YouCompleteMe, rtags and woboq. It is an improvement upon the rough sketch at https://github.com/ibab/laminate/blob/master/bazel/compilation_database.bzl

As for an approach based on action_listeners, I think it is better to use the code in Kythe to do it. It is a bit more involved than the aspect based approach, but at least you know it will be properly maintained. It is also very similar to the approach by Vincent (mentioned above), but uses Go instead of Python.
https://github.com/google/kythe/blob/master/tools/cpp/generate_compilation_database.sh

Note that the kythe repo also has an extra conf file for ycm at the repo top.
https://github.com/google/kythe/blob/master/.ycm_extra_conf.py

More (unrelated) on Kythe's approach at:
https://kythe.io/examples/#extracting-compilations-using-bazel

[siddharthab]

We open sourced our setup:
https://github.com/Grailbio/bazel-compilation-database

Contributions are welcome.

[m1ke-van-dyke]

+1 for compilation database support
I hope this feature is coming soon.

[tolikzinovyev]

I copied and slightly rearranged all necessary files from Kythe to generate compile_commands.json and run clang-tidy: https://github.com/aszinovyev/bazel-compilation-db.

@siddharthab Your approach didn't quite work for me. compile_flags += cpp_fragment.cxx_options(ctx.features) in aspects.bzl overrides my copts. Maybe you want to include these flags before including ctx.rule.attr.copts.

[siddharthab]

Thank you for the feedback. It's an easy fix if that's the only problem.

[atomgalaxy]

I'd like to chime in and say that I don't think this solution is good enough. Bazel should ship something like a standard target to do this instead of relying on 3rd-party rules. Compilation databases are really important for running any kind of tooling that does proper parsing of the source code, and having to rely on hacky (albeit mostly-working) solutions is not ideal.

[jxramos]

I second @atomgalaxy's sentiment, the compilation database has basically become a fairly standardized third party tooling interchange format especially surrounding libclang based tools with all their semantic proper c++ parsing goodness.

A brief glimpse of some built in options for some alternate build tools is provided below; it would be most excellent if bazel build held some analogous command line flag/option to trigger the rendering of a compile_commands.json in the working directory or whatever makes the most sense.

cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON .

https://sarcasm.github.io/notes/dev/compilation-database.html#cmake

ninja -t compdb [RULES...]

https://sarcasm.github.io/notes/dev/compilation-database.html#ninja

[mellery451]

I also agree that compilation databases are vital to proper use of certain tools (e.g. clangd is a great tool but it needs a compile DB for accurate results).

I've been playing with the https://github.com/Grailbio/bazel-compilation-database mentioned above. I've been able to get it to produce some output, but there is at least one issue for me related to sandboxing: generated files (e.g. config headers) are gone after the compilation because of sandbox cleanup. That means you will have missing source files if any were generated. Is there a way to get bazel to store generated files somewhere so that they can be found by tools using a compile database?

[siddharthab]

Hi @mellery451,

Thank you for trying out the tool. Perhaps you refer to grailbio/bazel-compilation-database#29.

I think bazel related work for that issue is being tracked in #10845.

Thanks.

[bhack]

Why this ticket is still closed?

Edit: We have a similar request in bazel-contrib/vscode-bazel#179

[cpsauer]

Hey all, following the retirement of the other solution to maintenance mode--and in response to lots of requests to release the one we'd built...

https://github.com/hedronvision/bazel-compile-commands-extractor
I'd love it if you'd give me a try and let me know what you think.

It uses a different, aquery-based approach to directly ask Bazel for the commands it would issue in a build--and then debazels/reformats those commands into a compile_commands.json. It's fast like aspects, accurate like action_listeners, and dodges a bunch of categories of issues that come with each.

[More discussion in the vscode-bazel issue]