Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

beanlabs/will: having cross-repo trouble using in bazel #7

Open
mhansen opened this issue Sep 11, 2022 · 4 comments
Open

beanlabs/will: having cross-repo trouble using in bazel #7

mhansen opened this issue Sep 11, 2022 · 4 comments

Comments

@mhansen
Copy link

mhansen commented Sep 11, 2022

Hi there, I'll preface this by saying this is super low priority -- I'm not really using will.py for anything important. And I'm probably doing something very weird that nobody else is. Just wanted to report.

I'm trying to use bazel to run a bunch of beancount scripts. Mostly this is straightforward.

But I'm having some trouble using https://github.com/beancount/beanlabs/blob/master/beanlabs/will/will.py.

Problem number 1: will.py has a main function, but is not exposed as a py_binary in the BUILD rule. That's probably fairly straightforward to fix.

Problem number 2: Build visibility.

The BUILD file (https://github.com/beancount/beanlabs/blob/master/beanlabs/will/BUILD) refers to targets in //beancount:

py_library(
    name = "will",
    srcs = ["will.py"],
    deps = [
        "//beancount/core:account",
        ...
    ],
)

//beancount maybe used to be accessible when beanlabs was a subfolder of the main beancount repository. But now it's in a different repository, and we need to import and rename them via a WORKSPACE file like this:

git_repository(                                                                                                                                                                    
    name = "bc",                                                                                                                                                                   
    remote = "https://github.com/beancount/beancount.git",                                                                                                                         
    commit = "5adf7b9a3be99f631f4b8cd161287f1479dce377",                                                                                                                           
    shallow_since = "1616264925 -0400",                                                                                                                                            
)

And then access them with the renamed name, like "@//beancount/core:account".

So I'm doing this, by writing my own BUILD.beanlabs file:

load("@rules_python//python:defs.bzl", "py_binary")                                                                                                                                
                                                                                                                                                                                   
package(                                                                                                                                                                           
    default_visibility = ["//visibility:public"],                                                                                                                                  
)                                                                                                                                                                                  
                                                                                                                                                                                   
# Can't use the existing build rules because they assume beanlabs will be run from inside beancount                                                                                
py_binary(
    name = "will",
    srcs = ["//beanlabs/will:will.py"],
    deps = [
        "@bc//beancount/core:account",
        "@bc//beancount/core:account_types",
        "@bc//beancount/core:convert",
        "@bc//beancount/core:data",
        "@bc//beancount/core:getters",
        "@bc//beancount/core:realization",
        "@bc//beancount:loader",
        "@bc//beancount/parser:options",
        "@bc//beancount/parser:version",
    ],                                                                                                                                                                             
)

But then that fails due to BUILD visibility of the depended targets:

$ bazel test ...
ERROR: /private/var/tmp/_bazel_mark/c85e0696457b3a2f404f0721275052b4/external/beanlabs/BUILD.bazel:8:10: in py_binary rule @beanlabs//:will: target '@bc//beancount/core:account' is not visible from target '@beanlabs//:will'. Check the visibility declaration of the former target if you think the dependency is legitimate
ERROR: /private/var/tmp/_bazel_mark/c85e0696457b3a2f404f0721275052b4/external/beanlabs/BUILD.bazel:8:10: in py_binary rule @beanlabs//:will: target '@bc//beancount/core:account_types' is not visible from target '@beanlabs//:will'. Check the visibility declaration of the former target if you think the dependency is legitimate
ERROR: /private/var/tmp/_bazel_mark/c85e0696457b3a2f404f0721275052b4/external/beanlabs/BUILD.bazel:8:10: in py_binary rule @beanlabs//:will: target '@bc//beancount/core:convert' is not visible from target '@beanlabs//:will'. Check the visibility declaration of the former target if you think the dependency is legitimate
ERROR: /private/var/tmp/_bazel_mark/c85e0696457b3a2f404f0721275052b4/external/beanlabs/BUILD.bazel:8:10: in py_binary rule @beanlabs//:will: target '@bc//beancount/core:data' is not visible from target '@beanlabs//:will'. Check the visibility declaration of the former target if you think the dependency is legitimate
ERROR: /private/var/tmp/_bazel_mark/c85e0696457b3a2f404f0721275052b4/external/beanlabs/BUILD.bazel:8:10: in py_binary rule @beanlabs//:will: target '@bc//beancount/core:getters' is not visible from target '@beanlabs//:will'. Check the visibility declaration of the former target if you think the dependency is legitimate
ERROR: /private/var/tmp/_bazel_mark/c85e0696457b3a2f404f0721275052b4/external/beanlabs/BUILD.bazel:8:10: in py_binary rule @beanlabs//:will: target '@bc//beancount/core:realization' is not visible from target '@beanlabs//:will'. Check the visibility declaration of the former target if you think the dependency is legitimate
ERROR: /private/var/tmp/_bazel_mark/c85e0696457b3a2f404f0721275052b4/external/beanlabs/BUILD.bazel:8:10: Analysis of target '@beanlabs//:will' failed

Indeed, the default visibility of beancount.core is fairly restrictive: https://github.com/beancount/beancount/blob/master/beancount/core/BUILD#L2

According to bazelbuild/bazel#3744, if you want visibility cross-repo, the only supported visibility is //visibility:public.

In the meantime, I'm working around this by specifying --nocheck_visibility. But it would be neat to avoid this :-)

I'm not sure, but I think there's an argument that beancount.core is part of the "public API" of beancount, depended on by anyone importing beancount, so maybe it should also be //visibility:public?

I'm guessing this beanlabs BUILD code is fairly vestigial (as there's no WORKSPACE in the repo).

I don't really expect you to fix this for me -- I'm happy to help contribute some fixes, just wanted to start a conversation about the desired state here.
@blais
Copy link
Member

blais commented Sep 11, 2022 via email

@mhansen
Copy link
Author

mhansen commented Sep 12, 2022

Thanks Martin. I've got an admittedly overengineered setup where I'm converting some historical hledger files to beancount and then running some tests on the hledger and some tests on the beancount (asserting that the numbers equal my tax returns for example), and some shellscript tests and generating a list of accounts with will.py, and I'm trying to tie together all these different languages with bazel. Is this silly? Yeah, probably. But it beats make, and lets me cache slow actions and get somewhat-hermetic results.

Not sure what the 'old process' and 'new code' is, sorry (is the old process an old build system, and the new code bazel BUILD)? But yeah, I was previously installing locally (maybe with pip, not sure) and inserting into PYTHONPATH, and that worked for a few years, the will.py python script wasn't really running hermetically under bazel so it all worked.

Is there any desire to get bazel ready for cross-repo work, or direction you'd like it to go (e.g. what do you think about making some of the modules visibility public and adding a WORKSPACE to this experimental git repo?) Totally understand if the focus is on v3 and this v2 stuff is sorta-deprecated, I'm happy to just continue with grabbing it from PYTHONPATH.

@blais
Copy link
Member

blais commented Sep 13, 2022 via email

@mhansen
Copy link
Author

mhansen commented Sep 13, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mhansen @blais