Chaining dependencies/non-header-only dependencies

[Lazauya]

So I've been working towards an MVP of a package server from the discussion here, but I've been stumbling over the fact that poac can only use header only (or system/pkg-config) libraries. I'm sure this is already on the radar, but what are the biggest hurdles to overcome at the moment?

To get the discussion going, here's my understanding of the problem: when you build an executable, you need to know the object dependencies, and you need to know the order of the dependencies. Poac currently doesn't support building libraries, but that doesn't seem like a huge hurdle unless I'm missing something.
So then:

1. We need to support building libraries
2. Once we have library support, we need to recursively build the dependencies

1 seems trivial(?). Effectively we just need to run ar. I don't think worrying about shared libraries is necessary yet.
2 seems also not too hard, though we need to make sure we specify the libraries in the correct order. We also need to be able to resolve where the library will be generated so that we can use it in the master Makefile. I'm guessing the best solution would be to keep all the Makefiles separate. I'd like some input as I'm going to be shifting focus to this until it's in a usable state!

[ken-matsui]

So I've been working towards an MVP of a package server from the discussion here, but I've been stumbling over the fact that poac can only use header only (or system/pkg-config) libraries. I'm sure this is already on the radar, but what are the biggest hurdles to overcome at the moment?

Thanks! The biggest problem is that I simply don’t have enough time, though I’ve been slowly working towards it, like c01d673 allows us to run build on dependencies.

To get the discussion going, here's my understanding of the problem: when you build an executable, you need to know the object dependencies, and you need to know the order of the dependencies. Poac currently doesn't support building libraries, but that doesn't seem like a huge hurdle unless I'm missing something.

So then:

1. We need to support building libraries

2. Once we have library support, we need to recursively build the dependencies

Yep, I think you are right. I simply didn't have enough time to sort them out.

1 seems trivial(?). Effectively we just need to run ar. I don't think worrying about shared libraries is necessary yet.

Basically, yes. But I think there would be a lot of (annoying) implementation details, such as naming and location (output directory) conventions.

2 seems also not too hard, though we need to make sure we specify the libraries in the correct order.

If we want to start with the dependency solver, we can use the existing code in srcOld, or implement a smarter solver like CDCL. Then, if the dependency graph is SAT solvable, we can say the graph is also topo sortable, which means we can gain a correct order. Note that this isn't necessary to be recursive, can be iterative, which is better in general.

We also need to be able to resolve where the library will be generated so that we can use it in the master Makefile.

I think this is why we need to establish naming and location conventions.

I'm guessing the best solution would be to keep all the Makefiles separate. I'd like some input as I'm going to be shifting focus to this until it's in a usable state!

Yes, I agree. This would be simpler.

[ken-matsui]

Starting from the dependency solver might be too much, so I would suggest starting with #991.

[Lazauya]

Starting from the dependency solver might be too much, so I would suggest starting with #991.

I think I want to start with building libraries before moving on to workspaces if you don't mind. I actually already have a simple MVP for library building but it probably doesn't match the project's expectations.

Currently the project supports no-build libraries, which is fine, but not good enough for easily porting old code to poac, which I feel should be a goal to make adoption easier.

Cargo makes you have a file named lib.rs which tells Cargo you're building a library. Similarly, my implementation searches for lib.cc (which is compiled the same way as main.cc until the end where we call ar) to determine if the poac package is a library. I don't think we should be worrying about shared libraries yet as mentioned above.

However, there are a few things I want to clear up. The only other discussion I found about library support was
here. You mention a libraries section of the poac.toml:

[libraries]
default = ["default1", "default2"]
option = ["option1", "option2"]

I'm not sure how this should fit into the implementation. In Cargo, unless I'm misunderstanding a core concept (very possible!), features are like macro #defines that allow you to conditionally include code in a build. Is this the function of libraries? The discussion was about building multiple libraries from one source. But It seems like maybe that's a feature better suited to workspaces? Any clarification would be appreciated.

[ken-matsui]

Starting from the dependency solver might be too much, so I would suggest starting with #991.

I think I want to start with building libraries before moving on to workspaces if you don't mind. I actually already have a simple MVP for library building but it probably doesn't match the project's expectations.

No problem, that makes sense. Library build is a great feature.

Currently the project supports no-build libraries, which is fine, but not good enough for easily porting old code to poac, which I feel should be a goal to make adoption easier.

Cargo makes you have a file named lib.rs which tells Cargo you're building a library. Similarly, my implementation searches for lib.cc (which is compiled the same way as main.cc until the end where we call ar) to determine if the poac package is a library. I don't think we should be worrying about shared libraries yet as mentioned above.

However, there are a few things I want to clear up. The only other discussion I found about library support was here. You mention a libraries section of the poac.toml:

[libraries]
default = ["default1", "default2"]
option = ["option1", "option2"]

I'm not sure how this should fit into the implementation. In Cargo, unless I'm misunderstanding a core concept (very possible!), features are like macro #defines that allow you to conditionally include code in a build. Is this the function of libraries? The discussion was about building multiple libraries from one source. But It seems like maybe that's a feature better suited to workspaces? Any clarification would be appreciated.

My initial idea was to allow users to include multiple libraries in a package (something Cargo does not support) in a similar way to binaries (which Cargo does support via src/bin). However, I like your suggestion of limiting packages to a single library and instead encouraging the use of workspaces, which aligns better with Cargo's design. I may have been considering edge cases with existing libraries, but I don’t recall specific issues. I think your approach provides a solid starting point without adding unnecessary complexity, and I hope we can proceed with it. If there is strong demand for supporting multiple libraries per package, we can consider adding it later.

Thanks!

[Lazauya]

Here's the PR for the library building: #1035

[Lazauya]

I've started looking at the recursive building aspect. I've considered two options, and I'm curious which one you think is the best:

1. We can simply cd into the package's directory and run poac build. This isn't too complicated, but it has a major flaw that I think rules it out. We'd have to make another pkg-config-like command that outputs the libraries to include.
2. With that in mind, I think the better but more complicated solution will be to load toml file for every dependency recursively and generate a BuildConfig etc. for everyone. That way we can use the already existing code to get the proper system libraries for linking.

One thing I'm not sure how to approach is version requirements. Since the package manager aspect isn't completed yet, how should we handle versions in git? Should we just treat different branches or tags as different packages and not worry about version satisfiability?