Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support List(T) passthrough #436

Open
wants to merge 1,000 commits into
base: master
Choose a base branch
from

Conversation

ErikMcClure
Copy link

This is an implementation of capnproto/capnproto#1807 for allowing List(T) in schemas. Essentially, it is treated as an AnyList parameter on the wire, but the type information is preserved in the schema, which allows the C++ version to implement a proxy List(T) over the AnyList that enforces the correct type usage.

I could not find an equivalent to capnp-c++'s AnyList class, it appears that AnyLists are simply converted to any_pointers and it is left to the user to cast them appropriately. So, this change simply replaces rejecting a List(AnyPointer) by replacing it with an any_pointer, the same way AnyList is handled. This change doesn't break any existing tests, but also does not add any tests either, because any test for this feature would have to be gated behind whatever version of capnp ends up incorporating List(T) support.

It may be possible to restrict the acceptance of List(AnyPointer) to only cases where the any_pointer is an actual parameter (which is what was done in the C++ version) but I wasn't sure how to do that.

dwrensha and others added 30 commits November 15, 2022 08:38
…o-util-0.7.4

Update tokio-util requirement from 0.6.0 to 0.7.4
…s-by-value

remove needless pass by value
…is_empty

implement 'is_empty' where it's useful,
use 'Self' keyword to refer to own type
use 'Self' keyword to refer to own type
The MSRV (minimum supported Rust version) is the earliest version capable of
building a project. In the case of the capnp crate, the MSRV is 1.65.0, due to
the use of GATs.

Having the MSRV in Cargo.toml helps display better error messages when the Rust
version is too old to build a project. eg:
> error: package `capnp v0.15.0 (/Users/maxime/Code/capnproto-rust/capnp)`
> cannot be built because it requires rustc 1.65.0 or newer, while the
> currently active rustc version is 1.64.0

Without it, the error messages are more confusing.
> error[E0658]: generic associated types are unstable
>   --> capnp/src/traits.rs:73:5
>    |
> 73 |     type Builder<'a>: FromStructBuilder<'a> + HasStructSize;
>   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   |
>   = note: see issue #44265 <rust-lang/rust#44265> for more information

Cargo.toml docs: https://doc.rust-lang.org/cargo/reference/manifest.html#the-rust-version-field
Specify the minimum supported Rust version for capnp crate
dwrensha and others added 27 commits August 29, 2023 15:21
I testing this change on the following simple program:

```
pub mod primlist_capnp {
    include!(concat!(env!("OUT_DIR"), "/primlist_capnp.rs"));
}

pub fn main() {
    let mut scratch_space = vec![0u8; 16000];

    let mut input: Vec<f64>= vec![0.0;1500];

    let mut cc = 1.1;
    for _ in 0 .. 500000 {
        for inp in &mut input[..] {
            *inp = cc;
            cc += 0.02;
        }
        cc -= input.len() as f64 * 0.02;
        cc += 0.001;
        let allocator = capnp::message::ScratchSpaceHeapAllocator::new(&mut scratch_space);
        let mut message = ::capnp::message::Builder::new(allocator);
        let root : primlist_capnp::foo::Builder = message.init_root();
        let mut list = root.init_float_list(input.len() as u32);

        for (i,n) in input.iter().enumerate() {
            list.set(i as u32,*n);
        }

        for (i,n) in input.iter().enumerate() {
            assert_eq!(*n, list.get(i as u32));
        }
    }
}
```

Adding the #[inline] attribute cut the run time from ~2.5 seconds to about 1.0 seconds.
Makes text::Reader a wrapper around &[u8] and defers
utf-8 validation until `to_str()` or `to_string()` is called.

This means that users will only need to pay the utf8 validation
cost if they actually need it. This also allows users to access
(broken) non-utf8 Text data. Previously such data was inaccessible.
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
The idea here is to lower the likelihood of any performance degradation
caused by the changes in the 0.18 release.
@dwrensha
Copy link
Member

dwrensha commented Oct 3, 2023

Thanks. This looks okay to me, but I'd like to wait until upstream actually adds this feature before we add support here.

It may be possible to restrict the acceptance of List(AnyPointer) to only cases where the any_pointer is an actual parameter (which is what was done in the C++ version) but I wasn't sure how to do that.

Hm... yeah TypedAnyList may be possible in Rust via traits with associated types. But I do worry that making it work would add a lot of complexity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.