Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: automatic conversion between Elixir Binary and Vec<u8> when using serde #668

Open
YizhePKU opened this issue Dec 14, 2024 · 2 comments
Labels

Comments

@YizhePKU
Copy link

YizhePKU commented Dec 14, 2024

Due to limitations of serde, Vec<u8> will serialize into "seq of u8", which is rendered into Elixir list instead of binary. There are libraries that force serde into treating Vec<u8> as bytes (e.g. serde_bytes, serde_with), but these libraries require users to annotate existing Rust code, which is not always possible or convenient.

Given that the goal of rustler is to make interop between Elixir and Rust as smooth as possible, I believe encoding Vec<u8> into binaries is a better default.

I propose the following breaking changes to rustler's serde conversion rules:

  1. Allow Elixir binary to be deserialized into "seq of u8". This means SerdeTerm<Vec<u8>> can be decoded from binary.
  2. Prefer to serialize "seq of u8" into binary when possible. This means SerdeTerm<Vec<u8>> will be encoded into binary, but SerdeTerm<Vec<i8>> will be encoded into list, as usual.

I'm currently working on an implementation. Any thoughts?

@YizhePKU
Copy link
Author

Now that I look at rustler's conversion table again, doesn't ruslter already claim that it encodes Vec<u8> into binary? If so, this might just be a bug fix rather than a new feature.

@filmor
Copy link
Member

filmor commented Dec 15, 2024

I'll have a look. The current serde suppprt is, apart from major refactorings and adjustments to make it work with modern rustler, basically still equivalent to serde_rustler. I didn't work on verifying or adjusting conversion rules, yet.

I would really like to be able to configure rustler in general and the serde support specifically at init time (which would also encompass a solution to #667), but I haven't gotten around prototyping this.

@filmor filmor added the serde label Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants