[Bug] Signature created outside VM is not valid in circuit

[Scratch-net]

🐛 Bug Report

Hey, I'm trying to sign an arbitrary string (byte array) outside VM (in SDK) and validate it inside Aleo transition but it always fails.

Steps to Reproduce

First, I convert a string to a field in the same way it's done in VM:

(i'm tapping into aleo-wasm, also new to Rust, so my code is not perfect)

        let s = "test";
        let bytes = s.as_bytes();
        let bits = &bytes.to_bits_le();
        let fields = bits.chunks(FieldNative::size_in_data_bits()).map(FieldNative::from_bits_le).collect::<Result<Vec<FieldNative>,anyhow::Error>>().unwrap();

the resulting field is 1953719668field

When I use SDK I just do

const bytes = textEncoder.encode('test')
const sign = account.sign(bytes)

The example signature for the following private key
APrivateKey1zkp5gkX61etNqNTibpEaZD7tC8FJV2NxYQQtwyzKgGY9vwD
and its address
aleo1zen02agt7udrz24ffqddfe633s8cuecjy4p4qg03p5vfukvrjqfqvx2tyc
and the string/field above is :
sign1mdks5v7yypepq6dqpk0zw3vlq3dha7l9wq6eghjdyvg8ehpwxgp7x0sj2mkhj9ywd5qltt7nuaeg55drmglju3p6uq97w5qfejlg6pr9c6pajt957g355dkue2l8mq00x7ur3zrlavsnvzyradh6kkcrqdpvfyd7pzwmn5s6f2v0mh6qcf92sqx7ng8lnxraqufle0dvkq2pz7na86y

which is valid if I verify in in both SDK and in sample Rust code
It's also valid if I verify it against field itself:

        let field = FieldNative::from_str("1953719668field").unwrap();
        let fields = vec![field];
        let signature = SignatureNative::from_str("sign1mdks5v7yypepq6dqpk0zw3vlq3dha7l9wq6eghjdyvg8ehpwxgp7x0sj2mkhj9ywd5qltt7nuaeg55drmglju3p6uq97w5qfejlg6pr9c6pajt957g355dkue2l8mq00x7ur3zrlavsnvzyradh6kkcrqdpvfyd7pzwmn5s6f2v0mh6qcf92sqx7ng8lnxraqufle0dvkq2pz7na86y").unwrap()
        let address = Address::from_str("aleo1zen02agt7udrz24ffqddfe633s8cuecjy4p4qg03p5vfukvrjqfqvx2tyc").unwrap();
        assert!(signature.verify(&address, &fields));

BUT

If I put it in leo transition:

    transition verify_field(public s: signature, public a: address, public v: field) ->bool {
        return signature::verify(s, a, v);
    }

with these inputs:

[verify_field]
public s: signature = sign1mdks5v7yypepq6dqpk0zw3vlq3dha7l9wq6eghjdyvg8ehpwxgp7x0sj2mkhj9ywd5qltt7nuaeg55drmglju3p6uq97w5qfejlg6pr9c6pajt957g355dkue2l8mq00x7ur3zrlavsnvzyradh6kkcrqdpvfyd7pzwmn5s6f2v0mh6qcf92sqx7ng8lnxraqufle0dvkq2pz7na86y;
public a: address = aleo1zen02agt7udrz24ffqddfe633s8cuecjy4p4qg03p5vfukvrjqfqvx2tyc;
public v: field = 1953719668field;

It returns false

I'm out of ideas, maybe you have any

[d0cd]

See this comment for a sample usage.

ProvableHQ/sdk#748 (comment)

[Scratch-net]

Sorry @d0cd this is not helpful
There is no way to sign anything but a byte array in SDK, so I can't use structures

[vicsn]

@onetrickwolf see the above discussion, do we have an existing example or test of creating a signature using the SDK's aleo-wasm and verifying it in a transition? If not can you make one whenever you have time?

[iamalwaysuncomfortable]

The SDK code for signing runs the signature algorithm outside of the context of a circuit. However when running a circuit via a leo function, and given the function executes properly and gives you a false there is no problem here.

What I imagine is happening here is that Leo types like Field Scalar Group etc. have a canonical byte encoding that is different from what you would get if you do:

const bytes = textEncoder.encode('123Field')
const sign = account.sign(bytes)

The code above will create utf-8 bytes which will lead to a different byte encoding than the 123Field type within Leo. And this why this function is likely failing:

    transition verify_field(public s: signature, public a: address, public v: field) ->bool {
        return signature::verify(s, a, v);
    }

Currently there's not yet a function that will first try to serialize these types into the correct byte encoding in the SDK, but I'll put a ticket in to support this.

[Scratch-net]

@iamalwaysuncomfortable I've traced what the word 'test' turns into when being signed and it turns into field 1953719668field which is when verified on its own out of the circuit returns true. There are examples of that above.
However if I put the very same field in the circuit it fails. That's what confuses me

[Scratch-net]

To confirm, 1953719668 is 0x74736574 which when turned to bytes gets you 'tset' which is 'test' in LE

[iamalwaysuncomfortable]

@iamalwaysuncomfortable I've traced what the word 'test' turns into when being signed and it turns into field 1953719668field which is when verified on its own out of the circuit returns true. There are examples of that above.
However if I put the very same field in the circuit it fails. That's what confuses me

Can you post your Leo code + the leo run invocation you're using? This will help us reproduce this.

[Scratch-net]

Sure @iamalwaysuncomfortable it's the same code that's in the first message though

1. raw field validation in aleo-wasm (I put this code in signature.rs to_string method so that I could call it from TS tests):

use crate::{
    account::{Address, PrivateKey},
    types::{SignatureNative, FieldNative},
};

use core::{fmt, ops::Deref, str::FromStr};
use rand::{rngs::StdRng, SeedableRng};
use snarkvm_console::account::{FromBits, SizeInDataBits, ToFields};
use wasm_bindgen::prelude::*;
use snarkvm_wasm::ToBits;
use std::convert::TryFrom;

...

        let field = FieldNative::from_str("1953719668field").unwrap();
        let fields = vec![field];
        let signature = SignatureNative::from_str("sign1mdks5v7yypepq6dqpk0zw3vlq3dha7l9wq6eghjdyvg8ehpwxgp7x0sj2mkhj9ywd5qltt7nuaeg55drmglju3p6uq97w5qfejlg6pr9c6pajt957g355dkue2l8mq00x7ur3zrlavsnvzyradh6kkcrqdpvfyd7pzwmn5s6f2v0mh6qcf92sqx7ng8lnxraqufle0dvkq2pz7na86y").unwrap();
        let address = Address::from_str("aleo1zen02agt7udrz24ffqddfe633s8cuecjy4p4qg03p5vfukvrjqfqvx2tyc").unwrap();
        assert!(signature.verify(&address, &fields)); // all good here

hello.aleo

// The 'hello' program.
program hello.aleo {
    transition verify_field(public s: signature, public a: address, public v: field) ->bool {
            return signature::verify(s, a, v);
        }
}

command

leo run verify_field sign1mdks5v7yypepq6dqpk0zw3vlq3dha7l9wq6eghjdyvg8ehpwxgp7x0sj2mkhj9ywd5qltt7nuaeg55drmglju3p6uq97w5qfejlg6pr9c6pajt957g355dkue2l8mq00x7ur3zrlavsnvzyradh6kkcrqdpvfyd7pzwmn5s6f2v0mh6qcf92sqx7ng8lnxraqufle0dvkq2pz7na86y aleo1zen02agt7udrz24ffqddfe633s8cuecjy4p4qg03p5vfukvrjqfqvx2tyc 1953719668field

and its output

       Leo ✅ Compiled 'main.leo' into Aleo instructions

⛓  Constraints

 •  'hello.aleo/verify_field' - 6,786 constraints (called 1 time)

➡️  Output

 • false

       Leo ✅ Finished 'hello.aleo/verify_field'