Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core Lib Documentation: byte_array module #6653

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
193 changes: 164 additions & 29 deletions corelib/src/byte_array.cairo
Original file line number Diff line number Diff line change
@@ -1,3 +1,44 @@
//! `ByteArray` is designed to handle large sequences of bytes with operations like appending,
//! concatenation, and accessing individual bytes. It uses a structure that combines an `Array` of
//! `bytes31` for full words and a `felt252` for handling partial words, optimizing for both space
//! and performance.
//!
//! # Examples
//!
//! Creating a new `ByteArray` and appending a single byte with [`ByteArrayTrait::append_byte`]:
//!
//! ```
//! let mut ba: ByteArray = "";
//! ba.append_byte(0x41); // Appending a single byte 'A'
//! ```
//!
//! Appending another `ByteArray` with [`ByteArrayTrait::append`]:
//!
//! ```
//! let mut ba: ByteArray = "";
//! let mut other_ba: ByteArray = "";
//! ba.append(@other_ba);
//! ```
//!
//! Accessing a single byte with [`ByteArrayTrait::at`]:
//!
//! ```
//! let mut ba: ByteArray = "";
//! ba.append_byte(0x41);
//! if let Option::Some(byte) = ba.at(0) {
//! assert!(byte == 0x41);
//! }
//! ```
//!
//! The `ByteArray` type also supports operations like addition, which can be used for
//! concatenation:
//!
//! ```
//! let mut ba: ByteArray = "";
//! let mut other_ba: ByteArray = "";
//! let concatenated = ba + other_ba;
//! ```

use crate::array::{ArrayTrait, SpanTrait};
#[allow(unused_imports)]
use crate::bytes_31::{
Expand All @@ -9,48 +50,61 @@ use crate::cmp::min;
#[allow(unused_imports)]
use crate::integer::{u128_safe_divmod, U32TryIntoNonZero};
use crate::option::OptionTrait;
use crate::traits::{Into, TryInto};
#[allow(unused_imports)]
use crate::serde::Serde;
use crate::traits::{Into, TryInto};
#[allow(unused_imports)]
use crate::zeroable::NonZeroIntoImpl;

/// A magic constant for identifying serialization of ByteArrays. An array of felt252s with this
/// magic as one of the felt252s indicates that right after it you should expect a serialized
/// ByteArray. This is currently used mainly for prints and panics.
/// A magic constant for identifying serialization of `ByteArray` variables. An array of `felt252`
/// with this magic value as one of the `felt252` indicates that you should expect right after it a
/// serialized `ByteArray`. This is currently used mainly for prints and panics.
pub const BYTE_ARRAY_MAGIC: felt252 =
0x46a6158a16a947e5916b2a2ca68501a45e93d7110e81aa2d6438b1c57c879a3;
const BYTES_IN_U128: usize = 16;
const BYTES_IN_BYTES31_MINUS_ONE: usize = BYTES_IN_BYTES31 - 1;

// TODO(yuval): don't allow creation of invalid ByteArray?
/// A dynamic, growable array of bytes where bytes are stored in multiples of 31 bytes,
/// with a `felt252` pending word used for bytes that don't complete a full `bytes31` word.
#[derive(Drop, Clone, PartialEq, Serde, Default)]
pub struct ByteArray {
// Full "words" of 31 bytes each. The first byte of each word in the byte array
// is the most significant byte in the word.
/// `data` field is an array of full "words" of 31 bytes each.
/// The first byte of each word in the byte array is the most significant byte in the word.
pub(crate) data: Array<bytes31>,
// This felt252 actually represents a bytes31, with < 31 bytes.
// It is represented as a felt252 to improve performance of building the byte array.
// The number of bytes in here is specified in `pending_word_len`.
// The first byte is the most significant byte among the `pending_word_len` bytes in the word.
/// `pending_word` field is a `felt252` that actually represents a `bytes31`, with < 31 bytes.
/// It is represented as a `felt252` to improve performance of building the byte array.
/// The first byte is the most significant byte among the `pending_word_len` bytes in the word.
pub(crate) pending_word: felt252,
// Should be in range [0, 30].
/// `pending_word_len` represents the number of bytes in `pending_word`.
/// Its value should be in the range [0, 30].
pub(crate) pending_word_len: usize,
}

pub(crate) impl ByteArrayStringLiteral of crate::string::StringLiteral<ByteArray>;

/// Basic trait for the `ByteArray` type.
#[generate_trait]
pub impl ByteArrayImpl of ByteArrayTrait {
// TODO(yuval): add a `new` function for initialization.

/// Appends a single word of `len` bytes to the end of the ByteArray.
/// Note: this function assumes that:
/// Appends a single word of `len` bytes to the end of the `ByteArray`.
///
/// This function assumes that:
/// 1. `word` could be validly converted to a `bytes31` which has no more than `len` bytes
/// of data.
/// 2. len <= BYTES_IN_BYTES31.
/// If these assumptions are not met, it can corrupt the ByteArray. Thus, this should be a
///
/// If these assumptions are not met, it can corrupt the `ByteArray`. Thus, this should be a
/// private function. We could add masking/assertions but it would be more expensive.
///
/// # Examples
///
/// ```
/// let mut ba = "";
/// ba.append_word('word', 4);
/// assert!(ba == "word");
/// ```
fn append_word(ref self: ByteArray, word: felt252, len: usize) {
if len == 0 {
return;
Expand Down Expand Up @@ -88,7 +142,15 @@ pub impl ByteArrayImpl of ByteArrayTrait {
self.pending_word_len = split_index;
}

/// Appends a byte array to the end of `self`.
/// Appends a `ByteArray` to the end of another `ByteArray`.
///
/// # Examples
///
/// ```
/// let mut ba: ByteArray = "1";
/// ba.append(@"2");
/// assert!(ba == "12");
/// ```
fn append(ref self: ByteArray, mut other: @ByteArray) {
let mut other_data = other.data.span();

Expand Down Expand Up @@ -141,14 +203,31 @@ pub impl ByteArrayImpl of ByteArrayTrait {
self.append_word(*other.pending_word, *other.pending_word_len);
}

/// Concatenates two byte arrays and returns the result.
/// Concatenates two `ByteArray` and returns the result.
///
/// # Examples
///
/// ```
/// let ba = "1";
/// let other_ba = "2";
/// let result = ByteArrayTrait::concat(@ba, @other_ba);
/// assert!(result == "12");
/// ```
fn concat(left: @ByteArray, right: @ByteArray) -> ByteArray {
let mut result = left.clone();
result.append(right);
result
}

/// Appends a single byte to the end of `self`.
/// Appends a single byte to the end of the `ByteArray`.
///
/// # Examples
///
/// ```
/// let mut ba = "";
/// ba.append_byte(0);
/// assert!(ba == "0");
/// ```
fn append_byte(ref self: ByteArray, byte: u8) {
if self.pending_word_len == 0 {
self.pending_word = byte.into();
Expand All @@ -170,12 +249,30 @@ pub impl ByteArrayImpl of ByteArrayTrait {
self.pending_word_len = 0;
}

/// Returns the length of the `ByteArray` as a `usize` value.
///
/// # Examples
///
/// ```
/// let ba: ByteArray = "byte array";
/// let len = ba.len();
/// assert!(len == 10);
/// ```
#[must_use]
fn len(self: @ByteArray) -> usize {
self.data.len() * BYTES_IN_BYTES31.into() + (*self.pending_word_len).into()
}

/// Returns the byte at the given index, or None if the index is out of bounds.
/// Returns an option of the byte at the given index of `self`
/// or `Option::None` if the index is out of bounds.
///
/// # Examples
///
/// ```
/// let ba: ByteArray = "byte array";
/// let byte = ba.at(0).unwrap();
/// assert!(byte == 98);
/// ```
fn at(self: @ByteArray, index: usize) -> Option<u8> {
let (word_index, index_in_word) = DivRem::div_rem(
index, BYTES_IN_BYTES31.try_into().unwrap()
Expand All @@ -202,7 +299,15 @@ pub impl ByteArrayImpl of ByteArrayTrait {
Option::Some(self.data.at(word_index).at(index_from_lsb))
}

/// Returns a ByteArray with the reverse order of `self`.
/// Returns a `ByteArray` with the reverse order of `self`.
///
/// # Examples
///
/// ```
/// let ba: ByteArray = "123";
/// let rev_ba = ba.rev();
/// assert!(rev_ba == "321");
/// ```
fn rev(self: @ByteArray) -> ByteArray {
let mut result = Default::default();

Expand All @@ -220,10 +325,19 @@ pub impl ByteArrayImpl of ByteArrayTrait {
result
}

/// Appends the reverse of the given word to the end `self`.
/// Assumptions:
/// Appends the reverse of the given word to the end of `self`.
///
/// This function assumes that:
/// 1. len < 31
/// 2. word is validly convertible to bytes31 of length `len`.
///
/// # Examples
///
/// ```
/// let mut ba: ByteArray = "";
/// ba.append_word_rev('123', 3);
/// assert!(ba == "321");
/// ```
fn append_word_rev(ref self: ByteArray, word: felt252, len: usize) {
let mut index = 0;

Expand Down Expand Up @@ -256,11 +370,11 @@ pub impl ByteArrayImpl of ByteArrayTrait {

// === Helpers ===

/// Appends a single word of `len` bytes to the end of the ByteArray, assuming there
/// Appends a single word of `len` bytes to the end of the `ByteArray`, assuming there
/// is enough space in the pending word (`self.pending_word_len + len < BYTES_IN_BYTES31`).
///
/// `word` is of type felt252 but actually represents a bytes31.
/// It is represented as a felt252 to improve performance of building the byte array.
/// `word` is of type `felt252` but actually represents a `bytes31`.
/// It is represented as a `felt252` to improve performance of building the `ByteArray`.
#[inline]
fn append_word_fits_into_pending(ref self: ByteArray, word: felt252, len: usize) {
if self.pending_word_len == 0 {
Expand All @@ -279,7 +393,7 @@ pub impl ByteArrayImpl of ByteArrayTrait {
/// `split_index` is the number of bytes left in `self.pending_word` after this function.
/// This is the index of the split (LSB's index is 0).
///
/// Note: this function doesn't update the new pending length of self. It's the caller's
/// Note: this function doesn't update the new pending length of `self`. It's the caller's
/// responsibility.
#[inline]
fn append_split_index_lt_16(ref self: ByteArray, word: felt252, split_index: usize) {
Expand All @@ -300,7 +414,7 @@ pub impl ByteArrayImpl of ByteArrayTrait {
/// `split_index` is the number of bytes left in `self.pending_word` after this function.
/// This is the index of the split (LSB's index is 0).
///
/// Note: this function doesn't update the new pending length of self. It's the caller's
/// Note: this function doesn't update the new pending length of `self`. It's the caller's
/// responsibility.
#[inline]
fn append_split_index_16(ref self: ByteArray, word: felt252) {
Expand All @@ -314,7 +428,7 @@ pub impl ByteArrayImpl of ByteArrayTrait {
/// `split_index` is the number of bytes left in `self.pending_word` after this function.
/// This is the index of the split (LSB's index is 0).
///
/// Note: this function doesn't update the new pending length of self. It's the caller's
/// Note: this function doesn't update the new pending length of `self`. It's the caller's
/// responsibility.
#[inline]
fn append_split_index_gt_16(ref self: ByteArray, word: felt252, split_index: usize) {
Expand All @@ -328,13 +442,13 @@ pub impl ByteArrayImpl of ByteArrayTrait {
self.append_split(high_quotient.into(), right);
}

/// A helper function to append a remainder to self, by:
/// A helper function to append a remainder to `self`, by:
/// 1. completing `self.pending_word` to a full word using `complete_full_word`, assuming it's
/// validly convertible to a `bytes31` of length exactly `BYTES_IN_BYTES31 -
/// self.pending_word_len`.
/// 2. Setting `self.pending_word` to `new_pending`.
///
/// Note: this function doesn't update the new pending length of self. It's the caller's
/// Note: this function doesn't update the new pending length of `self`. It's the caller's
/// responsibility.
#[inline]
fn append_split(ref self: ByteArray, complete_full_word: felt252, new_pending: felt252) {
Expand All @@ -347,13 +461,34 @@ pub impl ByteArrayImpl of ByteArrayTrait {
}

impl ByteArrayAdd of Add<ByteArray> {
/// Concatenates two `ByteArray` and returns the resulting `ByteArray`.
///
/// # Examples
///
/// ```
/// let ba: ByteArray = "1";
/// let other_ba: ByteArray = "2";
/// let result = ba + other_ba;
/// assert!(result == "12")
/// ```
#[inline]
fn add(lhs: ByteArray, rhs: ByteArray) -> ByteArray {
ByteArrayTrait::concat(@lhs, @rhs)
}
}

#[feature("deprecated-op-assign-traits")]
impl ByteArrayAddEq of crate::traits::AddEq<ByteArray> {
/// Appends a `ByteArray` to another `ByteArray`.
///
/// # Examples
///
/// ```
/// let mut ba: ByteArray = "1";
/// let other_ba: ByteArray = "2";
/// ba += other_ba;
/// assert!(ba == "12");
/// ```
#[inline]
fn add_eq(ref self: ByteArray, other: ByteArray) {
self.append(@other);
Expand Down