Skip to content

Commit

Permalink
Merge pull request #37 from orxfun/individual-test-methods-exposed
Browse files Browse the repository at this point in the history
individual test methods exposed
  • Loading branch information
orxfun authored Dec 12, 2024
2 parents 0b3841f + 3599177 commit 5c28b01
Show file tree
Hide file tree
Showing 13 changed files with 522 additions and 35 deletions.
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "orx-pinned-vec"
version = "3.10.1"
version = "3.11.0"
edition = "2021"
authors = ["orxfun <[email protected]>"]
description = "`PinnedVec` trait defines the interface for vectors which guarantee that elements added to the vector are pinned to their memory locations unless explicitly changed."
Expand Down
45 changes: 26 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,21 +5,25 @@

`PinnedVec` trait defines the interface for vectors which guarantee that elements added to the vector are pinned to their memory locations unless explicitly changed.

## A. Pinned Elements Guarantee
## Pinned Elements Guarantees

A `PinnedVec` guarantees that positions of its elements **do not change implicitly**.

To be specific, let's assume that a pinned vector currently has `n` elements:
Assume that we have a pinned vector which is at a state containing **n** elements. Pinned vector guarantees can then be summarized as follows.

| Method | Expected Behavior |
| -------- | ------- |
| `push(new_element)` | does not change the memory locations of the `n` elements |
| `extend_from_slice(slice)` | does not change the memory locations of the first `n` elements |
| `insert(a, new_element)` | does not change the memory locations of the first `a` elements, where `a <= n`; elements to the right of the inserted element might be changed, commonly shifted to right |
| `pop()` | does not change the memory locations of the first `n-1` elements, the `n`-th element is removed |
| `remove(a)` | does not change the memory locations of the first `a` elements, where `a < n`; elements to the right of the removed element might be changed, commonly shifted to left |
| `truncate(a)` | does not change the memory locations of the first `a` elements, where `a < n` |
||
* **G1: pinned elements on growth at the end**. This is one of the critical guarantees that the pinned vectors provide. We are adding **m** ≥ 1 elements to the end of the vector, and hence the vector reaches a length of **n + m**. Pinned vector guarantees that the memory locations of the **n** elements will not change due to this mutation.
* *Some such example mutation methods are **push**, **extend** or **extend_from_slice**.*
* *However, **insert** method is not covered since it is not an addition to the end of the vector.*
* *Notice that the standard vector does not satisfy this requirement.*
* *For many special data structures, such as concurrent collections or self referential collections, this is the necessary and sufficient pinned element guarantee.*
* **G2: pinned elements on removals from the end**. In this case, we are removing **m**[1, n] elements from the end of the vector leading to the final vector length of **n - m**. Pinned vector guarantees that memory locations of these remaining **n - m** elements do not change.
* *Some such example methods are **pop**, **truncate** or **clear**.*
* **G3: pinned prior elements on insertions to arbitrary position**. Assume we are adding **m** ≥ 1 elements; however, not necessarily to the end of the vector this time. The earliest position of the inserted elements is **p** < (n-1). In this case, pinned vector guarantees that memory locations of the elements at positions 0..(p-1) will remain intact.
* *The example method is the **insert** method.*
* **G4: pinned prior elements in removals from arbitrary position**. Lastly, assume that we are removing **m**[1, n] elements from the arbitrary positions of the vector leading to a final vector length of **n - m**. Let **p** be the earliest position of the removed elements. Pinned vector then guarantees that memory locations of the elements at positions 0..(p-1) will remain intact.
* *The example method is the **remove** method.*

## Motivation & Examples

Note that this eliminates a certain set of errors that are easy to make in some languages and forbidden by the borrow checker in rust. Consider, for example, the classical example that does not compile in rust. The reason this code has a bug is due to the fact that the elements of the standard vector are not pinned to their memory locations and it is possible that the `push` leads to changing them all together. Using a pinned vector, on the other hand, this would be a memory safe operation.

Expand All @@ -35,6 +39,13 @@ vec.push(4);
// assert_eq!(ref_to_first, &0);
```

An example data structure relying on pinned vector guarantees to enable immutable push operation is the [ImpVec](https://crates.io/crates/orx-imp-vec) which is also discussed in this [article](https://orxfun.github.io/orxfun-notes/#/imp-vec-motivation-2024-10-03).

Furthermore, self-referential-collections following thin references rather than wide pointers or index numbers rely on the consistency of memory positions of its elements. Pinned vectors again come in very useful for them. You may find an efficient and capable [LinkedList](https://crates.io/crates/orx-linked-list) implementation built on top of the pinned element guarantees.

Finally, it is easy to observe that pinned element guarantees make it extremely more convenient and safer to achieve data structures which enable concurrent growth. There are various concurrent data structures relying on pinned vectors, such as [ConcurrentVec](https://crates.io/crates/orx-concurrent-vec), [ConcurrentBag](https://crates.io/crates/orx-concurrent-bag) and [ConcurrentOrderedBag](https://crates.io/crates/orx-concurrent-ordered-bag). These concurrent collections play an essential role in efficiently collecting results of a parallel computation by the parallel iterator [Par](https://crates.io/crates/orx-parallel).

## Testing the Guarantees

`PinnedVec` trait on its own cannot provide the pinned element guarantee; hence, it could be considered as a marker trait.

Expand All @@ -50,17 +61,13 @@ This function performs an extensive test on the specific implementation `P` and

Note that `std::vec::Vec` does not provide the pinned elements during growth guarantee. You may find a wrapper struct `JustVec` which is nothing but the standard vec here: [src/pinned_vec_tests/test_all.rs](https://github.com/orxfun/orx-pinned-vec/blob/main/src/pinned_vec_tests/test_all.rs). As expected, `test_pinned_vec` method fails for this struct.

## B. Motivation

There are various situations where pinned elements are critical.
## Implementations

* It is critical in enabling **efficient, convenient and safe self-referential collections** with thin references, see [`SelfRefCol`](https://crates.io/crates/orx-selfref-col) for details, and its special cases such as [`LinkedList`](https://crates.io/crates/orx-linked-list).
* It is important for **concurrent** programs as it eliminates safety concerns related with elements implicitly carried to different memory locations. This helps reducing and dealing with the complexity of concurrency, and leads to efficient concurrent data structures. See [`ConcurrentIter`](https://crates.io/crates/orx-concurrent-iter), [`ConcurrentBag`](https://crates.io/crates/orx-concurrent-bag) or [`ConcurrentOrderedBag`](https://crates.io/crates/orx-concurrent-ordered-bag) for such concurrent data structures which are conveniently built on the pinned element guarantees of pinned vectors.
* It is essential in allowing an **immutable push** vector; i.e., [`ImpVec`](https://crates.io/crates/orx-imp-vec). This is a very useful operation when the desired collection is a bag or a container of things, rather than having a collective meaning. In such cases, `ImpVec` allows avoiding certain borrow checker complexities, heap allocations and wide pointers such as `Box` or `Rc` or etc.
[`SplitVec`](https://crates.io/crates/orx-split-vec) and [`FixedVec`](https://crates.io/crates/orx-fixed-vec) are two efficient PinnedVec implementations.

## C. Implementations
## Contributing

[`SplitVec`](https://crates.io/crates/orx-split-vec) and [`FixedVec`](https://crates.io/crates/orx-fixed-vec) are two efficient implementations.
Contributions are welcome! If you notice an error, have a question or think something could be improved, please open an [issue](https://github.com/orxfun/orx-pinned-vec/issues/new) or create a PR.

## License

Expand Down
5 changes: 3 additions & 2 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ mod concurrent_pinned_vec;
mod errors;
mod into_concurrent_pinned_vec;
mod pinned_vec;
mod pinned_vec_tests;
/// Tests methods to validate pinned element guarantees of an implementing type.
pub mod pinned_vec_tests;
/// Utility functions to make PinnedVec implementations more convenient.
pub mod utils;

Expand All @@ -28,4 +29,4 @@ pub use concurrent_pinned_vec::ConcurrentPinnedVec;
pub use errors::PinnedVecGrowthError;
pub use into_concurrent_pinned_vec::IntoConcurrentPinnedVec;
pub use pinned_vec::PinnedVec;
pub use pinned_vec_tests::test_all::test_pinned_vec;
pub use pinned_vec_tests::test_pinned_vec;
10 changes: 10 additions & 0 deletions src/pinned_vec_tests/extend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@ use super::refmap::RefMap;
use crate::PinnedVec;
use alloc::vec::Vec;

/// Tests the pinned vector guarantee on extending the vector;
/// panics if the pinned vector implementation `P` does not satisfy the required condition.
///
/// Tested pinned element guarantee:
///
/// * **G1: pinned elements on growth at the end**. This is one of the critical guarantees that the pinned vectors provide. We are adding **m** ≥ 1 elements to the end of the vector, and hence the vector reaches a length of **n + m**. Pinned vector guarantees that the memory locations of the **n** elements will not change due to this mutation.
/// * *Some such example mutation methods are **push**, **extend** or **extend_from_slice**.*
/// * *However, **insert** method is not covered since it is not an addition to the end of the vector.*
/// * *Notice that the standard vector does not satisfy this requirement.*
/// * *For many special data structures, such as concurrent collections or self referential collections, this is the necessary and sufficient pinned element guarantee.*
pub fn extend<P: PinnedVec<usize> + Sized>(pinned_vec: P, max_allowed_test_len: usize) -> P {
let mut vec = pinned_vec;
vec.clear();
Expand Down
9 changes: 9 additions & 0 deletions src/pinned_vec_tests/insert.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,19 @@
use super::refmap::RefMap;
use crate::PinnedVec;

/// Tests the pinned vector guarantee on removing elements from arbitrary positions;
/// panics if the pinned vector implementation `P` does not satisfy the required condition.
///
/// Tested pinned element guarantee:
///
/// * **G3: pinned prior elements on insertions to arbitrary position**. Assume we are adding **m** ≥ 1 elements; however, not necessarily to the end of the vector this time. The earliest position of the inserted elements is **p** < (n-1). In this case, pinned vector guarantees that memory locations of the elements at positions 0..(p-1) will remain intact.
/// * *The example method is the **insert** method.*
pub fn insert<P: PinnedVec<usize>>(pinned_vec: P, max_allowed_test_len: usize) -> P {
let mut vec = pinned_vec;
vec.clear();

// insert at the end

let first_half = max_allowed_test_len / 2;

let mut refmap = RefMap::new(200, first_half);
Expand Down
10 changes: 9 additions & 1 deletion src/pinned_vec_tests/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,19 @@ mod push;
mod refmap;
mod remove;
mod slices;
pub mod test_all;
mod test_all;
mod truncate;
mod unsafe_writer;

#[cfg(test)]
mod helpers;
#[cfg(test)]
pub(crate) mod testvec;

pub use extend::extend;
pub use insert::insert;
pub use pop::pop;
pub use push::push;
pub use remove::remove;
pub use test_all::test_pinned_vec;
pub use truncate::truncate;
7 changes: 7 additions & 0 deletions src/pinned_vec_tests/pop.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
use super::refmap::RefMap;
use crate::PinnedVec;

/// Tests the pinned vector guarantee on removing elements from the end;
/// panics if the pinned vector implementation `P` does not satisfy the required condition.
///
/// Tested pinned element guarantee:
///
/// * **G2: pinned elements on removals from the end**. In this case, we are removing **m** ∈ [1, n] elements from the end of the vector leading to the final vector length of **n - m**. Pinned vector guarantees that memory locations of these remaining **n - m** elements do not change.
/// * *Some such example methods are **pop**, **truncate** or **clear**.*
pub fn pop<P: PinnedVec<usize>>(pinned_vec: P, max_allowed_test_len: usize) -> P {
let mut vec = pinned_vec;
vec.clear();
Expand Down
10 changes: 10 additions & 0 deletions src/pinned_vec_tests/push.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
use super::refmap::RefMap;
use crate::PinnedVec;

/// Tests the pinned vector guarantee on extending the vector;
/// panics if the pinned vector implementation `P` does not satisfy the required condition.
///
/// Tested pinned element guarantee:
///
/// * **G1: pinned elements on growth at the end**. This is one of the critical guarantees that the pinned vectors provide. We are adding **m** ≥ 1 elements to the end of the vector, and hence the vector reaches a length of **n + m**. Pinned vector guarantees that the memory locations of the **n** elements will not change due to this mutation.
/// * *Some such example mutation methods are **push**, **extend** or **extend_from_slice**.*
/// * *However, **insert** method is not covered since it is not an addition to the end of the vector.*
/// * *Notice that the standard vector does not satisfy this requirement.*
/// * *For many special data structures, such as concurrent collections or self referential collections, this is the necessary and sufficient pinned element guarantee.*
pub fn push<P: PinnedVec<usize>>(pinned_vec: P, max_allowed_test_len: usize) -> P {
let mut vec = pinned_vec;
vec.clear();
Expand Down
7 changes: 7 additions & 0 deletions src/pinned_vec_tests/remove.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
use super::refmap::RefMap;
use crate::PinnedVec;

/// Tests the pinned vector guarantee on removing elements from arbitrary positions;
/// panics if the pinned vector implementation `P` does not satisfy the required condition.
///
/// Tested pinned element guarantee:
///
/// * **G4: pinned prior elements in removals from arbitrary position**. Lastly, assume that we are removing **m** ∈ [1, n] elements from the arbitrary positions of the vector leading to a final vector length of **n - m**. Let **p** be the earliest position of the removed elements. Pinned vector then guarantees that memory locations of the elements at positions 0..(p-1) will remain intact.
/// * *The example method is the **remove** method.*
pub fn remove<P: PinnedVec<usize>>(pinned_vec: P, max_allowed_test_len: usize) -> P {
let mut vec = pinned_vec;
vec.clear();
Expand Down
36 changes: 30 additions & 6 deletions src/pinned_vec_tests/test_all.rs
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,36 @@ mod tests {
}

impl<T> PinnedVec<T> for JustVec<T> {
type Iter<'a> = core::slice::Iter<'a, T> where T: 'a, Self: 'a;
type IterMut<'a> = core::slice::IterMut<'a, T> where T: 'a, Self: 'a;
type IterRev<'a> = Rev<core::slice::Iter<'a, T>> where T: 'a, Self: 'a;
type IterMutRev<'a> = Rev<core::slice::IterMut<'a, T>> where T: 'a, Self: 'a;
type SliceIter<'a> = Option<&'a [T]> where T: 'a, Self: 'a;
type SliceMutIter<'a> = Option<&'a mut [T]> where T: 'a, Self: 'a;
type Iter<'a>
= core::slice::Iter<'a, T>
where
T: 'a,
Self: 'a;
type IterMut<'a>
= core::slice::IterMut<'a, T>
where
T: 'a,
Self: 'a;
type IterRev<'a>
= Rev<core::slice::Iter<'a, T>>
where
T: 'a,
Self: 'a;
type IterMutRev<'a>
= Rev<core::slice::IterMut<'a, T>>
where
T: 'a,
Self: 'a;
type SliceIter<'a>
= Option<&'a [T]>
where
T: 'a,
Self: 'a;
type SliceMutIter<'a>
= Option<&'a mut [T]>
where
T: 'a,
Self: 'a;

fn index_of(&self, data: &T) -> Option<usize> {
crate::utils::slice::index_of(&self.0, data)
Expand Down
36 changes: 30 additions & 6 deletions src/pinned_vec_tests/testvec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,36 @@ impl<T> IntoIterator for TestVec<T> {
}

impl<T> PinnedVec<T> for TestVec<T> {
type Iter<'a> = core::slice::Iter<'a, T> where T: 'a, Self: 'a;
type IterMut<'a> = core::slice::IterMut<'a, T> where T: 'a, Self: 'a;
type IterRev<'a> = Rev<core::slice::Iter<'a, T>> where T: 'a, Self: 'a;
type IterMutRev<'a> = Rev<core::slice::IterMut<'a, T>> where T: 'a, Self: 'a;
type SliceIter<'a> = Option<&'a [T]> where T: 'a, Self: 'a;
type SliceMutIter<'a> = Option<&'a mut [T]> where T: 'a, Self: 'a;
type Iter<'a>
= core::slice::Iter<'a, T>
where
T: 'a,
Self: 'a;
type IterMut<'a>
= core::slice::IterMut<'a, T>
where
T: 'a,
Self: 'a;
type IterRev<'a>
= Rev<core::slice::Iter<'a, T>>
where
T: 'a,
Self: 'a;
type IterMutRev<'a>
= Rev<core::slice::IterMut<'a, T>>
where
T: 'a,
Self: 'a;
type SliceIter<'a>
= Option<&'a [T]>
where
T: 'a,
Self: 'a;
type SliceMutIter<'a>
= Option<&'a mut [T]>
where
T: 'a,
Self: 'a;

fn index_of(&self, data: &T) -> Option<usize> {
crate::utils::slice::index_of(&self.0, data)
Expand Down
7 changes: 7 additions & 0 deletions src/pinned_vec_tests/truncate.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
use super::refmap::RefMap;
use crate::PinnedVec;

/// Tests the pinned vector guarantee on removing elements from the end;
/// panics if the pinned vector implementation `P` does not satisfy the required condition.
///
/// Tested pinned element guarantee:
///
/// * **G2: pinned elements on removals from the end**. In this case, we are removing **m** ∈ [1, n] elements from the end of the vector leading to the final vector length of **n - m**. Pinned vector guarantees that memory locations of these remaining **n - m** elements do not change.
/// * *Some such example methods are **pop**, **truncate** or **clear**.*
pub fn truncate<P: PinnedVec<usize>>(pinned_vec: P, max_allowed_test_len: usize) -> P {
let mut vec = pinned_vec;
vec.clear();
Expand Down
Loading

0 comments on commit 5c28b01

Please sign in to comment.