Skip to content

Commit

Permalink
major refactoring applied
Browse files Browse the repository at this point in the history 
The following major revision is applied to the pinned vector:
* pinned elements guarantee is formalized and documented,
* unsafe methods such as `clone` or `insert` are now safe. pinned vector on its own cannot build inter-element references; hence, these methods are not unsafe. this responsibility is passed to the struct enabling these references, namely, orx_selfref_col.
* in addition to being a marker trait, this crate now provides `test_pinned_vec` function which is essential in asserting that a pinned vector implementation satisfies the required guarantees.
  • Loading branch information
@Ugur-Arikan
Ugur-Arikan committed Feb 27, 2024
1 parent 9ba3256 commit 0c83db0
Show file tree
Hide file tree
Showing 22 changed files with 936 additions and 880 deletions.
2 changes: 2 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,5 @@ Cargo.lock

# MSVC Windows builds of rustc generate these, which store debugging information
*.pdb

.vscode/
6 changes: 4 additions & 2 deletions Cargo.toml
View file
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
[package]
name = "orx-pinned-vec"
version = "1.0.1"
version = "2.0.0"
edition = "2021"
authors = ["orxfun <[email protected]>"]
description = "`PinnedVec` trait defines the interface for vectors which guarantee that elements are pinned to their memory locations with the aim to enable convenient self-referential collections."
description = "`PinnedVec` trait defines the interface for vectors which guarantee that elements added to the vector are pinned to their memory locations unless explicitly changed."
license = "MIT"
repository = "https://github.com/orxfun/orx-pinned-vec/"
keywords = ["vec", "array", "vector", "pinned", "memory"]
categories = ["data-structures", "rust-patterns"]

[dependencies]
rand = "0.8.5"
rand_chacha = "0.3.1"
90 changes: 29 additions & 61 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,81 +1,49 @@
# orx-pinned-vec

`PinnedVec` trait defines the interface for vectors which guarantee that elements are pinned to their memory locations with the aim to enable convenient self-referential collections.
`PinnedVec` trait defines the interface for vectors which guarantee that elements added to the vector are pinned to their memory locations unless explicitly changed.

## A. Implementations
## A. Pinned Elements Guarantee

A `PinnedVec` guarantees that positions of its elements are not changed implicitly. Note that `std::vec::Vec` does not satisfy this requirement.
A `PinnedVec` guarantees that positions of its elements **do not change implicitly**.

[`SplitVec`](https://crates.io/crates/orx-split-vec) and [`FixedVec`](https://crates.io/crates/orx-fixed-vec) are two efficient implementations.

## B. Motivation

There might be various situations where pinned elements are helpful.

* It is somehow required for async code, following [blog](https://blog.cloudflare.com/pin-and-unpin-in-rust) could be useful for the interested.
* It is crucial in representing self-referential types with thin references.

This crate focuses on the latter. Particularly, it aims to make it safe and convenient to build **performant self-referential collections** such as linked lists, trees or graphs.

As explained in rust-docs [here](https://doc.rust-lang.org/std/pin/index.html), there exist types `Pin` and `Unpin` for this very purpose. Through the theoretical discussions, one can easily agree on the safety. However, the solution is complicated with all words `PhantomPinned`, `NonNull`, `dangling`, `Box::pin`, etc. which are alien to the self-referential data structures we are trying to build.

This crate suggests the following approach:

* Instances of the self-referential type will be collected together in a vector.
* Referencing each other will be through the natural `&` way rather than requiring any of the smart pointers.
* In terms of convenience, building the collection will be close to building a regular vector.

## C. Self-Referential-Collection Element Traits
To be specific, let's assume that a pinned vector currently has `n` elements:

This crate also defines under the `orx_pinned_vec::self_referential_elements` module the required traits to enable building self referential collections with thin references.
| Method | Expected Behavior |
| -------- | ------- |
| `push(new_element)` | does not change the memory locations of the `n` elements |
| `extend_from_slice(slice)` | does not change the memory locations of the first `n` elements |
| `insert(a, new_element)` | does not change the memory locations of the first `a` elements, where `a <= n`; elements to the right of the inserted element might be changed, commonly shifted to right |
| `pop()` | does not change the memory locations of the first `n-1` elements, the `n`-th element is removed |
| `remove(a)` | does not change the memory locations of the first `a` elements, where `a < n`; elements to the right of the removed element might be changed, commonly shifted to left |
| `truncate(a)` | does not change the memory locations of the first `a` elements, where `a < n` |

* `SelfRefNext` trait simply requires:
* `fn next(&self) -> Option<&'a Self>;` and
* `fn set_next(&mut self, next: Option<&'a Self>);` methods.
`PinnedVec` trait on its own cannot provide the pinned element guarantee; hence, it could be considered as a marker trait.

`SelfRefPrev` is the previous counterpart.
However, this crate additionally provides the test function to assert these guarantees:

Notice that these two traits are sufficient to define a linked list. [`orx_linked_list::LinkedList`](https://crates.io/crates/orx-linked-list) implements `SelfRefPrev` and `SelfRefNext` to conveniently define a recurisve doubly linked list.
```rust ignore
pub fn test_pinned_vec<P: PinnedVec<usize>>(pinned_vec: P, test_vec_len: usize) {
// ...
}
```

Further, there exist multiple reference counterparts. They are useful in defining relations such as the *children* of a tree node or *heads of outgoing arcs* from a graph node, etc. There exist *vec* variants to be used for holding variable number of references. However, there also exist constant sized array versions which are useful in structures such as binary search trees where the number of references is bounded by a const.
This function performs an extensive test on the specific implementation `P` and fails if any of the above guarantees is not provided.

The table below presents the complete list of traits which suffice to define all aforementioned relations:
Note that `std::vec::Vec` does not provide the pinned elements during growth guarantee. You may find a wrapper struct `JustVec` which is nothing but the standard vec here: [src/pinned_vec_tests/test_all.rs](https://github.com/orxfun/orx-pinned-vec/blob/main/src/pinned_vec_tests/test_all.rs). As expected, `test_pinned_vec` method fails for this struct.

| | prev | next |
|---------------------------------------------|----------------|----------------|
| single reference | SelfRefPrev | SelfRefNext |
| dynamic number of references | SelfRefPrevVec | SelfRefNextVec |
| multiple elements with a `const` max-length | SelfRefPrevArr | SelfRefNextArr |

## D. Safety

With self referential collections, some mutating methods can lead to critical problems. These are the methods which change positions of already pushed elements or remove elements from the vector:

* `insert`
* `remove`
* `pop`
* `swap`
* `truncate`

These methods can invalidate the references among elements. Therefore, `PinnedVec` defines them as **unsafe**. One exception is the `clear` method which is safe since all elements are removed together with their references at once.

In addition, `clone` method as well is **unsafe**, since the elements of the clone would be referencing the elements of the original vector.

These are due to the fact that, naive implementations would cause false references. This does not mean that it is not possible to provide a safe implementation. Instead, it means that each data structure would need a different implementation (insert method of a tree and that of a linked-list cannot be implemented in the same way, they will need to update references differently).

Implementors can provide careful safe implementations, such as `orx_linked_list::LinkedList` safely implement `Clone`, although it uses any `PinnedVec` as the underlying storage.
## B. Motivation

There are a few cases other than self referencing collections, where a `PinnedVec` is useful. And there is no reason to treat these methods as unsafe if the elements are not referencing each other. For this purpose, `NotSelfRefVecItem` marker trait is defined. This trait works as follows:
There are various situations where pinned elements are necessary.

* if `V` implements `PinnedVec<T>`, and
* if `T` implements the marker trait `NotSelfRefVecItem`,
* => then, `V` also implements `PinnedVecSimple<T>` which provides the safe versions of the abovementioned methods.
* It is critical in enabling **efficient, convenient and safe self-referential collections** with thin references, see [`SelfRefCol`](https://crates.io/crates/orx-self-ref-col) for details.
* It is essential in allowing an **immutable push** vector; i.e., [`ImpVec`](https://crates.io/crates/orx-imp-vec). This is a very useful operation when the desired collection is a bag or a container of things, rather than having a collective meaning. In such cases, `ImpVec` avoids heap allocations and wide pointers such as `Box` or `Rc` or etc.
* It is important for **async** code; following [blog](https://blog.cloudflare.com/pin-and-unpin-in-rust) could be useful for the interested.

`NotSelfRefVecItem` trait is implemented for most primitives; however, one needs to implement for new types to explicitly state that the type is <ins>not</ins> self-referential.
*As explained in [rust-docs](https://doc.rust-lang.org/std/pin/index.html), there exist `Pin` and `Unpin` types for similar purposes. However, the solution is complicated and low level using `PhantomPinned`, `NonNull`, `dangling`, `Box::pin`, pointer accesses, etc.*

## E. Relation with the `ImpVec`
## C. Implementations

Providing pinned memory location elements with `PinnedVec` is the first block for building self referential structures; the second building block is the [`ImpVec`](https://crates.io/crates/orx-imp-vec). An `ImpVec` wraps any `PinnedVec` implementation and provides specialized methods built on the pinned element guarantee in order to allow building self referential collections.
[`SplitVec`](https://crates.io/crates/orx-split-vec) and [`FixedVec`](https://crates.io/crates/orx-fixed-vec) are two efficient implementations.

## License

Expand Down
98 changes: 31 additions & 67 deletions src/lib.rs
View file
Original file line number Diff line number Diff line change
@@ -1,81 +1,49 @@
//! # orx-pinned-vec
//!
//! `PinnedVec` trait defines the interface for vectors which guarantee that elements are pinned to their memory locations with the aim to enable convenient self-referential collections.
//! `PinnedVec` trait defines the interface for vectors which guarantee that elements added to the vector are pinned to their memory locations unless explicitly changed.
//!
//! ## A. Implementations
//! ## A. Pinned Elements Guarantee
//!
//! A `PinnedVec` guarantees that positions of its elements are not changed implicitly. Note that `std::vec::Vec` does not satisfy this requirement.
//! A `PinnedVec` guarantees that positions of its elements **do not change implicitly**.
//!
//! [`SplitVec`](https://crates.io/crates/orx-split-vec) and [`FixedVec`](https://crates.io/crates/orx-fixed-vec) are two efficient implementations.
//!
//! ## B. Motivation
//!
//! There might be various situations where pinned elements are helpful.
//!
//! * It is somehow required for async code, following [blog](https://blog.cloudflare.com/pin-and-unpin-in-rust) could be useful for the interested.
//! * It is crucial in representing self-referential types with thin references.
//!
//! This crate focuses on the latter. Particularly, it aims to make it safe and convenient to build **performant self-referential collections** such as linked lists, trees or graphs.
//!
//! As explained in rust-docs [here](https://doc.rust-lang.org/std/pin/index.html), there exist types `Pin` and `Unpin` for this very purpose. Through the theoretical discussions, one can easily agree on the safety. However, the solution is complicated with all words `PhantomPinned`, `NonNull`, `dangling`, `Box::pin`, etc. which are alien to the self-referential data structures we are trying to build.
//!
//! This crate suggests the following approach:
//!
//! * Instances of the self-referential type will be collected together in a vector.
//! * Referencing each other will be through the natural `&` way rather than requiring any of the smart pointers.
//! * In terms of convenience, building the collection will be close to building a regular vector.
//!
//! ## C. Self-Referential-Collection Element Traits
//! To be specific, let's assume that a pinned vector currently has `n` elements:
//!
//! This crate also defines under the `orx_pinned_vec::self_referential_elements` module the required traits to enable building self referential collections with thin references.
//! | Method | Expected Behavior |
//! | -------- | ------- |
//! | `push(new_element)` | does not change the memory locations of the `n` elements |
//! | `extend_from_slice(slice)` | does not change the memory locations of the first `n` elements |
//! | `insert(a, new_element)` | does not change the memory locations of the first `a` elements, where `a <= n`; elements to the right of the inserted element might be changed, commonly shifted to right |
//! | `pop()` | does not change the memory locations of the first `n-1` elements, the `n`-th element is removed |
//! | `remove(a)` | does not change the memory locations of the first `a` elements, where `a < n`; elements to the right of the removed element might be changed, commonly shifted to left |
//! | `truncate(a)` | does not change the memory locations of the first `a` elements, where `a < n` |
//!
//! * `SelfRefNext` trait simply requires:
//! * `fn next(&self) -> Option<&'a Self>;` and
//! * `fn set_next(&mut self, next: Option<&'a Self>);` methods.
//! `PinnedVec` trait on its own cannot provide the pinned element guarantee; hence, it could be considered as a marker trait.
//!
//! `SelfRefPrev` is the previous counterpart.
//! However, this crate additionally provides the test function to assert these guarantees:
//!
//! Notice that these two traits are sufficient to define a linked list. [`orx_linked_list::LinkedList`](https://crates.io/crates/orx-linked-list) implements `SelfRefPrev` and `SelfRefNext` to conveniently define a recurisve doubly linked list.
//! ```rust ignore
//! pub fn test_pinned_vec<P: PinnedVec<usize>>(pinned_vec: P, test_vec_len: usize) {
//! // ...
//! }
//! ```
//!
//! Further, there exist multiple reference counterparts. They are useful in defining relations such as the *children* of a tree node or *heads of outgoing arcs* from a graph node, etc. There exist *vec* variants to be used for holding variable number of references. However, there also exist constant sized array versions which are useful in structures such as binary search trees where the number of references is bounded by a const.
//! This function performs an extensive test on the specific implementation `P` and fails if any of the above guarantees is not provided.
//!
//! The table below presents the complete list of traits which suffice to define all aforementioned relations:
//! Note that `std::vec::Vec` does not provide the pinned elements during growth guarantee. You may find a wrapper struct `JustVec` which is nothing but the standard vec here: [src/pinned_vec_tests/test_all.rs](https://github.com/orxfun/orx-pinned-vec/blob/main/src/pinned_vec_tests/test_all.rs). As expected, `test_pinned_vec` method fails for this struct.
//!
//! | | prev | next |
//! |---------------------------------------------|----------------|----------------|
//! | single reference | SelfRefPrev | SelfRefNext |
//! | dynamic number of references | SelfRefPrevVec | SelfRefNextVec |
//! | multiple elements with a `const` max-length | SelfRefPrevArr | SelfRefNextArr |
//!
//! ## D. Safety
//!
//! With self referential collections, some mutating methods can lead to critical problems. These are the methods which change positions of already pushed elements or remove elements from the vector:
//!
//! * `insert`
//! * `remove`
//! * `pop`
//! * `swap`
//! * `truncate`
//!
//! These methods can invalidate the references among elements. Therefore, `PinnedVec` defines them as **unsafe**. One exception is the `clear` method which is safe since all elements are removed together with their references at once.
//!
//! In addition, `clone` method as well is **unsafe**, since the elements of the clone would be referencing the elements of the original vector.
//!
//! These are due to the fact that, naive implementations would cause false references. This does not mean that it is not possible to provide a safe implementation. Instead, it means that each data structure would need a different implementation (insert method of a tree and that of a linked-list cannot be implemented in the same way, they will need to update references differently).
//!
//! Implementors can provide careful safe implementations, such as `orx_linked_list::LinkedList` safely implement `Clone`, although it uses any `PinnedVec` as the underlying storage.
//! ## B. Motivation
//!
//! There are a few cases other than self referencing collections, where a `PinnedVec` is useful. And there is no reason to treat these methods as unsafe if the elements are not referencing each other. For this purpose, `NotSelfRefVecItem` marker trait is defined. This trait works as follows:
//! There are various situations where pinned elements are necessary.
//!
//! * if `V` implements `PinnedVec<T>`, and
//! * if `T` implements the marker trait `NotSelfRefVecItem`,
//! * => then, `V` also implements `PinnedVecSimple<T>` which provides the safe versions of the abovementioned methods.
//! * It is critical in enabling **efficient, convenient and safe self-referential collections** with thin references, see [`SelfRefCol`](https://crates.io/crates/orx-self-ref-col) for details.
//! * It is essential in allowing an **immutable push** vector; i.e., [`ImpVec`](https://crates.io/crates/orx-imp-vec). This is a very useful operation when the desired collection is a bag or a container of things, rather than having a collective meaning. In such cases, `ImpVec` avoids heap allocations and wide pointers such as `Box` or `Rc` or etc.
//! * It is important for **async** code; following [blog](https://blog.cloudflare.com/pin-and-unpin-in-rust) could be useful for the interested.
//!
//! `NotSelfRefVecItem` trait is implemented for most primitives; however, one needs to implement for new types to explicitly state that the type is <ins>not</ins> self-referential.
//! *As explained in [rust-docs](https://doc.rust-lang.org/std/pin/index.html), there exist `Pin` and `Unpin` types for similar purposes. However, the solution is complicated and low level using `PhantomPinned`, `NonNull`, `dangling`, `Box::pin`, pointer accesses, etc.*
//!
//! ## E. Relation with the `ImpVec`
//! ## C. Implementations
//!
//! Providing pinned memory location elements with `PinnedVec` is the first block for building self referential structures; the second building block is the [`ImpVec`](https://crates.io/crates/orx-imp-vec). An `ImpVec` wraps any `PinnedVec` implementation and provides specialized methods built on the pinned element guarantee in order to allow building self referential collections.
//! [`SplitVec`](https://crates.io/crates/orx-split-vec) and [`FixedVec`](https://crates.io/crates/orx-fixed-vec) are two efficient implementations.
//!
//! ## License
//!
Expand All @@ -93,14 +61,10 @@
clippy::todo
)]

mod not_self_ref;
mod pinned_vec;
mod pinned_vec_simple;
/// Traits to define variants of self-referential-collection elements.
pub mod self_referential_elements;
mod pinned_vec_tests;
/// Utility functions to make PinnedVec implementations more convenient.
pub mod utils;

pub use not_self_ref::NotSelfRefVecItem;
pub use pinned_vec::PinnedVec;
pub use pinned_vec_simple::PinnedVecSimple;
pub use pinned_vec_tests::test_all::test_pinned_vec;
Loading

0 comments on commit 0c83db0

Please sign in to comment.