Skip to content

Commit

Permalink
Rename offset to index to better capture its meaning
Browse files Browse the repository at this point in the history
Signed-off-by: Moritz Hoffmann <[email protected]>
  • Loading branch information
antiguru committed Jul 8, 2024
1 parent ebec264 commit afe0853
Show file tree
Hide file tree
Showing 9 changed files with 206 additions and 206 deletions.
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,9 @@ to look up a value. A region for slices has storage to remember slices of indexe
where the index can be used to look up the datum's representation in an inner
region.

`flatcontainer` provides [`FlatStack`], an exemplary implementation of how to
`flatcontainer` provides the [`FlatStack`] type, an exemplary implementation of how to
implement a collection of items that supports pushing additional elements,
and retrieval by offset of previously pushed elements. It can be used in many
and retrieval by an index to previously pushed elements. It can be used in many
places that simply want to use a flat data representation, but it leaves potential
for optimization behind. Specifically, indexes, although opaque, follow a
simple structure where a more efficient storage can save memory instead of blindly
Expand Down Expand Up @@ -101,19 +101,19 @@ we know more about the data, and can use a better approach to storing indexes.
e.g., 0, 2, 4, 8, ..., which we can represent using constant memory by remembering the
stride and length. We can extend this to storing a tail of elements equals to the last
stride by adding another count. Such an index container uses 0 bits in the limit!
* Consult the [offsets] module for types specialized to storing indexes using less bits.
* Consult the [index] module for types specialized to storing indexes using less bits.

Flatcontainer provides some implementations of these concepts. To merge adjacent start-end
pairs, wrap a region in a [`ConsecutiveOffsetPairs`] region. It stores indexes and presents
pairs, wrap a region in a [`ConsecutiveIndexPairs`] region. It stores indexes and presents
outwards as a dense sequence of 0, 1, 2, ...

A [`CollapseSequence`] region remembers the index of the last element, and if a new element
equals the last, it'll return the previous index again instead of storing the same element
multiple times. This is limited to the direct predecessor, because otherwise storing
indexes and scanning through previous elements would be too expensive.

[offsets]: impls::offsets
[`ConsecutiveOffsetPairs`]: impls::deduplicate::ConsecutiveOffsetPairs
[index]: impls::index
[`ConsecutiveIndexPairs`]: impls::deduplicate::ConsecutiveIndexPairs
[`CollapseSequence`]: impls::deduplicate::CollapseSequence

## Comparison to columnation
Expand Down
6 changes: 3 additions & 3 deletions benches/bench.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@

extern crate test;

use flatcontainer::impls::deduplicate::{CollapseSequence, ConsecutiveOffsetPairs};
use flatcontainer::impls::offsets::OffsetOptimized;
use flatcontainer::impls::deduplicate::{CollapseSequence, ConsecutiveIndexPairs};
use flatcontainer::impls::index::IndexOptimized;
use flatcontainer::impls::tuple::{TupleABCRegion, TupleABRegion};
use flatcontainer::{
ColumnsRegion, FlatStack, MirrorRegion, OwnedRegion, Push, Region, RegionPreference,
Expand Down Expand Up @@ -87,7 +87,7 @@ fn string10_copy_region(bencher: &mut Bencher) {
#[bench]
fn string10_copy_region_collapse(bencher: &mut Bencher) {
_bench_copy_region::<
SliceRegion<CollapseSequence<ConsecutiveOffsetPairs<StringRegion>>, OffsetOptimized>,
SliceRegion<CollapseSequence<ConsecutiveIndexPairs<StringRegion>>, IndexOptimized>,
_,
>(bencher, vec![format!("grawwwwrr!"); 1024]);
}
Expand Down
2 changes: 1 addition & 1 deletion src/impls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ pub mod codec;
pub mod columns;
pub mod deduplicate;
pub mod huffman_container;
pub mod index;
pub mod mirror;
pub mod offsets;
pub mod option;
pub mod result;
pub mod slice;
Expand Down
46 changes: 23 additions & 23 deletions src/impls/columns.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ use std::slice::Iter;
#[cfg(feature = "serde")]
use serde::{Deserialize, Serialize};

use crate::impls::deduplicate::ConsecutiveOffsetPairs;
use crate::impls::offsets::{OffsetContainer, OffsetOptimized};
use crate::impls::deduplicate::ConsecutiveIndexPairs;
use crate::impls::index::{IndexContainer, IndexOptimized};
use crate::{IntoOwned, PushIter};
use crate::{OwnedRegion, Push, Region};

Expand All @@ -25,7 +25,7 @@ use crate::{OwnedRegion, Push, Region};
///
/// Copy a table-like structure:
/// ```
/// # use flatcontainer::impls::deduplicate::ConsecutiveOffsetPairs;
/// # use flatcontainer::impls::deduplicate::ConsecutiveIndexPairs;
/// # use flatcontainer::{ColumnsRegion, Push, Region, StringRegion};
/// let data = [
/// vec![],
Expand All @@ -37,7 +37,7 @@ use crate::{OwnedRegion, Push, Region};
/// vec![],
/// ];
///
/// let mut r = <ColumnsRegion<ConsecutiveOffsetPairs<StringRegion>>>::default();
/// let mut r = <ColumnsRegion<ConsecutiveIndexPairs<StringRegion>>>::default();
///
/// let mut indices = Vec::with_capacity(data.len());
///
Expand All @@ -60,13 +60,13 @@ use crate::{OwnedRegion, Push, Region};
O: Serialize + for<'a> Deserialize<'a>,
")
)]
pub struct ColumnsRegion<R, O = OffsetOptimized>
pub struct ColumnsRegion<R, O = IndexOptimized>
where
R: Region,
{
/// Indices to address rows in `inner`. For each row, we remember
/// an index for each column.
indices: ConsecutiveOffsetPairs<OwnedRegion<R::Index>, O>,
indices: ConsecutiveIndexPairs<OwnedRegion<R::Index>, O>,
/// Storage for columns.
inner: Vec<R>,
}
Expand All @@ -92,11 +92,11 @@ where
impl<R, O> Region for ColumnsRegion<R, O>
where
R: Region,
O: OffsetContainer<usize>,
O: IndexContainer<usize>,
{
type Owned = Vec<R::Owned>;
type ReadItem<'a> = ReadColumns<'a, R> where Self: 'a;
type Index = <ConsecutiveOffsetPairs<OwnedRegion<R::Index>, OffsetOptimized> as Region>::Index;
type Index = <ConsecutiveIndexPairs<OwnedRegion<R::Index>, IndexOptimized> as Region>::Index;

fn merge_regions<'a>(regions: impl Iterator<Item = &'a Self> + Clone) -> Self
where
Expand All @@ -112,7 +112,7 @@ where
}

Self {
indices: ConsecutiveOffsetPairs::merge_regions(regions.map(|r| &r.indices)),
indices: ConsecutiveIndexPairs::merge_regions(regions.map(|r| &r.indices)),
inner,
}
}
Expand Down Expand Up @@ -169,11 +169,11 @@ where
impl<R, O> Default for ColumnsRegion<R, O>
where
R: Region,
O: OffsetContainer<usize>,
O: IndexContainer<usize>,
{
fn default() -> Self {
Self {
indices: ConsecutiveOffsetPairs::default(),
indices: ConsecutiveIndexPairs::default(),
inner: Vec::default(),
}
}
Expand Down Expand Up @@ -376,7 +376,7 @@ where
impl<R, O> Push<ReadColumns<'_, R>> for ColumnsRegion<R, O>
where
for<'a> R: Region + Push<<R as Region>::ReadItem<'a>>,
O: OffsetContainer<usize>,
O: IndexContainer<usize>,
{
fn push(&mut self, item: ReadColumns<'_, R>) -> <ColumnsRegion<R, O> as Region>::Index {
// Ensure all required regions exist.
Expand All @@ -395,7 +395,7 @@ where
impl<'a, R, O, T> Push<&'a [T]> for ColumnsRegion<R, O>
where
R: Region + Push<&'a T>,
O: OffsetContainer<usize>,
O: IndexContainer<usize>,
{
fn push(&mut self, item: &'a [T]) -> <ColumnsRegion<R, O> as Region>::Index {
// Ensure all required regions exist.
Expand All @@ -414,7 +414,7 @@ where
impl<R, O, T, const N: usize> Push<[T; N]> for ColumnsRegion<R, O>
where
R: Region + Push<T>,
O: OffsetContainer<usize>,
O: IndexContainer<usize>,
{
fn push(&mut self, item: [T; N]) -> <ColumnsRegion<R, O> as Region>::Index {
// Ensure all required regions exist.
Expand All @@ -433,7 +433,7 @@ where
impl<'a, R, O, T, const N: usize> Push<&'a [T; N]> for ColumnsRegion<R, O>
where
R: Region + Push<&'a T>,
O: OffsetContainer<usize>,
O: IndexContainer<usize>,
{
fn push(&mut self, item: &'a [T; N]) -> <ColumnsRegion<R, O> as Region>::Index {
// Ensure all required regions exist.
Expand All @@ -452,7 +452,7 @@ where
impl<R, O, T> Push<Vec<T>> for ColumnsRegion<R, O>
where
R: Region + Push<T>,
O: OffsetContainer<usize>,
O: IndexContainer<usize>,
{
fn push(&mut self, item: Vec<T>) -> <ColumnsRegion<R, O> as Region>::Index {
// Ensure all required regions exist.
Expand All @@ -471,7 +471,7 @@ where
impl<'a, R, O, T> Push<&'a Vec<T>> for ColumnsRegion<R, O>
where
R: Region + Push<&'a T>,
O: OffsetContainer<usize>,
O: IndexContainer<usize>,
{
fn push(&mut self, item: &'a Vec<T>) -> <ColumnsRegion<R, O> as Region>::Index {
// Ensure all required regions exist.
Expand All @@ -490,7 +490,7 @@ where
impl<R, O, T, I> Push<PushIter<I>> for ColumnsRegion<R, O>
where
R: Region + Push<T>,
O: OffsetContainer<usize>,
O: IndexContainer<usize>,
I: IntoIterator<Item = T>,
I::IntoIter: ExactSizeIterator,
{
Expand All @@ -509,7 +509,7 @@ where

#[cfg(test)]
mod tests {
use crate::impls::deduplicate::{CollapseSequence, ConsecutiveOffsetPairs};
use crate::impls::deduplicate::{CollapseSequence, ConsecutiveIndexPairs};
use crate::{MirrorRegion, OwnedRegion, Push, PushIter, Region, StringRegion};

use super::*;
Expand Down Expand Up @@ -573,7 +573,7 @@ mod tests {
];

let mut r =
ColumnsRegion::<CollapseSequence<ConsecutiveOffsetPairs<StringRegion>>>::default();
ColumnsRegion::<CollapseSequence<ConsecutiveIndexPairs<StringRegion>>>::default();

let mut indices = Vec::with_capacity(data.len());

Expand Down Expand Up @@ -601,7 +601,7 @@ mod tests {
vec![],
];

let mut r = ColumnsRegion::<ConsecutiveOffsetPairs<StringRegion>>::default();
let mut r = ColumnsRegion::<ConsecutiveIndexPairs<StringRegion>>::default();

let mut indices = Vec::with_capacity(data.len());

Expand Down Expand Up @@ -629,7 +629,7 @@ mod tests {
vec![],
];

let mut r = ColumnsRegion::<ConsecutiveOffsetPairs<StringRegion>>::default();
let mut r = ColumnsRegion::<ConsecutiveIndexPairs<StringRegion>>::default();

let mut indices = Vec::with_capacity(data.len());

Expand Down Expand Up @@ -713,7 +713,7 @@ mod tests {
vec![],
];

let mut r = ColumnsRegion::<ConsecutiveOffsetPairs<StringRegion>>::default();
let mut r = ColumnsRegion::<ConsecutiveIndexPairs<StringRegion>>::default();

for row in &data {
let _ = r.push(PushIter(row.iter()));
Expand Down
Loading

0 comments on commit afe0853

Please sign in to comment.