Skip to content

Commit

Permalink
Merge pull request #17 from christophhagen/improve-wrappers
Browse files Browse the repository at this point in the history
Introduce varint and zigzag wrappers
  • Loading branch information
christophhagen authored Apr 3, 2024
2 parents 2d3543b + b410cf1 commit dfd2547
Show file tree
Hide file tree
Showing 26 changed files with 1,210 additions and 237 deletions.
15 changes: 8 additions & 7 deletions BinaryFormat.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,14 @@ The basic types known to `Codable` are listed in the following table:
| --- | --- | --- |
| Bool | 1 | false: `0x00`, true: `0x01` |
| Data | ? | As itself
| Double | 8 | IEEE Double representation, little endian |
| Float | 4 |IEEE Double representation, little endian |
| Double | 8 | [IEEE Double representation, little endian](#floating-point-types) |
| Float | 4 | [IEEE Double representation, little endian](#floating-point-types) |
| Int8 | 1 | [Little endian](#little-endian) |
| Int16 | 2 | [Little endian](#little-endian) |
| Int32 | 1-5 | Zig-zag variable-length encoding |
| Int64 | 1-9 | Zig-zag variable-length encoding |
| Int | 1-9 | Zig-zag variable-length encoding |
| String | ? | UTF-8 data |
| Int32 | 1-5 | [Zig-zag variable-length encoding](#zig-zag-encoding) |
| Int64 | 1-9 | [Zig-zag variable-length encoding](#zig-zag-encoding) |
| Int | 1-9 | [Zig-zag variable-length encoding](#zig-zag-encoding) |
| String | ? | [UTF-8 data](#strings) |
| UInt8 | 1 | [Little endian](#little-endian) |
| UInt16 | 2 | [Little endian](#little-endian) |
| UInt32 | 1-5 | [Variable-length encoding](#variable-length-encoding) |
Expand Down Expand Up @@ -56,6 +56,7 @@ struct MyType: Codable {
### Boolean

`Bool` values are encoded as a single byte, using `1` for `true`, and `0` for `false`.
All other bytes will result in a decoding error.

### Little endian

Expand Down Expand Up @@ -268,4 +269,4 @@ For dictionaries with `String` keys (`[String: ...]`), the process is similar to
## Stream encoding

The encoding for data streams only differs from standard encoding in one key aspect.
Each top-level element is encoded as if it is part of an unkeyed container (which it essentially is), meaning that each element has the necessary length information prepended to determine its size.
Each top-level element is encoded as if it is part of an unkeyed container (which it essentially is), meaning that each element has the necessary length information prepended to determine its size.
56 changes: 45 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,21 +192,55 @@ Notes:
#### Fixed size integers

While varints are efficient for small numbers, their encoding introduces a storage and computation penalty when the integers are often large, e.g. for random numbers.
`BinaryCodable` provides the `FixedSize` wrapper, which forces integers to be encoded using their little-endian binary representations.
`BinaryCodable` provides the `@FixedSizeEncoded` property wrapper, which forces integers to be encoded using their little-endian binary representations.
This means that e.g. an `Int32` is always encoded as 4 byte (instead of 1-5 bytes using Varint encoding).
This makes 32-bit `FixedSize` types more efficient than `Varint` if values are often larger than `2^28` (`2^56` for 64-bit types).
This makes 32-bit `FixedSizeEncoded` types more efficient than `Varint` if values are often larger than `2^28` (`2^56` for 64-bit types).

Use the property wrapper within a `Codable` definition to enforce fixed-width encoding for a property:
```swift
struct MyStruct: Codable {
Use the property wrapper within a `Codable` definition to enforce fixed-width encoding for a property:
```swift
struct MyStruct: Codable {

/// Always encoded as 4 bytes
@FixedSize
var largeInteger: Int32
}
```
/// Always encoded as 4 bytes
@FixedSizeEncoded
var largeInteger: Int32
}
```

The `FixedSize` wrapper is available for `Int`, `Int32`, `Int64`, `UInt`, `UInt32`, and `UInt64`.
It has no effect for `Int16` and `UInt16`, which are already encoded with a fixed size by default.

#### Variable length integers

The `FixedSize` wrapper is available to all `Varint` types: `Int`, `UInt`, `Int32`, `UInt32`, `Int64`, and `UInt64`.
Some integers can be forced to use variable-length encoding instead of fixed-size or zig-zag encoding using the `@VariableLengthEncoded` property wrapper.

For `Int16` and `UInt16` (normally fixed-size encoded), this encoding can be more efficient if values are often smaller than `128` for `UInt16` and `63` for `Int16`.
For `Int`, `Int32` and `Int64` (normally zig-zag encoded), the encoding is (marginally) more efficient if values are mostly positive.
For `UInt`, `UInt32`, and `UInt64` the wrapper has no effect.

```swift
struct MyStruct: Codable {

/// Efficient for small, positive numbers
@VariableLengthEncoded
var value: Int16
}
```

#### Zig-Zag encoded integers

The signed integers `Int`, `Int32` and `Int64` are encoded using zig-zag encoding, which is more efficent than variable-length encoding if numbers are negative.
The `@ZigZagEncoded` wrapper can force `Int16` types to use zig-zag encoding instead of fixed-size encoding, which is more efficient for small (positive and negative) numbers.
The encoding is more efficient if values are between `-64` and `63`.
For `Int`, `Int32` and `Int64` the wrapper has no effect.

```swift
struct MyStruct: Codable {

/// More efficient between `-64` and `63`.
@ZigZagEncoded
var value: Int16
}
```

### Options

Expand Down
39 changes: 35 additions & 4 deletions Sources/BinaryCodable/Primitives/Int+Coding.swift
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,18 @@ extension Int: DecodablePrimitive {
}
}

// - MARK: Zig-zag encoding

extension Int: ZigZagEncodable {

var zigZagEncoded: Data {
public var zigZagEncoded: Data {
Int64(self).zigZagEncoded
}
}

extension Int: ZigZagDecodable {

init(fromZigZag data: Data) throws {
public init(fromZigZag data: Data) throws {
let raw = try Int64(data: data)
guard let value = Int(exactly: raw) else {
throw CorruptedDataError(outOfRange: raw, forType: "Int")
Expand All @@ -31,18 +33,27 @@ extension Int: ZigZagDecodable {
}
}

extension ZigZagEncoded where WrappedValue == Int {

@available(*, deprecated, message: "Property wrapper @ZigZagEncoded has no effect on type Int")
public init(wrappedValue: Int) {
self.wrappedValue = wrappedValue
}
}

// - MARK: Variable-length encoding

extension Int: VariableLengthEncodable {

/// The value encoded using variable length encoding
var variableLengthEncoding: Data {
public var variableLengthEncoding: Data {
Int64(self).variableLengthEncoding
}
}

extension Int: VariableLengthDecodable {

init(fromVarint data: Data) throws {
public init(fromVarint data: Data) throws {
let intValue = try Int64(fromVarint: data)
guard let value = Int(exactly: intValue) else {
throw CorruptedDataError(outOfRange: intValue, forType: "Int")
Expand All @@ -51,3 +62,23 @@ extension Int: VariableLengthDecodable {
}
}

// - MARK: Fixed-size encoding

extension Int: FixedSizeEncodable {

public var fixedSizeEncoded: Data {
Int64(self).fixedSizeEncoded
}
}

extension Int: FixedSizeDecodable {

public init(fromFixedSize data: Data) throws {
let signed = try Int64(fromFixedSize: data)
guard let value = Int(exactly: signed) else {
throw CorruptedDataError(outOfRange: signed, forType: "Int")
}
self = value
}
}

65 changes: 62 additions & 3 deletions Sources/BinaryCodable/Primitives/Int16+Coding.swift
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,77 @@ import Foundation

extension Int16: EncodablePrimitive {

var encodedData: Data {
.init(underlying: UInt16(bitPattern: self).littleEndian)
}
var encodedData: Data { fixedSizeEncoded }
}

extension Int16: DecodablePrimitive {

init(data: Data) throws {
try self.init(fromFixedSize: data)
}
}

// - MARK: Fixed size

extension Int16: FixedSizeEncodable {

public var fixedSizeEncoded: Data {
.init(underlying: UInt16(bitPattern: self).littleEndian)
}
}

extension Int16: FixedSizeDecodable {

public init(fromFixedSize data: Data) throws {
guard data.count == MemoryLayout<UInt16>.size else {
throw CorruptedDataError(invalidSize: data.count, for: "Int16")
}
let value = UInt16(littleEndian: data.interpreted())
self.init(bitPattern: value)
}
}

extension FixedSizeEncoded where WrappedValue == Int16 {

@available(*, deprecated, message: "Property wrapper @FixedSizeEncoded has no effect on type Int16")
public init(wrappedValue: Int16) {
self.wrappedValue = wrappedValue
}
}

// - MARK: Variable length

extension Int16: VariableLengthEncodable {

public var variableLengthEncoding: Data {
Int64(self).variableLengthEncoding
}
}

extension Int16: VariableLengthDecodable {

public init(fromVarint data: Data) throws {
let value = try UInt16(fromVarint: data)
self = Int16(bitPattern: value)
}
}

// - MARK: Zig-zag encoding

extension Int16: ZigZagEncodable {

public var zigZagEncoded: Data {
Int64(self).zigZagEncoded
}
}

extension Int16: ZigZagDecodable {

public init(fromZigZag data: Data) throws {
let raw = try Int64(fromZigZag: data)
guard let value = Int16(exactly: raw) else {
throw CorruptedDataError(outOfRange: raw, forType: "Int16")
}
self = value
}
}
41 changes: 37 additions & 4 deletions Sources/BinaryCodable/Primitives/Int32+Coding.swift
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,18 @@ extension Int32: DecodablePrimitive {
}
}

// - MARK: Zig-zag encoding

extension Int32: ZigZagEncodable {

var zigZagEncoded: Data {
public var zigZagEncoded: Data {
Int64(self).zigZagEncoded
}
}

extension Int32: ZigZagDecodable {

init(fromZigZag data: Data) throws {
public init(fromZigZag data: Data) throws {
let raw = try Int64(fromZigZag: data)
guard let value = Int32(exactly: raw) else {
throw CorruptedDataError(outOfRange: raw, forType: "Int32")
Expand All @@ -31,18 +33,49 @@ extension Int32: ZigZagDecodable {
}
}

extension ZigZagEncoded where WrappedValue == Int32 {

@available(*, deprecated, message: "Property wrapper @ZigZagEncoded has no effect on type Int32")
public init(wrappedValue: Int32) {
self.wrappedValue = wrappedValue
}
}

// - MARK: Variable-length encoding

extension Int32: VariableLengthEncodable {

/// The value encoded using variable length encoding
var variableLengthEncoding: Data {
public var variableLengthEncoding: Data {
UInt32(bitPattern: self).variableLengthEncoding
}
}

extension Int32: VariableLengthDecodable {

init(fromVarint data: Data) throws {
public init(fromVarint data: Data) throws {
let value = try UInt32(fromVarint: data)
self = Int32(bitPattern: value)
}
}

// - MARK: Fixed-size encoding

extension Int32: FixedSizeEncodable {

public var fixedSizeEncoded: Data {
let value = UInt32(bitPattern: littleEndian)
return Data(underlying: value)
}
}

extension Int32: FixedSizeDecodable {

public init(fromFixedSize data: Data) throws {
guard data.count == MemoryLayout<UInt32>.size else {
throw CorruptedDataError(invalidSize: data.count, for: "Int32")
}
let value = UInt32(littleEndian: data.interpreted())
self.init(bitPattern: value)
}
}
Loading

0 comments on commit dfd2547

Please sign in to comment.