Skip to content

Commit

Permalink
docs(FileSystem): improve
Browse files Browse the repository at this point in the history
  • Loading branch information
richardschneider committed Dec 31, 2018
1 parent ff08290 commit b693fc4
Show file tree
Hide file tree
Showing 8 changed files with 179 additions and 22 deletions.
22 changes: 1 addition & 21 deletions doc/articles/filesystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,9 @@ and [Object API](xref:Ipfs.CoreApi.IObjectApi).
A file has a unique [content id (CID)](xref:Ipfs.Cid) which is the cryptographic hash of the content; see
[CID concept](https://docs.ipfs.io/guides/concepts/cid/) for background information. The file's content is not just the file's
data but is encapsulated with a [protocol buffer](https://en.wikipedia.org/wiki/Protocol_Buffers) encoding of the
[PBNode](https://github.com/ipfs/go-ipfs/blob/0cb22ccf359e05fb5b55a9bf2f9c515bf7d4dba7/merkledag/pb/merkledag.proto#L31-L39)
[Merkle DAG](https://github.com/ipfs/go-ipfs/blob/0cb22ccf359e05fb5b55a9bf2f9c515bf7d4dba7/merkledag/pb/merkledag.proto#L31-L39)
and [UnixFS Data](https://github.com/ipfs/go-ipfs/blob/0cb22ccf359e05fb5b55a9bf2f9c515bf7d4dba7/unixfs/pb/unixfs.proto#L3-L20).

Where
- `PBNode.Data` contains unixfs message Data
- unixfs `Data.Data` contans file's data

When the file's data exceeds the [chunking size](xref:Ipfs.CoreApi.AddFileOptions.ChunkSize), multiple [blocks](xref:Ipfs.CoreApi.IBlockApi)
are generated. The returned CID points to a block that has `PBNode.Links` and no `PBNode.Data`.

### Adding a file

Expand Down Expand Up @@ -47,17 +41,3 @@ using (var stream = await ipfs.FileSystem.ReadFileAsyc(path))
// Do something with the data
}
```

### Getting a CID

Normally, you get the CID by [adding](xref:Ipfs.CoreApi.IFileSystemApi.AddAsync*) the file to IPFS. You can avoid adding it
to IPFS by using the [OnlyHash option](xref:Ipfs.CoreApi.AddFileOptions.OnlyHash).

```csharp
var options = new AddFileOptions { OnlyHash = true };
var fsn = await ipfs.FileSystem.AddTextAsync("hello world", options);
Console.WriteLine((string)fsn.Id)

// Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD
```

13 changes: 13 additions & 0 deletions doc/articles/fs/cid-only.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
### Getting a CID

Normally, you get the CID by [adding](xref:Ipfs.CoreApi.IFileSystemApi.AddAsync*) the file to IPFS. You can avoid adding it
to IPFS by using the [OnlyHash option](xref:Ipfs.CoreApi.AddFileOptions.OnlyHash).

```csharp
var options = new AddFileOptions { OnlyHash = true };
var fsn = await ipfs.FileSystem.AddTextAsync("hello world", options);
Console.WriteLine((string)fsn.Id)

// Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD
```

29 changes: 29 additions & 0 deletions doc/articles/fs/encryption.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Encryption

The [protection key option](xref:Ipfs.CoreApi.AddFileOptions.ProtectionKey) specifies that
the file's data blocks are encrypted using the specified [key name](../key.md).

Each data block maps to a [RFC652 - Cryptographic Message Syntax (CMS)](https://tools.ietf.org/html/rfc5652)
of type [Enveloped-data](https://tools.ietf.org/html/rfc5652#section-6) which is DER encoded
and has the following features:

- The data block is encrypted with a random IV and key using [aes-256-cbc](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard)
- The recipient is a key transport (ktri) with the Subject Key ID equal to the protection key's public ID
- The protection key is used to obtain the `aes key` to decrypt the data block.

The [Cid.ContentType](xref:Ipfs.Cid.ContentType) is set to `cms`.

```csharp
var options = new AddFileOptions
{
ProtectionKey = "me"
};
var node = await ipfs.FileSystem.AddTextAsync("hello world", options);
```

## Reading

The standard [read file methods](../filesystem.md#reading-a-file) are used to decrypt to file.
If the private key is not held by the local peer, then a `KeyNotFoundException` is thrown.


71 changes: 71 additions & 0 deletions doc/articles/fs/format.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
### Standard Format

Here is the [Merkle DAG](https://github.com/ipfs/go-ipfs/blob/0cb22ccf359e05fb5b55a9bf2f9c515bf7d4dba7/merkledag/pb/merkledag.proto#L31-L39)
and [UnixFS Data](https://github.com/ipfs/go-ipfs/blob/0cb22ccf359e05fb5b55a9bf2f9c515bf7d4dba7/unixfs/pb/unixfs.proto#L3-L20)
of a file containing the "hello world" string.

```json
{
"Links": [],
"Data": "\u0008\u0002\u0012\u000bhello world\u0018\u000b"
}
```

`Data` is the protobuf encoding of the UnixFS Data.

```json
{
"Type": 2,
"Data": "aGVsbG8gd29ybGQ=",
"FileSize": 11,
"BlockSizes": null,
"HashType": null,
"Fanout": null
}
```
### Chunked Format

When the file's data exceeds the [chunking size](xref:Ipfs.CoreApi.AddFileOptions.ChunkSize), multiple [blocks](xref:Ipfs.CoreApi.IBlockApi)
are generated. The returned CID points to a block that has `Merkle.Links`. Each link
contains a chunk of the file.

The following uses a chunking size of 6. A primary and two secondary blocks are created for "hello world".

#### Primary Block

```json
{
"Links": [
{"Name": "", "Hash": "QmPhmNbdBMtSQczNc4hnsMxRf5L4vfkU8jRTXDSHj8trSV", "Size": 14},
{"Name": "", "Hash": "QmNyJpQkU1cEkBwMDhDNFstr42q55mqG5GE5Mgwug4xyGk", "Size": 13}
],
"Data":"\u0008\u0002\u0018\u000b \u0006 \u0005"
}
{
"Type": 2,
"Data": null,
"FileSize": 11,
"BlockSizes": [6,5],
"HashType":null,
"Fanout":null
}
```

#### First Link

```json
{
"Links": [],
"Data": "\u0008\u0002\u0012\u0006hello \u0018\u0006"
}
```

#### Second Link

```json
{
"Links": [],
"Data": "\u0008\u0002\u0012\u0005world\u0018\u0005"
}
```

19 changes: 19 additions & 0 deletions doc/articles/fs/raw.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Raw Leaves

The [raw leaves option](xref:Ipfs.CoreApi.AddFileOptions.RawLeaves) specifies that
the file's data blocks are not [encapsulated](format.md)
with a Merkle DAG but simply contain the file's data.

The [Cid.ContentType](xref:Ipfs.Cid.ContentType) is set to `raw`.

```csharp
var options = new AddFileOptions
{
RawLeaves = true
};
var node = await ipfs.FileSystem.AddTextAsync("hello world", options);

// zb2rhj7crUKTQYRGCRATFaQ6YFLTde2YzdqbbhAASkL9uRDXn
// base58btc cidv1 raw sha2-256 QmaozNR7DZHQK1ZcU9p7QdrshMvXqWK6gpu5rmrkPdT3L4
```
34 changes: 34 additions & 0 deletions doc/articles/fs/wrap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Wrapping

The [wrap option](xref:Ipfs.CoreApi.AddFileOptions.Wrap) specifies that
the a directory is created for the file.

```csharp
var path = "hello.txt";
File.WriteAllText("hello.txt", "hello world");
var options = new AddFileOptions
{
Wrap = true
};
var node = await ipfs.FileSystem.AddFileAsync(path, options);

// QmNxvA5bwvPGgMXbmtyhxA1cKFdvQXnsGnZLCGor3AzYxJ
```
## Format

Two blocks are created, a directory object and a file object. The file object
is described in [standard format](format.md). The directory object looks
like this.

```json
{
"Links": [
{"Name": "hello.txt", "Hash": "Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD", "Size":19}
],
"Data": "\u0008\u0001"
}
{
"Type": 1,
}
```
2 changes: 1 addition & 1 deletion doc/articles/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ package is published on [NuGet](https://www.nuget.org/packages/Ipfs.Engine).
## Related projects

- [IPFS Core](https://github.com/richardschneider/net-ipfs-core)
- [IPFS API](https://github.com/richardschneider/net-ipfs-api)
- [IPFS HTTP Client](https://github.com/richardschneider/net-ipfs-http-client)

## Other implementations

Expand Down
11 changes: 11 additions & 0 deletions doc/articles/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,17 @@
href: core-api.md
- name: File System
href: filesystem.md
items:
- name: CID only
href: fs/cid-only.md
- name: Encryption
href: fs/encryption.md
- name: Raw Leaves
href: fs/raw.md
- name: Wrapping
href: fs/wrap.md
- name: Format
href: fs/format.md
- name: Repository
href: repository.md
items:
Expand Down

0 comments on commit b693fc4

Please sign in to comment.