From b693fc4d24a47cc45e722eff49c3bffece692db7 Mon Sep 17 00:00:00 2001 From: Richard Schneider Date: Mon, 31 Dec 2018 18:44:16 +1300 Subject: [PATCH] docs(FileSystem): improve --- doc/articles/filesystem.md | 22 +---------- doc/articles/fs/cid-only.md | 13 +++++++ doc/articles/fs/encryption.md | 29 ++++++++++++++ doc/articles/fs/format.md | 71 +++++++++++++++++++++++++++++++++++ doc/articles/fs/raw.md | 19 ++++++++++ doc/articles/fs/wrap.md | 34 +++++++++++++++++ doc/articles/intro.md | 2 +- doc/articles/toc.yml | 11 ++++++ 8 files changed, 179 insertions(+), 22 deletions(-) create mode 100644 doc/articles/fs/cid-only.md create mode 100644 doc/articles/fs/encryption.md create mode 100644 doc/articles/fs/format.md create mode 100644 doc/articles/fs/raw.md create mode 100644 doc/articles/fs/wrap.md diff --git a/doc/articles/filesystem.md b/doc/articles/filesystem.md index cc97db22..a9e96230 100644 --- a/doc/articles/filesystem.md +++ b/doc/articles/filesystem.md @@ -9,15 +9,9 @@ and [Object API](xref:Ipfs.CoreApi.IObjectApi). A file has a unique [content id (CID)](xref:Ipfs.Cid) which is the cryptographic hash of the content; see [CID concept](https://docs.ipfs.io/guides/concepts/cid/) for background information. The file's content is not just the file's data but is encapsulated with a [protocol buffer](https://en.wikipedia.org/wiki/Protocol_Buffers) encoding of the -[PBNode](https://github.com/ipfs/go-ipfs/blob/0cb22ccf359e05fb5b55a9bf2f9c515bf7d4dba7/merkledag/pb/merkledag.proto#L31-L39) +[Merkle DAG](https://github.com/ipfs/go-ipfs/blob/0cb22ccf359e05fb5b55a9bf2f9c515bf7d4dba7/merkledag/pb/merkledag.proto#L31-L39) and [UnixFS Data](https://github.com/ipfs/go-ipfs/blob/0cb22ccf359e05fb5b55a9bf2f9c515bf7d4dba7/unixfs/pb/unixfs.proto#L3-L20). -Where -- `PBNode.Data` contains unixfs message Data -- unixfs `Data.Data` contans file's data - -When the file's data exceeds the [chunking size](xref:Ipfs.CoreApi.AddFileOptions.ChunkSize), multiple [blocks](xref:Ipfs.CoreApi.IBlockApi) -are generated. The returned CID points to a block that has `PBNode.Links` and no `PBNode.Data`. ### Adding a file @@ -47,17 +41,3 @@ using (var stream = await ipfs.FileSystem.ReadFileAsyc(path)) // Do something with the data } ``` - -### Getting a CID - -Normally, you get the CID by [adding](xref:Ipfs.CoreApi.IFileSystemApi.AddAsync*) the file to IPFS. You can avoid adding it -to IPFS by using the [OnlyHash option](xref:Ipfs.CoreApi.AddFileOptions.OnlyHash). - -```csharp -var options = new AddFileOptions { OnlyHash = true }; -var fsn = await ipfs.FileSystem.AddTextAsync("hello world", options); -Console.WriteLine((string)fsn.Id) - -// Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD -``` - diff --git a/doc/articles/fs/cid-only.md b/doc/articles/fs/cid-only.md new file mode 100644 index 00000000..2fceacb0 --- /dev/null +++ b/doc/articles/fs/cid-only.md @@ -0,0 +1,13 @@ +### Getting a CID + +Normally, you get the CID by [adding](xref:Ipfs.CoreApi.IFileSystemApi.AddAsync*) the file to IPFS. You can avoid adding it +to IPFS by using the [OnlyHash option](xref:Ipfs.CoreApi.AddFileOptions.OnlyHash). + +```csharp +var options = new AddFileOptions { OnlyHash = true }; +var fsn = await ipfs.FileSystem.AddTextAsync("hello world", options); +Console.WriteLine((string)fsn.Id) + +// Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD +``` + diff --git a/doc/articles/fs/encryption.md b/doc/articles/fs/encryption.md new file mode 100644 index 00000000..3987b89f --- /dev/null +++ b/doc/articles/fs/encryption.md @@ -0,0 +1,29 @@ +# Encryption + +The [protection key option](xref:Ipfs.CoreApi.AddFileOptions.ProtectionKey) specifies that +the file's data blocks are encrypted using the specified [key name](../key.md). + +Each data block maps to a [RFC652 - Cryptographic Message Syntax (CMS)](https://tools.ietf.org/html/rfc5652) +of type [Enveloped-data](https://tools.ietf.org/html/rfc5652#section-6) which is DER encoded +and has the following features: + +- The data block is encrypted with a random IV and key using [aes-256-cbc](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) +- The recipient is a key transport (ktri) with the Subject Key ID equal to the protection key's public ID +- The protection key is used to obtain the `aes key` to decrypt the data block. + +The [Cid.ContentType](xref:Ipfs.Cid.ContentType) is set to `cms`. + +```csharp +var options = new AddFileOptions +{ + ProtectionKey = "me" +}; +var node = await ipfs.FileSystem.AddTextAsync("hello world", options); +``` + +## Reading + +The standard [read file methods](../filesystem.md#reading-a-file) are used to decrypt to file. +If the private key is not held by the local peer, then a `KeyNotFoundException` is thrown. + + diff --git a/doc/articles/fs/format.md b/doc/articles/fs/format.md new file mode 100644 index 00000000..6bde7687 --- /dev/null +++ b/doc/articles/fs/format.md @@ -0,0 +1,71 @@ +### Standard Format + +Here is the [Merkle DAG](https://github.com/ipfs/go-ipfs/blob/0cb22ccf359e05fb5b55a9bf2f9c515bf7d4dba7/merkledag/pb/merkledag.proto#L31-L39) +and [UnixFS Data](https://github.com/ipfs/go-ipfs/blob/0cb22ccf359e05fb5b55a9bf2f9c515bf7d4dba7/unixfs/pb/unixfs.proto#L3-L20) +of a file containing the "hello world" string. + +```json +{ + "Links": [], + "Data": "\u0008\u0002\u0012\u000bhello world\u0018\u000b" +} +``` + +`Data` is the protobuf encoding of the UnixFS Data. + +```json +{ + "Type": 2, + "Data": "aGVsbG8gd29ybGQ=", + "FileSize": 11, + "BlockSizes": null, + "HashType": null, + "Fanout": null +} +``` +### Chunked Format + +When the file's data exceeds the [chunking size](xref:Ipfs.CoreApi.AddFileOptions.ChunkSize), multiple [blocks](xref:Ipfs.CoreApi.IBlockApi) +are generated. The returned CID points to a block that has `Merkle.Links`. Each link +contains a chunk of the file. + +The following uses a chunking size of 6. A primary and two secondary blocks are created for "hello world". + +#### Primary Block + +```json +{ + "Links": [ + {"Name": "", "Hash": "QmPhmNbdBMtSQczNc4hnsMxRf5L4vfkU8jRTXDSHj8trSV", "Size": 14}, + {"Name": "", "Hash": "QmNyJpQkU1cEkBwMDhDNFstr42q55mqG5GE5Mgwug4xyGk", "Size": 13} + ], + "Data":"\u0008\u0002\u0018\u000b \u0006 \u0005" +} +{ + "Type": 2, + "Data": null, + "FileSize": 11, + "BlockSizes": [6,5], + "HashType":null, + "Fanout":null +} +``` + +#### First Link + +```json +{ + "Links": [], + "Data": "\u0008\u0002\u0012\u0006hello \u0018\u0006" +} +``` + +#### Second Link + +```json +{ + "Links": [], + "Data": "\u0008\u0002\u0012\u0005world\u0018\u0005" +} +``` + diff --git a/doc/articles/fs/raw.md b/doc/articles/fs/raw.md new file mode 100644 index 00000000..7012206d --- /dev/null +++ b/doc/articles/fs/raw.md @@ -0,0 +1,19 @@ +# Raw Leaves + +The [raw leaves option](xref:Ipfs.CoreApi.AddFileOptions.RawLeaves) specifies that +the file's data blocks are not [encapsulated](format.md) +with a Merkle DAG but simply contain the file's data. + +The [Cid.ContentType](xref:Ipfs.Cid.ContentType) is set to `raw`. + +```csharp +var options = new AddFileOptions +{ + RawLeaves = true +}; +var node = await ipfs.FileSystem.AddTextAsync("hello world", options); + +// zb2rhj7crUKTQYRGCRATFaQ6YFLTde2YzdqbbhAASkL9uRDXn +// base58btc cidv1 raw sha2-256 QmaozNR7DZHQK1ZcU9p7QdrshMvXqWK6gpu5rmrkPdT3L4 + +``` diff --git a/doc/articles/fs/wrap.md b/doc/articles/fs/wrap.md new file mode 100644 index 00000000..542e051a --- /dev/null +++ b/doc/articles/fs/wrap.md @@ -0,0 +1,34 @@ +# Wrapping + +The [wrap option](xref:Ipfs.CoreApi.AddFileOptions.Wrap) specifies that +the a directory is created for the file. + +```csharp +var path = "hello.txt"; +File.WriteAllText("hello.txt", "hello world"); +var options = new AddFileOptions +{ + Wrap = true +}; +var node = await ipfs.FileSystem.AddFileAsync(path, options); + +// QmNxvA5bwvPGgMXbmtyhxA1cKFdvQXnsGnZLCGor3AzYxJ + +``` +## Format + +Two blocks are created, a directory object and a file object. The file object +is described in [standard format](format.md). The directory object looks +like this. + +```json +{ + "Links": [ + {"Name": "hello.txt", "Hash": "Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD", "Size":19} + ], + "Data": "\u0008\u0001" +} +{ + "Type": 1, +} +``` diff --git a/doc/articles/intro.md b/doc/articles/intro.md index 8a4bdd36..555cf948 100644 --- a/doc/articles/intro.md +++ b/doc/articles/intro.md @@ -15,7 +15,7 @@ package is published on [NuGet](https://www.nuget.org/packages/Ipfs.Engine). ## Related projects - [IPFS Core](https://github.com/richardschneider/net-ipfs-core) -- [IPFS API](https://github.com/richardschneider/net-ipfs-api) +- [IPFS HTTP Client](https://github.com/richardschneider/net-ipfs-http-client) ## Other implementations diff --git a/doc/articles/toc.yml b/doc/articles/toc.yml index 2f946f82..14bde4ea 100644 --- a/doc/articles/toc.yml +++ b/doc/articles/toc.yml @@ -11,6 +11,17 @@ href: core-api.md - name: File System href: filesystem.md + items: + - name: CID only + href: fs/cid-only.md + - name: Encryption + href: fs/encryption.md + - name: Raw Leaves + href: fs/raw.md + - name: Wrapping + href: fs/wrap.md + - name: Format + href: fs/format.md - name: Repository href: repository.md items: