From e14e18627bcafbfd1cc448fe3ff98ff685e4d6e4 Mon Sep 17 00:00:00 2001 From: Dmitrii Kuvaiskii Date: Wed, 31 Jul 2024 01:13:39 -0700 Subject: [PATCH] fixup! [Docs] Add chapter on encrypted files implementation Signed-off-by: Dmitrii Kuvaiskii --- Documentation/devel/encfiles.rst | 29 +++++++++++-------- .../encfiles/02_encfiles_representation.svg | 2 +- .../img/encfiles/04_encfiles_write_less3k.svg | 2 +- .../img/encfiles/05_encfiles_read_less3k.svg | 2 +- .../encfiles/06_encfiles_write_greater3k.svg | 2 +- .../07_encfiles_write_greater3k_general.svg | 2 +- .../encfiles/08_encfiles_read_greater3k.svg | 2 +- .../09_encfiles_read_greater3k_general.svg | 2 +- 8 files changed, 24 insertions(+), 19 deletions(-) diff --git a/Documentation/devel/encfiles.rst b/Documentation/devel/encfiles.rst index 10640929a0..aaee7ad5e4 100644 --- a/Documentation/devel/encfiles.rst +++ b/Documentation/devel/encfiles.rst @@ -13,8 +13,8 @@ Encrypted Files in Gramine Gramine provides a feature of :ref:`encrypted-files`, which encrypts files and transparently decrypts them when the application reads or writes them. -Integrity- or confidentiality-sensitive files (or whole directories) accessed by -the application must be put under the "encrypted" FS mount in the Gramine +Integrity- and confidentiality-sensitive files (or whole directories) accessed +by the application must be put under the "encrypted" FS mount in the Gramine manifest. New files created in the "encrypted" FS mount are automatically treated as encrypted. The format used for encrypted and integrity-protected files is borrowed from the "protected files" feature of Intel SGX SDK (see the @@ -142,6 +142,10 @@ Crypto used for encrypted files `__ construction, with the required PRF (Pseudorandom Function) instantiated by AES-128-CMAC. +- Keys for node encryption (i.e., keys stored in the MHT nodes) are generated + randomly using a cryptographically secure pseudo-random number generator + (CSPRNG); for x86-64, this CSPRNG is the RDRAND instruction. + - Initialization vectors (IVs) are always all-zeros. This is allowed because each node-encryption key is generated randomly and is never re-used. @@ -228,10 +232,10 @@ between logical and physical numbers is clear on the below diagram. Note that there is a special MHT node -- the root MHT node. It has the same representation inside the SGX enclave and on host storage as all other MHT -nodes, but it is directly linked from the main data struct ``pf_handle`` via the -``root_mht_node`` field. Also, the root MHT node's encryption key and MAC are -stored directly in the encrypted header of the metadata node. The root MHT node -starts to be used when the plaintext file size exceeds 3KB. +nodes, but it is directly linked from the main data struct ``pf_context`` via +the ``root_mht_node`` field. Also, the root MHT node's encryption key and MAC +are stored directly in the encrypted header of the metadata node. The root MHT +node starts to be used when the plaintext file size exceeds 3KB. Note that the root MHT node is kept in trusted enclave memory for the lifetime of the file handle (i.e. as long as the file is opened). This is in contrast to @@ -290,7 +294,7 @@ represented solely in SGX enclave memory and is saved to untrusted host storage on a write (or more typically, on an explicit flush operation). Upon file creation, Gramine sets up three data structures representing the file: -the main ``pf_context`` struct that has the reference to the correspoding host +the main ``pf_context`` struct that has the reference to the corresponding host file and the user-supplied KDK, the ``metadata_node`` bounce buffer that will be copied out to host storage and the ``metadata_decrypted`` struct that has the file name, the file size and a 3KB buffer to hold file contents. @@ -329,7 +333,7 @@ already supplied by the user application, and must be the same as was used for file write. Then the app wants to read the file data. This triggers the read flow depicted -on the diagram above. The encrypted file is represented on the untrusted storage +in the diagram above. The encrypted file is represented on the untrusted storage as a single 4KB metadata node, which consists of a plaintext header, an encrypted part, and an unused padding. @@ -399,8 +403,7 @@ encrypted. Step 4 shows that the AES-GCM encryption happens in the ``encrypted`` bounce buffer of the data node, on the plaintext data-node buffer ``decrypted`` and with the newly generated key. As part of this encryption operation, the MAC is generated and is stored in the corresponding slot of the root MHT node (thus -shaping a key + MAC pair for data node 1). Since the MHT node's contents will -be encrypted, the MAC will not be leaked. +shaping a key + MAC pair for data node 1). At this point, the 4KB of the file data are stored as ciphertext in the bounce buffer of the data node and are ready to be flushed to storage. However, the @@ -418,7 +421,7 @@ encrypted. Step 6 shows that the AES-GCM encryption happens in the ``encrypted`` bounce buffer of the root MHT node, on the plaintext root-MHT-node ``decrypted`` and with the newly generated key. As part of this encryption operation, the MAC is generated and is stored in the ``root_mht_node_mac`` field of the metadata -node's header. Since the header will be encrypted, the MAC will not be leaked. +node's header. At this point, both the data node and the root MHT node are ready to be flushed to storage. Now steps 7-9 are performed, which correspond to steps 2-4 in the @@ -512,4 +515,6 @@ Additional details - It is worth pointing out that the format of encrypted files mostly uses one-time keys. The KDK is only used to derive the metadata-node key, thus it produces much less ciphertext than if it would be used to directly encrypt - file data. Therefore, the usual NIST limits would be reached much slower. + file data. Therefore, the usual NIST limits on the total number of + invocations of the encryption operation with the same key would be reached + much slower. diff --git a/Documentation/img/encfiles/02_encfiles_representation.svg b/Documentation/img/encfiles/02_encfiles_representation.svg index e847941123..e00651413e 100644 --- a/Documentation/img/encfiles/02_encfiles_representation.svg +++ b/Documentation/img/encfiles/02_encfiles_representation.svg @@ -1 +1 @@ -#0: Metadata node (header in plaintext)host_file_handlekdkmode = READ|WRITEmetadata_nodemetadata_decryptedroot_mht_nodeGlossary:•KDK = Key Derivation Key•MHT = Merkle Hash Tree•KDF = Key Derivation Function“GRAFS_PF”0x010x00KDF nonceMAC07894157encrypted metadataover583941Constant padding39424095file_path[772]file_sizeroot_mht_node_keyroot_mht_node_macfile_data[3072]type = MHT_NODElogical_node= 0physical_node= 1encrypteddecrypted#1: Root MHT node (all fields encrypted)data_key1data_mac1…mht_key1mht_mac1…data_key1data_mac1data_key96data_mac96…mht_key1mht_mac1mht_key32mht_mac32…0153130724095BouncebufferBounce bufferSGX enclave (trusted)Host storage (untrusted)3071encrypted file dataArrows legend:Encrypted on storageCopied in/out of enclave869 \ No newline at end of file +#0: Metadata node (first 58 bytesin plaintext)host_file_handlekdkmode = READ|WRITEmetadata_nodemetadata_decryptedroot_mht_nodeGlossary:•KDK = Key Derivation Key•MHT = Merkle Hash Tree•KDF = Key Derivation Function“GRAFS_PF”0x010x00KDF nonceMAC07894157encrypted metadataover583941Constant padding39424095file_path[772]file_sizeroot_mht_node_keyroot_mht_node_macfile_data[3072]type = MHT_NODElogical_node= 0physical_node= 1encrypteddecrypted#1: Root MHT node (all fields encrypted)data_key1data_mac1…mht_key1mht_mac1…data_key1data_mac1data_key96data_mac96…mht_key1mht_mac1mht_key32mht_mac32…0153130724095BouncebufferBounce bufferSGX enclave (trusted)Host storage (untrusted)3071encrypted file dataArrows legend:Encrypted on storageCopied in/out of enclave869 \ No newline at end of file diff --git a/Documentation/img/encfiles/04_encfiles_write_less3k.svg b/Documentation/img/encfiles/04_encfiles_write_less3k.svg index 1d2e3bd43d..1e772be333 100644 --- a/Documentation/img/encfiles/04_encfiles_write_less3k.svg +++ b/Documentation/img/encfiles/04_encfiles_write_less3k.svg @@ -1 +1 @@ -#0: Metadata node (header in plaintext)host_file_handlekdkmode = READ|WRITEmetadata_nodemetadata_decryptedroot_mht_nodeGlossary:•KDK = Key Derivation Key•MHT = Merkle Hash Tree•KDF = Key Derivation Function“GRAFS_PF”0x010x00KDF nonceMAC07894157encrypted metadataover583941Constant padding39424095file_path[772]file_sizeroot_mht_node_keyroot_mht_node_macfile_data[3072]type = MHT_NODElogical_node= 0physical_node= 1encrypteddecrypted#1: Root MHT node (all fields encrypted)data_key1data_mac1…mht_key1mht_mac1…data_key1data_mac1data_key96data_mac96…mht_key1mht_mac1mht_key32mht_mac32…0153130724095BouncebufferBounce bufferSGX enclave (trusted)Host storage (untrusted)3071encrypted file dataArrows legend:Encrypted on storageCopied out of enclaveUnused in this caseCase of file size < 3KB:Encrypting new file②KDF nonce = rand()③derive key④encrypt⑤copy out869①file_data← write(buf)②④ \ No newline at end of file +#0: Metadata node (first 58 bytes in plaintext)host_file_handlekdkmode = READ|WRITEmetadata_nodemetadata_decryptedroot_mht_nodeGlossary:•KDK = Key Derivation Key•MHT = Merkle Hash Tree•KDF = Key Derivation Function“GRAFS_PF”0x010x00KDF nonceMAC07894157encrypted metadataover583941Constant padding39424095file_path[772]file_sizeroot_mht_node_keyroot_mht_node_macfile_data[3072]type = MHT_NODElogical_node= 0physical_node= 1encrypteddecrypted#1: Root MHT node (all fields encrypted)data_key1data_mac1…mht_key1mht_mac1…data_key1data_mac1data_key96data_mac96…mht_key1mht_mac1mht_key32mht_mac32…0153130724095BouncebufferBounce bufferSGX enclave (trusted)Host storage (untrusted)3071encrypted file dataArrows legend:Encrypted on storageCopied out of enclaveUnused in this caseCase of file size < 3KB:Encrypting new file②KDF nonce = rand()③derive key④encrypt⑤copy out869①file_data← write(buf)②④ \ No newline at end of file diff --git a/Documentation/img/encfiles/05_encfiles_read_less3k.svg b/Documentation/img/encfiles/05_encfiles_read_less3k.svg index d133c8f60d..9d87f8860c 100644 --- a/Documentation/img/encfiles/05_encfiles_read_less3k.svg +++ b/Documentation/img/encfiles/05_encfiles_read_less3k.svg @@ -1 +1 @@ -#0: Metadata node (header in plaintext)host_file_handlekdkmode = READ|WRITEmetadata_nodemetadata_decryptedroot_mht_nodeGlossary:•KDK = Key Derivation Key•MHT = Merkle Hash Tree•KDF = Key Derivation Function“GRAFS_PF”0x010x00KDF nonceMAC07894157encrypted metadataover583941Constant padding39424095file_path[772]file_sizeroot_mht_node_keyroot_mht_node_macfile_data[3072]type = MHT_NODElogical_node= 0physical_node= 1encrypteddecrypted#1: Root MHT node (all fields encrypted)data_key1data_mac1…mht_key1mht_mac1…data_key1data_mac1data_key96data_mac96…mht_key1mht_mac1mht_key32mht_mac32…0153130724095BouncebufferBounce bufferSGX enclave (trusted)Host storage (untrusted)3071encrypted file dataArrows legend:Decrypted from storageCopied into enclaveUnused in this caseCase of file size < 3KB:Decrypting existing file②KDF nonce = loaded③derive key④decrypt①copy in (on file open)869⑤read(buf) ← file_data②④ \ No newline at end of file +#0: Metadata node (first 58 bytes in plaintext)host_file_handlekdkmode = READ|WRITEmetadata_nodemetadata_decryptedroot_mht_nodeGlossary:•KDK = Key Derivation Key•MHT = Merkle Hash Tree•KDF = Key Derivation Function“GRAFS_PF”0x010x00KDF nonceMAC07894157encrypted metadataover583941Constant padding39424095file_path[772]file_sizeroot_mht_node_keyroot_mht_node_macfile_data[3072]type = MHT_NODElogical_node= 0physical_node= 1encrypteddecrypted#1: Root MHT node (all fields encrypted)data_key1data_mac1…mht_key1mht_mac1…data_key1data_mac1data_key96data_mac96…mht_key1mht_mac1mht_key32mht_mac32…0153130724095BouncebufferBounce bufferSGX enclave (trusted)Host storage (untrusted)3071encrypted file dataArrows legend:Decrypted from storageCopied into enclaveUnused in this caseCase of file size < 3KB:Decrypting existing file②KDF nonce = loaded③derive key④decrypt①copy in (on file open)869⑤read(buf) ← file_data②④ \ No newline at end of file diff --git a/Documentation/img/encfiles/06_encfiles_write_greater3k.svg b/Documentation/img/encfiles/06_encfiles_write_greater3k.svg index 5a52ea42fc..dd8802f3f0 100644 --- a/Documentation/img/encfiles/06_encfiles_write_greater3k.svg +++ b/Documentation/img/encfiles/06_encfiles_write_greater3k.svg @@ -1 +1 @@ -#0: Metadata node (header in plaintext)host_file_handlekdkmode = READ|WRITEmetadata_nodemetadata_decryptedroot_mht_nodeGlossary:•KDK = Key Derivation Key•MHT = Merkle Hash Tree•KDF = Key Derivation Function“GRAFS_PF”0x010x00KDF nonceMAC07894157encrypted metadataover583941Constant padding39424095file_path[772]file_sizeroot_mht_node_keyroot_mht_node_macfile_data[3072]type = MHT_NODElogical_node= 0physical_node= 1encrypteddecrypted#1: Root MHT node (all fields encrypted)…data_key1data_mac1data_key96data_mac96…mht_key1mht_mac1mht_key32mht_mac32…0153130724095BouncebufferBounce bufferSGX enclave (trusted)Host storage (untrusted)3071encrypted file dataArrows legend:Encrypted on storageCopied out of enclaveUpdates MHT dataCase of file size > 3KB:Encrypting new file⑦KDF nonce = rand()⑧derive key⑨encrypt⑩copy out869①file_data← write(buf) …⑦⑨type = DATA_NODElogical_node= 0physical_node= 2encrypteddecrypted…Bouncebuffer#2: Data node (all encrypted)②… new.decrypted← write(buf)③key = rand()④encrypt⑤key = rand()⑥encrypt⑩copy out⑩copy out③④⑤⑥ \ No newline at end of file +#0: Metadata node (first 58 bytes in plaintext)host_file_handlekdkmode = READ|WRITEmetadata_nodemetadata_decryptedroot_mht_nodeGlossary:•KDK = Key Derivation Key•MHT = Merkle Hash Tree•KDF = Key Derivation Function“GRAFS_PF”0x010x00KDF nonceMAC07894157encrypted metadataover583941Constant padding39424095file_path[772]file_sizeroot_mht_node_keyroot_mht_node_macfile_data[3072]type = MHT_NODElogical_node= 0physical_node= 1encrypteddecrypted#1: Root MHT node (all fields encrypted)…data_key1data_mac1data_key96data_mac96…mht_key1mht_mac1mht_key32mht_mac32…0153130724095BouncebufferBounce bufferSGX enclave (trusted)Host storage (untrusted)3071encrypted file dataArrows legend:Encrypted on storageCopied out of enclaveUpdates MHT dataCase of file size > 3KB:Encrypting new file⑦KDF nonce = rand()⑧derive key⑨encrypt⑩copy out869①file_data← write(buf) …⑦⑨type = DATA_NODElogical_node= 0physical_node= 2encrypteddecrypted…Bouncebuffer#2: Data node (all encrypted)②… new.decrypted← write(buf)③key = rand()④encrypt⑤key = rand()⑥encrypt⑩copy out⑩copy out③④⑤⑥ \ No newline at end of file diff --git a/Documentation/img/encfiles/07_encfiles_write_greater3k_general.svg b/Documentation/img/encfiles/07_encfiles_write_greater3k_general.svg index 1f819bb1ca..6bae355a60 100644 --- a/Documentation/img/encfiles/07_encfiles_write_greater3k_general.svg +++ b/Documentation/img/encfiles/07_encfiles_write_greater3k_general.svg @@ -1 +1 @@ -#0: Metadata#1: Root MHT0#98: MHT1#3203: MHT33…KDK①write(buf)②③BouncebufferkeyMAC④⑤Bouncebuffer⑥keyMACBouncebuffer……keyMACBouncebuffer⑤⑥Root MHT keyRoot MHT MACderived keyrandom nonce⑦Bouncebuffer⑧⑨Metadata header is stored in plaintextCase of file size > 3KB:Encrypting new fileMetadata payload and MHT/data nodes encryptedMetadata MAC \ No newline at end of file +#0: Metadata#1: Root MHT0#98: MHT1#3203: MHT33…KDK①write(buf)②③BouncebufferkeyMAC④⑤Bouncebuffer⑥keyMACBouncebuffer……keyMACBouncebuffer⑤⑥Root MHT keyRoot MHT MACderived keyrandom nonce⑦Bouncebuffer⑧⑨First 58 bytes of metadata node are stored in plaintextCase of file size > 3KB:Encrypting new fileRest of metadata node and MHT/data nodes encryptedMetadata MAC \ No newline at end of file diff --git a/Documentation/img/encfiles/08_encfiles_read_greater3k.svg b/Documentation/img/encfiles/08_encfiles_read_greater3k.svg index 556f224262..696f2aeb68 100644 --- a/Documentation/img/encfiles/08_encfiles_read_greater3k.svg +++ b/Documentation/img/encfiles/08_encfiles_read_greater3k.svg @@ -1 +1 @@ -#0: Metadata node (header in plaintext)host_file_handlekdkmode = READ|WRITEmetadata_nodemetadata_decryptedroot_mht_nodeGlossary:•KDK = Key Derivation Key•MHT = Merkle Hash Tree•KDF = Key Derivation Function“GRAFS_PF”0x010x00KDF nonceMAC07894157encrypted metadataover583941Constant padding39424095file_path[772]file_sizeroot_mht_node_keyroot_mht_node_macfile_data[3072]type = MHT_NODElogical_node= 0physical_node= 1encrypteddecrypted#1: Root MHT node (all fields encrypted)…data_key1data_mac1data_key96data_mac96…mht_key1mht_mac1mht_key32mht_mac32…0153130724095BouncebufferBounce bufferSGX enclave (trusted)Host storage (untrusted)3071encrypted file dataArrows legend:Decrypted from storageCopied into enclaveCase of file size > 3KB:Decrypting existing file②KDF nonce = loaded③derive key④decrypt869②④type = DATA_NODElogical_node= 0physical_node= 2encrypteddecrypted…Bouncebuffer#2: Data node (all encrypted)⑧decrypt⑥decrypt⑤copy in⑦copy in①copy in (on file open)⑥⑥⑧⑧⑩… read(buf) ← node.decrypted⑨read(buf) ← file_data… \ No newline at end of file +#0: Metadata node (first 58 bytes in plaintext)host_file_handlekdkmode = READ|WRITEmetadata_nodemetadata_decryptedroot_mht_nodeGlossary:•KDK = Key Derivation Key•MHT = Merkle Hash Tree•KDF = Key Derivation Function“GRAFS_PF”0x010x00KDF nonceMAC07894157encrypted metadataover583941Constant padding39424095file_path[772]file_sizeroot_mht_node_keyroot_mht_node_macfile_data[3072]type = MHT_NODElogical_node= 0physical_node= 1encrypteddecrypted#1: Root MHT node (all fields encrypted)…data_key1data_mac1data_key96data_mac96…mht_key1mht_mac1mht_key32mht_mac32…0153130724095BouncebufferBounce bufferSGX enclave (trusted)Host storage (untrusted)3071encrypted file dataArrows legend:Decrypted from storageCopied into enclaveCase of file size > 3KB:Decrypting existing file②KDF nonce = loaded③derive key④decrypt869②④type = DATA_NODElogical_node= 0physical_node= 2encrypteddecrypted…Bouncebuffer#2: Data node (all encrypted)⑧decrypt⑥decrypt⑤copy in⑦copy in①copy in (on file open)⑥⑥⑧⑧⑩… read(buf) ← node.decrypted⑨read(buf) ← file_data… \ No newline at end of file diff --git a/Documentation/img/encfiles/09_encfiles_read_greater3k_general.svg b/Documentation/img/encfiles/09_encfiles_read_greater3k_general.svg index 09e41da393..624958edfc 100644 --- a/Documentation/img/encfiles/09_encfiles_read_greater3k_general.svg +++ b/Documentation/img/encfiles/09_encfiles_read_greater3k_general.svg @@ -1 +1 @@ -#0: Metadata#1: Root MHT0#98: MHT1#3203: MHT33…KDK⑨read(buf)⑩BouncebufferkeyMAC⑧⑤Bouncebuffer⑥keyMACBouncebuffer……keyMACBouncebuffer⑥Root MHT keyRoot MHT MACderived keyloaded nonce②Bouncebuffer③④Metadata header is stored in plaintextCase of file size > 3KB:Decrypting existing fileMetadata payload and MHT/data nodes encryptedMetadata MAC①…⑤⑦ \ No newline at end of file +#0: Metadata#1: Root MHT0#98: MHT1#3203: MHT33…KDK⑨read(buf)⑩BouncebufferkeyMAC⑧⑤Bouncebuffer⑥keyMACBouncebuffer……keyMACBouncebuffer⑥Root MHT keyRoot MHT MACderived keyloaded nonce②Bouncebuffer③④Case of file size > 3KB:Decrypting existing fileMetadata MAC①…⑤⑦First 58 bytes of metadata node are stored in plaintextRest of metadata node and MHT/data nodes encrypted \ No newline at end of file