[Docs] [txt|pdf|xml] [Tracker] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01 02

Network Working Group                                    P. Hallam-Baker
Internet-Draft                                         Comodo Group Inc.
Intended status: Informational                           August 27, 2018
Expires: February 28, 2019


             Data At Rest Encryption Part 2: DARE Container
                  draft-hallambaker-dare-container-02

Abstract

   This document describes DARE Container, a message and file syntax
   that allows an append-only sequence of data frames to be represented
   with cryptographic integrity, signature and encryption enhancements.
   The format supports data integrity checks using digest chains and
   Merkle trees.  The simplest supports efficient write operations and
   efficient read operations in either the forward or reverse direction.
   Support for efficient random-access reads may be provided through the
   use of binary trees or index records appended to the end of the file.

   This document is also available online at
   http://mathmesh.com/Documents/draft-hallambaker-dare-container.html
   [1] .

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on February 28, 2019.

Copyright Notice

   Copyright (c) 2018 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents



Hallam-Baker            Expires February 28, 2019               [Page 1]


Internet-Draft             DARE Data Container               August 2018


   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.1.  Container Format  . . . . . . . . . . . . . . . . . . . .   3
     1.2.  Write . . . . . . . . . . . . . . . . . . . . . . . . . .   4
     1.3.  Read Access . . . . . . . . . . . . . . . . . . . . . . .   5
     1.4.  Encryption and Authentication . . . . . . . . . . . . . .   5
     1.5.  Integrity and Signature . . . . . . . . . . . . . . . . .   5
     1.6.  Redaction . . . . . . . . . . . . . . . . . . . . . . . .   6
     1.7.  Alternative approaches  . . . . . . . . . . . . . . . . .   6
     1.8.  Efficiency  . . . . . . . . . . . . . . . . . . . . . . .   7
   2.  Definitions . . . . . . . . . . . . . . . . . . . . . . . . .   7
     2.1.  Related Specifications  . . . . . . . . . . . . . . . . .   7
     2.2.  Requirements Language . . . . . . . . . . . . . . . . . .   8
   3.  Container Navigation  . . . . . . . . . . . . . . . . . . . .   8
     3.1.  Tree  . . . . . . . . . . . . . . . . . . . . . . . . . .   9
     3.2.  Position Index  . . . . . . . . . . . . . . . . . . . . .   9
     3.3.  Metadata Index  . . . . . . . . . . . . . . . . . . . . .  10
   4.  Integrity Mechanisms  . . . . . . . . . . . . . . . . . . . .  10
     4.1.  Digest Chain calculation  . . . . . . . . . . . . . . . .  10
     4.2.  Binary Merkle tree calculation  . . . . . . . . . . . . .  10
   5.  Reference . . . . . . . . . . . . . . . . . . . . . . . . . .  11
     5.1.  Container Headers . . . . . . . . . . . . . . . . . . . .  11
       5.1.1.  Structure: ContainerHeaderFirst . . . . . . . . . . .  11
       5.1.2.  Structure: ContainerHeader  . . . . . . . . . . . . .  11
     5.2.  Content Metadata Structure  . . . . . . . . . . . . . . .  12
       5.2.1.  Structure: ContentMeta  . . . . . . . . . . . . . . .  12
     5.3.  Index Structures  . . . . . . . . . . . . . . . . . . . .  13
       5.3.1.  Structure: ContainerIndex . . . . . . . . . . . . . .  13
       5.3.2.  Structure: IndexPosition  . . . . . . . . . . . . . .  13
       5.3.3.  Structure: KeyValue . . . . . . . . . . . . . . . . .  13
       5.3.4.  Structure: IndexMeta  . . . . . . . . . . . . . . . .  13
     5.4.  Signature . . . . . . . . . . . . . . . . . . . . . . . .  14
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .  14
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  14
   8.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  14
   9.  Appendix A: Examples and Test Vectors . . . . . . . . . . . .  14
     9.1.  Simple container  . . . . . . . . . . . . . . . . . . . .  15
     9.2.  Payload and chain digests . . . . . . . . . . . . . . . .  16
     9.3.  Merkle Tree . . . . . . . . . . . . . . . . . . . . . . .  17



Hallam-Baker            Expires February 28, 2019               [Page 2]


Internet-Draft             DARE Data Container               August 2018


     9.4.  Signed container  . . . . . . . . . . . . . . . . . . . .  19
     9.5.  Encrypted container . . . . . . . . . . . . . . . . . . .  21
   10. Appendix B  . . . . . . . . . . . . . . . . . . . . . . . . .  25
   11. References  . . . . . . . . . . . . . . . . . . . . . . . . .  25
     11.1.  Normative References . . . . . . . . . . . . . . . . . .  25
     11.2.  Informative References . . . . . . . . . . . . . . . . .  26
     11.3.  URIs . . . . . . . . . . . . . . . . . . . . . . . . . .  26
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  26

1.  Introduction

   DARE Container is a message and file syntax that allows a sequence of
   data frames to be represented with cryptographic integrity,
   signature, and encryption enhancements to be constructed in an append
   only format.  DARE Container was developed in response to needs that
   arose out of the design of the Mathematical Mesh
   [draft-hallambaker-mesh-architecture] . It is built on the binary
   encodings of JSON data objects, JSON-B and JSON-C
   [draft-hallambaker-jsonbcd] and the DARE Message format
   [draft-hallambaker-dare-message] .

   The format is designed to meet the requirements of a wide range of
   use cases including:

   o  Recording transactions in persistent storage.

   o  Synchronizing transaction logs between hosts.

   o  File archive.

   o  Message spool.

   o  Signing and encrypting single data items.

   o  Incremental encryption and authentication of server logs.

1.1.  Container Format

   A DARE Container consists of a sequence of variable length frames.
   Each frame consists of a forward length indicator, the framed data
   and a reverse length indicator.  The reverse length indicator is
   written out backwards allowing the length and thus the frame to be
   read in the reverse direction:








Hallam-Baker            Expires February 28, 2019               [Page 3]


Internet-Draft             DARE Data Container               August 2018


   [[This figure is not viewable in this format.  The figure is
   available at http://mathmesh.com/Documents/draft-hallambaker-dare-
   container.html [2].]]


   JBCD Bidirectional Frame

   Each frame contains a single DARE Message consisting of a Header,
   Payload and Trailer (if required).  The first frame in a container
   describes the container format options and defaults.  These include
   the range of encoding options for frame metadata supported and the
   container profiles to which the container conforms.

   All internal data formats are 64 bit clean allowing for containers of
   up to 18 exabytes to be written.

   Five container types are currently specified but this may be reduced
   if the Digest and Tree types are withdrawn.  These are:

   Simple  The container does not provide any index or content integrity
      checks.

   Tree  Frame headers contain entries that specify the start position
      of previous frames at the apex of the immediately enclosing binary
      tree.  This enables efficient random access to any frame in the
      file.

   Digest  Each frame trailer contains a PayloadDigest field.
      Modification of the payload will cause verification of the
      PayloadDigest value to fail on that frame.

   Chain  Each frame trailer contains PayloadDigest and ChainDigest
      fields allowing modifications to the payload data to be detected.
      Modification of the payload will cause verification of the
      PayloadDigest value to fail on that frame and verification of the
      ChainDigest value to fail on all subsequent frames.

   Merkle Tree  Frame headers contain entries that specify the start
      position of previous frames at the apex of the immediately
      enclosing binary tree.  Frame Trailers contain
      TreeDigestPartial and TreeDigestFinal entries forming a Merkle
      digest tree.

1.2.  Write

   In normal circumstances, DARE Containers are written as an append
   only log.  As with DARE Messages, integrity information (payload
   digest, signatures) is written to the message trailer.  Thus, large



Hallam-Baker            Expires February 28, 2019               [Page 4]


Internet-Draft             DARE Data Container               August 2018


   payloads may be written without the need to buffer the payload data
   provided that the content length is known in advance.

1.3.  Read Access

   The use of reverse length indicators allows DARE containers to
   support efficient sequential access in either the forward or reverse
   directions.

   Random access to any part of a file MAY be supported by means of a
   binary tree index and/or an index record providing direct access to
   any part of the file.

1.4.  Encryption and Authentication

   Frame payloads and associated attributes MAY be encrypted and/or
   authenticated using the approach described in
   [draft-hallambaker-dare-message] .

   Incremental encryption is supported allowing encryption parameters
   from a single public key exchange operation to be applied to encrypt
   multiple frames.  The public key exchange information is specified in
   the first encrypted frame and subsequent frames encrypted under those
   parameters specify the location at which the key exchange information
   is to be found by means of the ExchangePosition field.

   The only restriction on the use of incremental encryption is that the
   frame containing the key exchange information MUST precede the frames
   that reference the exchange parameters.

   To avoid cryptographic vulnerabilities resulting from key re-use, the
   DARE key exchange requires that each encrypted sequence use an
   encryption key and initialization vector derived from the master key
   established in the public key exchange by means of a unique salt.

   Each DARE Message and by extension, each DARE Container frame MUST
   specify a unique salt value of at least 128 bits.  Since the
   encryption key is derived from the salt value by means of a Key
   Derivation Function, erasure of the salt MAY be used as a means of
   rendering the payload plaintext value inaccessible without changing
   the payload value.

1.5.  Integrity and Signature

   Signatures MAY be applied to a payload digest, the final digest in a
   chain or tree.  The chain and tree digest modes allow a single
   signature to be used to authenticate all frame payloads in a
   container.



Hallam-Baker            Expires February 28, 2019               [Page 5]


Internet-Draft             DARE Data Container               August 2018


   The tree signature mode is particularly suited to applications such
   as file archives as it allows files to be verified individually
   without requiring the signer to sign each individually.  Furthermore,
   in applications such as code signing, it allows a single signature to
   be used to verify both the integrity of the code and its membership
   of the distribution.

   As with DARE Message, the signature mechanism does not specify the
   interpretation of the signature semantics.  The presence of a
   signature demonstrates that the holder of the private key applied it
   to the specified digest value but not their motive for doing so.
   Describing such semantics is beyond the scope of this document and is
   deferred to future work.

1.6.  Redaction

   The chief disadvantage of using an append-only format is that
   containers only increase in size.  In many applications, much of the
   data in the container becomes redundant or obsolete and a process
   analogous to garbage collection is required.  This process is called
   redaction.

   The simplest method of redaction is to create a new container and
   sequentially copy each entry from the old container to the new,
   discarding redundant frames and obsolete header information.

   For example, partial index records may be consolidated into a single
   index record placed in the last frame of the container.  Unnecessary
   signature and integrity data may be discarded and so on.

   It is also possible but not necessarily advisable to perform such a
   redaction in-place provided that the redaction process does not
   increase the length of any individual frame and that appropriate
   provision is made for file locking to prevent conflicts and to
   provide for safe resumption should an interruption occur during the
   process.

1.7.  Alternative approaches

   Many file proprietary formats are in use that support some or all of
   these capabilities but only a handful have public, let alone open,
   standards.  DARE Container is designed to provide a superset of the
   capabilities of existing message and file syntaxes, including:

   o  Cryptographic Message Syntax [RFC5652] defines a syntax used to
      digitally sign, digest, authenticate, or encrypt arbitrary message
      content.




Hallam-Baker            Expires February 28, 2019               [Page 6]


Internet-Draft             DARE Data Container               August 2018


   o  The.ZIP File Format specification [ZIPFILE] developed by Phil
      Katz.

   o  The BitCoin Block chain [BLOCKCHAIN] .

   o  JSON Web Encryption and JSON Web Signature

   Attempting to make use of these specifications in a layered fashion
   would require at least three separate encoders and introduce
   unnecessary complexity.  Furthermore, there is considerable overlap
   between the specifications providing multiple means of achieving the
   same ends, all of which must be supported if decoders are to work
   reliably.

1.8.  Efficiency

   Every data format represents a compromise between different concerns,
   in particular:

   Compactness  The space required to record data in the encoding.

   Memory Overhead  The additional volatile storage (RAM) required to
      maintain indexes etc. to support efficient retrieval operations.

   Number of Operations  The number of operations required to retrieve
      data from or append data to an existing encoded sequence.

   Number of Disk Seek Operations  Optimizing the response time of
      magnetic storage media to random access read requests has
      traditionally been one of the central concerns of database design.
      The DARE Container format is designed to the assumption that this
      will cease to be a concern as solid state media replaces magnetic.

   While the cost of storage of all types has declined rapidly over the
   past decades, so has the amount of data to be stored.  DARE Container
   represents a pragmatic balance of these considerations for current
   technology.  In particular, since payload volumes are likely to be
   very large, memory and operational efficiency are considered higher
   priorities than compactness.

2.  Definitions

2.1.  Related Specifications

   DARE Container makes use of the following related standards and
   specifications.





Hallam-Baker            Expires February 28, 2019               [Page 7]


Internet-Draft             DARE Data Container               August 2018


   Encoding  Content frame headers are encoded using JavaScript Object
      Notation (JSON) [RFC7159] , JSON-B or JSON-C
      [draft-hallambaker-jsonbcd] .

   Cryptography  The encryption and signature schemes used are based on
      JSON Web Signature [RFC7515] and JSON Web Encryption [RFC7516] as
      applied in DARE Message [draft-hallambaker-dare-message] .

2.2.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119] .

3.  Container Navigation

   Three means of locating frames in a container are supported:

   Sequential  Access frames sequentially starting from the start or the
      end of the container.

   Binary search  Access any container frame by frame number in
      O(log2(n)) time by means of a binary tree constructed while the
      container is written.

   Index  Access and container frame by frame number or by key by means
      of an index record.

   All DARE Containers support sequential access.  Only tree and Merkle
   tree containers support binary search access.  An index frame MAY be
   written appended to any container and provides O(1) access to any
   frame listed in the index.

   Two modes of compilation are considered:

   Monolithic  Frames are added to the container in a single operation,
      e.g. file archives,

   Incremental  Additional frames are written to the container at
      various intervals after it was originally created, e.g. server
      logs, message spools.

   In the monolithic mode, navigation requirements are best met by
   writing an index frame to the end of the container when it is
   complete.  It is not necessary to construct a binary search tree
   unless a Merkle tree integrity check is required.





Hallam-Baker            Expires February 28, 2019               [Page 8]


Internet-Draft             DARE Data Container               August 2018


   In the incremental mode, Binary search provides an efficient means of
   locating frames by frame number but not by key.  Writing a complete
   index to the container every m write operations provides O(m) search
   access but requires O(n2) storage.

   Use of partial indexes provides a better compromise between speed and
   efficiency.  A partial index is written out every m frames where m is
   a power of two.  A complete index is written every time a binary tree
   apex record is written.  This approach provides for O(log2(n)) search
   with incremental compilation with approximately double the overhead
   of the monolithic case.

3.1.  Tree

   Binary search is supported by means of the TreePosition parameter
   specified in the FrameHeader.  This parameter specifies the value of
   the immediately preceding apex.

   Calculation of the immediately preceding apex is most easily
   described by representing the array index in binary with base of 1
   (rather than 0).  An array index that is a power of 2 (2, 4, 8, 16,
   etc.) will be the apex of a complete tree.  Every other array index
   has the value of the sum of a set of powers of 2 and the immediately
   preceding apex will be the value of the next smallest power of 2 in
   the sum.

   For example, to find the immediately preceding apex for frame 5, we
   add 1 to get 6. 6 = 4 + 2, so we ignore the 2 and the preceding frame
   is 4.

   The values of Tree Position are shown for the first 8 frames in
   figure xx below:

   [[This figure is not viewable in this format.  The figure is
   available at http://mathmesh.com/Documents/draft-hallambaker-dare-
   container.html [3].]]


   Merkle Tree Integrity check

   An algorithm for efficiently calculating the immediately preceding
   apex is provided in Appendix C.

3.2.  Position Index

   Contains a table of frame number, position pairs pointing to prior
   locations in the file.




Hallam-Baker            Expires February 28, 2019               [Page 9]


Internet-Draft             DARE Data Container               August 2018


3.3.  Metadata Index

   Contains a list of IndexMeta entries.  Each entry contains a metadata
   description and a list of frame indexes (not positions) of frames
   that match the description.

4.  Integrity Mechanisms

   Frame sequences in a DARE container MAY be protected against a frame
   insertion attack by means of a digest chain, a binary Merkle tree or
   both.

4.1.  Digest Chain calculation

   A digest chain is simple to implement but can only be verified if the
   full chain of values is known.  Appending a frame to the chain has
   O(1) complexity but verification has O(n) complexity:

   [[This figure is not viewable in this format.  The figure is
   available at http://mathmesh.com/Documents/draft-hallambaker-dare-
   container.html [4].]]


   Hash chain integrity check

   The value of the chain digest for the first frame (frame 0) is
   H(H(null)+H(Payload0)), where null is a zero length octet sequence
   and payloadn is the sequence of payload data bytes for frame n

   The value of the chain digest for frame n is H(H(Payloadn-1
   +H(Payloadn)), where A+B stands for concatenation of the byte
   sequences A and B.

4.2.  Binary Merkle tree calculation

   The tree index mechanism describe earlier may be used to implement a
   binary Merkle tree.  The value TreeDigest specifies the apex value of
   the tree for that node.

   Appending a frame to the chain has O(log2n) complexity provided that
   the container format supports at least the binary tree index.
   Verifying a chain has O(log2 n) complexity, provided that the set of
   necessary digest inputs is known.

   To calculate the value of the tree digest for a node, we first
   calculate the values of all the sub trees that have their apex at
   that node and then calculate the digest of that value and the
   immediately preceding local apex.



Hallam-Baker            Expires February 28, 2019              [Page 10]


Internet-Draft             DARE Data Container               August 2018


5.  Reference

   TBS stuff

5.1.  Container Headers

   TBS stuff

5.1.1.  Structure: ContainerHeaderFirst

   Inherits: ContainerHeader

   Inherits: ContainerHeader

   DataEncoding: String (Optional)  Specifies the data encoding for the
      header section of for the following frames.  This value is ONLY
      valid in Frame 0 which MUST have a header encoded in JSON.

5.1.2.  Structure: ContainerHeader

   Inherits: DAREHeader

   Describes a container header.  A container header MAY contain any
   DARE Message header.

   Index: Integer (Optional)  The record index within the file.  This
      MUST be unique and satisfy any additional requirements determined
      by the ContainerType.

   ContainerType: String (Optional)  Specifies the container type for
      the following records.

   IsMeta: Boolean (Optional)  If true, the current frame is a meta
      frame and does not contain a payload.

      Note: Meta frames MAY be present in any container.  Applications
      MUST accept containers that contain meta frames at any position in
      the file.  Applications MUST NOT interpret a meta frame as a data
      frame with an enpty payload.

   UniqueID: String (Optional)  Unique object identifier

   ContentMeta: ContentMeta (Optional)  Content meta data.

   TreePosition: Integer (Optional)  Position of the frame containing
      the apex of the preceding sub-tree.





Hallam-Baker            Expires February 28, 2019              [Page 11]


Internet-Draft             DARE Data Container               August 2018


   IndexPosition: Integer (Optional)  Specifies the position in the file
      at which the last index entry is to be found

   ExchangePosition: Integer (Optional)  Specifies the position in the
      file at which the key exchange data is to be found

   ContainerIndex: ContainerIndex (Optional)  An index of records in the
      current container up to but not including this one.

   PayloadDigest: Binary (Optional)  If present, contains the digest of
      the Payload.

   ChainDigest: Binary (Optional)  If present, contains the digest of
      the PayloadDigest values of this frame and the frame immediately
      preceding.

   TreeDigest: Binary (Optional)  If present, contains the Binary Merkle
      Tree digest value.

   Event: String (Optional)  Unique object identifier

   Labels: String [0..Many]  List of labels that are applied to the
      payload of the frame.

   KeyValues: KeyValue [0..Many]  List of key/value pairs describing the
      payload of the frame.

   First: Integer (Optional)  Frame number of the first object instance
      value.

   Previous: Integer (Optional)  Frame number of the immediately prior
      object instance value

5.2.  Content Metadata Structure

   TBS stuff

5.2.1.  Structure: ContentMeta

   Information describing the object instance

   ContentType: String (Optional)  The content type field as specified
      in JWE

   Paths: String [0..Many]  List of filename paths for the payload of
      the frame.

   UniqueID: String (Optional)  Unique object identifier



Hallam-Baker            Expires February 28, 2019              [Page 12]


Internet-Draft             DARE Data Container               August 2018


   Created: DateTime (Optional)  Initial creation date.

   Modified: DateTime (Optional)  Date of last modification.

5.3.  Index Structures

   TBS stuff

5.3.1.  Structure: ContainerIndex

   A container index

   Full: Boolean (Optional)  If true, the index is complete and contains
      position entries for all the frames in the file.  If absent or
      false, the index is incremental and only contains position entries
      for records added since the last frame containing a
      ContainerIndex.

   Positions: IndexPosition [0..Many]  List of container position
      entries

   Metas: IndexMeta [0..Many]  List of container position entries

5.3.2.  Structure: IndexPosition

   Specifies the position in a file at which a specified record index is
   found

   Index: Integer (Optional)  The record index within the file.

   Position: Integer (Optional)  The record position within the file
      relative to the index base.

5.3.3.  Structure: KeyValue

   Specifies a key/value entry

   Key: String (Optional)  The key

   Value: String (Optional)  The value corresponding to the key

5.3.4.  Structure: IndexMeta

   Specifies the list of index entries at which a record with the
   specified metadata occurrs.

   Index: Integer [0..Many]  List of record indicies within the file
      where frames matching the specified criteria are found.



Hallam-Baker            Expires February 28, 2019              [Page 13]


Internet-Draft             DARE Data Container               August 2018


   ContentType: String (Optional)  Content type parameter

   Paths: String [0..Many]  List of filename paths for the current
      frame.

   Labels: String [0..Many]  List of labels that are applied to the
      current frame.

5.4.  Signature

   Payload data MAY be signed using a JWS [RFC7515] as applied in the
   DARE Message format [draft-hallambaker-dare-message] .

   Signatures are specified by the Signatures parameter in the content
   header.  The data that the signature is calculated over is defined by
   the typ parameter of the Signature as follows.

   Payload  The value of the PayloadDigest parameter

   Chain  The value of the ChainDigest parameter

   Tree  The value of the TreeDigestFinal parameter

   If the typ parameter is absent, the value Payload is implied.

   A frame MAY contain multiple signatures created with the same signing
   key and different typ values.

   The use of signatures over chain and tree digest values permit
   multiple frames to be validated using a single signature verification
   operation.

6.  Security Considerations

7.  IANA Considerations

8.  Acknowledgements

9.  Appendix A: Examples and Test Vectors

   The data payloads in all the following examples are identical, only
   the authentication and/or encryption is different.

   o  Frame 1..n consists of 300 bytes being the byte sequence 00, 01,
      02, etc.  repeating after 256 bytes.






Hallam-Baker            Expires February 28, 2019              [Page 14]


Internet-Draft             DARE Data Container               August 2018


   For conciseness, the raw data format is omitted for examples after
   the first, except where the data payload has been transformed, (i.e.
   encrypted).

9.1.  Simple container

   the following example shows a simple container with first frame and a
   single data frame:

   f4 5d
   f0 59
   f0 00
   5d f4
   f5 01 40
   f0 0f
   f1 01 2c
   40 01 f5

                                 Figure 1

   Since there is no integrity check, there is no need for trailer
   entries.  The header values are:

   Frame 0

   {
     "Index":0,
     "ContainerType":"List",
     "ContentMeta":{},
     "DataEncoding":"JSON"}

   [Empty trailer]

                                 Figure 2

   Frame 1

   {
     "Index":1}

   [Empty trailer]

                                 Figure 3








Hallam-Baker            Expires February 28, 2019              [Page 15]


Internet-Draft             DARE Data Container               August 2018


9.2.  Payload and chain digests

   The following example shows a chain container with a first frame and
   three data frames.  The headers of these frames is the same as before
   but the frames now have trailers specifying the PayloadDigest and
   ChainDigest values:

   Frame 0

   {
     "Index":0,
     "ContainerType":"Chain",
     "ContentMeta":{},
     "DataEncoding":"JSON"}

   [Empty trailer]

                                 Figure 4

   Frame 1

   {
     "Index":1}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9
     tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}

                                 Figure 5

   Frame 2

   {
     "Index":2}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9
     tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}

                                 Figure 6

   Frame 3





Hallam-Baker            Expires February 28, 2019              [Page 16]


Internet-Draft             DARE Data Container               August 2018


   {
     "Index":3}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9
     tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}

                                 Figure 7

9.3.  Merkle Tree

   The following example shows a chain container with a first frame and
   six data frames.  The trailers now contain the TreePosition and
   TreeDigest values:

   Frame 0

   {
     "Index":0,
     "ContainerType":"Merkle",
     "ContentMeta":{},
     "DataEncoding":"JSON"}

   [Empty trailer]

                                 Figure 8

   Frame 1

   {
     "Index":1,
     "TreePosition":0}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t
     n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}

                                 Figure 9

   Frame 2







Hallam-Baker            Expires February 28, 2019              [Page 17]


Internet-Draft             DARE Data Container               August 2018


   {
     "Index":2,
     "TreePosition":319}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "TreeDigest":"7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokkFnZ
     QcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"}

                                 Figure 10

   Frame 3

   {
     "Index":3,
     "TreePosition":319}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t
     n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}

                                 Figure 11

   Frame 4

   {
     "Index":4,
     "TreePosition":1451}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "TreeDigest":"vJ6ngNATvZcXSMALi5IUqzl1GBxBnTNVcC87VL_BhMRCbAvKSj8
     gs0VFgxxLkZ2myrtaDIwhHoswiTiBMLNWug"}

                                 Figure 12

   Frame 5










Hallam-Baker            Expires February 28, 2019              [Page 18]


Internet-Draft             DARE Data Container               August 2018


   {
     "Index":5,
     "TreePosition":1451}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t
     n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}

                                 Figure 13

   Frame 6

   {
     "Index":6,
     "TreePosition":2586}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "TreeDigest":"WgHlz3EHczVPqgtpc39Arv7CFIsCbFVsk8wg0j2qLlEfur9SZ0m
     dr65Ka-HF0Qx8gg_DAoiJwUrwADDXyxVJOg"}

                                 Figure 14

9.4.  Signed container

   The following example shows a tree container with a signature in the
   final record.  The signing key parameters are:





















Hallam-Baker            Expires February 28, 2019              [Page 19]


Internet-Draft             DARE Data Container               August 2018


   {
     "PrivateKeyRSA":{
       "kid":"MBWNO-2J43U-ESWKW-XQWL6-6YGEW-UOPWU-A",
       "n":"1NzbmakMalVH1mRv7TEDEhwXDNojn5wZbq1tv1gp5PgZwzX-klYXuFhj0-
     MpO0zcwptsUaYJhwdvvgW_1udUpISQYluXOB3UMj2e_0yl64MvnqTL47SZQuAN3QQ
     9cuCw_-_Eyj_jerspauqa6RpNzGcabZrtRl7J7DPVZ3SNlw-H_Wxd4HkrFVW_Yqup
     htNL1JciQJYm2DSu9dbetqPZ80x6IBargY850mBYOzvNNE5S-dRJQHoJY4SG-ESYF
     BuAHtBlOMgbI0XNiq96EegA-vPW9XRF-SHdX5mkafefDGK4rT_RoE4gRwhDM3jbZ8
     1-F2ZA_GpNVEvB-25_vF96lQ",
       "e":"AQAB",
       "d":"k_v_h7Jo-TvUt44X6jSax-pTdBHrljk1zSYxGEe4yIBbmMVe-Gl2ECkTLe
     nNbnafO4RGJ_Vgxkk7PEZO-p7Uz5OBtX-rf83tCgihEyg8aaFIZ-h1_xY9Pqr5uGA
     MQGNJaoVMsLb99QNNZhE4JTquP56mVvDQaI3Zn6bhhA0ZqpxS_x6iRUV5KnHCRd47
     DKGHcAsQR_caxGec7M_XNPqpl0tcoGQz46-I0SVVcqtjb_YysEVez4eJhb_ZTU4C7
     pz4wXDj6B0ppFJVZEMaIhKo8FCnoQdXEJl4vSiBzUFrSlPc6gjQk2CBhxc_kb782w
     VwW5wtgOVxhV1k2piQ2NrlOQ",
       "p":"6osXfrJLiPfK0ky17vqzRs-M5mOxZU2LEFGyHTXxx6EYTWixEsx2Sdf6kx
     UD5K_QdYnqqd3yobbGdDpMwwEuwgogWVCz90nc9QdCUy4MCpz8lpOQd1i_tXMmtOg
     GBu9mapMTEOwc7HlLlSDRezv_TzTAH4izv-CUEZ_M5EcwyFM",
       "q":"6FYD6NV4rKU1ACtGQyIvwGrkrS_F6phB-whx0nSVFkbkwppJiPnC6XqjZ3
     OYPCZylxaTHFnxCs0nntrraeNEcWfNPrpTN5XIZjbOiIaKA2iCQkWlDoi8oduEtTK
     oIcuy32oBz6MpUeCWzl0ZQ4EeTF3RyCc_jpf8oRvZ0e3ItHc",
       "dp":"hfjbe9BmWx-HqCaPSanEW-9UQYmym_X2OGUiA5N7vxci5ZymgOFvs_B9v
     iQj7C4NOgaEl3EjFgJsS5m9nSoAxm-4WKxDkD6NyxzRYugLkshnc69otvNn1kKnWn
     CqeK2o57mJC4KDZwRGCzIK1oTH6jtsfta8Lh8fFQ4doEuV7uc",
       "dq":"r6R_ViE0Foja1aLhflU09mmZMViBbkXm86nBqtHZ97pmrLvJRdVTxgCh0
     c6w0yBZ1uEJHBDeykSoZE6qVCWtE3Le1kI0MTx6ANQENXBInCUA_Kr8Ck3TFSYIYJ
     fIRaxiMMZKUjfOQAji2WXGeKL_TcpLkt4hDWLXaNDOTgdOiSc",
       "qi":"DfHtLB1Ox1Kgp3E4jqy5Qxeb7-v7_uv8n_5-E1OQ3NLSRV2m_auojkR19
     nY3gokHKNSXM41qKlJLU00lROjOO2KUq57s8GZkheVfbJLNCJ6KAw_aRT2IgyJm2b
     e2v5OCHSkm88tgJWbtKj-OPKTFV5gOMVdeCzGX286ErjDHGCM"}}

                                 Figure 15

   The container headers and trailers are:

   Frame 0

   {
     "Index":0,
     "ContainerType":"Merkle",
     "ContentMeta":{},
     "DataEncoding":"JSON"}

   [Empty trailer]

                                 Figure 16




Hallam-Baker            Expires February 28, 2019              [Page 20]


Internet-Draft             DARE Data Container               August 2018


   Frame 1

   {
     "Index":1,
     "TreePosition":0}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRVz9t
     n_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}

                                 Figure 17

   Frame 2

   {
     "dig":"S512",
     "Index":2,
     "TreePosition":319}

   {
     "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZDlZe
     aWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw",
     "TreeDigest":"7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokkFnZ
     QcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"}

                                 Figure 18

9.5.  Encrypted container

   The following example shows a container in which all the frame
   payloads are encrypted under the same master secret established in a
   key agreement specified in the first frame.

   Frame 0















Hallam-Baker            Expires February 28, 2019              [Page 21]


Internet-Draft             DARE Data Container               August 2018


   {
     "enc":"A256CBC",
     "Salt":"Th7V_b2--KdV5rETQZbHsw",
     "recipients":[{
         "kid":"MDAD3-E4BYE-MK6CH-QA2HD-TKRS2-KIX5Y-A",
         "epk":{
           "PublicKeyDH":{
             "kid":"MANS7-VSJ2X-75NSK-4SEOM-FRITG-6Q7EO-A",
             "Domain":"YE6bnq1MlX5ojaJto6PLP_PEwA",
             "Public":"MYw__h5Ii4honclOGFKY42KdvJed5T47CZI3QTRSBXUPiOD
     0D0W49YIyf-m30K-BvMWVzNsh9qAQFpK53A5UJO_pQmfbmrZLHq3AjdYoRlc44YOn
     usUokZN43xtGly4m7Rcf_42M3cofa80UWejL3mz3zQKTojxwXz3cykLA2tsTjHdKE
     O0Q6jWef952YkPQNKJmR-42aX7Pfz5kZt1QtgfUV4QYk_b0NFfHSNk9CAs6Klav5K
     qGjxL4mfIxvmrksyIR5solCo2BMbh_AOdyZNuEa4nURvS9YtWrXZRjQtMZPKB8MXG
     _mSAxUco3U1em2hmdkgIkTveF7-da6otILw"}},
         "wmk":"0HfYk15gAvxngunLoHymHSunrb0KlGjGuTERlpSxB5JLpFJj6KJZXA"}
       ],
     "Index":0,
     "ContainerType":"List",
     "ContentMeta":{},
     "DataEncoding":"JSON"}

   [Empty trailer]

                                 Figure 19

   Frame 1

   {
     "enc":"A256CBC",
     "Salt":"BmY8ZYeD8Kg1snIrBIAUyA",
     "Index":1}

   [Empty trailer]

                                 Figure 20

   Frame 2

   {
     "enc":"A256CBC",
     "Salt":"pA-kigxfhhsGC5lM9xrAZQ",
     "Index":2}

   [Empty trailer]

                                 Figure 21




Hallam-Baker            Expires February 28, 2019              [Page 22]


Internet-Draft             DARE Data Container               August 2018


   Here are the container bytes.  Note that the content is now encrypted
   and has expanded by 25 bytes.  These are the salt (16 bytes), the AES
   padding (4 bytes) and the JSON-B framing (5 bytes).

   f5 03 3c
   f1 03 27
   f0 10
   3c 03 f5
   f5 01 7c
   f0 47
   f1 01 30
   7c 01 f5
   f5 01 7c
   f0 47
   f1 01 30
   7c 01 f5


                                 Figure 22

   The following example shows a container in which all the frame
   payloads are encrypted under separate key agreements specified in the
   payload frames.

   Frame 0

   {
     "Index":0,
     "ContainerType":"List",
     "ContentMeta":{},
     "DataEncoding":"JSON"}

   [Empty trailer]

                                 Figure 23

   Frame 1














Hallam-Baker            Expires February 28, 2019              [Page 23]


Internet-Draft             DARE Data Container               August 2018


   {
     "enc":"A256CBC",
     "Salt":"LUAkC3V-ztXJfF7OS-RNwQ",
     "recipients":[{
         "kid":"MDAD3-E4BYE-MK6CH-QA2HD-TKRS2-KIX5Y-A",
         "epk":{
           "PublicKeyDH":{
             "kid":"MCYIZ-PH7YZ-XAY6Q-2EMQR-GMRXG-66YPK-A",
             "Domain":"YE6bnq1MlX5ojaJto6PLP_PEwA",
             "Public":"4cDrUAlsZpyGZGhgVbLlunEyl7O9a8ku-6IGVZaSqyOmsb5
     rCt_pjnN49fjy7hqsS364S4eUDRj7e3vfIBEav-zeynaiJB6lY6eJBMmlaEuomyyy
     01Srq5eaX6i0A8CRWuuRK8IJAHQ31Q89Bah5XayMxc3bqPxciOL1gqU75w65fKmY9
     4mRdAIw2Yk-PTcQVztVDk2ooGJuvLENuIk29g-leFbMmCAXND9hhDErVupBND3Ljh
     BXylTvfIx-mrw0SMPO_IsVLbhrXQ6lTMygucttT3GPf25lgWBaEdiHVuPqmTBtagB
     Vgq9-gHySYtuCNrK7RfB5rtz6CUmENEwx1QA"}},
         "wmk":"25yqAeqIG1S6BrCSQxU_M8WKB4t8J_lHWI-68DNYORTUoCpaitosig"}
       ],
     "Index":1}

   [Empty trailer]

                                 Figure 24

   Frame 2

   {
     "enc":"A256CBC",
     "Salt":"EVrEz8ai_v9s9hZ0ZXMB1Q",
     "recipients":[{
         "kid":"MDAD3-E4BYE-MK6CH-QA2HD-TKRS2-KIX5Y-A",
         "epk":{
           "PublicKeyDH":{
             "kid":"MB43P-C7BVX-SROQ6-Z5LW5-5DLRO-PS675-A",
             "Domain":"YE6bnq1MlX5ojaJto6PLP_PEwA",
             "Public":"mJRyK45fAf2ksxW7M8W961hCKI1g7dQqn9rA2K6vdSh1o4v
     gLqjHDzLYqr9yQKteGUfl0PLIuHAOFq3p_oJDhaioJKX6Akv-y-dcfDdi2WgOOr_k
     R9NPtl-nmsbJ8fx6v1QdXLzdoGZjsj_t6wyIo5w287p7DylFdYkhriqVv7tgBELZI
     53Pur4F5QbK-qJYeSwY3fAksQYrviMZbFVwic8v5QQNoAGcoGguBOOz-aljFuo1g9
     0o7pNT-Soj3DAPfmXS1hmPR-hpqyzELAat-q_O22P4d_NtUOmaLwZXSZVYUBLE8Fj
     1okt4_8zGfCsGpsHl9rwqlyoVIZ4VadldWw"}},
         "wmk":"P6UjESsvq5bATPHc5vPfEcs5u1pCkLVBCv5_aYdHdwQG3rpPihHj2g"}
       ],
     "Index":2}

   [Empty trailer]

                                 Figure 25




Hallam-Baker            Expires February 28, 2019              [Page 24]


Internet-Draft             DARE Data Container               August 2018


10.  Appendix B

   public long PreviousFrame (long Frame) {
       long x2 = Frame + 1;
       long d = 1;

       while (x2 > 0) {
           if ((x2 & 1) == 1) {
               return x2 == 1 ? (d / 2) - 1 : Frame - d;
               }
           d = d * 2;
           x2 = x2 / 2;
           }
       return 0;
       }

                                 Figure 26

11.  References

11.1.  Normative References

   [draft-hallambaker-dare-message]
              Hallam-Baker, P., "Data At Rest Encryption Part 1: DARE
              Message", draft-hallambaker-dare-message-01 (work in
              progress), July 2018.

   [draft-hallambaker-jsonbcd]
              Hallam-Baker, P., "Binary Encodings for JavaScript Object
              Notation: JSON-B, JSON-C, JSON-D", draft-hallambaker-
              jsonbcd-13 (work in progress), July 2018.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997.

   [RFC7159]  Bray, T., "The JavaScript Object Notation (JSON) Data
              Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March
              2014.

   [RFC7515]  Jones, M., Bradley, J., and N. Sakimura, "JSON Web
              Signature (JWS)", RFC 7515, DOI 10.17487/RFC7515, May
              2015.

   [RFC7516]  Jones, M. and J. Hildebrand, "JSON Web Encryption (JWE)",
              RFC 7516, DOI 10.17487/RFC7516, May 2015.





Hallam-Baker            Expires February 28, 2019              [Page 25]


Internet-Draft             DARE Data Container               August 2018


11.2.  Informative References

   [BLOCKCHAIN]
              Chain.com, "Blockchain Specification".

   [draft-hallambaker-mesh-architecture]
              Hallam-Baker, P., "Mathematical Mesh: Architecture",
              draft-hallambaker-mesh-architecture-04 (work in progress),
              September 2017.

   [RFC5652]  Housley, R., "Cryptographic Message Syntax (CMS)", STD 70,
              RFC 5652, DOI 10.17487/RFC5652, September 2009.

   [ZIPFILE]  PKWARE Inc, "APPNOTE.TXT - .ZIP File Format
              Specification", October 2014.

11.3.  URIs

   [1] http://mathmesh.com/Documents/draft-hallambaker-dare-
       container.html

   [2] http://mathmesh.com/Documents/draft-hallambaker-dare-
       container.html

   [3] http://mathmesh.com/Documents/draft-hallambaker-dare-
       container.html

   [4] http://mathmesh.com/Documents/draft-hallambaker-dare-
       container.html

Author's Address

   Phillip Hallam-Baker
   Comodo Group Inc.

   Email: philliph@comodo.com















Hallam-Baker            Expires February 28, 2019              [Page 26]


Html markup produced by rfcmarkup 1.128b, available from https://tools.ietf.org/tools/rfcmarkup/