Derived Data Provider refactoring and documentation addition

docs/architecture/components/ddp.md

@@ -0,0 +1,30 @@
+# Derived Data Providers
+
+Derived Data Providers (DDPs) play a pivotal role in the OCEN 4.0 ecosystem by supplying additional data that significantly enhances the underwriting process. In an era where data is king, the OCEN protocol leverages not just the financial data digitally available through Account Aggregators (like banking data, GST data, NPS, equity etc.), but also additional auxiliary data coming from DDPs.
+
+Entities from diverse fields can serve as DDPs, providing valuable contextual data that may offer insights into a borrower's financial health and reliability. For instance, Swiggy or Zomato could participate as DDPs in the OCEN system, offering earnings data from their platforms which can assist lenders in underwriting loans to restaurants.
+
+In another scenario, consider Dairy Farmer Financing - dairies to which farmers sell their produce could register as DDPs. These dairies could then provide historic sales data, offering lenders a better picture of a farmer's consistent income, thus helping in the underwriting process.
+
+This data, originating from diverse sources, needs to be organized and systematically presented, thus calling for a generic and versatile API that could be implemented by any data source wishing to participate as a DDP. 
+
+## Design Principles
+
+DDPs should be able to integrate seamlessly with the OCEN ecosystem and should be adaptable to various data structures. To cater to this the APIs are designed keeping in mind the following principles:
+
+1. **Versioning**: To accommodate evolving data structures, APIs are designed to support versioning.
+2. **Transparency**: To facilitate accurate and efficient data exchange and as each DDP can expose different schema of data the APIs expose methods to understand the expected request and response data structures.
+
+## API Overview
+
+The following APIs are designed to be implemented by a DDP:
+
+1. **GET /ddp/versions**: This API provides a list of version IDs that the DDP supports, along with their descriptions. This ensures that the borrower's agent or lender is aware of the different versions of data schemas the DDP can provide.
+
+2. **GET /ddp/{version}/dataRequestSchema**: This API presents a JSON schema detailing the structure of the DataRequest object. This ensures that the data request sent by the borrower's agent or lender to the DDP is correctly structured.
+
+3. **GET /ddp/{version}/dataSchema**: This API returns a JSON schema describing the structure of the data provided by the DDP. This ensures that the borrower's agent or lender knows what to expect in the data response.
+
+4. **POST /ddp/{version}/fetchData**: This API accepts a DataRequest object and returns the data associated with the request. It enables the actual data exchange between the DDP and the requesting participant.
+
+By providing these APIs, DDPs will facilitate a structured, clear, and effective way to share valuable data in the OCEN ecosystem, aiming to improve the efficiency of underwriting processes and fostering data-driven decision-making.

static/dist.yaml

@@ -2639,105 +2639,93 @@ paths:
             application/json:
               schema:
                 $ref: '#/components/schemas/Ack'
-  /derivedDataProvider/schema:
-    post:
+  /ddp/versions:
+    get:
       tags:
-        - Data Provider
-      summary: Register schema
-      operationId: registerDPSchema
-      parameters:
-        - $ref: '#/components/parameters/Signature'
-      requestBody:
-        description: Schema registration endpoint for Dervied Data Provider
-        content:
-          application/json:
-            schema:
-              type: object
-              $ref: '#/components/schemas/DPRegisterSchemaRequest'
+        - Data Provider    
+      summary: Get a list of supported versions by the DDP
+      operationId: getDDPVersions      
       responses:
         '200':
-          description: lookup result
+          description: Successful operation
           content:
             application/json:
               schema:
-                $ref: '#/components/schemas/DPSchemaAddDeleteResponse'
-        '400':
-          description: Invalid request
-        '404':
-          description: Not found
-  /derivedDataProvider/schema/byDataProvider/{id}:
+                type: array
+                items:
+                  $ref: '#/components/schemas/DerivedDataVersion'
+  /ddp/{version}/dataRequestSchema:
     get:
       tags:
-        - Data Provider
-      summary: Get schemas supported by derived data provider id
-      operationId: getDPSchemas
+        - Data Provider        
+      summary: Get the Schema for the Data Request object of a specific version
+      operationId: getDataRequestSchema      
       parameters:
-        - $ref: '#/components/parameters/Signature'
-        - $ref: '#/components/parameters/DataProviderId'
+        - in: path
+          name: version
+          required: true
+          description: DDP data version
+          schema:
+            type: string
       responses:
         '200':
-          description: schemas supported by data provider
+          description: Json Schema object describing the structure of the Data Request object
           content:
             application/json:
               schema:
                 type: object
-                properties:
-                  dataProviderId:
-                    type: string
-                    format: uuid
-                    example: 5ebd671b-762c-49b0-99ce-65642b113d7b
-                  schemas:
-                    type: array
-                    items:
-                      $ref: '#/components/schemas/RegisteredDPSchema'
-        '400':
-          description: Invalid request
-        '404':
-          description: Not found
-  /derivedDataProvider/schema/{schemaId}:
+  /ddp/{version}/dataSchema:
     get:
       tags:
-        - Data Provider
-      summary: Get schema by schema id
-      operationId: getSchema
+        - Data Provider       
+      summary: Get the data schema for a specific version
+      operationId: getDataSchema      
       parameters:
-        - $ref: '#/components/parameters/Signature'
-        - $ref: '#/components/parameters/SchemaId'
+        - in: path
+          name: version
+          required: true
+          description: DDP data version
+          schema:
+            type: string
       responses:
         '200':
-          description: schema
+          description: Json Schema object describing the structure of the data that can be returned by fetchData API
           content:
             application/json:
               schema:
-                type: object
-                properties:
-                  schemas:
-                    type: array
-                    items:
-                      $ref: '#/components/schemas/RegisteredDPSchema'
-        '400':
-          description: Bad Request
-        '404':
-          description: Not Found
-    delete:
+                type: object 
+  /ddp/{version}/fetchData:
+    post:
       tags:
-        - Data Provider
-      summary: Delete schema by schema id
-      operationId: deleteSchema
+        - Data Provider     
+      summary: Fetch data from the DDP for a specific version
+      operationId: fetchDDPData        
       parameters:
-        - $ref: '#/components/parameters/Signature'
-        - $ref: '#/components/parameters/SchemaId'
+        - in: path
+          name: version
+          required: true
+          description: DDP data version
+          schema:
+            type: string
+      requestBody:
+        description: Data Request object
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
       responses:
         '200':
-          description: ''
+          description: Successful operation
           content:
             application/json:
               schema:
-                $ref: '#/components/schemas/DPSchemaAddDeleteResponse'
+                type: object
         '400':
-          description: Bad Request
+          description: Invalid request
         '404':
-          description: Not Found
+          description: Not found                
+
   /networks:
     post:
       operationId: createNetwork
@@ -5577,14 +5565,16 @@ components:
         extensibleData:
           desciption: Any extra metadata to be attached to the event
           $ref: '#/components/schemas/ExtensibleData'
-    DPRegisterSchemaRequest:
+    DerivedDataVersion:
       type: object
       properties:
         version:
           type: string
           description: follow semantic version, ref https://semver.org/ | <table> <thead> <tr> <td>Status</th> <td>Stage</th> <td>Rule</th> <td>ExampleVersion</th> </tr> </thead> <tbody> <tr> <td>First release</th> <td>New product</th> <td>Start with 1.0.0</th> <td>1.0.0</th> </tr> <tr> <td>-</th> </tr> <tr> <td>Backward compatible bug fixes</th> <td>Patch release</th> <td>Increment the third digit</th> <td>1.0.1</th> </tr> <tr> <td>-</th> </tr> <tr> <td>Backward compatible new features</th> <td>Minor release</th> <td>Increment the middle digit and reset last digit to zero</th> <td>1.1.0</th> </tr> <tr> <td>-</th> </tr> <tr> <td>Changes that break backward compatibility</th> <td>Major release</th> <td>Increment the first digit and reset middle and last digits to zero</th> <td>2.0.0</th> </tr> <tr> <td>-</th> </tr> </tbody> </table>
         name:
           type: string
+        description:
+          type: string
         jsonSchema:
           type: string
           format: json
@@ -5599,32 +5589,7 @@ components:
           type: string
           format: date-time
           example: '2021-12-06T11:39:57.153Z'
-    DPSchemaId:
-      type: object
-      properties:
-        schemaId:
-          type: string
-          format: uuid
-    DPSchemaAddDeleteResponse:
-      type: object
-      allOf:
-        - $ref: '#/components/schemas/Timestamp'
-        - $ref: '#/components/schemas/DPSchemaId'
-      properties:
-        message:
-          type: string
-          example: added successfully
-    RegisteredDPSchema:
-      type: object
-      allOf:
-        - $ref: '#/components/schemas/DPSchemaId'
-        - $ref: '#/components/schemas/DPRegisterSchemaRequest'
-      properties:
-        createdOn:
-          type: string
-          format: date-time
-          description: Creation timestamp of the message which will be updated at each leg
-          example: '2018-12-06T11:39:57.153Z'
+
     ProductPartner:
       type: object
       required: