Merge pull request #29 from gowthamsiddarthd/master

Updated the framework with additional test suites

api-tests/README.md

@@ -19,6 +19,16 @@ The test suite for this specification is located in the ff directory of this rep
 ### PSA Developer APIs
 The test suite for this specification is located in the dev_apis directory of this repository. See [PSA Developer APIs Readme](dev_apis/README.md) file for more details.
 
+## Release Update
+ - Release Version - 0.7
+ - Code Quality: Pre-Beta. Please use this opportunity to suggest enhancements and point out errors.
+ - Current release contains following Developer APIs tests:
+ 1. **Crypto tests** that are written for the crypto service APIs defined in the [crypto.h](../api-specs/include/crypto.h)
+ 2. **Protected Storage tests** that are written for the protected storage service APIs defined in the [protected_storage.h](../api-specs/include/protected_storage.h)
+ 3. **Internal Trusted Storage tests** that are written for the internal trusted storage service APIs defined in the [internal_trusted_storage.h](../api-specs/include/internal_trusted_storage.h)
+ - Current release contains following PSA-FF tests:
+ 1. **IPC tests** that are written for version 1.0-Beta-0 of the PSA FF specification.
+
 ## License
 
 Arm PSA test suite is distributed under Apache v2.0 License.
@@ -33,4 +43,4 @@ Arm PSA test suite is distributed under Apache v2.0 License.
 
 --------------
 
-*Copyright (c) 2018, Arm Limited and Contributors. All rights reserved.*
+*Copyright (c) 2018-2019, Arm Limited and Contributors. All rights reserved.*

api-tests/dev_apis/README.md

@@ -13,16 +13,9 @@ The Architecture Test Suite is a set of examples of the invariant behaviours tha
 
 The Architecture Test Suite contains the tests that are self-checking, portable C-based tests with directed stimulus.
 
-The tests are available as open source. The tests and the corresponding abstraction layers are available with an Apache v2.0 license allowing for external contribution.
+The tests are available as open source. The tests and the corresponding abstraction layers are available with an Apache v2.0 license allowing for external contribution. This test suite is not a substitute for design verification. To review the test logs, Arm licensees can contact Arm directly through their partner managers.
 
-For more information on Architecture Test Suite specification, refer the [Validation Methodology](../docs/Arm_PSA_FF_Arch_Test_Validation_Methodology.pdf) document.
-
-## Release Update
- - Release Version - 0.5
- - Code Quality - Alpha. This indicates the suite is in development and it contains tests which have not been validated on any platform. Please use this opportunity to suggest enhancements and point out errors.
- - Current release contains **Crypto tests** that are written for the crypto service APIs defined in the [crypto.h](../../api-specs/include/crypto.h)
- - This test suite is not a substitute for design verification.
- - To review the test logs, Arm licensees can contact Arm directly through their partner managers.
+For more information on Architecture Test Suite specification, refer the [Validation Methodology](../docs/Arm_PSA_APIs_Arch_Test_Validation_Methodology.pdf) document.
 
 ## Tests Scenarios
 
@@ -39,7 +32,7 @@ Please make sure you have all required software installed as explained in the [s
 
 ### Porting steps
 
-Refer to the [Porting Guide](../docs/porting_guide.md) document for porting steps.
+Refer to the [PSA Developer APIs Test Suite Porting Guide](../docs/porting_guide_dev_apis.md) document for porting steps.
 
 ### Build steps
 
@@ -54,15 +47,15 @@ cd api-tests
 -   <platform_name> is the same as the name of the target specific directory created in the platform/targets/ directory.  <br />
 -   <cpu_architecture_version> is the Arm Architecture version name for which test binaries should be compiled. For example, Armv7M, Armv8M-Baseline and Armv8M-Mainline Architecture.  <br />
 -   <suite_name> is the suite name and it is same as the suite name available in test_suites/ directory. <br />
--   <include_path> is an additional directory to be included into compiler search path. **Note** - To run crypto tests, include path must point to path where "psa/crypto.h" is located in your build system.<br />
+-   <include_path> is an additional directory to be included into compiler search path. Note- You must provide Developer APIs header file implementation to Test Suite build system using this option. For example - To compiler Crypto tests, include path must point to path where "psa/crypto.h" is located in your build system.<br />
 -   <build_dir> is an output directory to keep build files.
 
 Refer ./tools/scripts/setup.sh --help to know more about options.
 
-*To compile crypto tests for fvp_mps2_cm4_mbedos platform*
+*To compile crypto tests for tgt_dev_apis_mbedos_fvp_mps2_m4 platform*
 ```
 cd api-tests
-./tools/scripts/setup.sh --target fvp_mps2_cm4_mbedos --cpu_arch armv7m --suite crypto --build BUILD_CRYPTO --include <include_path>
+./tools/scripts/setup.sh --target tgt_dev_apis_mbedos_fvp_mps2_m4 --cpu_arch armv7m --suite crypto --build BUILD_CRYPTO --include <include_path>
 ```
 
 ### Build output
@@ -100,5 +93,5 @@ Arm PSA test suite is distributed under Apache v2.0 License.
 
 --------------
 
-*Copyright (c) 2018, Arm Limited and Contributors. All rights reserved.*
+*Copyright (c) 2018-2019, Arm Limited and Contributors. All rights reserved.*
 

api-tests/dev_apis/crypto/test_c001/test_c001.c

@@ -1,5 +1,5 @@
 /** @file
- * Copyright (c) 2018, Arm Limited or its affiliates. All rights reserved.
+ * Copyright (c) 2018-2019, Arm Limited or its affiliates. All rights reserved.
  * SPDX-License-Identifier : Apache-2.0
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
@@ -15,14 +15,8 @@
  * limitations under the License.
 **/
 
-#ifdef NONSECURE_TEST_BUILD
 #include "val_interfaces.h"
 #include "val_target.h"
-#else
-#include "val/common/val_client_defs.h"
-#include "val/spe/val_partition_common.h"
-#endif
-
 #include "test_c001.h"
 
 client_test_t test_c001_crypto_list[] = {
@@ -36,49 +30,44 @@ client_test_t test_c001_crypto_list[] = {
 int32_t psa_generate_random_without_init_test(security_t caller)
 {
     uint8_t         output[GENERATE_SIZE];
+    int32_t         status;
 
-    val->print(PRINT_TEST, "[Check1] Test calling crypto functions before psa_crypto_init \n", 0);
+    val->print(PRINT_TEST, "[Check 1] Test calling crypto functions before psa_crypto_init\n", 0);
 
     /* Generate random bytes */
-    if (val->crypto_function(VAL_CRYPTO_GENERATE_RANDOM, output, GENERATE_SIZE) == PSA_SUCCESS)
-    {
-        val->print(PRINT_ERROR, "The crypto function should have failed but succeeded\n", 0);
-        return VAL_STATUS_CRYPTO_FAILURE;
-    }
+    status = val->crypto_function(VAL_CRYPTO_GENERATE_RANDOM, output, GENERATE_SIZE);
+    if (status == PSA_SUCCESS)
+        return RESULT_SKIP(VAL_STATUS_INIT_ALREADY_DONE);
+    else
+        TEST_ASSERT_EQUAL(status, PSA_ERROR_BAD_STATE, TEST_CHECKPOINT_NUM(1));
 
     return VAL_STATUS_SUCCESS;
 }
 
 int32_t psa_crypto_init_test(security_t caller)
 {
-    int32_t         status = VAL_STATUS_SUCCESS;
+    int32_t        status;
 
-    val->print(PRINT_TEST, "[Check2] Test psa_crypto_init\n", status);
+    val->print(PRINT_TEST, "[Check 2] Test psa_crypto_init\n", 0);
 
     /* Initialize the PSA crypto library*/
-    if (val->crypto_function(VAL_CRYPTO_INIT) != PSA_SUCCESS)
-    {
-        val->print(PRINT_ERROR, "Crypto init failed\n", 0);
-        status = VAL_STATUS_INIT_FAILED;
-    }
+    status = val->crypto_function(VAL_CRYPTO_INIT);
+    TEST_ASSERT_EQUAL(status, PSA_SUCCESS, TEST_CHECKPOINT_NUM(1));
 
-   return status;
+    return VAL_STATUS_SUCCESS;
 }
 
 int32_t multiple_psa_crypto_init_test(security_t caller)
 {
-    int32_t         i, status = VAL_STATUS_SUCCESS;
+    int32_t         i, status;
 
-    val->print(PRINT_TEST, "[Check3] Test multiple psa_crypto_init \n", 0);
+    val->print(PRINT_TEST, "[Check 3] Test multiple psa_crypto_init \n", 0);
     for (i = 0; i < 5; i++)
     {
         /* Initialize the PSA crypto library*/
-        if (val->crypto_function(VAL_CRYPTO_INIT) != PSA_SUCCESS)
-        {
-            status = VAL_STATUS_INIT_FAILED;
-            return status;
-        }
+        status = val->crypto_function(VAL_CRYPTO_INIT);
+        TEST_ASSERT_EQUAL(status, PSA_SUCCESS, TEST_CHECKPOINT_NUM(1));
     }
 
-   return status;
+    return VAL_STATUS_SUCCESS;
 }

api-tests/dev_apis/crypto/test_c001/test_c001.h

@@ -1,5 +1,5 @@
 /** @file
- * Copyright (c) 2018, Arm Limited or its affiliates. All rights reserved.
+ * Copyright (c) 2018-2019, Arm Limited or its affiliates. All rights reserved.
  * SPDX-License-Identifier : Apache-2.0
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
@@ -18,7 +18,6 @@
 #define _TEST_C001_CLIENT_TESTS_H_
 
 #include "val_crypto.h"
-
 #define test_entry CONCAT(test_entry_,c001)
 #define val CONCAT(val,test_entry)
 #define psa CONCAT(psa,test_entry)