Skip when a test doesn't match the value or retrieve none

[j-suarezp]

No description provided.

[j-suarezp]

Method get_tag_value_in_given_dict_by_path of BasePageObject is not working as expected. When we evaluate the path of a yaml variable in the dict parsed from a get config response and the result of the evaluation is a list of elements (like more than one subinterface or protocols) the type of the result of the evaluation is a list. The software doesn't handle properly the list and threats it as a dict causing errors that doesn't seem to show. For making the tests run as expected we have to be sure to not get a list from the evaluation of the yaml variable path in the parsed response dict. We achieve this by deleting manually the items of a list in the router (Like deleting all the subinterfaces of the interface we want to test)

[sbarguil]

Date: 02/15/2020
Username: Samier Barguil
Issue: Handle exception when path does not match the value.

The Method get_tag_value_in_given_dict_by_path of BasePageObject is not working as expected.

The method needs to handle the following behaviors:

Description	Supported?
The response is an list with one element and the path match exactly with the defined in the base_page_objects.	Yes ✅
The response is an list with more than one element and the path match exactly with the defined in the base_page_objects.	No ❌
The response is an list with one or more than one element and the path do not match exactly with the defined in the base_page_objects.	No ❌
The response is an empty list.	No ❌

Examples:

1. One interface on the response

• page_object path: interfaces/interface/name

<rpc-reply message-id="urn:uuid:7aadc4ff-bf0b-4000-baf4-643bba2b994a" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
 <data>
  <interfaces xmlns="http://openconfig.net/yang/interfaces">
   <interface>
    <name>Loopback0</name>
    <config>
     <name>Loopback0</name>
     <type xmlns:idx="urn:ietf:params:xml:ns:yang:iana-if-type">idx:softwareLoopback</type>
     <enabled>true</enabled>
    </config>
    <state>
     <name>Loopback0</name>
     <enabled>true</enabled>
     <admin-status>UP</admin-status>
     <oper-status>UP</oper-status>
     <last-change>6657304</last-change>
     <ifindex>6</ifindex>
     <mtu>1500</mtu>
     <type xmlns:idx="urn:ietf:params:xml:ns:yang:iana-if-type">idx:softwareLoopback</type>
    </state>
    <subinterfaces>
     <subinterface>
      <index>0</index>
      <ipv4 xmlns="http://openconfig.net/yang/interfaces/ip">
       <addresses>
        <address>
         <ip>1.1.1.10</ip>
         <config>
          <ip>1.1.1.10</ip>
          <prefix-length>32</prefix-length>
         </config>
         <state>
          <ip>1.1.1.10</ip>
          <prefix-length>32</prefix-length>
          <origin>STATIC</origin>
         </state>
        </address>
       </addresses>
      </ipv4>
     </subinterface>
    </subinterfaces>
   </interface>
   <interface>
  </interfaces>
 </data>
</rpc-reply>

Once parced this XML the dictionary just have one entry after data. In this case the name can be accessed directed from the parsing.

2. Two interfaces on the response

• page_object path: interfaces/interface/name

<rpc-reply message-id="urn:uuid:7aadc4ff-bf0b-4000-baf4-643bba2b994a" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
 <data>
  <interfaces xmlns="http://openconfig.net/yang/interfaces">
   <interface>
    <name>Loopback0</name>
    <config>
     <name>Loopback0</name>
     <type xmlns:idx="urn:ietf:params:xml:ns:yang:iana-if-type">idx:softwareLoopback</type>
     <enabled>true</enabled>
    </config>
    <state>
     <name>Loopback0</name>
     <enabled>true</enabled>
     <admin-status>UP</admin-status>
     <oper-status>UP</oper-status>
     <last-change>6657304</last-change>
     <ifindex>6</ifindex>
     <mtu>1500</mtu>
     <type xmlns:idx="urn:ietf:params:xml:ns:yang:iana-if-type">idx:softwareLoopback</type>
    </state>
   </interface>
   <interface>
    <name>MgmtEth0/RP0/CPU0/0</name>
    <config>
     <name>MgmtEth0/RP0/CPU0/0</name>
     <type xmlns:idx="urn:ietf:params:xml:ns:yang:iana-if-type">idx:ethernetCsmacd</type>
     <enabled>true</enabled>
    </config>
    <state>
     <name>MgmtEth0/RP0/CPU0/0</name>
     <enabled>true</enabled>
     <admin-status>UP</admin-status>
     <oper-status>UP</oper-status>
     <last-change>6657211</last-change>
     <counters>
      <in-unicast-pkts>2358263</in-unicast-pkts>
      <in-octets>169061934</in-octets>
      <out-unicast-pkts>8628115</out-unicast-pkts>
      <out-octets>559632285</out-octets>
      <in-multicast-pkts>0</in-multicast-pkts>
      <in-broadcast-pkts>187156</in-broadcast-pkts>
      <out-multicast-pkts>0</out-multicast-pkts>
      <out-broadcast-pkts>0</out-broadcast-pkts>
      <out-discards>0</out-discards>
      <in-discards>0</in-discards>
      <in-unknown-protos>0</in-unknown-protos>
      <in-errors>0</in-errors>
      <out-errors>0</out-errors>
      <last-clear>2020-11-30T12:44:52.608+00:00</last-clear>
     </counters>
     <ifindex>1</ifindex>
     <mtu>1514</mtu>
     <type xmlns:idx="urn:ietf:params:xml:ns:yang:iana-if-type">idx:ethernetCsmacd</type>
    </state>
    <ethernet xmlns="http://openconfig.net/yang/interfaces/ethernet">
     <config>
      <auto-negotiate>false</auto-negotiate>
     </config>
     <state>
      <auto-negotiate>false</auto-negotiate>
      <counters>
       <in-8021q-frames>0</in-8021q-frames>
       <in-mac-pause-frames>0</in-mac-pause-frames>
       <in-oversize-frames>0</in-oversize-frames>
       <in-jabber-frames>0</in-jabber-frames>
       <in-fragment-frames>0</in-fragment-frames>
       <in-crc-errors>0</in-crc-errors>
       <out-8021q-frames>0</out-8021q-frames>
       <out-mac-pause-frames>0</out-mac-pause-frames>
      </counters>
     </state>
    </ethernet>
   </interface>
  </interfaces>
 </data>
</rpc-reply>

Once parced this XML the dictionary has two entries after data. In this case the name must be matched in both entries to validate if the configuration was correctly done.

Expected result:

Handle the 2,3 and 4 possible outcomes.

[josemtnzjmnz]

Hi @sbarguil. I have almost solved this issue but I don't understand this cause:

The response is an list with one or more than one element and the path do not match exactly with the defined in the base_page_objects.

I would like to test the code for different inputs. Could you provide me a set of inputs or do you prefer to try the code on you own?

[josemtnzjmnz]

@sbarguil, @oscargdd or @vlopezalvarez. Who wants to try and test my code? I tried it using the inputs contained in the uploaded file (commit 71d404a) but it is preferable that someone else tests the code with a wide variety of inputs.

Regards, Jose.

[josemtnzjmnz]

I have tried different inputs with successfully results, here the last commit cede6a2