Unable to delete external Data Group File List via automation

[f5killer]

I stuck in one of the automation case. It nightmare since long time. Can help to advice is there any automation way to delete Data Group File List from File Management.

COMPONENT NAME

System ›› File Management : Data Group File List

Environment

F5 v14, v15, v16

ANSIBLE VERSION

BIGIP VERSION

CONFIGURATION

OS / ENVIRONMENT

BIG-IP 16.1.4.1 Build 0

SUMMARY

Can help to advice is there any automation way to delete Data Group File List from File Management.

We have many unused file under System ›› File Management : Data Group File List, wanted to add automate deleation via ansible or python. Please suggest best approach. for reference this is locations.

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

[pgouband]

Hi @f5killer,

Are you using bigip_data_group?
https://clouddocs.f5.com/products/orchestration/ansible/devel/modules/bigip_data_group_module.html#bigip-data-group-module-3

With state absent?

[f5killer]

It works well in internal data group but external doesn't work.. Playbook will pass but still you will find external data group here.
ls -l /config/filestore/files_d/Common_d/data_group_d -- CLI
System ›› File Management : Data Group File List. -- GUI

is there any solutions?

[pgouband]

Hi @f5killer,

Not sure I understand your issue.
Playbook is working and datagroup object is not anymore in the configuration but the file in filestore is still there?

[f5killer]

Playbook works well with external datagroup but ifile never delete. Not sure if i am doing wrong

- name: Delete the external Data group file which is not used.
      bigip_data_group:
        provider: '{{ provider }}'
        name: ip_list_entry
        external_file_name: ip_list_entry
        type: address
      delegate_to: localhost

[pgouband]

Hi @f5killer,

Have you tried with 'state: absent'?

From:
https://clouddocs.f5.com/products/orchestration/ansible/devel/modules/bigip_data_group_module.html#bigip-data-group-module-3

When state is present, ensures the data group exists.
When state is absent, ensures the data group is removed.
The use of state in this module refers to the entire data group, not its members.

So it should look like:

- name: Delete the external Data group file which is not used.
      bigip_data_group:
        provider: '{{ provider }}'
        name: ip_list_entry
        external_file_name: ip_list_entry
        type: address
        state: absent
      delegate_to: localhost

[f5killer]

I feel its bug. Find the below code which i am using along with -vvv

        - name: Delete the external Data group file which is not used.
          bigip_data_group:
            provider: '{{ provider }}'
            name: ip_list_entry
            external_file_name: ip_list_entry
            type: address
            state: absent
          delegate_to: localhost

ok: [x.x.x.x -> localhost] => {
"changed": false,
"invocation": {
"module_args": {
"delete_data_group_file": false,
"description": null,
"external_file_name": "ip_list_entry",
"internal": false,
"name": "ip_list_entry",
"partition": "Common",
"provider": {
"auth_provider": null,
"no_f5_teem": true,
"password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"server": "x.x.x.x",
"server_port": 443,
"timeout": null,
"transport": "rest",
"user": "xxxx",
"validate_certs": false
},
"records": null,
"records_src": null,
"separator": ":=",
"state": "absent",
"type": "address"
}
}
}

Device Output:
[X:X:X:X::Active:Changes Pending] data_group_d # ls -l ip_list_entry
-rw-r--r--. 1 root root 3830 May 7 20:59 :Common:ip_list_entry_80553_1
[X:X:X:X::Active:Changes Pending] data_group_d #

I feel this is bug as F5 GUI file name shows ip_list_entry but cli file name append with prefix and postfix :Common:ip_list_entry_80553_1

Thank you

[pgouband]

Hi @f5killer,

Can you try to delete the datagroup without specifying the filename?

[f5killer]

Still the same result. I feel ansible-python library need to review.. Can you please ask someone to review.. in next version also some module will be used.

[pgouband]

Hi @f5killer,

I was able to delete a datagroup (external) using the following playbook.
So I don't think it's a bug.

- hosts: all
  collections:
    - f5networks.f5_modules
  connection: local
  gather_facts: no

  tasks:
    - name: create DG
      f5networks.f5_modules.bigip_data_group:
        name: test-dg
        type: address
        state: absent
        provider:
          server: 10.10.10.9
          user: "admin"
          password: "mypassword"
          server_port: 443
          validate_certs: no
          no_f5_teem: yes
      delegate_to: localhost

$ ansible-playbook -i hosts dg2.yml

PLAY [all] *******************************************************************************************************************************************************************************

TASK [create DG] *************************************************************************************************************************************************************************
ok: [10.10.10.9 -> localhost]

PLAY RECAP *******************************************************************************************************************************************************************************
10.10.10.9                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

[f5killer]

Its not working for me. May be i am not explaning clearly. To reproduce issue, I would suggets you upload file under
System ›› File Management : Data Group File List. And create irule which will reference external data group

when FLOW_INIT {
    # Data Group List - Type Address
    set dg "test-dg"

   if { [class match -- [IP::remote_addr] equals $dg]} {  
         log local0.debug "[clock format [clock seconds] -format %Y-%m-%dT%H:%M:%S%z] attacker:[IP::remote_addr]
        drop
        }
}

upload multiple file in external System ›› File Management : Data Group File test-dg_1, test-dg_2, test-dg_3 with different ip address. Later change the irule reference datagroup from test-dg to test-dg_1, test-dg_2 and final will be in iRule test-dg_3

Now delete unused datagroup test-dg, test-dg_1, test-dg_2 via ansible f5networks.f5_modules.bigip_data_group, you will find that ansible execute all but actaul datagroup remain there.

Device Output:
[X:X:X:X::Active:Changes Pending] data_group_d # ls -l ip_list_entry
-rw-r--r--. 1 root root 3830 May 7 20:59 :Common:ip_list_entry_80553_1
[X:X:X:X::Active:Changes Pending] data_group_d #

I feel this is bug as F5 GUI file name shows ip_list_entry but cli file name append with prefix and postfix :Common:ip_list_entry_80553_1

if you don't observe issue.. Best of Luck

[pgouband]

Hi @f5killer,

I did all you asked and datagroup are deleted (in the GUI Local Traffic >> iRules : Data Group List).
Like in GUI data group file you can see under System >> File Management >> Data Group File list are still present (not deleted when the Data Group is deleted from the GUI) so Ansible module is acting like GUI.

- hosts: all
  collections:
    - f5networks.f5_modules
  connection: local
  gather_facts: no

  tasks:
    - name: delete DG
      f5networks.f5_modules.bigip_data_group:
        name: test-dg
        type: address
        state: absent
        provider:
          server: 10.10.10.9
          user: "admin"
          password: "mypassword"
          server_port: 443
          validate_certs: no
          no_f5_teem: yes
      delegate_to: localhost

    - name: delete DG_1
      f5networks.f5_modules.bigip_data_group:
        name: test-dg_1
        type: address
        state: absent
        provider:
          server: 10.10.10.9
          user: "admin"
          password: "mypassword"
          server_port: 443
          validate_certs: no
          no_f5_teem: yes
      delegate_to: localhost

    - name: delete DG_2
      f5networks.f5_modules.bigip_data_group:
        name: test-dg_2
        type: address
        state: absent
        provider:
          server: 10.10.10.9
          user: "admin"
          password: "mypassword"
          server_port: 443
          validate_certs: no
          no_f5_teem: yes
      delegate_to: localhost

$ ansible-playbook -i hosts dg3.yml 

PLAY [all] *********************************************************************

TASK [delete DG] ***************************************************************
changed: [10.10.10.9 -> localhost]

TASK [delete DG_1] *************************************************************
changed: [10.10.10.9 -> localhost]

TASK [delete DG_2] *************************************************************
changed: [10.10.10.9 -> localhost]

PLAY RECAP *********************************************************************
10.10.10.9                   : ok=3    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[f5killer]

It mean you are able to delete file from external data group via above script? No or YES
Because i am not able to delete, i feel f5 treat this as file and module require aditional flag

[pgouband]

Hi @f5killer,

There are 2 components of a Data Group (external):

• LTM config object
• system file

In the GUI if you delete LTM config object, system file is not deleted.
Ansible is behaving like GUI.

Using the playbook, I was able to delete LTM config object but not the file but again Ansible module is behaving like GUI.

[f5killer]

Would you like suggest out of box idea or bug/improvement?

[pgouband]

Hi @f5killer,

You can use bigip_command module to delete the file after deleting Data Group LTM object.
If you need more help you should reach out your account team and ask for Professional Services.

- hosts: all
  collections:
    - f5networks.f5_modules
  connection: local
  gather_facts: no

  tasks:
    - name: delete DG
      f5networks.f5_modules.bigip_command:
        commands: delete sys file data-group test-dg
        provider:
          server: 10.10.10.9
          user: "admin"
          password: "mypassword"
          server_port: 443
          validate_certs: no
          no_f5_teem: yes
      delegate_to: localhost

[f5killer]

This is just workaround but i think 'bigip_data_group' module must be reviewed. External files 'external_file_name' parameter has added, which works for upload but never works for remove. Thanks

[pgouband]

Hi @f5killer,

Ansible is behaving like GUI so no the module will not be modified.