tap-twilio failing on usage_records stream

[lmmuniz]

Hi,

I have cloned the repo: https://github.com/singer-io/tap-twilio
And installed into my venv (Anaconda Python 3.6.5)

I am able to set the config.json to my Twilio account (passing account_sid, auth_token along with start_date and user_agent)

I can run the discovery mode and create the catalog.json:

(venv) (base) C:\Python\environments\tap-twilio\venv>tap-twilio --config config.json --discover > catalog.json
INFO Starting discover
INFO Finished discover
(venv) (base) C:\Python\environments\tap-twilio\venv>

Then I edit the catalog.json to add the attribute "selected": true to usage_records stream:

"stream": "usage_records",
      "metadata": [
        {
          "breadcrumb": [],
          "metadata": {
            "table-key-properties": [
              "account_sid",
              "category",
              "start_date"
            ],
            "forced-replication-method": "INCREMENTAL",
            "valid-replication-keys": [
              "end_date"
            ],
            "inclusion": "available",
			"selected": true
          }

But, when I run the singer-check-tap option in my tap-twilio, it show the following fail autput:

(venv) (base) C:\Python\environments\tap-twilio\venv>tap-twilio --config config.json --catalog catalog.json | singer-check-tap > state.json.tmp

(venv) (base) C:\Python\environments\tap-twilio\venv>tap-twilio --config config.json --catalog catalog.json | singer-check-tap > state.json.tmp
INFO last/currently syncing stream: None
INFO Skipping stream: accounts
INFO Skipping stream: addresses
INFO Skipping stream: dependent_phone_numbers
INFO Skipping stream: applications
INFO Skipping stream: available_phone_number_countries
INFO Skipping stream: available_phone_numbers_local
INFO Skipping stream: available_phone_numbers_mobile
INFO Skipping stream: available_phone_numbers_toll_free
INFO Skipping stream: incoming_phone_numbers
INFO Skipping stream: keys
INFO Skipping stream: calls
INFO Skipping stream: conferences
INFO Skipping stream: conference_participants
INFO Skipping stream: outgoing_caller_ids
INFO Skipping stream: recordings
INFO Skipping stream: transcriptions
INFO Skipping stream: queues
INFO Skipping stream: messages
INFO Skipping stream: message_media
INFO Skipping stream: usage_triggers
INFO Skipping stream: alerts
INFO selected_streams: ['usage_records']
INFO Sync Parent Streams: ['accounts']
INFO Sync Child Streams: ['usage_records']
INFO START Syncing: accounts
INFO stream: accounts, bookmark_field: date_updated, last_datetime: 2019-01-01T00:00:00Z
INFO START Sync for Stream: accounts
INFO URL for Stream accounts: https://api.twilio.com/2010-04-01/Accounts.json
INFO METRIC: {"type": "timer", "metric": "http_request_duration", "value": 0.21579551696777344, "tags": {"endpoint": "accounts", "http_status_code": 200, "status": "succeeded"}}
INFO START Syncing: usage_records
INFO START Sync for Stream: usage_records, parent_stream: accounts, parent_id: {AccountSID}
INFO No child stream usage_records for parent stream accounts in subresource uris
INFO FINISHED Sync for Stream: usage_records, parent_id: {AccountSID}, total_records: 0
INFO START Sync for Stream: usage_records, parent_stream: accounts, parent_id: {AccountSID}
INFO No child stream usage_records for parent stream accounts in subresource uris
INFO FINISHED Sync for Stream: usage_records, parent_id: {AccountSID}, total_records: 0
INFO START Sync for Stream: usage_records, parent_stream: accounts, parent_id: {AccountSID}
INFO No child stream usage_records for parent stream accounts in subresource uris
INFO FINISHED Sync for Stream: usage_records, parent_id: {AccountSID}, total_records: 0
INFO START Sync for Stream: usage_records, parent_stream: accounts, parent_id: {AccountSID}
INFO No child stream usage_records for parent stream accounts in subresource uris
INFO FINISHED Sync for Stream: usage_records, parent_id: {AccountSID}, total_records: 0
INFO Synced Stream: accounts, page: 1, 0 to 0 of total records: 0
INFO Write state for stream: accounts, value: 2019-01-01T00:00:00Z
INFO FINISHED Syncing: accounts, total_records: 0

(venv) (base) C:\Python\environments\tap-twilio\venv>

My state.json.tmp created shows:

Checking stdin for valid Singer-formatted data
The output is valid.
It contained 5 messages for 2 streams.

  2 schema messages
  0 record messages
  3 state messages

Details by stream:
+---------------+---------+---------+
| stream | records | schemas |
+---------------+---------+---------+
| accounts | 0 | 1 |
| usage_records | 0 | 1 |
+---------------+---------+---------+

My question, is why if shows a message that the usage_records is missing from subresource_uris in Accounts:

INFO No child stream usage_records for parent stream accounts in subresource uris

My accounts requests when I check in Postman shows:

"subresource_uris": {
"addresses": "/2010-04-01/Accounts/{AccountSID}/Addresses.json",
"conferences": "/2010-04-01/Accounts/{AccountSID}/Conferences.json",
"signing_keys": "/2010-04-01/Accounts/{AccountSID}/SigningKeys.json",
"transcriptions": "/2010-04-01/Accounts/{AccountSID}/Transcriptions.json",
"connect_apps": "/2010-04-01/Accounts/{AccountSID}/ConnectApps.json",
"sip": "/2010-04-01/Accounts/{AccountSID}/SIP.json",
"authorized_connect_apps": "/2010-04-01/Accounts/{AccountSID}/AuthorizedConnectApps.json",
"usage": "/2010-04-01/Accounts/{AccountSID}/Usage.json",
"keys": "/2010-04-01/Accounts/{AccountSID}/Keys.json",
"applications": "/2010-04-01/{AccountSID}/Applications.json",
"recordings": "/2010-04-01/Accounts/{AccountSID}/Recordings.json",
"short_codes": "/2010-04-01/Accounts/{AccountSID}/SMS/ShortCodes.json",
"calls": "/2010-04-01/Accounts/{AccountSID}/Calls.json",
"notifications": "/2010-04-01/Accounts/{AccountSID}/Notifications.json",
"incoming_phone_numbers": "/2010-04-01/Accounts/{AccountSID}/IncomingPhoneNumbers.json",
"queues": "/2010-04-01/Accounts/{AccountSID}/Queues.json",
"messages": "/2010-04-01/Accounts/{AccountSID}/Messages.json",
"outgoing_caller_ids": "/2010-04-01/Accounts/{AccountSID}/OutgoingCallerIds.json",
"available_phone_numbers": "/2010-04-01/Accounts/{AccountSID}/AvailablePhoneNumbers.json",
"balance": "/2010-04-01/Accounts/{AccountSID}/Balance.json"
}

And when i open usage request in Postman:

{
"subresource_uris": {
"records": "/2010-04-01/Accounts/{AccountSID}/Usage/Records.json",
"triggers": "/2010-04-01/Accounts/{AccountSID}/Usage/Triggers.json"
}
}

What is missing in the tap-twilio repo to make this API request to work?

Any help is kindly thankful.

Leonardo Muniz