[Kendra] S3 connector documentation for templateConfiguration usage

[ssmails]

Describe the issue

Referring to https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kendra/client/create_data_source.html

Amazon Kendra now supports an upgraded Amazon S3 connector.

You must now use the [TemplateConfiguration](https://docs.aws.amazon.com/kendra/latest/APIReference/API_TemplateConfiguration.html) object instead of the S3DataSourceConfiguration object to configure your connector.

Connectors configured using the older console and API architecture will continue to function as configured. However, you won’t be able to edit or update them. If you want to edit or update your connector configuration, you must create a new connector.

We recommended migrating your connector workflow to the upgraded version. Support for connectors configured using the older architecture is scheduled to end by June 2024.

please provide example of using the [TemplateConfiguration], since the orinal method documented using the S3DataSourceConfiguration is scheduled to end.

Links

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kendra/client/create_data_source.html

[tim-finnigan]

Thanks for reaching out. As mentioned in other issues that you opened, the create_data_source command makes a request to the CreateDataSource API, so any improvements to that documentation would need to be provided by the Kendra team.

Boto3-specific examples could also be added here: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/examples.html, or here in the Kendra Developer Guide: https://docs.aws.amazon.com/kendra/latest/dg/gs-python.html

The TemplateConfiguration object is already documented/linked from the command documentation: https://docs.aws.amazon.com/kendra/latest/APIReference/API_TemplateConfiguration.html. Are you having any issues using that parameter or do you have more details regarding the example you would link to see?

[ssmails]

@tim-finnigan Thanks. Would it possible to add an example of using this with boto3 with TemplatConfiguration or is this something thats not supported by boto3 and boto3 only supports the S3DataSourceConfiguration method ?

I see a 'Template' option in the boto3 create_data_source at https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kendra/client/create_data_source.html

Is this related to the S3 TeplateConfiguration ? If so, theres no example on how to use it in the docs for S3.

response = client.create_data_source(
    Name='string',
    IndexId='string',
    Type='S3'|'SHAREPOINT'|'DATABASE'|'SALESFORCE'|'ONEDRIVE'|'SERVICENOW'|'CUSTOM'|'CONFLUENCE'|'GOOGLEDRIVE'|'WEBCRAWLER'|'WORKDOCS'|'FSX'|'SLACK'|'BOX'|'QUIP'|'JIRA'|'GITHUB'|'ALFRESCO'|'TEMPLATE',

[tim-finnigan]

I see TemplateConfiguration in the documented snippet, below AlfrescoConfiguration:

Have you tried setting that parameter? Here is a list of data template schemas: https://docs.aws.amazon.com/kendra/latest/dg/ds-schemas.html, for example S3: https://docs.aws.amazon.com/kendra/latest/dg/ds-schemas.html#ds-s3-schema

[ssmails]

I am requesting an example for the S3 configuration with TemplateConfiguration. (Not AlfrescoConfiguration) @tim-finnigan if you can kindly provide documentation for that. or is this something that we ned to check with Kendra directly and not supported from boto3.

[tim-finnigan]

@ssmails can you try something like this: (I'm referring to TemplateConfiguration as documented, not AlfrescoConfiguration)

import boto3

# Create the Kendra client
kendra_client = boto3.client('kendra')

response = kendra_client.create_data_source(
    Name='string',
    IndexId='string',
    Type='TEMPLATE',
    TemplateConfiguration={
        'Template': {...}
    }
)

[ssmails]

@tim-finnigan do you have a sample of a working Template that can work for this ?
'Template': {...}

I also see the tag added to the issue as 'Feature-request' so wondering If this is something thats currently supported or not for S3 ? kindly confirm.

[tim-finnigan]

Hi, here are the various template schemas: https://docs.aws.amazon.com/kendra/latest/dg/ds-schemas.html. Which one are you trying to do, or can you share any more details on your use case?

[ssmails]

Pls provide a minimal working example of using TemplateConfiguration with S3 (at the least accepting a bucket name).
seems lot of connectors are inadequately documented.

[tim-finnigan]

Here you go, this worked for me:

import boto3

kendra_client = boto3.client('kendra', region_name='us-east-1')

template = {
    "connectionConfiguration":  {
        "repositoryEndpointMetadata": {
            "BucketName": "BUCKET"
        }
    },
    "repositoryConfigurations": {
        "document": {
            "fieldMappings": [
                {
                    "indexFieldName": "MyField",
                    "indexFieldType": "STRING",
                    "dataSourceFieldName": "MyField",
                }
            ]
        }
    },
    "syncMode": "FULL_CRAWL",
    "type": "S3"
}

response = kendra_client.create_data_source(
    Name='test-data-source',
    IndexId='f81421b2-b65b-4edd-b0fe-f53921a4',
    Type="TEMPLATE",
    Configuration={
        'TemplateConfiguration': {
            'Template': template
        }
    },
    RoleArn='arn:aws:iam::accountid:role/service-role/AmazonKendra-us-east-1-test',
)

print(response)

And here is the S3 template schema: https://docs.aws.amazon.com/kendra/latest/dg/ds-schemas.html#ds-s3-schema

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.