diff --git a/Debug-sessions/CONTRIBUTING.md b/Debug-sessions/CONTRIBUTING.md new file mode 100644 index 0000000..264b88e --- /dev/null +++ b/Debug-sessions/CONTRIBUTING.md @@ -0,0 +1,76 @@ +# Contributing to Azure Cognitive Search samples + +This project welcomes contributions and suggestions. Most contributions require you to agree to a +Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us +the rights to use your contribution. For details, visit https://cla.microsoft.com. + +When you submit a pull request, a CLA-bot will automatically determine whether you need to provide +a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions +provided by the bot. You will only need to do this once across all repos using our CLA. + +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). +For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or +contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. + + - [Code of Conduct](#coc) + - [Issues and Bugs](#issue) + - [Feature Requests](#feature) + - [Submission Guidelines](#submit) + +## Code of Conduct +Help us keep this project open and inclusive. Please read and follow our [Code of Conduct](https://opensource.microsoft.com/codeofconduct/). + +## Found an Issue? +If you find a bug in the source code or a mistake in the documentation, you can help us by +[submitting an issue](#submit-issue) to the GitHub Repository. Even better, you can +[submit a Pull Request](#submit-pr) with a fix. + +## Want a Feature? +You can *request* a new feature by [submitting an issue](#submit-issue) to the GitHub +Repository. If you would like to *implement* a new feature, please submit an issue with +a proposal for your work first, to be sure that we can use it. + +* **Small Features** can be crafted and directly [submitted as a Pull Request](#submit-pr). + +## Submission Guidelines + +### Submitting an Issue +Before you submit an issue, search the archive, maybe your question was already answered. + +If your issue appears to be a bug, and hasn't been reported, open a new issue. +Help us to maximize the effort we can spend fixing issues and adding new +features, by not reporting duplicate issues. Providing the following information will increase the +chances of your issue being dealt with quickly: + +* **Overview of the Issue** - if an error is being thrown a non-minified stack trace helps +* **Version** - what version is affected (e.g. 0.1.2) +* **Motivation for or Use Case** - explain what are you trying to do and why the current behavior is a bug for you +* **Browsers and Operating System** - is this a problem with all browsers? +* **Reproduce the Error** - provide a live example or a unambiguous set of steps +* **Related Issues** - has a similar issue been reported before? +* **Suggest a Fix** - if you can't fix the bug yourself, perhaps you can point to what might be + causing the problem (line of code or commit) + +You can file new issues by providing the above information at the corresponding repository's issues link: https://github.com/[organization-name]/[repository-name]/issues/new]. + +### Submitting a Pull Request (PR) +Before you submit your Pull Request (PR) consider the following guidelines: + +* Search the repository (https://github.com/[organization-name]/[repository-name]/pulls) for an open or closed PR + that relates to your submission. You don't want to duplicate effort. + +* Make your changes in a new git fork: + +* Commit your changes using a descriptive commit message +* Push your fork to GitHub: +* In GitHub, create a pull request +* If we suggest changes then: + * Make the required updates. + * Rebase your fork and force push to your GitHub repository (this will update your Pull Request): + + ```shell + git rebase master -i + git push -f + ``` + +That's it! Thank you for your contribution! diff --git a/Debug-sessions/DebugSessions.postman_collection.json b/Debug-sessions/DebugSessions.postman_collection.json new file mode 100644 index 0000000..1f01dc0 --- /dev/null +++ b/Debug-sessions/DebugSessions.postman_collection.json @@ -0,0 +1,289 @@ +{ + "info": { + "_postman_id": "40d4e8c9-b0eb-44ad-b3c2-0d0e3d5b971d", + "name": "DebugSessions", + "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json" + }, + "item": [ + { + "name": "CreateDatasource", + "request": { + "auth": { + "type": "apikey", + "apikey": [ + { + "key": "value", + "value": "", + "type": "string" + }, + { + "key": "key", + "value": "", + "type": "string" + } + ] + }, + "method": "POST", + "header": [ + { + "key": "api-key", + "value": "{{apiKey}}", + "type": "text" + } + ], + "body": { + "mode": "raw", + "raw": "{\r\n \"name\": \"clinical-trials-ds\",\r\n \"description\": null,\r\n \"type\": \"azureblob\",\r\n \"subtype\": null,\r\n \"credentials\": {\r\n \"connectionString\": \"{{storageConnectionString}}\"\r\n },\r\n \"container\": {\r\n \"name\": \"{{containerName}}\",\r\n \"query\": null\r\n },\r\n \"dataChangeDetectionPolicy\": null,\r\n \"dataDeletionDetectionPolicy\": null\r\n}", + "options": { + "raw": { + "language": "json" + } + } + }, + "url": { + "raw": "https://{{searchService}}.search.windows.net/datasources?api-version=2019-05-06", + "protocol": "https", + "host": [ + "{{searchService}}", + "search", + "windows", + "net" + ], + "path": [ + "datasources" + ], + "query": [ + { + "key": "api-version", + "value": "2019-05-06" + } + ] + } + }, + "response": [] + }, + { + "name": "CreateSkillset", + "request": { + "auth": { + "type": "apikey", + "apikey": [ + { + "key": "value", + "value": "", + "type": "string" + }, + { + "key": "key", + "value": "", + "type": "string" + } + ] + }, + "method": "PUT", + "header": [ + { + "key": "api-key", + "value": "{{apiKey}}", + "type": "text" + } + ], + "body": { + "mode": "raw", + "raw": "{\r\n \r\n \"name\": \"clinical-trials-ss\",\r\n \"description\": \"Demo;\",\r\n \"skills\": [\r\n {\r\n \"@odata.type\": \"#Microsoft.Skills.Text.EntityRecognitionSkill\",\r\n \"name\": \"#1\",\r\n \"description\": null,\r\n \"context\": \"/document/content\",\r\n \"categories\": [\r\n \"Person\",\r\n \"Quantity\",\r\n \"Organization\",\r\n \"URL\",\r\n \"Email\",\r\n \"Location\",\r\n \"DateTime\"\r\n ],\r\n \"defaultLanguageCode\": \"en\",\r\n \"minimumPrecision\": null,\r\n \"includeTypelessEntities\": null,\r\n \"inputs\": [\r\n {\r\n \"name\": \"text\",\r\n \"source\": \"/document/content\",\r\n \"sourceContext\": null,\r\n \"inputs\": []\r\n },\r\n {\r\n \"name\": \"languageCode\",\r\n \"source\": \"/document/languageCode\",\r\n \"sourceContext\": null,\r\n \"inputs\": []\r\n }\r\n ],\r\n \"outputs\": [\r\n {\r\n \"name\": \"organizations\",\r\n \"targetName\": \"organizations\"\r\n },\r\n {\r\n \"name\": \"locations\",\r\n \"targetName\": \"locations\"\r\n },\r\n {\r\n \"name\": \"entities\",\r\n \"targetName\": \"entities\"\r\n }\r\n ]\r\n },\r\n {\r\n \"@odata.type\": \"#Microsoft.Skills.Text.KeyPhraseExtractionSkill\",\r\n \"name\": \"#2\",\r\n \"description\": null,\r\n \"context\": \"/document/merged_content\",\r\n \"defaultLanguageCode\": \"en\",\r\n \"maxKeyPhraseCount\": null,\r\n \"inputs\": [\r\n {\r\n \"name\": \"text\",\r\n \"source\": \"/document/merged_content\",\r\n \"sourceContext\": null,\r\n \"inputs\": []\r\n },\r\n {\r\n \"name\": \"languageCode\",\r\n \"source\": \"/document/language\",\r\n \"sourceContext\": null,\r\n \"inputs\": []\r\n }\r\n ],\r\n \"outputs\": [\r\n {\r\n \"name\": \"keyPhrases\",\r\n \"targetName\": \"keyphrases\"\r\n }\r\n ]\r\n },\r\n {\r\n \"@odata.type\": \"#Microsoft.Skills.Text.LanguageDetectionSkill\",\r\n \"name\": \"#3\",\r\n \"description\": null,\r\n \"context\": \"/document\",\r\n \"inputs\": [\r\n {\r\n \"name\": \"text\",\r\n \"source\": \"/document/merged_content\",\r\n \"sourceContext\": null,\r\n \"inputs\": []\r\n }\r\n ],\r\n \"outputs\": [\r\n {\r\n \"name\": \"languageCode\",\r\n \"targetName\": \"language\"\r\n }\r\n ]\r\n },\r\n {\r\n \"@odata.type\": \"#Microsoft.Skills.Text.TranslationSkill\",\r\n \"name\": \"#4\",\r\n \"description\": null,\r\n \"context\": \"/document/merged_content\",\r\n \"defaultFromLanguageCode\": null,\r\n \"defaultToLanguageCode\": \"en\",\r\n \"suggestedFrom\": \"en\",\r\n \"inputs\": [\r\n {\r\n \"name\": \"text\",\r\n \"source\": \"/document/merged_content\",\r\n \"sourceContext\": null,\r\n \"inputs\": []\r\n }\r\n ],\r\n \"outputs\": [\r\n {\r\n \"name\": \"translatedText\",\r\n \"targetName\": \"translated_text\"\r\n }\r\n ]\r\n },\r\n \r\n {\r\n \"@odata.type\": \"#Microsoft.Skills.Text.MergeSkill\",\r\n \"name\": \"#5\",\r\n \"description\": null,\r\n \"context\": \"/document\",\r\n \"insertPreTag\": \" \",\r\n \"insertPostTag\": \" \",\r\n \"inputs\": [\r\n {\r\n \"name\": \"text\",\r\n \"source\": \"/document/content\",\r\n \"sourceContext\": null,\r\n \"inputs\": []\r\n },\r\n {\r\n \"name\": \"itemsToInsert\",\r\n \"source\": \"/document/normalized_images/*/text\",\r\n \"sourceContext\": null,\r\n \"inputs\": []\r\n },\r\n {\r\n \"name\": \"offsets\",\r\n \"source\": \"/document/normalized_images/*/contentOffset\",\r\n \"sourceContext\": null,\r\n \"inputs\": []\r\n }\r\n ],\r\n \"outputs\": [\r\n {\r\n \"name\": \"mergedText\",\r\n \"targetName\": \"merged_content\"\r\n }\r\n ]\r\n },\r\n {\r\n \"@odata.type\": \"#Microsoft.Skills.Vision.OcrSkill\",\r\n \"name\": \"#6\",\r\n \"description\": null,\r\n \"context\": \"/document/normalized_images/*\",\r\n \"textExtractionAlgorithm\": null,\r\n \"lineEnding\": \"Space\",\r\n \"defaultLanguageCode\": \"en\",\r\n \"detectOrientation\": true,\r\n \"inputs\": [\r\n {\r\n \"name\": \"image\",\r\n \"source\": \"/document/normalized_images/*\",\r\n \"sourceContext\": null,\r\n \"inputs\": []\r\n }\r\n ],\r\n \"outputs\": [\r\n {\r\n \"name\": \"text\",\r\n \"targetName\": \"text\"\r\n },\r\n {\r\n \"name\": \"layoutText\",\r\n \"targetName\": \"layoutText\"\r\n }\r\n ]\r\n }\r\n \r\n ],\r\n \"cognitiveServices\": null\r\n}", + "options": { + "raw": { + "language": "json" + } + } + }, + "url": { + "raw": "https://{{searchService}}.search.windows.net/skillsets/clinical-trials-ss?api-version=2019-05-06", + "protocol": "https", + "host": [ + "{{searchService}}", + "search", + "windows", + "net" + ], + "path": [ + "skillsets", + "clinical-trials-ss" + ], + "query": [ + { + "key": "api-version", + "value": "2019-05-06" + } + ] + } + }, + "response": [] + }, + { + "name": "CreateIndex", + "request": { + "auth": { + "type": "apikey", + "apikey": [ + { + "key": "value", + "value": "", + "type": "string" + }, + { + "key": "key", + "value": "", + "type": "string" + } + ] + }, + "method": "PUT", + "header": [ + { + "key": "api-key", + "value": "{{apiKey}}", + "type": "text" + } + ], + "body": { + "mode": "raw", + "raw": "{\r\n \"name\": \"clinical-trials\",\r\n \"defaultScoringProfile\": \"\",\r\n \"fields\": [\r\n {\r\n \"name\": \"content\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": true,\r\n \"filterable\": false,\r\n \"retrievable\": true,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": \"standard.lucene\",\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_storage_content_type\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_storage_size\",\r\n \"type\": \"Edm.Int64\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_storage_last_modified\",\r\n \"type\": \"Edm.DateTimeOffset\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_storage_content_md5\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_storage_name\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_storage_path\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": true,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": true,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_storage_file_extension\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_content_type\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_language\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_author\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_title\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"metadata_creation_date\",\r\n \"type\": \"Edm.DateTimeOffset\",\r\n \"searchable\": false,\r\n \"filterable\": false,\r\n \"retrievable\": false,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": null,\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"organizations\",\r\n \"type\": \"Collection(Edm.String)\",\r\n \"searchable\": true,\r\n \"filterable\": false,\r\n \"retrievable\": true,\r\n \"sortable\": false,\r\n \"facetable\": true,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": \"standard.lucene\",\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"locations\",\r\n \"type\": \"Collection(Edm.String)\",\r\n \"searchable\": true,\r\n \"filterable\": false,\r\n \"retrievable\": true,\r\n \"sortable\": false,\r\n \"facetable\": true,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": \"standard.lucene\",\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"keyphrases\",\r\n \"type\": \"Collection(Edm.String)\",\r\n \"searchable\": true,\r\n \"filterable\": true,\r\n \"retrievable\": true,\r\n \"sortable\": false,\r\n \"facetable\": true,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": \"standard.lucene\",\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"language\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": true,\r\n \"filterable\": false,\r\n \"retrievable\": true,\r\n \"sortable\": false,\r\n \"facetable\": true,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": \"standard.lucene\",\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"translated_text\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": true,\r\n \"filterable\": false,\r\n \"retrievable\": true,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": \"en.lucene\",\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"merged_content\",\r\n \"type\": \"Edm.String\",\r\n \"searchable\": true,\r\n \"filterable\": false,\r\n \"retrievable\": true,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": \"standard.lucene\",\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"text\",\r\n \"type\": \"Collection(Edm.String)\",\r\n \"searchable\": true,\r\n \"filterable\": false,\r\n \"retrievable\": true,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": \"standard.lucene\",\r\n \"synonymMaps\": []\r\n },\r\n {\r\n \"name\": \"layoutText\",\r\n \"type\": \"Collection(Edm.String)\",\r\n \"searchable\": true,\r\n \"filterable\": false,\r\n \"retrievable\": true,\r\n \"sortable\": false,\r\n \"facetable\": false,\r\n \"key\": false,\r\n \"indexAnalyzer\": null,\r\n \"searchAnalyzer\": null,\r\n \"analyzer\": \"standard.lucene\",\r\n \"synonymMaps\": []\r\n }\r\n ],\r\n \"scoringProfiles\": [],\r\n \"corsOptions\": null,\r\n \"suggesters\": [\r\n {\r\n \"name\": \"sg\",\r\n \"searchMode\": \"analyzingInfixMatching\",\r\n \"sourceFields\": [\r\n \"keyphrases\"\r\n ]\r\n }\r\n ],\r\n \"analyzers\": [],\r\n \"tokenizers\": [],\r\n \"tokenFilters\": [],\r\n \"charFilters\": [],\r\n \"encryptionKey\": null\r\n}", + "options": { + "raw": { + "language": "json" + } + } + }, + "url": { + "raw": "https://{{searchService}}.search.windows.net/indexes/clinical-trials?api-version=2019-05-06", + "protocol": "https", + "host": [ + "{{searchService}}", + "search", + "windows", + "net" + ], + "path": [ + "indexes", + "clinical-trials" + ], + "query": [ + { + "key": "api-version", + "value": "2019-05-06" + } + ] + } + }, + "response": [] + }, + { + "name": "CreateIndexer", + "request": { + "auth": { + "type": "apikey", + "apikey": [ + { + "key": "value", + "value": "", + "type": "string" + }, + { + "key": "key", + "value": "", + "type": "string" + } + ] + }, + "method": "PUT", + "header": [ + { + "key": "api-key", + "value": "{{apiKey}}", + "type": "text" + } + ], + "body": { + "mode": "raw", + "raw": "{\r\n \"name\": \"clinical-trials-idxr\",\r\n \"description\": \"\",\r\n \"dataSourceName\": \"clinical-trials-ds\",\r\n \"skillsetName\": \"clinical-trials-ss\",\r\n \"targetIndexName\": \"clinical-trials\",\r\n \"disabled\": null,\r\n \"schedule\": null,\r\n \"parameters\": {\r\n \"batchSize\": null,\r\n \"maxFailedItems\": 0,\r\n \"maxFailedItemsPerBatch\": 0,\r\n \"base64EncodeKeys\": null,\r\n \"configuration\": {\r\n \"imageAction\": \"generateNormalizedImages\",\r\n \"dataToExtract\": \"contentAndMetadata\",\r\n \"parsingMode\": \"default\"\r\n }\r\n },\r\n \"fieldMappings\": [\r\n {\r\n \"sourceFieldName\": \"metadata_storage_path\",\r\n \"targetFieldName\": \"metadata_storage_path\",\r\n \"mappingFunction\": {\r\n \"name\": \"base64Encode\",\r\n \"parameters\": null\r\n }\r\n }\r\n ],\r\n \"outputFieldMappings\": [\r\n {\r\n \"sourceFieldName\": \"/document/merged_content/organizations\",\r\n \"targetFieldName\": \"organizations\",\r\n \"mappingFunction\": null\r\n },\r\n {\r\n \"sourceFieldName\": \"/document/merged_content/locations\",\r\n \"targetFieldName\": \"locations\",\r\n \"mappingFunction\": null\r\n },\r\n {\r\n \"sourceFieldName\": \"/document/merged_content/keyphrases\",\r\n \"targetFieldName\": \"keyphrases\",\r\n \"mappingFunction\": null\r\n },\r\n {\r\n \"sourceFieldName\": \"/document/language\",\r\n \"targetFieldName\": \"language\",\r\n \"mappingFunction\": null\r\n },\r\n {\r\n \"sourceFieldName\": \"/document/merged_content/translated_text\",\r\n \"targetFieldName\": \"translated_text\",\r\n \"mappingFunction\": null\r\n },\r\n {\r\n \"sourceFieldName\": \"/document/merged_content\",\r\n \"targetFieldName\": \"merged_content\",\r\n \"mappingFunction\": null\r\n },\r\n {\r\n \"sourceFieldName\": \"/document/normalized_images/*/text\",\r\n \"targetFieldName\": \"text\",\r\n \"mappingFunction\": null\r\n },\r\n {\r\n \"sourceFieldName\": \"/document/normalized_images/*/layoutText\",\r\n \"targetFieldName\": \"layoutText\",\r\n \"mappingFunction\": null\r\n }\r\n ]\r\n}", + "options": { + "raw": { + "language": "json" + } + } + }, + "url": { + "raw": "https://{{searchService}}.search.windows.net/indexers/clinical-trials-idxr?api-version=2019-05-06", + "protocol": "https", + "host": [ + "{{searchService}}", + "search", + "windows", + "net" + ], + "path": [ + "indexers", + "clinical-trials-idxr" + ], + "query": [ + { + "key": "api-version", + "value": "2019-05-06" + } + ] + } + }, + "response": [] + } + ], + "event": [ + { + "listen": "prerequest", + "script": { + "id": "7c354fac-90fd-450d-b9a8-a8fe8c119af0", + "type": "text/javascript", + "exec": [ + "" + ] + } + }, + { + "listen": "test", + "script": { + "id": "eca8a3a9-bfaf-4c64-9b99-175d99b8c5ec", + "type": "text/javascript", + "exec": [ + "" + ] + } + } + ], + "variable": [ + { + "id": "a45f5c09-be88-4599-b8b7-b35bf9ff8352", + "key": "searchService", + "value": "", + "type": "string" + }, + { + "id": "4300adf2-37d0-4511-af6a-78db1f9186ef", + "key": "apiKey", + "value": "", + "type": "string" + }, + { + "id": "0ce78052-e07f-4d17-b435-b1593de79edf", + "key": "storageConnectionString", + "value": "", + "type": "string" + }, + { + "id": "65ed81ec-9658-4a1b-ab18-8eaa762c34b7", + "key": "containerName", + "value": "", + "type": "string" + } + ], + "protocolProfileBehavior": {} +} \ No newline at end of file diff --git a/Debug-sessions/LICENSE.md b/Debug-sessions/LICENSE.md new file mode 100644 index 0000000..d1ca00f --- /dev/null +++ b/Debug-sessions/LICENSE.md @@ -0,0 +1,21 @@ + MIT License + + Copyright (c) Microsoft Corporation. All rights reserved. + + Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in all + copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + SOFTWARE \ No newline at end of file diff --git a/Debug-sessions/README.md b/Debug-sessions/README.md new file mode 100644 index 0000000..7912187 --- /dev/null +++ b/Debug-sessions/README.md @@ -0,0 +1,93 @@ +--- +page_type: sample +languages: + - rest +name: Debug skillsets in Azure Cognitive Search +description: | + Learn how the portal tool Debug sessions can help you fix enrichment pipeline issues in Azure Cognitive Search. This collection creates a skillset with invalid fields and missing data, easily fixed in Debug sessions. +products: + - azure + - azure-cognitive-search +urlFragment: rest-api-debug-sessions +--- + +# Fix skillset issues using Debug sessions tool in Azure Cognitive Search + +![Flask sample MIT license badge](https://img.shields.io/badge/license-MIT-green.svg) + +This [Postman](https://www.getpostman.com/) collection uses the Azure Cognitive Search REST APIs to create an enrichment pipeline with warnings you can fix in the Debug sessions tool in Azure portal. Requests are provided in the V2 collection format, which you can import and then modify for connections to your search service. + +This readme also explains how to set up the clinical trials data used in this collection. + +## Contents + +| File/folder | Description | +|-------------|-------------| +| `Debug-sessions.postman_collection.json` | Import into Postman | +| `CONTRIBUTING.md` | Guidelines for contributing to the sample. | +| `README.md` | This README file. | +| `LICENSE.md` | The license for the sample. | + +## Prerequisites + +- [Postman Desktop app](https://www.getpostman.com/) +- [Azure Cognitive Search service](https://docs.microsoft.com/azure/search/search-create-service-portal) +- [Azure Storage account](https://docs.microsoft.com/azure/storage/common/storage-account-create?tabs=azure-portal) +- [Clinical Trials Data (19 documents)](https://github.com/Azure-Samples/azure-search-sample-data/tree/master/clinical-trials-pdf-19) + +## Set up the data + +1. Download the clinical trials data set. + +1. In Azure portal, in Azure Storage, [create a Blob container](https://docs.microsoft.com/azure/storage/blobs/storage-quickstart-blobs-portal) and upload the 19 documents. + +1. Make a note of the blob container name. + +## Get connection information + +Collect the connection information you'll need to provide on the requests. In a later step, you'll paste these values into variables on the collection. + +1. Still in Azure portal, in Azure Storage, select **Access keys** on the left. Copy one of the connection strings. It should be in this format: `DefaultEndpointsProtocol=https;AccountName=;AccountKey=;` + +1. Paste the connection string into Notepad and remove the last segment: `EndpointSuffix=core.windows.net`. When deleting part of the string, be careful to preserve the trailing semi-colon after the key. + +1. Back in the portal, go to the Azure Cognitive Search **Overview** page of your search service. + +1. Get the name of the service (for example, given an endpoint of `https://demo-svc.search.windows.net`, the service name is `demo-svc`). Paste the search service name into Notepad. + +1. Select **Keys** on the left and copy one of admin keys. Paste it into Notepad. + +## Setup + +1. Clone or download this sample repository. + +1. Extract contents if the download is a zip file. Make sure the files are read-write. + +1. Start Postman and select **Import** to load the Debug-sessions Postman collection. + +1. After importing the collection, expand the (`...`) action list and select **Edit**. + +1. Select **Variables** and enter the values you collected earlier: + + + In **searchService**, enter the name of your search service. + + In **apiKey**, enter the admin API key of your search service. + + In **storageConnectionString**, enter the connection string for your Azure Storage account. + + In **blobContainer**, enter the name of the blob container that stores the clinical trials documents. + +## Send the requests + +After you have finished adding the variable, you can run each request in turn, with no further modification. + +1. Start with **CreateDataSource**, click **Send** to submit the request to Azure Cognitive Search. This request creates a data source object that specifies the connection to the Blob container. + +1. Continue on to **CreateSkillset**. This request creates, but does not run, a skillset definition on the search service. + +1. Send the **CreateIndex** request to create the index definition. An enrichment pipeline sends output to a searchable index. The index stores the searchable content created by the pipeline. + +1. The **CreateIndexer** request executes the pipeline. Data movement occurs in this step: loading PDFs from the blob container, cracking the PDFs, formulating requests for AI processing, composing the enriched documents which include extracted and created data, and outputting the content to a search index. + +You have now created all of the objects necessary for using Debug sessions. This collection excludes monitoring and verification steps. You'll do that in the portal with Debug sessions. + +## Next steps + +By design, the indexer produces warnings and omits data. To learn how to find and fix these issues in Debug sessions, go to [Tutorial: Diagnose, repair, and commit changes to your skillset](https://docs.microsoft.com/azure/search/cognitive-search-tutorial-debug-sessions) for the next steps. \ No newline at end of file diff --git a/README.md b/README.md index 7706e2c..2c43d49 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,18 @@ This repository provides Postman collections used in REST API walkthroughs. All To use these collections, import them into **Postman**. +## Debug sessions + +This Postman collection creates a faulty enrichment pipeline with problems you can fix in Debug sessions. Requests create a data source, index, indexer, and skillset. The data source points to a Blob container that you create, using clinical trial PDFs that you upload. For instructions on how to set up this pipeline, see the readme file for this sample. + +## Knowledge-store + +This Postman collection contains requests for creating and populating an index with hotel reviews data from Kaggle. The collection also includes an indexer and skillset that contains instructions for expressing a knowledge store in Azure Storage. + +## Projections + +Contains requests that demonstrate how to configure table, object and file projections using the REST APIs. + ## Query examples: Full syntax, Patterns and special characters, Simple syntax Query requests in these collections demonstrate syntax for a broad range of scenarios described in these articles: [Full Lucene syntax examples](https://docs.microsoft.com/azure/search/search-query-lucene-examples), [Partial term and pattern matching](https://docs.microsoft.com/azure/search/search-query-partial-matching), [Simple syntax query examples](https://docs.microsoft.com/azure/search/query-simple-syntax). @@ -16,10 +28,6 @@ With the exception of Full Syntax, which queries the read-only NYCJobs index hos Most examples are self-explanatory. For the NOT query, see the Boolean operator section for either [simple](https://docs.microsoft.com/azure/search/query-simple-syntax). -## Knowledge-store collection - -Contains requests for creating and populating an index with hotel reviews data from Kaggle. The collection also includes an indexer and skillset that contains instructions for expressing a knowledge store in Azure Storage. - ## Quickstart collection Includes 4 requests used to create an index, load documents, search the index, and query system information. Request bodies include JSON documents that provides index and documents.