AMP-85211 added static robots.txt

[akramarev]

Amplitude Developer Docs PR

Description

added static robots.txt with one page that is excluded from navigation and website search

Deadline

no urgent

Change type

• Doc bug fix. Fixes #[insert issue number]. Amplitude contributors include Jira issue number.
• Doc update.
• New documentation.
• Non-documentation related fix or update.

PR checklist:

• My documentation follows the style guidelines of this project.
• I previewed my documentation on a local server using mkdocs serve.
• Running mkdocs serve didn't generate any failures.
• I have performed a self-review of my own documentation.

@amplitude-dev-docs

[github-actions]

🦙 MegaLinter status: ✅ SUCCESS

Descriptor	Linter	Files	Fixed	Errors	Elapsed time

See detailed report in MegaLinter reports
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by

[aws-amplify-us-east-1]

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-981.d19s7xzcva2mw3.amplifyapp.com