Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENCM-211-update-robots.txt #4429

Open
wants to merge 4 commits into
base: dev
Choose a base branch
from
Open

ENCM-211-update-robots.txt #4429

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
46d2d53
Update robots.txt
stuartmiyasato Jun 13, 2024
d22f593
Update robots.txt
stuartmiyasato Jun 18, 2024
ea49f72
Update robots.txt
stuartmiyasato Jun 20, 2024
57247b5
Update robots.txt
stuartmiyasato Jun 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 54 additions & 19 deletions src/encoded/static/robots.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,66 +36,101 @@ Disallow: /encode-software
Disallow: /region-search
Disallow: /brain-matrix
Disallow: /encyclopedia/*
Disallow: /deeply-profiled-uniform-batch-matrix
Disallow: /deeply-profiled-uniform-batch-matrix/
Disallow: /deeply-profiled-matrix
Disallow: /deeply-profiled-matrix/
Disallow: /degron-matrix
Disallow: /degron-matrix/
Disallow: /encore-matrix
Disallow: /encore-matrix/
Disallow: /stem-cell-matrix
Disallow: /stem-cell-matrix/
Disallow: /immune-cells
Disallow: /immune-cells/
Disallow: /mouse-development-matrix
Disallow: /mouse-development-matrix/
Disallow: /functional-characterization-matrix
Disallow: /functional-characterization-matrix/
Disallow: /human-donor-matrix
Disallow: /human-donor-matrix/
Allow: /encode-software/?type=Software
Allow: /region-search/
Allow: /rnaget/
Allow: /encyclopedia/
Allow: /search/?type=Cart&status=listed&status=released
Allow: /search/?type=Experiment$
Allow: /search/?type=Experiment&status=released$
Allow: /search/?type=Experiment&control_type!=*&status=released&perturbed=false$
Allow: /search/?type=FunctionalCharacterizationExperiment$
Allow: /search/?type=TransgenicEnhancerExperiment$
Allow: /search/?type=FunctionalCharacterizationExperiment&control_type!=*&audit.WARNING.category!=lacking+processed+data$
Allow: /search/?type=FunctionalCharacterizationExperiment&control_type!=*$
Allow: /search/?type=FunctionalCharacterizationExperiment&type=FunctionalCharacterizationSeries&type=TransgenicEnhancerExperiment&config=FunctionalCharacterization&datapoint=false&control_type!=*&status=released
Allow: /report/?type=Experiment$
Allow: /report/?type=Experiment&status=released$
Allow: /matrix/?type=Experiment$
Allow: /matrix/?type=Experiment&status=released$
Allow: /matrix/?type=Experiment&control_type!=*&status=released&perturbed=false$
Allow: /summary/?type=Experiment$
Allow: /summary/?type=Experiment&control_type!=*&replicates.library.biosample.donor.organism.scientific_name=Homo+sapiens&status=released
Allow: /summary/?type=Experiment&status=released$
Allow: /summary/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo+sapiens$
Allow: /summary/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus+musculus$
Allow: /summary/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Drosophila+melanogaster$
Allow: /summary/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Caenorhabditis+elegans$
Allow: /series-search/?type=OrganismDevelopmentSeries$
Allow: /series-search/?type=OrganismDevelopmentSeries&status=released
Allow: /series-search/?type=TreatmentTimeSeries$
Allow: /series-search/?type=TreatmentConcentrationSeries$
Allow: /series-search/?type=ReplicationTimingSeries$
Allow: /series-search/?type=GeneSilencingSeries$
Allow: /search/?type=Annotation$
Allow: /search/?type=Annotation&encyclopedia_version=ENCODE+v5$
Allow: /search/?type=Annotation&encyclopedia_version=ENCODE+v6$
Allow: /search/?type=Annotation&encyclopedia_version=current&status=released
Allow: /search/?type=Annotation&encyclopedia_version=current&annotation_type=imputation&status=released
Allow: /report/?type=Annotation$
Allow: /report/?type=Annotation&encyclopedia_version=ENCODE+v5$
Allow: /report/?type=Annotation&encyclopedia_version=ENCODE+v6$
Allow: /matrix/?type=Annotation$
Allow: /matrix/?type=Annotation&annotation_type!=imputation&status=released
Allow: /matrix/?type=Annotation&encyclopedia_version=ENCODE+v5$
Allow: /matrix/?type=Annotation&encyclopedia_version=ENCODE+v6$
Allow: /matrix/?type=Annotation&encyclopedia_version=ENCODE+v5&annotation_type=candidate+Cis-Regulatory+Elements$
Allow: /matrix/?type=Annotation&encyclopedia_version=ENCODE+v6&annotation_type=candidate+Cis-Regulatory+Elements$
Allow: /search/?type=AntibodyLot&status=released$
Allow: /entex-matrix/?type=Experiment&status=released&internal_tags=ENTEx$
Allow: /reference-epigenome-matrix/?type=Experiment&related_series.@type=ReferenceEpigenome&replicates.library.biosample.donor.organism.scientific_name=Homo+sapiens$
Allow: /reference-epigenome-matrix/?type=Experiment&related_series.@type=ReferenceEpigenome&replicates.library.biosample.donor.organism.scientific_name=Mus+musculus$
Allow: /reference-epigenome-matrix/?type=Experiment&control_type!=*&related_series.@type=ReferenceEpigenome&replicates.library.biosample.donor.organism.scientific_name=Homo+sapiens&status=released
Allow: /reference-epigenome-matrix/?type=Experiment&control_type!=*&related_series.@type=ReferenceEpigenome&replicates.library.biosample.donor.organism.scientific_name=Mus+musculus&status=released
Allow: /reference-epigenome-matrix/?type=Experiment&related_series.@type=ReferenceEpigenome&replicates.library.biosample.donor.organism.scientific_name=Homo+sapiens&status=released$
Allow: /reference-epigenome-matrix/?type=Experiment&related_series.@type=ReferenceEpigenome&replicates.library.biosample.donor.organism.scientific_name=Mus+musculus&status=released$
Allow: /matrix/?type=Experiment&status=released&internal_tags=ENCORE$
Allow: /sescc-stem-cell-matrix/?type=Experiment&internal_tags=SESCC$
Allow: /mouse-development-matrix/?type=Experiment&status=released&related_series.@type=OrganismDevelopmentSeries&replicates.library.biosample.organism.scientific_name=Mus+musculus$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens&assay_title=Histone%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens&assay_title=Histone%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens&assay_title=Histone%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens&assay_title=Histone%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens&assay_title=TF%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens&assay_title=TF%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens&assay_title=TF%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens&assay_title=TF%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus%20musculus&assay_title=Histone%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus%20musculus&assay_title=Histone%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus%20musculus&assay_title=Histone%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus%20musculus&assay_title=Histone%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus%20musculus&assay_title=TF%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus%20musculus&assay_title=TF%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus%20musculus&assay_title=TF%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus%20musculus&assay_title=TF%20ChIP-seq&status=released$
Allow: /search/?type=Annotation&accession=ENCSR471KRT&accession=ENCSR461KLY&accession=ENCSR770MVN&accession=ENCSR695LYW$
Allow: /search/?type=Annotation&accession=ENCSR405FRQ&accession=ENCSR216NPK$
Allow: /series-search/?type=DifferentiationSeries$
Allow: /series-search/?type=DiseaseSeries$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens&assay_title=Histone%20ChIP-seq&assay_title=Mint-ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens&assay_title=TF%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus%20musculus&assay_title=TF%20ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Mus%20musculus&assay_title=Histone%20ChIP-seq&assay_title=Mint-ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Caenorhabditis%20elegans&assay_title=TF%20ChIP-seq&assay_title=Mint-ChIP-seq&status=released$
Allow: /chip-seq-matrix/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Drosophila%20melanogaster&assay_title=TF%20ChIP-seq&status=released$
Allow: /brain-matrix/?type=Experiment&status=released&internal_tags=RushAD$
Allow: /deeply-profiled-uniform-batch-matrix/?type=Experiment&control_type!=*&status=released&replicates.library.biosample.biosample_ontology.term_id=EFO:0002106&replicates.library.biosample.biosample_ontology.term_id=EFO:0001203&replicates.library.biosample.biosample_ontology.term_id=EFO:0006711&replicates.library.biosample.biosample_ontology.term_id=EFO:0002713&replicates.library.biosample.biosample_ontology.term_id=EFO:0002847&replicates.library.biosample.biosample_ontology.term_id=EFO:0002074&replicates.library.biosample.biosample_ontology.term_id=EFO:0001200&replicates.library.biosample.biosample_ontology.term_id=EFO:0009747&replicates.library.biosample.biosample_ontology.term_id=EFO:0002824&replicates.library.biosample.biosample_ontology.term_id=CL:0002327&replicates.library.biosample.biosample_ontology.term_id=CL:0002618&replicates.library.biosample.biosample_ontology.term_id=EFO:0002784&replicates.library.biosample.biosample_ontology.term_id=EFO:0001196&replicates.library.biosample.biosample_ontology.term_id=EFO:0001187&replicates.library.biosample.biosample_ontology.term_id=EFO:0002067&replicates.library.biosample.biosample_ontology.term_id=EFO:0001099&replicates.library.biosample.biosample_ontology.term_id=EFO:0002819&replicates.library.biosample.biosample_ontology.term_id=EFO:0009318&replicates.library.biosample.biosample_ontology.term_id=EFO:0001086&replicates.library.biosample.biosample_ontology.term_id=EFO:0007950&replicates.library.biosample.biosample_ontology.term_id=EFO:0003045&replicates.library.biosample.biosample_ontology.term_id=EFO:0003042&replicates.library.biosample.internal_tags=Deeply%20Profiled
Allow: /deeply-profiled-matrix/?type=Experiment&control_type!=*&status=released&replicates.library.biosample.biosample_ontology.term_id=EFO:0002106&replicates.library.biosample.biosample_ontology.term_id=EFO:0001203&replicates.library.biosample.biosample_ontology.term_id=EFO:0006711&replicates.library.biosample.biosample_ontology.term_id=EFO:0002713&replicates.library.biosample.biosample_ontology.term_id=EFO:0002847&replicates.library.biosample.biosample_ontology.term_id=EFO:0002074&replicates.library.biosample.biosample_ontology.term_id=EFO:0001200&replicates.library.biosample.biosample_ontology.term_id=EFO:0009747&replicates.library.biosample.biosample_ontology.term_id=EFO:0002824&replicates.library.biosample.biosample_ontology.term_id=CL:0002327&replicates.library.biosample.biosample_ontology.term_id=CL:0002618&replicates.library.biosample.biosample_ontology.term_id=EFO:0002784&replicates.library.biosample.biosample_ontology.term_id=EFO:0001196&replicates.library.biosample.biosample_ontology.term_id=EFO:0001187&replicates.library.biosample.biosample_ontology.term_id=EFO:0002067&replicates.library.biosample.biosample_ontology.term_id=EFO:0001099&replicates.library.biosample.biosample_ontology.term_id=EFO:0002819&replicates.library.biosample.biosample_ontology.term_id=EFO:0009318&replicates.library.biosample.biosample_ontology.term_id=EFO:0001086&replicates.library.biosample.biosample_ontology.term_id=EFO:0007950&replicates.library.biosample.biosample_ontology.term_id=EFO:0003045&replicates.library.biosample.biosample_ontology.term_id=EFO:0003042$
Allow: /degron-matrix/?type=Experiment&control_type!=*&status=released&internal_tags=Degron$
Allow: /human-donor-matrix/?type=Experiment&control_type!=*&replicates.library.biosample.donor.organism.scientific_name=Homo+sapiens&biosample_ontology.classification=tissue&status=released&config=HumanDonorMatrix$
Allow: /human-donor-matrix/?type=Experiment&control_type!=*&replicates.library.biosample.donor.organism.scientific_name=Homo+sapiens&status=released&config=HumanDonorMatrix&config=HumanDonorMatrix&biosample_ontology.classification=primary+cell$
Allow: /human-donor-matrix/?type=Experiment&control_type!=*&replicates.library.biosample.donor.organism.scientific_name=Homo+sapiens&status=released&config=HumanDonorMatrix&biosample_ontology.classification=tissue&biosample_ontology.classification=primary+cell$
Allow: /encore-matrix/?type=Experiment&status=released&internal_tags=ENCORE$
Allow: /stem-cell-matrix/?type=Experiment&replicates.library.biosample.donor.accession=ENCDO222AAA&status=released&control_type!=*$
Allow: /immune-cells/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo+sapiens&biosample_ontology.cell_slims=hematopoietic+cell&biosample_ontology.classification=primary+cell&control_type!=*&status!=replaced&status!=revoked&status!=archived&biosample_ontology.system_slims=immune+system&biosample_ontology.system_slims=circulatory+system&config=immune$
Allow: /immune-cells/?type=Experiment&replicates.library.biosample.donor.organism.scientific_name=Homo+sapiens&biosample_ontology.cell_slims=hematopoietic+cell&biosample_ontology.classification=primary+cell&control_type!=*&status=released&biosample_ontology.system_slims=immune+system&biosample_ontology.system_slims=circulatory+system&config=immune
Allow: /single-cell/?type=Experiment&assay_slims=Single+cell&status=released&replicates.library.biosample.donor.organism.scientific_name=Homo%20sapiens$
Allow: /single-cell/?type=FunctionalCharacterizationExperiment&assay_slims=Single+cell&status=released$
Allow: /single-cell/?type=SingleCellRnaSeries&status=released$
Allow: /single-cell/?type=Annotation&annotation_type=cell+type+annotation&status=released$
Allow: /search/?type=Annotation&encyclopedia_version=current&annotation_type=candidate+Cis-Regulatory+Elements&annotation_type=chromatin+state&annotation_type=representative+DNase+hypersensitivity+sites&status=released$
Allow: /functional-characterization-matrix/?type=FunctionalCharacterizationExperiment&type=FunctionalCharacterizationSeries&type=TransgenicEnhancerExperiment&config=FunctionalCharacterization&datapoint=false&control_type!=*&status=released$