DESCRIPTION

Package: robotstxt
Type: Package
Title: A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker
Version: 0.7.15.9000
Authors@R: c(
        person(
          "Pedro", "Baltazar", role = c("ctb"),
          email = "pedrobtz@gmail.com"
        ),
         person(
          "Jordan", "Bradford", role = c("cre"),
          email = "jrdnbradford@gmail.com"
        ),
        person(
          "Peter", "Meissner", role = c("aut"),
          email = "retep.meissner@gmail.com"
        ),
        person(
            "Kun", "Ren", email = "mail@renkun.me", role = c("aut", "cph"),
            comment = "Author and copyright holder of list_merge.R."
        ),
	    person("Oliver", "Keys", role = "ctb", comment = "original release code review"),
	    person("Rich", "Fitz John", role = "ctb", comment = "original release code review")
	  )
Description: Provides functions to download and parse 'robots.txt' files.
        Ultimately the package makes it easy to check if bots
        (spiders, crawler, scrapers, ...) are allowed to access specific
        resources on a domain.
License: MIT + file LICENSE
BugReports: https://github.com/ropensci/robotstxt/issues
URL: https://docs.ropensci.org/robotstxt/, https://github.com/ropensci/robotstxt
Imports:
    stringr (>= 1.0.0),
    httr (>= 1.0.0),
    spiderbar (>= 0.2.0),
    future.apply (>= 1.0.0),
    magrittr,
    utils
Suggests:
    knitr,
    rmarkdown,
    dplyr,
    testthat (>= 3.0.0),
    covr,
    curl
Depends:
    R (>= 3.0.0)
VignetteBuilder: knitr
RoxygenNote: 7.3.2
Encoding: UTF-8
Config/testthat/edition: 3