Support for sidemap parsing from text instead of urls

[NiClassic]

While working with your library, I noticed that content can be extracted from a site by passing the response text to the extract() function. However, I found that the sitemap_search() function only supports directly passing a URL to the sitemap, which can be problematic in environments with limited internet access (e.g., behind a proxy). I expected a similar behavior to the following snippet:

import httpx
from trafilatura import sitemaps

with httpx.Client(mounts=proxy_mounts) as client:
    url = "http://example.com/sitemap.xml"
    res = client.get(url)

sitemaps = sitemaps.sitemap_search(res)

Please let me know if this is something that could be implemented in the future.

[adbar]

Hi @NiClassic, the sitemap_search function attemps to guess a sitemap's address if you pass a homepage instead of a direct sitemap URL.

If you pass https://example.org/ as input and if there is a sitemap at https://example.org/sitemap.xml it should be found automatically.

Does that solve your problem or is there an issue with the guessing mechanism?