Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow custom user agent #31

Open
j6s opened this issue Jan 13, 2019 · 1 comment · May be fixed by #62
Open

Allow custom user agent #31

j6s opened this issue Jan 13, 2019 · 1 comment · May be fixed by #62
Assignees
@mlj
Labels
enhancement
Milestone
Release 2.3.0

Comments

@j6s
Copy link

j6s commented Jan 13, 2019

I have one specific feed ( https://www.codingblocks.net/podcast-feed.xml ) which does some user agent filtering in the backend which will redirect all user agents from automated tools (e.g. curl) to a 403 error page instead of generating the feed.

This also means that castget cannot fetch the feed correctly.
It would be nice to have an option (either in castgetrc, as per feed option or as command line argument) to specify a custom user agent that will be used to fetch the feed.

In case the feed above changes, here is an example that illustrates the problem outside of castget:

$ curl 'https://www.codingblocks.net/podcast-feed.xml' | head 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   344  100   344    0     0    262      0  0:00:01  0:00:01 --:--:--   262
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /podcast-feed.xml
on this server.<br />
</p>
<p>Additionally, a 403 Forbidden
error was encountered while trying to use an ErrorDocument to handle the request.</p>

$ curl -H 'User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1' 'https://www.codingblocks.net/podcast-feed.xml' | head 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  419k    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
xmlns:rawvoice="http://www.rawvoice.com/rawvoiceRssModule/"
  7  419k    7 32768    0     0  31477      0  0:00:13  0:00:01  0:00:12 31477
@mlj mlj added the enhancement label Mar 9, 2019
@mlj mlj added this to the Release 2.0.0 milestone Mar 9, 2019
@mlj mlj self-assigned this Mar 9, 2019
@mlj
Copy link
Owner

mlj commented Mar 9, 2019

Thanks for the suggestion! Given the scenario you describe, it sounds like having this as an option in castgetrc is the way to go.

jetmore added a commit to jetmore/castget that referenced this issue Jan 22, 2023
@jetmore
add support for user_agent configuration
de69096 
fixes mlj#31
@jetmore jetmore linked a pull request Jan 22, 2023 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mlj mlj

Labels
enhancement
Projects
None yet
Milestone
Release 2.3.0
Development

Successfully merging a pull request may close this issue.

add support for user_agent configuration jetmore/castget
2 participants
@mlj @j6s