Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding configuration elements to block based on user-agents #20

Closed
wants to merge 7 commits into from
Closed

Adding configuration elements to block based on user-agents #20

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
2cf3d5d
Adding Caddyfile code blocks to generate user-agents block list
Sep 5, 2024
db40194
Documenting new variable in README
Sep 5, 2024
32409cb
Adding converge and Caddyfile.expected examples
Sep 5, 2024
eac7853
Changes according to Thomas comments
Sep 9, 2024
a06df43
Replying 403 code and basic access forbidden content, instead of clos…
Sep 13, 2024
1b14fb7
Adapting Caddyfile.excepted to new 403 response
Sep 13, 2024
fba99e3
Removing wildcard in the ansible role. Wildcards will have to be adde…
Sep 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ N/A
If you only want to install Caddy, you don't need to set any variables. If you want to configure Caddy as a reverse proxy as well, you can provide an array of objects named `caddy_sites` with the following values:

* `additional_forwarding_ports`: Allows to define a list with additional ports where Caddy should listen for this domain and forward to HTTPS.
* `allowlist`: An array if IP addresses in CIDR-notation which are allowed to access this site (Optional). All other visitors receive a 404 error.
* `allowlist`: An array of IP addresses in CIDR-notation which are allowed to access this site (Optional). All other visitors receive a 404 error.
* `useragent_blocklist`: An array of User-Agents which are blocked to access this site (Optional).
* `certificate_file`: You can set this variable if you want to provide the certificate by yourself (Optional). The certificate needs permissions `0640`, with root as Owner and Caddy as Group.
* `certificate_key`: You can set this variable if you want to provide the certificate by yourself (Optional).
* `domain`: The domain caddy should listen to.
Expand Down
2 changes: 2 additions & 0 deletions molecule/reverse-proxy/converge.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
code: 401
allowlist:
- 8.8.8.8/32
useragent_blocklist:
- amazonbot
additional_forwarding_ports:
- '8080'
- '1337'
Expand Down
8 changes: 8 additions & 0 deletions molecule/reverse-proxy/files/Caddyfile.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@


example.com {
@badbots {
header User-Agent *amazonbot*
}

handle @badbots {
abort
}

@allowlist {
remote_ip 8.8.8.8/32
}
Expand Down
12 changes: 12 additions & 0 deletions templates/Caddyfile.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,18 @@

{% for site in caddy_sites %}
{{ site.domain }} {
{%- if site.useragent_blocklist is defined %}
@badbots {
{%- for ua in site.useragent_blocklist %}
header User-Agent *{{ ua }}*
{%- endfor %}
}

handle @badbots {
abort
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be better to send a HTTP header code 403 Forbidden (https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403) instead of just closing the connection?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I would say that if they were respecting us, we could be respecting them. But there are not. We want to be sure that they do not come back and do not try again.

Copy link
Collaborator

@tomcbe tomcbe Sep 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Artefactual sends 403 status code by default in their Nginx Role for AtoM:

https://github.com/artefactual-labs/ansible-nginx/blob/master/defaults/main/main.yml#L70

I still wonder, if sending a 403 is better than no answer to keep them away.

}
{%- endif %}

{%- if site.allowlist is defined %}
@allowlist {
remote_ip {% for ip in site.allowlist %} {{ ip }}{% endfor %}
Expand Down