Merge pull request #50 from sudosammy/dev

Version 3.4.0 push

.circleci/config.yml

@@ -0,0 +1,35 @@
+version: 2.1
+orbs:
+  coveralls: coveralls/[email protected]
+jobs:
+  build:
+    working_directory: ~/repo
+    docker:
+      - image: cimg/go:1.16-node
+      - image: cimg/go:1.17-node
+    parallelism: 2
+    steps:
+      - checkout
+      - restore_cache:
+          keys:
+            - go-mod-v4-{{ checksum "go.sum" }}
+      - run:
+          name: Install Dependencies
+          command: go get -u -t
+      - run:
+          name: Install gcov2lcov
+          command: go install github.com/jandelgado/gcov2lcov@latest
+      - save_cache:
+          key: go-mod-v4-{{ checksum "go.sum" }}
+          paths:
+            - "/go/pkg/mod"
+      - run:
+          name: Run tests
+          command: go test -v -coverprofile=coverage.out ./...
+      - run:
+          name: Convert Coverage Profile to Coveralls
+          command: mkdir coverage && gcov2lcov -infile=coverage.out -outfile=coverage/lcov.info
+      - coveralls/upload
+      - run:
+          name: Run Go Vet
+          command: go vet ./...

.gitignore

@@ -2,6 +2,8 @@
 .env
 dev
 knary.log
+allowed.txt
+denied.txt
 blacklist.txt
 zone.txt
 knary.exe

README.md

@@ -1,12 +1,10 @@
 # knary - A simple HTTP(S) and DNS Canary
 
-[![Build Status](https://travis-ci.org/sudosammy/knary.svg?branch=master)](https://travis-ci.org/sudosammy/knary)  [![Coverage Status](https://coveralls.io/repos/github/sudosammy/knary/badge.svg?branch=master)](https://coveralls.io/github/sudosammy/knary?branch=master)
+[![Build Status](https://circleci.com/gh/sudosammy/knary/tree/master.svg?style=svg)](https://circleci.com/gh/sudosammy/knary/tree/master)  [![Coverage Status](https://coveralls.io/repos/github/sudosammy/knary/badge.svg?branch=master)](https://coveralls.io/github/sudosammy/knary?branch=master)
 
 >Like "Canary" but more hipster, which means better 😎😎😎
 
-⚠️ **Note: Upgrading from version 2? You need to change your DNS setup to make use of new knary features. See step [#2 and #3](#setup) below. You may also want to configure [DNS_SUBDOMAIN](https://github.com/sudosammy/knary/tree/master/examples#likely-recommended-optional-configurations) to mimic how knary operated previously.** ⚠️
-
-knary is a canary token server that notifies a Slack/Discord/Teams/Lark channel (or other webhook) when incoming HTTP(S) or DNS requests match a given domain or any of its subdomains. It also supports functionality useful in offensive engagements including subdomain denylisting, working with Burp Collaborator, and easy TLS certificate creation.
+knary is a canary token server that notifies a Slack/Discord/Teams/Lark/Telegram channel (or other webhook) when incoming HTTP(S) or DNS requests match a given domain or any of its subdomains. It also supports functionality useful in offensive engagements including subdomain allow/denylisting, working with Burp Collaborator, and automatic TLS certificate creation with Let's Encrypt.
 
 ![knary canary-ing](https://github.com/sudosammy/knary/raw/master/screenshots/canary.gif "knary canary-ing")
 
@@ -22,50 +20,59 @@ Defenders also use canaries as tripwires that can alert them of an attacker with
 
 __Prerequisite:__ You need Go >=1.16 to build knary.
 ```
-go get -u github.com/sudosammy/knary
+go install github.com/sudosammy/knary@latest
 ```
 
-2. Set your chosen knary domain nameserver(s) to point to a subdomain under itself; such as `ns.knary.tld`. If required, set multiple nameserver records such as `ns1.knary.tld`, `ns2.knary.ltd`.
+**Important:** The specifics of how to perform the next two steps will depend on your domain registrar. Google `How to set Glue Record on <registrar name>` to get started. Ultimately, you need to configure your knary domain(s) to make use of itself as the nameserver (i.e. `ns1.knary.tld` and `ns2.knary.tld`) and configure Glue Records to point these nameservers back to your knary host. You may need to raise a support ticket to have this performed by your registrar. 
 
-3. Create a "Glue Record", sometimes referred to as "Nameserver Registration" or "Nameserver IP address" to point to your knary server. This is what it looks like in `name.com`:
+2. Set your chosen knary domain(s) nameserver(s) to point to a subdomain under itself; such as `ns.knary.tld`. If required, set multiple nameserver records such as `ns1` and `ns2`.
 
- ![Setting a glue record](https://github.com/sudosammy/knary/raw/master/screenshots/nameserver-ip.png "Setting a glue record")
+3. Create a "Glue Record" (sometimes referred to as "Nameserver Registration" or "Nameserver IP address") to point to your knary server. This is what it looks like in `name.com`:
 
-If your registry requires you to have multiple nameservers with different IP addresses, set the second nameserver to an IP address such as `8.8.8.8` or `1.1.1.1`. 
+ ![Setting a glue record](https://github.com/sudosammy/knary/raw/master/screenshots/nameserver-ip.png "Setting a glue record")
 
-**Note:** You may need to raise a support ticket to have step #2 and #3 performed by your registrar. 
+If your registry requires you to have multiple nameservers with **different** IP addresses, set the second nameserver to an IP address such as `8.8.8.8` or `1.1.1.1`. 
 
-4. This will take some time to propagate, so go setup your [webhook](#supported-webhook-configurations).
+4. This **will** take time to propagate, so go setup your [webhook(s)](#supported-webhook-configurations) while you wait. You can use [this tool](https://www.whatsmydns.net/#NS/) to check the propagation. Within a few hours you should see some DNS servers reflecting your knary domain as the nameserver.
 
 5. Create a `.env` file in the same directory as the knary binary and [configure](https://github.com/sudosammy/knary/tree/master/examples) it as necessary. You can also use environment variables to set these configurations. Environment variables will take precedence over the `.env` file.
 
-6. __Optional__ For accepting TLS (HTTPS) connections set the `LETS_ENCRYPT=<email address>` variable and knary will automagically manage wildcard certificates for you. Otherwise, you can specify the path to your own certificates with `TLS_CRT=<path>` and `TLS_KEY=<path>`.
+6. __Optional__ For accepting TLS (HTTPS) connections set the `LETS_ENCRYPT=<email address>` variable and knary will automagically manage wildcard certificates for you (see [OPSEC note](#opsec-notes) below). Otherwise, you can specify the path to your own certificates with `TLS_CRT=<path>` and `TLS_KEY=<path>`.
 
-7. Run the binary (probably in `screen`, `tmux`, or similar) and hope for output that looks something like this: 
+7. Run the binary (via the provided [Docker container](#knary-docker), or in `tmux` / `screen`) and hope for output that looks something like this: 
 
 ![knary go-ing](https://github.com/sudosammy/knary/raw/master/screenshots/run.png "knary go-ing")
 
-## Denying matches
-You **will** find systems that spam your knary even long after an engagement has ended. You will also find several DNS requests to mundane subdomains hitting your knary every day. To stop these from cluttering your notifications knary has two features:
+## Allowing or denying matches
+You **will** find systems that spam your knary even long after an engagement has ended. You will also find several DNS requests to mundane subdomains hitting your knary every day. To stop these from cluttering your notifications knary has a few features:
 
-1. A simple text-based denylist (location specified with `DENYLIST_FILE`). Add the offending subdomains or IP addresses separated by a newline (case-insensitive):
+1. A simple text-based deny and/or allowlist (location specified with `DENYLIST_FILE` and/or `ALLOWLIST_FILE`). Add the subdomains or IP addresses separated by a newline (case-insensitive):
 ```
 knary.tld
 www.knary.tld
 171.244.140.247
 test.dns.knary.tld
+sam.knary.tld
 ```
-This would stop knary from alerting on `www.knary.tld` but not `another.www.knary.tld`. **Note:** wildcards are not supported. An entry of `*.knary.tld` will match that string exactly.
+If this were a denylist, it would stop knary from alerting on `www.knary.tld` but not `another.www.knary.tld`.
 
-2. The `DNS_SUBDOMAIN` configuration allows you to specify that knary should only alert on DNS hits that are `*.<DNS_SUBDOMAIN>.knary.tld`.
+If this were an allowlist, knary would alert on exact matches (`sam.knary.tld`) and subdomain matches (`website1.sam.knary.tld`). Use `ALLOWLIST_STRICT=true` to prevent this fuzzy matching and only alert on hits to `sam.knary.tld`.
 
-A configuration of `DNS_SUBDOMAIN=dns` would stop knary from alerting on DNS hits to `blah.knary.tld` but not `blah.dns.knary.tld`. This configuration only affects DNS traffic. A HTTP request to `blah.knary.tld` would still notify you unless prevented by the denylist. Use a combination of both deny methods if you wish to prevent this.
+You can use both a deny and allowlist simultaneously. **Note:** wildcards in these files are not supported. An entry of `*.knary.tld` will match that string exactly.
+
+2. The `DNS_SUBDOMAIN` configuration allows you to specify a subdomain that knary must fuzzy match (i.e. `*.DNS_SUBDOMAIN.knary.tld`) before alerting on DNS hits. This configuration does not affect HTTP(S) requests and remains primarily to mimic legacy knary v2 functionality. **Consider using a deny/allowlist instead.**
+
+A configuration of `DNS_SUBDOMAIN=dns` would stop knary from alerting on DNS hits to `blah.knary.tld` but not `blah.dns.knary.tld`. A HTTP request to `blah.knary.tld` would still notify you unless prevented by an allow- or denylist.
 
 Sample configurations can be found [in the examples](https://github.com/sudosammy/knary/tree/master/examples) with common subdomains to deny.
 
 ## knary Docker
 Using knary in a container is as simple as creating your `.env` file (or setting environment variables in the `docker-compose.yaml` file) and running `sudo docker compose up -d`
 
+## OPSEC notes
+* Let's Encrypt will dox all the domains you are using with knary (and your `DNS_SUBDOMAIN` and `BURP_DOMAIN` if you are using those configurations). This is due to these domains being included in the SAN certificate generated for you. A remote adversary can read the certificate and extract the list of domains within it. To avoid this, don't configure `LETS_ENCRYPT`. You can use self-signed certificates with `TLS_CRT=<path>` and `TLS_KEY=<path>`; however, many hosts will refuse to connect reducing your visibility of incoming HTTPS connections.
+* With enough effort, knary is likely fingerprint-able by a remote host. i.e. it's plausible an adversary could determine you are running knary on a given host. This is because knary is not an RFC compliant nameserver (because doing so involves dark magic) and it likely behaves in an unusual / unique manner when compared to other nameservers.
+
 ## Supported Webhook Configurations
 These are environment variables / `.env` file configurations. You can configure none, one, or many. Most common usage would be to configure one. Refer to [the examples](https://github.com/sudosammy/knary/tree/master/examples) for usage help.
 
@@ -76,3 +83,5 @@ These are environment variables / `.env` file configurations. You can configure
 * `PUSHOVER_USER` The user token of the Pushover user you want knary to notify
 * `LARK_WEBHOOK` The full URL of the [webhook](https://www.feishu.cn/hc/en-US/articles/360024984973-Bot-Use-bots-in-groups) for the Lark/Feishu bot you want knary to notify
 * `LARK_SECRET` The [secret token](https://www.feishu.cn/hc/en-US/articles/360024984973-Bot-Use-bots-in-groups) used to sign messages to your Lark/Feishu bot
+* `TELEGRAM_CHATID` The [Telegram Bot](https://core.telegram.org/bots) chat ID you want knary to notify
+* `TELEGRAM_BOT_TOKEN` The Telegram Bot token

VERSION

@@ -1 +1 @@
-3.3.1
+3.4.0

docker-compose.yaml

@@ -10,7 +10,7 @@ services:
     #   - DNS=true
     #   - HTTP=true
     #   - BIND_ADDR=127.0.0.1
-    #   - CANARY_DOMAIN=knary.tld
+    #   - CANARY_DOMAIN=knary.tld,knary2.tld
     # (etc. etc.)
     build: .
     ports:

examples/README.md

@@ -1,28 +1,26 @@
 # Configuration Options & Example Files
 
 ## Sample Files
-* `default_env` - A recommended quick start configuration file with Let's Encrypt configuration.
-* `burp_env` - A recommended quick start configuration file if you are also using Burp Collaborator on the same server as knary.
-* `denylist.txt` - This is a good starting set of subdomains you should consider denying from notifying your webhook. Setting [DNS_SUBDOMAIN](#likely-recommended-optional-configurations) will cut down the noise to your knary too. Find & Replace `knary.tld` with your knary domain.
-* `zone_file.txt` - Although an uncommon configuration, this file demonstrates the proper format for configuring a Zone file for custom responses to DNS queries made to knary.
+* `default_env` - A recommended quick start configuration file with Let's Encrypt configuration
+* `burp_env` - A recommended quick start configuration file if you are also using Burp Collaborator on the same server as knary
+* `multidomain_env` - An example of how to specify multiple knary domains. Domains must be comma-delimited. Whitespace is stripped automatically
+* `allowlist.txt` - This is an example allowlist showing how knary could be used with your team's names. This has the added benefit of allowing your team to configure [keyword notifications](https://slack.com/intl/en-au/help/articles/201355156-Configure-your-Slack-notifications#keyword-notifications) in Slack
+* `denylist.txt` - If you are not going to use an allowlist, this is a good starting set of subdomains you should consider denying. Setting [DNS_SUBDOMAIN](#optional-configurations) will cut down the noise to your knary too. Find & Replace `knary.tld` with your knary domain
+* `zone_file.txt` - Although an uncommon configuration, this file demonstrates the proper format for configuring a Zone file for custom responses to DNS queries made to knary
 
 ## Minimum Necessary Configuration
 * `DNS` Enable/Disable the DNS canary (true/false)
 * `HTTP` Enable/Disable the HTTP canary (true/false)
 * `BIND_ADDR` The IP address you want knary to listen on. Example input: `0.0.0.0` to bind to all addresses available
-* `CANARY_DOMAIN` The domain + TLD to match canary hits on. Example input: `mycanary.com` (knary will match `*.mycanary.com`)
+* `CANARY_DOMAIN` The domain + TLD to match canary hits on. Example input: `mycanary.com` (knary will match `*.mycanary.com`). Multiple domains can be provided comma-delimited. Example input: `mycanary.com,knarytwo.zyz,knary3.io`
 * `*_WEBHOOK` One (or many) webhooks for knary to alert. Refer to the [webhook section in the README](https://github.com/sudosammy/knary#supported-webhook-configurations) for options
 
 ## Recommended Optional Configurations
 * `LETS_ENCRYPT` Enable Let's Encrypt management of your knary domain. If you do not configure this, or `TLS_*` as [detailed below](#optional-configurations), knary will only listen on port 80 and notify of HTTP hits. Example input: `[email protected]`
 * `LOG_FILE` Location for a file that knary will log greppable and timestamped warnings/errors. Example input: `/var/log/knary.log` or `knary.log` for current working directory
+* `ALLOWLIST_FILE` Location for a file containing case-insensitive subdomains or IP addresses (separated by newlines) that should trigger a notification for knary (unless also included in the denylist). Example input: `allowlist.txt`
 * `DENYLIST_FILE` Location for a file containing case-insensitive subdomains or IP addresses (separated by newlines) that should be ignored by knary and not logged or notified. Example input: `denylist.txt` 
 
-## Likely Recommended Optional Configurations
-* `DNS_SUBDOMAIN` Tell knary to only notify on `*.<DNS_SUBDOMAIN>.<CANARY_DOMAIN>` DNS hits. This is useful if you your webhook is getting too noisy with DNS hits to your knary TLD. Setting this configuration will mimic how knary operated prior to version 3. Example input: `dns`
-
-**Note:** If you have previously been running knary with Let's Encrypt and have now configured `DNS_SUBDOMAIN`, you should delete the files in the `certs/` folder so that knary can re-generate certificates that include this subdomain as a SAN. Otherwise knary may exhibit strange behaviour / failures when attempting to renew the certificate.
-
 ## Burp Collaborator Configuration
 If you are running Burp Collaborator on the same server as knary, you will need to configure the following.
 * `BURP_DOMAIN` The domain + TLD to match Collaborator hits on (e.g. `burp.knary.tld`).
@@ -31,12 +29,15 @@ If you are running Burp Collaborator on the same server as knary, you will need
 * `BURP_HTTPS_PORT` Much like the above - set to `8443` (or whatever you set the Burp HTTPS port to be)
 * `BURP_INT_IP` __Optional__ The internal IP address that Burp Collaborator is bound to. In most cases this will be `127.0.0.1` (which is the default); however, if you run knary in Docker you may need to set this to the Burp Collaborator IP address reachable from within the knary container
 
-**Note:** If you have previously been running knary with Let's Encrypt and have now configured Burp Collaborator, you should delete the files in the `certs/` folder so that knary can re-generate certificates that include your Burp Collaborator subdomain as a SAN. Otherwise knary may exhibit strange behaviour / failures when attempting to renew the certificate.
-
 ## Optional Configurations
-* `TLS_*` (CRT/KEY). If you're not using the `LETS_ENCRYPT` configuration use these environment variables to configure the location of your certificate and private key for accepting TLS (HTTPS) requests
+* `TLS_*` (CRT/KEY). If you're not using the `LETS_ENCRYPT` configuration use these environment variables to configure the location of your certificate and private key for accepting TLS (HTTPS) requests. Example input `TLS_KEY=certs/knary.key`
 * `DEBUG` Enable/Disable displaying incoming requests in the terminal and some additional info. Default disabled (true/false)
-* `LE_ENV` Set to `staging` to use the Let's Encrypt's staging environment. Useful if you are testing configurations with Let's Encrypt and do not want to hit the rate limit.
+* `ALLOWLIST_STRICT` Set to `true` to prevent fuzzy matching on allowlist items and only alert on exact matches
+* `LE_ENV` Set to `staging` to use the Let's Encrypt's staging environment. Useful if you are testing configurations with Let's Encrypt and do not want to hit the rate limit
 * `EXT_IP` The IP address the DNS canary will answer `A` questions with. By default knary will use the nameserver glue record. Setting this option will overrule that behaviour
 * `DENYLIST_ALERTING` By default knary will alert on items in the denylist that haven't triggered in >14 days. Set to `false` to disable this behaviour
+* `DNS_SUBDOMAIN` Tell knary to only notify on `*.<DNS_SUBDOMAIN>.<CANARY_DOMAIN>` DNS hits. This is useful if you your webhook is getting too noisy with DNS hits to your knary TLD and you do not maintain an allow or denylist. Setting this configuration will mimic how knary operated prior to version 3. Example input: `dns`
 * `ZONE_FILE` knary supports responding to DNS requests based on an RFC 1034/1035 compliant zone file. Example input: `zone_file.txt`
+
+## Note about editing configuration and Let's Encrypt
+If you have previously been running knary with Let's Encrypt and have now configured Burp Collaborator or `DNS_SUBDOMAIN`, you should delete the files in the `certs/` folder so that knary can re-generate certificates that include these subdomains as a SAN. Otherwise knary may exhibit strange behaviour / failures when attempting to renew the certificate.