-tiny can cause confusion when hiding panics entirely

[Ne0nd0g]

What version of Garble and Go are you using?

$ garble version

mvdan.cc/garble v0.7.2

Build settings:
       -compiler gc
     CGO_ENABLED 1
          GOARCH amd64
            GOOS linux
         GOAMD64 v1


$ go version

go version go1.19.3 linux/amd64

What environment are you running Garble on?

go env Output

$ go env
GO111MODULE=""

GOARCH="amd64"

GOBIN=""

GOCACHE="/home/rastley/.cache/go-build"

GOENV="/home/rastley/.config/go/env"

GOEXE=""

GOEXPERIMENT=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOINSECURE=""

GOMODCACHE="/home/rastley/go/pkg/mod"

GONOPROXY=""

GONOSUMDB=""

GOOS="linux"

GOPATH="/home/rastley/go"

GOPRIVATE=""

GOPROXY="https://proxy.golang.org,direct"

GOROOT="/usr/local/go"

GOSUMDB="sum.golang.org"

GOTMPDIR=""

GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"

GOVCS=""

GOVERSION="go1.19.3"

GCCGO="gccgo"

GOAMD64="v1"

AR="ar"

CC="gcc"

CXX="g++"

CGO_ENABLED="1"

GOMOD="/tmp/test/merlin-agent/go.mod"

GOWORK=""

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3883891253=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Set GOGARBLE=*

rastley@ubuntu:/tmp/test$ git clone https://github.com/Ne0nd0g/merlin-agent
Cloning into 'merlin-agent'...
remote: Enumerating objects: 873, done.
remote: Counting objects: 100% (245/245), done.
remote: Compressing objects: 100% (140/140), done.
remote: Total 873 (delta 125), reused 178 (delta 90), pack-reused 628
Receiving objects: 100% (873/873), 352.33 KiB | 3.67 MiB/s, done.
Resolving deltas: 100% (495/495), done.
rastley@ubuntu:/tmp/test$ cd merlin-agent/
rastley@ubuntu:/tmp/test/merlin-agent$ sed -i 's/GOGARBLE=golang.org,gopkg.in,github.com/GOGARBLE=*/g' Makefile 
rastley@ubuntu:/tmp/test/merlin-agent$ make linux-garble 
export GOGARBLE=*;export GOOS=linux GOARCH=amd64;garble -tiny -literals -seed d0d03a0ae4722535a0e1d5d0c8385ce42015511e68d960fadef4b4eaf5942feb build -trimpath -ldflags '-s -w -X "main.build=890d80f1e940c46a915e6a2d8abfd97b6655c78c" -X "github.com/Ne0nd0g/merlin-agent/agent.build=890d80f1e940c46a915e6a2d8abfd97b6655c78c" -X "main.protocol=h2" -X "main.url=https://127.0.0.1:443" -X "main.host=" -X "main.psk=merlin" -X "main.sleep=30s" -X "main.proxy=" -X "main.useragent=Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.85 Safari/537.36" -X "main.headers=" -X "main.skew=3000" -X "main.padding=4096" -X "main.killdate=0" -X "main.maxretry=7" -X "main.parrot=" -buildid=' -gcflags=all=-trimpath= -asmflags=all=-trimpath= -o bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64 ./main.go
rastley@ubuntu:/tmp/test/merlin-agent$ bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64 -h
rastley@ubuntu:/tmp/test/merlin-agent$ 

What did you expect to see?

Expected to see the agent's help menu printed to STDOUT. Set GOGARBLE=golang.org,gopkg.in,github.com

rastley@ubuntu:/tmp/test/merlin-agent$ make linux-garble 
export GOGARBLE=golang.org,gopkg.in,github.com;export GOOS=linux GOARCH=amd64;garble -tiny -literals -seed d0d03a0ae4722535a0e1d5d0c8385ce42015511e68d960fadef4b4eaf5942feb build -trimpath -ldflags '-s -w -X "main.build=890d80f1e940c46a915e6a2d8abfd97b6655c78c" -X "github.com/Ne0nd0g/merlin-agent/agent.build=890d80f1e940c46a915e6a2d8abfd97b6655c78c" -X "main.protocol=h2" -X "main.url=https://127.0.0.1:443" -X "main.host=" -X "main.psk=merlin" -X "main.sleep=30s" -X "main.proxy=" -X "main.useragent=Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.85 Safari/537.36" -X "main.headers=" -X "main.skew=3000" -X "main.padding=4096" -X "main.killdate=0" -X "main.maxretry=7" -X "main.parrot=" -buildid=' -gcflags=all=-trimpath= -asmflags=all=-trimpath= -o bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64 ./main.go
rastley@ubuntu:/tmp/test/merlin-agent$ bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64 -h
Merlin Agent
  -debug
    	Enable debug output
  -headers string
    	A new line separated (e.g., \n) list of additional HTTP headers to use
  -host string
    	HTTP Host header
  -ja3 string
    	JA3 signature string (not the MD5 hash). Overrides -proto & -parrot flags
  -killdate string
    	The date, as a Unix EPOCH timestamp, that the agent will quit running (default "0")
  -maxretry string
    	The maximum amount of failed checkins before the agent will quit running (default "7")
  -padding string
    	The maximum amount of data that will be randomly selected and appended to every message (default "4096")
  -parrot string
    	parrot or mimic a specific browser from github.com/refraction-networking/utls (e.g., HelloChrome_Auto
  -proto string
    	Protocol for the agent to connect with [https (HTTP/1.1), http (HTTP/1.1 Clear-Text), h2 (HTTP/2), h2c (HTTP/2 Clear-Text), http3 (QUIC or HTTP/3.0)] (default "h2")
  -proxy string
    	Hardcoded proxy to use for http/1.1 traffic only that will override host configuration
  -psk string
    	Pre-Shared Key used to encrypt initial communications (default "merlin")
  -skew string
    	Amount of skew, or variance, between agent checkins (default "3000")
  -sleep string
    	Time for agent to sleep (default "30s")
  -url string
    	Full URL for agent to connect to (default "https://127.0.0.1:443")
  -useragent string
    	The HTTP User-Agent header string that the Agent will use while sending traffic (default "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.85 Safari/537.36")
  -v	Enable verbose output
  -version
    	Print the agent version and exit

What did you see instead?

Nothing, the process seems to immediately exit. I even added a print line as the very first thing in main() and it isn't called.

astley@ubuntu:/tmp/test/merlin-agent$ make linux-garble 
export GOGARBLE=*;export GOOS=linux GOARCH=amd64;garble -tiny -literals -seed d0d03a0ae4722535a0e1d5d0c8385ce42015511e68d960fadef4b4eaf5942feb build -trimpath -ldflags '-s -w -X "main.build=890d80f1e940c46a915e6a2d8abfd97b6655c78c" -X "github.com/Ne0nd0g/merlin-agent/agent.build=890d80f1e940c46a915e6a2d8abfd97b6655c78c" -X "main.protocol=h2" -X "main.url=https://127.0.0.1:443" -X "main.host=" -X "main.psk=merlin" -X "main.sleep=30s" -X "main.proxy=" -X "main.useragent=Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.85 Safari/537.36" -X "main.headers=" -X "main.skew=3000" -X "main.padding=4096" -X "main.killdate=0" -X "main.maxretry=7" -X "main.parrot=" -buildid=' -gcflags=all=-trimpath= -asmflags=all=-trimpath= -o bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64 ./main.go
rastley@ubuntu:/tmp/test/merlin-agent$ bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64 -h
rastley@ubuntu:/tmp/test/merlin-agent$ 

[mvdan]

Can you reproduce this without make? What is the shortest GOGARBLE=* garble build command that seems to do nothing at all?

I just did your sed line followed by your make line, and the build appears to have worked - I now see bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64.

[mvdan]

For what it's worth, we use build caching, so it's possible that a garble build command would exit almost immediately - akin to a go build. When it's run twice in a row, the second time should do little work.

[Ne0nd0g]

What happens when you run bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64 -h? You should see output. The build doesn't fail, the resulting program fails to run.

[Ne0nd0g]

I set GOGARBLE to all of the domains in my go.mod file and the resulting program executes as expected

rastley@ubuntu:/tmp/test/merlin-agent$ export GOGARBLE=golang.org,gopkg.in,github.com,go.dedis.ch
rastley@ubuntu:/tmp/test/merlin-agent$ garble -tiny -literals -seed d0d03a0ae4722535a0e1d5d0c8385ce42015511e68d960fadef4b4eaf5942feb build -trimpath -ldflags '-s -w -X "main.build=890d80f1e940c46a915e6a2d8abfd97b6655c78c" -X "github.com/Ne0nd0g/merlin-agent/agent.build=890d80f1e940c46a915e6a2d8abfd97b6655c78c" -X "main.protocol=h2" -X "main.url=https://127.0.0.1:443" -X "main.host=" -X "main.psk=merlin" -X "main.sleep=30s" -X "main.proxy=" -X "main.useragent=Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.85 Safari/537.36" -X "main.headers=" -X "main.skew=3000" -X "main.padding=4096" -X "main.killdate=0" -X "main.maxretry=7" -X "main.parrot=" -buildid=' -gcflags=all=-trimpath= -asmflags=all=-trimpath= -o bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64 ./main.go
rastley@ubuntu:/tmp/test/merlin-agent$ bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64 -h
Merlin Agent
  -debug
    	Enable debug output
  -headers string
    	A new line separated (e.g., \n) list of additional HTTP headers to use
<SNIP>

[Ne0nd0g]

Interesting

rastley@ubuntu:/tmp/test/merlin-agent$ export GOGARBLE=*
rastley@ubuntu:/tmp/test/merlin-agent$ garble -seed d0d03a0ae4722535a0e1d5d0c8385ce42015511e68d960fadef4b4eaf5942feb build -o gTest ./main.go
rastley@ubuntu:/tmp/test/merlin-agent$ ./gTest -h
panic: qtls.ConnectionState doesn't match

goroutine 1 [running]:
NO0yOYBm.init.0()
	ITRuZ2Jj.go:1 +0x1be

[mvdan]

Oh, my bad, I misread your bug report.

I think the issue is -tiny. It makes panics print nothing, because in theory they could help with reverse engineering. And in theory they should not make correct programs behave any differently - if a program does not crash, it should be unaffected. And if it crashes, it's not like the user of an obfuscated program should be trying to investigate what happened.

In your case, if I remove the use of -tiny, I see:

$ ./bin/v1.6.0/890d80f1e940c46a915e6a2d8abfd97b6655c78c/merlinAgent-Linux-x64
panic: qtls.ConnectionState doesn't match

goroutine 1 [running]:
NO0yOYBmqAVmCZ.init.0()
	ITRuZ2Jj_WcvU.go:1 +0x2c6

The source of the problem is likely https://github.com/marten-seemann/qtls-go1-19 - since it mirrors types in the crypto/tls package, and we likely obfuscate those in different ways, then you get panics.

[mvdan]

That said, we should probably make panics still print something, because I agree this is very confusing. Perhaps we could make it print panic: nil or some other generic message, to at least tell the user that the program did panic, without leaking the real message or obfuscated stack trace. Then the garble user could at least try to reproduce without -tiny.

[mvdan]

cc @capnspacehook since you wrote most of -tiny

[mvdan]

Renamed this issue to be about the confusion with panics when -tiny is used. For qtls perhaps we can split into a new issue, as the two are unrelated.

[mvdan]

Dropping by to note that panic(nil) will now do something slightly different: golang/go#25448

I think that's fine for our use case here.