Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(client): introduce extensible YAML config in Go #2306

Open
wants to merge 49 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
c615901
Initial Implementation
fortuna Nov 22, 2024
a4c894d
Add DialEndpoint
fortuna Nov 25, 2024
0d1b9ec
Wire new config
fortuna Nov 26, 2024
e81cb45
Add URL
fortuna Nov 26, 2024
455a2ad
Wire Endpoint
fortuna Nov 27, 2024
5a85e73
Cleanup
fortuna Nov 27, 2024
87eaa89
Tweaks
fortuna Dec 2, 2024
fbddc57
Merge branch 'master' into fortuna-ws-config
fortuna Dec 6, 2024
131e564
Progress
fortuna Dec 6, 2024
a26708c
Make it work
fortuna Dec 9, 2024
1f89c98
Simplify
fortuna Dec 10, 2024
0aa3dbc
More simplification
fortuna Dec 10, 2024
e3738e7
Merge branch 'master' into fortuna-ws-config
fortuna Dec 10, 2024
bc20623
Revert Go version
fortuna Dec 10, 2024
cde35c5
Progress
fortuna Dec 10, 2024
0128549
Fixes
fortuna Dec 10, 2024
e05542f
Fix tests
fortuna Dec 11, 2024
6c199bf
Fix
fortuna Dec 11, 2024
dbaf0a2
Add test
fortuna Dec 11, 2024
cffd70a
Add default provider
fortuna Dec 11, 2024
d46455e
Clean up
fortuna Dec 10, 2024
f63d2b9
Update tests
fortuna Dec 20, 2024
8df547f
Revamp config
fortuna Dec 24, 2024
52dcd84
Clean up and TODOs
fortuna Dec 24, 2024
5b80741
Naming
fortuna Dec 26, 2024
abcbb42
Clean up test
fortuna Dec 26, 2024
70a4a28
More tests
fortuna Dec 26, 2024
5ed3701
Better test
fortuna Dec 26, 2024
6141ff2
Add tcpudp
fortuna Dec 26, 2024
3962502
More tests
fortuna Dec 26, 2024
cacf818
Add Websocket support
fortuna Dec 26, 2024
2c59ed5
Fix
fortuna Dec 27, 2024
7304fdc
Lint fixes
fortuna Dec 27, 2024
b3f7dd6
It works!
fortuna Dec 27, 2024
5b0de18
Fix display
fortuna Dec 30, 2024
fa05c42
Try Coder
fortuna Jan 2, 2025
79c0a73
Comment
fortuna Jan 6, 2025
24f0544
Remove Websocket
fortuna Jan 6, 2025
777da9b
Merge branch 'master' into fortuna-ws-config
fortuna Jan 6, 2025
612d5db
Fix Android
fortuna Jan 6, 2025
ebb993f
Add net
fortuna Jan 6, 2025
01d95d2
Fix Electron
fortuna Jan 6, 2025
486aabb
Review changes
fortuna Jan 6, 2025
fdd54d5
Merge branch 'master' into fortuna-ws-config
fortuna Jan 6, 2025
5ab7ea9
fixes
fortuna Jan 6, 2025
d18deae
Fix Linux
fortuna Jan 6, 2025
b4a6e1f
Fixes
fortuna Jan 7, 2025
025f102
Tests
fortuna Jan 7, 2025
3334519
Lint
fortuna Jan 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions client/electron/go_helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@

import {pathToEmbeddedTun2socksBinary} from './app_paths';
import {ChildProcessHelper} from './process';
import {TransportConfigJson} from '../src/www/app/outline_server_repository/config';

/**
* Verifies the UDP connectivity of the server specified in `config`.
Expand All @@ -34,7 +33,7 @@ import {TransportConfigJson} from '../src/www/app/outline_server_repository/conf
* @throws ProcessTerminatedExitCodeError if tun2socks failed to run.
*/
export async function checkUDPConnectivity(
config: TransportConfigJson,
config: string,
debugMode: boolean = false
): Promise<boolean> {
const tun2socks = new ChildProcessHelper(pathToEmbeddedTun2socksBinary());
Expand Down
10 changes: 3 additions & 7 deletions client/electron/go_vpn_tunnel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ import {checkUDPConnectivity} from './go_helpers';
import {ChildProcessHelper, ProcessTerminatedSignalError} from './process';
import {RoutingDaemon} from './routing_service';
import {VpnTunnel} from './vpn_tunnel';
import {TransportConfigJson} from '../src/www/app/outline_server_repository/config';
import {TunnelStatus} from '../src/www/app/outline_server_repository/vpn';

const isLinux = platform() === 'linux';
Expand Down Expand Up @@ -64,7 +63,7 @@ export class GoVpnTunnel implements VpnTunnel {

constructor(
private readonly routing: RoutingDaemon,
readonly transportConfig: TransportConfigJson
readonly transportConfig: string
) {
this.tun2socks = new GoTun2socks();

Expand Down Expand Up @@ -248,10 +247,7 @@ class GoTun2socks {
* Otherwise, an error containing a JSON-formatted message will be thrown.
* @param isUdpEnabled Indicates whether the remote Outline server supports UDP.
*/
async start(
config: TransportConfigJson,
isUdpEnabled: boolean
): Promise<void> {
async start(transportConfig: string, isUdpEnabled: boolean): Promise<void> {
// ./tun2socks.exe \
// -tunName outline-tap0 -tunDNS 1.1.1.1,9.9.9.9 \
// -tunAddr 10.0.85.2 -tunGw 10.0.85.1 -tunMask 255.255.255.0 \
Expand All @@ -263,7 +259,7 @@ class GoTun2socks {
args.push('-tunGw', TUN2SOCKS_VIRTUAL_ROUTER_IP);
args.push('-tunMask', TUN2SOCKS_VIRTUAL_ROUTER_NETMASK);
args.push('-tunDNS', DNS_RESOLVERS.join(','));
args.push('-transport', JSON.stringify(config));
args.push('-transport', transportConfig);
args.push('-logLevel', this.process.isDebugModeEnabled ? 'debug' : 'info');
if (!isUdpEnabled) {
args.push('-dnsFallback');
Expand Down
12 changes: 7 additions & 5 deletions client/electron/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import * as path from 'path';
import * as process from 'process';
import * as url from 'url';

import * as net from '@outline/infrastructure/net';
import * as Sentry from '@sentry/electron/main';
import autoLaunch = require('auto-launch'); // tslint:disable-line
import {
Expand Down Expand Up @@ -352,17 +353,18 @@ async function createVpnTunnel(
// because startVpn will add a routing table entry that prefixed with this
// host (e.g. "<host>/32"), therefore <host> must be an IP address.
// TODO: make sure we resolve it in the native code
const host = tunnelConfig.firstHop.host;
const {host} = net.splitHostPort(tunnelConfig.firstHop);
if (!host) {
throw new errors.IllegalServerConfiguration('host is missing');
}
const hostIp = await lookupIp(host);
const routing = new RoutingDaemon(hostIp || '', isAutoConnect);
// Make sure the transport will use the IP we will allowlist.
const resolvedTransport =
config.setTransportConfigHost(tunnelConfig.transport, hostIp) ??
tunnelConfig.transport;
const tunnel = new GoVpnTunnel(routing, resolvedTransport);
// HACK: We do a simple string replacement in the config here. This may not always work with general configs
// but it works for simple configs.
// TODO: Remove the need to allowlisting the host IP.
tunnelConfig.transport = tunnelConfig.transport.replaceAll(host, hostIp);
const tunnel = new GoVpnTunnel(routing, tunnelConfig.transport);
routing.onNetworkChange = tunnel.networkChanged.bind(tunnel);
return tunnel;
}
Expand Down
82 changes: 28 additions & 54 deletions client/go/outline/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,21 +15,29 @@
package outline

import (
"fmt"
"context"
"net"

"github.com/Jigsaw-Code/outline-apps/client/go/outline/config"
"github.com/Jigsaw-Code/outline-apps/client/go/outline/platerrors"
"github.com/Jigsaw-Code/outline-sdk/transport"
"github.com/Jigsaw-Code/outline-sdk/transport/shadowsocks"
"github.com/eycorsican/go-tun2socks/common/log"
)

// Client provides a transparent container for [transport.StreamDialer] and [transport.PacketListener]
// that is exportable (as an opaque object) via gobind.
// It's used by the connectivity test and the tun2socks handlers.
// TODO: Rename to Transport. Needs to update per-platform code.
type Client struct {
transport.StreamDialer
transport.PacketListener
sd *config.Dialer[transport.StreamConn]
pl *config.PacketListener
}

func (c *Client) DialStream(ctx context.Context, address string) (transport.StreamConn, error) {
return c.sd.Dial(ctx, address)
}

func (c *Client) ListenPacket(ctx context.Context) (net.PacketConn, error) {
return c.pl.ListenPacket(ctx)
}

// NewClientResult represents the result of [NewClientAndReturnError].
Expand All @@ -42,67 +50,33 @@ type NewClientResult struct {

// NewClient creates a new Outline client from a configuration string.
func NewClient(transportConfig string) *NewClientResult {
client, err := newClientWithBaseDialers(transportConfig, net.Dialer{KeepAlive: -1}, net.Dialer{})
return &NewClientResult{
Client: client,
Error: platerrors.ToPlatformError(err),
}
}

func newClientWithBaseDialers(transportConfig string, tcpDialer, udpDialer net.Dialer) (*Client, error) {
conf, err := parseConfigFromJSON(transportConfig)
if err != nil {
return nil, err
}
prefixBytes, err := ParseConfigPrefixFromString(conf.Prefix)
tcpDialer := transport.TCPDialer{Dialer: net.Dialer{KeepAlive: -1}}
udpDialer := transport.UDPDialer{}
client, err := newClientWithBaseDialers(transportConfig, &tcpDialer, &udpDialer)
if err != nil {
return nil, err
return &NewClientResult{Error: platerrors.ToPlatformError(err)}
}

return newShadowsocksClient(conf.Host, int(conf.Port), conf.Method, conf.Password, prefixBytes, tcpDialer, udpDialer)
return &NewClientResult{Client: client}
}

func newShadowsocksClient(
host string, port int, cipherName, password string, prefix []byte, tcpDialer, udpDialer net.Dialer,
) (*Client, error) {
if err := validateConfig(host, port, cipherName, password); err != nil {
return nil, err
}

// TODO: consider using net.LookupIP to get a list of IPs, and add logic for optimal selection.
proxyAddress := net.JoinHostPort(host, fmt.Sprint(port))

cryptoKey, err := shadowsocks.NewEncryptionKey(cipherName, password)
func newClientWithBaseDialers(transportConfig string, tcpDialer transport.StreamDialer, udpDialer transport.PacketDialer) (*Client, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are changing the UDP handler from a PacketListener to a PacketDialer here, will this cause any unexpected errors?

transportYAML, err := config.ParseConfigYAML(transportConfig)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we still accepting the old JSON format here? Existing dynamic keys are still using JSON.

if err != nil {
return nil, newIllegalConfigErrorWithDetails("cipher&password pair is not valid",
"cipher|password", cipherName+"|"+password, "valid combination", err)
}

// We disable Keep-Alive as per https://datatracker.ietf.org/doc/html/rfc1122#page-101, which states that it should only be
// enabled in server applications. This prevents the device from unnecessarily waking up to send keep alives.
streamDialer, err := shadowsocks.NewStreamDialer(&transport.TCPEndpoint{Address: proxyAddress, Dialer: tcpDialer}, cryptoKey)
if err != nil {
return nil, platerrors.PlatformError{
Code: platerrors.SetupTrafficHandlerFailed,
Message: "failed to create TCP traffic handler",
Details: platerrors.ErrorDetails{"proxy-protocol": "shadowsocks", "handler": "tcp"},
return nil, &platerrors.PlatformError{
Code: platerrors.IllegalConfig,
Message: "config is not valid YAML",
Cause: platerrors.ToPlatformError(err),
}
}
if len(prefix) > 0 {
log.Debugf("Using salt prefix: %s", string(prefix))
streamDialer.SaltGenerator = shadowsocks.NewPrefixSaltGenerator(prefix)
}

packetListener, err := shadowsocks.NewPacketListener(&transport.UDPEndpoint{Address: proxyAddress, Dialer: udpDialer}, cryptoKey)
transportPair, err := config.NewDefaultTransportProvider(tcpDialer, udpDialer).Parse(context.Background(), transportYAML)
if err != nil {
return nil, platerrors.PlatformError{
Code: platerrors.SetupTrafficHandlerFailed,
Message: "failed to create UDP traffic handler",
Details: platerrors.ErrorDetails{"proxy-protocol": "shadowsocks", "handler": "udp"},
return nil, &platerrors.PlatformError{
Code: platerrors.IllegalConfig,
Message: "failed to create transport",
Cause: platerrors.ToPlatformError(err),
}
}

return &Client{StreamDialer: streamDialer, PacketListener: packetListener}, nil
return &Client{sd: transportPair.StreamDialer, pl: transportPair.PacketListener}, nil
}
Loading
Loading