From 3fb1201b3fbd7a3f8ba743a002b6d29c424abc0f Mon Sep 17 00:00:00 2001 From: yrutschle Date: Sun, 26 Sep 2021 16:28:06 +0200 Subject: [PATCH] merged transparent proxy setups together, clarifying what is not known to work --- doc/config.md | 156 ++--------------------------------------- doc/tproxy.md | 187 +++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 182 insertions(+), 161 deletions(-) diff --git a/doc/config.md b/doc/config.md index ad1c903..42d609f 100644 --- a/doc/config.md +++ b/doc/config.md @@ -99,158 +99,12 @@ Then you can run sslh-select as an unpriviledged user, e.g.: Transparent proxy support ------------------------- -On Linux and FreeBSD you can use the `--transparent` option to -request transparent proxying. This means services behind `sslh` -(Apache, `sshd` and so on) will see the external IP and ports -as if the external world connected directly to them. This -simplifies IP-based access control (or makes it possible at -all). +Transparent proxying allows the target server to see the +original client IP address, i.e. `sslh` becomes invisible. +This makes it easier to use the server's logs, and potential +IP-based banning ability. -You can refer to Sean Warn'ѕ [tutorial](tproxy.md) for a -different set-up which enables transparent proxying between -two different machines. The following may only work if -`sslh` and the final servers are on the same machine. - -Note that getting this to work is very tricky and -detail-dependant: depending on whether the target server and -sslh are on the same machine, different machines, or -different dockers, and tool versions, all seem to change the -required network configuration somewhat. If it doesn't work, -it's almost certain that the problem is not linked to `sslh` -but to the network setup that surrounds it. - -Linux: - -`sslh` needs extended rights to perform this: you'll need to -give it `CAP_NET_RAW` capabilities (see appropriate chapter) -or run it as root (but don't do that). - -The firewalling tables also need to be adjusted as follows. -I don't think it is possible to have `httpd` and `sslh` both listen to 443 in -this scheme -- let me know if you manage that: - - # Set route_localnet = 1 on all interfaces so that ssl can use "localhost" as destination - sysctl -w net.ipv4.conf.default.route_localnet=1 - sysctl -w net.ipv4.conf.all.route_localnet=1 - - # DROP martian packets as they would have been if route_localnet was zero - # Note: packets not leaving the server aren't affected by this, thus sslh will still work - iptables -t raw -A PREROUTING ! -i lo -d 127.0.0.0/8 -j DROP - iptables -t mangle -A POSTROUTING ! -o lo -s 127.0.0.0/8 -j DROP - - # Mark all connections made by ssl for special treatment (here sslh is run as user "sslh") - iptables -t nat -A OUTPUT -m owner --uid-owner sslh -p tcp --tcp-flags FIN,SYN,RST,ACK SYN -j CONNMARK --set-xmark 0x01/0x0f - - # Outgoing packets that should go to sslh instead have to be rerouted, so mark them accordingly (copying over the connection mark) - iptables -t mangle -A OUTPUT ! -o lo -p tcp -m connmark --mark 0x01/0x0f -j CONNMARK --restore-mark --mask 0x0f - - # Configure routing for those marked packets - ip rule add fwmark 0x1 lookup 100 - ip route add local 0.0.0.0/0 dev lo table 100 - -Tranparent proxying with IPv6 is similarly set up as follows: - - # Set route_localnet = 1 on all interfaces so that ssl can use "localhost" as destination - # Not sure if this is needed for ipv6 though - sysctl -w net.ipv4.conf.default.route_localnet=1 - sysctl -w net.ipv4.conf.all.route_localnet=1 - - # DROP martian packets as they would have been if route_localnet was zero - # Note: packets not leaving the server aren't affected by this, thus sslh will still work - ip6tables -t raw -A PREROUTING ! -i lo -d ::1/128 -j DROP - ip6tables -t mangle -A POSTROUTING ! -o lo -s ::1/128 -j DROP - - # Mark all connections made by ssl for special treatment (here sslh is run as user "sslh") - ip6tables -t nat -A OUTPUT -m owner --uid-owner sslh -p tcp --tcp-flags FIN,SYN,RST,ACK SYN -j CONNMARK --set-xmark 0x01/0x0f - - # Outgoing packets that should go to sslh instead have to be rerouted, so mark them accordingly (copying over the connection mark) - ip6tables -t mangle -A OUTPUT ! -o lo -p tcp -m connmark --mark 0x01/0x0f -j CONNMARK --restore-mark --mask 0x0f - - # Configure routing for those marked packets - ip -6 rule add fwmark 0x1 lookup 100 - ip -6 route add local ::/0 dev lo table 100 - -Explanation: -To be able to use `localhost` as destination in your sslh config along with transparent proxying -you have to allow routing of loopback addresses as done above. -This is something you usually should not do (see [this stackoverflow post](https://serverfault.com/questions/656279/how-to-force-linux-to-accept-packet-with-loopback-ip/656484#656484)) -The two `DROP` iptables rules emulate the behaviour of `route_localnet` set to off (with one small difference: -allowing the reroute-check to happen after the fwmark is set on packets destined for sslh). -See [this diagram](https://upload.wikimedia.org/wikipedia/commons/3/37/Netfilter-packet-flow.svg) for a good visualisation -showing how packets will traverse the iptables chains. - -Note: -You have to run `sslh` as dedicated user (in this example the user is also named `sslh`), to not mess up with your normal networking. -These rules will allow you to connect directly to ssh on port -22 (or to any other service behind sslh) as well as through sslh on port 443. - -Also remember that iptables configuration and ip routes and -rules won't be necessarily persisted after you reboot. Make -sure to save them properly. For example in CentOS7, you would -do `iptables-save > /etc/sysconfig/iptables`, and add both -`ip` commands to your `/etc/rc.local`. - -FreeBSD: - -Given you have no firewall defined yet, you can use the following configuration -to have ipfw properly redirect traffic back to sslh - - /etc/rc.conf - firewall_enable="YES" - firewall_type="open" - firewall_logif="YES" - firewall_coscripts="/etc/ipfw/sslh.rules" - - -/etc/ipfw/sslh.rules - - #! /bin/sh - - # ssl - ipfw add 20000 fwd 192.0.2.1,443 log tcp from 192.0.2.1 8443 to any out - ipfw add 20010 fwd 2001:db8::1,443 log tcp from 2001:db8::1 8443 to any out - - # ssh - ipfw add 20100 fwd 192.0.2.1,443 log tcp from 192.0.2.1 8022 to any out - ipfw add 20110 fwd 2001:db8::1,443 log tcp from 2001:db8::1 8022 to any out - - # xmpp - ipfw add 20200 fwd 192.0.2.1,443 log tcp from 192.0.2.1 5222 to any out - ipfw add 20210 fwd 2001:db8::1,443 log tcp from 2001:db8::1 5222 to any out - - # openvpn (running on other internal system) - ipfw add 20300 fwd 192.0.2.1,443 log tcp from 198.51.100.7 1194 to any out - ipfw add 20310 fwd 2001:db8::1,443 log tcp from 2001:db8:1::7 1194 to any out - -General notes: - - -This will only work if `sslh` does not use any loopback -addresses (no `127.0.0.1` or `localhost`), you'll need to use -explicit IP addresses (or names): - - sslh --listen 192.168.0.1:443 --ssh 192.168.0.1:22 --tls 192.168.0.1:4443 - -This will not work: - - sslh --listen 192.168.0.1:443 --ssh 127.0.0.1:22 --tls 127.0.0.1:4443 - -Transparent proxying means the target server sees the real -origin address, so it means if the client connects using -IPv6, the server must also support IPv6. It is easy to -support both IPv4 and IPv6 by configuring the server -accordingly, and setting `sslh` to connect to a name that -resolves to both IPv4 and IPv6, e.g.: - - sslh --transparent --listen :443 --ssh insideaddr:22 - - /etc/hosts: - 192.168.0.1 insideaddr - 201::::2 insideaddr - -Upon incoming IPv6 connection, `sslh` will first try to -connect to the IPv4 address (which will fail), then connect -to the IPv6 address. +Set up can get complicated, so it has it's own [document](tproxy.md). Systemd Socket Activation ------------------------- diff --git a/doc/tproxy.md b/doc/tproxy.md index 249b077..0a91773 100644 --- a/doc/tproxy.md +++ b/doc/tproxy.md @@ -1,14 +1,181 @@ -# Transparent Proxy to Two Hosts +# Transparent proxy + +On Linux and FreeBSD you can use the `--transparent` option to +request transparent proxying. This means services behind `sslh` +(Apache, `sshd` and so on) will see the external IP and ports +as if the external world connected directly to them. This +simplifies IP-based access control (or makes it possible at +all). + +This document shows recipes that may help to do that. + +Note that getting this to work is very tricky and +detail-dependant: depending on whether the target server and +sslh are on the same machine, different machines, or +different dockers, and tool versions, all seem to change the +required network configuration somewhat. If it doesn't work, +it's almost certain that the problem is not linked to `sslh` +but to the network setup that surrounds it. If in trouble, +it might be worth trying to set up the network rules +with a simpler server than `sslh`, such as +[`socat`](http://www.dest-unreach.org/socat/) + + +Users have tried to do at least the following: + + * `sslh` and the target servers run on the same host (see [below](#transparent-proxy-to-one-host)) + * `sslh` runs on a host, and the target servers run on LXC or dockers running on the same host. No known working setup. + * `sslh` runs on a host, and the target servers run on different hosts on the same local network(see [below](#transparent-proxy-to-two-hosts)) + * `sslh` runs on a host, and the target servers run on a different host on a different network (there is a [usecase](https://github.com/yrutschle/sslh/issues/295) for this). No known working setup, and it's unclear it is possible. + + +## Transparent proxy to one host + +### Linux + +`sslh` needs extended rights to perform this: you'll need to +give it `CAP_NET_RAW` capabilities (see appropriate chapter) +or run it as root (but don't do that). + +The firewalling tables also need to be adjusted as follows. +I don't think it is possible to have `httpd` and `sslh` both listen to 443 in +this scheme -- let me know if you manage that: + + # Set route_localnet = 1 on all interfaces so that ssl can use "localhost" as destination + sysctl -w net.ipv4.conf.default.route_localnet=1 + sysctl -w net.ipv4.conf.all.route_localnet=1 + + # DROP martian packets as they would have been if route_localnet was zero + # Note: packets not leaving the server aren't affected by this, thus sslh will still work + iptables -t raw -A PREROUTING ! -i lo -d 127.0.0.0/8 -j DROP + iptables -t mangle -A POSTROUTING ! -o lo -s 127.0.0.0/8 -j DROP + + # Mark all connections made by ssl for special treatment (here sslh is run as user "sslh") + iptables -t nat -A OUTPUT -m owner --uid-owner sslh -p tcp --tcp-flags FIN,SYN,RST,ACK SYN -j CONNMARK --set-xmark 0x01/0x0f + + # Outgoing packets that should go to sslh instead have to be rerouted, so mark them accordingly (copying over the connection mark) + iptables -t mangle -A OUTPUT ! -o lo -p tcp -m connmark --mark 0x01/0x0f -j CONNMARK --restore-mark --mask 0x0f + + # Configure routing for those marked packets + ip rule add fwmark 0x1 lookup 100 + ip route add local 0.0.0.0/0 dev lo table 100 + +Tranparent proxying with IPv6 is similarly set up as follows: + + # Set route_localnet = 1 on all interfaces so that ssl can use "localhost" as destination + # Not sure if this is needed for ipv6 though + sysctl -w net.ipv4.conf.default.route_localnet=1 + sysctl -w net.ipv4.conf.all.route_localnet=1 + + # DROP martian packets as they would have been if route_localnet was zero + # Note: packets not leaving the server aren't affected by this, thus sslh will still work + ip6tables -t raw -A PREROUTING ! -i lo -d ::1/128 -j DROP + ip6tables -t mangle -A POSTROUTING ! -o lo -s ::1/128 -j DROP + + # Mark all connections made by ssl for special treatment (here sslh is run as user "sslh") + ip6tables -t nat -A OUTPUT -m owner --uid-owner sslh -p tcp --tcp-flags FIN,SYN,RST,ACK SYN -j CONNMARK --set-xmark 0x01/0x0f + + # Outgoing packets that should go to sslh instead have to be rerouted, so mark them accordingly (copying over the connection mark) + ip6tables -t mangle -A OUTPUT ! -o lo -p tcp -m connmark --mark 0x01/0x0f -j CONNMARK --restore-mark --mask 0x0f + + # Configure routing for those marked packets + ip -6 rule add fwmark 0x1 lookup 100 + ip -6 route add local ::/0 dev lo table 100 + +Explanation: +To be able to use `localhost` as destination in your sslh config along with transparent proxying +you have to allow routing of loopback addresses as done above. +This is something you usually should not do (see [this stackoverflow post](https://serverfault.com/questions/656279/how-to-force-linux-to-accept-packet-with-loopback-ip/656484#656484)) +The two `DROP` iptables rules emulate the behaviour of `route_localnet` set to off (with one small difference: +allowing the reroute-check to happen after the fwmark is set on packets destined for sslh). +See [this diagram](https://upload.wikimedia.org/wikipedia/commons/3/37/Netfilter-packet-flow.svg) for a good visualisation +showing how packets will traverse the iptables chains. + +Note: +You have to run `sslh` as dedicated user (in this example the user is also named `sslh`), to not mess up with your normal networking. +These rules will allow you to connect directly to ssh on port +22 (or to any other service behind sslh) as well as through sslh on port 443. + +Also remember that iptables configuration and ip routes and +rules won't be necessarily persisted after you reboot. Make +sure to save them properly. For example in CentOS7, you would +do `iptables-save > /etc/sysconfig/iptables`, and add both +`ip` commands to your `/etc/rc.local`. + +### FreeBSD + +Given you have no firewall defined yet, you can use the following configuration +to have ipfw properly redirect traffic back to sslh + + /etc/rc.conf + firewall_enable="YES" + firewall_type="open" + firewall_logif="YES" + firewall_coscripts="/etc/ipfw/sslh.rules" + + +/etc/ipfw/sslh.rules + + #! /bin/sh + + # ssl + ipfw add 20000 fwd 192.0.2.1,443 log tcp from 192.0.2.1 8443 to any out + ipfw add 20010 fwd 2001:db8::1,443 log tcp from 2001:db8::1 8443 to any out + + # ssh + ipfw add 20100 fwd 192.0.2.1,443 log tcp from 192.0.2.1 8022 to any out + ipfw add 20110 fwd 2001:db8::1,443 log tcp from 2001:db8::1 8022 to any out + + # xmpp + ipfw add 20200 fwd 192.0.2.1,443 log tcp from 192.0.2.1 5222 to any out + ipfw add 20210 fwd 2001:db8::1,443 log tcp from 2001:db8::1 5222 to any out + + # openvpn (running on other internal system) + ipfw add 20300 fwd 192.0.2.1,443 log tcp from 198.51.100.7 1194 to any out + ipfw add 20310 fwd 2001:db8::1,443 log tcp from 2001:db8:1::7 1194 to any out + +General notes: + + +This will only work if `sslh` does not use any loopback +addresses (no `127.0.0.1` or `localhost`), you'll need to use +explicit IP addresses (or names): + + sslh --listen 192.168.0.1:443 --ssh 192.168.0.1:22 --tls 192.168.0.1:4443 + +This will not work: + + sslh --listen 192.168.0.1:443 --ssh 127.0.0.1:22 --tls 127.0.0.1:4443 + +Transparent proxying means the target server sees the real +origin address, so it means if the client connects using +IPv6, the server must also support IPv6. It is easy to +support both IPv4 and IPv6 by configuring the server +accordingly, and setting `sslh` to connect to a name that +resolves to both IPv4 and IPv6, e.g.: + + sslh --transparent --listen :443 --ssh insideaddr:22 + + /etc/hosts: + 192.168.0.1 insideaddr + 201::::2 insideaddr + +Upon incoming IPv6 connection, `sslh` will first try to +connect to the IPv4 address (which will fail), then connect +to the IPv6 address. + + +## Transparent Proxy to Two Hosts Tutorial by Sean Warner. 19 June 2019 20:35 -## Aim +### Aim * Show that `sslh` can transparently proxy requests from the internet to services on two separate hosts that are both on the same LAN. * The IP address of the client initiating the request is what the destination should see… and not the IP address of the host that `sslh` is running on, which is what happens when `sslh` is not running in transparent mode. * The solution here only works for my very specific use-case but hopefully others can adapt it to suits their needs. -## Overview of my Network +### Overview of my Network Two Raspberry Pis on my home LAN: * Pi A: 192.168.1.124 – `sslh` (Port 4433), Apache2 web server for https (port 443), `stunnel` (port 4480) to decrypt ssh traffic and forward to SSH server (also on Pi A at Port 1022) @@ -17,7 +184,7 @@ Two Raspberry Pis on my home LAN: ![Architecture](tproxy.svg) -## `sslh` build +### `sslh` build   `sslh` Version: sslh v1.19c-2-gf451cc8-dirty. @@ -43,7 +210,7 @@ MAN=sslh.8.gz         # man page name # itself ```   -## systemd setup +### systemd setup Create an sslh systemd service file... ``` @@ -78,7 +245,7 @@ Start it again to test… # systemctl start sslh ```   -## Configure `sslh` +### Configure `sslh` First stop `sslh` then open the config file and replace with below, save and start `sslh` again ``` @@ -117,7 +284,7 @@ protocols: ); ```   -## Configure `stunnel` +### Configure `stunnel` First stop `stunnel` then open the config file and replace with below, save and start `stunnel` again ``` @@ -144,7 +311,7 @@ connect = 192.168.1.124:1022 TIMEOUTclose = 0 ```   -## Configure iptables for Pi A +### Configure iptables for Pi A The `_add.sh` script creates the rules, the `_rm.sh` script removes the rules. They will be lost if you reboot but there are ways to make them load again on start-up.. @@ -186,7 +353,7 @@ Now run the "add" script on Pi A! # piA_tproxy_rm.sh ``` -# Configure iptables for Pi B +## Configure iptables for Pi B ``` # nano /usr/local/sbin/piB_tproxy_add.sh @@ -226,7 +393,7 @@ Now run the "add" script on Pi B! # piB_tproxy_rm.sh ```   -## Testing +### Testing * Getting to sshd on PiA