diff --git a/doc/podman.md b/doc/podman.md index 7e268ff..757e192 100644 --- a/doc/podman.md +++ b/doc/podman.md @@ -33,6 +33,8 @@ olegstepura](https://github.com/yrutschle/sslh/issues/448) # ... (make sure to mount config as you like) command: --transparent -F/etc/sslh/sslh.cfg # parameter --transparent here is needed to trigger configure_iptables in init script state: started + # ⚠️ Important: Never add a `network:` here, otherwise this setup will NOT work. + # Adding a network disconnects the sslh container from the pod's shared localhost. - name: "Create the caddy container" containers.podman.podman_container: @@ -95,26 +97,60 @@ protocols: ); ``` -I omitted caddy configs here as it's not important. Some unrelated container configs were also dropped. -In the example above `sslh`, `caddy` and `ssh-proxy` are 3 containers in the same pod, all listening on `localhost`. SSLH has to listen on `444` because caddy already listens on `443` and it's more complex to reconfigure caddy port because of let's encrypt (caddy itself "thinks" it is bound to your host interface). +I omitted Caddy configs here since they're irrelevant, and some unrelated container configs were also dropped for clarity. -Scheme of port mapping is (all containers share same `localhost`): +In the example above, `sslh`, `caddy`, and `ssh-proxy` are **three containers inside the same Podman pod**, all sharing the same **pod-local `localhost`**. + +`sslh` listens on port **444** instead of **443**, because `caddy` already binds to port `443`, and reconfiguring Caddy is more complex due to its Let's Encrypt integration (Caddy itself "thinks" it’s bound to your host interface). + +#### Port mapping scheme +All containers **share the same `localhost`** inside the pod: ``` host 443 → pod 444 (sslh) → pod 443 (caddy) ``` -- podman connects host `443` port to pod's `444` port -- `sslh` listens `444` on `localhost`, reroutes tls to `caddy` on `localhost:443` -- `caddy` listens `443` on `localhost` (reverse-proxies to other apps on private network) -Reverse-proxied services get correct IP in `X-Forwarded-For` header. -Takeaways: -- `net.ipv4.conf.default.route_localnet` is setup only in container, not on host, which is nice. -- same with iptables and route rules applied by `init` script in sslh container -- `sslh` proxy to other services in the private network will not work (because transparent mode is enabled) even if that other service does not need real IP - - with transparent mode all services that `sslh` will connect to should be attached to this pod making it listen on `localhost` of the pod - - ports of containers should not clash - - ports should be published on pod level, not on containers - - containers should not be configured to connect to custom networks -- because of the above proxying ssh to host will also not work out of the box, `sslh` should connect to `localhost` and not to a random IP. While adding another proxy as a container to the same pod sounds like an overkill I don't see any other solution, so `socat` is used as an additional proxy. -- `reverse_proxy` by caddy to other containers in the same custom network (as pod is attached to) works (e.g. caddy can connect to other IPs from custom network). `socat` can also connect to host and/or other containers in custom network. +- **Podman** maps the host’s `443` → pod’s `444`. +- **SSHL** listens on `localhost:444` inside the pod and forwards TLS traffic to `caddy` on `localhost:443`. +- **Caddy** listens on `localhost:443` and reverse-proxies to other services in the private network. + +This ensures that **reverse-proxied services** receive the correct client IP in the `X-Forwarded-For` header. + +--- + +### Important note + +⚠️ Do **not** set a `network:` parameter for the `sslh` container when it belongs to a Podman pod. +Setting a custom network **forces Podman to detach the container from the pod’s shared network namespace** and creates a **separate `localhost`** for that container, breaking transparent routing. + +For example: + +```yaml +command: --transparent -F/etc/sslh/sslh.cfg # parameter --transparent here is needed to trigger configure_iptables in init script +state: started +# ⚠️ Never add a `network:` here! +# Adding a network disconnects the sslh container from the pod's shared localhost, +# causing transparent routing to fail. +``` + +### Key takeaways + +- `net.ipv4.conf.default.route_localnet` and iptables rules are applied **inside the `sslh` container only**, not on the host. +- The `sslh` **init** script manages all required `iptables` and routing rules inside of the container. +- **Transparent mode restrictions**: + - Any service `sslh` proxies traffic to **must** be inside the same pod and bound to its **pod-local `localhost`**. + - Container ports must not conflict within the pod. + - Ports should be published **at the pod level**, **not per-container**. + - Containers inside this pod **must not** be attached to any custom networks. +- Because of transparent routing, `sslh` cannot directly proxy SSH to the host IP. + A workaround is to run a lightweight **socat** container in the same pod: + `TCP-LISTEN:222 → TCP:host.containers.internal:22` +- Caddy’s `reverse_proxy` **to other containers** in the **custom network attached to the pod** still works properly. +- `socat` can also connect to both the **host** and **other containers** inside the custom network without issues. + +### Tested setup + +- Confirmed working on **Debian Trixie** +- Using **Podman 5.4.2** +- Pinned `podman` and related dependencies (`conmon`, `slirp4netns`, `aardvark-dns`, etc.) from **Debian testing**, + since **stable** often ships outdated versions that can break this setup. diff --git a/hash.c b/hash.c index 5105254..e39f569 100644 --- a/hash.c +++ b/hash.c @@ -155,6 +155,10 @@ static int distance(int current_index, hash* h, hash_item item) return current_index - wanted_index + h->hash_size; } +int hash_insertable(hash* h) +{ + return (h->item_cnt < h->hash_size); +} int hash_insert(hash* h, hash_item new) { diff --git a/hash.h b/hash.h index 621790b..0a35d26 100644 --- a/hash.h +++ b/hash.h @@ -13,6 +13,7 @@ typedef int (*hash_cmp_item_fn)(hash_item item1, hash_item item2); hash* hash_init(int hash_size, hash_make_key_fn make_key, hash_cmp_item_fn cmp_item); +int hash_insertable(hash* h); /* Is it possible to insert: any space left? */ int hash_insert(hash* h, hash_item new); int hash_remove(hash* h, hash_item item); diff --git a/probe.c b/probe.c index ed07a1c..eeb16b2 100644 --- a/probe.c +++ b/probe.c @@ -162,8 +162,6 @@ static int is_openvpn_protocol (const char*p,ssize_t len, struct sslhcfg_protoco if (len < 1) return PROBE_NEXT; - printf("opcode: %d\n", (p[0] & OVPN_OPCODE_MASK) >> 3); - if ((p[0] & OVPN_OPCODE_MASK) != OVPN_CONTROL_HARD_RESET_CLIENT_V1 && (p[0] & OVPN_OPCODE_MASK) != OVPN_CONTROL_HARD_RESET_CLIENT_V2 && (p[0] & OVPN_OPCODE_MASK) != OVPN_CONTROL_HARD_RESET_CLIENT_V3 diff --git a/processes.c b/processes.c index 65be5bc..27a10d6 100644 --- a/processes.c +++ b/processes.c @@ -46,9 +46,9 @@ int tidy_connection(struct connection *cnx, struct loop_info* fd_info) watchers_del_read(fd_info->watchers, cnx->q[i].fd); watchers_del_write(fd_info->watchers, cnx->q[i].fd); close(cnx->q[i].fd); - if (cnx->q[i].deferred_data) - free(cnx->q[i].deferred_data); } + if (cnx->q[i].begin_deferred_data) + free(cnx->q[i].begin_deferred_data); } if (cnx->type == SOCK_DGRAM) @@ -152,6 +152,13 @@ void decrease_forked_connection(struct loop_info* loop, pid_t pid) { typedef struct pid2proto* hash_item; #include "hash.h" + +int has_space_to_fork(struct loop_info* fd_info) +{ + return hash_insertable(fd_info->pid2proto); +} + + void remember_child_data(struct loop_info* fd_info, struct connection* cnx, pid_t pid) { @@ -160,7 +167,10 @@ void remember_child_data(struct loop_info* fd_info, pid2proto->proto = cnx->proto; pid2proto->endpoint = cnx->endpoint; if (hash_insert(fd_info->pid2proto, pid2proto)) { - /* TODO something if it fails */ + print_message(msg_int_error, "hash_insert for pid %d failed\n", pid); + /* This should not happen as called is supposed to check that space is + * available in the hash */ + exit(1); } } @@ -168,8 +178,11 @@ static int max_forking_connections(void) { int max_cnx = 0; for (int i = 0; i < cfg.protocols_len; i++) { - if (cfg.protocols[i].fork) - max_cnx += cfg.protocols[i].max_connections; + if (cfg.protocols[i].fork) { + if (cfg.protocols[i].max_connections_is_present) + max_cnx += cfg.protocols[i].max_connections; + else max_cnx += 512; /* A magic number that's hopefully high enough */ + } } return max_cnx; } diff --git a/processes.h b/processes.h index 6eb50ea..20566cc 100644 --- a/processes.h +++ b/processes.h @@ -37,6 +37,7 @@ void loop_init(struct loop_info* loop, int num_addr_listen); void remember_child_data(struct loop_info* fd_info, struct connection* cnx, pid_t pid); void decrease_forked_connection(struct loop_info* loop, pid_t pid); +int has_space_to_fork(struct loop_info* loop); /* These must be declared in the loop handler, sslh-ev or sslh-select */ void watchers_add_read(watchers* w, int fd); diff --git a/sslh-select.c b/sslh-select.c index f43d28d..5893123 100644 --- a/sslh-select.c +++ b/sslh-select.c @@ -131,6 +131,7 @@ static void sigchld_process(struct loop_info* loop) received_sigchld = 0; do { chld = waitpid(-1, NULL, WNOHANG); + if ((chld == -1) && (errno == ECHILD)) return; CHECK_RES_RETURN(chld, "waitpid", ); if (chld) { decrease_forked_connection(loop, chld); @@ -221,8 +222,9 @@ void main_loop(struct listen_endpoint listen_sockets[], int num_addr_listen) fd_info.watchers->max_fd, fd_info.num_probing); res = select(fd_info.watchers->max_fd, &readfds, &writefds, NULL, fd_info.num_probing ? &tv : NULL); - if (res < 0) - perror("select"); + if ((res == -1) && ((errno != EAGAIN) && (errno != EINTR))) + print_message(msg_system_error, "%s:%d:%d:%s\n", + __FILE__, __LINE__, errno, strerror(errno)); \ /* Check main socket for new connections */ for (i = 0; i < num_addr_listen; i++) { diff --git a/tcp-listener.c b/tcp-listener.c index 7572e62..4840a58 100644 --- a/tcp-listener.c +++ b/tcp-listener.c @@ -165,6 +165,7 @@ struct connection* accept_new_connection(struct listen_endpoint* endpoint, struc print_message(msg_fd, "accepting from %d\n", listen_socket); in_socket = accept(listen_socket, 0, 0); + if ((in_socket == -1) && (errno == EAGAIN)) return NULL; /* Do not log if we're just retring next iteration */ CHECK_RES_RETURN(in_socket, "accept", NULL); res = set_nonblock(in_socket); @@ -298,6 +299,12 @@ static void connect_proxy(struct connection *cnx) /* Forks a shoveler process for one protocol */ void fork_shoveling_process(struct loop_info* fd_info, struct connection* cnx) { pid_t pid; + + if (!has_space_to_fork(fd_info)) { + print_message(msg_connections_error, "forking for %s: out of hash space\n", cnx->proto->name); + return; + } + switch (pid = fork()) { case 0: /* child */ /* TODO: close all file descriptors except 2 */ @@ -311,11 +318,6 @@ void fork_shoveling_process(struct loop_info* fd_info, struct connection* cnx) { watcher_sigchld(fd_info, cnx, pid); break; } - /* Free file descriptor (used only in child), but do not reduce connection - * counts */ - cnx->proto = NULL; - cnx->endpoint = NULL; - tidy_connection(cnx, fd_info); } /* Process read activity on a socket in probe state @@ -349,6 +351,11 @@ void probing_read_process(struct connection* cnx, tidy_connection(cnx, fd_info); } else if (cnx->proto->fork) { fork_shoveling_process(fd_info, cnx); + /* Free file descriptor (used only in child), but do not reduce connection + * counts */ + cnx->proto = NULL; + cnx->endpoint = NULL; + tidy_connection(cnx, fd_info); } else { connect_queue(cnx, fd_info); }