HAproxy Reverse Proxy Setup

[awptechnologies]

Reverse proxy is supported from what i can tell in the research i have done. i am having troubles setting up a relay server that is accessible from outside my network through the reverse proxy.

[eltorio]

@awptechnologies
I tried to host hbbs/hbbr and a minimalist admin server in a Kubernetes cluster.
It uses Haproxy 2.7 as the ingress proxy and a modified Haproxy-ingress-controler 1.10 as the Haproxy controler.
First issue is that Haproxy cannot handle udp -except for quic- and it probably never supports it.
Second issue hbbs will see the IP address of Haproxy and shows a log like this one:

2024-03-26T07:41:22.860694378Z [2024-03-26 07:41:22.860521 +00:00] DEBUG [src/rendezvous_server.rs:1097] Tcp connection from [::ffff:172.28.5.218]:58786, ws: false
2024-03-26T07:41:22.861155579Z [2024-03-26 07:41:22.861075 +00:00] DEBUG [src/rendezvous_server.rs:1137] Tcp connection from [::ffff:172.28.5.218]:58786 closed

but this is the generated haproxy config:

frontend https
  mode http
  bind 0.0.0.0:443 name v4 crt /etc/haproxy-ingress/certs/frontend ssl alpn h2,http/1.1
  bind [::]:443 name v6 crt /etc/haproxy-ingress/certs/frontend ssl alpn h2,http/1.1
  http-request set-var(txn.base) base
  http-request set-var(txn.path) path
  http-request set-var(txn.host) req.hdr(Host),field(1,:),lower
  http-request set-var(txn.host_match) var(txn.host),map(/etc/haproxy-ingress/maps/host.map)
  http-request set-var(txn.host_match) var(txn.host),regsub(^[^.]*,,),map(/etc/haproxy-ingress/maps/host.map,'') if !{ var(txn.host_match) -m found }
  http-request set-var(txn.path_match) var(txn.host_match),concat(,txn.path,),map(/etc/haproxy-ingress/maps/path-exact.map)
  http-request set-var(txn.path_match) var(txn.host_match),concat(,txn.path,),map_beg(/etc/haproxy-ingress/maps/path-prefix.map) if !{ var(txn.path_match) -m found }
  http-request auth realm Protected-Content if { var(txn.path_match) -m dom 706d41ac94352467187ae04f79c5413b } !{ http_auth_group(briac-code-server-hcf-coder-ingress-coder-briac-cf) authenticated-users }
  http-request auth realm Protected-Content if { var(txn.path_match) -m dom 1a8a05768edae0436e25b1e49cc824c2 } !{ http_auth_group(longhorn-system-longhorn-ui) authenticated-users }
  http-request auth realm Protected-Content if { var(txn.path_match) -m dom 67785cfd7a61cf6cf21ad6ac536622cd } !{ http_auth_group(monitoring-prometheus-kube-prometheus-prometheus) authenticated-users }
  http-request set-header X-Forwarded-Proto https
  use_backend %[var(txn.path_match),field(1,.)]
  default_backend _default-local-service_http

frontend sctgdesk-hbbr-hbbs-gateway-port-21115
  mode tcp
  bind 0.0.0.0:21115 name v4
  bind [::]:21115 name v6
  option tcplog
  default_backend sctgdesk_port-21115

frontend sctgdesk-hbbr-hbbs-gateway-port-21116
  mode tcp
  bind 0.0.0.0:21116 name v4
  bind [::]:21116 name v6
  option tcplog
  default_backend sctgdesk_port-21116

frontend sctgdesk-hbbr-hbbs-gateway-port-21117
  mode tcp
  bind 0.0.0.0:21117 name v4
  bind [::]:21117 name v6
  option tcplog
  default_backend sctgdesk_port-21117

frontend sctgdesk-hbbr-hbbs-gateway-port-21118
  mode tcp
  bind 0.0.0.0:21118 name v4
  bind [::]:21118 name v6
  option tcplog
  default_backend sctgdesk_port-21118

frontend sctgdesk-hbbr-hbbs-gateway-port-21119
  mode tcp
  bind 0.0.0.0:21119 name v4
  bind [::]:21119 name v6
  option tcplog
  default_backend sctgdesk_port-21119

backend sctgdesk_port-21116
  mode tcp
  default-server check
  server SRV_1 172.28.2.86:21116 enabled

backend sctgdesk_port-21117
  mode tcp
  default-server check
  server SRV_1 172.28.2.86:21117 enabled

backend sctgdesk_port-21118
  mode tcp
  default-server check
  server SRV_1 172.28.2.86:21118 enabled

backend sctgdesk_port-21119
  mode tcp
  default-server check
  server SRV_1 172.28.2.86:21119 enabled

backend sctgdesk_sctgdesk-api-server-service_port-21114
  mode http
  balance roundrobin
  option forwardfor
  no option abortonclose
  default-server check
  server SRV_1 172.28.5.91:21114 enabled

I also added a iptables rule for forwarding udp:

iptables -t nat -A PREROUTING -i enp0s6 -p udp --dport 21116 -j DNAT --to-destination 172.31.255.221:21116

But yet it doesn't work…
I studied the possibility to use Haproxy v2 protocol. It needs a small modification of the code. I made some tests at https://github.com/sctg-development/proxyv2-test/blob/main/src/main.rs#L47-L89 …

In fact it will be better to study the possibility to use the Websockets. I use the RustDesk web it uses websockets on ports 21118/21119. I just modified the code for using secure websockets on 21120/21121 . It is easy to proxy websockets and to proxy secure websockets to websockets.
hbbs/hbbr use tokio_tungstenite for getting the inner stream via websocket, we'll need to add that support on RustDesk

[eltorio]

I made a fork publically available with my test https://github.com/sctg-development/sctgdesk-server I don't have yet enough time to test but docker image are built at https://hub.docker.com/search?q=eltorio%2Frustdesk-server and the built package are at https://github.com/sctg-development/sctgdesk-server/releases
But as I wrote before Haproxy cannot handle UDP, you need something else.

[eltorio]

Another idea may use QUIC which can be enabled in HAProxy, I saw that the hbb_common code contains a quic feature. Currently it does not compile.