Lab 16.2 HA Proxy is unable to restart when multiple backend servers

josephkwong

The HA Proxy started okay when 1 backend server based on the file provided though k8s first master node didn't install yet.

[email protected]:~# cat /etc/haproxy/haproxy.cfg
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
stats timeout 30s
user haproxy
group haproxy
daemon
maxconn 64

    # Default SSL material locations
    ca-base /etc/ssl/certs
    crt-base /etc/ssl/private

    # See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
    ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets

defaults
log global
mode tcp
option tcplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http

frontend proxynode #<-- Add the following lines to bottom of file
bind *:80
bind *:6443
stats uri /proxystats
default_backend k8sServers

backend k8sServers
option tcp-check
tcp-check connect
balance roundrobin
server cp 192.168.3.67:6443 check fall 3 rise 2 maxconn 16

server cp2 192.168.3.68.6443 check fall 3 rise 2

server cp3 192.168.3.69.6443 check fall 3 rise 2

listen stats
bind :9999
mode http
stats enable
stats hide-version
stats uri /stats

[email protected]:~# systemctl restart haproxy
[email protected]:~# systemctl status haproxy
● haproxy.service - HAProxy Load Balancer
Loaded: loaded (/lib/systemd/system/haproxy.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2021-11-08 15:00:20 UTC; 14s ago
Docs: man:haproxy(1)
file:/usr/share/doc/haproxy/configuration.txt.gz
Process: 81032 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS (code=exited, status=0/SUCCESS)
Main PID: 81033 (haproxy)
Tasks: 5 (limit: 4616)
Memory: 2.6M
CGroup: /system.slice/haproxy.service
├─81033 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock
└─81034 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock

Nov 08 15:00:20 ha-proxy haproxy[81033]: Proxy proxynode started.
Nov 08 15:00:20 ha-proxy haproxy[81033]: Proxy proxynode started.
Nov 08 15:00:20 ha-proxy haproxy[81033]: Proxy k8sServers started.
Nov 08 15:00:20 ha-proxy haproxy[81033]: Proxy k8sServers started.
Nov 08 15:00:20 ha-proxy haproxy[81033]: Proxy stats started.
Nov 08 15:00:20 ha-proxy haproxy[81033]: Proxy stats started.
Nov 08 15:00:20 ha-proxy haproxy[81033]: [NOTICE] 311/150020 (81033) : New worker #1 (81034) forked
Nov 08 15:00:20 ha-proxy systemd[1]: Started HAProxy Load Balancer.
Nov 08 15:00:20 ha-proxy haproxy[81034]: [WARNING] 311/150020 (81034) : Server k8sServers/cp is DOWN, reason: Layer4 connection problem, info: "Con>
Nov 08 15:00:20 ha-proxy haproxy[81034]: [ALERT] 311/150020 (81034) : backend 'k8sServers' has no server available!

[email protected]:~# haproxy -c -f /etc/haproxy/haproxy.cfg
[WARNING] 311/145918 (80970) : config : 'stats' statement ignored for frontend 'proxynode' as it requires HTTP mode.
Configuration file is valid

Any impact of this WARNING? How can I reconfigure to remove this warning?

Thanks for feedback in advance.
Joseph

chrispokorni

Hi @josephkwong,

The SOLUTIONS/s_16/haproxy.cfg file could help to get your haproxy service up and running without errors.

Regards,
-Chris

josephkwong

Thank you Chris.

I copy and paste the SOLUTIONS/s_16/haproxy.cfg file
...
# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNUL$
ssl-default-bind-options no-sslv3

It came out the following fatal errors.
[email protected]:/# haproxy -c -f /etc/haproxy/haproxy.cfg
[ALERT] 312/085340 (109028) : parsing [/etc/haproxy/haproxy.cfg:34] : unknown keyword 'ssl-default-bind-ciphers' in 'defaults' section
[ALERT] 312/085340 (109028) : parsing [/etc/haproxy/haproxy.cfg:35] : unknown keyword 'ssl-default-bind-options' in 'defaults' section
[ALERT] 312/085340 (109028) : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg
[ALERT] 312/085340 (109028) : Fatal errors found in configuration.

When I reverted back my previously configuration,
...
# See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA38>
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets

The file check is valid but it still showed this [WARNING].
[email protected]:/# haproxy -c -f /etc/haproxy/haproxy.cfg
[WARNING] 312/090237 (109260) : config : 'stats' statement ignored for frontend 'proxynode' as it requires HTTP mode.
Configuration file is valid

Kindly your further advice with thank. Have a great day, Joseph...

chrispokorni

Hi @josephkwong,

Instead of copy/paste can you try to scp the entire file? Or download the tarball on the haproxy node?
Even with the displayed warnings, can you access the haproxy console?

Regards,
-Chris

josephkwong

I can get access the haproxy console as attached though the displayed warnings shown, where I am figuring out why having such warning.