Can't change lighttpd's port for HaProxy reverse proxy

Troubleshooting
1

Hey,

I’m trying to set up a reverse proxy so that each service/port has its own subdomain. So as an example, gitea would be git.example.com, a Matrix synapse server would be matrix.example.com and my main website would be just example.com with no subdomain.

I’m using this guide: https://developweekly.com/docs/haproxy-load-balancing-and-subdomain-port-redirection/

However, I’ve encountered a problem. HaProxy needs to be the service pointed at port 80, which is handled currently by lighttpd. However, I cant change lighttpd’s port from 80 to something else. I tried setting it in /etc/lighttpd/lighttpd.conf, using server.port = 8080, but for some reason it still uses port 80.

Is there any way I can fix this? Some tips on this reverse proxy setup would be nice, too!

2

Hi,

many thanks for your request. I did a test on my RPi3B+ and changing server port within /etc/lighttpd/lighttpd.conf is working quite well

root@DietPi3:/etc/lighttpd# cat /etc/lighttpd/lighttpd.conf | grep server.port
server.port                 = 8080

LISTEN port is 8080 now

root@DietPi3:/etc/lighttpd# lsof -i -P -n | grep LISTEN
dropbear 474     root    3u  IPv4  10923      0t0  TCP *:22 (LISTEN)
dropbear 474     root    4u  IPv6  10924      0t0  TCP *:22 (LISTEN)
lighttpd 697 www-data    4u  IPv4  14511      0t0  TCP *:8080 (LISTEN)
lighttpd 697 www-data    5u  IPv6  14512      0t0  TCP *:8080 (LISTEN)

Did you restart lighttpd service after changing the port?

systemctl restart lighttpd.service
3

Alright, so I’m dumb. Turns out it is changing the port and it is saying that it is being used as the LISTEN port using the command you did. However, the webserver is still accessible from port 80 and is not accessible from port 8080. Is there something in lighttpd’s configs controlling this that I don’t know about?

I’m new to both HAProxy and lighttpd, so it really might just be me being dumb.

4

can you past output of lsof -i -P -n | grep LISTEN pls

5 
root@dietpi:~# lsof -i -P -n | grep LISTEN
pihole-FT  350      pihole    5u  IPv4  11603      0t0  TCP *:53 (LISTEN)
pihole-FT  350      pihole    7u  IPv6  11605      0t0  TCP *:53 (LISTEN)
pihole-FT  350      pihole   10u  IPv4  11609      0t0  TCP 127.0.0.1:4711 (LISTEN)
pihole-FT  350      pihole   11u  IPv6  11611      0t0  TCP [::1]:4711 (LISTEN)
cloudflar  439 cloudflared    3u  IPv4  12640      0t0  TCP 127.0.0.1:33167 (LISTEN)
cloudflar  439 cloudflared    6u  IPv4  12649      0t0  TCP 127.0.0.1:5053 (LISTEN)
sshd       550        root    3u  IPv4  12744      0t0  TCP *:22 (LISTEN)
sshd       550        root    4u  IPv6  12746      0t0  TCP *:22 (LISTEN)
redis-ser  577       redis    7u  IPv4  12775      0t0  TCP 127.0.0.1:6379 (LISTEN)
redis-ser  577       redis    8u  IPv6  12776      0t0  TCP [::1]:6379 (LISTEN)
mysqld     644       mysql   19u  IPv4  12985      0t0  TCP 127.0.0.1:3306 (LISTEN)
gitea      725      dietpi   18u  IPv6  17484      0t0  TCP *:3000 (LISTEN)
Cuberite   749    cuberite   24u  IPv6  15670      0t0  TCP *:25565 (LISTEN)
Cuberite   749    cuberite   25u  IPv6  15671      0t0  TCP *:1339 (LISTEN)
lighttpd  5803    www-data    4u  IPv4  74810      0t0  TCP *:8080 (LISTEN)
lighttpd  5803    www-data    5u  IPv6  74811      0t0  TCP *:8080 (LISTEN)
lighttpd  5803    www-data    6u  IPv4  74812      0t0  TCP *:443 (LISTEN)
6

pls make sure you are using HTTP and not HTTPS. As far as I can see lighttpd is still running on port 443.

http://<your-ip>:8080

But I guess you need to change lighttpd port 443 anyway if you like to use HaPoxy + SSL.

7

I cannot access the server using :8080, nor :443. Only :80 works, which should be impossible as it isn’t being used…

Port 443 gives me a “connection reset” error, while 8080 just times out without a response. Port 80 is still working fine…

8

did you have done a full reboot of your system?

9

I have now done a full reboot with the port change saved in lighttpd.conf, but it still seems to be using port 80 and is still accessible from the internet. I’m not forwarding port 8080, either.

10

do you have HaProxy already installed? What about access from local notwork (let’s not focus on internet access)?

11

HaProxy is installed, but isn’t running. dietpi-services says it has failed, probably because I haven’t fully set up the configuration yet. Here is its log:

● haproxy.service - HAProxy Load Balancer 
│    Loaded: loaded (/etc/systemd/system/haproxy.service; disabled; vendor preset: enabled)
│    Active: failed (Result: exit-code) since Mon 2020-06-08 09:05:49 BST; 4h 27min ago
│   Process: 648 ExecStartPre=/usr/local/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS (code=exited, status=203/EXEC)
│                                                                                                                      │
│ Jun 08 09:05:49 Ouroboros systemd[1]: haproxy.service: Service RestartSec=100ms expired, scheduling restart. 
│ Jun 08 09:05:49 Ouroboros systemd[1]: haproxy.service: Scheduled restart job, restart counter is at 5. 
│ Jun 08 09:05:49 Ouroboros systemd[1]: Stopped HAProxy Load Balancer.
│ Jun 08 09:05:49 Ouroboros systemd[1]: haproxy.service: Start request repeated too quickly.
│ Jun 08 09:05:49 Ouroboros systemd[1]: haproxy.service: Failed with result 'exit-code'.
│ Jun 08 09:05:49 Ouroboros systemd[1]: Failed to start HAProxy Load Balancer.

I also cannot access my website using the internal IP and port 8080, so it really is just accessible through port 80…

12

can you remove HaProxy and deactivate port 443 on lighttpd. than restart everything

13

I have now uininstalled HaProxy and disabled port 443. Port 80 is still apparently being used!

Here’s the output of lsof -i -P -n | grep LISTEN:

pihole-FT 351      pihole    5u  IPv4  12341      0t0  TCP *:53 (LISTEN)
pihole-FT 351      pihole    7u  IPv6  12343      0t0  TCP *:53 (LISTEN)
pihole-FT 351      pihole   10u  IPv4  11292      0t0  TCP 127.0.0.1:4711 (LISTEN)
pihole-FT 351      pihole   11u  IPv6  11294      0t0  TCP [::1]:4711 (LISTEN)
cloudflar 436 cloudflared    3u  IPv4  13642      0t0  TCP 127.0.0.1:45997 (LISTEN)
cloudflar 436 cloudflared    6u  IPv4  12886      0t0  TCP 127.0.0.1:5053 (LISTEN)
sshd      571        root    3u  IPv4  12975      0t0  TCP *:22 (LISTEN)
sshd      571        root    4u  IPv6  12977      0t0  TCP *:22 (LISTEN)
redis-ser 595       redis    7u  IPv4  13723      0t0  TCP 127.0.0.1:6379 (LISTEN)
redis-ser 595       redis    8u  IPv6  13724      0t0  TCP [::1]:6379 (LISTEN)
mysqld    660       mysql   21u  IPv4  13054      0t0  TCP 127.0.0.1:3306 (LISTEN)
lighttpd  727    www-data    4u  IPv4  13223      0t0  TCP *:8080 (LISTEN)
lighttpd  727    www-data    5u  IPv6  13224      0t0  TCP *:8080 (LISTEN)
gitea     731      dietpi    5u  IPv6  15796      0t0  TCP *:3000 (LISTEN)
Cuberite  736    cuberite   24u  IPv6  14150      0t0  TCP *:25565 (LISTEN)
Cuberite  736    cuberite   25u  IPv6  14151      0t0  TCP *:1339 (LISTEN)

I’m at a loss.

14

well me2. Can you stop service by service to have lighttpd running only

15

Yeah, that seems to change nothing. Do you think there is an additional config file in /etc/lighttpd/config-enabled causing this, or something that I set up during the dietpi installation?

16

doesn’t matter how much config files you have, there is no LISTEN port 80. Means you should not be able to reach anything at all on port 80. Can you try to clear cache on your browser

17

I cleared my browser cache and now something weirder is happening. It seems to accept a connection on port 80 via HTTP, which then redirects to HTTPS because of an option I checked during install., where it is “unable to connect” because port 443 is no longer open.

I changed the port to 8035, just in case it was the fact that 8080 was overlapping with 80 or something, but that didn’t have any effect…

18

Extra: I tried on local IPs again, looks like it just rejects the connection on port 80 while my new port just times out. Weird.

EDIT: Alright, we’re getting somewhere! It looks like the new port is working, but the connection is timing out.

19

ahh yeah you redirect http to https. pls can you have a look into /etc/lighttpd/conf-enabled. There you should have 2 files.

-rw-r--r-- 1 root root 1735 Jun  8 21:12 letsencrypt.conf
-rw-r--r-- 1 root root  223 Jun  8 21:12 redirect.conf

pls remove them and restart your webserver

20

Removing the files and rebooting did nothing to change anything… There are other symlinks in there but I don’t want to touch them:

root@Ouroboros:~# ls /etc/lighttpd/conf-enabled
10-fastcgi.conf  15-fastcgi-php.conf          99-dietpi-nextcloud.conf                  99-dietpi-pihole.conf
10-rewrite.conf  99-dietpi-dav_redirect.conf  99-dietpi-pihole-block_public_admin.conf  99-unconfigured.conf

EDIT: It’s working! Firefox was screwing with me with dodgy cache.

Now then, how should I go about adding SSL to the webserver from HaProxy? I’d prefer it to be compatible with the guide I linked at the start of thread. Is there a way I could possibly use a wildcard certificate with certbot?