First boot - Not able to fetch data with HTTPS/443

Having issues with your DietPi installation or found a bug? Post it here.
gokhan
Posts: 15
Joined: Sun Jan 17, 2021 11:39 am

Re: First boot - Not able to fetch data with HTTPS/443

Post by gokhan »

config

Code: Select all

                            ●─ Adapter Options ──────────────●                │
│               Ethernet     : Available | [On] | Connected                    │
│               WiFi         : Not Found | [Off] | Disconnected                │
│               Onboard WiFi : [Off]                                           │
│                            ●─ Additional Options ───────────●                │
│               IPv6         : [On]                                            │
│               Prefer IPv4  : [On]                                            │
│               Proxy        : [Off]                                           │

Try_1

Code: Select all

root@DietPi:~# wget -v --spider https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Spider mode enabled. Check if remote file exists.
--2021-01-18 16:24:42--  https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.140.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.140.133|:443... connected.
Try_2

Code: Select all

root@DietPi:~# curl -IL https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
HTTP/2 200 
content-type: text/plain; charset=utf-8
cache-control: max-age=300
content-security-policy: default-src 'none'; style-src 'unsafe-inline'; sandbox
etag: "c03658be5fa4c240351ae67f33338ac48a4916d6c115177fa3f8e6f2736b7bb9"
strict-transport-security: max-age=31536000
x-content-type-options: nosniff
x-frame-options: deny
x-xss-protection: 1; mode=block
via: 1.1 varnish (Varnish/6.0), 1.1 varnish
x-github-request-id: 73A0:675E:1041E9:12599B:6005B747
accept-ranges: bytes
date: Mon, 18 Jan 2021 16:28:55 GMT
x-served-by: cache-fjr7920-FJR
x-cache: HFM, MISS
x-cache-hits: 0, 0
x-timer: S1610987335.108782,VS0,VE279
vary: Authorization,Accept-Encoding, Accept-Encoding
access-control-allow-origin: *
x-fastly-request-id: de19f6f9ca2dfb01be602a5fbe007c54c5181957
expires: Mon, 18 Jan 2021 16:33:55 GMT
source-age: 0
content-length: 12914
Try_3

Code: Select all

root@DietPi:~# wget -v --spider -6 https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Spider mode enabled. Check if remote file exists.
--2021-01-18 16:29:30--  https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... failed: Name or service not known.
wget: unable to resolve host address ‘raw.githubusercontent.com’
I may need to listen it with wireshark :/

This my very first raspberry experience, lol.
gokhan
Posts: 15
Joined: Sun Jan 17, 2021 11:39 am

Re: First boot - Not able to fetch data with HTTPS/443

Post by gokhan »

Same result if I turn off IPv6 in dietpi-config.
gokhan
Posts: 15
Joined: Sun Jan 17, 2021 11:39 am

Re: First boot - Not able to fetch data with HTTPS/443

Post by gokhan »

So I installed Ubuntu 20.04.1 for Raspberry Pi.

Code: Select all

ubuntu@ubuntu:/home$ wget -v --spider https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Spider mode enabled. Check if remote file exists.
--2021-01-18 17:04:13--  https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi.txt
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.140.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.140.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12914 (13K) [text/plain]
Remote file exists.
ta-da it works!

Apologies if I spam you guys with my problem but I believe something is wrong with base image.

Or maybe the flash utility. balenaEtcher vs Raspberry Pi Imager. I really do not know.
User avatar
Joulinar
Posts: 5148
Joined: Sat Nov 16, 2019 12:49 am

Re: First boot - Not able to fetch data with HTTPS/443

Post by Joulinar »

but you told it's failing on plain Raspbian OS as well if I understood right. So it would not be an issue of DietPi, it's more a challenge of Raspbian OS. Probably wget is behaving different between Ubuntu and Raspbian OS :?
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
gokhan
Posts: 15
Joined: Sun Jan 17, 2021 11:39 am

Re: First boot - Not able to fetch data with HTTPS/443

Post by gokhan »

Indeed Joulinar, it fails both in Raspberry OS and DietPi.
User avatar
MichaIng
Site Admin
Posts: 3106
Joined: Sat Nov 18, 2017 6:21 pm

Re: First boot - Not able to fetch data with HTTPS/443

Post by MichaIng »

Raspbian Buster (hence DietPi 32-bit RPi image) ships a different version of wget and a different build (for armv6hf). But that image is used by hundreds of thousands of users, so it issue is very specific, but I have no idea what could be related.

raw.githubusercontent.com has no AAAA (IPv6) record indeed, so that check was obsolete :P:

Code: Select all

2021-01-18 20:11:10 root@micha:~# getent hosts raw.githubusercontent.com
151.101.112.133 github.map.fastly.net raw.githubusercontent.com
Remarkable that even such major hostname has not enabled IPv6, and sad the same way that we're still at this stage where IPv6 causes enough issues that huge companies disable it... :(.

Since curl succeeds while wget fails, probably it's related to how the underlying SSL library is called. curl uses OpenSSL (libssl) while wget uses GnuTLS (libgnutls). Reminds me a bit of the SSL_ERROR_SYSCALL failures of curl (https://github.com/MichaIng/DietPi/issues/3306) which were reported upstream by others as well but neither we nor the curl devs found any clue where this came from. Often switching the network adapter or even only the IP address solved it.

We switched to curl completely for v6.35 btw, but the above shows that it is no guarantee for such kind of mysterious issues ;).
User avatar
Joulinar
Posts: 5148
Joined: Sat Nov 16, 2019 12:49 am

Re: First boot - Not able to fetch data with HTTPS/443

Post by Joulinar »

but why is it working using a VPN? So it can't be something on the system themselves, isn't it?
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
User avatar
MichaIng
Site Admin
Posts: 3106
Joined: Sat Nov 18, 2017 6:21 pm

Re: First boot - Not able to fetch data with HTTPS/443

Post by MichaIng »

The (virtual) network interface and the routes are changed when using a VPN, a bit similar to solutions for the SSL_ERROR_SYSCALL that I have seen.

The effect should be different, but did you actually rule out the MTU size?
gokhan
Posts: 15
Joined: Sun Jan 17, 2021 11:39 am

Re: First boot - Not able to fetch data with HTTPS/443

Post by gokhan »

No, I have not checked the MTU size yet. I will update you once I do. Thank you.
Post Reply