Error when trying curl | jq

Hi guys, I’m trying to run this cmd line to pull one of my pihole stats:

curl -f -s http://127.0.0.1/admin/api.php | jq .dns_queries_today

but I must have the syntax wrong because I keep getting this error:

jq: error (at :0): Cannot index array with string “dns_queries_today”

Any suggestions? This query works on my other Ubuntu instance but here I’ve tried everything I can think of and still keep getting this error.

Hardware: NanoPi Neo2 Black
OS: latest Bookworm with latest updates on everything I thought might help.

When you just call api.php you just get an empty array back. Also you need to authenticate yourself, try something like:

http://127.0.0.1/admin/api.php?status&auth=<TOKEN>

Where TOKEN is your API token.

Also with Pihole6 a REST API will be introduced :sunglasses:

on web browser it’s working :smiley:

Trying a few different commands with/without the auth I still don’t think I’m getting the correct return:



In the web browser it does work for me too.

Maybe this helps? What is “X-Frame-Options: deny”

Ok after some testing and research I come to the conclusion that you can not grab JSON with curl. But you can get it with wget. I tried

wget -O temp 'http://<IP_ADDRESS>/admin/api.php?auth=<AUTH_KEY>&summaryRaw' --no-check-certificate

to save it to the file temp.
I needed the --no-check-certificate since it throws some errors (I think because of HTTPS redirect, but I call the local IP and not the domain, I think it’s bc of this, but not directly related to the topic)

Makes sense, my other instance is way out of date and unsupported, which is why I’m trying to re-set it up on DietPi. My goal is to get the python script updated for my NanoHatOLED screen.

Like this? On line 8 it’s still references JSON.

Yes, but it’s no saved to file temp (with the flag -O you can specify a output file) and from there you can process it further.

You’re talking about your pihole instance? You could use teleporter and export all your settings and adlists and import them on the other device.
You can find the teleporter on the pihole web panel Settings > Teleporter
http://PIHOLE_IP/admin/settings.php?tab=teleporter

Both my OS and my PiHole were way out of date. I’ve already teleported and updated everything into the new DietPi OS and everything is working great. I’ve even got the OLED screen working with the stock python file. But trying to implement my old python file it has those curl | jq lines that are failing, so just being nitpicky about what I seen on the screen is all. :slight_smile:

And example of one of the lines:
cmd = “curl -f -s http://127.0.0.1/admin/api.php | jq .dns_queries_today”
Queries = subprocess.check_output(cmd, shell = True ).strip()

bakebit_nanohat_oled_v2.txt (14.0 KB)

It also just occured to me, when I was reinstalling PiHole using DietPi Software it asked me if I wanted to “block public domains” and listed a bunch of examples like “127.0.0.1” , “192.168.1…” etc. Could that possibly be the issue?

You are mixing up things. 127.0.0.1 and 192.168.x.x are local IPs and they are not blocked at all. Otherwise you could not reach the web interface.

Do I get it right, your old system was running on older version of Pihole? I guess Pihole developers changed the behaviour of the software in recent version. This might be the reason why your old scripts are not working anymore.

There is pihole padd available that could display status and other information on a small screen.

It was asking to block public access to pihole, which is good. I guess the example were about the privat IPs, from where you still can reach it. This is not the problem.
I think something changed with the API and so part of the script is not working anymore. It looks like it grabbed everything and then filtered it with jq. On my machine the line curl -f -s http://pihole_IP/admin/api.php$auth=.... returns also nothing .
It also lookes like (in the script) it’s just getting the raw data, so maybe we can use another way. You can directly communicate with pihole FTL via telnet (https://docs.pi-hole.net/ftldns/telnet-api/).
So you could use something like

echo ">stats>quit" | nc 127.0.0.1 4711

(4711 is the port where FTL is running on)

Oh Ok. Correct, yes my old Ubuntu OS was no longer supported and then PiHole would no longer let me update on it. I did see a few things about padd while I was searching around for solutions but didn’t think it pertained to my setup. But I will go back and try looking into the padd stuff and see if I can make that work. Thank you!

Awesome, thank you for the link above too. I’ll fiddle around with it some more and see if I can get it working again. Thanks!

Indeed PADD could be a good replacement, the attached script (bakebit nanohat script) show also “only” various system stats and the pihole stats, but comes also with some button control? :thinking:

Yeah, I borrowed it from this Reddit post and modified it a bit.

https://www.reddit.com/r/pihole/comments/f9ztfs/just_wanted_to_show_you_my_little_pihole_machine/

Ok, after testing all the commands it looks like the Telnet API (Telnet API - Pi-hole documentation) is the only one I can get to return any data. But now I’m stuck on trying to parse out a single value, say returning dns_queries_today only, from this command: echo “>stats>quit” | nc 127.0.0.1 4711

image

It looks like I’m going to have the same question for the new v6 REST API (Pi-hole API documentation) as well. How would I return just a single stat?

Is this something that can’t be done with a single line command? Would it have to be done using additional code in the python file? Because I’ve been trying to test that too.

Probably better to raise such questions within Pihole forum. There you have the experts that might be able to answer your question regarding the API usage.

This depends on how the devs implemented it.

You can filter your output e.g. with grep and cut

echo “>stats>quit” | nc 127.0.0.1 4711 | grep dns_queries_today |  cut -d\   -f2

This should return 30163 (number from your example)

To get all queries for the last 24 hours it should something like (I can not really test this, I do not log my queries):

timediff=86400 | echo ">getallqueries-time $(expr $EPOCHSECONDS-$timediff) $EPOCHSECONDS>quit" | nc 127.0.0.1 4711

This is not possible, you need to process the ouput further, with grep and cut or jq.

Awesome, grep and cut returns exactly what I was looking for. No way I would have been able to piece that together, still trying to work it into my python script without it failing but it does give me the same return as my original cmd line. Thank you for all the help.

1 Like

Well I’m stumped. All of the commands below work in the terminal and return the same result, but as soon as I try to add any of them into my python file it causes it to fail. I can’t see what I’m missing so I’m dumping everything here and will try to come back to it later with fresh eyes.

These commands all work, both as root and dietpi user, and with all ip address versions:
Each of these commands works and returns the same result when I run them dierctly in the terminal:

curl -f -s "http://127.0.0.1/admin/api.php?summaryRaw&auth=69xx"
curl -f -s "http://192.168.1.100/admin/api.php?summaryRaw&auth=69xx" | jq .dns_queries_today
echo ">stats>quit" | nc 127.0.0.1 4711
echo ">stats>quit" | nc 127.0.0.1 4711 | grep dns_queries_today |  cut -d\   -f2
echo -e ">stats>quit" | nc 127.0.0.1 4711 | grep dns_queries_today |  cut -d\   -f2

In addition I also ran the same things in python and got the same results, again mixing/matching everything above:

python3
import json
import requests
api_url = 'http://127.0.0.1/admin/api.php?summaryRaw&auth=69xx
r = requests. Get(api_url)
data = json.loads(r.text)
Queries = data['dns_queries_today']
print(Queries)
python3
import subprocess
cmd = "curl -f -s \"http://127.0.0.1/admin/api.php?summaryRaw" + "&auth=$(grep -oPi \"(?<=WEBPASSWORD\=).+\" /etc/pihole/setupVars.conf)\" | jq .dns_queries_today"
Queries = subprocess.check_output(cmd, shell = True ).decode("utf-8", errors="ignore").strip()
print(Queries)

Here’re my python tests just to make sure I’m getting the syntax correct. Again I tried every combination I could think of using single quotes/double quotes/backslashes to escape quotes where necessary.

list of relevant things imported:
import subprocess
import os
import json
import requests

Examples:

cmd = "curl -f -s 'http://127.0.0.1/admin/api.php?summaryRaw&auth=94xx' | jq .dns_queries_today"
cmd = "curl -f -s \"http://127.0.0.1/admin/api.php?summaryRaw" + "&auth=$(grep -oPi \"(?<=WEBPASSWORD\=).+\" /etc/pihole/setupVars.conf)\" | jq .dns_queries_today"
cmd = "os.system('curl -f -s \"http://127.0.0.1/admin/api.php?summaryRaw&auth=94xx" | jq .dns_queries_today')"
cmd = "echo -e \">stats>quit\" | nc 127.0.0.1 4711 | grep dns_queries_today |  cut -d\   -f2"

Queries = subprocess.check_output(cmd, shell = True ).decode("utf-8", errors="ignore").strip()

try:

	api_url = 'http://127.0.0.1/admin/api.php?summaryRaw&auth=94xx'
	r = requests.get(api_url) or r = requests.get(api_url, headers={"Content-Type":"text"}) or r = requests.get(api_url, headers={"Content-Type":"application/json"})
	data = json.loads(r.text)
	Queries = data['dns_queries_today']
except KeyError:
	time.sleep(1)
	continue

This is my working file with my tests on row 195-209:
bakebit_nanohat_oled.txt (11.2 KB)