Separate DNS for Transmission and Pihole

Hello everybody
is it possible to use at the same time on the same Raspberry Pi

  • Transmission with wireguard (Mullvad)
  • PiHole with unbound without wireguard?

I’m asking because I would like to keep my torrent client behind a wireguard connection and use unbound for all other devices. I’ve tried modifying the conf file of wireguard and use Pihole as Wireguard DNS and Pihole set to use Unbound, but somehow its functioning is inconsistent - sometimes it works, sometimes it doesn’t.

It is possible. You’d have to mark unbound traffic in mangle table of iptables. Then create a rule in iputils to use a special routing table for the marked packets. Then create a routing table which will have default gateway the ISP, not the VPN.

It seems easy but it’s way too advanced for me.
Do you know any good tutorial?

No, these are separate functions of different programs.
You’ll have to search on how to mark interesting traffic in iptables, then how to create a rule based on marked packets, finally how to create a new routing table and add the default route in there.