How do I set up a framebuffer terminal emulator and get more than 8 or 16 colors without running a display server?

Exactly, the linux console can only display 16 colors.
As far a I know fbterm also only supports 16 colors and the development is abondened.
To achieve your goal (256 colors, view PDFs and emoijs), without wayland or xserver, you would need to use

  • vc4-kms-v3d driver
  • use a wayland terminal like foot, but without using wayland (yes, this is possible) and build foot by your own, with the use of some special flags
  • create a systemd service to replace getty with foot

I will test this on my RPi 3B before I post a step-by-step guide

edit: It did not work