usually time sync should be done via VBoxService if I’m not mistaken. However you could exclude VBoxService from being checked.
nano /lib/systemd/system/systemd-timesyncd.service.d/disable-with-time-daemon.conf
comment out line: ConditionFileIsExecutable=!/usr/sbin/VBoxService
Once done restart the service
systemctl daemon-reload
systemctl restart systemd-timesyncd.service