Local mirror for install?


We have just started using DietPi instead of Raspbian as it is so much better!
I have also just gotten started on automating the installation process so its quicker to make new units.
One thing I would like to do is mirror the install archive to our local Synology NAS, the whole raspbian is not needed but mainly the DietPi main install and it’s application.
The main reason for the local install is not to download every archive every time we are making a new device. Our internet connection is also a bit slow so having a local mirror would speed things up a lot.

Any suggestion on how to do this?
If needed we can set up a Docker or VM machine in our ESX but using the Synology web and file service would be cleaner.

Currently it is not easily possible for non-APT downloads.

For APT packages it should be natively possible, although you most likely don’t want a whole local APT mirror :thinking:. Not sure if it’s possible to somehow let local APT clients trigger a central “server” to download and cache the packages once, so that all local clients can download this particular package(s) from it. Web search might reveal a possibility to achieve this. If possible, it will work with DietPi as well.

About the DietPi archive: Currently via dietpi.txt dev settings, you can use a different GitHub repo, but we do not natively support a local Git repo. This is actually an interesting idea to support: Allowing to add a custom domain/IP to download the Git archive and single files from it for multi DietPi networks, kind of a master<>salve/server<>client feature. We already though about this in the past, to merge local NTP, DHCP/WiFi-AP, DNS and such in one scheme.
BUT: Note that the whole DietPi Git archive is only 2,4M in size. The question is, if for this the effort makes sense. According our release schedule you will only download 2,4M ~every month for each device, nothing that should kill your traffic plan or where your bandwidth has much effect.

About 3rd party software downloads:

  • For DietPi-Software downloads (outside of APT), it is very difficult, since non-APT based software downloads all have individual URLs that do not follow any scheme to use for a consistent master<>slave setup. This would need a larger rework/much effort to implement something for this.

But I am teased for this as large project, although no chance to have such short or middle term: DietPi clusters

  • most likely/easiest with master<>slave solution
  • I am fans of dezentral solutions, but whooo then a communication systems would be required as well, such as: “Which device has sufficient free space and is in idle to do the download for the others?”, then broadcasting: “Software X ready for local download here” :sunglasses: