- Français (fr)
- English (en)
debmirror (tips and tricks)
debmirror is a program who aims to download the content of a debian repository in order to set a local mirror up and maintain it up to date.
The parameters are numerous and depend on each repository, so it is sometimes hard to find right settings on the web. That's why I gather here the parameters that were good for me + a touch of methodology.
Note: As I actually do not use debmirror, some informations on this page may be quite outdated.
— Médéric 02/12/2018 23:35
Getting the public GPG keys
debmirror runs in userspace and tries to get each public GnuPG key repository from the curent user trusted keyring ~/.gnupg/trustedkeys.gpg
.
How to retrieve repository keys ?
Each GPG repository key must be requested by its ID to key servers which return the public key content if available:
- Search for the public GPG key ID on the official repository web site
- Send a query to a key server for that ID
The official repository procedures tell the root user to import the public key into the APT keyring, by providing the ID of the public GPG Key either like this:
apt-key adv --keyserver a-server.tld --recv-keys ID
or like this:
gpg --keyserver a-server.tld --recv-keys ID && gpg --export -a ID | apt-key add -
With the key ID and the server name ("a-server.tld"), the user can download the public repository GPG key into its own trusted keyring ~/.gnupg/trustedkeys.gpg
either by importing first to its personal public keyring ~/.gnupg/pubring.gpg
:
gpg --keyserver a-server.tld --recv-keys ID gpg --export -a ID | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -
or directly to its trusted keyring ~/.gnupg/trustedkeys.gpg
:
gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver a-server.tld --recv-keys ID
In most cases, the key server keys.gnupg.net
will be sufficient to retrieve keys so that the only variable parameter will be the ID:
gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys ID
GPG keys may change so it's important to perform an update from time to time. A failsafe strategy is to request the public repository key before each debmirror session.
Rely on the APT trusted keyring
Export the APT trusted keyring to the user's one
If all the repositories that one want to mirror are already configured within the system (/etc/apt/sources.list and /etc/apt/sources.list.d/…), then the respectives public GPG keys should be up to date. In this case, it's convenient to update the user GPG trusted keyring by importing from the APT's one as follow:
gpg --keyring /usr/share/keyrings/debian-archive-keyring.gpg --export | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import
Aternate way: using debmirror with APT keyring
An alternate way is to tell debmirror to look directly into the APT keyring:
debmirror --keyring /usr/share/keyrings/debian-archive-keyring.gpg ...
Check the repository key validation
Looking at standard output, debmirror shows if a GPG key is missing in ~/.gnupg/trustedkeys.gpg
.
Example:
[GNUPG:] ERRSIG 8B48AD6246925553 1 8 00 1441453660 9 [GNUPG:] NO_PUBKEY 8B48AD6246925553 [GNUPG:] ERRSIG 7638D0442B90D010 1 8 00 1441453660 9 [GNUPG:] NO_PUBKEY 7638D0442B90D010 [GNUPG:] ERRSIG 6FB2A1C265FFB764 1 8 00 1441453675 9 [GNUPG:] NO_PUBKEY 6FB2A1C265FFB764 gpgv: keyblock resource `/home/demo/.gnupg/trustedkeys.gpg': file open error gpgv: Signature made Sat Sep 5 13:47:40 2015 CEST using RSA key ID 46925553 gpgv: Can't check signature: public key not found gpgv: Signature made Sat Sep 5 13:47:40 2015 CEST using RSA key ID 2B90D010 gpgv: Can't check signature: public key not found gpgv: Signature made Sat Sep 5 13:47:55 2015 CEST using RSA key ID 65FFB764 gpgv: Can't check signature: public key not found gpgv: keyblock resource `/home/demo/.gnupg/trustedkeys.gpg': file open error gpgv: Signature made Sat Sep 5 13:47:40 2015 CEST using RSA key ID 46925553 gpgv: Can't check signature: public key not found :Failed to download some Release or Release.gpg files!
In this case, one have to retrieve the public GPG keys whose IDs are:
- 46925553
- 2B90D010
- 65FFB764
Repository timeout
Warning!!
Some repositories have a validation timeout. In this case, if one exceeds that limit before updating the repository, all datas will be regarded as outdated and will be re-downloaded and overwritten!! This can lead an overload bandwidth!!
In this case, I advise to adjust the cron task that launches your debmirror script in order to save resources.
In the following examples, the base directory of the local mirrored repositories is stored in the shell variable TARGET_DIR.
If one want to know the repositories that have a timeout and their respectives values the following command do the trick:
find $TARGET_DIR -name Release -exec grep -HE '^Valid-Until:' "{}" \; | \
grep -v '/.temp/' | gawk -F '/' '{date=gensub(/^.*Valid-Until: */,"",$0); print $4": "date}'
Example:
find $TARGET_DIR -name Release -exec grep -HE '^Valid-Until:' "{}" \; | grep -v '/.temp/' | gawk -F '/' '{date=gensub(/^.*Valid-Until: */,"",$0); print $4" : "date}' debian.org: Tue, 22 Sep 2015 02:42:42 UTC debian.org: Tue, 22 Sep 2015 02:42:42 UTC security.debian.org : Wed, 23 Sep 2015 20:01:59 UTC
Each day, the Debian repositories are refreshed and their timeout is set to 7 days up.
Debian
- Official site: http://www.debian.org/
- Debian Wiki: https://wiki.debian.org/
- Debian packages: https://www.debian.org/distrib/packages
If you're concerned with Debian, see also the Devuan project from the "Veteran Unix Admins" (VUA) team who aims to revert to an UNIX authentic distribution : http://devuan.org/ and newsletter_22dec.html.
debian.org
Debian 7 (wheezy):
## Debian Archive Automatic Signing Key (Comment: 7.0/wheezy) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 46925553 ## Debian Archive Automatic Signing Key (Comment: 8/jessie) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 2B90D010 ## Wheezy Stable Release Key (Comment: none) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 65FFB764 debmirror -v --verbose --arch=i386,amd64 --dist=wheezy,wheezy-updates,wheezy-backports --source --method=rsync --host=ftp.fr.debian.org --root=:debian --section=main,contrib,non-free --i18n --postcleanup --diff=mirror $TARGET_DIR/debian.org/debian
One can add that local repository by adding it to a future sources.list file under construction:
echo deb file:$TARGET_DIR/debian.org/debian/ wheezy main contrib non-free >>/etc/apt/sources.list.local echo deb file:$TARGET_DIR/debian.org/debian/ wheezy-updates main contrib >>/etc/apt/sources.list.local
Debian 8 (jessie):
## Debian Archive Automatic Signing Key (Comment: 7.0/wheezy) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 46925553 ## Debian Archive Automatic Signing Key (Comment: 8/jessie) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 2B90D010 ## Jessie Stable Release Key (Comment: none) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 518E17E1 debmirror -v --verbose --arch=i386,amd64 --dist=jessie,jessie-updates,jessie-backports --source --method=rsync --host=ftp.fr.debian.org --root=:debian --section=main,contrib,non-free --i18n --postcleanup --diff=mirror $TARGET_DIR/debian.org/debian
One can add that local repository by adding it to a future sources.list file under construction:
echo deb file:$TARGET_DIR/debian.org/debian/ jessie main contrib non-free >>/etc/apt/sources.list.local echo deb file:$TARGET_DIR/debian.org/debian/ jessie-updates main contrib >>/etc/apt/sources.list.local
security.debian.org
Debian 7 (wheezy):
## Debian Security Archive Automatic Signing Key (Comment: 8/jessie) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys C857C906 ## Debian Archive Automatic Signing Key (Comment: 7.0/wheezy) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 46925553 debmirror -v --verbose --arch=i386,amd64 --dist=wheezy/updates --source --method=rsync --host=security.debian.org --root=:debian-security/ --section=main,contrib,non-free --i18n --postcleanup --diff=mirror $TARGET_DIR/debian.org/debian
One can add that local repository by adding it to a future sources.list file under construction:
echo deb file:$TARGET_DIR/security.debian.org/ wheezy/updates main contrib non-free >>/etc/apt/sources.list.local
Debian 8 (jessie):
## Debian Security Archive Automatic Signing Key (Comment: 8/jessie) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys C857C906 ## Debian Archive Automatic Signing Key (Comment: 7.0/wheezy) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 46925553 debmirror -v --verbose --arch=i386,amd64 --dist=jessie/updates --source --method=rsync --host=security.debian.org --root=:debian-security/ --section=main,contrib,non-free --i18n --postcleanup --diff=mirror $TARGET_DIR/debian.org/debian
One can add that local repository by adding it to a future sources.list file under construction:
echo deb file:$TARGET_DIR/security.debian.org/ jessie/updates main contrib non-free >>/etc/apt/sources.list.local
mozilla.debian.net
The Debian Mozilla Team maintains the Mozilla Foundation softwares rebranded by the Debian project.
## Debian Mozilla team APT archive (Comment: none) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 06C4AE2A debmirror -v --verbose --arch=i386,amd64 --host=mozilla.debian.net --root=:. --dist=wheezy-backports --section=iceweasel-aurora,iceweasel-beta,iceweasel-release,iceweasel-esr,icedove-esr --i18n --postcleanup --source --method=http --diff=mirror $TARGET_DIR/mozilla.debian.net
One can add that local repository by adding it to a future sources.list file under construction:
echo deb file:$TARGET_DIR/mozilla.debian.net/ wheezy-backports iceweasel-release >>/etc/apt/sources.list.local
ignorantguru.github.com
Ignorant Guru is well know to be the systemd-free creator of SpaceFM. His main web pages are:
- Downloads: https://igurublog.wordpress.com/downloads/
- GitHub: https://github.com/IgnorantGuru
## IgnorantGuru (Comment: none) (Expiration: 01/02/2015) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 01937621 debmirror -v --verbose --arch=i386,amd64 --host=ignorantguru.github.com --root=:debian --dist=unstable --section=main --i18n --postcleanup --source --method=http --diff=mirror $TARGET_DIR/ignorantguru.github.com/debian
One can add that local repository by adding it to a future sources.list file under construction:
echo deb file:$TARGET_DIR/ignorantguru.github.com/debian/ unstable main >>/etc/apt/sources.list.local
deb-multimedia.org
Some usefull multimedia softwares are banned from debian repository for ethic and legitimate reasons: not enough free to the Debian Social Contract.
But some of this softwares are sometimes necessary – like the famous FFmpeg from the famous Fabrice Bellard –. Fortunatly, Christian Marillat and Thibaut Varene maintain theses softwares up to date for debian users from the site deb-multimedia.org:
## Christian Marillat (Comment: none) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 65558117 debmirror -v --verbose --arch=i386,amd64 --host=deb-multimedia.org --root=/ --dist=wheezy --section=main,non-free --i18n --postcleanup --source --method=http --diff=mirror $TARGET_DIR/www.deb-multimedia.org
One can add that local repository by adding it to a future sources.list file under construction:
echo deb file:$TARGET_DIR/www.deb-multimedia.org/ wheezy main non-free >>/etc/apt/sources.list.local
Trinity Desktop
Trinity Desktop is a fork of KDE 3.5 maintained for debian based systems.
The following code downloads official Trinity R14.0.x packages from the repository of the University of Kent http://mirrorservice.org/sites/trinitydesktop.org/trinity/:
## QuickBuild Trinity Desktop Environment (Comment: none) gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keyserver.quickbuild.pearsoncomputing.net --recv-keys F5CFC95C debmirror -v --verbose --arch=i386,amd64 --host=mirrorservice.org --root=:sites/trinitydesktop.org/trinity/trinity-r14.0.0/debian --dist=wheezy --section=main --i18n --postcleanup --source --method=http --diff=mirror $TARGET_DIR/trinity-r14.0.0/debian debmirror -v --verbose --arch=i386,amd64 --host=mirrorservice.org --root=:sites/trinitydesktop.org/trinity/trinity-builddeps-r14.0.0/debian --dist=wheezy --section=main --i18n --postcleanup --source --method=http --diff=mirror $TARGET_DIR/trinity-builddeps-r14.0.0/debian
One can add that local repository by adding it to a future sources.list file under construction:
echo deb file:$TARGET_DIR/trinity-r14.0.0/debian wheezy main >>/etc/apt/sources.list.local echo deb file:$TARGET_DIR/trinity-builddeps-r14.0.0/debian wheezy main >>/etc/apt/sources.list.local
See also:
- forums.debian.net > Install TDE (Trinity Desktop Environment) on Jessie
virtualbox.org
VirtualBox is a multiplateform software to host various PC's operating systems.
## Oracle Corporation (comment: VirtualBox archive signing key) wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import - ## or: #gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 54422A4B98AB5139 debmirror -v --verbose --arch=i386,amd64 --host=download.virtualbox.org --root=:virtualbox/debian --dist=wheezy --section=non-free,contrib --postcleanup --source --method=http --diff=mirror $TARGET_DIR/download.virtualbox.org/virtualbox/debian
One can add that local repository by adding it to a future sources.list file under construction:
echo deb file:$TARGET_DIR/download.virtualbox.org/virtualbox/debian wheezy non-free contrib >>/etc/apt/sources.list.local
References
- debian.org > Chapter 2. Debian package management (fr)
- debmirror:
- help.ubuntu.com > debmirror
- Miroir Debian (fr)
- debmirror manpage : http://pwet.fr/man/linux/commandes/debmirror
- sourceforge.net > Digital Preservation Software Platform > update-deb-mirror
See also
- Aptly : (Debian repository management tool)
aptly is a swiss army knife for Debian repository management: it allows you to mirror remote repositories, manage local package repositories, take snapshots, pull new versions of packages along with dependencies, publish as Debian repository. :- Official site : www.aptly.info
- Github : github.com/smira/aptly
- packages.debian.org > packages.debian.org > aptly
Comments