gnu/linux, debian

debmirror (tips and tricks)

debmirror is a program who aims to download the content of a debian repository in order to set a local mirror up and maintain it up to date.

The parameters are numerous and depend on each repository, so it is sometimes hard to find right settings on the web. That's why I gather here the parameters that were good for me + a touch of methodology.

Note: As I actually do not use debmirror, some informations on this page may be quite outdated.
— Médéric 02/12/2018 23:35

Getting the public GPG keys

debmirror runs in userspace and tries to get each public GnuPG key repository from the curent user trusted keyring ~/.gnupg/trustedkeys.gpg.

How to retrieve repository keys ?

Each GPG repository key must be requested by its ID to key servers which return the public key content if available:

Search for the public GPG key ID on the official repository web site
Send a query to a key server for that ID

The official repository procedures tell the root user to import the public key into the APT keyring, by providing the ID of the public GPG Key either like this:
apt-key adv --keyserver a-server.tld --recv-keys ID

or like this:
gpg --keyserver a-server.tld --recv-keys ID && gpg --export -a ID | apt-key add -

With the key ID and the server name ("a-server.tld"), the user can download the public repository GPG key into its own trusted keyring ~/.gnupg/trustedkeys.gpg either by importing first to its personal public keyring ~/.gnupg/pubring.gpg:

gpg --keyserver a-server.tld --recv-keys ID
gpg --export -a ID | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -

or directly to its trusted keyring ~/.gnupg/trustedkeys.gpg:

gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver a-server.tld --recv-keys ID

In most cases, the key server keys.gnupg.net will be sufficient to retrieve keys so that the only variable parameter will be the ID:
gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys ID

GPG keys may change so it's important to perform an update from time to time. A failsafe strategy is to request the public repository key before each debmirror session.

Rely on the APT trusted keyring

Export the APT trusted keyring to the user's one

If all the repositories that one want to mirror are already configured within the system (/etc/apt/sources.list and /etc/apt/sources.list.d/…), then the respectives public GPG keys should be up to date. In this case, it's convenient to update the user GPG trusted keyring by importing from the APT's one as follow:

gpg --keyring /usr/share/keyrings/debian-archive-keyring.gpg  --export | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import

Aternate way: using debmirror with APT keyring

An alternate way is to tell debmirror to look directly into the APT keyring:
debmirror --keyring /usr/share/keyrings/debian-archive-keyring.gpg ...

Check the repository key validation

Looking at standard output, debmirror shows if a GPG key is missing in ~/.gnupg/trustedkeys.gpg.

Example:

[GNUPG:] ERRSIG 8B48AD6246925553 1 8 00 1441453660 9
[GNUPG:] NO_PUBKEY 8B48AD6246925553
[GNUPG:] ERRSIG 7638D0442B90D010 1 8 00 1441453660 9
[GNUPG:] NO_PUBKEY 7638D0442B90D010
[GNUPG:] ERRSIG 6FB2A1C265FFB764 1 8 00 1441453675 9
[GNUPG:] NO_PUBKEY 6FB2A1C265FFB764
gpgv: keyblock resource `/home/demo/.gnupg/trustedkeys.gpg': file open error
gpgv: Signature made Sat Sep  5 13:47:40 2015 CEST using RSA key ID 46925553
gpgv: Can't check signature: public key not found
gpgv: Signature made Sat Sep  5 13:47:40 2015 CEST using RSA key ID 2B90D010
gpgv: Can't check signature: public key not found
gpgv: Signature made Sat Sep  5 13:47:55 2015 CEST using RSA key ID 65FFB764
gpgv: Can't check signature: public key not found
gpgv: keyblock resource `/home/demo/.gnupg/trustedkeys.gpg': file open error
gpgv: Signature made Sat Sep  5 13:47:40 2015 CEST using RSA key ID 46925553
gpgv: Can't check signature: public key not found
:Failed to download some Release or Release.gpg files!

In this case, one have to retrieve the public GPG keys whose IDs are:

46925553
2B90D010
65FFB764

Repository timeout

Warning!!
Some repositories have a validation timeout. In this case, if one exceeds that limit before updating the repository, all datas will be regarded as outdated and will be re-downloaded and overwritten!! This can lead an overload bandwidth!!

In this case, I advise to adjust the cron task that launches your debmirror script in order to save resources.

In the following examples, the base directory of the local mirrored repositories is stored in the shell variable TARGET_DIR.

If one want to know the repositories that have a timeout and their respectives values the following command do the trick:

find $TARGET_DIR -name Release -exec grep -HE '^Valid-Until:' "{}" \; | \
grep -v '/.temp/' | gawk -F '/' '{date=gensub(/^.*Valid-Until: */,"",$0); print $4": "date}'

Example:

find $TARGET_DIR -name Release -exec grep -HE '^Valid-Until:' "{}" \; | grep -v '/.temp/' | gawk -F '/' '{date=gensub(/^.*Valid-Until: */,"",$0); print $4" : "date}'
debian.org: Tue, 22 Sep 2015 02:42:42 UTC
debian.org: Tue, 22 Sep 2015 02:42:42 UTC
security.debian.org : Wed, 23 Sep 2015 20:01:59 UTC

Each day, the Debian repositories are refreshed and their timeout is set to 7 days up.

Debian

Official site: http://www.debian.org/
Debian Social Contract: https://www.debian.org/social_contract
Debian Wiki: https://wiki.debian.org/
Debian packages: https://www.debian.org/distrib/packages

If you're concerned with Debian, see also the Devuan project from the "Veteran Unix Admins" (VUA) team who aims to revert to an UNIX authentic distribution : http://devuan.org/ and newsletter_22dec.html.