fedora-createrepo-image/docs/scripts.md
XLion 964aa339d1
Set rpm's default backend to sq; sign rpm to both v4 and v6 format; import key to sq; (#525)
* Add sq

* repomd sq

* shellcheck

* fix: request both v4 and v6 RPM signatures

* fix: complete Sequoia signing contract

* fix: require RPM v4-compatible signing keys

* docs: streamline agent guidance and track downstream compatibility

* fix: reject invalid RPM verification and match explicit signing keys

* refactor: share RPM signing command across retries

* refactor: simplify key import and signing test fixtures
2026-10-28 10:32:32 +08:00

20 KiB

Scripts

Overview

Files under scripts/ are copied into /usr/bin by the container image. Package repository workflows use these commands as CI building blocks.

The scripts should stay small. They should wrap one clear responsibility, expose stable flags, and avoid hiding repository-specific decisions behind a generic abstraction.

General Design

CLI Conventions

  • Use Bash for small orchestration around existing CLI tools.
  • Start scripts with #!/bin/bash and set -euo pipefail unless there is a concrete reason to use something else.
  • Keep option names descriptive and stable; let callers supply environment variable names, such as --key-env GPG_PRIVATE_KEY.
  • Validate required dependencies near startup and report missing tools clearly.
  • Validate required inputs before network requests, signing, or destructive file operations.
  • Prefer explicit arguments for caller-specific behavior.

Keep Scripts Focused

Each script should have one main job.

Good examples:

  • xlion-repo-utils-gh handles GitHub release metadata and checksum verification.
  • xlion-repo-gpg-import imports a private key into GnuPG and Sequoia and reports the fingerprint.
  • xlion-repo-repackage-deb repackages DEB files with consistent fpm options.
  • xlion-repo-repackage-rpm repackages RPM files with consistent fpm options.
  • xlion-repo-rpm-createrepo creates RPM repository metadata for one or more directories.
  • xlion-repo-rpm-sign-repomd signs RPM repository metadata for one or more directories.
  • xlion-repo-rpm-sign-packages signs RPM packages and verifies signatures.

Avoid scripts that do too much, such as one command that downloads assets, repackages packages, signs metadata, publishes repositories, and uploads artifacts. Those workflows are easier to reason about when they remain separate steps.

Keep Product Rules Near the Caller

Not every repeated shell line needs a shared abstraction. For example, RustDesk release assets have product-specific naming rules around architecture, sciter, nightly builds, and SUSE RPMs. Simple workflow filtering with grep is often clearer than a generic selector command with many flags.

Use shared scripts for stable mechanics:

  • GitHub release API access
  • checksum verification
  • GPG key import
  • package reversion or repackaging when the behavior is identical
  • RPM signing and metadata creation when those helpers exist

Keep caller-specific choices in the workflow:

  • which upstream repository to use
  • which release tag to use
  • which package names or architectures to include
  • how to lay out product-specific publish directories

stdout and stderr

If a script is useful in command substitution, stdout should be machine-readable.

For example, xlion-repo-gpg-import prints only the fingerprint to stdout. Status messages go to stderr. This allows safe usage like:

fingerprint="$(xlion-repo-gpg-import --key-env GPG_PRIVATE_KEY)"

Scripts that generate human-readable reports can write normal logs to stdout if they are not expected to be used this way. Be deliberate and document the behavior.

Secrets

Scripts must not print secrets. This includes:

  • GPG private keys
  • GitHub tokens
  • Cloudflare R2 credentials
  • full environment dumps

Accept secret material through environment variables or stdin. Prefer letting the caller choose the environment variable name with a flag such as --key-env.

Temporary State

Temporary state should have an obvious owner.

For GPG, GNUPGHOME must remain available to later steps that run gpg or aptly. RPM signing and RPM repository metadata signing use Sequoia, so SEQUOIA_HOME must remain available to later steps that run sq or the rpmsign sq backend. xlion-repo-gpg-import may create both homes, but it intentionally does not delete either one. With --github-env, it writes the paths to $GITHUB_ENV; GitHub Actions makes them available to subsequent steps. The workflow should remove both homes in an if: always() cleanup step after signing and publishing are finished.

Validation Requirements

For every changed Bash script, run bash -n scripts/SCRIPT_NAME and scripts/SCRIPT_NAME --help. Each script reference below lists its minimum commands and practical functional checks.

Use temporary local keys and packages for functional tests. Verify behavior without real signing secrets or repository publishing. For key import, confirm the fingerprint on stdout, private-key availability in both stores, and the documented GitHub environment/output entries. Keep the generated homes available for subsequent signing checks; the caller owns cleanup.

When adding or changing a script, update its reference here with its purpose, ownership boundaries, common examples, and minimum test commands.

Script Reference

xlion-repo-utils-gh

xlion-repo-utils-gh has two modes.

Fetch mode:

xlion-repo-utils-gh --workfolder "$WORKFOLDER" --repo rustdesk/rustdesk

Fetch a specific tag:

xlion-repo-utils-gh --workfolder "$WORKFOLDER" --repo rustdesk/rustdesk --tag nightly

Verify downloaded files:

xlion-repo-utils-gh --verify --workfolder "$WORKFOLDER"

Fetch mode writes:

  • api.json
  • dl_urls.txt
  • sha256.txt when release asset digests are available

Verify mode reads sha256.txt from the workfolder, checks only files actually downloaded there, and fails if any downloaded file has a checksum mismatch.

Typical caller flow:

xlion-repo-utils-gh --workfolder "$WORKFOLDER" --repo pendulum-project/ntpd-rs
cd "$WORKFOLDER"
grep -E '\.deb$|\.rpm$' dl_urls.txt | wget2 --input-file=-
xlion-repo-utils-gh --verify --workfolder "$WORKFOLDER"

What this script owns:

  • GitHub release API calls
  • writing release URL and checksum files
  • checksum verification

What this script does not own:

  • deciding which packages a product repo wants
  • downloading every possible asset automatically
  • repository metadata generation
  • package signing

Minimum tests:

bash -n scripts/xlion-repo-utils-gh
scripts/xlion-repo-utils-gh --help

If network access is acceptable, also test a small fetch-only scenario against a known public release.

xlion-repo-gpg-import

xlion-repo-gpg-import imports a private key into both GnuPG and Sequoia and prints the imported fingerprint. The two key stores are separate: GnuPG consumers such as aptly use GNUPGHOME, while RPM signing helpers use SEQUOIA_HOME for sq.

Recommended GitHub Actions usage:

xlion-repo-gpg-import \
  --key-env GPG_PRIVATE_KEY \
  --create-gnupghome \
  --github-env \
  --name GPG_FINGERPRINT

This writes these variables for later workflow steps:

GNUPGHOME=/tmp/...
SEQUOIA_HOME=/tmp/...
GPG_TTY=
GPG_FINGERPRINT=...

--github-env appends these values to $GITHUB_ENV; GitHub Actions applies that file when it starts the next step. The importer cannot change the environment of the step that invoked it. GNUPGHOME is created when it is unset and --create-gnupghome is given. SEQUOIA_HOME is created when it is unset. A caller-provided home is reused.

Use stdin for local testing or when the caller does not want to store the key in an environment variable:

xlion-repo-gpg-import --stdin --create-gnupghome < private-key.asc

Use RPM key import when later steps call rpmsign or verify RPM signatures:

xlion-repo-gpg-import \
  --key-env GPG_PRIVATE_KEY \
  --create-gnupghome \
  --github-env \
  --rpmkeys-import

What this script owns:

  • reading private key material from stdin or a caller-selected environment variable
  • creating GNUPGHOME when requested and creating SEQUOIA_HOME when needed
  • importing the private key with both gpg and sq
  • parsing and reporting the fingerprint
  • optionally importing the public key into rpm's key database

What this script does not own:

  • publishing APT repositories
  • signing RPM packages
  • signing repomd.xml
  • deleting GNUPGHOME or SEQUOIA_HOME; the calling workflow owns cleanup
  • choosing the secret name used by a workflow

Minimum tests:

bash -n scripts/xlion-repo-gpg-import
scripts/xlion-repo-gpg-import --help

Functional test with a throwaway key:

tmp_src="$(mktemp -d)"
tmp_env="$(mktemp)"
chmod 700 "$tmp_src"

GNUPGHOME="$tmp_src" \
  gpg --batch --passphrase '' \
  --quick-generate-key "XLion Repo Test <[email protected]>" \
  default default 1d >/dev/null 2>&1

key_data="$(GNUPGHOME="$tmp_src" gpg --batch --armor --export-secret-keys "[email protected]")"

fingerprint="$({
  GITHUB_ENV="$tmp_env" \
  GPG_PRIVATE_KEY="$key_data" \
  scripts/xlion-repo-gpg-import \
    --key-env GPG_PRIVATE_KEY \
    --create-gnupghome \
    --github-env
})"

test -n "$fingerprint"
grep -q '^GNUPGHOME=' "$tmp_env"
grep -q '^SEQUOIA_HOME=' "$tmp_env"
grep -qx 'GPG_TTY=' "$tmp_env"
grep -q '^GPG_FINGERPRINT=' "$tmp_env"

imported_home="$(grep '^GNUPGHOME=' "$tmp_env" | cut -d= -f2-)"
sequoia_home="$(grep '^SEQUOIA_HOME=' "$tmp_env" | cut -d= -f2-)"
GNUPGHOME="$imported_home" gpg --batch --list-secret-keys "$fingerprint" >/dev/null
test -d "$sequoia_home"
SEQUOIA_HOME="$sequoia_home" sq key list --cert "$fingerprint" >/dev/null

rm -rf "$tmp_src" "$imported_home" "$sequoia_home" "$tmp_env"

xlion-repo-repackage-deb

xlion-repo-repackage-deb repackages .deb files from a source directory into a target directory.

Latest-style repackaging without changing the package version:

xlion-repo-repackage-deb \
  --source-dir ori \
  --target-dir . \
  --deb-recommends xlion-repo-archive-keyring \
  --remove-source-dir

Nightly-style repackaging with a date suffix:

xlion-repo-repackage-deb \
  --source-dir ori \
  --target-dir . \
  --date-version \
  --deb-recommends xlion-repo-archive-keyring \
  --remove-source-dir

This turns an original version such as 1.2.3 into 1.2.3+YYYYMMDD.

What this script owns:

  • scanning one source directory for .deb files
  • reading original package versions with dpkg-deb
  • calling fpm -t deb -s deb
  • applying --deb-compression
  • optionally adding --deb-recommends
  • optionally removing the source directory after success

What this script does not own:

  • downloading upstream release assets
  • deciding which architecture packages should exist
  • creating APT repository metadata
  • signing APT repositories

Minimum tests:

bash -n scripts/xlion-repo-repackage-deb
scripts/xlion-repo-repackage-deb --help

Functional testing should use a small throwaway .deb package. Confirm that the repackaged output exists, and when --date-version is used, confirm the output package Version contains +YYYYMMDD.

xlion-repo-repackage-rpm

xlion-repo-repackage-rpm repackages .rpm files from a source directory into a target directory. By default, the target directory is the parent of the source directory. This matches common wwwroot/channel/ori usage.

Latest-style repackaging without changing the package version:

xlion-repo-repackage-rpm \
  --source-dir wwwroot/latest/ori \
  --remove-source-dir

Nightly-style repackaging with a date suffix:

xlion-repo-repackage-rpm \
  --source-dir wwwroot/nightly/ori \
  --date-version \
  --remove-source-dir

This turns an original version such as 1.2.3 into 1.2.3+YYYYMMDD.

What this script owns:

  • scanning one source directory for .rpm files
  • reading original package versions with rpm -qp
  • calling fpm -t rpm -s rpm
  • applying --rpm-compression
  • optionally removing the source directory after success

What this script does not own:

  • splitting SUSE and non-SUSE packages
  • creating RPM repository metadata
  • importing GPG keys
  • signing RPM packages
  • signing repomd.xml

Minimum tests:

bash -n scripts/xlion-repo-repackage-rpm
scripts/xlion-repo-repackage-rpm --help

Functional testing should use a small throwaway .rpm package. Confirm that the repackaged output exists, and when --date-version is used, confirm the output package version contains +YYYYMMDD.

xlion-repo-rpm-createrepo

xlion-repo-rpm-createrepo runs createrepo_c for one or more RPM repository directories. Multiple --dir values are processed concurrently, and each background job is checked with wait.

RustDesk RPM-style usage:

xlion-repo-rpm-createrepo \
  --dir wwwroot/latest \
  --dir wwwroot/latest-suse \
  --dir wwwroot/nightly \
  --dir wwwroot/nightly-suse

Single-directory usage:

xlion-repo-rpm-createrepo --dir ~/.aptly/public/rpm

Pass extra arguments to createrepo_c after --:

xlion-repo-rpm-createrepo --dir wwwroot/latest -- --update

What this script owns:

  • validating that each repo directory exists
  • warning when a repo directory has no direct .rpm files
  • running createrepo_c for each directory
  • running multiple directories concurrently
  • failing if any createrepo_c process fails

What this script does not own:

  • splitting product-specific RPM directories
  • signing RPM packages
  • signing repomd.xml
  • uploading repository artifacts

Minimum tests:

bash -n scripts/xlion-repo-rpm-createrepo
scripts/xlion-repo-rpm-createrepo --help

Functional testing should create one or more throwaway directories containing small RPM packages, run the helper, and confirm each directory contains repodata/repomd.xml.

xlion-repo-rpm-sign-repomd

xlion-repo-rpm-sign-repomd signs repodata/repomd.xml with Sequoia's sq backend for one or more RPM repository directories. Multiple --dir values are processed concurrently, and each background job is checked with wait.

The helper requires sq and checks that it is available before signing.

RustDesk RPM-style usage:

xlion-repo-rpm-sign-repomd \
  --dir wwwroot/latest \
  --dir wwwroot/latest-suse \
  --dir wwwroot/nightly \
  --dir wwwroot/nightly-suse \
  --gpg-key "$GPG_FINGERPRINT"

Single-directory usage:

xlion-repo-rpm-sign-repomd --dir ~/.aptly/public/rpm --gpg-key "$GPG_FINGERPRINT"

The --gpg-key value can be an OpenPGP key ID or fingerprint; a full fingerprint is recommended. The helper passes it to sq as --signer. The --gpg-key flag name is retained for compatibility with existing workflows.

What this script owns:

  • validating that each repo directory exists
  • validating that each repodata/repomd.xml exists
  • signing each repomd.xml with sq --batch --overwrite sign
  • running multiple directories concurrently
  • failing if any signing process fails

What this script does not own:

  • importing GPG keys
  • importing the signing key into Sequoia
  • choosing the signing key secret name
  • signing RPM packages
  • creating RPM repository metadata

Minimum tests:

bash -n scripts/xlion-repo-rpm-sign-repomd
scripts/xlion-repo-rpm-sign-repomd --help

Functional testing should use a throwaway OpenPGP key imported into a throwaway Sequoia key store and a throwaway RPM repo directory with repodata/repomd.xml. Set SEQUOIA_HOME to that store when needed, then confirm that repomd.xml.asc is created.

xlion-repo-rpm-sign-packages

xlion-repo-rpm-sign-packages signs .rpm files in one or more directories with Sequoia's sq backend, verifies the resulting signatures with rpmkeys -Kv, and retries signing when verification fails. It captures each rpmkeys -Kv result once and rejects any non-zero exit or reported BAD/NOT OK result. A package is accepted only when a signature line reports OK, includes the requested key in its explicit fingerprint or key ID field, and uses a supported RPM v4 signing algorithm (RSA, DSA, ECDSA, or EdDSA); this is the signing-algorithm compatibility gate.

Signing uses the Sequoia sq backend and RPM 6's --rpmv4 --rpmv6 options to request both RPM signature formats, including during retries. The signing key must already be imported into sq, and SEQUOIA_HOME must point to that key store when it is not the default. The --gpg-key value should be an OpenPGP key ID or fingerprint; a full fingerprint is recommended.

RPM v4 compatibility requires a signing algorithm supported by RPM v4. The helper uses the matching rpmkeys -Kv signature algorithm token as this compatibility gate and rejects other algorithms, such as ML-DSA. Key matching is case-insensitive and exact. It accepts full fingerprint equality; it derives a 16-hex-character key ID from a v4 fingerprint's final 64 bits or a v6 fingerprint's first 64 bits. An 8-hex-character legacy ID matches the final eight characters of that 16-character key ID. A v3 fingerprint is matched only in full because its key ID comes from the RSA modulus. rpmsign --rpmv4 --rpmv6 remains responsible for RPM v4 and v6 signature storage.

RustDesk RPM-style usage:

xlion-repo-rpm-sign-packages \
  --dir wwwroot/latest \
  --dir wwwroot/latest-suse \
  --dir wwwroot/nightly \
  --dir wwwroot/nightly-suse \
  --gpg-key "$GPG_FINGERPRINT"

Single-directory usage:

xlion-repo-rpm-sign-packages --dir ~/.aptly/public/rpm --gpg-key "$GPG_FINGERPRINT"

Retry options:

xlion-repo-rpm-sign-packages \
  --dir wwwroot/latest \
  --gpg-key "$GPG_FINGERPRINT" \
  --retry 3 \
  --retry-sleep 2

What this script owns:

  • collecting direct child .rpm files from one or more directories
  • signing each RPM with rpmsign --key-id --rpmv4 --rpmv6 using sq
  • requiring successful overall rpmkeys -Kv verification and a matching signature line for the requested key with an RPM v4-supported algorithm (RSA, DSA, ECDSA, or EdDSA)
  • verifying signatures with rpmkeys -Kv
  • checking that the signature line's explicit fingerprint or key ID field matches the requested --gpg-key
  • retrying signing when verification fails
  • checking that find, rpmkeys, rpmsign, sleep, sort, and sq are available

What this script does not own:

  • importing the OpenPGP signing key into Sequoia
  • importing public keys into rpmkeys
  • creating RPM repository metadata
  • signing repomd.xml
  • deciding whether PR workflows should skip signing

Minimum tests:

bash -n scripts/xlion-repo-rpm-sign-packages
scripts/xlion-repo-rpm-sign-packages --help
tests/test-rpm-sign-packages.sh

The regression test runs the public CLI with temporary controlled rpmkeys, rpmsign, and sq commands in a temporary directory. It covers v3/v4/v6 fingerprint and key ID matching, key ID collisions, unsupported algorithms, failed verifier output/status, payload digest failures, invalid-key rejection before signing, and the retry path; all fixtures are removed when it exits.

Keep RPM package signing sequential unless there is a proven need to parallelize; signing is more sensitive to concurrent access than metadata generation.

Functional testing should use a throwaway RPM and a throwaway RSA signing key. Import the private key into sq and the public key into rpmkeys before running this helper, then confirm rpmkeys -Kv reports the expected key with a supported algorithm and OK status and the helper accepts it. Test a key whose matching output has an unsupported algorithm such as ML-DSA and confirm that the helper rejects it. Also reject a supported algorithm with the wrong fingerprint or a NOT OK status. Exercise each of the RPM v4-supported signing algorithms (RSA, DSA, ECDSA, and EdDSA) where available; rpmsign --rpmv4 --rpmv6 remains responsible for signature storage.

Adding a New Script

Before adding a new script, check whether it is actually a stable shared mechanic. If the logic is mostly product-specific filtering or directory layout, keep it in the calling workflow.

When a new script is justified:

  • Add it under scripts/ with an executable bit.
  • Include --help output.
  • Validate dependencies and inputs.
  • Keep stdout behavior documented.
  • Add or update documentation in this file.
  • Run bash -n and at least one practical functional test.

Container Integration

The Dockerfile currently contains:

COPY scripts/* /usr/bin

That means every file under scripts/ becomes available in the image PATH. Do not add non-executable notes, fixtures, or test data under scripts/; put documentation under docs/ instead.