One feature our users have been asking us ever since we officially launched over a year ago was to provide binary updates for base and kernel. We are excited to announce that we are launching the framework for binary updates today! We still need to tie in the update build script to our continuous integration infrastructure. For now, updates for the hardened/current/master branch of the HardenedBSD repo will be done manually. When we create the next installers/distsets for the HardenedBSD-stable repo, we'll also support updates there. You will notice two new programs, /usr/sbin/hbsd-update and /usr/sbin/hbsd-update-build, which apply and build update packages, respectively. This work was sponsored by G2, Inc, who has an immediate need for binary updates.

Please note that this feature is still experimental.

Here's the design of the update mechanisms:

Goals

1. Provide binary updates for base and kernel
2. Cryptographically sign binary updates
3. Support mirrors and a massively scalable design
4. Easy maintenance
5. Supports jails
6. Support ZFS BEs

Goal 1 - Provide binary updates for base and kernel
This can be achieved with base.txz and kernel.txz. We would combine both of these tarballs into a new tarball that contains metadata about files to skip (like /etc/master.passwd), obsoleted files, etc. Building the update archive will require git. The signature files listed below will be discussed later.

Example tarball contents (in no particular order):

• base.txz
• kernel-HARDENEDBSD.txz
• etcupdate.tbz
• skip.txt
• UPDATING-HardenedBSD.txt
• base.txz.sig (Cryptographic signature for base.txz)
• kernel-HARDENEDBSD.txz.sig (cryptographic signature for kernel-HARDENEDBSD.txz)
• skip.txt.sig (Cryptographic signature for skip.txt)
• etcupdate.tbz.sig (Cryptographic signature for etcupdate.tbz)
• pubkey.asc (X.509 public key used to sign the artifacts)

Adding the kernel name to the kernel tarball also allows us to support multiple kernels in a given update package. A user can therefore choose which kernel to be installed.

Goal 2 - Cryptographically sign binary updates

In order to provide assurance of data integrity and authenticity, all artifacts that alter the filesystem need to be signed. The digital signatures will be detached from the files they pertain to. As OpenSSL is included in base, rely on that to perform the signature procedures.

Unsigned updates will be supported by a command-line argument. Cryptographic signing will be enforced by default.

In order to support third-party signatures along with certificate chaining, HardenedBSD will use a standard that allows for a form of PKI. That standard will be RSA keys embedded within X.509 certificates. A trusted keystore will need to be maintained by the sysadmin/user of the system. This is no different in concept than what exists for existing FreeBSD utilities like pkg(7).

By default, the trusted root store will be /usr/share/keys/hbsd-update/trusted. This directory can be overridden in a client's configuration file. OpenSSL will be instructed to load the trusted root certificate and validate public keys based on the loaded root certificates.

Upon downloading the update tarball, the tarball will be unpacked and the public cert extracted. Any file that ends in .sig will be validated with that public cert that matches against a certificate in the trusted root certificate store. If the public cert does not validate or if the signature doesn't validate, an error will be thrown and the user will be forced to diagnose.

Distributing the public cert used for signing the artifacts along with the artifacts allows a distributor to immediately and efficiently update key material in case of private key compromise or expiration. Compromise or expiration of the trusted root certificate is a much more serious issue that should be resolved by human intervention.

Validation steps:

1. Extract main tarball
2. Validate public cert against trusted root store
1. If public cert does not belong to any root cert in the trust store, gracefully fail with an error message
3. Validate each file that causes changes to the system has a corresponding .sig file
1. If signature doesn't match, gracefully fail with an error message
4. Continue on with update application

Goal 3 - Scalability

This goal is relatively easy to achieve. Given that only a single tarball is produced, the tarball can be distributed across many nodes. To distribute version information to users regarding updates, two options are available: propigation through DNS TXT records or a file containing the version number information is placed in the root of the URL that contains all update packages.

Option 1 - DNS

DNS TXT records will be used to distribute version information. This allows for bandwidth relief by using existing massively-distributed systems. It also allows us to somewhat stagger updates due to DNS record caching. The drawback to this could be an attacker having control over DNS records preventing a user from applying updates. However, if an attacker can control a client's DNS results, the client has much bigger problems to solve. DNSSEC may be able to help mitigate these issues.

The TXT record name is structured such that the update publisher can publish different update archives for different repos and branches. For example, update archives for amd64 systems running the bleeding-edge development branch of HardenedBSD's main repo would check the DNS entry amd64.master.current.hardened.hardenedbsd.updates.hardenedbsd.org. Update archives for amd64 systems running the 10-stable branch of HardenedBSD's stable repo would check the DNS entry for amd64.master.10-stable.hardened.hardenedbsd-stable.updates.hardenedbsd.org. The DNS record names may seem redundant and cumbersome, but they allow for flexibility. Third-party update publishes do not need to follow this example.

The TXT record values will be pipe-delimited.

Current TXT record fields:

1. Timestamp
2. Git object hash

Option 2 - Version number in file
The same exact text that appears in the DNS TXT record will be placed in a file called update-latest.txt. That file will be uploaded to the same directory as the update archive. This will allow users to determine via an HTTP GET (if using HTTP) request what the latest version number is.

Goal 4 - Easy Maintenance

There will be one script for building the update tarball along with its artifacts. It will only rely on /bin/sh, OpenSSL, and git. Everything will be automated with the exception of DNS updates. However, the tarball build script will provide output on stdout that will lend itself to DNS automation. The script for applying updates will only require applications in base. The sole exception to that will be the optional use of the beadm application to manage ZFS BEs.

Goal 5 - Jail support

Follow the example of pkg(7) and support updating jails from the host.

Goal 6 - ZFS Boot Environment support

Optionally install updates into a newly-created Boot Environment (BE). This feature will require the beadm tool, available in our package repo. Since this requires a third-party tool, make this feature optional via a command-line flag.