HardenedBSD March 2024 Status Report

What a busy month it has been! And not just for HardenedBSD, but for the rest of the security and IT industries as we work through the xz backdoor (CVE-2024-3094).

In src, the hbsdcontrol utility, and the library implementing the core logic (libhbsdcontrol) were rewritten from the ground up. While the implementation is now feature complete, there is still a bit of work to be done. Chiefly, rewriting the manual pages. After the documentation is updated, I plan to also integrate libucl support, to support JSON output and perhaps also support applying rules specified by a configuration file.

In ports, www/firefox was fixed and the minimum llvm version number was bumped for devel/boost.

Updates were applied across the entire infrastructure. A new build of hbsdfw (a HardenedBSD-basd fork of OPNsense) was deployed. This build has some issues, so I would recommend others not to deploy it, though it works fine enough for us to keep this current build deployed.

Here's what to look for in April:

  1. Continued work on {,lib}hbsdcontrol.
  2. I'm hoping to study more the dance between the CSU, libc, libthr, and the RTLD.
  3. More work on libhijack, perhaps a new shim library that gets injected to help aid further process injection work.

And, lastly:


$ fetch -q -o - https://api.github.com/repos/HardenedBSD/HardenedBSD | jq -r .created_at
2014-04-08T10:10:24Z

Happy birthday, HardenedBSD! May the next decade be as impactful as the previous.

HardenedBSD Unaffected By CVE-2024-3094 (Backdoor in xz/lzma 5.6.0/5.6.1)

A backdoor targeting amd64 linux glibc based systems was recently found in the xz project. The versions of xz impacted are 5.6.0 and 5.6.1.

Neither FreeBSD nor HardenedBSD are directly affected by this issue. However, I suspect that those running an amd64 linux glibc jail on FreeBSD (or HardenedBSD) have the potential to be affected.

Note that the linux.ko and linux64.ko kernel modules are tagged as insecure/untrustworthy by default in HardenedBSD. Those wishing to deploy a Linux environment on HardenedBSD must explicitly enable the Linux syscall translation kernel modules (linux.ko and linux64.ko).

HardenedBSD February 2024 Status Report

I spent most of February getting 15-CURRENT working again. FreeBSD introduced a new library, libsys, which is where the userland side of performing syscalls is implemented. There's an intricate dance between libsys, libc, libthr, and the CSU. I spent some time learning about that dance, and I still feel like there's more to learn.

HardenedBSD 15-CURRENT is mostly fixed. Prior to the libsys change, we built libc with Link-Time Optimizations (LTO). Building libc with LTO was part of the problem, though not the only issue. Once all the issues are resolved, I will re-enable LTO for libc.

FreeBSD also introduced a new pam_xdg(8) PAM module. This module had a few vulnerabilities, which are fixed in HardenedBSD. The two NULL deref bugs are fixed in FreeBSD now, too. The filesystem race condition and recursion limit issues are somewhat mitigated in HardenedBSD, but not completely.

HardenedBSD now has two VisionFive StarFive2 64-bit RISCV SBCs. I spent a little bit of time toying around with them. The kernel boots to the mountroot prompt. I've been wanting to learn hardware hacking, including writing drivers, so these little SBCs might be great for that.

In ports:

  1. the u-boot ports are fixed
  2. dns/unbound was updated to 1.19.1
  3. the net/vnstat port was fixed
  4. graphics/mupdf is now built as a PIE with SafeStack enabled
  5. The secadm ports were updated

Looking forward into March: I'm hoping to close two gaps of knowledge: the dance mentioned above and I'd like to return to jemalloc hardening. I plan to also do some infrastructure maintenance--routine updates.

HardenedBSD January 2024 Status Report

Happy new year! January saw a few changes in the src repository:

  1. Update some internal netlink(4) functions to use an unsigned integer for buffer size.
  2. The NETLINK option is now effectively required by our upstream FreeBSD. I would encourage the community to study its code for potential issues.
  3. The sysctl knobs vm.objects and vm.swap_objects are now only made available to privileged unjailed processes.
  4. A new sysctl node is introduced that disables new USB device connections: hardening.pax.prohibit_new_usb. Possible values to set it to:
    • 0: disabled
    • 1: enabled
    • 2: enabled without possibility to disable without incurring a reboot

    It would be cool to see a new option: 3, enforce a USB device allowlist. I would like to delegate that to anyone who wants to volunteer to do that work. :-)

In ports:

  1. An unneeded patch for dns/c-ares was remooved now that the port has been updated by upstream
  2. Fix build of devel/ivykis
  3. Bump default llvm ports version to 17

In the infrastructure:

As detailed in this announcement, HardenedBSD will slow its OS/update build cadence from bi-weekly to monthly.

The issue plaguing git clone https://... of src or ports has been resolved! Major thanks to h3artbl33d for helping resolve this issue.

Change in Build Cadence

Building packages takes quite a while, especially for 14-STABLE. The last package build for that branch took 371 hours, around 15-ish days. We currently build installer images and base OS updates every 14 days. This means that the 14-STABLE package builds will always be at least one build behind.

So, we need to slow the cadence of our installer image and base OS update builds. Instead of building installer images and base OS updates bi-weekly, we will move to monthly builds.

This will allow us to, at the start of every month, perform a clean package build. After that initial clean package build completes and is synced, we can then perform incremental package builds for the remainder of the month.

There are still a few potential issues with this. Sometimes, a port gets updated that 20,000+ other ports depend on, effectively meaning that we're doing a clean build even though an incremental build was requested.

If an incremental package build is unable to complete by the time the 1st day of the next month comes around, we will cancel that build in favor of starting the clean build on the 1st.

Slowing the frequency to monthly for the installers and base OS updates also opens up new opportunities for the community to explore. For example, we would welcome generation of .torrent files. Who knows, maybe we could help perform the initial seeding. :-)

These changes will take place on 01 Feb 2024.

HardenedBSD December 2023 Status Report

Happy new year! In December, the focus was on finishing the move to our new home. Updates were applied across the entire infrastructure.

In src, we reverted commit 8cf35a2cbe0270666845a5f2634cfc50c90696f1, which had originally set the default umask to 027. This is a bit too strict to work in the real world as a default for HardenedBSD, so we chose to revert back to 022.

In ports, textproc/jq was updated to 1.7.1. CFI and SafeStack were enabled for textproc/jq. ports-mgmt/poudriere-hbsd was updated to 3.4.

The secadm project was brought up-to-date to account for recent VFS-related changes in FreeBSD. The ports entry was updated accordingly.

While not an official HardenedBSD project, libhijack has given inspiration to new hardening techniques. libhijack is a post-exploitation tool that can be used to inject arbitrary code and hook dynamically-loaded functions. It's a tool that I created with origins back to the early 2000's. libhijack can now inject shared objects over the ptrace boundary anonymously. For those curious, libhijack can be found here: https://github.com/SoldierX/libhijack

HardenedBSD November 2023 Status Report

The focus in November was navigating and closing the purchase of a new home. I worked a little bit on the change to jemalloc that will optionally return NULL for zero-sized allocation requests. I realized that I lack a lot of knowledge on how jemalloc works and I need to fill those gaps fully before landing this feature.

I worked a bit on Cross-DSO CFI, including working in the ports tree. I started toying around a little with applying CFI (via LLVM kCFI) to select kernel modules. Some work needs to happen in the kernel ELF linker to support newer relocation types ld.lld emits when linking with -fsanitize=kcfi.

In src:

  1. The vfs.zfs.bclone_enabled sysctl tunable is set to 0 by default. We hope to re-enable it after a long soak time in FreeBSD.
  2. Generation of OpenSSH RSA host keys is disabled by default.

In ports:

  1. Loic F fixed the lang/gcc11 and lang/gcc12 ports
  2. Shawn Webb fixed the databases/postgresql*-server ports
  3. Shawn Webb fixed dns/void-zones-tools
  4. Shawn Webb added a new port: security/evilginx2

IMPORTANT INFRASTRUCTURE NOTE:

I'm hoping to move the HardenedBSD development/build infrastructure over to the new house as soon as this weekend. Electrical work still needs to happen, but I might have a workaround available until a proper solution is in place.

If the move does not happen this weekend (02-03 Dec 2023), then it will definitely happen the following weekend.

Other projects:

While writing this very status report, I've kicked off another build of hbsdfw. This includes the latest ZFS changes from upstream and more OpenSSL fixes. I'm hoping to have it tested and uploaded this weekend.

Once I have the jemalloc feature sorted out, I plan to resume work on libhijack. I've been letting my brain think about what needs to happen next over the past few months. Implementing an RTLD over the PTrace boundary is a bit more difficult than one might think. :-)

I've also started writing a little HardenedBSD testing framework. This will help us identify and resolve regressions (like the PaX NOEXEC regression I still have yet to fully resolve.)

If you would like to help with HardenedBSD development, but aren't sure where to start, there's this nifty issue board that shows all the bugs, features, and other work we would like help with: https://git.hardenedbsd.org/groups/hardenedbsd/-/boards/11

HardenedBSD October 2023 Status Report

This status is going to be pretty short. The focus in October has been on the home purchase.

News regarding the home purchase: if things go according to plan, we will take possession of the property towards the end of November 2023 (this month).

In ports:

  1. Loic F fixed archivers/rpm4
  2. Loic F fixed graphics/sane-backends
  3. Shawn Webb fixed lang/perl5.36
  4. Shawn Webb updated ports-mgmt/pkg to 1.20.8
  5. Shawn Webb "fixed" audio/pulseaudio by ripping out SIMD support. A more proper fix is needed. If the community can help us re-gain SIMD support for this port, that would be very much appreciated.

I worked on hbsdfw a little bit, figured out some of the issues in building the base OS packages. Now I need to figure out why some ports fail to build.

For November, I'm working on a new malloc.conf(5) option: bool nullonzero, default: false. This malloc.conf(5) option will cause malloc(3)-and-friends to return NULL when a zero size is passed in. This would prevent a class of integer overflow vulnerabilities that can lead to buffer overflows.

For example, in the common OpenSolaris CTF code (used in DTrace), we find the following function:

    char *
    xstrndup(char *str, size_t len)
    {
        char *newstr;
        
        /*
         * HardenedBSD note:
         *
         * If len happens to be SIZE_MAX, the size passed to malloc is zero.
         */
        if ((newstr = malloc(len + 1)) == NULL)
            memory_bailout();
    
        /*
         *HardenedBSD note:
         *
         * malloc(0) happily handed us a non-NULL pointer, sized zero. We now
         * copy SIZE_MAX bytes from str to newstr.
         */
        (void) strncpy(newstr, str, len);
        newstr[len] = '\0';
    
        return (newstr);
    }

HardenedBSD would prefer developers get used to providing reasonableness checks regarding size calculations. This xstrndup function could include this defensive coding technique of applying reasonable checks like so:

    char *
    xstrndup(char *str, size_t len)
    {
        char *newstr;
    
        if (str == NULL || len + 1 == 0) {
            return (NULL);
        }
    
        if (len == 0 && str[0] != '\0') {
            return (NULL);
        }
    
        /*
         * If len is still 0, that means we're duplicating an empty string.
         * That's fine, since strdup("") is a thing.
         */
        
        if ((newstr = malloc(len + 1)) == NULL)
            memory_bailout();
    
        (void) strncpy(newstr, str, len);
        newstr[len] = '\0';
    
        return (newstr);
    }

These new reasonable checks ensure that sleepy, caffeine-deprived developers have a harder time making understandable mistakes. We want to encourage safe and defensive programming techniques, especially before we launch this feature. I will fix the issues that come up in the base OS. When it lands, this feature will be defaulted to disabled, but the reasonableness checks and fixes will remain in place regardless.

So that's a preview of what's going on. I hope to land this before the end of the month. Perhaps mid-2024 we'll have it enabled by default.

HardenedBSD September 2023 Status Report

The HardenedBSD 14-STABLE build infrastructure is back online. A new package build is running. I apologize for the outage, and I appreciate the patience.

My wife and I are investigating some potential opportunities to purchase a home and plant our roots in Colorado. There is a chance we might significantly accelerate our plans at purchasing a home, moving the date from around June 2024 to even potentially November or December 2023. Should things go the way I'm thinking they might, the downtime for the HardenedBSD infrastructure would be limited to a single weekend, perhaps even a single Saturday.

We would like to ask for more public mirrors. Please reach out to netops@hardenedbsd.org if you would like to mirror our installation media and OS update artifacts. This may be especially useful in case we find unexpected dead trees in the metaphorical forest of purchasing a new-to-us home.

In HardenedBSD's src repo:

  1. A conditional in the virtual memory subsystem pertaining to our PaX NOEXEC-inspired strict W^X implementation. I suspect there may be one or two more conditionals to double-check.
  2. The output provided by the `newvers.sh` build script should be more correct.

In the ports tree:

  1. First-time submitter Shion Yorigami provided a fix for lang/gcc-aux.
  2. Shion Yorigami provided a fix for security/py-cryptography.
  3. Shawn Webb patched ports-mgmt/poudriere-hbsd to take into account the hardening of the vfs.lookup_cap_dotdot and vfs.lookup_cap_dotdot_nonlocal sysctl nodes.
  4. ports-mgmt/pkg is now built with PIE and LTO.
  5. devel/boost-libs now builds.
  6. math/symengine now builds.
  7. The default version of llvm in ports was bumped from 15 to 16. Because we build base system libraries with LTO, the default minimum ports llvm version needs to match the base llvm version.

Additional infrastructure info: the rsync service was moved to a new VM to account for the additional 14-STABLE build artifacts. I hope to deploy the Tor Onion Service endpoints for the 14-STABLE build infrastructure this week.

I also worked a bit on hbsdfw, forward-porting the changes to its HardenedBSD 14-STABLE feature branch. I'm still hoping to get a new build out soon-ish, but it is indeed taking longer than I originally anticipated.

We still have a number of ports that are broken, graphics/sane-backends being broken prevents editors/libreoffice from building. I'm hoping we can get some help from the community in fixing broken ports. I really appreciate those who contribute, no matter the form of contribution--code patches, advocacy, funding, documentation, etc. It's all equally important and very much appreciated.

Pages

Subscribe to HardenedBSD RSS