System Design

Hardening C++ you can't drop yet: libc++ hardening cut Google's segfaults 30%

TL;DR

Google enabled libc++ hardening across server production: over 1,000 bugs found, segfaults down 30%, about 0.3% overhead. One build flag turns out-of-bounds bugs into safe crashes for the C++ you can't rewrite yet.

Moving to Rust is the long-term direction, but in practice every company still has millions, sometimes hundreds of millions, of lines of C++ that will live on for years. The practical question is: while we wait, how do we cut vulnerabilities in that very code? Google answered with a change that sounds tiny: turn on bounds checking in the C++ standard library (libc++ hardening) across its server fleet. The result: more than 1,000 bugs found, a 30% drop in production segfaults, at an average performance cost of just 0.30%.

The problem: standard C++ checks nothing

By default, v[i] on a std::vector does not check that i is in range, and *opt on an empty std::optional reads uninitialised memory. That is undefined behaviour, and the door to spatial-safety bugs: reading or writing outside the allocated region. According to Project Zero tracking cited by Google, this class accounts for about 40% of in-the-wild memory-safety exploits over the past decade.

libc++ hardening modes

ModeWhat it checksWhen to use
noneNothing (the default unless your vendor changes it)Not for new code
fastLow-cost, security-critical checks: valid element access, valid input rangesProduction (recommended by LLVM)
extensivefast plus: non-null, non-overlapping ranges, valid deallocation, compatible allocators…After benchmarking and accepting the cost
debugEvery check, including expensive heuristics and internal assertionsTest environments only

Enable it with one macro, defined before any #include (usually via a build flag):

clang++ -stdlib=libc++ \
  -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_FAST \
  main.cpp

When a check fails in fast or extensive mode, the program traps immediately by default. An out-of-bounds bug that could have been silently exploited becomes a loud, debuggable crash.

How Google rolled it out, and what it got

  • Path: more than a year in testing, a bake period in test and pre-production, then a gradual production rollout over several months. Chrome had enabled it since 2022.
  • Scope: now the default for server production, including Search, Gmail, Drive, YouTube and Maps; only a handful of components opted out.
  • Cost: 0.30% on average. Google also found and removed a redundant check (from 0.35% down to 0.30%) and upstreamed the fix to LLVM.
  • Benefit: over 1,000 bugs surfaced, some more than a decade old; an expected 1,000–2,000 new bugs prevented per year; production segfaults down 30%; one internal red-team exercise disrupted.

Risks and caveats

  • More crashes at first: old bugs that "sort of worked" now crash. That is why a long test period comes before production.
  • Only protects standard-library code: raw C arrays and hand-rolled pointer arithmetic are not checked. Use tools such as Clang's -Wunsafe-buffer-usage warning to find them.
  • Does not stop use-after-free: hardening addresses spatial safety; temporal safety (UAF) needs other mechanisms such as MiraclePtr, or Rust.
  • Observe mode: there is a log-and-continue mode, but continuing after a failed check is still undefined behaviour; use it only during adoption.
  • ABI: hardening modes don't change the ABI, but some checks (such as bounded iterators) depend on ABI options chosen when the library is built.

How to apply it

  • Dev: Turn on debug mode in CI and fast mode in release builds. Run your existing tests: every trap is a real bug waiting to be fixed.
  • Tech Lead: Follow Google's path: test, bake, then enable service by service, with an exemption process that records reasons. Measure on real workloads before concluding it is "too slow".
  • PM: This is the cheapest investment in a memory-safety strategy: no rewrite, under 1% cost, measurable benefit right away. Put it on the roadmap before debating a Rust rewrite.

Sources & references

  1. Retrofitting spatial safety to hundreds of millions of lines of C++ — Google Security Blog
  2. Hardening Modes — LLVM libc++ documentation