Assorted links for Thursday, June 20:
- How we improved push processing on GitHub
A push triggers a Kafka event, which is fanned out via independent consumers to many isolated jobs that can process the event without worrying about any other consumers.
- Leveraging Rust in High-Performance Web Services
Rust’s ownership model is a fundamental feature that enhances both speed and safety. Every value in Rust has a unique owner, responsible for its cleanup when it’s no longer needed. This eliminates the need for a garbage collector and ensures efficient memory management. The ownership rules are enforced at compile time, which means there’s no runtime overhead.
- systemd 256 Released With run0, systemd-vpick, importctl & Other New Features
- Maintaining large-scale AI capacity at Meta
Outside of special cases, Meta maintains its fleet of clusters using a technique called maintenance trains. This is used for all capacity, including compute and storage capacity. A small number of servers are taken out of production and maintained with all applicable upgrades. Trains provide the guarantee that all capacity minus one maintenance domain is up and running 24/7, thus providing capacity predictability. This is mandatory for all capacity that is used for online and recurring training.
- How Meta trains large language models at scale