As data is ingested into a database, it must be constantly rewritten for easy querying. Scylla writes incoming data to immutable files that must later be compacted into fewer files in order to maintain good read performance. The question becomes how fast should you compact? The traditional approach is to expose throughput tunables so the user can control the compaction speed. That means finding a good value involves a lot of trial and error. And what if the workload changes?
We take a different approach at ScyllaDB. We use the mathematical foundation of control theory to make automatic decisions about compactions, putting an end to compaction tuning altogether.
In this webinar you will learn:
- How we created mathematical models of compaction backlog
- How to use that model to feed a control theory framework that can automatically tune compactions.
- Other exciting developments that are coming in this area