Customer story

IHME Scales Global Health Analytics to Billions of Records with MariaDB

When data growth outpaced its MySQL environment, IHME turned to MariaDB columnar analytics to scale to petabytes of health data with no application rewrites.

About IHME

Commissioned by the World Bank, the Global Burden of Disease (GBD) project, led by the Institute for Health Metrics and Evaluation (IHME), analyzes disability and death from multiple causes worldwide. Originally relying on a MySQL-compliant infrastructure, IHME faced massive data growth over a short period:

2010: Approximately 2 billion data points.
2016: Results exceeded 10 billion per table.

Tasked with processing petabytes of health data, IHME needed a highly scalable, flexible, and cost-effective data warehouse and Big Data analytical engine.

Deciding factors: evaluation and selection

IHME initially invested heavily in performance-tuning their legacy database environment for their 16,000-CPU-core cluster but eventually hit scaling limits. Their key criteria for a new solution included cost-effectiveness, MySQL compatibility, and an open-source offering. They specifically did not want to rewrite vast amounts of application code.

They evaluated other options before deciding:

MemSQL performed well at scale, but relied on non-standard constructs for database design and lacked definable security mechanisms in their non-commercial product.
MySQL 5.7: Offered higher benchmark speeds, but could not support the transformational improvements needed for scalability and workload efficiency.

Ultimately, IHME needed a platform offering multi-host sharding. To avoid making all applications shard-aware with a small database team, they selected MariaDB.

“MariaDB, with the ColumnStore storage engine, solved both a volume and scale problem in our environment that allows us to seamlessly handle both current and planned increases in workload.”

Andrew Ernst

Former Assistant Director, Infrastructure, IHME

Architectural Solution: MariaDB Enterprise Platform

MariaDB Cloud stood out because it runs on AWS, fitting seamlessly into Flightradar24’s existing cloud footprint while providing capabilities that RDS and self-managed alternatives could not.

By deploying MariaDB Platform—a hybrid transactional and analytical processing (HTAP) architecture—IHME achieved substantial core performance and operational improvements:

Analytical Performance Optimization: The columnar storage engine minimizes disk I/O, outperforming traditional row-based engines during intensive read operations on massive datasets.
Footprint Reduction & Resource Efficiency: Advanced columnar compression reduced the physical storage footprint to approximately 40% of original capacity, significantly optimizing hardware utilization.
Enterprise Security Posture: ColumnStore integrates directly with MariaDB Server enterprise security capabilities, including wire-encryption for data in transit, role-based access control (RBAC), and comprehensive audit logging.
Unified SQL Frontend: Standard ANSI SQL support for dual OLTP and OLAP workloads allows IHME to execute analytics without modifying existing application logic
or codebase.