What’s the best cloud for OLTP?
Connecting state and local government leaders
A new report can help agencies assess the tradeoffs between AWS, Azure and the Google Cloud Platform when it comes to running online transaction processing applications.
When it comes to throughput, CPU and disk performance and network latency, not all clouds perform equally. A new report evaluated Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP) in four areas to help agencies assess the performance tradeoffs within each cloud and each cloud’s machines for online transaction processing (OLTP) applications.
“The 2021 Cloud Report” by Cockroach Labs found that GCP provides the fastest throughput, or processing rates. For the first time, GCP provided the highest amount of raw throughput, and for the third year that the organization has put out the report, Google won the network throughput benchmark, delivering nearly triple the speed of AWS and Azure. Even worst-performing GCP machine outpaced the other two’s best performers, the report notes.
The researchers assessed each cloud’s CPU performance using the CoreMark Version 1.0 benchmark, an open source, cloud-agnostic standard. They looked at the average number of iterations per second for single-core and 16-core runs. Intel was the clear winner in the single-core category, but AWS’s custom Graviton2 Processor beat out GCP and Azure’s machines.
AWS also took the top spot for latency for the third year in a row. Its top-performing machine’s 99th percentile network latency was 28% and 37% lower than Azure and GCP, respectively, the report states. AWS has room for improvement, however, on storage input/output. It “demonstrated the lowest performance on seven of the 12 benchmarks, including all storage I/O benchmarks and the single core CPU benchmark,” according to the report.
What’s more, AWS’s advanced io2 disks averaged 51% lower latency than the machines with general-purpose disks. Still, GCP’s general-purpose disk held up well against the other two’s advanced disks. Its top-performing machine for storage I/O read and write achieved only 5% fewer read storage input/output per second (IOPS) than the others’ top performers, the report states.
But Azure’s expensive ultra-disks took the top spot for IOPS. Specifically, the disks enabled Azure machines to beat out the other cloud providers in storage I/O, read IOPS, write IOPS and write latency, the report states. At the same time, Azure had the highest estimated monthly disk cost and was the second least cost-efficient cloud provider in terms of cost per throughput.
The higher cost may not be worth it, the report adds, depending on users’ needs. “To improve OLTP performance, we found it is better to spend on more nodes, memory, and better CPUs,” the report states. “Machines with advanced disks could be more appropriate for heavier read or write tasks, and critical workloads with demanding latency requirements.”
GCP enabled the highest throughput at 37,048 throughput-per-minute on a three-node cluster, but AWS provides the most cost-efficient machine at $0.816 per tpm.
For the evaluation, Cockroach Labs assessed 54 machines and conducted nearly 1,000 benchmarks to measure CPU performance using CoreMark, network performance using Netperf, storage I/O performance using FIO, OLTP performance using a Cockroach Labs derivative of TPC-C.
“We believe each provider has a compelling case for how their machines are ready to support today’s application,” the report concludes.
Read the full report here.
NEXT STORY: Now that’s edge computing