CryoCloud v3.0: EMPIAR-10288 & 11807 benchmarks
v3.0 adds our own 3D refinement engine. We benchmarked it on two EMPIAR datasets against RELION 5.0 on the same hardware and against published CryoSPARC results. This article gives the runtimes of the fully unsupervised benchmark runs.
01 Introduction
CryoCloud v3.0 is the first release that adds our own 3D refinement engine, alongside the existing 17 algorithms including CryoCloud MotionCorr, CTF estimation, picker, AutoMasking, AutoClass3D and polishing. The two GPU steps in the default workflow that remain RELION’s are 3D classification and initial model generation.
The two datasets differ in kind: a 2,755-movie GPCR membrane-protein complex, CB1–Gi, and a 10,016-movie small soluble kinase complex, CAK–CT7030. Table 1 summarises the headline figures; the rest of the article shows how we ran the benchmark.
Dataset
End to end
Result
CB1–Gi · EMPIAR-10288
6.41 h
2,755 movies to a 2.68 Å map, 21 jobs, unsupervised
CAK–CT7030 · EMPIAR-11807
12.85 h
10,016 movies to a 2.16 Å map, 17 jobs, unsupervised
vs RELION, end to end
3.0× faster
CB1, same data and hardware, 0.6 Å higher resolution
3D refinement job of CAK
7.2× faster
Our engine against RELION Refine3D: 3 h 52 min to 32 min
vs CryoSPARC, end to end
1.7× faster
CB1, published runtime as reported; 2.6× after GPU scaling (Appendix 2)
Table 1. The four headline figures. Both runs were unsupervised from raw movies to final map.
02 Method
The workflow
Both datasets were processed with a decoy-based 3D classification workflow; 2D classification was omitted entirely. CB1, an active-state GPCR complex, went through our standard two rounds of 3D classification. For CAK a single round was enough to obtain a clean particle set and a 2.16 Å reconstruction. Both workflows ran fully unsupervised: no parameters were tuned and no particle sets were curated by hand between jobs. Figure 1 shows the sequence.
Hardware
Every GPU job in the CB1 run executed on one g6.12xlarge instance with 4 × NVIDIA L4. Extraction, selection and post-processing are CPU jobs and ran on small CPU instances. Particle picking used one L4. Because every other GPU job ran on the same 4 × L4 node, one hardware normalisation covers the run. The CAK run used two GPU types, classification on 4 × L4 and refinement and polishing on 4 × A10G, as instances were available, so its CryoSPARC comparison is given as a range.
Reference runs
RELION 5.0, run by us on our own hardware. For CB1 we processed the same 2,755 movies through an equivalent unsupervised RELION workflow on the same instances, using RELION’s motion correction, CtfFind4 and the spherical masks of RELION’s default unsupervised path. RELION’s motion correction and Bayesian polishing run on CPU and used the CPUs of the same node. The comparison is like-for-like with regard to the hardware and the jobs each package provides.
CryoSPARC, as published. The CryoSPARC comparisons rest on the preprint for unsupervised GPCR analysis (bioRxiv 2025.10.17.682689, Table S4) and on the CryoSPARC blog post covering the CAK runs . We take the runtimes its authors published for the same two EMPIAR entries and scale them to our GPU configuration using the geometric-mean model in Appendix 2, in order to compare end-to-end runtimes while accounting for hardware differences.
What we measure
Runtime per job is wall time from start to completion as recorded in the project report, excluding instance launch time, which averages five minutes across thousands of jobs on our platform. Stage runtime sums the jobs in each of four stages: preprocessing, classification, refinement, and CTF refinement and polishing. End to end sums every job in the sequence, with queue time excluded because it measures cluster load. Final resolution is the gold-standard FSC at 0.143 from the final post-processing job, reported for the same mask in the RELION comparison.
03 Results
CB1–Gi against RELION, run within CryoCloud
We motion-corrected 2,755 movies in 34 minutes, estimated CTF patch by patch, and picked 1.32 million particles with our ML picker at default settings in 10 minutes. The RELION branch used RELION’s motion correction and CtfFind4; to keep the comparison to the algorithms, it used our picker too, so both branches started from the same particles.
Two rounds of decoy-based 3D classification left 520,257 particles, extracted at 2.46 Å/px in a 112 px box and re-extracted at 1.29 Å/px in a 256 px box for refinement. From here the branches diverge: our refinement engine with dynamic shape masking on one side, RELION’s Refine3D with a spherical mask on the other, followed by each package’s own CTF refinement and polishing and a final refinement.
The first 3D refinement with our algorithm finished in 64 minutes at 3.00 Å; RELION’s took 119 minutes to reach 3.79 Å. Polishing, including our training job, ran 4.8× faster and the final refinement 8.5×, and the finished maps stand at 2.68 Å against 3.30 Å. Table 2 gives the stage totals, Figure 3 the same data to one axis, and Appendix 1 the job-by-job pairing.
CB1–Gi against CryoSPARC
CryoSPARC’s authors report 10.9 h for EMPIAR-10288 on a two-GPU workstation with 2 × RTX 4090, reaching 2.7 Å. As published, v3.0 is 1.7× faster at the same resolution. That configuration carries 1.51× the compute of our 4 × L4 node by the measure in Appendix 2. Scaling their runtime by that factor gives 2.6×.
Stage
RELION 5.0
CryoCloud v3.0
Speed-up
Preprocessing
1.52 h
1.20 h
1.3×
Classification
4.42 h
1.71 h
2.6×
Refinement
7.32 h
1.88 h
3.9×
CTF refinement
0.72 h
0.58 h
1.2×
Polishing
4.96 h
1.04 h
4.8×
End to end
18.94 h
6.41 h
3.0×
Final resolution
3.30 Å
2.68 Å
0.6 Å better
Table 2. CryoCloud v3.0 completes CB1 in 6.41 h against 18.94 h for RELION 5.0 on the same micrographs and the same instances, and resolves the map 0.6 Å further. The RELION run never reaches 3 Å: its first refinement yields 3.79 Å and the final map 3.30 Å.
CAK–CT7030 against CryoSPARC
CAK–CT7030 is the larger dataset: 10,016 movies of a small soluble complex, where a single round of decoy classification cleaned the stack in 54 minutes, taking 855,623 picked particles to 460,055. Particles were extracted at 2.67 Å/px in a 64 px box and re-extracted at 1.02 Å/px in a 168 px box. In every other respect the sequence matched CB1.
This run used two GPU types, depending on instance availability: classification ran on 4 × L4, refinement and polishing on 4 × A10G.
Our refinement engine ran the first refinement in 32 min to 2.41 Å and the final refinement in 16 min, for which post-processing gave a 2.16 Å map.
CryoSPARC’s case study reports 25.2 h on a single node with 8 × A100 40 GB, with a final map at 2.10 Å, 0.06 Å beyond ours, with comparable detail in the ligand density (see Figure 5). The workflows differ, and we did not set out to replicate that work. Table 4 compares the unsupervised path to a final map on each platform.
Measure
CryoSPARC, as published
CryoSPARC, GPU-equivalent
CryoCloud v3.0
Hardware
2 × RTX 4090
scaled ×1.51
4 × L4
End to end
10.9 h
16.5 h
6.41 h
Speed-up of CryoCloud v3.0
1.7×
2.6×
—
Final resolution
2.7 Å
2.7 Å
2.68 Å
Table 3. Normalised for GPU configuration, CryoCloud v3.0 completes CB1 in 6.41 h against 16.5 h for CryoSPARC, at the same resolution: 2.68 Å against 2.7 Å.
04 Summary
In this benchmark we processed EMPIAR entries 10288 and 11807 on our own platform, using both our own algorithms and RELION. CryoCloud v3.0 completes CB1 3.0× faster end to end than RELION 5.0 and resolves the map to a resolution 0.6 Å higher .
Our algorithms outperform RELION at every stage, though not in every individual job: our patch-based CTF estimation is slower than CtfFind4 because it fits nine patches per micrograph in place of one global value, as Appendix 1 shows. Our refinement engine took the first CB1 refinement from 119 min to 64 min and the final one from 313 min to 37 min, and on CAK the first refinement from 3 h 52 min to 32 min while improving its resolution from 2.48 Å to 2.41 Å. Particle picking was not tested here, since both branches used our picker; earlier isolated benchmarks show considerably higher particle retention than RELION’s picker or Topaz, which we will cover separately.
Against published CryoSPARC benchmarks, CryoCloud v3.0 is 1.7× faster end to end on CB1 and 2.0× faster on CAK as published, at comparable final resolution. Normalised for hardware on the geometric mean of FP32 compute and memory bandwidth, the figures are 2.6× and 5.0×. On CB1 the margin is widest in preprocessing and classification. In refinement CryoSPARC’s published stage is faster, 1.7 h against 1.9 h, on GPUs with over three times the memory bandwidth.
Test it on your own data
The fastest way to check these numbers is to run CryoCloud on your own dataset. A 30-day trial includes onboarding, weekly check-ins and a summary report, on your data or on an EMPIAR entry of your choice. We are also glad to walk through any individual job in these runs.
Appendix 1: where the time goes, job by job
Job (RELION equivalent)
Stage
CryoCloud, min
RELION, min
Speed-up
CCMotionCorr · MotionCorr
Preprocessing
34.1
71.2
2.10×
JoinStar · JoinStar
Preprocessing
0.5
0.5
1.00×
C3TFEstimation · CtfFind
Preprocessing
18.5
2.7
0.10×
Select · Select
Preprocessing
0.4
0.5
1.20×
CryoCloudPicker · CryoCloudPicker
Preprocessing
9.8
9.7
1.00×
Extract · Extract
Preprocessing
8.9
6.8
0.80×
Class3D · Class3D
Classification
58.0
171.9
3.00×
Select · Select
Classification
0.8
0.9
1.10×
Class3D · Class3D
Classification
27.4
82.6
3.00×
Select · Select
Classification
0.9
0.7
0.80×
InitialModel · InitialModel
Classification
9.0
8.9
1.00×
AutoClass3D · no RELION equivalent
Classification
6.3
No equivalent
Not applicable
Extract · Extract
Refinement
8.8
6.6
0.70×
CCRefine · Refine3D
Refinement
63.8
118.7
1.90×
C3TFRefinement · CtfRefine ×2
CTF & polishing
34.9
43.0
1.20×
CCMotionRefineTrain · no RELION equivalent
CTF & polishing
9.4
60.6
6.44x
CCMotionRefine · Polish
CTF & polishing
47.7
237.9
4.96 ×
CCRefine · Refine3D
Refinement
36.9
313.1
8.50×
AutoMask · AutoMask
Refinement
2.5
0.5
0.20×
PostProcess · PostProcess
Refinement
0.6
0.6
1.00×
Total, 20 job slots
21 jobs
379.2
1,137.4
3.00×
Table 5. The CB1 sequence in execution order, each job beside its RELION counterpart. AutoClass3D has no RELION equivalent. Queue time excluded; runtimes from the job database.
Appendix 2. How the GPU multipliers were calculated
CryoSPARC’s published runtimes were produced on different hardware we do not use. For comparison, we normalize the GPU differences. Each GPU is scored against an NVIDIA L4, the GPU in our node, on FP32 throughput and on memory bandwidth, and the two ratios are combined as a geometric mean. Table 6 gives the scores, and the multiplier for each published configuration compared to 4 × L4 instances CryoCLoud uses by default.
GPU
CUDA cores
Boost clock
FP32 compute
Memory bandwidth
Compute vs L4
Bandwidth vs L4
Geometric mean · multiplier vs 4 × L4
L4
the base
7,424
2,040 MHz
30.3 TFLOPS
300 GB/s
1.00
1.00
1.00 · —
A10
9,216
1,695 MHz
31.2 TFLOPS
600 GB/s
1.03
2.00
1.44 · —
RTX 4090
16,384
2,520 MHz
82.6 TFLOPS
1,008 GB/s
2.73
3.36
3.03 · —
A100 40 GB
6,912
1,410 MHz
19.5 TFLOPS
1,555 GB/s
0.64
5.18
1.83 · —
4 × L4
CryoCloud node
4 × 1.00 L4-equivalents
4.00
× 1.00
2 × RTX 4090
CryoSPARC, CB1
2 × 3.03 L4-equivalents
6.05
× 1.51
8 × A100 40 GB
CryoSPARC, CAK
8 × 1.83 L4-equivalents
14.61
× 3.65
Table 6. GPU equivalency relative to the L4. Each GPU is scored on FP32 compute and memory bandwidth, combined as a geometric mean; configurations sum their GPUs in L4-equivalents, and the multiplier divides by our 4 × L4 node. Specifications from NVIDIA datasheets; FP32 is cores × clock × 2.
CB1: 10.9 h published × 1.51 = 16.5 h GPU-equivalent
CAK: 25.2 h published × 3.65 = 92 h on L4; × 2.55 against 4 × A10G = 64 h
Sources. CryoSPARC CB1 runtime and resolution: Table S4, bioRxiv 2025.10.17.682689 , 2 × RTX 4090. CryoSPARC CAK runtime and resolution: CryoSPARC CAK case study , 8 × A100 40 GB. GPU specifications: NVIDIA datasheets, October 2026. CryoCloud runtimes: project reports for EMPIAR-10288 and EMPIAR-11807, available on request.
Trademarks. CryoSPARC is a trademark of Structura Biotechnology Inc. RELION is developed at the MRC Laboratory of Molecular Biology. CryoCloud is not affiliated with, endorsed by or connected to either; the names are used only to identify the software compared.