Frozen reference · EVA‑02‑L
Know the target precisely.
- Parameters
- 305,080,232 exact tensor count · FP32
- Compute
- 362.3 GMAC per 448² image
- Feature sequence
- 1,025 × 1,024 tokens × width
- Published top‑1
- 90.054% ImageNet‑1K validation · not the contest gate
Architecture
- Input
- RGB · fixed 448 × 448
- Patch embedding
- 14 × 14 convolution, stride 14 · 32 × 32 grid
- Backbone
- 24 blocks · width 1,024 · 16 attention heads
- Transformer block
- global attention · 2D RoPE · SwiGLU · sub-LayerNorm
- Pooling
- mean of the 1,024 patch tokens, excluding the prefix token
- Head
- LayerNorm · linear projection · 1,000 logits · argmax
Training provenance
- Teacher
- frozen one-billion-parameter EVA‑CLIP vision tower
- Objective
- reconstruct masked teacher features with negative-cosine loss
- Mask
- 40% block-wise masking
- Pretraining
- 56 epochs on the 38-million-image Merged‑38M corpus
- Fine-tuning
- ImageNet‑21K, then ImageNet‑1K
- Published format
- timm Safetensors checkpoint stored in float32
Checkpoint identifier
eva02_large_patch14_448.mim_m38m_ft_in22k_in1k
- eva02_large
- 24-block large EVA‑02 model
- patch14_448
- 14-pixel patches at a fixed 448² input
- mim_m38m
- masked-image pretraining on Merged‑38M
- ft_in22k
- intermediate supervised fine-tuning on ImageNet‑21K
- in1k
- final ImageNet‑1K classification fine-tune
Context · a decade of vision in one checkpoint
What EVA‑02 means in vision AI.
A finish line means more when you know the race. This section places the frozen reference in the research it came out of: what it learned from, what it displaced, and why a 304M-parameter model drew the attention usually reserved for models several times its size. Every number here is its authors' own published measurement.
- Three threads matured separately. 2012: visual features should be learned, not engineered. 2020: the plain Transformer from language modeling reads images once they are cut into patch tokens (the same 14 × 14 patches described above). 2021: CLIP showed that supervising vision with web-scale natural language yields features that transfer almost anywhere.
- EVA (2022) fused them. A one-billion-parameter plain ViT was pre-trained to reconstruct the language-aligned CLIP features of image patches it could not see. That single pretext task set records across classification, detection, and segmentation, and made plain ViT plus a strong teacher the recipe to beat.
- EVA‑02 (2023) ran the recipe toward efficiency. Four sizes from 6M to 304M parameters, pre-trained against a frozen EVA‑CLIP teacher on only publicly accessible data. The large model, the checkpoint frozen above, reached 90.0% fine-tuned top‑1 on ImageNet‑1K and beat its billion-parameter predecessor on ten of eleven benchmarks. Its authors call that result, fairly, giant-killing.
| Model | Published | Parameters | Top-1 (%) | Training data |
|---|---|---|---|---|
| EVA-02-Ti | 2023 | 6M | 80.7 | public |
| EVA-02-S | 2023 | 22M | 85.8 | public |
| EVA-02-B | 2023 | 86M | 88.6 | public |
| EVA-02-L (the contest reference) | 2023 | 304M | 90.0 | public |
| MobileViTv3-1.0 | 2022 | 5M | 79.7 | public |
| DeiT-III-S | 2022 | 22M | 84.8 | public |
| MOAT-0 | 2022 | 28M | 85.7 | public |
| MobileNetV4-Hybrid-L | 2024 | 38M | 87.0 | private |
| BEiTv2-B | 2022 | 86M | 87.5 | public |
| ViT-B distilled from ViT-22B | 2023 | 86M | 88.6 | private |
| BEiTv2-L | 2022 | 304M | 89.2 | public |
| ViT-L distilled from ViT-22B | 2023 | 304M | 89.6 | private |
| ConvNeXt V2-H | 2023 | 659M | 88.9 | public |
| EVA | 2022 | 1B | 89.7 | public |
| InternImage-H | 2023 | 1.08B | 89.6 | public |
| ONE-PEACE | 2023 | 1.5B | 89.8 | public |
| BEiT-3 | 2022 | 1.9B | 89.6 | public |
| CoCa | 2022 | 2.1B | 91.0 | private |
| RevCol-H | 2023 | 2.2B | 90.0 | private |
| SwinV2-G | 2022 | 3B | 90.2 | private |
| MAWS ViT-6.5B | 2023 | 6.5B | 90.1 | private |
Hollow diamonds mark results whose training requires a private image corpus (JFT, Instagram‑3B, or a proprietary ImageNet extension); frozen publicly downloadable teacher weights count as public artifacts.
The two points marked ← 22B are ViTs distilled from Google's
22-billion-parameter teacher on the private four-billion-image JFT‑4B corpus;
pre-trained on 14 million public images, EVA‑02‑B ties one and
EVA‑02‑L clears the other. Three more years of results barely moved the
ceiling (CoCa's 91.0% from 2022 still tops the axis), and every point above the
reference is a hollow diamond: trained on a private corpus, unrepeatable outside its
lab. No circle of any size has passed the reference, so among models anyone can retrain
end to end from public data, it remains the most accurate ever published.
Frontier behavior stopped being the exclusive property of frontier budgets.
Trained by imitation, not by labels.
- During pretraining each image is cut into 1,024 patches and 40% are blanked out in contiguous blocks; the student sees only the survivors.
- The frozen one-billion-parameter EVA‑CLIP teacher encodes the complete image, and the student must predict its feature vector at every blanked position, scored by cosine similarity.
- Answering reliably requires objects, parts, textures, and the language-shaped semantics the teacher carries, not the statistics of a thousand class labels.
Masked image
448² · 1,024 patches · 40% blanked block-wise
EVA‑02 · the student, trained
plain TrV transformer · 6M–304M parameters
Predicted features
one guess per blanked patch
Full image
nothing hidden
EVA‑CLIP · the teacher, frozen
1.0B-parameter language-aligned vision tower
Target features
for the same patch positions
Loss: negative cosine similarity between prediction and target, at blanked positions only. The student must infer, from context alone, how the teacher would describe what it cannot see.
MIM on Merged‑38M · 56 epochs
ImageNet‑21K fine-tune
ImageNet‑1K fine-tune
the frozen reference
A 2020 architecture, modernized in three edits.
- Deliberately plain: no pyramid, no convolutional stem, no windowed attention, just the same stack of identical global-attention blocks the Vision Transformer introduced in 2020.
- Three edits borrowed from language models (a gated SwiGLU feed-forward, sub-LayerNorm on the output projection, and 2D rotary positions) lift ImageNet top‑1 by 1.6 points in the paper's controlled pilot.
- For an entrant this is not trivia: these are precisely the ops a submission executes, 24 blocks deep, once per image, 10,000 times. How they are fused, laid out in memory, and quantized is where the machine's bandwidth budget is won or lost.
Plain ViT block · 2020
LayerNorm
Global attention
learned absolute positions
+ residual
LayerNorm
MLP
GELU activation
+ residual
TrV block · EVA‑02
LayerNorm
Global attention
2D rotary position embedding (RoPE)
+ residual
LayerNorm
SwiGLU feed-forward
two branches, one gating the other · sub-LN before the output projection
+ residual
Same 86M model, data, and schedule:
84.0baseline
85.0SwiGLU + xavier-normal init
85.2sub-LN
85.62D RoPE
Harder tests than its own benchmark.
- The identical fine-tuned weights, with no re-tuning and no test-time tricks, were measured on five harder ImageNet variants: new photographs, adversarially filtered images, art and renditions, and sketches.
- EVA‑02‑L posted both the highest averaged accuracy (85.2%) and the smallest home-to-variant gap (4.8 points) of every model in the comparison, including the billion-parameter EVA. The contest froze a generalist, not a validation-set specialist.
ImageNet‑1K the home benchmark
90.0
ReaL relabeled validation set
91.1
Rendition art, cartoons, sculpture
89.9
Adversarial naturally hard photographs
87.7
V2 fresh test photographs
82.4
Sketch black-and-white drawings
70.1
Origin story
The inspiration for this contest.
Why this checkpoint, this machine, and a single clock.
- Open end to end. Weights, recipe, code, and every pretraining image are public, so the provenance standard this contest's honor rules demand of entrants is one the reference itself already meets.
- Still unmatched in public. Three years after publication, EVA‑02‑L remains the most accurate ImageNet‑1K classifier at any parameter count that anyone can rebuild from scratch from public artifacts; every published result above its 90.0% was trained on a private corpus no one outside its lab can retrain on.
- Sized for the Spark, frozen for the clock. Unhurried FP32 inference fits the box's unified memory with room to spare, and one Safetensors file with one preprocessing config yields 10,000 deterministic answers, content-addressed below.
- The symmetry. EVA‑02 exists because a student was trained to reproduce a frozen teacher's outputs. This contest hands you the same move, one generation on: EVA‑02‑L is the frozen teacher and your submission is the student. The pass gate scores behavior, not lineage: within the rules' provenance limits, doing to EVA‑02 what EVA‑02 did to EVA‑CLIP is not a loophole. It is the point.
- The machine completes the inversion. Research measured EVA‑02 in accuracy per parameter; this contest measures it in seconds per 10,000 images on one 240 W box that is bandwidth-bound long before it is compute-bound. The starter kit gives you the correct, slow baseline; the clock hands you the rest.
The deeper inspiration is the paper's own experiment, inverted:
- The paper's question · 2023
- Fix the budget, maximize capability. Given 304M parameters and public data only, how close can a compact model come to the billion-parameter frontier? Answer: it can pass it.
- This contest's question · 2026
- Fix the capability, minimize time. Given the reference's answers on 10,000 images (match at least 9,900 of them), how few seconds can one DGX Spark need? Answer: yours to set.
Content-addressed artifacts
Verify the exact reference.
The name identifies a model family. These digests identify the bytes used to produce the canonical deterministic FP32 answers.
- Checkpoint · SHA‑256
1b54182f1f92613b96bc9e325f3795bde38bd1708233bb77a07d5c52eed2d638- Preprocessing config · SHA‑256
3fd38abc37443bc6e8a5edaf91e5e1f16a5d3dacd0d8c29eaadf172539d6c80b
- Resize
- Bicubic · direct
squash· 448² - Normalize · mean
0.48145466 · 0.4578275 · 0.40821073- Normalize · std
0.26862954 · 0.26130258 · 0.27577711- Reference math
- FP32 · deterministic algorithms · argmax
- Output ABI
- one little-endian signed int32 per image · 40,000 bytes total
- Pass gate
- at least 9,900 of 10,000 exact top‑1 matches
Agreement is not ImageNet accuracy. The scorer compares submitted class IDs with the frozen reference class IDs, not with dataset labels. Any model, checkpoint, or general inference approach permitted by the rules may be used.