Security reasoning.
Measured.
Explore AEGIS alongside leading models on security detection and structured analysis.
Internal benchmarkTested on a security detection dataset
AEGIS · F1 score
Model performance
Higher is better · 0–100%
Showing 7 of 14 models · Ranked by F1 score
Reported scores from OpenSec’s internal evaluation. Scope varies by model. See methodology.
A closer comparison.
Compare AEGIS with another model across key security metrics.
F1 score+15.8 pp
Precision+14.7 pp
Recall+16.7 pp
Differences are percentage points between reported scores; they are not estimates of statistical significance.
All results
Explore reported results for all 14 evaluated models.
| Model | ||||
|---|---|---|---|---|
| AEGIS | 82.5% | 85.1% | 80.0% | 83.0% |
| Claude Sonnet 4.6 Thinking | 71.3% | 58.2% | 92.0% | 59.0% |
| Claude Opus 4.6 Thinking | 68.3% | 56.6% | 86.0% | 59.0% |
| Qwen3.6 27B FP8 Base | 66.7% | 70.5% | 63.3% | 67.0% |
Scores shown as percentages. — Not reported.
Behind the numbers.
Our evaluation methodology, in brief.
Internal evaluation
Security detection and structured analysis, evaluated by OpenSec. AEGIS is the published name for our checkpoint-1400 candidate, compared with the Qwen3.6 27B FP8 base and API models.
Reading the results
These are reported results from the previous OpenSec benchmark. Evaluation scope and sample sizes vary by model. They describe these runs, rather than a universal ranking of model capability.
Metric definitions
- F1 score
- Balances precision and recall in one score.
- Precision
- Of the positive predictions, the proportion that were correct.
- Recall
- Of the positive cases, the proportion identified.
- Accuracy
- The reported proportion of correct classifications.
- CWE alignment
- Reported agreement with the reference weakness classification.
- Severity alignment
- Reported agreement with the reference severity.
- Triplet accuracy
- Reported joint agreement across verdict, CWE and severity.
- JSON validity
- Reported rate of structurally valid JSON responses.
Evaluation scope and source notes
The source describes a shared security detection dataset, but does not specify its public name, evaluation date, prompts, or full scoring protocol. No new evaluation was run for this redesign.
The original sample-count metadata does not consistently agree with the listed confusion-matrix totals. Scores are preserved as published; they have not been recalculated from those totals. GLM-5 is marked as an incomplete run in the source. Missing triplet scores remain unavailable rather than being represented as zero.
The original paired-run summary reports different AEGIS verdict and severity rates from the leaderboard. It is shown separately below to preserve that distinction. The comparison above uses only leaderboard scores.
| Reported match rate | AEGIS | Qwen base |
|---|---|---|
| Verdict | 70.0% | 67.0% |
| CWE | 73.0% | 43.0% |
| Severity | 60.0% | 26.0% |
Put security reasoning to work.
Partner with OpenSec to evaluate AEGIS for your use cases.