# Exploit Test Files
**Note**: Each tool is designed for different purposes and performing according to its specifications.
## Tool Comparison Results
| Exploit | LeanParanoia | lean4checker | SafeVerify | Comparator |
|---------|--------------|--------------|------------|------------|
| AuxiliaryShadowing/MatcherShadowing [β](tests/lean_exploit_files/AuxiliaryShadowing/MatcherShadowing.lean) | π’ 1460ms (1488ms) | π’ 2313ms | π 1480ms
definition LeanTestProject.AuxiliaryShadowing.MatcherShadowing.foo does not match the requirement | π‘ 2843ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| AuxiliaryShadowing/PrivateShadowing [β](tests/lean_exploit_files/AuxiliaryShadowing/PrivateShadowing.lean) | π’ 1463ms (1445ms) | π’ 2304ms | π 1499ms
definition LeanTestProject.AuxiliaryShadowing.PrivateShadowing.foo does not match the requirement | π‘ 2907ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| AuxiliaryShadowing/ProofShadowing [β](tests/lean_exploit_files/AuxiliaryShadowing/ProofShadowing.lean) | π’ 1587ms (1538ms) | π’ 2444ms | π 1575ms
definition LeanTestProject.AuxiliaryShadowing.ProofShadowing.foo does not match the requirement | π‘ 3193ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| AuxiliaryShadowing/TheoremShadowing [β](tests/lean_exploit_files/AuxiliaryShadowing/TheoremShadowing.lean) | π 1650ms (1580ms)
Replay | π’ 2382ms | π 1604ms
theorem LeanTestProject.AuxiliaryShadowing.TheoremShadowing.exploit does not have the same type as the requirement | π‘ 3216ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| AuxiliaryShadowing/TypeSignature [β](tests/lean_exploit_files/AuxiliaryShadowing/TypeSignature.lean) | π 1616ms (1680ms)
Replay | π’ 2446ms | π 1532ms
definition LeanTestProject.AuxiliaryShadowing.TypeSignature.exploit does not match the requirement | π‘ 3204ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| CSimp/WithAxiom [β](tests/lean_exploit_files/CSimp/WithAxiom.lean) | π 1652ms (1096ms)
CSimp; CustomAxioms | π’ 2527ms | π 1572ms
exploit_axiom is not in the allowed set of standard axioms | π 4532ms
Illegal axiom detected: 'sorryAx' |
| CSimp/WithUnsafe [β](tests/lean_exploit_files/CSimp/WithUnsafe.lean) | π 1722ms (1114ms)
CSimp; CustomAxioms | π’ 2483ms | π 1619ms
exploit_axiom is not in the allowed set of standard axioms | π 4483ms
Illegal axiom detected: 'sorryAx' |
| ConstructorIntegrity/ManualConstructor [β](tests/lean_exploit_files/ConstructorIntegrity/ManualConstructor.lean) | π 1511ms (1049ms)
ConstructorIntegrity; CustomAxioms; RecursorIntegrity | π’ 2471ms | π 1687ms
definition exploit_theorem does not match the requirement | π 4360ms
Challenge and solution constant kind don't match: 'exploit_theorem' |
| CustomAxioms/FakeStdLib [β](tests/lean_exploit_files/CustomAxioms/FakeStdLib.lean) | π 3765ms (1552ms)
CustomAxioms | π’ 2908ms | π 2035ms
Std.TrustMe.forgedFalse is not in the allowed set of standard axioms | π 4879ms
Illegal axiom detected: 'sorryAx' |
| CustomAxioms/ForgeRunCmd [β](tests/lean_exploit_files/CustomAxioms/ForgeRunCmd.lean) | π 3725ms (1549ms)
CustomAxioms | π’ 3067ms | π 2087ms
RunCmdForgedFalse is not in the allowed set of standard axioms | π 4962ms
Challenge and solution constant kind don't match: 'exploit_theorem' |
| CustomAxioms/ForgeRunTac [β](tests/lean_exploit_files/CustomAxioms/ForgeRunTac.lean) | π 3855ms (1487ms)
CustomAxioms | π’ 2840ms | π 2105ms
RunTacForgedFalse is not in the allowed set of standard axioms | π 5002ms
Challenge and solution constant kind don't match: 'exploit_theorem' |
| CustomAxioms/HiddenInInstance [β](tests/lean_exploit_files/CustomAxioms/HiddenInInstance.lean) | π 1652ms (1072ms)
CustomAxioms | π’ 2443ms | π 1527ms
LeanTestProject.CustomAxioms.HiddenInInstance.hidden_axiom is not in the allowed set of standard axioms | π‘ 3068ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| CustomAxioms/HiddenInMacro [β](tests/lean_exploit_files/CustomAxioms/HiddenInMacro.lean) | π 1604ms (999ms)
CustomAxioms | π’ 2352ms | π 1530ms
LeanTestProject.CustomAxioms.HiddenInMacro.hidden_axiom is not in the allowed set of standard axioms | π‘ 2968ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| CustomAxioms/ProveAnything [β](tests/lean_exploit_files/CustomAxioms/ProveAnything.lean) | π 1559ms (985ms)
CustomAxioms | π’ 2449ms | π 1532ms
magic is not in the allowed set of standard axioms | π 4061ms
Illegal axiom detected: 'sorryAx' |
| CustomAxioms/ProveFalse [β](tests/lean_exploit_files/CustomAxioms/ProveFalse.lean) | π 1499ms (940ms)
CustomAxioms | π’ 2355ms | π 1550ms
exploit_axiom is not in the allowed set of standard axioms | π 4414ms
Illegal axiom detected: 'sorryAx' |
| CustomAxioms/SkipKernelTC [β](tests/lean_exploit_files/CustomAxioms/SkipKernelTC.lean) | π 1625ms (1052ms)
CustomAxioms | π’ 2459ms | π’ 1651ms | π 4347ms
Solution constant is not a theorem: 'exploit_theorem' |
| Extern/BuiltinInit [β](tests/lean_exploit_files/Extern/BuiltinInit.lean) | π 2004ms (1052ms)
CustomAxioms; Extern | π’ 2486ms | π 1636ms
theorem exploit_theorem does not have the same type as the requirement | π 4351ms
Challenge and solution theorem statement do not match: 'exploit_theorem' |
| Extern/CoreNamespace [β](tests/lean_exploit_files/Extern/CoreNamespace.lean) | π 1720ms (1102ms)
CustomAxioms; Extern | π’ 2515ms | π 1621ms
exploit_axiom is not in the allowed set of standard axioms | π 4389ms
Illegal axiom detected: 'sorryAx' |
| Extern/ExportC [β](tests/lean_exploit_files/Extern/ExportC.lean) | π 1679ms (1061ms)
CustomAxioms; Extern | π’ 2494ms | π 1675ms
exploit_axiom is not in the allowed set of standard axioms | π 4442ms
Illegal axiom detected: 'sorryAx' |
| Extern/ExternFFI [β](tests/lean_exploit_files/Extern/ExternFFI.lean) | π 1619ms (1081ms)
CustomAxioms; Extern | π’ 2515ms | π 1753ms
exploit_axiom is not in the allowed set of standard axioms | π 4345ms
Illegal axiom detected: 'sorryAx' |
| Extern/InitHook [β](tests/lean_exploit_files/Extern/InitHook.lean) | π 2064ms (1105ms)
CustomAxioms; Extern | π’ 2557ms | π 1563ms
theorem exploit_theorem does not have the same type as the requirement | π 4436ms
Challenge and solution theorem statement do not match: 'exploit_theorem' |
| Extern/PrivateExtern [β](tests/lean_exploit_files/Extern/PrivateExtern.lean) | π 1583ms (1053ms)
CustomAxioms; Extern | π’ 2357ms | π 1654ms
exploit_axiom is not in the allowed set of standard axioms | π 4482ms
Illegal axiom detected: 'sorryAx' |
| ImplementedBy/ChainedReplacement [β](tests/lean_exploit_files/ImplementedBy/ChainedReplacement.lean) | π 1707ms (1072ms)
CustomAxioms; Extern; ImplementedBy | π’ 2505ms | π 1673ms
exploit_axiom is not in the allowed set of standard axioms | π 4660ms
Illegal axiom detected: 'sorryAx' |
| ImplementedBy/DirectReplacement [β](tests/lean_exploit_files/ImplementedBy/DirectReplacement.lean) | π 1710ms (1099ms)
CustomAxioms; ImplementedBy; Unsafe | π’ 2567ms | π 1660ms
exploit_axiom is not in the allowed set of standard axioms | π 4418ms
Illegal axiom detected: 'sorryAx' |
| ImplementedBy/PrivateImpl [β](tests/lean_exploit_files/ImplementedBy/PrivateImpl.lean) | π 1678ms (1110ms)
CustomAxioms; ImplementedBy; Unsafe | π’ 2459ms | π 1485ms
exploit_axiom is not in the allowed set of standard axioms | π 4390ms
Illegal axiom detected: 'sorryAx' |
| KernelRejection/NonPositive [β](tests/lean_exploit_files/KernelRejection/NonPositive.lean) | π 918ms (943ms)
KernelRejection | π 2367ms
Could not find any oleans for: LeanTestProject.KernelRejection.NonPositive | π‘ N/A
Kernel rejected during compilation | π 3015ms
Child exited with 1 |
| KernelRejection/UnsafeCast [β](tests/lean_exploit_files/KernelRejection/UnsafeCast.lean) | π 893ms (915ms)
KernelRejection | π 2279ms
Could not find any oleans for: LeanTestProject.KernelRejection.UnsafeCast | π‘ N/A
Kernel rejected during compilation | π 3031ms
Child exited with 1 |
| Metavariables/Timeout [β](tests/lean_exploit_files/Metavariables/Timeout.lean) | π 1583ms (1104ms)
CustomAxioms; Sorry; Unsafe | π’ 2536ms | π 1646ms
sorryAx is not in the allowed set of standard axioms | π 4515ms
Illegal axiom detected: 'sorryAx' |
| Metavariables/TypeclassFail [β](tests/lean_exploit_files/Metavariables/TypeclassFail.lean) | π 1669ms (1109ms)
CustomAxioms; Sorry; Unsafe | π’ 2516ms | π 1642ms
sorryAx is not in the allowed set of standard axioms | π 4478ms
Illegal axiom detected: 'sorryAx' |
| NativeComputation/NativeDecide [β](tests/lean_exploit_files/NativeComputation/NativeDecide.lean) | π 1821ms (1081ms)
CustomAxioms; NativeComputation; Replay | π 2514ms
(kernel) (interpreter) unknown declaration 'exploit_theorem._nativeDecide_1_1' | π 1670ms
(kernel) (interpreter) unknown declaration 'exploit_theorem._nativeDecide_1_1' | π 4454ms
Challenge and solution theorem statement do not match: 'exploit_theorem' |
| NativeComputation/OfReduce [β](tests/lean_exploit_files/NativeComputation/OfReduce.lean) | π 2180ms (1212ms)
CustomAxioms; NativeComputation; Replay | π 2598ms
(kernel) (interpreter) unknown declaration 'exploit_theorem._nativeDecide_1_1' | π 1654ms
(kernel) (interpreter) unknown declaration 'exploit_theorem._nativeDecide_1_1' | π 4069ms
Illegal axiom detected: 'sorryAx' |
| Partial/NonTerminating [β](tests/lean_exploit_files/Partial/NonTerminating.lean) | π 1569ms (938ms)
CustomAxioms; Sorry; Unsafe | π’ 2334ms | π 1503ms
sorryAx is not in the allowed set of standard axioms | π‘ 3137ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| RecursorIntegrity/MissingRecursor [β](tests/lean_exploit_files/RecursorIntegrity/MissingRecursor.lean) | π 1573ms (1032ms)
ConstructorIntegrity; CustomAxioms; RecursorIntegrity | π’ 2479ms | π 1586ms
definition exploit_theorem does not match the requirement | π 4047ms
Challenge and solution constant kind don't match: 'exploit_theorem' |
| Sorry/Admit [β](tests/lean_exploit_files/Sorry/Admit.lean) | π 1492ms (1013ms)
CustomAxioms; Sorry; Unsafe | π’ 2423ms | π 1647ms
sorryAx is not in the allowed set of standard axioms | π 4434ms
Illegal axiom detected: 'sorryAx' |
| Sorry/ByAsSorry [β](tests/lean_exploit_files/Sorry/ByAsSorry.lean) | π 1630ms (1043ms)
CustomAxioms; Sorry; Unsafe | π’ 2434ms | π 1677ms
sorryAx is not in the allowed set of standard axioms | π 4199ms
Illegal axiom detected: 'sorryAx' |
| Sorry/Direct [β](tests/lean_exploit_files/Sorry/Direct.lean) | π 1545ms (935ms)
CustomAxioms; Sorry; Unsafe | π’ 2289ms | π 1472ms
sorryAx is not in the allowed set of standard axioms | π‘ 3087ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| Sorry/Intermediate [β](tests/lean_exploit_files/Sorry/Intermediate.lean) | π 1711ms (1083ms)
CustomAxioms; Sorry; Unsafe | π’ 2499ms | π 1743ms
sorryAx is not in the allowed set of standard axioms | π 4925ms
Illegal axiom detected: 'sorryAx' |
| Sorry/Opaque [β](tests/lean_exploit_files/Sorry/Opaque.lean) | π 4574ms (1725ms)
CustomAxioms; Sorry | π’ 3100ms | π 2037ms
sorryAx is not in the allowed set of standard axioms | π 4506ms
Challenge and solution constant kind don't match: 'exploit_theorem' |
| Sorry/ProofAsSorry [β](tests/lean_exploit_files/Sorry/ProofAsSorry.lean) | π 1551ms (1082ms)
CustomAxioms; Sorry | π’ 2537ms | π 1698ms
sorryAx is not in the allowed set of standard axioms | π 4483ms
Illegal axiom detected: 'sorryAx' |
| Sorry/TerminalTactics [β](tests/lean_exploit_files/Sorry/TerminalTactics.lean) | π 1687ms (1052ms)
CustomAxioms; Sorry; Unsafe | π’ 2430ms | π 1605ms
sorryAx is not in the allowed set of standard axioms | π 4367ms
Illegal axiom detected: 'sorryAx' |
| Sorry/Underscore [β](tests/lean_exploit_files/Sorry/Underscore.lean) | π 1672ms (1116ms)
CustomAxioms; Sorry; Unsafe | π’ 2456ms | π 1692ms
sorryAx is not in the allowed set of standard axioms | π 4457ms
Illegal axiom detected: 'sorryAx' |
| SourcePatterns/LocalInstance [β](tests/lean_exploit_files/SourcePatterns/LocalInstance.lean) | π 1553ms (998ms)
SourcePatterns | π’ 2331ms | π 1478ms
theorem LeanTestProject.SourcePatterns.LocalInstance.exploit_theorem does not have the same type as the requirement | π‘ 2948ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| SourcePatterns/LocalInstanceArithmetic [β](tests/lean_exploit_files/SourcePatterns/LocalInstanceArithmetic.lean) | π 1628ms (1060ms)
SourcePatterns | π’ 2472ms | π 1639ms
theorem exploit_theorem does not have the same type as the requirement | π 4371ms
Challenge and solution theorem statement do not match: 'exploit_theorem' |
| SourcePatterns/LocalMacroRules [β](tests/lean_exploit_files/SourcePatterns/LocalMacroRules.lean) | π 1627ms (1052ms)
CustomAxioms; SourcePatterns | π’ 2412ms | π 1601ms
LeanTestProject.SourcePatterns.LocalMacroRules.hidden_axiom is not in the allowed set of standard axioms | π‘ 3165ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| SourcePatterns/LocalNotation [β](tests/lean_exploit_files/SourcePatterns/LocalNotation.lean) | π 1676ms (1074ms)
SourcePatterns | π’ 2532ms | π 1680ms
LeanTestProject.SourcePatterns.LocalNotation.Β«term_β_Β» not found in submission | π‘ 3232ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| SourcePatterns/NotationRedefinition [β](tests/lean_exploit_files/SourcePatterns/NotationRedefinition.lean) | π 1682ms (1026ms)
CustomAxioms; SourcePatterns | π’ 2425ms | π 1628ms
theorem LeanTestProject.SourcePatterns.NotationRedefinition.exploit_theorem does not have the same type as the requirement | π‘ 3292ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| SourcePatterns/ScopedNotation [β](tests/lean_exploit_files/SourcePatterns/ScopedNotation.lean) | π 1723ms (1101ms)
CustomAxioms; SourcePatterns | π’ 2508ms | π 1689ms
LeanTestProject.SourcePatterns.ScopedNotation._aux_SourcePatterns_ScopedNotation___unexpand_False_1 not found in submission | π‘ 3208ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| Transitive/DeepAxiom_L1 [β](tests/lean_exploit_files/Transitive/DeepAxiom_L1.lean) | π 2132ms (1106ms)
CustomAxioms | π’ 2536ms | π 1627ms
custom_axiom is not in the allowed set of standard axioms | π 4022ms
Illegal axiom detected: 'sorryAx' |
| Transitive/DeepSorry_L1 [β](tests/lean_exploit_files/Transitive/DeepSorry_L1.lean) | π 2016ms (1018ms)
CustomAxioms; Sorry; Unsafe | π’ 2455ms | π 1632ms
sorryAx is not in the allowed set of standard axioms | π 4243ms
Illegal axiom detected: 'sorryAx' |
| Transitive/DeepSorry_L2 [β](tests/lean_exploit_files/Transitive/DeepSorry_L2.lean) | π 2448ms (1081ms)
CustomAxioms; Sorry; Unsafe | π’ 2501ms | π 1644ms
sorryAx is not in the allowed set of standard axioms | π 4491ms
Illegal axiom detected: 'sorryAx' |
| Transitive/Level2_UsesBoth [β](tests/lean_exploit_files/Transitive/Level2_UsesBoth.lean) | π’ 2129ms (2095ms) | π’ 2424ms | π’ 1622ms | π’ 4224ms
Your solution is okay! |
| Transitive/UsesBadLib [β](tests/lean_exploit_files/Transitive/UsesBadLib.lean) | π 1972ms (1016ms)
CustomAxioms | π’ 2398ms | π 1597ms
BadLib.hiddenAssumption is not in the allowed set of standard axioms | π 4239ms
Illegal axiom detected: 'sorryAx' |
| Unsafe/UnsafeDefinition [β](tests/lean_exploit_files/Unsafe/UnsafeDefinition.lean) | π 1616ms (1102ms)
CustomAxioms; ImplementedBy; Unsafe | π’ 2473ms | π 1648ms
exploit_axiom is not in the allowed set of standard axioms | π 4308ms
Illegal axiom detected: 'sorryAx' |
| Valid/ComplexExample [β](tests/lean_exploit_files/Valid/ComplexExample.lean) | π’ 1759ms (1702ms) | π’ 2521ms | π 1710ms
_private.Valid.ComplexExample.0.LeanTestProject.Valid.ComplexExample.Tree.size.match_1.eq_1 not found in submission | π‘ 2894ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
| Valid/Dependencies [β](tests/lean_exploit_files/Valid/Dependencies.lean) | π’ 1911ms (1899ms) | π’ 2292ms | π’ 1509ms | π’ 3966ms
Your solution is okay! |
| Valid/Helper [β](tests/lean_exploit_files/Valid/Helper.lean) | π’ 1631ms (1554ms) | π’ 2332ms | π’ 1541ms | π’ 4254ms
Your solution is okay! |
| Valid/Simple [β](tests/lean_exploit_files/Valid/Simple.lean) | π’ 1607ms (1631ms) | π’ 2477ms | π’ 1635ms | π’ 4363ms
Your solution is okay! |
| Valid/UnsafeReducibility [β](tests/lean_exploit_files/Valid/UnsafeReducibility.lean) | π’ 1627ms (1619ms) | π’ 2459ms | π’ 1630ms | π’ 4334ms
Your solution is okay! |
| Valid/WithAxioms [β](tests/lean_exploit_files/Valid/WithAxioms.lean) | π’ 1807ms (1821ms) | π’ 2497ms | π’ 1663ms | π‘ 3247ms
lean4export: PANIC at Option.get! Init.Data.Option.BasicAux:22:14: value is none |
---
### Legend
- π **Detected**: Tool identified an exploit
- π’ **Passed**: Tool did not detect any exploit
- π‘ **N/A**: Test setup issue or methodology limitation
- Click to show details
Verification results or tool output
### Timing
- **LeanParanoia**: Verification of pre-compiled `.olean` files (fail-fast time in brackets)
- **Lean4checker**: Kernel re-verification of pre-compiled `.olean` files
- **SafeVerify**: Comparison of pre-compiled `.olean` files
- **Comparator**: Equivalence checking of pre-compiled `.olean` files
Note: WIP, might contain inaccuracies.