Skip to content

Runtime evaluator measurements (2026-09-13)

The end-of-branch tables, produced by benchmarks/Corvus.Text.Json.RuntimeEvaluator.Benchmarks/tools/measure.sh (see its README) on one machine, each corpus in its own process, against Blaze 16.10.0. The CSV behind the four-axis table is docs/measurements/2026-09-13/four-axis-table.csv in the repository.

The four-axis table

cold_ns: one fresh process reading the schema, preparing it (compile, template or image load, nothing for generated code), reading the instances and validating every instance once (median of 3). warm_ns: one pass over the corpus at steady state, the sum over instances of the per-evaluation mean over 200 loops with the clock overhead subtracted, each corpus in its own process. parse_eval_ns: one pass over the corpus parsing and evaluating each instance once (the shape of a service validating each request), the sum over instances of the per-instance mean over 100 loops with the clock overhead subtracted; Blaze's by difference, its CLI validating ten copies of the corpus minus one instance, per instance, which also carries the CLI's per-instance bookkeeping (about 2 us per instance). compile_ns: the preparation step alone (Blaze: its compile --fast --minify command less a bare --version; the in-process compile call for corvus-runtime; the image load for corvus-image; the first evaluation for the generated rows). memory: bytes allocated per evaluation at steady state; not measurable for Blaze from outside (its binary carries its own allocator; its heap is flat across loops).

Medians

implementation cold_ns (median) warm_ns (median) parse_eval_ns (median) compile_ns (median) memory (median B/eval) warm faster than blaze on parse+eval faster than blaze on
blaze-compile 9,700,000 111,890 3,861,450 2,700,000 n/a 0 of 37 0 of 37
blaze-template 9,300,000 111,890 3,861,450 0 n/a 0 of 37 0 of 37
corvus-runtime-jit 149,300,000 77,670 484,360 82,120,000 0 31 of 37 37 of 37
corvus-runtime-r2r 52,100,000 75,740 504,550 n/a 0 31 of 37 37 of 37
corvus-runtime-aot 4,600,000 87,580 549,960 320,000 0 29 of 37 37 of 37
corvus-image-jit 121,200,000 77,670 484,360 25,170,000 0 31 of 37 37 of 37
corvus-image-aot 4,200,000 87,580 549,960 150,000 0 29 of 37 37 of 37
corvus-generated-jit 115,800,000 75,080 484,260 35,480,000 0 30 of 37 37 of 37
corvus-generated-aot 4,800,000 88,690 551,080 170,000 0 27 of 37 37 of 37

Every corpus

implementation corpus_name cold_ns warm_ns parse_eval_ns compile_ns memory
blaze-compile ansible-meta 13,200,000 111,890 1,442,433 10,800,000 n/a
blaze-template ansible-meta 11,500,000 111,890 1,442,433 0 n/a
corvus-runtime-jit ansible-meta 152,300,000 77,670 213,350 114,990,000 0
corvus-runtime-r2r ansible-meta 55,800,000 75,740 222,800 n/a 0
corvus-runtime-aot ansible-meta 4,200,000 87,580 225,290 1,230,000 0
corvus-image-jit ansible-meta 120,300,000 77,670 213,350 33,560,000 0
corvus-image-aot ansible-meta 3,800,000 87,580 225,290 430,000 0
corvus-generated-jit ansible-meta 122,000,000 82,290 209,660 43,830,000 0
corvus-generated-aot ansible-meta 4,100,000 91,880 231,340 530,000 0
blaze-compile aws-cdk 7,400,000 24,570 3,105,643 200,000 n/a
blaze-template aws-cdk 6,200,000 24,570 3,105,643 0 n/a
corvus-runtime-jit aws-cdk 145,100,000 16,580 358,160 65,880,000 0
corvus-runtime-r2r aws-cdk 45,500,000 15,190 382,700 n/a 0
corvus-runtime-aot aws-cdk 4,200,000 16,860 422,980 100,000 0
corvus-image-jit aws-cdk 113,200,000 16,580 358,160 16,680,000 0
corvus-image-aot aws-cdk 4,100,000 16,860 422,980 40,000 0
corvus-generated-jit aws-cdk 115,800,000 18,480 325,330 29,600,000 0
corvus-generated-aot aws-cdk 4,500,000 16,730 424,560 60,000 0
blaze-compile babelrc 6,300,000 48,010 2,179,274 1,400,000 n/a
blaze-template babelrc 5,900,000 48,010 2,179,274 0 n/a
corvus-runtime-jit babelrc 141,900,000 50,730 279,230 72,180,000 0
corvus-runtime-r2r babelrc 46,500,000 51,930 288,440 n/a 0
corvus-runtime-aot babelrc 3,500,000 52,470 266,060 230,000 0
corvus-image-jit babelrc 108,000,000 50,730 279,230 23,800,000 0
corvus-image-aot babelrc 3,400,000 52,470 266,060 90,000 0
corvus-generated-jit babelrc 106,300,000 53,870 240,450 32,700,000 0
corvus-generated-aot babelrc 3,700,000 56,320 259,490 110,000 0
blaze-compile clang-format 9,600,000 47,200 508,382 7,000,000 n/a
blaze-template clang-format 9,300,000 47,200 508,382 0 n/a
corvus-runtime-jit clang-format 145,800,000 16,600 59,130 104,480,000 0
corvus-runtime-r2r clang-format 43,900,000 17,370 59,440 n/a 0
corvus-runtime-aot clang-format 3,800,000 17,330 145,820 940,000 0
corvus-image-jit clang-format 121,600,000 16,600 59,130 25,170,000 0
corvus-image-aot clang-format 3,100,000 17,330 145,820 410,000 0
corvus-generated-jit clang-format 124,300,000 17,380 64,030 35,480,000 0
corvus-generated-aot clang-format 3,600,000 25,160 67,950 500,000 0
blaze-compile cmake-presets 39,200,000 1,820,000 20,013,070 21,200,000 n/a
blaze-template cmake-presets 34,300,000 1,820,000 20,013,070 0 n/a
corvus-runtime-jit cmake-presets 197,400,000 1,620,000 3,630,000 84,660,000 0
corvus-runtime-r2r cmake-presets 108,100,000 1,640,000 3,890,000 n/a 0
corvus-runtime-aot cmake-presets 14,000,000 1,760,000 4,460,000 1,810,000 0
corvus-image-jit cmake-presets 171,000,000 1,620,000 3,630,000 25,590,000 0
corvus-image-aot cmake-presets 12,800,000 1,760,000 4,460,000 680,000 0
corvus-generated-jit cmake-presets 162,400,000 1,610,000 3,680,000 45,700,000 0
corvus-generated-aot cmake-presets 13,600,000 1,850,000 4,470,000 810,000 0
blaze-compile code-climate 14,300,000 81,830 10,726,432 1,300,000 n/a
blaze-template code-climate 14,100,000 81,830 10,726,432 0 n/a
corvus-runtime-jit code-climate 149,300,000 61,620 1,080,000 83,960,000 0
corvus-runtime-r2r code-climate 54,400,000 64,930 1,250,000 n/a 0
corvus-runtime-aot code-climate 6,000,000 70,700 1,130,000 190,000 0
corvus-image-jit code-climate 111,700,000 61,620 1,080,000 16,260,000 0
corvus-image-aot code-climate 5,300,000 70,700 1,130,000 60,000 0
corvus-generated-jit code-climate 102,600,000 61,290 948,830 28,530,000 0
corvus-generated-aot code-climate 6,000,000 70,560 1,170,000 80,000 0
blaze-compile cql2 13,700,000 105,770 487,447 19,800,000 n/a
blaze-template cql2 20,600,000 105,770 487,447 0 n/a
corvus-runtime-jit cql2 156,800,000 52,970 95,710 86,350,000 0
corvus-runtime-r2r cql2 50,400,000 54,580 99,280 n/a 0
corvus-runtime-aot cql2 3,700,000 58,200 107,000 880,000 0
corvus-image-jit cql2 114,400,000 52,970 95,710 29,590,000 0
corvus-image-aot cql2 3,000,000 58,200 107,000 370,000 0
corvus-generated-jit cql2 114,500,000 55,500 96,280 44,810,000 0
corvus-generated-aot cql2 3,900,000 60,090 106,430 430,000 0
blaze-compile cspell 24,500,000 250,890 6,834,697 17,900,000 n/a
blaze-template cspell 23,600,000 250,890 6,834,697 0 n/a
corvus-runtime-jit cspell 176,000,000 168,560 897,460 83,170,000 0
corvus-runtime-r2r cspell 67,900,000 168,410 927,630 n/a 0
corvus-runtime-aot cspell 6,700,000 173,310 1,010,000 1,540,000 0
corvus-image-jit cspell 122,700,000 168,560 897,460 32,280,000 0
corvus-image-aot cspell 5,600,000 173,310 1,010,000 630,000 0
corvus-generated-jit cspell 121,800,000 163,940 873,410 43,110,000 0
corvus-generated-aot cspell 6,200,000 182,700 1,020,000 520,000 0
blaze-compile cypress 9,200,000 77,670 4,394,448 2,500,000 n/a
blaze-template cypress 9,000,000 77,670 4,394,448 0 n/a
corvus-runtime-jit cypress 152,500,000 51,300 542,080 76,990,000 0
corvus-runtime-r2r cypress 52,100,000 51,750 593,060 n/a 0
corvus-runtime-aot cypress 5,100,000 50,040 615,260 500,000 0
corvus-image-jit cypress 123,000,000 51,300 542,080 26,660,000 0
corvus-image-aot cypress 4,900,000 50,040 615,260 150,000 0
corvus-generated-jit cypress 122,400,000 48,370 519,960 35,130,000 0
corvus-generated-aot cypress 5,500,000 50,410 601,530 180,000 0
blaze-compile deno 13,800,000 117,520 6,420,651 3,900,000 n/a
blaze-template deno 13,400,000 117,520 6,420,651 0 n/a
corvus-runtime-jit deno 151,900,000 81,090 904,820 88,100,000 0
corvus-runtime-r2r deno 56,000,000 79,540 945,030 n/a 0
corvus-runtime-aot deno 6,400,000 88,290 1,090,000 410,000 0
corvus-image-jit deno 125,700,000 81,090 904,820 26,110,000 0
corvus-image-aot deno 6,200,000 88,290 1,090,000 150,000 0
corvus-generated-jit deno 126,500,000 75,080 880,900 34,910,000 0
corvus-generated-aot deno 6,400,000 88,690 1,020,000 190,000 0
blaze-compile dependabot 8,400,000 145,980 4,298,445 1,600,000 n/a
blaze-template dependabot 8,000,000 145,980 4,298,445 0 n/a
corvus-runtime-jit dependabot 145,200,000 127,690 591,520 94,510,000 0
corvus-runtime-r2r dependabot 50,200,000 132,940 692,470 n/a 0
corvus-runtime-aot dependabot 4,900,000 134,470 643,070 270,000 0
corvus-image-jit dependabot 119,600,000 127,690 591,520 23,510,000 0
corvus-image-aot dependabot 4,500,000 134,470 643,070 240,000 0
corvus-generated-jit dependabot 108,200,000 131,920 555,080 32,110,000 0
corvus-generated-aot dependabot 4,800,000 133,010 649,640 130,000 0
blaze-compile draft-04 47,200,000 3,150,000 41,740,414 1,900,000 n/a
blaze-template draft-04 42,700,000 3,150,000 41,740,414 0 n/a
corvus-runtime-jit draft-04 220,200,000 1,700,000 5,600,000 73,800,000 0
corvus-runtime-r2r draft-04 170,400,000 1,800,000 6,420,000 n/a 0
corvus-runtime-aot draft-04 21,300,000 1,950,000 7,160,000 280,000 0
corvus-image-jit draft-04 187,500,000 1,700,000 5,600,000 23,710,000 0
corvus-image-aot draft-04 20,200,000 1,950,000 7,160,000 100,000 0
corvus-generated-jit draft-04 175,000,000 1,850,000 5,740,000 37,670,000 0
corvus-generated-aot draft-04 20,200,000 1,850,000 7,150,000 170,000 0
blaze-compile fabric-mod 10,600,000 332,870 5,618,617 2,700,000 n/a
blaze-template fabric-mod 9,700,000 332,870 5,618,617 0 n/a
corvus-runtime-jit fabric-mod 151,400,000 168,740 729,340 77,700,000 0
corvus-runtime-r2r fabric-mod 56,600,000 170,550 788,530 n/a 0
corvus-runtime-aot fabric-mod 4,800,000 166,480 842,090 320,000 0
corvus-image-jit fabric-mod 125,600,000 168,740 729,340 29,270,000 0
corvus-image-aot fabric-mod 4,600,000 166,480 842,090 120,000 0
corvus-generated-jit fabric-mod 130,300,000 164,810 648,960 39,810,000 0
corvus-generated-aot fabric-mod 5,300,000 163,070 792,820 160,000 0
blaze-compile geojson 193,400,000 6,410,000 191,609,322 10,400,000 n/a
blaze-template geojson 191,700,000 6,410,000 191,609,322 0 n/a
corvus-runtime-jit geojson 459,100,000 5,170,000 51,780,000 79,450,000 0
corvus-runtime-r2r geojson 334,800,000 5,150,000 55,220,000 n/a 0
corvus-runtime-aot geojson 138,000,000 5,840,000 58,870,000 910,000 0
corvus-image-jit geojson 397,900,000 5,170,000 51,780,000 23,740,000 0
corvus-image-aot geojson 136,900,000 5,840,000 58,870,000 500,000 0
corvus-generated-jit geojson 391,200,000 5,400,000 50,540,000 35,970,000 0
corvus-generated-aot geojson 133,400,000 6,260,000 58,950,000 470,000 0
blaze-compile gitpod-configuration 8,400,000 101,870 3,787,384 2,700,000 n/a
blaze-template gitpod-configuration 7,700,000 101,870 3,787,384 0 n/a
corvus-runtime-jit gitpod-configuration 148,900,000 70,700 484,360 82,120,000 0
corvus-runtime-r2r gitpod-configuration 47,000,000 70,410 504,550 n/a 0
corvus-runtime-aot gitpod-configuration 4,200,000 67,180 514,180 350,000 0
corvus-image-jit gitpod-configuration 110,400,000 70,700 484,360 27,150,000 0
corvus-image-aot gitpod-configuration 4,200,000 67,180 514,180 120,000 0
corvus-generated-jit gitpod-configuration 110,000,000 68,290 490,680 36,420,000 0
corvus-generated-aot gitpod-configuration 4,400,000 68,660 487,990 160,000 0
blaze-compile helm-chart-lock 14,400,000 81,260 12,789,329 400,000 n/a
blaze-template helm-chart-lock 15,300,000 81,260 12,789,329 0 n/a
corvus-runtime-jit helm-chart-lock 133,400,000 186,770 1,270,000 88,370,000 0
corvus-runtime-r2r helm-chart-lock 63,100,000 188,300 1,460,000 n/a 0
corvus-runtime-aot helm-chart-lock 6,300,000 197,360 1,360,000 170,000 0
corvus-image-jit helm-chart-lock 121,700,000 186,770 1,270,000 15,840,000 0
corvus-image-aot helm-chart-lock 6,600,000 197,360 1,360,000 40,000 0
corvus-generated-jit helm-chart-lock 119,100,000 184,970 1,120,000 28,710,000 0
corvus-generated-aot helm-chart-lock 7,100,000 199,490 1,300,000 60,000 0
blaze-compile importmap 6,500,000 15,680 3,522,365 100,000 n/a
blaze-template importmap 6,500,000 15,680 3,522,365 0 n/a
corvus-runtime-jit importmap 129,800,000 20,280 328,170 55,760,000 0
corvus-runtime-r2r importmap 44,900,000 19,870 322,330 n/a 0
corvus-runtime-aot importmap 4,100,000 22,470 362,800 90,000 0
corvus-image-jit importmap 106,900,000 20,280 328,170 15,200,000 0
corvus-image-aot importmap 4,100,000 22,470 362,800 30,000 0
corvus-generated-jit importmap 99,200,000 21,080 279,930 28,190,000 0
corvus-generated-aot importmap 4,800,000 23,600 348,120 50,000 0
blaze-compile jasmine 5,600,000 53,270 2,402,245 1,100,000 n/a
blaze-template jasmine 5,500,000 53,270 2,402,245 0 n/a
corvus-runtime-jit jasmine 138,600,000 52,150 289,070 99,430,000 0
corvus-runtime-r2r jasmine 44,700,000 51,600 273,910 n/a 0
corvus-runtime-aot jasmine 3,300,000 53,880 243,140 170,000 0
corvus-image-jit jasmine 111,000,000 52,150 289,070 18,310,000 0
corvus-image-aot jasmine 3,200,000 53,880 243,140 70,000 0
corvus-generated-jit jasmine 107,900,000 52,960 231,250 31,720,000 0
corvus-generated-aot jasmine 3,500,000 57,110 270,520 90,000 0
blaze-compile jsconfig 13,900,000 179,690 3,366,343 8,200,000 n/a
blaze-template jsconfig 12,200,000 179,690 3,366,343 0 n/a
corvus-runtime-jit jsconfig 153,300,000 145,720 464,810 92,000,000 0
corvus-runtime-r2r jsconfig 53,400,000 145,660 476,010 n/a 0
corvus-runtime-aot jsconfig 4,500,000 155,710 483,620 910,000 0
corvus-image-jit jsconfig 135,700,000 145,720 464,810 30,090,000 0
corvus-image-aot jsconfig 3,900,000 155,710 483,620 410,000 0
corvus-generated-jit jsconfig 118,700,000 145,950 428,040 39,280,000 0
corvus-generated-aot jsconfig 4,700,000 148,890 480,530 490,000 0
blaze-compile jshintrc 9,400,000 466,250 4,897,507 2,300,000 n/a
blaze-template jshintrc 9,200,000 466,250 4,897,507 0 n/a
corvus-runtime-jit jshintrc 127,500,000 107,500 624,760 79,450,000 0
corvus-runtime-r2r jshintrc 47,900,000 108,130 651,690 n/a 0
corvus-runtime-aot jshintrc 4,700,000 108,090 732,530 250,000 0
corvus-image-jit jshintrc 102,900,000 107,500 624,760 21,580,000 0
corvus-image-aot jshintrc 4,100,000 108,090 732,530 100,000 0
corvus-generated-jit jshintrc 107,800,000 99,830 568,510 31,440,000 0
corvus-generated-aot jshintrc 5,000,000 112,090 716,480 150,000 0
blaze-compile krakend 55,300,000 85,330 989,104 62,500,000 n/a
blaze-template krakend 44,600,000 85,330 989,104 0 n/a
corvus-runtime-jit krakend 177,000,000 66,350 155,060 109,830,000 0
corvus-runtime-r2r krakend 61,900,000 66,490 178,080 n/a 0
corvus-runtime-aot krakend 8,600,000 75,640 197,710 5,200,000 0
corvus-image-jit krakend 119,900,000 66,350 155,060 36,060,000 0
corvus-image-aot krakend 5,500,000 75,640 197,710 2,090,000 0
corvus-generated-jit krakend 124,500,000 64,290 158,620 52,790,000 0
corvus-generated-aot krakend 5,800,000 79,280 188,090 2,360,000 0
blaze-compile lazygit 16,000,000 56,730 989,353 13,000,000 n/a
blaze-template lazygit 13,400,000 56,730 989,353 0 n/a
corvus-runtime-jit lazygit 159,800,000 38,180 146,730 109,520,000 0
corvus-runtime-r2r lazygit 56,900,000 39,410 150,680 n/a 0
corvus-runtime-aot lazygit 4,500,000 42,100 151,930 1,370,000 0
corvus-image-jit lazygit 123,800,000 38,180 146,730 37,070,000 0
corvus-image-aot lazygit 3,500,000 42,100 151,930 870,000 0
corvus-generated-jit lazygit 123,000,000 39,260 141,390 45,210,000 0
corvus-generated-aot lazygit 4,100,000 42,800 153,820 1,000,000 0
blaze-compile lerna 6,400,000 60,500 3,105,315 700,000 n/a
blaze-template lerna 6,300,000 60,500 3,105,315 0 n/a
corvus-runtime-jit lerna 153,600,000 34,890 319,900 69,380,000 0
corvus-runtime-r2r lerna 42,300,000 34,240 297,190 n/a 0
corvus-runtime-aot lerna 3,700,000 35,840 300,740 150,000 0
corvus-image-jit lerna 100,400,000 34,890 319,900 16,060,000 0
corvus-image-aot lerna 3,600,000 35,840 300,740 60,000 0
corvus-generated-jit lerna 102,900,000 33,840 255,960 28,770,000 0
corvus-generated-aot lerna 3,900,000 37,510 286,330 100,000 0
blaze-compile nest-cli 9,700,000 90,630 3,642,355 3,900,000 n/a
blaze-template nest-cli 9,000,000 90,630 3,642,355 0 n/a
corvus-runtime-jit nest-cli 140,000,000 63,270 389,370 78,180,000 0
corvus-runtime-r2r nest-cli 49,500,000 62,820 411,020 n/a 0
corvus-runtime-aot nest-cli 4,400,000 63,020 395,230 420,000 0
corvus-image-jit nest-cli 113,100,000 63,270 389,370 25,110,000 0
corvus-image-aot nest-cli 4,400,000 63,020 395,230 170,000 0
corvus-generated-jit nest-cli 110,200,000 61,930 344,130 31,480,000 0
corvus-generated-aot nest-cli 4,400,000 66,730 400,180 220,000 0
blaze-compile omnisharp 9,700,000 239,020 4,369,443 3,700,000 n/a
blaze-template omnisharp 9,300,000 239,020 4,369,443 0 n/a
corvus-runtime-jit omnisharp 134,600,000 67,490 465,530 73,260,000 0
corvus-runtime-r2r omnisharp 50,300,000 70,580 470,390 n/a 0
corvus-runtime-aot omnisharp 4,500,000 74,680 549,960 380,000 0
corvus-image-jit omnisharp 108,200,000 67,490 465,530 22,640,000 0
corvus-image-aot omnisharp 4,200,000 74,680 549,960 160,000 0
corvus-generated-jit omnisharp 117,500,000 68,320 403,250 30,150,000 0
corvus-generated-aot omnisharp 4,900,000 79,060 551,080 210,000 0
blaze-compile openapi 90,600,000 5,820,000 77,685,603 11,800,000 n/a
blaze-template openapi 86,000,000 5,820,000 77,685,603 0 n/a
corvus-runtime-jit openapi 298,900,000 2,990,000 13,040,000 97,570,000 0
corvus-runtime-r2r openapi 250,200,000 2,850,000 14,330,000 n/a 0
corvus-runtime-aot openapi 50,600,000 2,930,000 16,070,000 1,270,000 0
corvus-image-jit openapi 278,900,000 2,990,000 13,040,000 35,210,000 0
corvus-image-aot openapi 49,200,000 2,930,000 16,070,000 790,000 0
corvus-generated-jit openapi 259,900,000 2,910,000 13,110,000 55,290,000 0
corvus-generated-aot openapi 48,800,000 2,980,000 16,300,000 970,000 0
blaze-compile pre-commit-hooks 8,700,000 218,530 4,840,491 1,500,000 n/a
blaze-template pre-commit-hooks 8,800,000 218,530 4,840,491 0 n/a
corvus-runtime-jit pre-commit-hooks 142,500,000 95,500 547,110 77,020,000 0
corvus-runtime-r2r pre-commit-hooks 48,800,000 96,820 604,010 n/a 0
corvus-runtime-aot pre-commit-hooks 4,700,000 100,730 617,440 280,000 0
corvus-image-jit pre-commit-hooks 121,200,000 95,500 547,110 21,770,000 0
corvus-image-aot pre-commit-hooks 4,300,000 100,730 617,440 120,000 0
corvus-generated-jit pre-commit-hooks 102,100,000 97,270 515,760 29,990,000 0
corvus-generated-aot pre-commit-hooks 4,700,000 101,100 620,010 160,000 0
blaze-compile pulumi 16,500,000 257,040 12,880,338 2,400,000 n/a
blaze-template pulumi 16,700,000 257,040 12,880,338 0 n/a
corvus-runtime-jit pulumi 153,500,000 181,360 1,140,000 75,810,000 0
corvus-runtime-r2r pulumi 66,000,000 181,470 1,200,000 n/a 0
corvus-runtime-aot pulumi 6,600,000 187,500 1,150,000 310,000 0
corvus-image-jit pulumi 132,700,000 181,360 1,140,000 27,390,000 0
corvus-image-aot pulumi 6,400,000 187,500 1,150,000 130,000 0
corvus-generated-jit pulumi 118,400,000 178,570 970,280 36,670,000 0
corvus-generated-aot pulumi 6,500,000 192,980 1,160,000 160,000 0
blaze-compile semantic-release 7,300,000 43,050 3,736,471 900,000 n/a
blaze-template semantic-release 6,800,000 43,050 3,736,471 0 n/a
corvus-runtime-jit semantic-release 145,900,000 37,550 415,530 76,340,000 0
corvus-runtime-r2r semantic-release 45,000,000 37,770 441,720 n/a 0
corvus-runtime-aot semantic-release 3,900,000 40,010 455,270 230,000 0
corvus-image-jit semantic-release 110,600,000 37,550 415,530 17,520,000 0
corvus-image-aot semantic-release 3,800,000 40,010 455,270 60,000 0
corvus-generated-jit semantic-release 107,700,000 36,320 382,870 30,290,000 0
corvus-generated-aot semantic-release 4,200,000 38,370 435,720 80,000 0
blaze-compile stale 6,200,000 72,020 3,040,316 600,000 n/a
blaze-template stale 6,500,000 72,020 3,040,316 0 n/a
corvus-runtime-jit stale 144,100,000 70,180 402,820 75,470,000 0
corvus-runtime-r2r stale 47,400,000 71,870 409,370 n/a 0
corvus-runtime-aot stale 4,000,000 76,130 410,320 170,000 0
corvus-image-jit stale 111,300,000 70,180 402,820 23,660,000 0
corvus-image-aot stale 4,000,000 76,130 410,320 80,000 0
corvus-generated-jit stale 106,700,000 73,090 369,440 32,920,000 0
corvus-generated-aot stale 4,400,000 74,390 422,400 100,000 0
blaze-compile stylecop 9,200,000 121,080 3,962,403 2,500,000 n/a
blaze-template stylecop 8,100,000 121,080 3,962,403 0 n/a
corvus-runtime-jit stylecop 145,200,000 103,870 566,240 93,000,000 0
corvus-runtime-r2r stylecop 49,200,000 103,680 564,890 n/a 0
corvus-runtime-aot stylecop 4,700,000 112,710 570,830 290,000 0
corvus-image-jit stylecop 129,300,000 103,870 566,240 26,590,000 0
corvus-image-aot stylecop 4,200,000 112,710 570,830 120,000 0
corvus-generated-jit stylecop 114,800,000 102,410 484,260 39,760,000 0
corvus-generated-aot stylecop 4,800,000 112,670 565,570 160,000 0
blaze-compile tmuxinator 5,500,000 52,530 1,907,499 1,300,000 n/a
blaze-template tmuxinator 5,600,000 52,530 1,907,499 0 n/a
corvus-runtime-jit tmuxinator 134,900,000 23,070 231,740 71,270,000 0
corvus-runtime-r2r tmuxinator 43,900,000 23,310 262,150 n/a 0
corvus-runtime-aot tmuxinator 3,500,000 26,020 268,880 220,000 0
corvus-image-jit tmuxinator 107,400,000 23,070 231,740 17,280,000 0
corvus-image-aot tmuxinator 3,500,000 26,020 268,880 140,000 0
corvus-generated-jit tmuxinator 107,400,000 24,540 235,650 30,640,000 0
corvus-generated-aot tmuxinator 3,800,000 26,560 280,010 100,000 0
blaze-compile ui5 35,500,000 236,230 5,037,535 37,300,000 n/a
blaze-template ui5 27,000,000 236,230 5,037,535 0 n/a
corvus-runtime-jit ui5 193,500,000 373,350 864,660 116,140,000 0
corvus-runtime-r2r ui5 89,500,000 356,270 971,460 n/a 0
corvus-runtime-aot ui5 8,700,000 376,110 980,900 3,460,000 0
corvus-image-jit ui5 166,900,000 373,350 864,660 73,160,000 0
corvus-image-aot ui5 7,000,000 376,110 980,900 2,120,000 0
corvus-generated-jit ui5 134,400,000 372,810 850,560 51,270,000 0
corvus-generated-aot ui5 7,500,000 372,630 952,860 2,010,000 0
blaze-compile ui5-manifest 145,800,000 1,180,000 13,836,265 199,500,000 n/a
blaze-template ui5-manifest 194,600,000 1,180,000 13,836,265 0 n/a
corvus-runtime-jit ui5-manifest 260,500,000 1,280,000 2,520,000 145,970,000 0
corvus-runtime-r2r ui5-manifest 133,500,000 1,240,000 2,670,000 n/a 0
corvus-runtime-aot ui5-manifest 23,700,000 1,290,000 2,890,000 10,450,000 0
corvus-image-jit ui5-manifest 183,300,000 1,280,000 2,520,000 82,930,000 0
corvus-image-aot ui5-manifest 13,700,000 1,290,000 2,890,000 4,800,000 0
corvus-generated-jit ui5-manifest 174,200,000 1,240,000 2,570,000 68,380,000 0
corvus-generated-aot ui5-manifest 14,200,000 1,270,000 2,860,000 4,880,000 0
blaze-compile unreal-engine-uproject 8,400,000 217,520 3,861,450 2,900,000 n/a
blaze-template unreal-engine-uproject 8,000,000 217,520 3,861,450 0 n/a
corvus-runtime-jit unreal-engine-uproject 146,200,000 160,620 570,060 82,740,000 0
corvus-runtime-r2r unreal-engine-uproject 52,900,000 159,290 647,250 n/a 0
corvus-runtime-aot unreal-engine-uproject 4,500,000 190,960 661,130 320,000 0
corvus-image-jit unreal-engine-uproject 121,500,000 160,620 570,060 26,710,000 0
corvus-image-aot unreal-engine-uproject 4,000,000 190,960 661,130 510,000 0
corvus-generated-jit unreal-engine-uproject 113,300,000 159,420 565,460 40,830,000 0
corvus-generated-aot unreal-engine-uproject 4,700,000 219,100 730,750 530,000 0
blaze-compile vercel 12,400,000 114,670 2,760,389 7,800,000 n/a
blaze-template vercel 10,000,000 114,670 2,760,389 0 n/a
corvus-runtime-jit vercel 161,400,000 87,700 358,930 77,630,000 0
corvus-runtime-r2r vercel 55,200,000 87,570 411,120 n/a 0
corvus-runtime-aot vercel 4,600,000 100,090 423,320 750,000 0
corvus-image-jit vercel 130,000,000 87,700 358,930 27,490,000 0
corvus-image-aot vercel 4,300,000 100,090 423,320 680,000 0
corvus-generated-jit vercel 114,700,000 87,020 360,370 36,810,000 0
corvus-generated-aot vercel 4,800,000 103,740 407,850 740,000 0
blaze-compile yamllint 6,400,000 6,340 3,811,387 300,000 n/a
blaze-template yamllint 6,400,000 6,340 3,811,387 0 n/a
corvus-runtime-jit yamllint 134,200,000 11,730 424,090 58,760,000 0
corvus-runtime-r2r yamllint 43,900,000 14,980 434,350 n/a 0
corvus-runtime-aot yamllint 3,900,000 13,260 448,540 110,000 0
corvus-image-jit yamllint 107,600,000 11,730 424,090 15,730,000 0
corvus-image-aot yamllint 3,800,000 13,260 448,540 30,000 0
corvus-generated-jit yamllint 107,300,000 11,700 380,350 28,210,000 0
corvus-generated-aot yamllint 4,200,000 15,180 445,960 60,000 0

Parse and evaluate once

One pass over each Sourcemeta corpus, every instance parsed and evaluated once (the shape of a service validating each request), at steady state. Per corpus: the sum over instances of the per-instance mean. Blaze 16.10.0 (jsonschema validate --fast), Corvus runtime evaluator on .NET 10 (JIT). Same machine, same instances, each corpus in its own process.

corpus instances Blaze via CLI Blaze less CLI bookkeeping Corvus Corvus / Blaze
ansible-meta 333 1.44 ms 1.03 ms 213.3 µs 0.21
aws-cdk 483 3.11 ms 2.50 ms 358.2 µs 0.14
babelrc 794 2.18 ms 1.19 ms 279.2 µs 0.24
clang-format 133 508.4 µs 342.1 µs 59.1 µs 0.17
cmake-presets 967 20.01 ms 18.80 ms 3.63 ms 0.19
code-climate 2484 10.73 ms 7.62 ms 1.08 ms 0.14
cql2 109 487.4 µs 351.2 µs 95.7 µs 0.27
cspell 981 6.83 ms 5.61 ms 897.5 µs 0.16
cypress 981 4.39 ms 3.17 ms 542.1 µs 0.17
deno 987 6.42 ms 5.19 ms 904.8 µs 0.17
dependabot 967 4.30 ms 3.09 ms 591.5 µs 0.19
draft-04 563 41.74 ms 41.04 ms 5.60 ms 0.14
fabric-mod 911 5.62 ms 4.48 ms 729.3 µs 0.16
geojson 500 191.61 ms 190.98 ms 51.78 ms 0.27
gitpod-configuration 986 3.79 ms 2.55 ms 484.4 µs 0.19
helm-chart-lock 3888 12.79 ms 7.93 ms 1.27 ms 0.16
importmap 964 3.52 ms 2.32 ms 328.2 µs 0.14
jasmine 980 2.40 ms 1.18 ms 289.1 µs 0.25
jsconfig 981 3.37 ms 2.14 ms 464.8 µs 0.22
jshintrc 966 4.90 ms 3.69 ms 624.8 µs 0.17
krakend 47 989.1 µs 930.4 µs 155.1 µs 0.17
lazygit 280 989.4 µs 639.4 µs 146.7 µs 0.23
lerna 985 3.11 ms 1.87 ms 319.9 µs 0.17
nest-cli 1025 3.64 ms 2.36 ms 389.4 µs 0.16
omnisharp 987 4.37 ms 3.14 ms 465.5 µs 0.15
openapi 107 77.69 ms 77.55 ms 13.04 ms 0.17
pre-commit-hooks 985 4.84 ms 3.61 ms 547.1 µs 0.15
pulumi 3807 12.88 ms 8.12 ms 1.14 ms 0.14
semantic-release 794 3.74 ms 2.74 ms 415.5 µs 0.15
stale 961 3.04 ms 1.84 ms 402.8 µs 0.22
stylecop 983 3.96 ms 2.73 ms 566.2 µs 0.21
tmuxinator 382 1.91 ms 1.43 ms 231.7 µs 0.16
ui5 942 5.04 ms 3.86 ms 864.7 µs 0.22
ui5-manifest 611 13.84 ms 13.07 ms 2.52 ms 0.19
unreal-engine-uproject 859 3.86 ms 2.79 ms 570.1 µs 0.20
vercel 710 2.76 ms 1.87 ms 358.9 µs 0.19
yamllint 984 3.81 ms 2.58 ms 424.1 µs 0.16
median 3.86 ms 2.74 ms 484.4 µs 0.17

Corvus is faster on 37 of 37 corpora; geometric mean of Corvus over Blaze (less bookkeeping) 0.18.

How the two sides were timed. Corvus: in process, a timer around parse, evaluate and dispose per instance, 100 loops, clock overhead subtracted. Blaze: its benchmark mode times evaluation of an already parsed instance only, so the parse-inclusive figure is taken by difference: the CLI validating ten copies of the corpus (validate --fast, output discarded) minus the same over one instance, per instance, median of three. That difference carries the CLI's per-instance bookkeeping (reading the line, result handling, output), measured at 1.25 µs per instance with trivial instances ({}, 1) against the empty schema; the fourth column subtracts it, and the ratio uses that column. What remains in Blaze's figure is its JSON parse and its evaluation; an in-process harness would remove the last of the doubt.

Warm evaluation, Corvus against Blaze

Corpus Instances Corvus runtime (JIT) Blaze Corvus / Blaze Corvus fastest
ansible-meta 333 77.7 µs 111.9 µs 0.69 x
aws-cdk 483 16.6 µs 24.6 µs 0.67 x
babelrc 794 50.7 µs 48.0 µs 1.06
clang-format 133 16.6 µs 47.2 µs 0.35 x
cmake-presets 967 1.62 ms 1.82 ms 0.89 x
code-climate 2484 61.6 µs 81.8 µs 0.75 x
cql2 109 53.0 µs 105.8 µs 0.50 x
cspell 981 168.6 µs 250.9 µs 0.67 x
cypress 981 51.3 µs 77.7 µs 0.66 x
deno 987 81.1 µs 117.5 µs 0.69 x
dependabot 967 127.7 µs 146.0 µs 0.87 x
draft-04 563 1.70 ms 3.15 ms 0.54 x
fabric-mod 911 168.7 µs 332.9 µs 0.51 x
geojson 500 5.17 ms 6.41 ms 0.81 x
gitpod-configuration 986 70.7 µs 101.9 µs 0.69 x
helm-chart-lock 3888 186.8 µs 81.3 µs 2.30
importmap 964 20.3 µs 15.7 µs 1.29
jasmine 980 52.1 µs 53.3 µs 0.98 x
jsconfig 981 145.7 µs 179.7 µs 0.81 x
jshintrc 966 107.5 µs 466.2 µs 0.23 x
krakend 47 66.3 µs 85.3 µs 0.78 x
lazygit 280 38.2 µs 56.7 µs 0.67 x
lerna 985 34.9 µs 60.5 µs 0.58 x
nest-cli 1025 63.3 µs 90.6 µs 0.70 x
omnisharp 987 67.5 µs 239.0 µs 0.28 x
openapi 107 2.99 ms 5.82 ms 0.51 x
pre-commit-hooks 985 95.5 µs 218.5 µs 0.44 x
pulumi 3807 181.4 µs 257.0 µs 0.71 x
semantic-release 794 37.5 µs 43.0 µs 0.87 x
stale 961 70.2 µs 72.0 µs 0.97 x
stylecop 983 103.9 µs 121.1 µs 0.86 x
tmuxinator 382 23.1 µs 52.5 µs 0.44 x
ui5 942 373.4 µs 236.2 µs 1.58
ui5-manifest 611 1.28 ms 1.18 ms 1.08
unreal-engine-uproject 859 160.6 µs 217.5 µs 0.74 x
vercel 710 87.7 µs 114.7 µs 0.76 x
yamllint 984 11.7 µs 6.3 µs 1.85

Corvus is fastest on 31 of 37 corpora. Geometric mean of Corvus over Blaze: 0.73.