docs / build
suite format
on this page
A suite is a JSON file of calls to one JavaScript function and the answers expected.
the file
{
"version": 1,
"runtime": "js-boa/1",
"entry": "fib",
"tests": [
{ "args": [10], "expected": 55 },
{ "args": [0], "expected": 0 }
]
}
A test passes when fib(...args) returns a value equal to expected. Numbers compare by value (1 equals 1.0), object key order is ignored, and array order matters.
a check function
For a job with no single right answer, such as a trading strategy, the suite can carry a check function instead of relying on exact answers. A test then passes when check(result, expected, args) returns exactly true, so expected can hold thresholds and hidden data rather than an answer.
{
"version": 1,
"runtime": "js-boa/1",
"entry": "positions",
"check": "function check(r, e, args) { /* score r on e.futures */ return pnl >= e.min_pnl }",
"tests": [
{ "args": [[[101, 102, 104], [88, 87, 85]]], "expected": { "futures": [[105, 107], [84, 82]], "min_pnl": 4 } }
]
}
checkruns in its own fresh sandbox, apart from the submission, with its own budget, so a submission can neither reach it nor starve it- anything but
true, a throw, a timeout or another value, fails the test - the submission sees only
args; prices it is scored on go inexpected, which onlychecksees - keep each window's future out of every other window's history, or a strategy could read it from there
- when the job is posted, the grader runs the reference through
checkin the enclave, so a broken checker cannot open a job
rules
| Rule | Limit |
|---|---|
| file size | 256 KiB |
| tests | 1 to 32 |
| nesting | 32 levels |
| numbers | finite; whole numbers within ±(2^53 − 1) |
| object keys | no __proto__ |
| entry | a plain identifier, not a keyword or built-in |
| check | optional, 1 byte to 64 KiB of JavaScript defining check |
the hash
The suite hash is the SHA-256 of the file's exact bytes, so anyone can recompute it:
shasum -a 256 suite.json
sphinx suite hash suite.json # the same, and checks the rules
the runtime
js-boa/1 runs plain JavaScript in a sandboxed engine with no network, no files, no regular expressions and fixed limits. Each test gets a fresh sandbox, Date is a fixed clock and Math.random throws, so a run gives the same result every time. Budgets are in grader limits.