Skip to content

Measuring on the judge itself#

rbx time measures wherever rbx runs, so the straightforward way to get limits for a judge is to run it on that judge's hardware. --runner is the other way: it measures your solutions on the judge park, through the judge's own CLI, and feeds those timings into the same estimation you would get locally.

rbx time -p moj --runner moj

Why measure there#

A time limit is a claim about hardware. Estimate on your laptop and you have a limit that is correct for your laptop — different CPU, different compiler flags, none of the neighbouring load a judge machine carries. The margin you thought you left may not be there.

Measuring on the park removes the guess: the timings are the judge's own, so the limit is derived from the machine that will enforce it.

If what you want is a verdict rather than a limit, reach for rbx run --runner instead. The two commands share everything below.

Timing on MOJ#

You must be logged in to the moj CLI — rbx reuses its session and never handles your credentials. On macOS the CLI also needs a bash newer than the one the system ships; if it refuses to start, see The MOJ CLI needs bash 4 or newer.

rbx uploads a throwaway problem of its own, named <your login>#rbxt-<slug> from the slug in .rbx-id at the package root — the one identity your package has on any remote judge, shared with the DOMjudge backend. The slug is minted once and then kept, so every later run reuses the same problem instead of orphaning one; commit .rbx-id if you want a co-setter to reach that problem too. The id is also written to .moj-id, which is what the moj CLI itself reads.

It never touches a problem it did not create: a package already bound to a real MOJ problem is refused by name rather than overwritten.

A judge reports less than the local sandbox does. It hands back a verdict, not the bytes your solution wrote, so some of what a local run shows you has no counterpart at all:

  • No memory usage, no .out/.err artifacts, and a verdict code rather than the checker's own message.
  • --runs greater than one, a sanitizer, and interactive (communication) problems are refused by name before anything is uploaded — each would produce a report answering a different question than the one you asked.

rbx refuses whatever it cannot answer honestly on a given backend, rather than quietly reporting less than you asked for.

The two phases, and the two uploads#

MOJ enforces the time limit from inside the package, so the limit rbx is measuring under has to be in what it uploads. The two phases of a run measure under different limits, and therefore need different packages:

  1. Estimating runs the accepted solutions under the estimation cap — one limit for every language.
  2. Checking the upper bound runs the too-slow solutions at the bound the estimate demands of them, which differs per language group.

So a full run uploads twice, and the second upload usually costs a calibration wait as well. Each trip through the language-group picker that changes a limit costs another one.

Finished testruns are cached, so re-running rbx time, or regrouping back onto limits already probed, costs no judge time at all. And --skip-slow stops after the estimate, which is the one-upload path:

rbx time -p moj --runner moj --skip-slow

Each command uploads to a throwaway problem of its own — …-run for rbx run, … and …-slow for the two phases of rbx time — so alternating between the commands never costs a re-upload.

Timing on DOMjudge#

rbx time -p domjudge --runner domjudge

What it needs, and what it cannot tell you#

rbx talks to DOMjudge's REST API directly, so there is no CLI to install. Point it at the instance with three environment variables:

export RBX_DOMJUDGE_SERVER=https://judge.example.edu/
export RBX_DOMJUDGE_USERNAME=your-admin-user
export RBX_DOMJUDGE_PASSWORD=...

They are environment variables rather than package settings on purpose: which DOMjudge you can reach is a property of your machine, and committing one setter's server would break the package for everyone else.

The account needs admin rights. rbx creates a private rbx-timing contest the first time it runs, uploads a throwaway rbxt-… problem into it, and submits as the built-in domjudge team — none of which a plain jury account may do. The problem is named from the slug in .rbx-id at the package root — the one identity your package has on any remote judge, shared with the MOJ backend — so it stays the same problem as your testset grows, rather than leaving a trail of abandoned ones behind. The contest has no scoreboard anyone reads, and nothing rbx does there touches a real contest.

Before uploading anything, rbx checks the instance and refuses by name what cannot work, rather than leaving you watching a run that will never finish:

  • verification_required turned on. It hides every judgement and run until a human verifies it, so a timing run would poll an empty list until it gave up.
  • No enabled judgehost, or one that is enabled but has not asked for work recently. Either way submissions would be stored and never judged — but the fix differs, so the two are reported differently.

A judge reports less than the local sandbox does. It hands back a verdict, not the bytes your solution wrote:

  • No memory usage, no .out/.err artifacts, and a verdict rather than the checker's own message. DOMjudge has no memory-limit verdict at all, so a solution that runs out of memory comes back as a runtime error.
  • Times are CPU time, not wall clock. A solution killed for exceeding the limit reports the CPU it actually burned, which for a sleeping or I/O-bound solution is close to zero. The verdict is what says it was too slow.
  • --runs greater than one and sanitizers are refused by name before anything is uploaded — each would produce a report answering a different question than the one you asked.

Interactive (communication) problems do work. DOMjudge runs the interactor rbx ships as the problem's run script, so the verdict comes from the same program that judges locally — and a legacy interactor paired with a checker is chained so that both still run.

Solutions are submitted one at a time. A judgehost judges one submission at a time anyway, and two in flight on a multi-judgehost instance would land on different machines — whose timings are not comparable, which is the whole point of measuring remotely.

The two phases, and the two uploads#

DOMjudge enforces the time limit from inside the package, so — exactly as on MOJ — the two phases measure under different limits and therefore need different packages. Each phase uploads to a problem of its own, rbxt-…-estimation and rbxt-…-validation, so alternating between them never costs a re-upload.

The estimation package pins a deliberately generous limit. DOMjudge kills a run at the limit, so a package pinned near the answer you are looking for would truncate exactly the measurements that matter most.

Finished judgings are cached, and so is the upload: re-running rbx time, or regrouping back onto limits already probed, re-submits only the solutions whose source changed, and re-uploads the package only when the package itself changed. A judging rbx could not read — a compile error above all — is never cached, so fixing it and running again really does try again.

--skip-slow stops after the estimate, which is the one-upload path:

rbx time -p domjudge --runner domjudge --skip-slow