orcaLog in

Pipelines

mojo_history

Progress

Runs at onceRuns of this pipeline going now, out of the most the scheduler starts at once (max_runs); the others wait in the queue. Fill runs come on top, up to fill_max_runs. Docs ↗
0 running of 1, 245 queued

Config

PartitionsHow the work is split: each run does some partitions (months, days, hours or named keys), from the first one on. Backfills and the coverage calendar count in partitions. Docs ↗
keyed from the start
ScheduleThe cron line the scheduler fires on, in its time zone, and the partition each fire runs. None means runs start only from a sensor, a backfill or a person. Docs ↗
none
Run retriesHow many more times the scheduler starts a run that failed before leaving it failed for a person. A step's own retries, inside a run, come first. Docs ↗
3
PlacementWhere its runs go: control is the scheduler's own host, ec2 an instance launched for the run, worker:<pool> a remote orca worker of that pool, and default the scheduler's runner (ORCA_RUNNER). Docs ↗
worker:bgent (its runs wait for a worker of pool bgent, Workers)
BinaryThe program each run starts: the pipeline's Mojo code built into one executable, as orca register recorded it. Docs ↗
/opt/orca/pipelines/mojo_history

CodeThe functions this pipeline is made of, at the commit it's registered with. Hover a name to see its source; click it to open the file on its host. Docs ↗

PipelineWhere the pipeline is declared: its Pipeline[...] line. Docs ↗
mojo_history
StepsEach step's function, in the order the steps run. The Steps table below has what each reads and writes. Docs ↗
histories
TasksThe dista tasks its steps fan out to, each a function. Docs ↗
read_tags · read_history

Code hash 0x951171cd4bca7616A hash of what the binary says about itself: its steps, partitions and schedule. It changes when they do, not on every build. Docs ↗

ResourcesWhat its latest runs used, as the median (p50) and the most of them: what a run of it needs. A run's peak is its attempts' largest, its CPU their sum. Docs ↗

MeasuredHow many of its latest runs these are: the last 20 that recorded what they used.
its last 20 runs
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
p50 87 MB · max 252 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
p50 3.0 s · max 7.0 s
Peak coresThe most cores a run's processes kept busy over one sample's second: the CPUs to ask for. Not measured on macOS. Docs ↗
p50 0.1 · max 1.4

Steps docs

GraphTable
StepA step of the pipeline: a function that runs once per partition (mapped) or once over all of them (gather).KindHow the step runs: mapped, once per partition, or gather, once over all of them. Hover a kind for more.AfterThe steps this one waits for: it starts once they have succeeded.ReadsThe datasets the step reads.WritesThe datasets the step commits to.
histories pipelines/mojo_history.mojo:119gathermojo.reposmojo.commits, mojo.migrations, mojo.tags

Keys docs

From
mojo.repos @ 2296263712729538280 read 2026-11-08T10:36:52Z
Keys
901
Done
560
In live runs
341
Failed
0

Configuration

Registered from /tmp/orca.toml @ ac0c766d3f15 of https://github.com/magmalake/mojo-census, sha256 9ac40fcb5922.

KeyThe setting's name, as orca.toml and orca register's flags call it. Hover the name for what it sets.ValueThe value it is registered with.FromWhere the value came from: the code's default, orca.toml (and which table), a flag of orca register, or the scheduler's environment. A link opens the line that sets it.
placementWhere its runs start: default (the scheduler's runner), control, ec2, or worker:<pool>.worker:bgentorca.toml [pipelines.mojo_history]
head_typeThe EC2 instance type of a run's dista head.c7g.largecode default the ec2 runner's
worker_typeThe EC2 instance type of each dista worker.c7g.xlargecode default the ec2 runner's
workersHow many dista workers a run asks for, or min..max to scale between.4code default the pipeline's code the Pipeline[...] line
worker_procsDista worker processes on each worker instance; auto for one per core.1code default
max_runsThe most runs of the pipeline the scheduler starts at once.1orca.toml [defaults]
fillsall: the scheduler fills the whole range by itself. since_registration: from registration on.since_registrationcode default
env.DISTA_WORKER_SLOTSA variable set in its runs' environment.3orca.toml [pipelines.mojo_history]
env.ORCA_PUBLIC_WAREHOUSEA variable set in its runs' environment.s3://census-mlake-dev/warehouseenvironment ORCA_RUN_ENV