orcaLog in

Runs · run 2865

load_day 2024-08-20, attempt 1 succeeded

Host
73ccef5e07c2:166569
Started
2026-11-08T07:31:37Z
Ended
2026-11-08T07:33:36Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
711 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
181.3 s, up to 2.8 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/2865/load_day/2024-08-20/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T07:31:37Z [load_day 2024-08-20] attempt 1 started
2026-11-08T07:31:41Z [load_day 2024-08-20] 2024-08-20: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T07:31:47Z [load_day 2024-08-20] 2024-08-20 hour 0: 192504 events in 0 files, snapshot none (task 5997 ms: download 3725 ms overlapping gunzip 856 ms, parse 3668 ms, sort 0 ms, write 0 ms, index 652 ms: 6 candidates, 101119 repo-hours, 1115 releases; waiting 512 ms; commit 0 ms)
2026-11-08T07:31:48Z [load_day 2024-08-20] 2024-08-20 hour 1: 202118 events in 0 files, snapshot none (task 6090 ms: download 3498 ms overlapping gunzip 860 ms, parse 3722 ms, sort 0 ms, write 0 ms, index 730 ms: 5 candidates, 112596 repo-hours, 790 releases; waiting 490 ms; commit 0 ms)
2026-11-08T07:31:57Z [load_day 2024-08-20] 2024-08-20 hour 2: 164326 events in 0 files, snapshot none (task 8039 ms: download 4654 ms overlapping gunzip 1272 ms, parse 5228 ms, sort 0 ms, write 0 ms, index 778 ms: 11 candidates, 70909 repo-hours, 787 releases; waiting 511 ms; commit 0 ms)
2026-11-08T07:31:57Z [load_day 2024-08-20] 2024-08-20 hour 3: 178789 events in 0 files, snapshot none (task 8136 ms: download 4330 ms overlapping gunzip 1245 ms, parse 5082 ms, sort 0 ms, write 0 ms, index 914 ms: 7 candidates, 73629 repo-hours, 4743 releases; waiting 610 ms; commit 0 ms)
2026-11-08T07:32:02Z [load_day 2024-08-20] 2024-08-20 hour 4: 156404 events in 0 files, snapshot none (task 5327 ms: download 2449 ms overlapping gunzip 754 ms, parse 3134 ms, sort 0 ms, write 0 ms, index 554 ms: 7 candidates, 59122 repo-hours, 680 releases; waiting 528 ms; commit 0 ms)
2026-11-08T07:32:04Z [load_day 2024-08-20] 2024-08-20 hour 5: 175899 events in 0 files, snapshot none (task 6588 ms: download 3460 ms overlapping gunzip 1008 ms, parse 4184 ms, sort 0 ms, write 0 ms, index 621 ms: 4 candidates, 65893 repo-hours, 841 releases; waiting 514 ms; commit 0 ms)
2026-11-08T07:32:10Z [load_day 2024-08-20] 2024-08-20 hour 6: 208513 events in 0 files, snapshot none (task 7378 ms: download 4591 ms overlapping gunzip 1171 ms, parse 4824 ms, sort 0 ms, write 0 ms, index 643 ms: 7 candidates, 90523 repo-hours, 1043 releases; waiting 497 ms; commit 0 ms)
2026-11-08T07:32:13Z [load_day 2024-08-20] 2024-08-20 hour 7: 211243 events in 0 files, snapshot none (task 9050 ms: download 5489 ms overlapping gunzip 1389 ms, parse 5753 ms, sort 0 ms, write 0 ms, index 1071 ms: 8 candidates, 82167 repo-hours, 1019 releases; waiting 542 ms; commit 0 ms)
2026-11-08T07:32:19Z [load_day 2024-08-20] 2024-08-20 hour 8: 216887 events in 0 files, snapshot none (task 7927 ms: download 5489 ms overlapping gunzip 1269 ms, parse 5194 ms, sort 0 ms, write 0 ms, index 633 ms: 10 candidates, 89201 repo-hours, 1158 releases; waiting 515 ms; commit 0 ms)
2026-11-08T07:32:22Z [load_day 2024-08-20] 2024-08-20 hour 9: 227459 events in 0 files, snapshot none (task 8380 ms: download 5894 ms overlapping gunzip 1279 ms, parse 5470 ms, sort 0 ms, write 0 ms, index 719 ms: 9 candidates, 95533 repo-hours, 1349 releases; waiting 595 ms; commit 0 ms)
2026-11-08T07:32:26Z [load_day 2024-08-20] 2024-08-20 hour 10: 213365 events in 0 files, snapshot none (task 6372 ms: download 3927 ms overlapping gunzip 1073 ms, parse 4207 ms, sort 0 ms, write 0 ms, index 606 ms: 14 candidates, 86922 repo-hours, 1094 releases; waiting 178 ms; commit 0 ms)
2026-11-08T07:32:30Z [load_day 2024-08-20] 2024-08-20 hour 11: 213953 events in 0 files, snapshot none (task 6840 ms: download 4179 ms overlapping gunzip 1016 ms, parse 4292 ms, sort 0 ms, write 0 ms, index 723 ms: 15 candidates, 84054 repo-hours, 1191 releases; waiting 519 ms; commit 0 ms)
2026-11-08T07:32:35Z [load_day 2024-08-20] 2024-08-20 hour 12: 268062 events in 0 files, snapshot none (task 9276 ms: download 4943 ms overlapping gunzip 1389 ms, parse 5622 ms, sort 0 ms, write 0 ms, index 1398 ms: 15 candidates, 131176 repo-hours, 1476 releases; waiting 527 ms; commit 0 ms)
2026-11-08T07:32:40Z [load_day 2024-08-20] 2024-08-20 hour 13: 249074 events in 0 files, snapshot none (task 9932 ms: download 7029 ms overlapping gunzip 1516 ms, parse 6652 ms, sort 0 ms, write 0 ms, index 919 ms: 12 candidates, 113274 repo-hours, 1279 releases; waiting 542 ms; commit 0 ms)
2026-11-08T07:32:49Z [load_day 2024-08-20] 2024-08-20 hour 14: 231553 events in 0 files, snapshot none (task 12898 ms: download 10162 ms overlapping gunzip 1966 ms, parse 7830 ms, sort 0 ms, write 0 ms, index 791 ms: 12 candidates, 98068 repo-hours, 1267 releases; waiting 1933 ms; commit 0 ms)
2026-11-08T07:32:52Z [load_day 2024-08-20] 2024-08-20 hour 15: 249753 events in 0 files, snapshot none (task 11368 ms: download 7635 ms overlapping gunzip 1780 ms, parse 7523 ms, sort 0 ms, write 0 ms, index 1280 ms: 11 candidates, 106158 repo-hours, 1485 releases; waiting 458 ms; commit 0 ms)
2026-11-08T07:32:59Z [load_day 2024-08-20] 2024-08-20 hour 16: 230850 events in 0 files, snapshot none (task 9846 ms: download 5824 ms overlapping gunzip 1548 ms, parse 6367 ms, sort 0 ms, write 0 ms, index 1092 ms: 13 candidates, 94479 repo-hours, 1146 releases; waiting 480 ms; commit 0 ms)
2026-11-08T07:33:01Z [load_day 2024-08-20] 2024-08-20 hour 17: 219318 events in 0 files, snapshot none (task 8460 ms: download 5762 ms overlapping gunzip 1397 ms, parse 5608 ms, sort 0 ms, write 0 ms, index 681 ms: 15 candidates, 86707 repo-hours, 1130 releases; waiting 479 ms; commit 0 ms)
2026-11-08T07:33:09Z [load_day 2024-08-20] 2024-08-20 hour 18: 223861 events in 0 files, snapshot none (task 8683 ms: download 4958 ms overlapping gunzip 1340 ms, parse 5739 ms, sort 0 ms, write 0 ms, index 818 ms: 5 candidates, 100412 repo-hours, 1337 releases; waiting 520 ms; commit 0 ms)
2026-11-08T07:33:10Z [load_day 2024-08-20] 2024-08-20 hour 19: 202018 events in 0 files, snapshot none (task 8331 ms: download 5665 ms overlapping gunzip 1315 ms, parse 5594 ms, sort 0 ms, write 0 ms, index 560 ms: 8 candidates, 78569 repo-hours, 1055 releases; waiting 566 ms; commit 0 ms)
2026-11-08T07:33:17Z [load_day 2024-08-20] 2024-08-20 hour 20: 193053 events in 0 files, snapshot none (task 8195 ms: download 4787 ms overlapping gunzip 1307 ms, parse 5268 ms, sort 0 ms, write 0 ms, index 703 ms: 8 candidates, 77153 repo-hours, 1088 releases; waiting 595 ms; commit 0 ms)
2026-11-08T07:33:18Z [load_day 2024-08-20] 2024-08-20 hour 21: 187335 events in 0 files, snapshot none (task 7346 ms: download 4427 ms overlapping gunzip 1162 ms, parse 4625 ms, sort 0 ms, write 0 ms, index 677 ms: 9 candidates, 75992 repo-hours, 1245 releases; waiting 498 ms; commit 0 ms)
2026-11-08T07:33:25Z [load_day 2024-08-20] 2024-08-20 hour 22: 166817 events in 0 files, snapshot none (task 6924 ms: download 3862 ms overlapping gunzip 1028 ms, parse 4605 ms, sort 0 ms, write 0 ms, index 448 ms: 8 candidates, 62900 repo-hours, 1082 releases; waiting 478 ms; commit 0 ms)
2026-11-08T07:33:25Z [load_day 2024-08-20] 2024-08-20 hour 23: 160545 events in 0 files, snapshot none (task 6621 ms: download 3730 ms overlapping gunzip 997 ms, parse 4170 ms, sort 0 ms, write 0 ms, index 697 ms: 4 candidates, 60484 repo-hours, 789 releases; waiting 498 ms; commit 0 ms)
2026-11-08T07:33:34Z [load_day 2024-08-20] 2024-08-20: repo_hour merged from 24 files to 1 (2097040 rows, 8622 ms)
2026-11-08T07:33:36Z [load_day 2024-08-20] 2024-08-20: index committed in 11032 ms (24 + 1 + 24 files)
2026-11-08T07:33:36Z [load_day 2024-08-20] 2024-08-20: 4943699 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T07:33:36Z [load_day 2024-08-20] attempt 1 succeeded