orcaLog in

Runs · run 3223

load_day 2024-11-20, attempt 1 succeeded

Host
73ccef5e07c2:209976
Started
2026-11-08T08:56:13Z
Ended
2026-11-08T08:58:40Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
827 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
237.1 s, up to 2.9 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3223/load_day/2024-11-20/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T08:56:13Z [load_day 2024-11-20] attempt 1 started
2026-11-08T08:56:14Z [load_day 2024-11-20] 2024-11-20: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:56:24Z [load_day 2024-11-20] 2024-11-20 hour 0: 199263 events in 0 files, snapshot none (task 8715 ms: download 5521 ms overlapping gunzip 1341 ms, parse 5676 ms, sort 0 ms, write 0 ms, index 1020 ms: 8 candidates, 92277 repo-hours, 1217 releases; waiting 346 ms; commit 0 ms)
2026-11-08T08:56:24Z [load_day 2024-11-20] 2024-11-20 hour 1: 216785 events in 0 files, snapshot none (task 7830 ms: download 4688 ms overlapping gunzip 1218 ms, parse 5200 ms, sort 0 ms, write 0 ms, index 885 ms: 8 candidates, 114643 repo-hours, 880 releases; waiting 176 ms; commit 0 ms)
2026-11-08T08:56:32Z [load_day 2024-11-20] 2024-11-20 hour 2: 193177 events in 0 files, snapshot none (task 8155 ms: download 5242 ms overlapping gunzip 1247 ms, parse 5394 ms, sort 0 ms, write 0 ms, index 635 ms: 8 candidates, 84225 repo-hours, 876 releases; waiting 588 ms; commit 0 ms)
2026-11-08T08:56:34Z [load_day 2024-11-20] 2024-11-20 hour 3: 217187 events in 0 files, snapshot none (task 9529 ms: download 7002 ms overlapping gunzip 1530 ms, parse 6537 ms, sort 0 ms, write 0 ms, index 635 ms: 4 candidates, 82070 repo-hours, 3888 releases; waiting 487 ms; commit 0 ms)
2026-11-08T08:56:42Z [load_day 2024-11-20] 2024-11-20 hour 4: 196970 events in 0 files, snapshot none (task 9245 ms: download 5858 ms overlapping gunzip 1509 ms, parse 6092 ms, sort 0 ms, write 0 ms, index 885 ms: 6 candidates, 68483 repo-hours, 644 releases; waiting 463 ms; commit 0 ms)
2026-11-08T08:56:45Z [load_day 2024-11-20] 2024-11-20 hour 5: 207493 events in 0 files, snapshot none (task 10836 ms: download 8447 ms overlapping gunzip 1476 ms, parse 6183 ms, sort 0 ms, write 0 ms, index 549 ms: 5 candidates, 71921 repo-hours, 832 releases; waiting 2274 ms; commit 0 ms)
2026-11-08T08:56:53Z [load_day 2024-11-20] 2024-11-20 hour 6: 222697 events in 0 files, snapshot none (task 10078 ms: download 7177 ms overlapping gunzip 1591 ms, parse 6223 ms, sort 0 ms, write 0 ms, index 688 ms: 6 candidates, 91737 repo-hours, 938 releases; waiting 1251 ms; commit 0 ms)
2026-11-08T08:56:55Z [load_day 2024-11-20] 2024-11-20 hour 7: 222578 events in 0 files, snapshot none (task 9593 ms: download 7221 ms overlapping gunzip 1687 ms, parse 6435 ms, sort 0 ms, write 0 ms, index 611 ms: 8 candidates, 82543 repo-hours, 792 releases; waiting 593 ms; commit 0 ms)
2026-11-08T08:57:02Z [load_day 2024-11-20] 2024-11-20 hour 8: 239197 events in 0 files, snapshot none (task 8923 ms: download 6346 ms overlapping gunzip 1538 ms, parse 6008 ms, sort 0 ms, write 0 ms, index 759 ms: 6 candidates, 91979 repo-hours, 1125 releases; waiting 323 ms; commit 0 ms)
2026-11-08T08:57:05Z [load_day 2024-11-20] 2024-11-20 hour 9: 252177 events in 0 files, snapshot none (task 8943 ms: download 6504 ms overlapping gunzip 1492 ms, parse 5900 ms, sort 0 ms, write 0 ms, index 755 ms: 5 candidates, 101833 repo-hours, 1220 releases; waiting 531 ms; commit 0 ms)
2026-11-08T08:57:13Z [load_day 2024-11-20] 2024-11-20 hour 10: 244996 events in 0 files, snapshot none (task 10178 ms: download 6148 ms overlapping gunzip 1477 ms, parse 5835 ms, sort 0 ms, write 0 ms, index 2291 ms: 3 candidates, 94889 repo-hours, 1180 releases; waiting 263 ms; commit 0 ms)
2026-11-08T08:57:16Z [load_day 2024-11-20] 2024-11-20 hour 11: 245963 events in 0 files, snapshot none (task 10336 ms: download 7018 ms overlapping gunzip 1789 ms, parse 6815 ms, sort 0 ms, write 0 ms, index 807 ms: 4 candidates, 91077 repo-hours, 1128 releases; waiting 587 ms; commit 0 ms)
2026-11-08T08:57:25Z [load_day 2024-11-20] 2024-11-20 hour 12: 272369 events in 0 files, snapshot none (task 11148 ms: download 6437 ms overlapping gunzip 1663 ms, parse 7118 ms, sort 0 ms, write 0 ms, index 1720 ms: 14 candidates, 136749 repo-hours, 1614 releases; waiting 328 ms; commit 0 ms)
2026-11-08T08:57:27Z [load_day 2024-11-20] 2024-11-20 hour 13: 270529 events in 0 files, snapshot none (task 10791 ms: download 8154 ms overlapping gunzip 1788 ms, parse 7502 ms, sort 0 ms, write 0 ms, index 892 ms: 8 candidates, 116628 repo-hours, 1326 releases; waiting 297 ms; commit 0 ms)
2026-11-08T08:57:38Z [load_day 2024-11-20] 2024-11-20 hour 14: 259175 events in 0 files, snapshot none (task 12524 ms: download 9907 ms overlapping gunzip 2123 ms, parse 8667 ms, sort 0 ms, write 0 ms, index 744 ms: 8 candidates, 103673 repo-hours, 1230 releases; waiting 691 ms; commit 0 ms)
2026-11-08T08:57:40Z [load_day 2024-11-20] 2024-11-20 hour 15: 266191 events in 0 files, snapshot none (task 11967 ms: download 9404 ms overlapping gunzip 2139 ms, parse 8164 ms, sort 0 ms, write 0 ms, index 849 ms: 10 candidates, 112563 repo-hours, 1403 releases; waiting 522 ms; commit 0 ms)
2026-11-08T08:57:50Z [load_day 2024-11-20] 2024-11-20 hour 16: 258713 events in 0 files, snapshot none (task 11529 ms: download 8681 ms overlapping gunzip 1898 ms, parse 7691 ms, sort 0 ms, write 0 ms, index 855 ms: 13 candidates, 104623 repo-hours, 1206 releases; waiting 568 ms; commit 0 ms)
2026-11-08T08:57:52Z [load_day 2024-11-20] 2024-11-20 hour 17: 249986 events in 0 files, snapshot none (task 11641 ms: download 8683 ms overlapping gunzip 1834 ms, parse 7819 ms, sort 0 ms, write 0 ms, index 1062 ms: 10 candidates, 96764 repo-hours, 960 releases; waiting 589 ms; commit 0 ms)
2026-11-08T08:58:02Z [load_day 2024-11-20] 2024-11-20 hour 18: 255411 events in 0 files, snapshot none (task 11148 ms: download 8187 ms overlapping gunzip 1764 ms, parse 7416 ms, sort 0 ms, write 0 ms, index 1102 ms: 13 candidates, 106506 repo-hours, 1125 releases; waiting 547 ms; commit 0 ms)
2026-11-08T08:58:05Z [load_day 2024-11-20] 2024-11-20 hour 19: 235277 events in 0 files, snapshot none (task 12476 ms: download 9499 ms overlapping gunzip 1747 ms, parse 7013 ms, sort 0 ms, write 0 ms, index 1056 ms: 6 candidates, 86016 repo-hours, 1030 releases; waiting 2352 ms; commit 0 ms)
2026-11-08T08:58:16Z [load_day 2024-11-20] 2024-11-20 hour 20: 234952 events in 0 files, snapshot none (task 13147 ms: download 9041 ms overlapping gunzip 2206 ms, parse 8790 ms, sort 0 ms, write 0 ms, index 1288 ms: 4 candidates, 86678 repo-hours, 1022 releases; waiting 535 ms; commit 0 ms)
2026-11-08T08:58:17Z [load_day 2024-11-20] 2024-11-20 hour 21: 236889 events in 0 files, snapshot none (task 11344 ms: download 8105 ms overlapping gunzip 1989 ms, parse 7815 ms, sort 0 ms, write 0 ms, index 808 ms: 11 candidates, 86905 repo-hours, 1105 releases; waiting 480 ms; commit 0 ms)
2026-11-08T08:58:27Z [load_day 2024-11-20] 2024-11-20 hour 22: 220175 events in 0 files, snapshot none (task 10093 ms: download 7421 ms overlapping gunzip 1785 ms, parse 6863 ms, sort 0 ms, write 0 ms, index 618 ms: 9 candidates, 74091 repo-hours, 1166 releases; waiting 516 ms; commit 0 ms)
2026-11-08T08:58:27Z [load_day 2024-11-20] 2024-11-20 hour 23: 213355 events in 0 files, snapshot none (task 9000 ms: download 6364 ms overlapping gunzip 1483 ms, parse 5985 ms, sort 0 ms, write 0 ms, index 616 ms: 3 candidates, 73010 repo-hours, 851 releases; waiting 505 ms; commit 0 ms)
2026-11-08T08:58:37Z [load_day 2024-11-20] 2024-11-20: repo_hour merged from 24 files to 1 (2251883 rows, 10011 ms)
2026-11-08T08:58:40Z [load_day 2024-11-20] 2024-11-20: index committed in 12742 ms (24 + 1 + 24 files)
2026-11-08T08:58:40Z [load_day 2024-11-20] 2024-11-20: 5631505 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:58:40Z [load_day 2024-11-20] attempt 1 succeeded