load_day 2024-01-30, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:77256

Started

2026-11-08T04:24:01Z

Ended

2026-11-08T04:26:15Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

701 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

188.8 s, up to 2.9 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2024/load_day/2024-01-30/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:24:01Z [load_day 2024-01-30] attempt 1 started
2026-11-08T04:24:02Z [load_day 2024-01-30] 2024-01-30: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:24:13Z [load_day 2024-01-30] 2024-01-30 hour 0: 241125 events in 0 files, snapshot none (task 10835 ms: download 6948 ms overlapping gunzip 1699 ms, parse 7467 ms, sort 0 ms, write 0 ms, index 832 ms: 5 candidates, 94909 repo-hours, 1180 releases; waiting 508 ms; commit 0 ms)
2026-11-08T04:24:13Z [load_day 2024-01-30] 2024-01-30 hour 1: 235450 events in 0 files, snapshot none (task 9607 ms: download 6277 ms overlapping gunzip 1449 ms, parse 6429 ms, sort 0 ms, write 0 ms, index 913 ms: 6 candidates, 97390 repo-hours, 847 releases; waiting 484 ms; commit 0 ms)
2026-11-08T04:24:20Z [load_day 2024-01-30] 2024-01-30 hour 2: 211058 events in 0 files, snapshot none (task 6627 ms: download 3514 ms overlapping gunzip 1019 ms, parse 4073 ms, sort 0 ms, write 0 ms, index 706 ms: 12 candidates, 66863 repo-hours, 869 releases; waiting 540 ms; commit 0 ms)
2026-11-08T04:24:20Z [load_day 2024-01-30] 2024-01-30 hour 3: 202467 events in 0 files, snapshot none (task 5758 ms: download 2907 ms overlapping gunzip 817 ms, parse 3537 ms, sort 0 ms, write 0 ms, index 641 ms: 6 candidates, 63320 repo-hours, 796 releases; waiting 532 ms; commit 0 ms)
2026-11-08T04:24:27Z [load_day 2024-01-30] 2024-01-30 hour 4: 195357 events in 0 files, snapshot none (task 6611 ms: download 3919 ms overlapping gunzip 1010 ms, parse 4293 ms, sort 0 ms, write 0 ms, index 574 ms: 6 candidates, 58537 repo-hours, 645 releases; waiting 442 ms; commit 0 ms)
2026-11-08T04:24:27Z [load_day 2024-01-30] 2024-01-30 hour 5: 204138 events in 0 files, snapshot none (task 6708 ms: download 4140 ms overlapping gunzip 1038 ms, parse 4350 ms, sort 0 ms, write 0 ms, index 541 ms: 7 candidates, 64546 repo-hours, 656 releases; waiting 525 ms; commit 0 ms)
2026-11-08T04:24:37Z [load_day 2024-01-30] 2024-01-30 hour 6: 223722 events in 0 files, snapshot none (task 9486 ms: download 6127 ms overlapping gunzip 1480 ms, parse 6476 ms, sort 0 ms, write 0 ms, index 685 ms: 6 candidates, 77510 repo-hours, 818 releases; waiting 556 ms; commit 0 ms)
2026-11-08T04:24:38Z [load_day 2024-01-30] 2024-01-30 hour 7: 212563 events in 0 files, snapshot none (task 9541 ms: download 6396 ms overlapping gunzip 1546 ms, parse 6453 ms, sort 0 ms, write 0 ms, index 768 ms: 3 candidates, 75055 repo-hours, 829 releases; waiting 485 ms; commit 0 ms)
2026-11-08T04:24:46Z [load_day 2024-01-30] 2024-01-30 hour 8: 230265 events in 0 files, snapshot none (task 8232 ms: download 5238 ms overlapping gunzip 1258 ms, parse 5375 ms, sort 0 ms, write 0 ms, index 703 ms: 8 candidates, 83695 repo-hours, 1009 releases; waiting 574 ms; commit 0 ms)
2026-11-08T04:24:47Z [load_day 2024-01-30] 2024-01-30 hour 9: 259560 events in 0 files, snapshot none (task 9342 ms: download 6469 ms overlapping gunzip 1524 ms, parse 6321 ms, sort 0 ms, write 0 ms, index 649 ms: 9 candidates, 89027 repo-hours, 1282 releases; waiting 508 ms; commit 0 ms)
2026-11-08T04:24:55Z [load_day 2024-01-30] 2024-01-30 hour 10: 261522 events in 0 files, snapshot none (task 7934 ms: download 5392 ms overlapping gunzip 1227 ms, parse 5250 ms, sort 0 ms, write 0 ms, index 629 ms: 8 candidates, 87363 repo-hours, 1083 releases; waiting 565 ms; commit 0 ms)
2026-11-08T04:24:56Z [load_day 2024-01-30] 2024-01-30 hour 11: 256284 events in 0 files, snapshot none (task 7652 ms: download 4926 ms overlapping gunzip 1149 ms, parse 4974 ms, sort 0 ms, write 0 ms, index 678 ms: 3 candidates, 81814 repo-hours, 1005 releases; waiting 512 ms; commit 0 ms)
2026-11-08T04:25:07Z [load_day 2024-01-30] 2024-01-30 hour 12: 311038 events in 0 files, snapshot none (task 11285 ms: download 7973 ms overlapping gunzip 1747 ms, parse 7773 ms, sort 0 ms, write 0 ms, index 971 ms: 8 candidates, 112952 repo-hours, 1192 releases; waiting 526 ms; commit 0 ms)
2026-11-08T04:25:07Z [load_day 2024-01-30] 2024-01-30 hour 13: 283056 events in 0 files, snapshot none (task 10977 ms: download 7942 ms overlapping gunzip 1717 ms, parse 7501 ms, sort 0 ms, write 0 ms, index 975 ms: 10 candidates, 109044 repo-hours, 1127 releases; waiting 481 ms; commit 0 ms)
2026-11-08T04:25:19Z [load_day 2024-01-30] 2024-01-30 hour 14: 273813 events in 0 files, snapshot none (task 11836 ms: download 8591 ms overlapping gunzip 1880 ms, parse 8347 ms, sort 0 ms, write 0 ms, index 768 ms: 6 candidates, 100654 repo-hours, 1220 releases; waiting 513 ms; commit 0 ms)
2026-11-08T04:25:19Z [load_day 2024-01-30] 2024-01-30 hour 15: 244635 events in 0 files, snapshot none (task 10284 ms: download 6412 ms overlapping gunzip 1637 ms, parse 6787 ms, sort 0 ms, write 0 ms, index 866 ms: 6 candidates, 96710 repo-hours, 1244 releases; waiting 524 ms; commit 0 ms)
2026-11-08T04:25:31Z [load_day 2024-01-30] 2024-01-30 hour 16: 268120 events in 0 files, snapshot none (task 10717 ms: download 7883 ms overlapping gunzip 1746 ms, parse 7344 ms, sort 0 ms, write 0 ms, index 854 ms: 5 candidates, 99695 repo-hours, 1157 releases; waiting 543 ms; commit 0 ms)
2026-11-08T04:25:31Z [load_day 2024-01-30] 2024-01-30 hour 17: 270429 events in 0 files, snapshot none (task 10447 ms: download 7631 ms overlapping gunzip 1693 ms, parse 7392 ms, sort 0 ms, write 0 ms, index 723 ms: 8 candidates, 91823 repo-hours, 1009 releases; waiting 358 ms; commit 0 ms)
2026-11-08T04:25:42Z [load_day 2024-01-30] 2024-01-30 hour 18: 276623 events in 0 files, snapshot none (task 11355 ms: download 7984 ms overlapping gunzip 1558 ms, parse 7806 ms, sort 0 ms, write 0 ms, index 1143 ms: 5 candidates, 94210 repo-hours, 1171 releases; waiting 543 ms; commit 0 ms)
2026-11-08T04:25:42Z [load_day 2024-01-30] 2024-01-30 hour 19: 256085 events in 0 files, snapshot none (task 10255 ms: download 7327 ms overlapping gunzip 1632 ms, parse 6915 ms, sort 0 ms, write 0 ms, index 831 ms: 5 candidates, 83217 repo-hours, 955 releases; waiting 517 ms; commit 0 ms)
2026-11-08T04:25:53Z [load_day 2024-01-30] 2024-01-30 hour 20: 253422 events in 0 files, snapshot none (task 9921 ms: download 7191 ms overlapping gunzip 1550 ms, parse 6788 ms, sort 0 ms, write 0 ms, index 788 ms: 5 candidates, 80645 repo-hours, 1098 releases; waiting 541 ms; commit 0 ms)
2026-11-08T04:25:53Z [load_day 2024-01-30] 2024-01-30 hour 21: 235144 events in 0 files, snapshot none (task 9565 ms: download 6968 ms overlapping gunzip 1558 ms, parse 6597 ms, sort 0 ms, write 0 ms, index 609 ms: 4 candidates, 75305 repo-hours, 1196 releases; waiting 501 ms; commit 0 ms)
2026-11-08T04:26:03Z [load_day 2024-01-30] 2024-01-30 hour 22: 224490 events in 0 files, snapshot none (task 9124 ms: download 6450 ms overlapping gunzip 1572 ms, parse 6223 ms, sort 0 ms, write 0 ms, index 536 ms: 5 candidates, 68946 repo-hours, 1105 releases; waiting 479 ms; commit 0 ms)
2026-11-08T04:26:03Z [load_day 2024-01-30] 2024-01-30 hour 23: 212930 events in 0 files, snapshot none (task 7870 ms: download 5093 ms overlapping gunzip 1153 ms, parse 5316 ms, sort 0 ms, write 0 ms, index 574 ms: 4 candidates, 62444 repo-hours, 845 releases; waiting 511 ms; commit 0 ms)
2026-11-08T04:26:12Z [load_day 2024-01-30] 2024-01-30: repo_hour merged from 24 files to 1 (2015674 rows, 9825 ms)
2026-11-08T04:26:15Z [load_day 2024-01-30] 2024-01-30: index committed in 12611 ms (24 + 1 + 24 files)
2026-11-08T04:26:15Z [load_day 2024-01-30] 2024-01-30: 5843296 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:26:15Z [load_day 2024-01-30] attempt 1 succeeded