load_day 2024-03-16, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:98486

Started

2026-11-08T05:08:38Z

Ended

2026-11-08T05:10:01Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

633 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

121.2 s, up to 2.8 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2204/load_day/2024-03-16/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T05:08:38Z [load_day 2024-03-16] attempt 1 started
2026-11-08T05:08:39Z [load_day 2024-03-16] 2024-03-16: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T05:08:46Z [load_day 2024-03-16] 2024-03-16 hour 0: 175858 events in 0 files, snapshot none (task 6544 ms: download 3497 ms overlapping gunzip 1053 ms, parse 4297 ms, sort 0 ms, write 0 ms, index 672 ms: 5 candidates, 88666 repo-hours, 1081 releases; waiting 179 ms; commit 0 ms)
2026-11-08T05:08:46Z [load_day 2024-03-16] 2024-03-16 hour 1: 163192 events in 0 files, snapshot none (task 6310 ms: download 3602 ms overlapping gunzip 892 ms, parse 3983 ms, sort 0 ms, write 0 ms, index 661 ms: 9 candidates, 84217 repo-hours, 775 releases; waiting 496 ms; commit 0 ms)
2026-11-08T05:08:52Z [load_day 2024-03-16] 2024-03-16 hour 2: 129417 events in 0 files, snapshot none (task 5270 ms: download 2950 ms overlapping gunzip 738 ms, parse 3161 ms, sort 0 ms, write 0 ms, index 493 ms: 7 candidates, 48711 repo-hours, 663 releases; waiting 515 ms; commit 0 ms)
2026-11-08T05:08:52Z [load_day 2024-03-16] 2024-03-16 hour 3: 126569 events in 0 files, snapshot none (task 5124 ms: download 2157 ms overlapping gunzip 729 ms, parse 3085 ms, sort 0 ms, write 0 ms, index 569 ms: 1 candidates, 47988 repo-hours, 719 releases; waiting 463 ms; commit 0 ms)
2026-11-08T05:08:56Z [load_day 2024-03-16] 2024-03-16 hour 4: 125621 events in 0 files, snapshot none (task 3678 ms: download 2148 ms overlapping gunzip 486 ms, parse 2028 ms, sort 0 ms, write 0 ms, index 344 ms: 3 candidates, 47862 repo-hours, 537 releases; waiting 513 ms; commit 0 ms)
2026-11-08T05:08:57Z [load_day 2024-03-16] 2024-03-16 hour 5: 142120 events in 0 files, snapshot none (task 4900 ms: download 3096 ms overlapping gunzip 760 ms, parse 2876 ms, sort 0 ms, write 0 ms, index 400 ms: 6 candidates, 50329 repo-hours, 561 releases; waiting 593 ms; commit 0 ms)
2026-11-08T05:09:00Z [load_day 2024-03-16] 2024-03-16 hour 6: 145844 events in 0 files, snapshot none (task 3793 ms: download 2030 ms overlapping gunzip 485 ms, parse 2085 ms, sort 0 ms, write 0 ms, index 437 ms: 5 candidates, 59618 repo-hours, 883 releases; waiting 514 ms; commit 0 ms)
2026-11-08T05:09:01Z [load_day 2024-03-16] 2024-03-16 hour 7: 142769 events in 0 files, snapshot none (task 3851 ms: download 2242 ms overlapping gunzip 499 ms, parse 2037 ms, sort 0 ms, write 0 ms, index 415 ms: 13 candidates, 54960 repo-hours, 762 releases; waiting 550 ms; commit 0 ms)
2026-11-08T05:09:04Z [load_day 2024-03-16] 2024-03-16 hour 8: 148555 events in 0 files, snapshot none (task 4066 ms: download 2296 ms overlapping gunzip 572 ms, parse 2341 ms, sort 0 ms, write 0 ms, index 439 ms: 14 candidates, 58110 repo-hours, 792 releases; waiting 374 ms; commit 0 ms)
2026-11-08T05:09:06Z [load_day 2024-03-16] 2024-03-16 hour 9: 157179 events in 0 files, snapshot none (task 4258 ms: download 2340 ms overlapping gunzip 616 ms, parse 2426 ms, sort 0 ms, write 0 ms, index 458 ms: 3 candidates, 59922 repo-hours, 995 releases; waiting 499 ms; commit 0 ms)
2026-11-08T05:09:10Z [load_day 2024-03-16] 2024-03-16 hour 10: 155139 events in 0 files, snapshot none (task 5587 ms: download 2862 ms overlapping gunzip 891 ms, parse 3407 ms, sort 0 ms, write 0 ms, index 590 ms: 10 candidates, 61662 repo-hours, 684 releases; waiting 366 ms; commit 0 ms)
2026-11-08T05:09:11Z [load_day 2024-03-16] 2024-03-16 hour 11: 157817 events in 0 files, snapshot none (task 5241 ms: download 2747 ms overlapping gunzip 758 ms, parse 3247 ms, sort 0 ms, write 0 ms, index 485 ms: 10 candidates, 60482 repo-hours, 606 releases; waiting 481 ms; commit 0 ms)
2026-11-08T05:09:17Z [load_day 2024-03-16] 2024-03-16 hour 12: 241926 events in 0 files, snapshot none (task 6528 ms: download 3465 ms overlapping gunzip 871 ms, parse 4098 ms, sort 0 ms, write 0 ms, index 714 ms: 5 candidates, 95569 repo-hours, 1499 releases; waiting 498 ms; commit 0 ms)
2026-11-08T05:09:18Z [load_day 2024-03-16] 2024-03-16 hour 13: 204945 events in 0 files, snapshot none (task 5704 ms: download 3257 ms overlapping gunzip 795 ms, parse 3524 ms, sort 0 ms, write 0 ms, index 659 ms: 7 candidates, 103003 repo-hours, 805 releases; waiting 501 ms; commit 0 ms)
2026-11-08T05:09:25Z [load_day 2024-03-16] 2024-03-16 hour 14: 181166 events in 0 files, snapshot none (task 6675 ms: download 3957 ms overlapping gunzip 969 ms, parse 4033 ms, sort 0 ms, write 0 ms, index 852 ms: 9 candidates, 72878 repo-hours, 914 releases; waiting 536 ms; commit 0 ms)
2026-11-08T05:09:25Z [load_day 2024-03-16] 2024-03-16 hour 15: 185386 events in 0 files, snapshot none (task 6904 ms: download 4001 ms overlapping gunzip 1051 ms, parse 4425 ms, sort 0 ms, write 0 ms, index 698 ms: 7 candidates, 73423 repo-hours, 907 releases; waiting 432 ms; commit 0 ms)
2026-11-08T05:09:32Z [load_day 2024-03-16] 2024-03-16 hour 16: 179082 events in 0 files, snapshot none (task 6685 ms: download 4093 ms overlapping gunzip 910 ms, parse 3663 ms, sort 0 ms, write 0 ms, index 631 ms: 10 candidates, 70653 repo-hours, 1051 releases; waiting 1145 ms; commit 0 ms)
2026-11-08T05:09:32Z [load_day 2024-03-16] 2024-03-16 hour 17: 171816 events in 0 files, snapshot none (task 6135 ms: download 3102 ms overlapping gunzip 939 ms, parse 3880 ms, sort 0 ms, write 0 ms, index 523 ms: 6 candidates, 67243 repo-hours, 737 releases; waiting 511 ms; commit 0 ms)
2026-11-08T05:09:38Z [load_day 2024-03-16] 2024-03-16 hour 18: 190751 events in 0 files, snapshot none (task 5745 ms: download 3361 ms overlapping gunzip 951 ms, parse 3787 ms, sort 0 ms, write 0 ms, index 556 ms: 3 candidates, 81329 repo-hours, 1717 releases; waiting 174 ms; commit 0 ms)
2026-11-08T05:09:38Z [load_day 2024-03-16] 2024-03-16 hour 19: 166746 events in 0 files, snapshot none (task 5996 ms: download 3715 ms overlapping gunzip 1035 ms, parse 3729 ms, sort 0 ms, write 0 ms, index 430 ms: 8 candidates, 64244 repo-hours, 748 releases; waiting 515 ms; commit 0 ms)
2026-11-08T05:09:44Z [load_day 2024-03-16] 2024-03-16 hour 20: 161091 events in 0 files, snapshot none (task 5761 ms: download 3634 ms overlapping gunzip 897 ms, parse 3714 ms, sort 0 ms, write 0 ms, index 433 ms: 8 candidates, 61959 repo-hours, 712 releases; waiting 479 ms; commit 0 ms)
2026-11-08T05:09:44Z [load_day 2024-03-16] 2024-03-16 hour 21: 149161 events in 0 files, snapshot none (task 5648 ms: download 3388 ms overlapping gunzip 878 ms, parse 3578 ms, sort 0 ms, write 0 ms, index 413 ms: 6 candidates, 54752 repo-hours, 770 releases; waiting 499 ms; commit 0 ms)
2026-11-08T05:09:51Z [load_day 2024-03-16] 2024-03-16 hour 22: 153218 events in 0 files, snapshot none (task 6339 ms: download 3755 ms overlapping gunzip 988 ms, parse 4135 ms, sort 0 ms, write 0 ms, index 457 ms: 11 candidates, 58518 repo-hours, 730 releases; waiting 525 ms; commit 0 ms)
2026-11-08T05:09:52Z [load_day 2024-03-16] 2024-03-16 hour 23: 155075 events in 0 files, snapshot none (task 6820 ms: download 4259 ms overlapping gunzip 1043 ms, parse 4438 ms, sort 0 ms, write 0 ms, index 447 ms: 4 candidates, 64446 repo-hours, 569 releases; waiting 526 ms; commit 0 ms)
2026-11-08T05:09:59Z [load_day 2024-03-16] 2024-03-16: repo_hour merged from 24 files to 1 (1590544 rows, 7139 ms)
2026-11-08T05:10:01Z [load_day 2024-03-16] 2024-03-16: index committed in 9429 ms (24 + 1 + 24 files)
2026-11-08T05:10:01Z [load_day 2024-03-16] 2024-03-16: 3910443 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T05:10:01Z [load_day 2024-03-16] attempt 1 succeeded