load_day 2024-01-20, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:50286

Started

2026-11-08T04:14:47Z

Ended

2026-11-08T04:16:17Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

600 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

112.5 s, up to 2.8 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1988/load_day/2024-01-20/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:14:47Z [load_day 2024-01-20] attempt 1 started
2026-11-08T04:14:48Z [load_day 2024-01-20] 2024-01-20: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:14:56Z [load_day 2024-01-20] 2024-01-20 hour 0: 179977 events in 0 files, snapshot none (task 7322 ms: download 4098 ms overlapping gunzip 1069 ms, parse 4593 ms, sort 0 ms, write 0 ms, index 861 ms: 7 candidates, 78516 repo-hours, 991 releases; waiting 469 ms; commit 0 ms)
2026-11-08T04:14:56Z [load_day 2024-01-20] 2024-01-20 hour 1: 175630 events in 0 files, snapshot none (task 7049 ms: download 3769 ms overlapping gunzip 1076 ms, parse 4688 ms, sort 0 ms, write 0 ms, index 792 ms: 7 candidates, 81464 repo-hours, 777 releases; waiting 211 ms; commit 0 ms)
2026-11-08T04:15:03Z [load_day 2024-01-20] 2024-01-20 hour 2: 143210 events in 0 files, snapshot none (task 6487 ms: download 3275 ms overlapping gunzip 1039 ms, parse 4126 ms, sort 0 ms, write 0 ms, index 514 ms: 24 candidates, 46885 repo-hours, 591 releases; waiting 501 ms; commit 0 ms)
2026-11-08T04:15:03Z [load_day 2024-01-20] 2024-01-20 hour 3: 142195 events in 0 files, snapshot none (task 5884 ms: download 2894 ms overlapping gunzip 890 ms, parse 3590 ms, sort 0 ms, write 0 ms, index 585 ms: 13 candidates, 44658 repo-hours, 745 releases; waiting 466 ms; commit 0 ms)
2026-11-08T04:15:09Z [load_day 2024-01-20] 2024-01-20 hour 4: 146385 events in 0 files, snapshot none (task 5575 ms: download 2945 ms overlapping gunzip 854 ms, parse 3619 ms, sort 0 ms, write 0 ms, index 329 ms: 3 candidates, 46284 repo-hours, 483 releases; waiting 488 ms; commit 0 ms)
2026-11-08T04:15:10Z [load_day 2024-01-20] 2024-01-20 hour 5: 157028 events in 0 files, snapshot none (task 6193 ms: download 3146 ms overlapping gunzip 996 ms, parse 3932 ms, sort 0 ms, write 0 ms, index 473 ms: 13 candidates, 51128 repo-hours, 524 releases; waiting 156 ms; commit 0 ms)
2026-11-08T04:15:14Z [load_day 2024-01-20] 2024-01-20 hour 6: 157911 events in 0 files, snapshot none (task 4613 ms: download 2377 ms overlapping gunzip 543 ms, parse 2584 ms, sort 0 ms, write 0 ms, index 498 ms: 9 candidates, 56860 repo-hours, 904 releases; waiting 691 ms; commit 0 ms)
2026-11-08T04:15:15Z [load_day 2024-01-20] 2024-01-20 hour 7: 154755 events in 0 files, snapshot none (task 4212 ms: download 2014 ms overlapping gunzip 562 ms, parse 2451 ms, sort 0 ms, write 0 ms, index 383 ms: 3 candidates, 50563 repo-hours, 773 releases; waiting 555 ms; commit 0 ms)
2026-11-08T04:15:19Z [load_day 2024-01-20] 2024-01-20 hour 8: 160281 events in 0 files, snapshot none (task 4542 ms: download 2738 ms overlapping gunzip 706 ms, parse 2797 ms, sort 0 ms, write 0 ms, index 448 ms: 9 candidates, 54787 repo-hours, 842 releases; waiting 304 ms; commit 0 ms)
2026-11-08T04:15:20Z [load_day 2024-01-20] 2024-01-20 hour 9: 163721 events in 0 files, snapshot none (task 4781 ms: download 2706 ms overlapping gunzip 671 ms, parse 2886 ms, sort 0 ms, write 0 ms, index 424 ms: 5 candidates, 56734 repo-hours, 808 releases; waiting 538 ms; commit 0 ms)
2026-11-08T04:15:25Z [load_day 2024-01-20] 2024-01-20 hour 10: 167319 events in 0 files, snapshot none (task 4955 ms: download 3711 ms overlapping gunzip 623 ms, parse 2753 ms, sort 0 ms, write 0 ms, index 447 ms: 6 candidates, 58596 repo-hours, 775 releases; waiting 750 ms; commit 0 ms)
2026-11-08T04:15:25Z [load_day 2024-01-20] 2024-01-20 hour 11: 168648 events in 0 files, snapshot none (task 4751 ms: download 2143 ms overlapping gunzip 630 ms, parse 2847 ms, sort 0 ms, write 0 ms, index 505 ms: 3 candidates, 57885 repo-hours, 764 releases; waiting 496 ms; commit 0 ms)
2026-11-08T04:15:32Z [load_day 2024-01-20] 2024-01-20 hour 12: 234305 events in 0 files, snapshot none (task 7092 ms: download 3539 ms overlapping gunzip 971 ms, parse 4567 ms, sort 0 ms, write 0 ms, index 741 ms: 20 candidates, 90585 repo-hours, 824 releases; waiting 483 ms; commit 0 ms)
2026-11-08T04:15:33Z [load_day 2024-01-20] 2024-01-20 hour 13: 194055 events in 0 files, snapshot none (task 6772 ms: download 3623 ms overlapping gunzip 978 ms, parse 4449 ms, sort 0 ms, write 0 ms, index 616 ms: 30 candidates, 81743 repo-hours, 747 releases; waiting 468 ms; commit 0 ms)
2026-11-08T04:15:39Z [load_day 2024-01-20] 2024-01-20 hour 14: 186249 events in 0 files, snapshot none (task 6579 ms: download 3636 ms overlapping gunzip 957 ms, parse 4108 ms, sort 0 ms, write 0 ms, index 755 ms: 4 candidates, 67014 repo-hours, 764 releases; waiting 494 ms; commit 0 ms)
2026-11-08T04:15:39Z [load_day 2024-01-20] 2024-01-20 hour 15: 188440 events in 0 files, snapshot none (task 6455 ms: download 3628 ms overlapping gunzip 945 ms, parse 4098 ms, sort 0 ms, write 0 ms, index 644 ms: 8 candidates, 69177 repo-hours, 959 releases; waiting 499 ms; commit 0 ms)
2026-11-08T04:15:47Z [load_day 2024-01-20] 2024-01-20 hour 16: 189143 events in 0 files, snapshot none (task 6892 ms: download 3433 ms overlapping gunzip 958 ms, parse 4389 ms, sort 0 ms, write 0 ms, index 771 ms: 6 candidates, 68472 repo-hours, 791 releases; waiting 518 ms; commit 0 ms)
2026-11-08T04:15:47Z [load_day 2024-01-20] 2024-01-20 hour 17: 186023 events in 0 files, snapshot none (task 6648 ms: download 2984 ms overlapping gunzip 916 ms, parse 3942 ms, sort 0 ms, write 0 ms, index 994 ms: 10 candidates, 68206 repo-hours, 804 releases; waiting 500 ms; commit 0 ms)
2026-11-08T04:15:54Z [load_day 2024-01-20] 2024-01-20 hour 18: 189562 events in 0 files, snapshot none (task 6351 ms: download 2822 ms overlapping gunzip 927 ms, parse 3991 ms, sort 0 ms, write 0 ms, index 654 ms: 8 candidates, 74382 repo-hours, 801 releases; waiting 509 ms; commit 0 ms)
2026-11-08T04:15:54Z [load_day 2024-01-20] 2024-01-20 hour 19: 171721 events in 0 files, snapshot none (task 5997 ms: download 2715 ms overlapping gunzip 785 ms, parse 3739 ms, sort 0 ms, write 0 ms, index 696 ms: 4 candidates, 60170 repo-hours, 805 releases; waiting 484 ms; commit 0 ms)
2026-11-08T04:16:00Z [load_day 2024-01-20] 2024-01-20 hour 20: 169135 events in 0 files, snapshot none (task 6219 ms: download 3459 ms overlapping gunzip 867 ms, parse 4107 ms, sort 0 ms, write 0 ms, index 442 ms: 5 candidates, 61098 repo-hours, 659 releases; waiting 504 ms; commit 0 ms)
2026-11-08T04:16:00Z [load_day 2024-01-20] 2024-01-20 hour 21: 158568 events in 0 files, snapshot none (task 6065 ms: download 3493 ms overlapping gunzip 854 ms, parse 3886 ms, sort 0 ms, write 0 ms, index 514 ms: 7 candidates, 53186 repo-hours, 731 releases; waiting 473 ms; commit 0 ms)
2026-11-08T04:16:06Z [load_day 2024-01-20] 2024-01-20 hour 22: 146828 events in 0 files, snapshot none (task 5550 ms: download 3236 ms overlapping gunzip 764 ms, parse 3584 ms, sort 0 ms, write 0 ms, index 405 ms: 2 candidates, 48434 repo-hours, 675 releases; waiting 517 ms; commit 0 ms)
2026-11-08T04:16:06Z [load_day 2024-01-20] 2024-01-20 hour 23: 139752 events in 0 files, snapshot none (task 5228 ms: download 2767 ms overlapping gunzip 742 ms, parse 3316 ms, sort 0 ms, write 0 ms, index 388 ms: 5 candidates, 45001 repo-hours, 556 releases; waiting 497 ms; commit 0 ms)
2026-11-08T04:16:14Z [load_day 2024-01-20] 2024-01-20: repo_hour merged from 24 files to 1 (1471828 rows, 7477 ms)
2026-11-08T04:16:17Z [load_day 2024-01-20] 2024-01-20: index committed in 10480 ms (24 + 1 + 24 files)
2026-11-08T04:16:17Z [load_day 2024-01-20] 2024-01-20: 4070841 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:16:17Z [load_day 2024-01-20] attempt 1 succeeded