load_day 2024-01-01, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:14721

Started

2026-11-08T03:58:06Z

Ended

2026-11-08T03:59:29Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

555 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

130.4 s, up to 3.1 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1921/load_day/2024-01-01/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T03:58:06Z [load_day 2024-01-01] attempt 1 started
2026-11-08T03:58:07Z [load_day 2024-01-01] 2024-01-01: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T03:58:12Z [load_day 2024-01-01] 2024-01-01 hour 0: 153619 events in 0 files, snapshot none (task 4237 ms: download 2014 ms overlapping gunzip 766 ms, parse 2865 ms, sort 0 ms, write 0 ms, index 547 ms: 7 candidates, 62925 repo-hours, 858 releases; waiting 8 ms; commit 0 ms)
2026-11-08T03:58:13Z [load_day 2024-01-01] 2024-01-01 hour 1: 169672 events in 0 files, snapshot none (task 5197 ms: download 3094 ms overlapping gunzip 953 ms, parse 3648 ms, sort 0 ms, write 0 ms, index 532 ms: 4 candidates, 73731 repo-hours, 902 releases; waiting 4 ms; commit 0 ms)
2026-11-08T03:58:17Z [load_day 2024-01-01] 2024-01-01 hour 2: 130276 events in 0 files, snapshot none (task 4302 ms: download 1623 ms overlapping gunzip 763 ms, parse 2886 ms, sort 0 ms, write 0 ms, index 585 ms: 5 candidates, 38803 repo-hours, 664 releases; waiting 12 ms; commit 0 ms)
2026-11-08T03:58:18Z [load_day 2024-01-01] 2024-01-01 hour 3: 136620 events in 0 files, snapshot none (task 4782 ms: download 2575 ms overlapping gunzip 925 ms, parse 3263 ms, sort 0 ms, write 0 ms, index 528 ms: 4 candidates, 37165 repo-hours, 708 releases; waiting 13 ms; commit 0 ms)
2026-11-08T03:58:21Z [load_day 2024-01-01] 2024-01-01 hour 4: 135528 events in 0 files, snapshot none (task 3991 ms: download 1711 ms overlapping gunzip 721 ms, parse 2642 ms, sort 0 ms, write 0 ms, index 528 ms: 4 candidates, 37758 repo-hours, 490 releases; waiting 17 ms; commit 0 ms)
2026-11-08T03:58:23Z [load_day 2024-01-01] 2024-01-01 hour 5: 137293 events in 0 files, snapshot none (task 4034 ms: download 1679 ms overlapping gunzip 693 ms, parse 2673 ms, sort 0 ms, write 0 ms, index 595 ms: 1 candidates, 38904 repo-hours, 511 releases; waiting 10 ms; commit 0 ms)
2026-11-08T03:58:26Z [load_day 2024-01-01] 2024-01-01 hour 6: 147114 events in 0 files, snapshot none (task 3967 ms: download 1766 ms overlapping gunzip 683 ms, parse 2623 ms, sort 0 ms, write 0 ms, index 575 ms: 2 candidates, 46647 repo-hours, 640 releases; waiting 17 ms; commit 0 ms)
2026-11-08T03:58:27Z [load_day 2024-01-01] 2024-01-01 hour 7: 142101 events in 0 files, snapshot none (task 3686 ms: download 1595 ms overlapping gunzip 666 ms, parse 2609 ms, sort 0 ms, write 0 ms, index 337 ms: 0 candidates, 40793 repo-hours, 506 releases; waiting 9 ms; commit 0 ms)
2026-11-08T03:58:31Z [load_day 2024-01-01] 2024-01-01 hour 8: 147821 events in 0 files, snapshot none (task 4523 ms: download 2469 ms overlapping gunzip 666 ms, parse 2436 ms, sort 0 ms, write 0 ms, index 619 ms: 4 candidates, 44056 repo-hours, 594 releases; waiting 529 ms; commit 0 ms)
2026-11-08T03:58:33Z [load_day 2024-01-01] 2024-01-01 hour 9: 149533 events in 0 files, snapshot none (task 4924 ms: download 2732 ms overlapping gunzip 742 ms, parse 2957 ms, sort 0 ms, write 0 ms, index 357 ms: 1 candidates, 44748 repo-hours, 740 releases; waiting 576 ms; commit 0 ms)
2026-11-08T03:58:37Z [load_day 2024-01-01] 2024-01-01 hour 10: 158214 events in 0 files, snapshot none (task 5184 ms: download 2589 ms overlapping gunzip 785 ms, parse 2930 ms, sort 0 ms, write 0 ms, index 676 ms: 2 candidates, 48231 repo-hours, 510 releases; waiting 527 ms; commit 0 ms)
2026-11-08T03:58:38Z [load_day 2024-01-01] 2024-01-01 hour 11: 160724 events in 0 files, snapshot none (task 5407 ms: download 2666 ms overlapping gunzip 910 ms, parse 3310 ms, sort 0 ms, write 0 ms, index 378 ms: 5 candidates, 48224 repo-hours, 638 releases; waiting 213 ms; commit 0 ms)
2026-11-08T03:58:44Z [load_day 2024-01-01] 2024-01-01 hour 12: 225075 events in 0 files, snapshot none (task 6873 ms: download 4198 ms overlapping gunzip 1021 ms, parse 4532 ms, sort 0 ms, write 0 ms, index 605 ms: 5 candidates, 82750 repo-hours, 790 releases; waiting 521 ms; commit 0 ms)
2026-11-08T03:58:45Z [load_day 2024-01-01] 2024-01-01 hour 13: 190568 events in 0 files, snapshot none (task 6049 ms: download 3816 ms overlapping gunzip 965 ms, parse 3986 ms, sort 0 ms, write 0 ms, index 502 ms: 8 candidates, 73499 repo-hours, 646 releases; waiting 315 ms; commit 0 ms)
2026-11-08T03:58:50Z [load_day 2024-01-01] 2024-01-01 hour 14: 176699 events in 0 files, snapshot none (task 5161 ms: download 2959 ms overlapping gunzip 805 ms, parse 3112 ms, sort 0 ms, write 0 ms, index 419 ms: 5 candidates, 58798 repo-hours, 676 releases; waiting 527 ms; commit 0 ms)
2026-11-08T03:58:51Z [load_day 2024-01-01] 2024-01-01 hour 15: 180387 events in 0 files, snapshot none (task 5351 ms: download 3118 ms overlapping gunzip 839 ms, parse 3251 ms, sort 0 ms, write 0 ms, index 454 ms: 11 candidates, 60767 repo-hours, 885 releases; waiting 547 ms; commit 0 ms)
2026-11-08T03:58:56Z [load_day 2024-01-01] 2024-01-01 hour 16: 178135 events in 0 files, snapshot none (task 5883 ms: download 3659 ms overlapping gunzip 928 ms, parse 3525 ms, sort 0 ms, write 0 ms, index 591 ms: 9 candidates, 59460 repo-hours, 811 releases; waiting 536 ms; commit 0 ms)
2026-11-08T03:58:57Z [load_day 2024-01-01] 2024-01-01 hour 17: 178980 events in 0 files, snapshot none (task 5671 ms: download 3386 ms overlapping gunzip 914 ms, parse 3476 ms, sort 0 ms, write 0 ms, index 459 ms: 5 candidates, 57072 repo-hours, 2311 releases; waiting 520 ms; commit 0 ms)
2026-11-08T03:59:03Z [load_day 2024-01-01] 2024-01-01 hour 18: 185244 events in 0 files, snapshot none (task 5929 ms: download 3105 ms overlapping gunzip 986 ms, parse 3663 ms, sort 0 ms, write 0 ms, index 480 ms: 5 candidates, 64963 repo-hours, 873 releases; waiting 516 ms; commit 0 ms)
2026-11-08T03:59:03Z [load_day 2024-01-01] 2024-01-01 hour 19: 172448 events in 0 files, snapshot none (task 5920 ms: download 3663 ms overlapping gunzip 964 ms, parse 3828 ms, sort 0 ms, write 0 ms, index 378 ms: 6 candidates, 52744 repo-hours, 1536 releases; waiting 523 ms; commit 0 ms)
2026-11-08T03:59:10Z [load_day 2024-01-01] 2024-01-01 hour 20: 166962 events in 0 files, snapshot none (task 6293 ms: download 4112 ms overlapping gunzip 1090 ms, parse 4086 ms, sort 0 ms, write 0 ms, index 355 ms: 1 candidates, 52088 repo-hours, 627 releases; waiting 526 ms; commit 0 ms)
2026-11-08T03:59:10Z [load_day 2024-01-01] 2024-01-01 hour 21: 159181 events in 0 files, snapshot none (task 6072 ms: download 3911 ms overlapping gunzip 977 ms, parse 3959 ms, sort 0 ms, write 0 ms, index 339 ms: 5 candidates, 48191 repo-hours, 909 releases; waiting 569 ms; commit 0 ms)
2026-11-08T03:59:16Z [load_day 2024-01-01] 2024-01-01 hour 22: 151168 events in 0 files, snapshot none (task 5754 ms: download 3623 ms overlapping gunzip 982 ms, parse 3718 ms, sort 0 ms, write 0 ms, index 347 ms: 8 candidates, 43497 repo-hours, 836 releases; waiting 508 ms; commit 0 ms)
2026-11-08T03:59:17Z [load_day 2024-01-01] 2024-01-01 hour 23: 146481 events in 0 files, snapshot none (task 5838 ms: download 3613 ms overlapping gunzip 951 ms, parse 3743 ms, sort 0 ms, write 0 ms, index 313 ms: 2 candidates, 43037 repo-hours, 1451 releases; waiting 527 ms; commit 0 ms)
2026-11-08T03:59:23Z [load_day 2024-01-01] 2024-01-01: repo_hour merged from 24 files to 1 (1258851 rows, 6358 ms)
2026-11-08T03:59:29Z [load_day 2024-01-01] 2024-01-01: index committed in 12836 ms (23 + 1 + 24 files)
2026-11-08T03:59:29Z [load_day 2024-01-01] 2024-01-01: 3879843 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T03:59:29Z [load_day 2024-01-01] attempt 1 succeeded