load_day 2024-02-04, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:82820

Started

2026-11-08T04:29:10Z

Ended

2026-11-08T04:30:42Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

592 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

120.2 s, up to 2.7 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2046/load_day/2024-02-04/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:29:10Z [load_day 2024-02-04] attempt 1 started
2026-11-08T04:29:11Z [load_day 2024-02-04] 2024-02-04: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:29:19Z [load_day 2024-02-04] 2024-02-04 hour 0: 202667 events in 0 files, snapshot none (task 7453 ms: download 4383 ms overlapping gunzip 907 ms, parse 4310 ms, sort 0 ms, write 0 ms, index 960 ms: 5 candidates, 75339 repo-hours, 1055 releases; waiting 712 ms; commit 0 ms)
2026-11-08T04:29:19Z [load_day 2024-02-04] 2024-02-04 hour 1: 207403 events in 0 files, snapshot none (task 6727 ms: download 2844 ms overlapping gunzip 855 ms, parse 4014 ms, sort 0 ms, write 0 ms, index 1156 ms: 7 candidates, 82332 repo-hours, 758 releases; waiting 447 ms; commit 0 ms)
2026-11-08T04:29:25Z [load_day 2024-02-04] 2024-02-04 hour 2: 177078 events in 0 files, snapshot none (task 5131 ms: download 2518 ms overlapping gunzip 703 ms, parse 3114 ms, sort 0 ms, write 0 ms, index 550 ms: 7 candidates, 48987 repo-hours, 638 releases; waiting 472 ms; commit 0 ms)
2026-11-08T04:29:25Z [load_day 2024-02-04] 2024-02-04 hour 3: 173948 events in 0 files, snapshot none (task 5480 ms: download 2186 ms overlapping gunzip 680 ms, parse 3226 ms, sort 0 ms, write 0 ms, index 806 ms: 1 candidates, 47520 repo-hours, 721 releases; waiting 488 ms; commit 0 ms)
2026-11-08T04:29:30Z [load_day 2024-02-04] 2024-02-04 hour 4: 172687 events in 0 files, snapshot none (task 5025 ms: download 2058 ms overlapping gunzip 687 ms, parse 3213 ms, sort 0 ms, write 0 ms, index 338 ms: 0 candidates, 44935 repo-hours, 453 releases; waiting 521 ms; commit 0 ms)
2026-11-08T04:29:31Z [load_day 2024-02-04] 2024-02-04 hour 5: 181967 events in 0 files, snapshot none (task 5009 ms: download 2370 ms overlapping gunzip 721 ms, parse 3114 ms, sort 0 ms, write 0 ms, index 369 ms: 10 candidates, 50082 repo-hours, 633 releases; waiting 228 ms; commit 0 ms)
2026-11-08T04:29:37Z [load_day 2024-02-04] 2024-02-04 hour 6: 199506 events in 0 files, snapshot none (task 5996 ms: download 3268 ms overlapping gunzip 881 ms, parse 3904 ms, sort 0 ms, write 0 ms, index 474 ms: 1 candidates, 62761 repo-hours, 666 releases; waiting 510 ms; commit 0 ms)
2026-11-08T04:29:37Z [load_day 2024-02-04] 2024-02-04 hour 7: 196071 events in 0 files, snapshot none (task 6267 ms: download 3332 ms overlapping gunzip 946 ms, parse 4082 ms, sort 0 ms, write 0 ms, index 453 ms: 9 candidates, 56993 repo-hours, 645 releases; waiting 508 ms; commit 0 ms)
2026-11-08T04:29:43Z [load_day 2024-02-04] 2024-02-04 hour 8: 199964 events in 0 files, snapshot none (task 5921 ms: download 3311 ms overlapping gunzip 824 ms, parse 3806 ms, sort 0 ms, write 0 ms, index 518 ms: 8 candidates, 59365 repo-hours, 738 releases; waiting 526 ms; commit 0 ms)
2026-11-08T04:29:44Z [load_day 2024-02-04] 2024-02-04 hour 9: 205505 events in 0 files, snapshot none (task 5724 ms: download 3180 ms overlapping gunzip 836 ms, parse 3659 ms, sort 0 ms, write 0 ms, index 488 ms: 6 candidates, 61457 repo-hours, 785 releases; waiting 461 ms; commit 0 ms)
2026-11-08T04:29:50Z [load_day 2024-02-04] 2024-02-04 hour 10: 209194 events in 0 files, snapshot none (task 6132 ms: download 3620 ms overlapping gunzip 867 ms, parse 4028 ms, sort 0 ms, write 0 ms, index 440 ms: 11 candidates, 63604 repo-hours, 774 releases; waiting 504 ms; commit 0 ms)
2026-11-08T04:29:50Z [load_day 2024-02-04] 2024-02-04 hour 11: 209201 events in 0 files, snapshot none (task 5517 ms: download 3124 ms overlapping gunzip 759 ms, parse 3505 ms, sort 0 ms, write 0 ms, index 440 ms: 6 candidates, 61944 repo-hours, 708 releases; waiting 488 ms; commit 0 ms)
2026-11-08T04:29:56Z [load_day 2024-02-04] 2024-02-04 hour 12: 275806 events in 0 files, snapshot none (task 5907 ms: download 3042 ms overlapping gunzip 788 ms, parse 3728 ms, sort 0 ms, write 0 ms, index 635 ms: 6 candidates, 92375 repo-hours, 769 releases; waiting 498 ms; commit 0 ms)
2026-11-08T04:29:56Z [load_day 2024-02-04] 2024-02-04 hour 13: 231836 events in 0 files, snapshot none (task 5515 ms: download 2852 ms overlapping gunzip 741 ms, parse 3402 ms, sort 0 ms, write 0 ms, index 616 ms: 9 candidates, 83245 repo-hours, 641 releases; waiting 479 ms; commit 0 ms)
2026-11-08T04:30:03Z [load_day 2024-02-04] 2024-02-04 hour 14: 225745 events in 0 files, snapshot none (task 5970 ms: download 3021 ms overlapping gunzip 776 ms, parse 3517 ms, sort 0 ms, write 0 ms, index 866 ms: 15 candidates, 71211 repo-hours, 752 releases; waiting 513 ms; commit 0 ms)
2026-11-08T04:30:03Z [load_day 2024-02-04] 2024-02-04 hour 15: 226744 events in 0 files, snapshot none (task 6322 ms: download 3044 ms overlapping gunzip 856 ms, parse 3842 ms, sort 0 ms, write 0 ms, index 914 ms: 10 candidates, 72482 repo-hours, 984 releases; waiting 494 ms; commit 0 ms)
2026-11-08T04:30:10Z [load_day 2024-02-04] 2024-02-04 hour 16: 226238 events in 0 files, snapshot none (task 6366 ms: download 3606 ms overlapping gunzip 922 ms, parse 4100 ms, sort 0 ms, write 0 ms, index 622 ms: 10 candidates, 71483 repo-hours, 854 releases; waiting 497 ms; commit 0 ms)
2026-11-08T04:30:10Z [load_day 2024-02-04] 2024-02-04 hour 17: 217746 events in 0 files, snapshot none (task 6141 ms: download 3533 ms overlapping gunzip 896 ms, parse 3947 ms, sort 0 ms, write 0 ms, index 555 ms: 9 candidates, 66302 repo-hours, 751 releases; waiting 485 ms; commit 0 ms)
2026-11-08T04:30:17Z [load_day 2024-02-04] 2024-02-04 hour 18: 224489 events in 0 files, snapshot none (task 6791 ms: download 3814 ms overlapping gunzip 935 ms, parse 4462 ms, sort 0 ms, write 0 ms, index 642 ms: 7 candidates, 74490 repo-hours, 915 releases; waiting 466 ms; commit 0 ms)
2026-11-08T04:30:17Z [load_day 2024-02-04] 2024-02-04 hour 19: 210267 events in 0 files, snapshot none (task 6221 ms: download 3573 ms overlapping gunzip 853 ms, parse 4096 ms, sort 0 ms, write 0 ms, index 530 ms: 16 candidates, 61063 repo-hours, 836 releases; waiting 467 ms; commit 0 ms)
2026-11-08T04:30:24Z [load_day 2024-02-04] 2024-02-04 hour 20: 208348 events in 0 files, snapshot none (task 5970 ms: download 3145 ms overlapping gunzip 863 ms, parse 3839 ms, sort 0 ms, write 0 ms, index 525 ms: 10 candidates, 60915 repo-hours, 815 releases; waiting 508 ms; commit 0 ms)
2026-11-08T04:30:24Z [load_day 2024-02-04] 2024-02-04 hour 21: 200164 events in 0 files, snapshot none (task 5981 ms: download 3123 ms overlapping gunzip 814 ms, parse 3827 ms, sort 0 ms, write 0 ms, index 578 ms: 21 candidates, 56575 repo-hours, 825 releases; waiting 476 ms; commit 0 ms)
2026-11-08T04:30:30Z [load_day 2024-02-04] 2024-02-04 hour 22: 194723 events in 0 files, snapshot none (task 5927 ms: download 3074 ms overlapping gunzip 816 ms, parse 3690 ms, sort 0 ms, write 0 ms, index 456 ms: 10 candidates, 56174 repo-hours, 869 releases; waiting 492 ms; commit 0 ms)
2026-11-08T04:30:30Z [load_day 2024-02-04] 2024-02-04 hour 23: 178735 events in 0 files, snapshot none (task 5731 ms: download 2788 ms overlapping gunzip 850 ms, parse 3692 ms, sort 0 ms, write 0 ms, index 485 ms: 8 candidates, 48694 repo-hours, 642 releases; waiting 495 ms; commit 0 ms)
2026-11-08T04:30:38Z [load_day 2024-02-04] 2024-02-04: repo_hour merged from 24 files to 1 (1530328 rows, 7666 ms)
2026-11-08T04:30:42Z [load_day 2024-02-04] 2024-02-04: index committed in 12269 ms (23 + 1 + 24 files)
2026-11-08T04:30:42Z [load_day 2024-02-04] 2024-02-04: 4956032 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:30:42Z [load_day 2024-02-04] attempt 1 succeeded