load_day 2024-10-15, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:192325

Started

2026-11-08T08:21:20Z

Ended

2026-11-08T08:23:07Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

750 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

168.3 s, up to 3.2 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/3083/load_day/2024-10-15/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T08:21:20Z [load_day 2024-10-15] attempt 1 started
2026-11-08T08:21:21Z [load_day 2024-10-15] 2024-10-15: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:21:28Z [load_day 2024-10-15] 2024-10-15 hour 0: 178811 events in 0 files, snapshot none (task 6507 ms: download 3066 ms overlapping gunzip 967 ms, parse 4083 ms, sort 0 ms, write 0 ms, index 879 ms: 9 candidates, 86864 repo-hours, 1111 releases; waiting 369 ms; commit 0 ms)
2026-11-08T08:21:28Z [load_day 2024-10-15] 2024-10-15 hour 1: 194622 events in 0 files, snapshot none (task 6701 ms: download 3828 ms overlapping gunzip 1013 ms, parse 4110 ms, sort 0 ms, write 0 ms, index 734 ms: 3 candidates, 108015 repo-hours, 805 releases; waiting 489 ms; commit 0 ms)
2026-11-08T08:21:34Z [load_day 2024-10-15] 2024-10-15 hour 2: 178289 events in 0 files, snapshot none (task 6113 ms: download 2496 ms overlapping gunzip 945 ms, parse 3432 ms, sort 0 ms, write 0 ms, index 925 ms: 9 candidates, 81314 repo-hours, 819 releases; waiting 508 ms; commit 0 ms)
2026-11-08T08:21:34Z [load_day 2024-10-15] 2024-10-15 hour 3: 191646 events in 0 files, snapshot none (task 5659 ms: download 2736 ms overlapping gunzip 909 ms, parse 3331 ms, sort 0 ms, write 0 ms, index 674 ms: 6 candidates, 78196 repo-hours, 3987 releases; waiting 502 ms; commit 0 ms)
2026-11-08T08:21:40Z [load_day 2024-10-15] 2024-10-15 hour 4: 173323 events in 0 files, snapshot none (task 5133 ms: download 2818 ms overlapping gunzip 761 ms, parse 2981 ms, sort 0 ms, write 0 ms, index 591 ms: 10 candidates, 65394 repo-hours, 769 releases; waiting 532 ms; commit 0 ms)
2026-11-08T08:21:41Z [load_day 2024-10-15] 2024-10-15 hour 5: 195151 events in 0 files, snapshot none (task 6298 ms: download 3394 ms overlapping gunzip 1001 ms, parse 3588 ms, sort 0 ms, write 0 ms, index 879 ms: 8 candidates, 72493 repo-hours, 926 releases; waiting 510 ms; commit 0 ms)
2026-11-08T08:21:47Z [load_day 2024-10-15] 2024-10-15 hour 6: 223875 events in 0 files, snapshot none (task 6661 ms: download 4128 ms overlapping gunzip 1059 ms, parse 3983 ms, sort 0 ms, write 0 ms, index 686 ms: 6 candidates, 99953 repo-hours, 1060 releases; waiting 643 ms; commit 0 ms)
2026-11-08T08:21:48Z [load_day 2024-10-15] 2024-10-15 hour 7: 229261 events in 0 files, snapshot none (task 6626 ms: download 4252 ms overlapping gunzip 1071 ms, parse 4057 ms, sort 0 ms, write 0 ms, index 655 ms: 9 candidates, 91476 repo-hours, 1090 releases; waiting 523 ms; commit 0 ms)
2026-11-08T08:21:56Z [load_day 2024-10-15] 2024-10-15 hour 8: 242427 events in 0 files, snapshot none (task 8799 ms: download 5688 ms overlapping gunzip 1431 ms, parse 5507 ms, sort 0 ms, write 0 ms, index 757 ms: 9 candidates, 98430 repo-hours, 1234 releases; waiting 655 ms; commit 0 ms)
2026-11-08T08:21:58Z [load_day 2024-10-15] 2024-10-15 hour 9: 251494 events in 0 files, snapshot none (task 8682 ms: download 6069 ms overlapping gunzip 1436 ms, parse 5475 ms, sort 0 ms, write 0 ms, index 892 ms: 8 candidates, 103669 repo-hours, 1442 releases; waiting 577 ms; commit 0 ms)
2026-11-08T08:22:02Z [load_day 2024-10-15] 2024-10-15 hour 10: 236464 events in 0 files, snapshot none (task 5686 ms: download 3328 ms overlapping gunzip 1028 ms, parse 3747 ms, sort 0 ms, write 0 ms, index 714 ms: 7 candidates, 94798 repo-hours, 1114 releases; waiting 9 ms; commit 0 ms)
2026-11-08T08:22:05Z [load_day 2024-10-15] 2024-10-15 hour 11: 238243 events in 0 files, snapshot none (task 6423 ms: download 3546 ms overlapping gunzip 1039 ms, parse 3844 ms, sort 0 ms, write 0 ms, index 1298 ms: 3 candidates, 91619 repo-hours, 1095 releases; waiting 17 ms; commit 0 ms)
2026-11-08T08:22:13Z [load_day 2024-10-15] 2024-10-15 hour 12: 279315 events in 0 files, snapshot none (task 10181 ms: download 6008 ms overlapping gunzip 1481 ms, parse 6054 ms, sort 0 ms, write 0 ms, index 1873 ms: 14 candidates, 139724 repo-hours, 1326 releases; waiting 529 ms; commit 0 ms)
2026-11-08T08:22:14Z [load_day 2024-10-15] 2024-10-15 hour 13: 275313 events in 0 files, snapshot none (task 8717 ms: download 5295 ms overlapping gunzip 1600 ms, parse 5976 ms, sort 0 ms, write 0 ms, index 895 ms: 11 candidates, 124798 repo-hours, 1224 releases; waiting 27 ms; commit 0 ms)
2026-11-08T08:22:21Z [load_day 2024-10-15] 2024-10-15 hour 14: 265786 events in 0 files, snapshot none (task 7891 ms: download 4893 ms overlapping gunzip 1449 ms, parse 5231 ms, sort 0 ms, write 0 ms, index 1015 ms: 9 candidates, 109746 repo-hours, 1341 releases; waiting 9 ms; commit 0 ms)
2026-11-08T08:22:22Z [load_day 2024-10-15] 2024-10-15 hour 15: 264940 events in 0 files, snapshot none (task 8154 ms: download 5763 ms overlapping gunzip 1430 ms, parse 5674 ms, sort 0 ms, write 0 ms, index 813 ms: 2 candidates, 112956 repo-hours, 1485 releases; waiting 25 ms; commit 0 ms)
2026-11-08T08:22:30Z [load_day 2024-10-15] 2024-10-15 hour 16: 247250 events in 0 files, snapshot none (task 8114 ms: download 4854 ms overlapping gunzip 1406 ms, parse 5528 ms, sort 0 ms, write 0 ms, index 983 ms: 4 candidates, 101114 repo-hours, 1277 releases; waiting 4 ms; commit 0 ms)
2026-11-08T08:22:31Z [load_day 2024-10-15] 2024-10-15 hour 17: 241842 events in 0 files, snapshot none (task 7760 ms: download 5312 ms overlapping gunzip 1423 ms, parse 5431 ms, sort 0 ms, write 0 ms, index 684 ms: 6 candidates, 94585 repo-hours, 1227 releases; waiting 12 ms; commit 0 ms)
2026-11-08T08:22:39Z [load_day 2024-10-15] 2024-10-15 hour 18: 245679 events in 0 files, snapshot none (task 8447 ms: download 4851 ms overlapping gunzip 1446 ms, parse 5597 ms, sort 0 ms, write 0 ms, index 1183 ms: 9 candidates, 107178 repo-hours, 1237 releases; waiting 19 ms; commit 0 ms)
2026-11-08T08:22:40Z [load_day 2024-10-15] 2024-10-15 hour 19: 225799 events in 0 files, snapshot none (task 8617 ms: download 5875 ms overlapping gunzip 1574 ms, parse 6010 ms, sort 0 ms, write 0 ms, index 831 ms: 7 candidates, 84726 repo-hours, 1153 releases; waiting 8 ms; commit 0 ms)
2026-11-08T08:22:48Z [load_day 2024-10-15] 2024-10-15 hour 20: 217822 events in 0 files, snapshot none (task 8261 ms: download 4809 ms overlapping gunzip 1526 ms, parse 5515 ms, sort 0 ms, write 0 ms, index 1022 ms: 5 candidates, 82947 repo-hours, 1087 releases; waiting 2 ms; commit 0 ms)
2026-11-08T08:22:48Z [load_day 2024-10-15] 2024-10-15 hour 21: 212178 events in 0 files, snapshot none (task 7506 ms: download 4673 ms overlapping gunzip 1379 ms, parse 5166 ms, sort 0 ms, write 0 ms, index 713 ms: 19 candidates, 79603 repo-hours, 1204 releases; waiting 18 ms; commit 0 ms)
2026-11-08T08:22:54Z [load_day 2024-10-15] 2024-10-15 hour 22: 186623 events in 0 files, snapshot none (task 5545 ms: download 3196 ms overlapping gunzip 996 ms, parse 3714 ms, sort 0 ms, write 0 ms, index 619 ms: 8 candidates, 65715 repo-hours, 1176 releases; waiting 17 ms; commit 0 ms)
2026-11-08T08:22:54Z [load_day 2024-10-15] 2024-10-15 hour 23: 182533 events in 0 files, snapshot none (task 5442 ms: download 3085 ms overlapping gunzip 962 ms, parse 3619 ms, sort 0 ms, write 0 ms, index 628 ms: 6 candidates, 62894 repo-hours, 762 releases; waiting 24 ms; commit 0 ms)
2026-11-08T08:23:05Z [load_day 2024-10-15] 2024-10-15: repo_hour merged from 24 files to 1 (2238207 rows, 10770 ms)
2026-11-08T08:23:07Z [load_day 2024-10-15] 2024-10-15: index committed in 13336 ms (24 + 1 + 24 files)
2026-11-08T08:23:07Z [load_day 2024-10-15] 2024-10-15: 5378686 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:23:07Z [load_day 2024-10-15] attempt 1 succeeded