load_day 2024-01-07, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:16669

Started

2026-11-08T04:02:49Z

Ended

2026-11-08T04:04:02Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

587 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

101.5 s, up to 3.2 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1943/load_day/2024-01-07/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:02:49Z [load_day 2024-01-07] attempt 1 started
2026-11-08T04:02:50Z [load_day 2024-01-07] 2024-01-07: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:02:56Z [load_day 2024-01-07] 2024-01-07 hour 0: 158137 events in 0 files, snapshot none (task 5513 ms: download 2853 ms overlapping gunzip 760 ms, parse 3403 ms, sort 0 ms, write 0 ms, index 483 ms: 7 candidates, 66798 repo-hours, 947 releases; waiting 549 ms; commit 0 ms)
2026-11-08T04:02:56Z [load_day 2024-01-07] 2024-01-07 hour 1: 161987 events in 0 files, snapshot none (task 5089 ms: download 2184 ms overlapping gunzip 674 ms, parse 3072 ms, sort 0 ms, write 0 ms, index 593 ms: 2 candidates, 74441 repo-hours, 842 releases; waiting 484 ms; commit 0 ms)
2026-11-08T04:03:01Z [load_day 2024-01-07] 2024-01-07 hour 2: 132881 events in 0 files, snapshot none (task 4006 ms: download 2279 ms overlapping gunzip 515 ms, parse 2191 ms, sort 0 ms, write 0 ms, index 453 ms: 4 candidates, 42622 repo-hours, 642 releases; waiting 504 ms; commit 0 ms)
2026-11-08T04:03:01Z [load_day 2024-01-07] 2024-01-07 hour 3: 132288 events in 0 files, snapshot none (task 3978 ms: download 2203 ms overlapping gunzip 503 ms, parse 2224 ms, sort 0 ms, write 0 ms, index 417 ms: 3 candidates, 39559 repo-hours, 649 releases; waiting 500 ms; commit 0 ms)
2026-11-08T04:03:05Z [load_day 2024-01-07] 2024-01-07 hour 4: 132625 events in 0 files, snapshot none (task 3439 ms: download 1945 ms overlapping gunzip 399 ms, parse 1740 ms, sort 0 ms, write 0 ms, index 433 ms: 6 candidates, 41341 repo-hours, 476 releases; waiting 506 ms; commit 0 ms)
2026-11-08T04:03:05Z [load_day 2024-01-07] 2024-01-07 hour 5: 135850 events in 0 files, snapshot none (task 3669 ms: download 1769 ms overlapping gunzip 471 ms, parse 1977 ms, sort 0 ms, write 0 ms, index 395 ms: 2 candidates, 42391 repo-hours, 485 releases; waiting 507 ms; commit 0 ms)
2026-11-08T04:03:09Z [load_day 2024-01-07] 2024-01-07 hour 6: 150373 events in 0 files, snapshot none (task 3603 ms: download 1942 ms overlapping gunzip 435 ms, parse 2000 ms, sort 0 ms, write 0 ms, index 361 ms: 4 candidates, 52518 repo-hours, 643 releases; waiting 511 ms; commit 0 ms)
2026-11-08T04:03:09Z [load_day 2024-01-07] 2024-01-07 hour 7: 143675 events in 0 files, snapshot none (task 3811 ms: download 2791 ms overlapping gunzip 497 ms, parse 2109 ms, sort 0 ms, write 0 ms, index 385 ms: 3 candidates, 45741 repo-hours, 541 releases; waiting 469 ms; commit 0 ms)
2026-11-08T04:03:13Z [load_day 2024-01-07] 2024-01-07 hour 8: 148931 events in 0 files, snapshot none (task 3963 ms: download 2255 ms overlapping gunzip 522 ms, parse 2217 ms, sort 0 ms, write 0 ms, index 380 ms: 2 candidates, 49119 repo-hours, 541 releases; waiting 481 ms; commit 0 ms)
2026-11-08T04:03:14Z [load_day 2024-01-07] 2024-01-07 hour 9: 155229 events in 0 files, snapshot none (task 4012 ms: download 1956 ms overlapping gunzip 546 ms, parse 2344 ms, sort 0 ms, write 0 ms, index 376 ms: 3 candidates, 51625 repo-hours, 661 releases; waiting 483 ms; commit 0 ms)
2026-11-08T04:03:18Z [load_day 2024-01-07] 2024-01-07 hour 10: 161035 events in 0 files, snapshot none (task 4184 ms: download 2731 ms overlapping gunzip 558 ms, parse 2358 ms, sort 0 ms, write 0 ms, index 445 ms: 5 candidates, 54936 repo-hours, 679 releases; waiting 525 ms; commit 0 ms)
2026-11-08T04:03:18Z [load_day 2024-01-07] 2024-01-07 hour 11: 163075 events in 0 files, snapshot none (task 4141 ms: download 2063 ms overlapping gunzip 541 ms, parse 2390 ms, sort 0 ms, write 0 ms, index 380 ms: 2 candidates, 54616 repo-hours, 693 releases; waiting 519 ms; commit 0 ms)
2026-11-08T04:03:23Z [load_day 2024-01-07] 2024-01-07 hour 12: 226982 events in 0 files, snapshot none (task 4854 ms: download 2208 ms overlapping gunzip 586 ms, parse 2843 ms, sort 0 ms, write 0 ms, index 636 ms: 8 candidates, 88800 repo-hours, 784 releases; waiting 497 ms; commit 0 ms)
2026-11-08T04:03:23Z [load_day 2024-01-07] 2024-01-07 hour 13: 195483 events in 0 files, snapshot none (task 4635 ms: download 2135 ms overlapping gunzip 579 ms, parse 2678 ms, sort 0 ms, write 0 ms, index 611 ms: 6 candidates, 81427 repo-hours, 733 releases; waiting 490 ms; commit 0 ms)
2026-11-08T04:03:29Z [load_day 2024-01-07] 2024-01-07 hour 14: 184825 events in 0 files, snapshot none (task 5488 ms: download 2764 ms overlapping gunzip 747 ms, parse 3342 ms, sort 0 ms, write 0 ms, index 601 ms: 4 candidates, 67171 repo-hours, 745 releases; waiting 510 ms; commit 0 ms)
2026-11-08T04:03:30Z [load_day 2024-01-07] 2024-01-07 hour 15: 189501 events in 0 files, snapshot none (task 5803 ms: download 3347 ms overlapping gunzip 854 ms, parse 3640 ms, sort 0 ms, write 0 ms, index 542 ms: 9 candidates, 69173 repo-hours, 820 releases; waiting 494 ms; commit 0 ms)
2026-11-08T04:03:36Z [load_day 2024-01-07] 2024-01-07 hour 16: 195737 events in 0 files, snapshot none (task 6250 ms: download 3612 ms overlapping gunzip 898 ms, parse 3918 ms, sort 0 ms, write 0 ms, index 552 ms: 5 candidates, 71185 repo-hours, 803 releases; waiting 517 ms; commit 0 ms)
2026-11-08T04:03:36Z [load_day 2024-01-07] 2024-01-07 hour 17: 187620 events in 0 files, snapshot none (task 5955 ms: download 3335 ms overlapping gunzip 849 ms, parse 3900 ms, sort 0 ms, write 0 ms, index 467 ms: 10 candidates, 65342 repo-hours, 787 releases; waiting 480 ms; commit 0 ms)
2026-11-08T04:03:42Z [load_day 2024-01-07] 2024-01-07 hour 18: 197007 events in 0 files, snapshot none (task 6183 ms: download 3474 ms overlapping gunzip 884 ms, parse 4036 ms, sort 0 ms, write 0 ms, index 553 ms: 6 candidates, 71142 repo-hours, 919 releases; waiting 490 ms; commit 0 ms)
2026-11-08T04:03:43Z [load_day 2024-01-07] 2024-01-07 hour 19: 176857 events in 0 files, snapshot none (task 6228 ms: download 3681 ms overlapping gunzip 990 ms, parse 4118 ms, sort 0 ms, write 0 ms, index 408 ms: 8 candidates, 57886 repo-hours, 832 releases; waiting 510 ms; commit 0 ms)
2026-11-08T04:03:48Z [load_day 2024-01-07] 2024-01-07 hour 20: 173250 events in 0 files, snapshot none (task 5244 ms: download 2904 ms overlapping gunzip 694 ms, parse 3241 ms, sort 0 ms, write 0 ms, index 524 ms: 4 candidates, 58724 repo-hours, 939 releases; waiting 559 ms; commit 0 ms)
2026-11-08T04:03:48Z [load_day 2024-01-07] 2024-01-07 hour 21: 162545 events in 0 files, snapshot none (task 5146 ms: download 2324 ms overlapping gunzip 723 ms, parse 3280 ms, sort 0 ms, write 0 ms, index 459 ms: 5 candidates, 52672 repo-hours, 960 releases; waiting 485 ms; commit 0 ms)
2026-11-08T04:03:53Z [load_day 2024-01-07] 2024-01-07 hour 22: 155642 events in 0 files, snapshot none (task 4485 ms: download 3116 ms overlapping gunzip 561 ms, parse 2414 ms, sort 0 ms, write 0 ms, index 460 ms: 3 candidates, 49285 repo-hours, 845 releases; waiting 795 ms; commit 0 ms)
2026-11-08T04:03:53Z [load_day 2024-01-07] 2024-01-07 hour 23: 147068 events in 0 files, snapshot none (task 3967 ms: download 2480 ms overlapping gunzip 500 ms, parse 2237 ms, sort 0 ms, write 0 ms, index 438 ms: 5 candidates, 46336 repo-hours, 654 releases; waiting 486 ms; commit 0 ms)
2026-11-08T04:03:59Z [load_day 2024-01-07] 2024-01-07: repo_hour merged from 24 files to 1 (1394850 rows, 6196 ms)
2026-11-08T04:04:02Z [load_day 2024-01-07] 2024-01-07: index committed in 8589 ms (24 + 1 + 24 files)
2026-11-08T04:04:02Z [load_day 2024-01-07] 2024-01-07: 3968603 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:04:02Z [load_day 2024-01-07] attempt 1 succeeded