load_day 2023-12-31, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:14512

Started

2026-11-08T03:57:47Z

Ended

2026-11-08T03:58:51Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

539 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

84.7 s, up to 2.8 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1918/load_day/2023-12-31/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T03:57:47Z [load_day 2023-12-31] attempt 1 started
2026-11-08T03:57:48Z [load_day 2023-12-31] 2023-12-31: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T03:57:53Z [load_day 2023-12-31] 2023-12-31 hour 0: 150026 events in 0 files, snapshot none (task 4653 ms: download 2521 ms overlapping gunzip 597 ms, parse 2751 ms, sort 0 ms, write 0 ms, index 480 ms: 8 candidates, 66383 repo-hours, 920 releases; waiting 489 ms; commit 0 ms)
2026-11-08T03:57:53Z [load_day 2023-12-31] 2023-12-31 hour 1: 151737 events in 0 files, snapshot none (task 4396 ms: download 2368 ms overlapping gunzip 548 ms, parse 2538 ms, sort 0 ms, write 0 ms, index 509 ms: 6 candidates, 72995 repo-hours, 768 releases; waiting 489 ms; commit 0 ms)
2026-11-08T03:57:57Z [load_day 2023-12-31] 2023-12-31 hour 2: 122265 events in 0 files, snapshot none (task 3852 ms: download 2081 ms overlapping gunzip 545 ms, parse 2255 ms, sort 0 ms, write 0 ms, index 312 ms: 3 candidates, 37217 repo-hours, 670 releases; waiting 526 ms; commit 0 ms)
2026-11-08T03:57:58Z [load_day 2023-12-31] 2023-12-31 hour 3: 123218 events in 0 files, snapshot none (task 3677 ms: download 1912 ms overlapping gunzip 483 ms, parse 2149 ms, sort 0 ms, write 0 ms, index 302 ms: 4 candidates, 36817 repo-hours, 680 releases; waiting 507 ms; commit 0 ms)
2026-11-08T03:58:01Z [load_day 2023-12-31] 2023-12-31 hour 4: 124968 events in 0 files, snapshot none (task 3239 ms: download 2020 ms overlapping gunzip 377 ms, parse 1655 ms, sort 0 ms, write 0 ms, index 328 ms: 5 candidates, 37229 repo-hours, 439 releases; waiting 513 ms; commit 0 ms)
2026-11-08T03:58:01Z [load_day 2023-12-31] 2023-12-31 hour 5: 129286 events in 0 files, snapshot none (task 3390 ms: download 1940 ms overlapping gunzip 421 ms, parse 1842 ms, sort 0 ms, write 0 ms, index 353 ms: 4 candidates, 39053 repo-hours, 484 releases; waiting 522 ms; commit 0 ms)
2026-11-08T03:58:05Z [load_day 2023-12-31] 2023-12-31 hour 6: 141876 events in 0 files, snapshot none (task 3002 ms: download 1428 ms overlapping gunzip 374 ms, parse 1733 ms, sort 0 ms, write 0 ms, index 343 ms: 2 candidates, 46455 repo-hours, 616 releases; waiting 294 ms; commit 0 ms)
2026-11-08T03:58:05Z [load_day 2023-12-31] 2023-12-31 hour 7: 134843 events in 0 files, snapshot none (task 3212 ms: download 1691 ms overlapping gunzip 375 ms, parse 1692 ms, sort 0 ms, write 0 ms, index 306 ms: 2 candidates, 40478 repo-hours, 486 releases; waiting 514 ms; commit 0 ms)
2026-11-08T03:58:08Z [load_day 2023-12-31] 2023-12-31 hour 8: 140016 events in 0 files, snapshot none (task 3023 ms: download 1964 ms overlapping gunzip 389 ms, parse 1709 ms, sort 0 ms, write 0 ms, index 407 ms: 4 candidates, 43013 repo-hours, 508 releases; waiting 263 ms; commit 0 ms)
2026-11-08T03:58:09Z [load_day 2023-12-31] 2023-12-31 hour 9: 145908 events in 0 files, snapshot none (task 3691 ms: download 1888 ms overlapping gunzip 413 ms, parse 1902 ms, sort 0 ms, write 0 ms, index 543 ms: 2 candidates, 43806 repo-hours, 746 releases; waiting 498 ms; commit 0 ms)
2026-11-08T03:58:13Z [load_day 2023-12-31] 2023-12-31 hour 10: 146111 events in 0 files, snapshot none (task 3757 ms: download 1928 ms overlapping gunzip 491 ms, parse 2122 ms, sort 0 ms, write 0 ms, index 408 ms: 2 candidates, 46812 repo-hours, 592 releases; waiting 493 ms; commit 0 ms)
2026-11-08T03:58:13Z [load_day 2023-12-31] 2023-12-31 hour 11: 144022 events in 0 files, snapshot none (task 3856 ms: download 3041 ms overlapping gunzip 440 ms, parse 1902 ms, sort 0 ms, write 0 ms, index 364 ms: 6 candidates, 44830 repo-hours, 653 releases; waiting 764 ms; commit 0 ms)
2026-11-08T03:58:18Z [load_day 2023-12-31] 2023-12-31 hour 12: 214208 events in 0 files, snapshot none (task 5351 ms: download 2630 ms overlapping gunzip 597 ms, parse 3068 ms, sort 0 ms, write 0 ms, index 849 ms: 6 candidates, 79067 repo-hours, 741 releases; waiting 498 ms; commit 0 ms)
2026-11-08T03:58:19Z [load_day 2023-12-31] 2023-12-31 hour 13: 169840 events in 0 files, snapshot none (task 4846 ms: download 2380 ms overlapping gunzip 685 ms, parse 2884 ms, sort 0 ms, write 0 ms, index 545 ms: 7 candidates, 68595 repo-hours, 648 releases; waiting 466 ms; commit 0 ms)
2026-11-08T03:58:23Z [load_day 2023-12-31] 2023-12-31 hour 14: 156286 events in 0 files, snapshot none (task 4457 ms: download 1900 ms overlapping gunzip 692 ms, parse 2930 ms, sort 0 ms, write 0 ms, index 415 ms: 6 candidates, 52535 repo-hours, 841 releases; waiting 169 ms; commit 0 ms)
2026-11-08T03:58:24Z [load_day 2023-12-31] 2023-12-31 hour 15: 159997 events in 0 files, snapshot none (task 4218 ms: download 2200 ms overlapping gunzip 600 ms, parse 2483 ms, sort 0 ms, write 0 ms, index 404 ms: 8 candidates, 54037 repo-hours, 907 releases; waiting 389 ms; commit 0 ms)
2026-11-08T03:58:28Z [load_day 2023-12-31] 2023-12-31 hour 16: 154070 events in 0 files, snapshot none (task 4536 ms: download 3162 ms overlapping gunzip 582 ms, parse 2483 ms, sort 0 ms, write 0 ms, index 374 ms: 4 candidates, 51685 repo-hours, 807 releases; waiting 678 ms; commit 0 ms)
2026-11-08T03:58:28Z [load_day 2023-12-31] 2023-12-31 hour 17: 150041 events in 0 files, snapshot none (task 3752 ms: download 1995 ms overlapping gunzip 484 ms, parse 2103 ms, sort 0 ms, write 0 ms, index 372 ms: 3 candidates, 46366 repo-hours, 677 releases; waiting 499 ms; commit 0 ms)
2026-11-08T03:58:33Z [load_day 2023-12-31] 2023-12-31 hour 18: 150924 events in 0 files, snapshot none (task 4319 ms: download 1962 ms overlapping gunzip 578 ms, parse 2550 ms, sort 0 ms, write 0 ms, index 456 ms: 3 candidates, 51203 repo-hours, 716 releases; waiting 469 ms; commit 0 ms)
2026-11-08T03:58:33Z [load_day 2023-12-31] 2023-12-31 hour 19: 136500 events in 0 files, snapshot none (task 4299 ms: download 3470 ms overlapping gunzip 599 ms, parse 2424 ms, sort 0 ms, write 0 ms, index 468 ms: 7 candidates, 38508 repo-hours, 592 releases; waiting 502 ms; commit 0 ms)
2026-11-08T03:58:38Z [load_day 2023-12-31] 2023-12-31 hour 20: 129151 events in 0 files, snapshot none (task 4095 ms: download 2377 ms overlapping gunzip 583 ms, parse 2374 ms, sort 0 ms, write 0 ms, index 302 ms: 4 candidates, 37783 repo-hours, 657 releases; waiting 496 ms; commit 0 ms)
2026-11-08T03:58:38Z [load_day 2023-12-31] 2023-12-31 hour 21: 117164 events in 0 files, snapshot none (task 3518 ms: download 1698 ms overlapping gunzip 443 ms, parse 2018 ms, sort 0 ms, write 0 ms, index 377 ms: 2 candidates, 32801 repo-hours, 621 releases; waiting 476 ms; commit 0 ms)
2026-11-08T03:58:43Z [load_day 2023-12-31] 2023-12-31 hour 22: 115690 events in 0 files, snapshot none (task 4366 ms: download 2006 ms overlapping gunzip 605 ms, parse 2699 ms, sort 0 ms, write 0 ms, index 263 ms: 2 candidates, 30565 repo-hours, 740 releases; waiting 522 ms; commit 0 ms)
2026-11-08T03:58:43Z [load_day 2023-12-31] 2023-12-31 hour 23: 112622 events in 0 files, snapshot none (task 4345 ms: download 1921 ms overlapping gunzip 600 ms, parse 2745 ms, sort 0 ms, write 0 ms, index 304 ms: 3 candidates, 31126 repo-hours, 527 releases; waiting 464 ms; commit 0 ms)
2026-11-08T03:58:48Z [load_day 2023-12-31] 2023-12-31: repo_hour merged from 24 files to 1 (1129359 rows, 5583 ms)
2026-11-08T03:58:51Z [load_day 2023-12-31] 2023-12-31: index committed in 7968 ms (24 + 1 + 24 files)
2026-11-08T03:58:51Z [load_day 2023-12-31] 2023-12-31: 3420769 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T03:58:51Z [load_day 2023-12-31] attempt 1 succeeded