load_day 2026-03-04, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:450969

Started

2026-11-08T15:28:46Z

Ended

2026-11-08T15:30:01Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

724 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

81.7 s, up to 2.2 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/5001/load_day/2026-03-04/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T15:28:46Z [load_day 2026-03-04] attempt 1 started
2026-11-08T15:28:47Z [load_day 2026-03-04] 2026-03-04: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T15:28:52Z [load_day 2026-03-04] 2026-03-04 hour 0: 148708 events in 0 files, snapshot none (task 4014 ms: download 2383 ms overlapping gunzip 595 ms, parse 1905 ms, sort 0 ms, write 0 ms, index 619 ms: 4 candidates, 77196 repo-hours, 970 releases; waiting 537 ms; commit 0 ms)
2026-11-08T15:28:52Z [load_day 2026-03-04] 2026-03-04 hour 1: 155844 events in 0 files, snapshot none (task 3986 ms: download 2288 ms overlapping gunzip 543 ms, parse 1605 ms, sort 0 ms, write 0 ms, index 766 ms: 10 candidates, 91589 repo-hours, 622 releases; waiting 529 ms; commit 0 ms)
2026-11-08T15:28:56Z [load_day 2026-03-04] 2026-03-04 hour 2: 155360 events in 0 files, snapshot none (task 4207 ms: download 2408 ms overlapping gunzip 630 ms, parse 1929 ms, sort 0 ms, write 0 ms, index 731 ms: 9 candidates, 89509 repo-hours, 707 releases; waiting 556 ms; commit 0 ms)
2026-11-08T15:28:57Z [load_day 2026-03-04] 2026-03-04 hour 3: 151628 events in 0 files, snapshot none (task 4273 ms: download 2364 ms overlapping gunzip 652 ms, parse 2026 ms, sort 0 ms, write 0 ms, index 693 ms: 7 candidates, 76797 repo-hours, 915 releases; waiting 600 ms; commit 0 ms)
2026-11-08T15:29:01Z [load_day 2026-03-04] 2026-03-04 hour 4: 147550 events in 0 files, snapshot none (task 4410 ms: download 2410 ms overlapping gunzip 743 ms, parse 2204 ms, sort 0 ms, write 0 ms, index 605 ms: 7 candidates, 76208 repo-hours, 1025 releases; waiting 514 ms; commit 0 ms)
2026-11-08T15:29:01Z [load_day 2026-03-04] 2026-03-04 hour 5: 146887 events in 0 files, snapshot none (task 4117 ms: download 2735 ms overlapping gunzip 623 ms, parse 1964 ms, sort 0 ms, write 0 ms, index 685 ms: 6 candidates, 78067 repo-hours, 807 releases; waiting 585 ms; commit 0 ms)
2026-11-08T15:29:06Z [load_day 2026-03-04] 2026-03-04 hour 6: 150989 events in 0 files, snapshot none (task 3980 ms: download 2587 ms overlapping gunzip 553 ms, parse 1690 ms, sort 0 ms, write 0 ms, index 714 ms: 6 candidates, 81068 repo-hours, 783 releases; waiting 726 ms; commit 0 ms)
2026-11-08T15:29:06Z [load_day 2026-03-04] 2026-03-04 hour 7: 153157 events in 0 files, snapshot none (task 4195 ms: download 2383 ms overlapping gunzip 543 ms, parse 1531 ms, sort 0 ms, write 0 ms, index 655 ms: 8 candidates, 81698 repo-hours, 690 releases; waiting 1121 ms; commit 0 ms)
2026-11-08T15:29:11Z [load_day 2026-03-04] 2026-03-04 hour 8: 153588 events in 0 files, snapshot none (task 3731 ms: download 2214 ms overlapping gunzip 488 ms, parse 1514 ms, sort 0 ms, write 0 ms, index 702 ms: 12 candidates, 85647 repo-hours, 602 releases; waiting 734 ms; commit 0 ms)
2026-11-08T15:29:11Z [load_day 2026-03-04] 2026-03-04 hour 9: 147190 events in 0 files, snapshot none (task 3580 ms: download 2523 ms overlapping gunzip 421 ms, parse 1290 ms, sort 0 ms, write 0 ms, index 706 ms: 5 candidates, 84525 repo-hours, 652 releases; waiting 892 ms; commit 0 ms)
2026-11-08T15:29:15Z [load_day 2026-03-04] 2026-03-04 hour 10: 153291 events in 0 files, snapshot none (task 3972 ms: download 2012 ms overlapping gunzip 620 ms, parse 1737 ms, sort 0 ms, write 0 ms, index 828 ms: 5 candidates, 87295 repo-hours, 613 releases; waiting 526 ms; commit 0 ms)
2026-11-08T15:29:15Z [load_day 2026-03-04] 2026-03-04 hour 11: 152065 events in 0 files, snapshot none (task 3999 ms: download 2666 ms overlapping gunzip 583 ms, parse 1812 ms, sort 0 ms, write 0 ms, index 664 ms: 4 candidates, 84406 repo-hours, 681 releases; waiting 606 ms; commit 0 ms)
2026-11-08T15:29:19Z [load_day 2026-03-04] 2026-03-04 hour 12: 158602 events in 0 files, snapshot none (task 3813 ms: download 2174 ms overlapping gunzip 490 ms, parse 1551 ms, sort 0 ms, write 0 ms, index 768 ms: 7 candidates, 96063 repo-hours, 459 releases; waiting 692 ms; commit 0 ms)
2026-11-08T15:29:19Z [load_day 2026-03-04] 2026-03-04 hour 13: 160788 events in 0 files, snapshot none (task 3477 ms: download 1762 ms overlapping gunzip 443 ms, parse 1394 ms, sort 0 ms, write 0 ms, index 898 ms: 8 candidates, 99351 repo-hours, 314 releases; waiting 392 ms; commit 0 ms)
2026-11-08T15:29:24Z [load_day 2026-03-04] 2026-03-04 hour 14: 152064 events in 0 files, snapshot none (task 3593 ms: download 2172 ms overlapping gunzip 444 ms, parse 1383 ms, sort 0 ms, write 0 ms, index 983 ms: 3 candidates, 89915 repo-hours, 448 releases; waiting 536 ms; commit 0 ms)
2026-11-08T15:29:24Z [load_day 2026-03-04] 2026-03-04 hour 15: 158509 events in 0 files, snapshot none (task 3612 ms: download 2073 ms overlapping gunzip 495 ms, parse 1504 ms, sort 0 ms, write 0 ms, index 756 ms: 9 candidates, 93453 repo-hours, 383 releases; waiting 568 ms; commit 0 ms)
2026-11-08T15:29:28Z [load_day 2026-03-04] 2026-03-04 hour 16: 158299 events in 0 files, snapshot none (task 3565 ms: download 1969 ms overlapping gunzip 391 ms, parse 1232 ms, sort 0 ms, write 0 ms, index 724 ms: 7 candidates, 92730 repo-hours, 342 releases; waiting 792 ms; commit 0 ms)
2026-11-08T15:29:28Z [load_day 2026-03-04] 2026-03-04 hour 17: 151784 events in 0 files, snapshot none (task 3366 ms: download 2048 ms overlapping gunzip 410 ms, parse 1278 ms, sort 0 ms, write 0 ms, index 690 ms: 6 candidates, 87324 repo-hours, 452 releases; waiting 604 ms; commit 0 ms)
2026-11-08T15:29:33Z [load_day 2026-03-04] 2026-03-04 hour 18: 153906 events in 0 files, snapshot none (task 4414 ms: download 3342 ms overlapping gunzip 444 ms, parse 1401 ms, sort 0 ms, write 0 ms, index 683 ms: 10 candidates, 87633 repo-hours, 527 releases; waiting 1577 ms; commit 0 ms)
2026-11-08T15:29:33Z [load_day 2026-03-04] 2026-03-04 hour 19: 152262 events in 0 files, snapshot none (task 3702 ms: download 2355 ms overlapping gunzip 450 ms, parse 1431 ms, sort 0 ms, write 0 ms, index 749 ms: 4 candidates, 84910 repo-hours, 617 releases; waiting 753 ms; commit 0 ms)
2026-11-08T15:29:36Z [load_day 2026-03-04] 2026-03-04 hour 20: 151432 events in 0 files, snapshot none (task 3176 ms: download 1834 ms overlapping gunzip 373 ms, parse 1156 ms, sort 0 ms, write 0 ms, index 624 ms: 6 candidates, 82587 repo-hours, 538 releases; waiting 733 ms; commit 0 ms)
2026-11-08T15:29:37Z [load_day 2026-03-04] 2026-03-04 hour 21: 155252 events in 0 files, snapshot none (task 3804 ms: download 2576 ms overlapping gunzip 402 ms, parse 1227 ms, sort 0 ms, write 0 ms, index 622 ms: 2 candidates, 82969 repo-hours, 625 releases; waiting 1294 ms; commit 0 ms)
2026-11-08T15:29:40Z [load_day 2026-03-04] 2026-03-04 hour 22: 154616 events in 0 files, snapshot none (task 3697 ms: download 2395 ms overlapping gunzip 433 ms, parse 1267 ms, sort 0 ms, write 0 ms, index 768 ms: 2 candidates, 81940 repo-hours, 843 releases; waiting 839 ms; commit 0 ms)
2026-11-08T15:29:41Z [load_day 2026-03-04] 2026-03-04 hour 23: 145271 events in 0 files, snapshot none (task 3677 ms: download 2624 ms overlapping gunzip 483 ms, parse 1403 ms, sort 0 ms, write 0 ms, index 539 ms: 5 candidates, 72882 repo-hours, 977 releases; waiting 951 ms; commit 0 ms)
2026-11-08T15:29:50Z [load_day 2026-03-04] 2026-03-04: repo_hour merged from 24 files to 1 (2045762 rows, 9210 ms)
2026-11-08T15:30:01Z [load_day 2026-03-04] 2026-03-04: index committed in 20345 ms (24 + 1 + 24 files)
2026-11-08T15:30:01Z [load_day 2026-03-04] 2026-03-04: 3669042 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T15:30:01Z [load_day 2026-03-04] attempt 1 succeeded