load_day 2024-03-09, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:95821

Started

2026-11-08T05:01:46Z

Ended

2026-11-08T05:03:13Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

586 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

121.7 s, up to 2.6 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2175/load_day/2024-03-09/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T05:01:46Z [load_day 2024-03-09] attempt 1 started
2026-11-08T05:01:47Z [load_day 2024-03-09] 2024-03-09: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T05:01:55Z [load_day 2024-03-09] 2024-03-09 hour 0: 237996 events in 0 files, snapshot none (task 7340 ms: download 4340 ms overlapping gunzip 1049 ms, parse 4869 ms, sort 0 ms, write 0 ms, index 673 ms: 13 candidates, 87128 repo-hours, 1087 releases; waiting 480 ms; commit 0 ms)
2026-11-08T05:01:55Z [load_day 2024-03-09] 2024-03-09 hour 1: 229209 events in 0 files, snapshot none (task 7721 ms: download 4616 ms overlapping gunzip 1085 ms, parse 5260 ms, sort 0 ms, write 0 ms, index 640 ms: 9 candidates, 81277 repo-hours, 709 releases; waiting 465 ms; commit 0 ms)
2026-11-08T05:02:02Z [load_day 2024-03-09] 2024-03-09 hour 2: 187024 events in 0 files, snapshot none (task 6407 ms: download 2535 ms overlapping gunzip 877 ms, parse 4014 ms, sort 0 ms, write 0 ms, index 684 ms: 5 candidates, 46012 repo-hours, 581 releases; waiting 486 ms; commit 0 ms)
2026-11-08T05:02:02Z [load_day 2024-03-09] 2024-03-09 hour 3: 192783 events in 0 files, snapshot none (task 6145 ms: download 2445 ms overlapping gunzip 876 ms, parse 3946 ms, sort 0 ms, write 0 ms, index 723 ms: 7 candidates, 47432 repo-hours, 651 releases; waiting 279 ms; commit 0 ms)
2026-11-08T05:02:07Z [load_day 2024-03-09] 2024-03-09 hour 4: 195929 events in 0 files, snapshot none (task 4846 ms: download 2470 ms overlapping gunzip 692 ms, parse 3081 ms, sort 0 ms, write 0 ms, index 349 ms: 6 candidates, 48643 repo-hours, 667 releases; waiting 493 ms; commit 0 ms)
2026-11-08T05:02:07Z [load_day 2024-03-09] 2024-03-09 hour 5: 198619 events in 0 files, snapshot none (task 4687 ms: download 2527 ms overlapping gunzip 704 ms, parse 3026 ms, sort 0 ms, write 0 ms, index 338 ms: 2 candidates, 49643 repo-hours, 572 releases; waiting 247 ms; commit 0 ms)
2026-11-08T05:02:14Z [load_day 2024-03-09] 2024-03-09 hour 6: 211495 events in 0 files, snapshot none (task 5848 ms: download 3243 ms overlapping gunzip 821 ms, parse 3725 ms, sort 0 ms, write 0 ms, index 501 ms: 4 candidates, 58731 repo-hours, 841 releases; waiting 496 ms; commit 0 ms)
2026-11-08T05:02:14Z [load_day 2024-03-09] 2024-03-09 hour 7: 207298 events in 0 files, snapshot none (task 5534 ms: download 2756 ms overlapping gunzip 806 ms, parse 3540 ms, sort 0 ms, write 0 ms, index 464 ms: 6 candidates, 52308 repo-hours, 761 releases; waiting 451 ms; commit 0 ms)
2026-11-08T05:02:20Z [load_day 2024-03-09] 2024-03-09 hour 8: 210608 events in 0 files, snapshot none (task 5616 ms: download 2626 ms overlapping gunzip 753 ms, parse 3501 ms, sort 0 ms, write 0 ms, index 610 ms: 8 candidates, 56978 repo-hours, 724 releases; waiting 492 ms; commit 0 ms)
2026-11-08T05:02:20Z [load_day 2024-03-09] 2024-03-09 hour 9: 211975 events in 0 files, snapshot none (task 5449 ms: download 2847 ms overlapping gunzip 763 ms, parse 3473 ms, sort 0 ms, write 0 ms, index 444 ms: 5 candidates, 58695 repo-hours, 863 releases; waiting 497 ms; commit 0 ms)
2026-11-08T05:02:25Z [load_day 2024-03-09] 2024-03-09 hour 10: 217566 events in 0 files, snapshot none (task 4867 ms: download 2454 ms overlapping gunzip 664 ms, parse 3042 ms, sort 0 ms, write 0 ms, index 415 ms: 8 candidates, 59793 repo-hours, 757 releases; waiting 459 ms; commit 0 ms)
2026-11-08T05:02:26Z [load_day 2024-03-09] 2024-03-09 hour 11: 218616 events in 0 files, snapshot none (task 5446 ms: download 2983 ms overlapping gunzip 765 ms, parse 3507 ms, sort 0 ms, write 0 ms, index 420 ms: 6 candidates, 58848 repo-hours, 1133 releases; waiting 475 ms; commit 0 ms)
2026-11-08T05:02:33Z [load_day 2024-03-09] 2024-03-09 hour 12: 297877 events in 0 files, snapshot none (task 6878 ms: download 3613 ms overlapping gunzip 883 ms, parse 4516 ms, sort 0 ms, write 0 ms, index 699 ms: 9 candidates, 92933 repo-hours, 789 releases; waiting 492 ms; commit 0 ms)
2026-11-08T05:02:33Z [load_day 2024-03-09] 2024-03-09 hour 13: 246907 events in 0 files, snapshot none (task 6660 ms: download 3982 ms overlapping gunzip 968 ms, parse 4270 ms, sort 0 ms, write 0 ms, index 609 ms: 7 candidates, 82323 repo-hours, 644 releases; waiting 473 ms; commit 0 ms)
2026-11-08T05:02:40Z [load_day 2024-03-09] 2024-03-09 hour 14: 239757 events in 0 files, snapshot none (task 6915 ms: download 3462 ms overlapping gunzip 981 ms, parse 4515 ms, sort 0 ms, write 0 ms, index 595 ms: 3 candidates, 68792 repo-hours, 731 releases; waiting 506 ms; commit 0 ms)
2026-11-08T05:02:41Z [load_day 2024-03-09] 2024-03-09 hour 15: 246858 events in 0 files, snapshot none (task 7425 ms: download 4393 ms overlapping gunzip 1096 ms, parse 4889 ms, sort 0 ms, write 0 ms, index 621 ms: 9 candidates, 72715 repo-hours, 828 releases; waiting 509 ms; commit 0 ms)
2026-11-08T05:02:48Z [load_day 2024-03-09] 2024-03-09 hour 16: 239775 events in 0 files, snapshot none (task 6943 ms: download 3953 ms overlapping gunzip 949 ms, parse 4222 ms, sort 0 ms, write 0 ms, index 804 ms: 9 candidates, 69132 repo-hours, 773 releases; waiting 631 ms; commit 0 ms)
2026-11-08T05:02:48Z [load_day 2024-03-09] 2024-03-09 hour 17: 235957 events in 0 files, snapshot none (task 6105 ms: download 3325 ms overlapping gunzip 823 ms, parse 3955 ms, sort 0 ms, write 0 ms, index 590 ms: 13 candidates, 64125 repo-hours, 766 releases; waiting 512 ms; commit 0 ms)
2026-11-08T05:02:55Z [load_day 2024-03-09] 2024-03-09 hour 18: 242260 events in 0 files, snapshot none (task 6693 ms: download 3453 ms overlapping gunzip 922 ms, parse 4238 ms, sort 0 ms, write 0 ms, index 750 ms: 9 candidates, 71703 repo-hours, 891 releases; waiting 502 ms; commit 0 ms)
2026-11-08T05:02:55Z [load_day 2024-03-09] 2024-03-09 hour 19: 219490 events in 0 files, snapshot none (task 6644 ms: download 3218 ms overlapping gunzip 1014 ms, parse 4325 ms, sort 0 ms, write 0 ms, index 556 ms: 12 candidates, 57700 repo-hours, 758 releases; waiting 494 ms; commit 0 ms)
2026-11-08T05:03:00Z [load_day 2024-03-09] 2024-03-09 hour 20: 217151 events in 0 files, snapshot none (task 4663 ms: download 2329 ms overlapping gunzip 622 ms, parse 2835 ms, sort 0 ms, write 0 ms, index 402 ms: 5 candidates, 57233 repo-hours, 661 releases; waiting 498 ms; commit 0 ms)
2026-11-08T05:03:00Z [load_day 2024-03-09] 2024-03-09 hour 21: 209713 events in 0 files, snapshot none (task 4527 ms: download 3372 ms overlapping gunzip 618 ms, parse 2801 ms, sort 0 ms, write 0 ms, index 374 ms: 3 candidates, 52256 repo-hours, 751 releases; waiting 477 ms; commit 0 ms)
2026-11-08T05:03:05Z [load_day 2024-03-09] 2024-03-09 hour 22: 200814 events in 0 files, snapshot none (task 4450 ms: download 2494 ms overlapping gunzip 587 ms, parse 2683 ms, sort 0 ms, write 0 ms, index 330 ms: 11 candidates, 48532 repo-hours, 789 releases; waiting 550 ms; commit 0 ms)
2026-11-08T05:03:05Z [load_day 2024-03-09] 2024-03-09 hour 23: 193069 events in 0 files, snapshot none (task 4204 ms: download 2251 ms overlapping gunzip 550 ms, parse 2490 ms, sort 0 ms, write 0 ms, index 332 ms: 3 candidates, 47305 repo-hours, 628 releases; waiting 525 ms; commit 0 ms)
2026-11-08T05:03:11Z [load_day 2024-03-09] 2024-03-09: repo_hour merged from 24 files to 1 (1490237 rows, 6248 ms)
2026-11-08T05:03:13Z [load_day 2024-03-09] 2024-03-09: index committed in 8441 ms (24 + 1 + 24 files)
2026-11-08T05:03:13Z [load_day 2024-03-09] 2024-03-09: 5308746 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T05:03:13Z [load_day 2024-03-09] attempt 1 succeeded