load_day 2024-01-06, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:16356

Started

2026-11-08T04:02:08Z

Ended

2026-11-08T04:03:20Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

569 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

99.0 s, up to 2.9 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1940/load_day/2024-01-06/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:02:08Z [load_day 2024-01-06] attempt 1 started
2026-11-08T04:02:09Z [load_day 2024-01-06] 2024-01-06: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:02:16Z [load_day 2024-01-06] 2024-01-06 hour 0: 157043 events in 0 files, snapshot none (task 6967 ms: download 4053 ms overlapping gunzip 1029 ms, parse 4343 ms, sort 0 ms, write 0 ms, index 638 ms: 3 candidates, 79334 repo-hours, 1038 releases; waiting 692 ms; commit 0 ms)
2026-11-08T04:02:16Z [load_day 2024-01-06] 2024-01-06 hour 1: 141968 events in 0 files, snapshot none (task 5691 ms: download 2755 ms overlapping gunzip 814 ms, parse 3421 ms, sort 0 ms, write 0 ms, index 652 ms: 4 candidates, 74841 repo-hours, 805 releases; waiting 520 ms; commit 0 ms)
2026-11-08T04:02:21Z [load_day 2024-01-06] 2024-01-06 hour 2: 114785 events in 0 files, snapshot none (task 4608 ms: download 2564 ms overlapping gunzip 662 ms, parse 2648 ms, sort 0 ms, write 0 ms, index 456 ms: 1 candidates, 44466 repo-hours, 662 releases; waiting 533 ms; commit 0 ms)
2026-11-08T04:02:21Z [load_day 2024-01-06] 2024-01-06 hour 3: 116162 events in 0 files, snapshot none (task 4078 ms: download 2501 ms overlapping gunzip 531 ms, parse 2163 ms, sort 0 ms, write 0 ms, index 524 ms: 3 candidates, 43644 repo-hours, 741 releases; waiting 552 ms; commit 0 ms)
2026-11-08T04:02:26Z [load_day 2024-01-06] 2024-01-06 hour 4: 115620 events in 0 files, snapshot none (task 3958 ms: download 2116 ms overlapping gunzip 474 ms, parse 2087 ms, sort 0 ms, write 0 ms, index 338 ms: 3 candidates, 43841 repo-hours, 486 releases; waiting 494 ms; commit 0 ms)
2026-11-08T04:02:26Z [load_day 2024-01-06] 2024-01-06 hour 5: 126474 events in 0 files, snapshot none (task 4664 ms: download 2292 ms overlapping gunzip 673 ms, parse 2607 ms, sort 0 ms, write 0 ms, index 362 ms: 5 candidates, 47612 repo-hours, 582 releases; waiting 468 ms; commit 0 ms)
2026-11-08T04:02:31Z [load_day 2024-01-06] 2024-01-06 hour 6: 132407 events in 0 files, snapshot none (task 4431 ms: download 2166 ms overlapping gunzip 569 ms, parse 2652 ms, sort 0 ms, write 0 ms, index 443 ms: 1 candidates, 54537 repo-hours, 879 releases; waiting 468 ms; commit 0 ms)
2026-11-08T04:02:31Z [load_day 2024-01-06] 2024-01-06 hour 7: 129385 events in 0 files, snapshot none (task 4452 ms: download 2146 ms overlapping gunzip 698 ms, parse 2825 ms, sort 0 ms, write 0 ms, index 363 ms: 4 candidates, 49087 repo-hours, 759 releases; waiting 204 ms; commit 0 ms)
2026-11-08T04:02:35Z [load_day 2024-01-06] 2024-01-06 hour 8: 128024 events in 0 files, snapshot none (task 4253 ms: download 1779 ms overlapping gunzip 542 ms, parse 2361 ms, sort 0 ms, write 0 ms, index 546 ms: 8 candidates, 51372 repo-hours, 764 releases; waiting 493 ms; commit 0 ms)
2026-11-08T04:02:36Z [load_day 2024-01-06] 2024-01-06 hour 9: 132479 events in 0 files, snapshot none (task 4655 ms: download 2603 ms overlapping gunzip 658 ms, parse 2600 ms, sort 0 ms, write 0 ms, index 387 ms: 3 candidates, 53163 repo-hours, 850 releases; waiting 674 ms; commit 0 ms)
2026-11-08T04:02:40Z [load_day 2024-01-06] 2024-01-06 hour 10: 137226 events in 0 files, snapshot none (task 3962 ms: download 2421 ms overlapping gunzip 483 ms, parse 2033 ms, sort 0 ms, write 0 ms, index 426 ms: 9 candidates, 56654 repo-hours, 759 releases; waiting 597 ms; commit 0 ms)
2026-11-08T04:02:41Z [load_day 2024-01-06] 2024-01-06 hour 11: 137706 events in 0 files, snapshot none (task 3670 ms: download 1857 ms overlapping gunzip 468 ms, parse 2013 ms, sort 0 ms, write 0 ms, index 388 ms: 3 candidates, 55116 repo-hours, 716 releases; waiting 519 ms; commit 0 ms)
2026-11-08T04:02:45Z [load_day 2024-01-06] 2024-01-06 hour 12: 198871 events in 0 files, snapshot none (task 4631 ms: download 2599 ms overlapping gunzip 562 ms, parse 2605 ms, sort 0 ms, write 0 ms, index 628 ms: 8 candidates, 88382 repo-hours, 731 releases; waiting 491 ms; commit 0 ms)
2026-11-08T04:02:46Z [load_day 2024-01-06] 2024-01-06 hour 13: 169715 events in 0 files, snapshot none (task 4781 ms: download 2396 ms overlapping gunzip 625 ms, parse 2759 ms, sort 0 ms, write 0 ms, index 601 ms: 9 candidates, 79800 repo-hours, 898 releases; waiting 517 ms; commit 0 ms)
2026-11-08T04:02:51Z [load_day 2024-01-06] 2024-01-06 hour 14: 158578 events in 0 files, snapshot none (task 5132 ms: download 3757 ms overlapping gunzip 583 ms, parse 2479 ms, sort 0 ms, write 0 ms, index 564 ms: 3 candidates, 65651 repo-hours, 705 releases; waiting 1153 ms; commit 0 ms)
2026-11-08T04:02:51Z [load_day 2024-01-06] 2024-01-06 hour 15: 162017 events in 0 files, snapshot none (task 4351 ms: download 2113 ms overlapping gunzip 582 ms, parse 2522 ms, sort 0 ms, write 0 ms, index 541 ms: 4 candidates, 68915 repo-hours, 967 releases; waiting 490 ms; commit 0 ms)
2026-11-08T04:02:57Z [load_day 2024-01-06] 2024-01-06 hour 16: 166162 events in 0 files, snapshot none (task 6253 ms: download 3520 ms overlapping gunzip 895 ms, parse 3946 ms, sort 0 ms, write 0 ms, index 551 ms: 3 candidates, 67411 repo-hours, 876 releases; waiting 500 ms; commit 0 ms)
2026-11-08T04:02:57Z [load_day 2024-01-06] 2024-01-06 hour 17: 153425 events in 0 files, snapshot none (task 5307 ms: download 2950 ms overlapping gunzip 822 ms, parse 3397 ms, sort 0 ms, write 0 ms, index 453 ms: 3 candidates, 64070 repo-hours, 819 releases; waiting 445 ms; commit 0 ms)
2026-11-08T04:03:03Z [load_day 2024-01-06] 2024-01-06 hour 18: 153018 events in 0 files, snapshot none (task 5294 ms: download 2782 ms overlapping gunzip 730 ms, parse 3161 ms, sort 0 ms, write 0 ms, index 619 ms: 4 candidates, 66996 repo-hours, 833 releases; waiting 514 ms; commit 0 ms)
2026-11-08T04:03:03Z [load_day 2024-01-06] 2024-01-06 hour 19: 134208 events in 0 files, snapshot none (task 4619 ms: download 2608 ms overlapping gunzip 645 ms, parse 2737 ms, sort 0 ms, write 0 ms, index 446 ms: 7 candidates, 52857 repo-hours, 705 releases; waiting 476 ms; commit 0 ms)
2026-11-08T04:03:08Z [load_day 2024-01-06] 2024-01-06 hour 20: 131812 events in 0 files, snapshot none (task 3834 ms: download 1918 ms overlapping gunzip 463 ms, parse 2060 ms, sort 0 ms, write 0 ms, index 465 ms: 11 candidates, 53078 repo-hours, 723 releases; waiting 538 ms; commit 0 ms)
2026-11-08T04:03:08Z [load_day 2024-01-06] 2024-01-06 hour 21: 121743 events in 0 files, snapshot none (task 4137 ms: download 1816 ms overlapping gunzip 515 ms, parse 2233 ms, sort 0 ms, write 0 ms, index 627 ms: 8 candidates, 47533 repo-hours, 874 releases; waiting 447 ms; commit 0 ms)
2026-11-08T04:03:12Z [load_day 2024-01-06] 2024-01-06 hour 22: 113949 events in 0 files, snapshot none (task 3953 ms: download 1766 ms overlapping gunzip 441 ms, parse 1955 ms, sort 0 ms, write 0 ms, index 702 ms: 3 candidates, 44940 repo-hours, 825 releases; waiting 514 ms; commit 0 ms)
2026-11-08T04:03:12Z [load_day 2024-01-06] 2024-01-06 hour 23: 114752 events in 0 files, snapshot none (task 3808 ms: download 2052 ms overlapping gunzip 451 ms, parse 1931 ms, sort 0 ms, write 0 ms, index 398 ms: 4 candidates, 43216 repo-hours, 630 releases; waiting 534 ms; commit 0 ms)
2026-11-08T04:03:18Z [load_day 2024-01-06] 2024-01-06: repo_hour merged from 24 files to 1 (1396516 rows, 6259 ms)
2026-11-08T04:03:20Z [load_day 2024-01-06] 2024-01-06: index committed in 8402 ms (24 + 1 + 24 files)
2026-11-08T04:03:20Z [load_day 2024-01-06] 2024-01-06: 3347529 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:03:20Z [load_day 2024-01-06] attempt 1 succeeded