load_day 2023-12-30, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:14095

Started

2026-11-08T03:56:57Z

Ended

2026-11-08T03:58:06Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

562 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

92.3 s, up to 2.8 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1913/load_day/2023-12-30/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T03:56:57Z [load_day 2023-12-30] attempt 1 started
2026-11-08T03:56:58Z [load_day 2023-12-30] 2023-12-30: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T03:57:04Z [load_day 2023-12-30] 2023-12-30 hour 0: 162939 events in 0 files, snapshot none (task 5581 ms: download 2442 ms overlapping gunzip 832 ms, parse 3358 ms, sort 0 ms, write 0 ms, index 586 ms: 6 candidates, 71759 repo-hours, 849 releases; waiting 538 ms; commit 0 ms)
2026-11-08T03:57:04Z [load_day 2023-12-30] 2023-12-30 hour 1: 160260 events in 0 files, snapshot none (task 5370 ms: download 3033 ms overlapping gunzip 753 ms, parse 3226 ms, sort 0 ms, write 0 ms, index 611 ms: 5 candidates, 72194 repo-hours, 1479 releases; waiting 520 ms; commit 0 ms)
2026-11-08T03:57:08Z [load_day 2023-12-30] 2023-12-30 hour 2: 129741 events in 0 files, snapshot none (task 4016 ms: download 2035 ms overlapping gunzip 540 ms, parse 2358 ms, sort 0 ms, write 0 ms, index 361 ms: 6 candidates, 39397 repo-hours, 621 releases; waiting 486 ms; commit 0 ms)
2026-11-08T03:57:08Z [load_day 2023-12-30] 2023-12-30 hour 3: 128083 events in 0 files, snapshot none (task 4017 ms: download 1883 ms overlapping gunzip 559 ms, parse 2285 ms, sort 0 ms, write 0 ms, index 375 ms: 5 candidates, 39407 repo-hours, 710 releases; waiting 513 ms; commit 0 ms)
2026-11-08T03:57:13Z [load_day 2023-12-30] 2023-12-30 hour 4: 129149 events in 0 files, snapshot none (task 4307 ms: download 1809 ms overlapping gunzip 534 ms, parse 2590 ms, sort 0 ms, write 0 ms, index 374 ms: 2 candidates, 39811 repo-hours, 557 releases; waiting 508 ms; commit 0 ms)
2026-11-08T03:57:13Z [load_day 2023-12-30] 2023-12-30 hour 5: 133226 events in 0 files, snapshot none (task 4249 ms: download 3045 ms overlapping gunzip 541 ms, parse 2511 ms, sort 0 ms, write 0 ms, index 367 ms: 1 candidates, 42736 repo-hours, 519 releases; waiting 483 ms; commit 0 ms)
2026-11-08T03:57:18Z [load_day 2023-12-30] 2023-12-30 hour 6: 146623 events in 0 files, snapshot none (task 4502 ms: download 2113 ms overlapping gunzip 576 ms, parse 2703 ms, sort 0 ms, write 0 ms, index 511 ms: 1 candidates, 50039 repo-hours, 910 releases; waiting 514 ms; commit 0 ms)
2026-11-08T03:57:18Z [load_day 2023-12-30] 2023-12-30 hour 7: 141293 events in 0 files, snapshot none (task 4023 ms: download 2194 ms overlapping gunzip 452 ms, parse 2202 ms, sort 0 ms, write 0 ms, index 576 ms: 2 candidates, 43112 repo-hours, 885 releases; waiting 524 ms; commit 0 ms)
2026-11-08T03:57:22Z [load_day 2023-12-30] 2023-12-30 hour 8: 143971 events in 0 files, snapshot none (task 3845 ms: download 1958 ms overlapping gunzip 486 ms, parse 2260 ms, sort 0 ms, write 0 ms, index 371 ms: 3 candidates, 46304 repo-hours, 810 releases; waiting 477 ms; commit 0 ms)
2026-11-08T03:57:22Z [load_day 2023-12-30] 2023-12-30 hour 9: 145120 events in 0 files, snapshot none (task 3986 ms: download 1827 ms overlapping gunzip 542 ms, parse 2370 ms, sort 0 ms, write 0 ms, index 369 ms: 4 candidates, 46370 repo-hours, 916 releases; waiting 498 ms; commit 0 ms)
2026-11-08T03:57:27Z [load_day 2023-12-30] 2023-12-30 hour 10: 151977 events in 0 files, snapshot none (task 4363 ms: download 2445 ms overlapping gunzip 619 ms, parse 2649 ms, sort 0 ms, write 0 ms, index 368 ms: 6 candidates, 49005 repo-hours, 646 releases; waiting 505 ms; commit 0 ms)
2026-11-08T03:57:27Z [load_day 2023-12-30] 2023-12-30 hour 11: 149905 events in 0 files, snapshot none (task 4153 ms: download 2246 ms overlapping gunzip 575 ms, parse 2448 ms, sort 0 ms, write 0 ms, index 358 ms: 10 candidates, 47684 repo-hours, 724 releases; waiting 483 ms; commit 0 ms)
2026-11-08T03:57:33Z [load_day 2023-12-30] 2023-12-30 hour 12: 220711 events in 0 files, snapshot none (task 5541 ms: download 2888 ms overlapping gunzip 705 ms, parse 3552 ms, sort 0 ms, write 0 ms, index 574 ms: 9 candidates, 81183 repo-hours, 698 releases; waiting 502 ms; commit 0 ms)
2026-11-08T03:57:33Z [load_day 2023-12-30] 2023-12-30 hour 13: 180381 events in 0 files, snapshot none (task 4888 ms: download 2484 ms overlapping gunzip 650 ms, parse 3059 ms, sort 0 ms, write 0 ms, index 618 ms: 8 candidates, 72228 repo-hours, 834 releases; waiting 366 ms; commit 0 ms)
2026-11-08T03:57:39Z [load_day 2023-12-30] 2023-12-30 hour 14: 165429 events in 0 files, snapshot none (task 5041 ms: download 2906 ms overlapping gunzip 777 ms, parse 3113 ms, sort 0 ms, write 0 ms, index 385 ms: 2 candidates, 56387 repo-hours, 786 releases; waiting 469 ms; commit 0 ms)
2026-11-08T03:57:39Z [load_day 2023-12-30] 2023-12-30 hour 15: 167959 events in 0 files, snapshot none (task 4913 ms: download 2631 ms overlapping gunzip 700 ms, parse 2971 ms, sort 0 ms, write 0 ms, index 460 ms: 5 candidates, 58684 repo-hours, 928 releases; waiting 512 ms; commit 0 ms)
2026-11-08T03:57:43Z [load_day 2023-12-30] 2023-12-30 hour 16: 168852 events in 0 files, snapshot none (task 4089 ms: download 2485 ms overlapping gunzip 521 ms, parse 2339 ms, sort 0 ms, write 0 ms, index 457 ms: 10 candidates, 58107 repo-hours, 752 releases; waiting 492 ms; commit 0 ms)
2026-11-08T03:57:43Z [load_day 2023-12-30] 2023-12-30 hour 17: 162981 events in 0 files, snapshot none (task 3959 ms: download 2225 ms overlapping gunzip 513 ms, parse 2306 ms, sort 0 ms, write 0 ms, index 393 ms: 5 candidates, 54709 repo-hours, 718 releases; waiting 529 ms; commit 0 ms)
2026-11-08T03:57:48Z [load_day 2023-12-30] 2023-12-30 hour 18: 171546 events in 0 files, snapshot none (task 4312 ms: download 3306 ms overlapping gunzip 505 ms, parse 2260 ms, sort 0 ms, write 0 ms, index 419 ms: 2 candidates, 58914 repo-hours, 887 releases; waiting 779 ms; commit 0 ms)
2026-11-08T03:57:48Z [load_day 2023-12-30] 2023-12-30 hour 19: 162440 events in 0 files, snapshot none (task 3916 ms: download 2245 ms overlapping gunzip 478 ms, parse 2103 ms, sort 0 ms, write 0 ms, index 364 ms: 5 candidates, 46125 repo-hours, 764 releases; waiting 695 ms; commit 0 ms)
2026-11-08T03:57:53Z [load_day 2023-12-30] 2023-12-30 hour 20: 153233 events in 0 files, snapshot none (task 4545 ms: download 2253 ms overlapping gunzip 598 ms, parse 2713 ms, sort 0 ms, write 0 ms, index 468 ms: 3 candidates, 46227 repo-hours, 636 releases; waiting 501 ms; commit 0 ms)
2026-11-08T03:57:53Z [load_day 2023-12-30] 2023-12-30 hour 21: 135905 events in 0 files, snapshot none (task 4546 ms: download 2169 ms overlapping gunzip 647 ms, parse 2798 ms, sort 0 ms, write 0 ms, index 337 ms: 4 candidates, 41814 repo-hours, 744 releases; waiting 505 ms; commit 0 ms)
2026-11-08T03:57:57Z [load_day 2023-12-30] 2023-12-30 hour 22: 135733 events in 0 files, snapshot none (task 4084 ms: download 2145 ms overlapping gunzip 525 ms, parse 2423 ms, sort 0 ms, write 0 ms, index 293 ms: 4 candidates, 39594 repo-hours, 712 releases; waiting 550 ms; commit 0 ms)
2026-11-08T03:57:57Z [load_day 2023-12-30] 2023-12-30 hour 23: 122931 events in 0 files, snapshot none (task 3619 ms: download 1994 ms overlapping gunzip 450 ms, parse 2052 ms, sort 0 ms, write 0 ms, index 399 ms: 4 candidates, 38431 repo-hours, 625 releases; waiting 503 ms; commit 0 ms)
2026-11-08T03:58:03Z [load_day 2023-12-30] 2023-12-30: repo_hour merged from 24 files to 1 (1240221 rows, 5870 ms)
2026-11-08T03:58:06Z [load_day 2023-12-30] 2023-12-30: index committed in 8054 ms (24 + 1 + 24 files)
2026-11-08T03:58:06Z [load_day 2023-12-30] 2023-12-30: 3670388 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T03:58:06Z [load_day 2023-12-30] attempt 1 succeeded