orcaLog in

Runs · run 1834

load_day 2023-12-09, attempt 1 succeeded

Host
73ccef5e07c2:7782
Started
2026-11-08T03:41:32Z
Ended
2026-11-08T03:42:51Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
567 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
111.4 s, up to 3.1 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/1834/load_day/2023-12-09/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T03:41:32Z [load_day 2023-12-09] attempt 1 started
2026-11-08T03:41:33Z [load_day 2023-12-09] 2023-12-09: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T03:41:40Z [load_day 2023-12-09] 2023-12-09 hour 0: 205523 events in 0 files, snapshot none (task 5650 ms: download 3157 ms overlapping gunzip 783 ms, parse 3409 ms, sort 0 ms, write 0 ms, index 554 ms: 5 candidates, 79105 repo-hours, 983 releases; waiting 637 ms; commit 0 ms)
2026-11-08T03:41:40Z [load_day 2023-12-09] 2023-12-09 hour 1: 196547 events in 0 files, snapshot none (task 5217 ms: download 2554 ms overlapping gunzip 696 ms, parse 3134 ms, sort 0 ms, write 0 ms, index 579 ms: 5 candidates, 79501 repo-hours, 781 releases; waiting 483 ms; commit 0 ms)
2026-11-08T03:41:45Z [load_day 2023-12-09] 2023-12-09 hour 2: 161453 events in 0 files, snapshot none (task 4261 ms: download 2230 ms overlapping gunzip 589 ms, parse 2512 ms, sort 0 ms, write 0 ms, index 366 ms: 7 candidates, 44775 repo-hours, 584 releases; waiting 495 ms; commit 0 ms)
2026-11-08T03:41:45Z [load_day 2023-12-09] 2023-12-09 hour 3: 148715 events in 0 files, snapshot none (task 4063 ms: download 2105 ms overlapping gunzip 544 ms, parse 2378 ms, sort 0 ms, write 0 ms, index 365 ms: 11 candidates, 44865 repo-hours, 746 releases; waiting 531 ms; commit 0 ms)
2026-11-08T03:41:49Z [load_day 2023-12-09] 2023-12-09 hour 4: 161303 events in 0 files, snapshot none (task 3939 ms: download 1980 ms overlapping gunzip 511 ms, parse 2331 ms, sort 0 ms, write 0 ms, index 353 ms: 11 candidates, 45120 repo-hours, 623 releases; waiting 467 ms; commit 0 ms)
2026-11-08T03:41:49Z [load_day 2023-12-09] 2023-12-09 hour 5: 173144 events in 0 files, snapshot none (task 4029 ms: download 2328 ms overlapping gunzip 538 ms, parse 2364 ms, sort 0 ms, write 0 ms, index 339 ms: 5 candidates, 48799 repo-hours, 651 releases; waiting 536 ms; commit 0 ms)
2026-11-08T03:41:54Z [load_day 2023-12-09] 2023-12-09 hour 6: 183240 events in 0 files, snapshot none (task 4488 ms: download 2026 ms overlapping gunzip 562 ms, parse 2698 ms, sort 0 ms, write 0 ms, index 432 ms: 5 candidates, 55888 repo-hours, 917 releases; waiting 538 ms; commit 0 ms)
2026-11-08T03:41:54Z [load_day 2023-12-09] 2023-12-09 hour 7: 180856 events in 0 files, snapshot none (task 4825 ms: download 2475 ms overlapping gunzip 630 ms, parse 2865 ms, sort 0 ms, write 0 ms, index 359 ms: 3 candidates, 50271 repo-hours, 900 releases; waiting 659 ms; commit 0 ms)
2026-11-08T03:42:00Z [load_day 2023-12-09] 2023-12-09 hour 8: 172488 events in 0 files, snapshot none (task 4729 ms: download 2387 ms overlapping gunzip 632 ms, parse 2861 ms, sort 0 ms, write 0 ms, index 396 ms: 2 candidates, 52540 repo-hours, 841 releases; waiting 543 ms; commit 0 ms)
2026-11-08T03:42:00Z [load_day 2023-12-09] 2023-12-09 hour 9: 184338 events in 0 files, snapshot none (task 4553 ms: download 2319 ms overlapping gunzip 627 ms, parse 2758 ms, sort 0 ms, write 0 ms, index 385 ms: 4 candidates, 53683 repo-hours, 916 releases; waiting 520 ms; commit 0 ms)
2026-11-08T03:42:05Z [load_day 2023-12-09] 2023-12-09 hour 10: 197356 events in 0 files, snapshot none (task 5359 ms: download 2877 ms overlapping gunzip 745 ms, parse 3411 ms, sort 0 ms, write 0 ms, index 421 ms: 5 candidates, 57196 repo-hours, 614 releases; waiting 520 ms; commit 0 ms)
2026-11-08T03:42:05Z [load_day 2023-12-09] 2023-12-09 hour 11: 195381 events in 0 files, snapshot none (task 5285 ms: download 2905 ms overlapping gunzip 830 ms, parse 3292 ms, sort 0 ms, write 0 ms, index 383 ms: 7 candidates, 53812 repo-hours, 629 releases; waiting 550 ms; commit 0 ms)
2026-11-08T03:42:12Z [load_day 2023-12-09] 2023-12-09 hour 12: 267143 events in 0 files, snapshot none (task 6573 ms: download 3286 ms overlapping gunzip 873 ms, parse 3971 ms, sort 0 ms, write 0 ms, index 924 ms: 10 candidates, 86593 repo-hours, 738 releases; waiting 510 ms; commit 0 ms)
2026-11-08T03:42:12Z [load_day 2023-12-09] 2023-12-09 hour 13: 240790 events in 0 files, snapshot none (task 6470 ms: download 3439 ms overlapping gunzip 838 ms, parse 4051 ms, sort 0 ms, write 0 ms, index 822 ms: 9 candidates, 78975 repo-hours, 650 releases; waiting 498 ms; commit 0 ms)
2026-11-08T03:42:21Z [load_day 2023-12-09] 2023-12-09 hour 14: 230647 events in 0 files, snapshot none (task 7548 ms: download 4455 ms overlapping gunzip 985 ms, parse 4573 ms, sort 0 ms, write 0 ms, index 685 ms: 5 candidates, 63680 repo-hours, 632 releases; waiting 979 ms; commit 0 ms)
2026-11-08T03:42:21Z [load_day 2023-12-09] 2023-12-09 hour 15: 167025 events in 0 files, snapshot none (task 6225 ms: download 2384 ms overlapping gunzip 868 ms, parse 3808 ms, sort 0 ms, write 0 ms, index 748 ms: 8 candidates, 68772 repo-hours, 965 releases; waiting 493 ms; commit 0 ms)
2026-11-08T03:42:27Z [load_day 2023-12-09] 2023-12-09 hour 16: 166736 events in 0 files, snapshot none (task 5816 ms: download 3156 ms overlapping gunzip 827 ms, parse 3699 ms, sort 0 ms, write 0 ms, index 481 ms: 9 candidates, 67780 repo-hours, 839 releases; waiting 492 ms; commit 0 ms)
2026-11-08T03:42:27Z [load_day 2023-12-09] 2023-12-09 hour 17: 160201 events in 0 files, snapshot none (task 5558 ms: download 2792 ms overlapping gunzip 743 ms, parse 3370 ms, sort 0 ms, write 0 ms, index 684 ms: 9 candidates, 66010 repo-hours, 1021 releases; waiting 484 ms; commit 0 ms)
2026-11-08T03:42:33Z [load_day 2023-12-09] 2023-12-09 hour 18: 160571 events in 0 files, snapshot none (task 5330 ms: download 2845 ms overlapping gunzip 792 ms, parse 3293 ms, sort 0 ms, write 0 ms, index 502 ms: 11 candidates, 68040 repo-hours, 899 releases; waiting 495 ms; commit 0 ms)
2026-11-08T03:42:33Z [load_day 2023-12-09] 2023-12-09 hour 19: 132334 events in 0 files, snapshot none (task 5169 ms: download 2682 ms overlapping gunzip 750 ms, parse 3190 ms, sort 0 ms, write 0 ms, index 483 ms: 8 candidates, 55204 repo-hours, 828 releases; waiting 465 ms; commit 0 ms)
2026-11-08T03:42:38Z [load_day 2023-12-09] 2023-12-09 hour 20: 127259 events in 0 files, snapshot none (task 4623 ms: download 2606 ms overlapping gunzip 667 ms, parse 2781 ms, sort 0 ms, write 0 ms, index 386 ms: 6 candidates, 54089 repo-hours, 686 releases; waiting 516 ms; commit 0 ms)
2026-11-08T03:42:38Z [load_day 2023-12-09] 2023-12-09 hour 21: 122595 events in 0 files, snapshot none (task 4605 ms: download 2563 ms overlapping gunzip 641 ms, parse 2722 ms, sort 0 ms, write 0 ms, index 360 ms: 9 candidates, 49858 repo-hours, 777 releases; waiting 508 ms; commit 0 ms)
2026-11-08T03:42:43Z [load_day 2023-12-09] 2023-12-09 hour 22: 113975 events in 0 files, snapshot none (task 3844 ms: download 1954 ms overlapping gunzip 488 ms, parse 2151 ms, sort 0 ms, write 0 ms, index 434 ms: 10 candidates, 47186 repo-hours, 740 releases; waiting 522 ms; commit 0 ms)
2026-11-08T03:42:43Z [load_day 2023-12-09] 2023-12-09 hour 23: 106124 events in 0 files, snapshot none (task 3479 ms: download 2125 ms overlapping gunzip 413 ms, parse 1819 ms, sort 0 ms, write 0 ms, index 436 ms: 6 candidates, 45181 repo-hours, 605 releases; waiting 516 ms; commit 0 ms)
2026-11-08T03:42:49Z [load_day 2023-12-09] 2023-12-09: repo_hour merged from 24 files to 1 (1416923 rows, 6537 ms)
2026-11-08T03:42:51Z [load_day 2023-12-09] 2023-12-09: index committed in 8832 ms (24 + 1 + 24 files)
2026-11-08T03:42:51Z [load_day 2023-12-09] 2023-12-09: 4155744 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T03:42:51Z [load_day 2023-12-09] attempt 1 succeeded