orcaLog in

Runs · run 2687

load_day 2024-07-07, attempt 1 succeeded

Host
73ccef5e07c2:146619
Started
2026-11-08T06:53:07Z
Ended
2026-11-08T06:54:26Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
665 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
107.6 s, up to 2.9 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/2687/load_day/2024-07-07/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T06:53:07Z [load_day 2024-07-07] attempt 1 started
2026-11-08T06:53:08Z [load_day 2024-07-07] 2024-07-07: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T06:53:12Z [load_day 2024-07-07] 2024-07-07 hour 0: 130248 events in 0 files, snapshot none (task 3673 ms: download 2119 ms overlapping gunzip 459 ms, parse 2125 ms, sort 0 ms, write 0 ms, index 519 ms: 3 candidates, 72888 repo-hours, 923 releases; waiting 243 ms; commit 0 ms)
2026-11-08T06:53:12Z [load_day 2024-07-07] 2024-07-07 hour 1: 152955 events in 0 files, snapshot none (task 4164 ms: download 2284 ms overlapping gunzip 482 ms, parse 2261 ms, sort 0 ms, write 0 ms, index 607 ms: 8 candidates, 90699 repo-hours, 844 releases; waiting 488 ms; commit 0 ms)
2026-11-08T06:53:15Z [load_day 2024-07-07] 2024-07-07 hour 2: 114067 events in 0 files, snapshot none (task 3217 ms: download 1599 ms overlapping gunzip 383 ms, parse 1705 ms, sort 0 ms, write 0 ms, index 314 ms: 6 candidates, 46129 repo-hours, 686 releases; waiting 489 ms; commit 0 ms)
2026-11-08T06:53:16Z [load_day 2024-07-07] 2024-07-07 hour 3: 139499 events in 0 files, snapshot none (task 3435 ms: download 1690 ms overlapping gunzip 421 ms, parse 1995 ms, sort 0 ms, write 0 ms, index 423 ms: 3 candidates, 56321 repo-hours, 5290 releases; waiting 295 ms; commit 0 ms)
2026-11-08T06:53:20Z [load_day 2024-07-07] 2024-07-07 hour 4: 119037 events in 0 files, snapshot none (task 4161 ms: download 2110 ms overlapping gunzip 524 ms, parse 2379 ms, sort 0 ms, write 0 ms, index 569 ms: 1 candidates, 44349 repo-hours, 496 releases; waiting 440 ms; commit 0 ms)
2026-11-08T06:53:21Z [load_day 2024-07-07] 2024-07-07 hour 5: 128335 events in 0 files, snapshot none (task 4447 ms: download 2255 ms overlapping gunzip 585 ms, parse 2807 ms, sort 0 ms, write 0 ms, index 340 ms: 1 candidates, 48097 repo-hours, 605 releases; waiting 370 ms; commit 0 ms)
2026-11-08T06:53:25Z [load_day 2024-07-07] 2024-07-07 hour 6: 151927 events in 0 files, snapshot none (task 4514 ms: download 2667 ms overlapping gunzip 519 ms, parse 2343 ms, sort 0 ms, write 0 ms, index 568 ms: 4 candidates, 67957 repo-hours, 679 releases; waiting 682 ms; commit 0 ms)
2026-11-08T06:53:26Z [load_day 2024-07-07] 2024-07-07 hour 7: 141447 events in 0 files, snapshot none (task 3766 ms: download 2134 ms overlapping gunzip 480 ms, parse 2142 ms, sort 0 ms, write 0 ms, index 396 ms: 0 candidates, 54469 repo-hours, 567 releases; waiting 488 ms; commit 0 ms)
2026-11-08T06:53:31Z [load_day 2024-07-07] 2024-07-07 hour 8: 150310 events in 0 files, snapshot none (task 5013 ms: download 2365 ms overlapping gunzip 686 ms, parse 3001 ms, sort 0 ms, write 0 ms, index 499 ms: 7 candidates, 59136 repo-hours, 647 releases; waiting 519 ms; commit 0 ms)
2026-11-08T06:53:31Z [load_day 2024-07-07] 2024-07-07 hour 9: 154426 events in 0 files, snapshot none (task 4784 ms: download 2348 ms overlapping gunzip 653 ms, parse 2804 ms, sort 0 ms, write 0 ms, index 519 ms: 7 candidates, 65863 repo-hours, 869 releases; waiting 449 ms; commit 0 ms)
2026-11-08T06:53:36Z [load_day 2024-07-07] 2024-07-07 hour 10: 150102 events in 0 files, snapshot none (task 5333 ms: download 2765 ms overlapping gunzip 762 ms, parse 3398 ms, sort 0 ms, write 0 ms, index 510 ms: 3 candidates, 61076 repo-hours, 735 releases; waiting 384 ms; commit 0 ms)
2026-11-08T06:53:37Z [load_day 2024-07-07] 2024-07-07 hour 11: 154094 events in 0 files, snapshot none (task 5577 ms: download 2926 ms overlapping gunzip 810 ms, parse 3494 ms, sort 0 ms, write 0 ms, index 480 ms: 15 candidates, 61139 repo-hours, 802 releases; waiting 494 ms; commit 0 ms)
2026-11-08T06:53:45Z [load_day 2024-07-07] 2024-07-07 hour 12: 236914 events in 0 files, snapshot none (task 7723 ms: download 2548 ms overlapping gunzip 948 ms, parse 4926 ms, sort 0 ms, write 0 ms, index 1099 ms: 9 candidates, 113001 repo-hours, 982 releases; waiting 492 ms; commit 0 ms)
2026-11-08T06:53:45Z [load_day 2024-07-07] 2024-07-07 hour 13: 187681 events in 0 files, snapshot none (task 7034 ms: download 3469 ms overlapping gunzip 938 ms, parse 4203 ms, sort 0 ms, write 0 ms, index 1000 ms: 12 candidates, 89459 repo-hours, 846 releases; waiting 639 ms; commit 0 ms)
2026-11-08T06:53:51Z [load_day 2024-07-07] 2024-07-07 hour 14: 191571 events in 0 files, snapshot none (task 6181 ms: download 3327 ms overlapping gunzip 721 ms, parse 3195 ms, sort 0 ms, write 0 ms, index 763 ms: 10 candidates, 92353 repo-hours, 816 releases; waiting 1254 ms; commit 0 ms)
2026-11-08T06:53:51Z [load_day 2024-07-07] 2024-07-07 hour 15: 191067 events in 0 files, snapshot none (task 5790 ms: download 3085 ms overlapping gunzip 788 ms, parse 3512 ms, sort 0 ms, write 0 ms, index 738 ms: 13 candidates, 86977 repo-hours, 972 releases; waiting 484 ms; commit 0 ms)
2026-11-08T06:53:59Z [load_day 2024-07-07] 2024-07-07 hour 16: 178836 events in 0 files, snapshot none (task 6683 ms: download 3453 ms overlapping gunzip 1047 ms, parse 4514 ms, sort 0 ms, write 0 ms, index 627 ms: 6 candidates, 75410 repo-hours, 793 releases; waiting 278 ms; commit 0 ms)
2026-11-08T06:53:59Z [load_day 2024-07-07] 2024-07-07 hour 17: 166467 events in 0 files, snapshot none (task 6347 ms: download 3045 ms overlapping gunzip 908 ms, parse 4020 ms, sort 0 ms, write 0 ms, index 742 ms: 5 candidates, 66258 repo-hours, 729 releases; waiting 486 ms; commit 0 ms)
2026-11-08T06:54:05Z [load_day 2024-07-07] 2024-07-07 hour 18: 180328 events in 0 files, snapshot none (task 5637 ms: download 2437 ms overlapping gunzip 730 ms, parse 3192 ms, sort 0 ms, write 0 ms, index 725 ms: 10 candidates, 83338 repo-hours, 894 releases; waiting 696 ms; commit 0 ms)
2026-11-08T06:54:05Z [load_day 2024-07-07] 2024-07-07 hour 19: 164188 events in 0 files, snapshot none (task 5445 ms: download 3592 ms overlapping gunzip 722 ms, parse 3046 ms, sort 0 ms, write 0 ms, index 527 ms: 11 candidates, 72685 repo-hours, 737 releases; waiting 798 ms; commit 0 ms)
2026-11-08T06:54:10Z [load_day 2024-07-07] 2024-07-07 hour 20: 151821 events in 0 files, snapshot none (task 5117 ms: download 2841 ms overlapping gunzip 747 ms, parse 3120 ms, sort 0 ms, write 0 ms, index 458 ms: 8 candidates, 60560 repo-hours, 908 releases; waiting 508 ms; commit 0 ms)
2026-11-08T06:54:10Z [load_day 2024-07-07] 2024-07-07 hour 21: 148057 events in 0 files, snapshot none (task 4826 ms: download 2472 ms overlapping gunzip 688 ms, parse 2994 ms, sort 0 ms, write 0 ms, index 418 ms: 8 candidates, 60623 repo-hours, 1012 releases; waiting 464 ms; commit 0 ms)
2026-11-08T06:54:16Z [load_day 2024-07-07] 2024-07-07 hour 22: 170204 events in 0 files, snapshot none (task 5295 ms: download 2564 ms overlapping gunzip 754 ms, parse 3110 ms, sort 0 ms, write 0 ms, index 691 ms: 3 candidates, 90568 repo-hours, 831 releases; waiting 476 ms; commit 0 ms)
2026-11-08T06:54:16Z [load_day 2024-07-07] 2024-07-07 hour 23: 126991 events in 0 files, snapshot none (task 4943 ms: download 2542 ms overlapping gunzip 673 ms, parse 3059 ms, sort 0 ms, write 0 ms, index 417 ms: 3 candidates, 46415 repo-hours, 653 releases; waiting 532 ms; commit 0 ms)
2026-11-08T06:54:23Z [load_day 2024-07-07] 2024-07-07: repo_hour merged from 24 files to 1 (1665770 rows, 6937 ms)
2026-11-08T06:54:26Z [load_day 2024-07-07] 2024-07-07: index committed in 9738 ms (23 + 1 + 24 files)
2026-11-08T06:54:26Z [load_day 2024-07-07] 2024-07-07: 3780572 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T06:54:26Z [load_day 2024-07-07] attempt 1 succeeded