orcaLog in

Runs · run 1571

load_day 2023-09-30, attempt 1 succeeded

Host
3c2b29713ce8:386954
Started
2026-11-08T01:20:04Z
Ended
2026-11-08T01:22:33Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1571/load_day/2023-09-30/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:20:04Z [load_day 2023-09-30] attempt 1 started
2026-11-08T01:20:06Z [load_day 2023-09-30] 2023-09-30: 2 hours in flight, about 1200 MB
2026-11-08T01:20:21Z [load_day 2023-09-30] 2023-09-30 hour 0: 138098 events in 3 files, snapshot 6940373854658116633 (task 10837 ms: download 2537 ms overlapping gunzip 650 ms, parse 3068 ms, sort 99 ms, write 5743 ms, index 478 ms: 10 candidates, 75402 repo-hours, 933 releases; waiting 488 ms; commit 2366 ms)
2026-11-08T01:20:25Z [load_day 2023-09-30] 2023-09-30 hour 1: 130102 events in 3 files, snapshot 2026988990032662227 (task 9954 ms: download 1529 ms overlapping gunzip 501 ms, parse 2280 ms, sort 113 ms, write 6057 ms, index 566 ms: 6 candidates, 72082 repo-hours, 626 releases; waiting 184 ms; commit 4794 ms)
2026-11-08T01:20:30Z [load_day 2023-09-30] 2023-09-30 hour 2: 96520 events in 2 files, snapshot 7464380894030020565 (task 8416 ms: download 1945 ms overlapping gunzip 542 ms, parse 2172 ms, sort 109 ms, write 4333 ms, index 302 ms: 6 candidates, 39585 repo-hours, 562 releases; waiting 713 ms; commit 2499 ms)
2026-11-08T01:20:33Z [load_day 2023-09-30] 2023-09-30 hour 3: 96076 events in 2 files, snapshot 837923121345511845 (task 8647 ms: download 2084 ms overlapping gunzip 514 ms, parse 2318 ms, sort 68 ms, write 4473 ms, index 493 ms: 7 candidates, 39690 repo-hours, 656 releases; waiting 487 ms; commit 2447 ms)
2026-11-08T01:20:43Z [load_day 2023-09-30] 2023-09-30 hour 4: 100533 events in 3 files, snapshot 7744603677209437999 (task 8229 ms: download 2268 ms overlapping gunzip 477 ms, parse 2137 ms, sort 71 ms, write 4304 ms, index 388 ms: 3 candidates, 41408 repo-hours, 453 releases; waiting 484 ms; commit 5596 ms)
2026-11-08T01:20:47Z [load_day 2023-09-30] 2023-09-30 hour 5: 102302 events in 3 files, snapshot 6711580396254345205 (task 8249 ms: download 1710 ms overlapping gunzip 403 ms, parse 1924 ms, sort 114 ms, write 4557 ms, index 471 ms: 4 candidates, 43105 repo-hours, 574 releases; waiting 548 ms; commit 4126 ms)
2026-11-08T01:20:51Z [load_day 2023-09-30] 2023-09-30 hour 6: 114252 events in 3 files, snapshot 8265504299832507059 (task 8096 ms: download 2014 ms overlapping gunzip 440 ms, parse 2014 ms, sort 74 ms, write 4294 ms, index 468 ms: 2 candidates, 51993 repo-hours, 658 releases; waiting 502 ms; commit 3582 ms)
2026-11-08T01:20:56Z [load_day 2023-09-30] 2023-09-30 hour 7: 111926 events in 3 files, snapshot 2540018442852278991 (task 7888 ms: download 1710 ms overlapping gunzip 393 ms, parse 1849 ms, sort 63 ms, write 4492 ms, index 320 ms: 4 candidates, 47975 repo-hours, 564 releases; waiting 502 ms; commit 2123 ms)
2026-11-08T01:20:59Z [load_day 2023-09-30] 2023-09-30 hour 8: 115923 events in 3 files, snapshot 3731338314668618952 (task 7693 ms: download 1643 ms overlapping gunzip 439 ms, parse 1932 ms, sort 71 ms, write 4103 ms, index 373 ms: 7 candidates, 50524 repo-hours, 630 releases; waiting 535 ms; commit 2440 ms)
2026-11-08T01:21:06Z [load_day 2023-09-30] 2023-09-30 hour 9: 121260 events in 3 files, snapshot 3777159603603253802 (task 8743 ms: download 1964 ms overlapping gunzip 466 ms, parse 2078 ms, sort 82 ms, write 4409 ms, index 914 ms: 6 candidates, 51465 repo-hours, 763 releases; waiting 505 ms; commit 2155 ms)
2026-11-08T01:21:08Z [load_day 2023-09-30] 2023-09-30 hour 10: 121971 events in 3 files, snapshot 3879111189803307960 (task 8158 ms: download 2037 ms overlapping gunzip 412 ms, parse 1874 ms, sort 74 ms, write 4578 ms, index 420 ms: 4 candidates, 53375 repo-hours, 599 releases; waiting 479 ms; commit 2524 ms)
2026-11-08T01:21:17Z [load_day 2023-09-30] 2023-09-30 hour 11: 119068 events in 3 files, snapshot 2616966118738702519 (task 9702 ms: download 2497 ms overlapping gunzip 500 ms, parse 2279 ms, sort 113 ms, write 5336 ms, index 426 ms: 6 candidates, 50586 repo-hours, 681 releases; waiting 756 ms; commit 2886 ms)
2026-11-08T01:21:22Z [load_day 2023-09-30] 2023-09-30 hour 12: 196835 events in 4 files, snapshot 2857992940416877373 (task 12856 ms: download 3456 ms overlapping gunzip 705 ms, parse 3646 ms, sort 175 ms, write 6856 ms, index 662 ms: 6 candidates, 84485 repo-hours, 753 releases; waiting 493 ms; commit 2559 ms)
2026-11-08T01:21:30Z [load_day 2023-09-30] 2023-09-30 hour 13: 154454 events in 4 files, snapshot 2839112329323371745 (task 11503 ms: download 3681 ms overlapping gunzip 593 ms, parse 2712 ms, sort 90 ms, write 6304 ms, index 1031 ms: 9 candidates, 77933 repo-hours, 651 releases; waiting 500 ms; commit 3109 ms)
2026-11-08T01:21:34Z [load_day 2023-09-30] 2023-09-30 hour 14: 140490 events in 3 files, snapshot 2208053914512784999 (task 10569 ms: download 2060 ms overlapping gunzip 546 ms, parse 2451 ms, sort 87 ms, write 6149 ms, index 487 ms: 7 candidates, 60990 repo-hours, 695 releases; waiting 528 ms; commit 2722 ms)
2026-11-08T01:21:43Z [load_day 2023-09-30] 2023-09-30 hour 15: 146053 events in 3 files, snapshot 995403446766423965 (task 11183 ms: download 4068 ms overlapping gunzip 603 ms, parse 2714 ms, sort 125 ms, write 6422 ms, index 513 ms: 6 candidates, 63393 repo-hours, 997 releases; waiting 537 ms; commit 2915 ms)
2026-11-08T01:21:47Z [load_day 2023-09-30] 2023-09-30 hour 16: 142971 events in 3 files, snapshot 7407517083610725576 (task 11995 ms: download 4473 ms overlapping gunzip 666 ms, parse 3019 ms, sort 123 ms, write 6778 ms, index 650 ms: 11 candidates, 61505 repo-hours, 809 releases; waiting 497 ms; commit 2651 ms)
2026-11-08T01:21:54Z [load_day 2023-09-30] 2023-09-30 hour 17: 136443 events in 3 files, snapshot 3814657358921638749 (task 10300 ms: download 2207 ms overlapping gunzip 548 ms, parse 2370 ms, sort 80 ms, write 6005 ms, index 435 ms: 5 candidates, 57652 repo-hours, 699 releases; waiting 518 ms; commit 2871 ms)
2026-11-08T01:21:59Z [load_day 2023-09-30] 2023-09-30 hour 18: 142368 events in 3 files, snapshot 597805643714308996 (task 11464 ms: download 3012 ms overlapping gunzip 545 ms, parse 2585 ms, sort 91 ms, write 6732 ms, index 538 ms: 5 candidates, 63952 repo-hours, 811 releases; waiting 491 ms; commit 2688 ms)
2026-11-08T01:22:08Z [load_day 2023-09-30] 2023-09-30 hour 19: 128210 events in 3 files, snapshot 4722607827176381388 (task 10756 ms: download 2265 ms overlapping gunzip 600 ms, parse 2793 ms, sort 145 ms, write 6054 ms, index 369 ms: 5 candidates, 52196 repo-hours, 744 releases; waiting 462 ms; commit 4133 ms)
2026-11-08T01:22:12Z [load_day 2023-09-30] 2023-09-30 hour 20: 123476 events in 3 files, snapshot 4561047652032886774 (task 11070 ms: download 2266 ms overlapping gunzip 715 ms, parse 3253 ms, sort 100 ms, write 5773 ms, index 428 ms: 7 candidates, 50575 repo-hours, 656 releases; waiting 495 ms; commit 2994 ms)
2026-11-08T01:22:17Z [load_day 2023-09-30] 2023-09-30 hour 21: 108336 events in 3 files, snapshot 7195089331383065011 (task 9374 ms: download 2490 ms overlapping gunzip 521 ms, parse 2424 ms, sort 111 ms, write 5176 ms, index 327 ms: 5 candidates, 44257 repo-hours, 766 releases; waiting 570 ms; commit 2700 ms)
2026-11-08T01:22:22Z [load_day 2023-09-30] 2023-09-30 hour 22: 97685 events in 2 files, snapshot 190985405833444642 (task 8368 ms: download 1980 ms overlapping gunzip 473 ms, parse 2285 ms, sort 68 ms, write 4289 ms, index 393 ms: 4 candidates, 39521 repo-hours, 633 releases; waiting 509 ms; commit 3800 ms)
2026-11-08T01:22:24Z [load_day 2023-09-30] 2023-09-30 hour 23: 92161 events in 2 files, snapshot 1220123548953639176 (task 7040 ms: download 1640 ms overlapping gunzip 379 ms, parse 1937 ms, sort 117 ms, write 3531 ms, index 286 ms: 6 candidates, 37991 repo-hours, 498 releases; waiting 516 ms; commit 2407 ms)
2026-11-08T01:22:31Z [load_day 2023-09-30] 2023-09-30: repo_hour merged from 24 files to 1 (1311640 rows, 6140 ms)
2026-11-08T01:22:33Z [load_day 2023-09-30] 2023-09-30: index committed in 8119 ms (24 + 1 + 24 files)
2026-11-08T01:22:33Z [load_day 2023-09-30] 2023-09-30: 2977513 events in 24 hours
2026-11-08T01:22:33Z [load_day 2023-09-30] attempt 1 succeeded