orcaLog in

Runs · run 1540

load_day 2023-09-23, attempt 1 succeeded

Host
3c2b29713ce8:360752
Started
2026-11-08T01:05:54Z
Ended
2026-11-08T01:08:39Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1540/load_day/2023-09-23/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:05:54Z [load_day 2023-09-23] attempt 1 started
2026-11-08T01:05:55Z [load_day 2023-09-23] 2023-09-23: 2 hours in flight, about 1200 MB
2026-11-08T01:06:11Z [load_day 2023-09-23] 2023-09-23 hour 0: 142283 events in 3 files, snapshot 9121122926426382320 (task 12087 ms: download 2781 ms overlapping gunzip 706 ms, parse 3377 ms, sort 139 ms, write 6600 ms, index 496 ms: 8 candidates, 78447 repo-hours, 931 releases; waiting 503 ms; commit 2309 ms)
2026-11-08T01:06:17Z [load_day 2023-09-23] 2023-09-23 hour 1: 133214 events in 3 files, snapshot 789475161166793473 (task 11773 ms: download 2615 ms overlapping gunzip 701 ms, parse 3219 ms, sort 106 ms, write 6467 ms, index 595 ms: 7 candidates, 74560 repo-hours, 653 releases; waiting 470 ms; commit 5437 ms)
2026-11-08T01:06:22Z [load_day 2023-09-23] 2023-09-23 hour 2: 105133 events in 3 files, snapshot 2909228306310990851 (task 9523 ms: download 2231 ms overlapping gunzip 479 ms, parse 2143 ms, sort 123 ms, write 5646 ms, index 372 ms: 7 candidates, 43596 repo-hours, 548 releases; waiting 520 ms; commit 2540 ms)
2026-11-08T01:06:27Z [load_day 2023-09-23] 2023-09-23 hour 3: 103983 events in 3 files, snapshot 7553917244411269304 (task 9437 ms: download 2087 ms overlapping gunzip 518 ms, parse 2402 ms, sort 123 ms, write 5014 ms, index 545 ms: 5 candidates, 42286 repo-hours, 622 releases; waiting 498 ms; commit 4799 ms)
2026-11-08T01:06:36Z [load_day 2023-09-23] 2023-09-23 hour 4: 106546 events in 3 files, snapshot 8825876994693213548 (task 9786 ms: download 1931 ms overlapping gunzip 522 ms, parse 2239 ms, sort 111 ms, write 5829 ms, index 345 ms: 6 candidates, 42651 repo-hours, 529 releases; waiting 506 ms; commit 5106 ms)
2026-11-08T01:06:38Z [load_day 2023-09-23] 2023-09-23 hour 5: 109300 events in 3 files, snapshot 229030531827426989 (task 10136 ms: download 2438 ms overlapping gunzip 708 ms, parse 2957 ms, sort 99 ms, write 5086 ms, index 457 ms: 5 candidates, 45288 repo-hours, 689 releases; waiting 471 ms; commit 2326 ms)
2026-11-08T01:06:45Z [load_day 2023-09-23] 2023-09-23 hour 6: 120326 events in 3 files, snapshot 2157599284651199939 (task 10123 ms: download 2277 ms overlapping gunzip 538 ms, parse 2377 ms, sort 140 ms, write 5879 ms, index 422 ms: 6 candidates, 54655 repo-hours, 665 releases; waiting 443 ms; commit 2657 ms)
2026-11-08T01:06:50Z [load_day 2023-09-23] 2023-09-23 hour 7: 119531 events in 3 files, snapshot 6511834862655762368 (task 10682 ms: download 2485 ms overlapping gunzip 642 ms, parse 2821 ms, sort 75 ms, write 5727 ms, index 545 ms: 5 candidates, 49942 repo-hours, 567 releases; waiting 553 ms; commit 2945 ms)
2026-11-08T01:06:59Z [load_day 2023-09-23] 2023-09-23 hour 8: 123864 events in 3 files, snapshot 7564968537698572634 (task 10239 ms: download 2657 ms overlapping gunzip 591 ms, parse 2570 ms, sort 73 ms, write 5741 ms, index 386 ms: 13 candidates, 54159 repo-hours, 652 releases; waiting 534 ms; commit 5191 ms)
2026-11-08T01:07:04Z [load_day 2023-09-23] 2023-09-23 hour 9: 127689 events in 3 files, snapshot 2983427233567520317 (task 12031 ms: download 2184 ms overlapping gunzip 707 ms, parse 2887 ms, sort 85 ms, write 7135 ms, index 385 ms: 9 candidates, 53958 repo-hours, 803 releases; waiting 483 ms; commit 2995 ms)
2026-11-08T01:07:09Z [load_day 2023-09-23] 2023-09-23 hour 10: 126694 events in 3 files, snapshot 5059184326214134295 (task 11741 ms: download 2705 ms overlapping gunzip 762 ms, parse 3313 ms, sort 147 ms, write 6270 ms, index 437 ms: 5 candidates, 54537 repo-hours, 637 releases; waiting 480 ms; commit 2694 ms)
2026-11-08T01:07:15Z [load_day 2023-09-23] 2023-09-23 hour 11: 122368 events in 3 files, snapshot 5564337301787971276 (task 10327 ms: download 2258 ms overlapping gunzip 528 ms, parse 2426 ms, sort 141 ms, write 5761 ms, index 556 ms: 12 candidates, 51366 repo-hours, 652 releases; waiting 570 ms; commit 2708 ms)
2026-11-08T01:07:24Z [load_day 2023-09-23] 2023-09-23 hour 12: 206781 events in 5 files, snapshot 8778140392577818060 (task 14324 ms: download 5096 ms overlapping gunzip 769 ms, parse 3902 ms, sort 202 ms, write 8074 ms, index 607 ms: 13 candidates, 86631 repo-hours, 824 releases; waiting 509 ms; commit 2809 ms)
2026-11-08T01:07:30Z [load_day 2023-09-23] 2023-09-23 hour 13: 158146 events in 4 files, snapshot 3978592722963687179 (task 13046 ms: download 4514 ms overlapping gunzip 668 ms, parse 2970 ms, sort 123 ms, write 7802 ms, index 648 ms: 12 candidates, 79493 repo-hours, 755 releases; waiting 524 ms; commit 2978 ms)
2026-11-08T01:07:41Z [load_day 2023-09-23] 2023-09-23 hour 14: 151751 events in 4 files, snapshot 1086553764711513013 (task 13444 ms: download 4566 ms overlapping gunzip 716 ms, parse 3172 ms, sort 207 ms, write 7949 ms, index 555 ms: 9 candidates, 65575 repo-hours, 797 releases; waiting 528 ms; commit 4879 ms)
2026-11-08T01:07:44Z [load_day 2023-09-23] 2023-09-23 hour 15: 153242 events in 4 files, snapshot 2411086427759274941 (task 13269 ms: download 5673 ms overlapping gunzip 793 ms, parse 3438 ms, sort 155 ms, write 7495 ms, index 640 ms: 8 candidates, 66473 repo-hours, 979 releases; waiting 501 ms; commit 3175 ms)
2026-11-08T01:07:54Z [load_day 2023-09-23] 2023-09-23 hour 16: 147172 events in 3 files, snapshot 6565088284853039217 (task 13713 ms: download 5704 ms overlapping gunzip 806 ms, parse 3504 ms, sort 142 ms, write 7183 ms, index 663 ms: 13 candidates, 62790 repo-hours, 813 releases; waiting 1036 ms; commit 2542 ms)
2026-11-08T01:07:57Z [load_day 2023-09-23] 2023-09-23 hour 17: 139168 events in 3 files, snapshot 2584895714786103996 (task 10710 ms: download 4209 ms overlapping gunzip 541 ms, parse 2468 ms, sort 87 ms, write 6390 ms, index 624 ms: 9 candidates, 58282 repo-hours, 719 releases; waiting 285 ms; commit 3064 ms)
2026-11-08T01:08:06Z [load_day 2023-09-23] 2023-09-23 hour 18: 144823 events in 3 files, snapshot 4972509651677318232 (task 11424 ms: download 3884 ms overlapping gunzip 691 ms, parse 3030 ms, sort 116 ms, write 6339 ms, index 447 ms: 8 candidates, 63977 repo-hours, 805 releases; waiting 524 ms; commit 2272 ms)
2026-11-08T01:08:11Z [load_day 2023-09-23] 2023-09-23 hour 19: 126659 events in 3 files, snapshot 5377793424351655432 (task 10692 ms: download 2001 ms overlapping gunzip 499 ms, parse 2349 ms, sort 132 ms, write 6545 ms, index 370 ms: 7 candidates, 51815 repo-hours, 724 releases; waiting 502 ms; commit 5335 ms)
2026-11-08T01:08:18Z [load_day 2023-09-23] 2023-09-23 hour 20: 125129 events in 3 files, snapshot 3541844393723786540 (task 10265 ms: download 2477 ms overlapping gunzip 645 ms, parse 2884 ms, sort 68 ms, write 5184 ms, index 591 ms: 6 candidates, 51004 repo-hours, 710 releases; waiting 577 ms; commit 2829 ms)
2026-11-08T01:08:20Z [load_day 2023-09-23] 2023-09-23 hour 21: 120506 events in 3 files, snapshot 7779885209110755809 (task 10707 ms: download 2507 ms overlapping gunzip 601 ms, parse 2751 ms, sort 116 ms, write 5803 ms, index 537 ms: 6 candidates, 52217 repo-hours, 1360 releases; waiting 561 ms; commit 2618 ms)
2026-11-08T01:08:29Z [load_day 2023-09-23] 2023-09-23 hour 22: 105416 events in 3 files, snapshot 7336232549297985711 (task 8048 ms: download 1889 ms overlapping gunzip 451 ms, parse 1947 ms, sort 103 ms, write 4378 ms, index 319 ms: 13 candidates, 43158 repo-hours, 649 releases; waiting 509 ms; commit 4779 ms)
2026-11-08T01:08:31Z [load_day 2023-09-23] 2023-09-23 hour 23: 99033 events in 2 files, snapshot 8016434779980823194 (task 7078 ms: download 1794 ms overlapping gunzip 406 ms, parse 1803 ms, sort 77 ms, write 3793 ms, index 309 ms: 14 candidates, 40473 repo-hours, 562 releases; waiting 478 ms; commit 2122 ms)
2026-11-08T01:08:37Z [load_day 2023-09-23] 2023-09-23: repo_hour merged from 24 files to 1 (1367333 rows, 5985 ms)
2026-11-08T01:08:39Z [load_day 2023-09-23] 2023-09-23: index committed in 8287 ms (24 + 1 + 24 files)
2026-11-08T01:08:39Z [load_day 2023-09-23] 2023-09-23: 3118757 events in 24 hours
2026-11-08T01:08:39Z [load_day 2023-09-23] attempt 1 succeeded