orcaLog in

Runs · run 1537

load_day 2023-09-22, attempt 1 succeeded

Host
3c2b29713ce8:358704
Started
2026-11-08T01:04:44Z
Ended
2026-11-08T01:08:37Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1537/load_day/2023-09-22/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:04:44Z [load_day 2023-09-22] attempt 1 started
2026-11-08T01:04:46Z [load_day 2023-09-22] 2023-09-22: 2 hours in flight, about 1200 MB
2026-11-08T01:05:05Z [load_day 2023-09-22] 2023-09-22 hour 0: 163347 events in 4 files, snapshot 6163019796680068327 (task 15654 ms: download 6165 ms overlapping gunzip 947 ms, parse 4331 ms, sort 126 ms, write 8493 ms, index 554 ms: 15 candidates, 86995 repo-hours, 991 releases; waiting 211 ms; commit 2085 ms)
2026-11-08T01:05:10Z [load_day 2023-09-22] 2023-09-22 hour 1: 153484 events in 4 files, snapshot 439763458773593270 (task 15026 ms: download 5915 ms overlapping gunzip 801 ms, parse 3838 ms, sort 155 ms, write 8515 ms, index 707 ms: 11 candidates, 85358 repo-hours, 724 releases; waiting 729 ms; commit 5255 ms)
2026-11-08T01:05:19Z [load_day 2023-09-22] 2023-09-22 hour 2: 132903 events in 3 files, snapshot 283667349318289418 (task 12166 ms: download 4995 ms overlapping gunzip 650 ms, parse 2761 ms, sort 77 ms, write 7424 ms, index 490 ms: 8 candidates, 58306 repo-hours, 712 releases; waiting 507 ms; commit 2487 ms)
2026-11-08T01:05:21Z [load_day 2023-09-22] 2023-09-22 hour 3: 132863 events in 3 files, snapshot 7448049071880684235 (task 11908 ms: download 5066 ms overlapping gunzip 760 ms, parse 3384 ms, sort 101 ms, write 6484 ms, index 454 ms: 9 candidates, 57404 repo-hours, 824 releases; waiting 464 ms; commit 2401 ms)
2026-11-08T01:05:31Z [load_day 2023-09-22] 2023-09-22 hour 4: 129376 events in 3 files, snapshot 1976173795075589472 (task 10479 ms: download 3989 ms overlapping gunzip 585 ms, parse 2526 ms, sort 98 ms, write 6060 ms, index 420 ms: 11 candidates, 53427 repo-hours, 682 releases; waiting 522 ms; commit 2721 ms)
2026-11-08T01:05:36Z [load_day 2023-09-22] 2023-09-22 hour 5: 141605 events in 3 files, snapshot 370211300167649790 (task 13464 ms: download 4865 ms overlapping gunzip 805 ms, parse 3527 ms, sort 205 ms, write 7572 ms, index 452 ms: 13 candidates, 59740 repo-hours, 970 releases; waiting 543 ms; commit 2581 ms)
2026-11-08T01:05:46Z [load_day 2023-09-22] 2023-09-22 hour 6: 169429 events in 4 files, snapshot 3067492232984633398 (task 14034 ms: download 8501 ms overlapping gunzip 881 ms, parse 3929 ms, sort 108 ms, write 8176 ms, index 506 ms: 7 candidates, 78506 repo-hours, 979 releases; waiting 213 ms; commit 2458 ms)
2026-11-08T01:05:52Z [load_day 2023-09-22] 2023-09-22 hour 7: 174815 events in 4 files, snapshot 8448875721692386061 (task 15064 ms: download 8549 ms overlapping gunzip 960 ms, parse 4264 ms, sort 117 ms, write 8510 ms, index 497 ms: 6 candidates, 77772 repo-hours, 990 releases; waiting 487 ms; commit 2207 ms)
2026-11-08T01:06:04Z [load_day 2023-09-22] 2023-09-22 hour 8: 186145 events in 4 files, snapshot 4173865084723735502 (task 17073 ms: download 8614 ms overlapping gunzip 954 ms, parse 4139 ms, sort 124 ms, write 10166 ms, index 921 ms: 13 candidates, 83362 repo-hours, 1064 releases; waiting 512 ms; commit 2567 ms)
2026-11-08T01:06:14Z [load_day 2023-09-22] 2023-09-22 hour 9: 188057 events in 4 files, snapshot 5900557864779777654 (task 18528 ms: download 10238 ms overlapping gunzip 960 ms, parse 4379 ms, sort 127 ms, write 11669 ms, index 576 ms: 9 candidates, 84112 repo-hours, 1212 releases; waiting 500 ms; commit 4516 ms)
2026-11-08T01:06:25Z [load_day 2023-09-22] 2023-09-22 hour 10: 176191 events in 4 files, snapshot 8221890747345783913 (task 18275 ms: download 9846 ms overlapping gunzip 1030 ms, parse 4581 ms, sort 148 ms, write 10898 ms, index 565 ms: 6 candidates, 78425 repo-hours, 1059 releases; waiting 529 ms; commit 3992 ms)
2026-11-08T01:06:33Z [load_day 2023-09-22] 2023-09-22 hour 11: 174972 events in 4 files, snapshot 7960627275216159043 (task 19582 ms: download 10343 ms overlapping gunzip 1097 ms, parse 4961 ms, sort 113 ms, write 12015 ms, index 574 ms: 5 candidates, 75143 repo-hours, 1011 releases; waiting 501 ms; commit 2669 ms)
2026-11-08T01:06:47Z [load_day 2023-09-22] 2023-09-22 hour 12: 232494 events in 5 files, snapshot 9135982604297853383 (task 22885 ms: download 11554 ms overlapping gunzip 1399 ms, parse 6598 ms, sort 259 ms, write 12682 ms, index 1004 ms: 12 candidates, 109399 repo-hours, 1148 releases; waiting 491 ms; commit 2309 ms)
2026-11-08T01:06:56Z [load_day 2023-09-22] 2023-09-22 hour 13: 209381 events in 5 files, snapshot 3624962609022959021 (task 21775 ms: download 11050 ms overlapping gunzip 1224 ms, parse 5423 ms, sort 200 ms, write 13082 ms, index 1063 ms: 10 candidates, 104882 repo-hours, 1145 releases; waiting 516 ms; commit 2900 ms)
2026-11-08T01:07:11Z [load_day 2023-09-22] 2023-09-22 hour 14: 196187 events in 4 files, snapshot 4034191360460391398 (task 21459 ms: download 11892 ms overlapping gunzip 1211 ms, parse 5427 ms, sort 158 ms, write 13186 ms, index 638 ms: 12 candidates, 88786 repo-hours, 1140 releases; waiting 526 ms; commit 4096 ms)
2026-11-08T01:07:19Z [load_day 2023-09-22] 2023-09-22 hour 15: 183624 events in 4 files, snapshot 2006161563079194291 (task 21188 ms: download 12245 ms overlapping gunzip 1132 ms, parse 5063 ms, sort 122 ms, write 13245 ms, index 814 ms: 13 candidates, 82977 repo-hours, 1118 releases; waiting 520 ms; commit 3198 ms)
2026-11-08T01:07:32Z [load_day 2023-09-22] 2023-09-22 hour 16: 183926 events in 4 files, snapshot 2729712610655834131 (task 19583 ms: download 11335 ms overlapping gunzip 1163 ms, parse 5107 ms, sort 158 ms, write 11691 ms, index 608 ms: 7 candidates, 82380 repo-hours, 1001 releases; waiting 536 ms; commit 3803 ms)
2026-11-08T01:07:38Z [load_day 2023-09-22] 2023-09-22 hour 17: 177538 events in 4 files, snapshot 7232064705781702049 (task 19022 ms: download 10378 ms overlapping gunzip 1019 ms, parse 4581 ms, sort 238 ms, write 12066 ms, index 515 ms: 9 candidates, 75810 repo-hours, 980 releases; waiting 256 ms; commit 2408 ms)
2026-11-08T01:07:51Z [load_day 2023-09-22] 2023-09-22 hour 18: 184488 events in 4 files, snapshot 641060088708925666 (task 18835 ms: download 10198 ms overlapping gunzip 1268 ms, parse 5478 ms, sort 120 ms, write 10543 ms, index 615 ms: 12 candidates, 82800 repo-hours, 1046 releases; waiting 479 ms; commit 2542 ms)
2026-11-08T01:07:59Z [load_day 2023-09-22] 2023-09-22 hour 19: 167594 events in 4 files, snapshot 3829453162557393898 (task 17961 ms: download 10992 ms overlapping gunzip 1110 ms, parse 4672 ms, sort 109 ms, write 10745 ms, index 485 ms: 11 candidates, 70574 repo-hours, 963 releases; waiting 513 ms; commit 3711 ms)
2026-11-08T01:08:09Z [load_day 2023-09-22] 2023-09-22 hour 20: 158717 events in 4 files, snapshot 1999418040453281160 (task 16037 ms: download 6337 ms overlapping gunzip 874 ms, parse 3902 ms, sort 101 ms, write 9628 ms, index 440 ms: 11 candidates, 64949 repo-hours, 930 releases; waiting 629 ms; commit 3050 ms)
2026-11-08T01:08:15Z [load_day 2023-09-22] 2023-09-22 hour 21: 146797 events in 3 files, snapshot 4747463069469102857 (task 16017 ms: download 6815 ms overlapping gunzip 973 ms, parse 4379 ms, sort 137 ms, write 9367 ms, index 391 ms: 8 candidates, 59408 repo-hours, 969 releases; waiting 393 ms; commit 2235 ms)
2026-11-08T01:08:23Z [load_day 2023-09-22] 2023-09-22 hour 22: 135108 events in 3 files, snapshot 9047891682165643327 (task 14570 ms: download 6423 ms overlapping gunzip 902 ms, parse 3947 ms, sort 152 ms, write 8283 ms, index 524 ms: 12 candidates, 53260 repo-hours, 871 releases; waiting 450 ms; commit 2022 ms)
2026-11-08T01:08:27Z [load_day 2023-09-22] 2023-09-22 hour 23: 116494 events in 3 files, snapshot 1750590354400579692 (task 10829 ms: download 2593 ms overlapping gunzip 622 ms, parse 2566 ms, sort 72 ms, write 6387 ms, index 340 ms: 10 candidates, 48353 repo-hours, 650 releases; waiting 526 ms; commit 2948 ms)
2026-11-08T01:08:35Z [load_day 2023-09-22] 2023-09-22: repo_hour merged from 24 files to 1 (1802128 rows, 7638 ms)
2026-11-08T01:08:37Z [load_day 2023-09-22] 2023-09-22: index committed in 9406 ms (24 + 1 + 24 files)
2026-11-08T01:08:37Z [load_day 2023-09-22] 2023-09-22: 4015535 events in 24 hours
2026-11-08T01:08:37Z [load_day 2023-09-22] attempt 1 succeeded