orcaLog in

Runs · run 1506

load_day 2023-09-15, attempt 1 succeeded

Host
3c2b29713ce8:333488
Started
2026-11-08T00:51:23Z
Ended
2026-11-08T00:55:10Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1506/load_day/2023-09-15/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:51:23Z [load_day 2023-09-15] attempt 1 started
2026-11-08T00:51:24Z [load_day 2023-09-15] 2023-09-15: 2 hours in flight, about 1200 MB
2026-11-08T00:51:44Z [load_day 2023-09-15] 2023-09-15 hour 0: 164022 events in 4 files, snapshot 7508743763556318877 (task 15367 ms: download 6947 ms overlapping gunzip 897 ms, parse 4310 ms, sort 135 ms, write 8561 ms, index 664 ms: 9 candidates, 89170 repo-hours, 897 releases; waiting 521 ms; commit 2804 ms)
2026-11-08T00:51:47Z [load_day 2023-09-15] 2023-09-15 hour 1: 154871 events in 4 files, snapshot 3467434602674310819 (task 14065 ms: download 5621 ms overlapping gunzip 695 ms, parse 3227 ms, sort 117 ms, write 8458 ms, index 692 ms: 8 candidates, 88259 repo-hours, 802 releases; waiting 576 ms; commit 2356 ms)
2026-11-08T00:51:58Z [load_day 2023-09-15] 2023-09-15 hour 2: 137281 events in 3 files, snapshot 6350571142750433725 (task 13162 ms: download 5158 ms overlapping gunzip 784 ms, parse 3426 ms, sort 107 ms, write 7544 ms, index 480 ms: 3 candidates, 59280 repo-hours, 894 releases; waiting 503 ms; commit 2045 ms)
2026-11-08T00:52:01Z [load_day 2023-09-15] 2023-09-15 hour 3: 135481 events in 3 files, snapshot 3100664602238106911 (task 12282 ms: download 5612 ms overlapping gunzip 713 ms, parse 3072 ms, sort 117 ms, write 7217 ms, index 405 ms: 7 candidates, 56780 repo-hours, 889 releases; waiting 493 ms; commit 3207 ms)
2026-11-08T00:52:13Z [load_day 2023-09-15] 2023-09-15 hour 4: 128190 events in 3 files, snapshot 8495531211135558614 (task 11956 ms: download 5211 ms overlapping gunzip 655 ms, parse 2760 ms, sort 83 ms, write 7295 ms, index 363 ms: 10 candidates, 54103 repo-hours, 682 releases; waiting 535 ms; commit 4091 ms)
2026-11-08T00:52:16Z [load_day 2023-09-15] 2023-09-15 hour 5: 142205 events in 3 files, snapshot 6319906002180340825 (task 14686 ms: download 5885 ms overlapping gunzip 939 ms, parse 4092 ms, sort 105 ms, write 8134 ms, index 447 ms: 8 candidates, 61533 repo-hours, 815 releases; waiting 502 ms; commit 2463 ms)
2026-11-08T00:52:30Z [load_day 2023-09-15] 2023-09-15 hour 6: 170233 events in 4 files, snapshot 3869393873075049163 (task 16940 ms: download 9512 ms overlapping gunzip 1064 ms, parse 4449 ms, sort 179 ms, write 9654 ms, index 791 ms: 10 candidates, 78309 repo-hours, 919 releases; waiting 508 ms; commit 2933 ms)
2026-11-08T00:52:38Z [load_day 2023-09-15] 2023-09-15 hour 7: 177681 events in 4 files, snapshot 3145997323572961875 (task 19984 ms: download 10846 ms overlapping gunzip 1335 ms, parse 5888 ms, sort 171 ms, write 11220 ms, index 602 ms: 8 candidates, 78757 repo-hours, 1087 releases; waiting 478 ms; commit 2742 ms)
2026-11-08T00:52:52Z [load_day 2023-09-15] 2023-09-15 hour 8: 188762 events in 4 files, snapshot 119485733656718614 (task 19315 ms: download 11432 ms overlapping gunzip 1062 ms, parse 4594 ms, sort 170 ms, write 11856 ms, index 804 ms: 13 candidates, 84157 repo-hours, 1181 releases; waiting 499 ms; commit 4386 ms)
2026-11-08T00:53:00Z [load_day 2023-09-15] 2023-09-15 hour 9: 191635 events in 4 files, snapshot 6091836465085377589 (task 21017 ms: download 11365 ms overlapping gunzip 1231 ms, parse 5549 ms, sort 124 ms, write 12545 ms, index 756 ms: 9 candidates, 84689 repo-hours, 1423 releases; waiting 531 ms; commit 2748 ms)
2026-11-08T00:53:08Z [load_day 2023-09-15] 2023-09-15 hour 10: 181913 events in 4 files, snapshot 1319139108399790768 (task 16424 ms: download 9952 ms overlapping gunzip 1118 ms, parse 5076 ms, sort 165 ms, write 9276 ms, index 550 ms: 7 candidates, 81038 repo-hours, 1065 releases; waiting 17 ms; commit 2356 ms)
2026-11-08T00:53:14Z [load_day 2023-09-15] 2023-09-15 hour 11: 177560 events in 4 files, snapshot 2971227082877974307 (task 14220 ms: download 7997 ms overlapping gunzip 928 ms, parse 3915 ms, sort 118 ms, write 8556 ms, index 501 ms: 8 candidates, 74910 repo-hours, 1119 releases; waiting 15 ms; commit 2340 ms)
2026-11-08T00:53:27Z [load_day 2023-09-15] 2023-09-15 hour 12: 239056 events in 5 files, snapshot 662679088270845027 (task 18328 ms: download 10447 ms overlapping gunzip 1110 ms, parse 5087 ms, sort 149 ms, write 10866 ms, index 898 ms: 9 candidates, 108057 repo-hours, 1092 releases; waiting 26 ms; commit 2383 ms)
2026-11-08T00:53:36Z [load_day 2023-09-15] 2023-09-15 hour 13: 211811 events in 5 files, snapshot 5070582115682951543 (task 19522 ms: download 9514 ms overlapping gunzip 1143 ms, parse 5200 ms, sort 206 ms, write 12102 ms, index 676 ms: 9 candidates, 103328 repo-hours, 1130 releases; waiting 16 ms; commit 3026 ms)
2026-11-08T00:53:49Z [load_day 2023-09-15] 2023-09-15 hour 14: 201448 events in 5 files, snapshot 5942217964054756917 (task 20238 ms: download 10774 ms overlapping gunzip 1216 ms, parse 5475 ms, sort 141 ms, write 12448 ms, index 739 ms: 9 candidates, 88399 repo-hours, 1219 releases; waiting 23 ms; commit 2731 ms)
2026-11-08T00:54:05Z [load_day 2023-09-15] 2023-09-15 hour 15: 191792 events in 4 files, snapshot 3291688233942334459 (task 19955 ms: download 10557 ms overlapping gunzip 1252 ms, parse 5003 ms, sort 169 ms, write 12487 ms, index 800 ms: 11 candidates, 87253 repo-hours, 1234 releases; waiting 15 ms; commit 11208 ms)
2026-11-08T00:54:10Z [load_day 2023-09-15] 2023-09-15 hour 16: 181417 events in 4 files, snapshot 2973573135803712702 (task 19815 ms: download 10886 ms overlapping gunzip 1087 ms, parse 5013 ms, sort 103 ms, write 12486 ms, index 907 ms: 14 candidates, 78943 repo-hours, 948 releases; waiting 23 ms; commit 2986 ms)
2026-11-08T00:54:16Z [load_day 2023-09-15] 2023-09-15 hour 17: 174086 events in 4 files, snapshot 8715418509853164174 (task 17694 ms: download 10441 ms overlapping gunzip 1131 ms, parse 4719 ms, sort 120 ms, write 10965 ms, index 505 ms: 12 candidates, 74006 repo-hours, 1019 releases; waiting 25 ms; commit 3044 ms)
2026-11-08T00:54:28Z [load_day 2023-09-15] 2023-09-15 hour 18: 172385 events in 4 files, snapshot 9084198863778417901 (task 17220 ms: download 9692 ms overlapping gunzip 1015 ms, parse 4812 ms, sort 184 ms, write 10403 ms, index 603 ms: 4 candidates, 77480 repo-hours, 1099 releases; waiting 17 ms; commit 2988 ms)
2026-11-08T00:54:35Z [load_day 2023-09-15] 2023-09-15 hour 19: 162914 events in 4 files, snapshot 9002162293350227093 (task 17609 ms: download 10083 ms overlapping gunzip 1081 ms, parse 4789 ms, sort 166 ms, write 10790 ms, index 560 ms: 9 candidates, 66213 repo-hours, 1036 releases; waiting 25 ms; commit 2845 ms)
2026-11-08T00:54:46Z [load_day 2023-09-15] 2023-09-15 hour 20: 156819 events in 4 files, snapshot 7099907297806197632 (task 15820 ms: download 6338 ms overlapping gunzip 975 ms, parse 4283 ms, sort 165 ms, write 9776 ms, index 408 ms: 7 candidates, 63984 repo-hours, 967 releases; waiting 21 ms; commit 3201 ms)
2026-11-08T00:54:50Z [load_day 2023-09-15] 2023-09-15 hour 21: 143314 events in 3 files, snapshot 3581554211150611340 (task 14340 ms: download 5729 ms overlapping gunzip 945 ms, parse 4023 ms, sort 98 ms, write 8334 ms, index 720 ms: 8 candidates, 56920 repo-hours, 1045 releases; waiting 32 ms; commit 2665 ms)
2026-11-08T00:54:58Z [load_day 2023-09-15] 2023-09-15 hour 22: 125178 events in 3 files, snapshot 7608270018291719127 (task 11436 ms: download 4532 ms overlapping gunzip 700 ms, parse 3090 ms, sort 85 ms, write 6943 ms, index 396 ms: 8 candidates, 49859 repo-hours, 813 releases; waiting 22 ms; commit 2100 ms)
2026-11-08T00:55:00Z [load_day 2023-09-15] 2023-09-15 hour 23: 115757 events in 3 files, snapshot 2563830502417328557 (task 9878 ms: download 1448 ms overlapping gunzip 592 ms, parse 2505 ms, sort 90 ms, write 6184 ms, index 296 ms: 11 candidates, 46191 repo-hours, 776 releases; waiting 17 ms; commit 2065 ms)
2026-11-08T00:55:08Z [load_day 2023-09-15] 2023-09-15: repo_hour merged from 24 files to 1 (1791618 rows, 7964 ms)
2026-11-08T00:55:10Z [load_day 2023-09-15] 2023-09-15: index committed in 9883 ms (24 + 1 + 24 files)
2026-11-08T00:55:10Z [load_day 2023-09-15] 2023-09-15: 4025811 events in 24 hours
2026-11-08T00:55:10Z [load_day 2023-09-15] attempt 1 succeeded