orcaLog in

Runs · run 1773

load_day 2023-11-24, attempt 1 succeeded

Host
3c2b29713ce8:593010
Started
2026-11-08T03:08:07Z
Ended
2026-11-08T03:11:43Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1773/load_day/2023-11-24/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:08:07Z [load_day 2023-11-24] attempt 1 started
2026-11-08T03:08:09Z [load_day 2023-11-24] 2023-11-24: 2 hours in flight, about 1200 MB
2026-11-08T03:08:27Z [load_day 2023-11-24] 2023-11-24 hour 0: 166711 events in 4 files, snapshot 540633488093636531 (task 13906 ms: download 5059 ms overlapping gunzip 812 ms, parse 3586 ms, sort 117 ms, write 8631 ms, index 538 ms: 8 candidates, 80820 repo-hours, 890 releases; waiting 15 ms; commit 2547 ms)
2026-11-08T03:08:30Z [load_day 2023-11-24] 2023-11-24 hour 1: 172111 events in 4 files, snapshot 2542583725346823426 (task 14418 ms: download 5039 ms overlapping gunzip 787 ms, parse 3650 ms, sort 148 ms, write 8254 ms, index 582 ms: 10 candidates, 88086 repo-hours, 784 releases; waiting 483 ms; commit 3054 ms)
2026-11-08T03:08:40Z [load_day 2023-11-24] 2023-11-24 hour 2: 143191 events in 3 files, snapshot 8649872261846171801 (task 12115 ms: download 4270 ms overlapping gunzip 762 ms, parse 3179 ms, sort 110 ms, write 7281 ms, index 564 ms: 6 candidates, 58376 repo-hours, 840 releases; waiting 20 ms; commit 2856 ms)
2026-11-08T03:08:44Z [load_day 2023-11-24] 2023-11-24 hour 3: 142606 events in 3 files, snapshot 4630044925816808280 (task 11689 ms: download 4786 ms overlapping gunzip 696 ms, parse 2955 ms, sort 119 ms, write 7226 ms, index 448 ms: 10 candidates, 56350 repo-hours, 802 releases; waiting 26 ms; commit 3364 ms)
2026-11-08T03:08:52Z [load_day 2023-11-24] 2023-11-24 hour 4: 136655 events in 3 files, snapshot 6163011171646971196 (task 10069 ms: download 3542 ms overlapping gunzip 571 ms, parse 2532 ms, sort 127 ms, write 6268 ms, index 365 ms: 9 candidates, 52221 repo-hours, 616 releases; waiting 19 ms; commit 2676 ms)
2026-11-08T03:08:58Z [load_day 2023-11-24] 2023-11-24 hour 5: 146701 events in 3 files, snapshot 4482634956632705496 (task 12047 ms: download 4066 ms overlapping gunzip 733 ms, parse 3146 ms, sort 141 ms, write 7262 ms, index 532 ms: 4 candidates, 57200 repo-hours, 635 releases; waiting 20 ms; commit 4856 ms)
2026-11-08T03:09:15Z [load_day 2023-11-24] 2023-11-24 hour 6: 165921 events in 4 files, snapshot 3360613322384684804 (task 14634 ms: download 6097 ms overlapping gunzip 804 ms, parse 3975 ms, sort 129 ms, write 8420 ms, index 556 ms: 10 candidates, 70285 repo-hours, 909 releases; waiting 520 ms; commit 10697 ms)
2026-11-08T03:09:18Z [load_day 2023-11-24] 2023-11-24 hour 7: 175128 events in 4 files, snapshot 8092209077596685089 (task 16703 ms: download 9249 ms overlapping gunzip 997 ms, parse 4331 ms, sort 167 ms, write 10326 ms, index 649 ms: 5 candidates, 69545 repo-hours, 886 releases; waiting 16 ms; commit 2747 ms)
2026-11-08T03:09:27Z [load_day 2023-11-24] 2023-11-24 hour 8: 187624 events in 4 files, snapshot 9077721526168346777 (task 17955 ms: download 9817 ms overlapping gunzip 1071 ms, parse 4731 ms, sort 106 ms, write 10865 ms, index 948 ms: 6 candidates, 76951 repo-hours, 1019 releases; waiting 24 ms; commit 3007 ms)
2026-11-08T03:09:40Z [load_day 2023-11-24] 2023-11-24 hour 9: 202713 events in 5 files, snapshot 5957227849806566915 (task 19856 ms: download 10208 ms overlapping gunzip 1178 ms, parse 5593 ms, sort 130 ms, write 12102 ms, index 646 ms: 7 candidates, 83303 repo-hours, 1257 releases; waiting 20 ms; commit 3229 ms)
2026-11-08T03:09:48Z [load_day 2023-11-24] 2023-11-24 hour 10: 197408 events in 4 files, snapshot 6214923727080406910 (task 20010 ms: download 11073 ms overlapping gunzip 1141 ms, parse 5142 ms, sort 123 ms, write 12459 ms, index 913 ms: 8 candidates, 82986 repo-hours, 1141 releases; waiting 24 ms; commit 3189 ms)
2026-11-08T03:10:00Z [load_day 2023-11-24] 2023-11-24 hour 11: 184370 events in 4 files, snapshot 6179597220089635138 (task 18328 ms: download 10147 ms overlapping gunzip 1099 ms, parse 4971 ms, sort 170 ms, write 11347 ms, index 510 ms: 8 candidates, 77728 repo-hours, 1018 releases; waiting 24 ms; commit 4513 ms)
2026-11-08T03:10:08Z [load_day 2023-11-24] 2023-11-24 hour 12: 235783 events in 5 files, snapshot 3235114561955546693 (task 18790 ms: download 9712 ms overlapping gunzip 976 ms, parse 4647 ms, sort 203 ms, write 12053 ms, index 715 ms: 11 candidates, 110089 repo-hours, 1055 releases; waiting 20 ms; commit 2953 ms)
2026-11-08T03:10:16Z [load_day 2023-11-24] 2023-11-24 hour 13: 214231 events in 5 files, snapshot 1111185965764695103 (task 16817 ms: download 8544 ms overlapping gunzip 965 ms, parse 4237 ms, sort 148 ms, write 10100 ms, index 652 ms: 8 candidates, 99719 repo-hours, 1053 releases; waiting 505 ms; commit 2388 ms)
2026-11-08T03:10:26Z [load_day 2023-11-24] 2023-11-24 hour 14: 205222 events in 5 files, snapshot 7580402374095243850 (task 16616 ms: download 8177 ms overlapping gunzip 972 ms, parse 4456 ms, sort 129 ms, write 10185 ms, index 657 ms: 7 candidates, 85223 repo-hours, 1131 releases; waiting 24 ms; commit 3312 ms)
2026-11-08T03:10:37Z [load_day 2023-11-24] 2023-11-24 hour 15: 201672 events in 5 files, snapshot 3297957449216347796 (task 17969 ms: download 8747 ms overlapping gunzip 993 ms, parse 4475 ms, sort 148 ms, write 11564 ms, index 587 ms: 13 candidates, 86041 repo-hours, 1351 releases; waiting 16 ms; commit 4394 ms)
2026-11-08T03:10:47Z [load_day 2023-11-24] 2023-11-24 hour 16: 192037 events in 4 files, snapshot 2612338021404071515 (task 18356 ms: download 9502 ms overlapping gunzip 1143 ms, parse 5229 ms, sort 181 ms, write 10688 ms, index 898 ms: 8 candidates, 80753 repo-hours, 1051 releases; waiting 17 ms; commit 4581 ms)
2026-11-08T03:10:55Z [load_day 2023-11-24] 2023-11-24 hour 17: 178798 events in 4 files, snapshot 3924942103546602756 (task 16881 ms: download 9042 ms overlapping gunzip 1074 ms, parse 4801 ms, sort 238 ms, write 9985 ms, index 552 ms: 10 candidates, 72041 repo-hours, 850 releases; waiting 25 ms; commit 3305 ms)
2026-11-08T03:11:02Z [load_day 2023-11-24] 2023-11-24 hour 18: 178064 events in 4 files, snapshot 8059833160724504011 (task 15346 ms: download 8122 ms overlapping gunzip 881 ms, parse 3963 ms, sort 149 ms, write 9296 ms, index 800 ms: 8 candidates, 73994 repo-hours, 946 releases; waiting 25 ms; commit 3061 ms)
2026-11-08T03:11:09Z [load_day 2023-11-24] 2023-11-24 hour 19: 158541 events in 4 files, snapshot 5560627611670210736 (task 13468 ms: download 5025 ms overlapping gunzip 842 ms, parse 3746 ms, sort 235 ms, write 7942 ms, index 497 ms: 10 candidates, 60822 repo-hours, 913 releases; waiting 22 ms; commit 3079 ms)
2026-11-08T03:11:15Z [load_day 2023-11-24] 2023-11-24 hour 20: 149903 events in 3 files, snapshot 8288208229601446976 (task 12122 ms: download 4112 ms overlapping gunzip 715 ms, parse 3307 ms, sort 140 ms, write 7239 ms, index 502 ms: 10 candidates, 58376 repo-hours, 786 releases; waiting 20 ms; commit 3091 ms)
2026-11-08T03:11:22Z [load_day 2023-11-24] 2023-11-24 hour 21: 148436 events in 3 files, snapshot 6889767461475360083 (task 12809 ms: download 5027 ms overlapping gunzip 810 ms, parse 3722 ms, sort 126 ms, write 7237 ms, index 699 ms: 9 candidates, 57019 repo-hours, 887 releases; waiting 14 ms; commit 2743 ms)
2026-11-08T03:11:30Z [load_day 2023-11-24] 2023-11-24 hour 22: 131658 events in 3 files, snapshot 6843820052272608718 (task 10864 ms: download 1660 ms overlapping gunzip 678 ms, parse 2982 ms, sort 125 ms, write 6462 ms, index 387 ms: 8 candidates, 49233 repo-hours, 888 releases; waiting 25 ms; commit 5586 ms)
2026-11-08T03:11:33Z [load_day 2023-11-24] 2023-11-24 hour 23: 125274 events in 3 files, snapshot 2249683032195647866 (task 8402 ms: download 968 ms overlapping gunzip 450 ms, parse 2113 ms, sort 104 ms, write 5138 ms, index 394 ms: 10 candidates, 45447 repo-hours, 661 releases; waiting 13 ms; commit 3413 ms)
2026-11-08T03:11:41Z [load_day 2023-11-24] 2023-11-24: repo_hour merged from 24 files to 1 (1732608 rows, 7551 ms)
2026-11-08T03:11:43Z [load_day 2023-11-24] 2023-11-24: index committed in 9935 ms (24 + 1 + 24 files)
2026-11-08T03:11:43Z [load_day 2023-11-24] 2023-11-24: 4140758 events in 24 hours
2026-11-08T03:11:43Z [load_day 2023-11-24] attempt 1 succeeded