orcaLog in

Runs · run 1762

load_day 2023-11-21, attempt 1 succeeded

Host
3c2b29713ce8:580243
Started
2026-11-08T03:01:39Z
Ended
2026-11-08T03:06:16Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1762/load_day/2023-11-21/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:01:39Z [load_day 2023-11-21] attempt 1 started
2026-11-08T03:01:41Z [load_day 2023-11-21] 2023-11-21: 2 hours in flight, about 1200 MB
2026-11-08T03:02:09Z [load_day 2023-11-21] 2023-11-21 hour 0: 183668 events in 4 files, snapshot 1182410219232900809 (task 22769 ms: download 13708 ms overlapping gunzip 1545 ms, parse 6692 ms, sort 197 ms, write 12984 ms, index 567 ms: 9 candidates, 88642 repo-hours, 1003 releases; waiting 545 ms; commit 3115 ms)
2026-11-08T03:02:15Z [load_day 2023-11-21] 2023-11-21 hour 1: 188904 events in 4 files, snapshot 437359447815112093 (task 21158 ms: download 11651 ms overlapping gunzip 1348 ms, parse 6050 ms, sort 222 ms, write 12309 ms, index 1001 ms: 10 candidates, 94969 repo-hours, 855 releases; waiting 24 ms; commit 6334 ms)
2026-11-08T03:02:29Z [load_day 2023-11-21] 2023-11-21 hour 2: 169367 events in 4 files, snapshot 5526221862535785058 (task 17683 ms: download 8341 ms overlapping gunzip 1173 ms, parse 4864 ms, sort 106 ms, write 10309 ms, index 497 ms: 9 candidates, 69454 repo-hours, 820 releases; waiting 498 ms; commit 4411 ms)
2026-11-08T03:02:32Z [load_day 2023-11-21] 2023-11-21 hour 3: 164831 events in 4 files, snapshot 7470147017623661735 (task 16194 ms: download 6695 ms overlapping gunzip 992 ms, parse 4321 ms, sort 133 ms, write 9678 ms, index 839 ms: 2 candidates, 66481 repo-hours, 920 releases; waiting 14 ms; commit 3281 ms)
2026-11-08T03:02:44Z [load_day 2023-11-21] 2023-11-21 hour 4: 161546 events in 4 files, snapshot 8101707328005427450 (task 14366 ms: download 5390 ms overlapping gunzip 882 ms, parse 3757 ms, sort 225 ms, write 8848 ms, index 420 ms: 7 candidates, 61089 repo-hours, 708 releases; waiting 24 ms; commit 3364 ms)
2026-11-08T03:02:50Z [load_day 2023-11-21] 2023-11-21 hour 5: 168556 events in 4 files, snapshot 2386255685867349538 (task 16335 ms: download 6951 ms overlapping gunzip 1109 ms, parse 4724 ms, sort 152 ms, write 9068 ms, index 492 ms: 9 candidates, 65880 repo-hours, 854 releases; waiting 482 ms; commit 2971 ms)
2026-11-08T03:03:03Z [load_day 2023-11-21] 2023-11-21 hour 6: 180926 events in 4 files, snapshot 4472233939942440447 (task 18518 ms: download 10078 ms overlapping gunzip 1145 ms, parse 5232 ms, sort 136 ms, write 11121 ms, index 663 ms: 6 candidates, 77696 repo-hours, 890 releases; waiting 16 ms; commit 2838 ms)
2026-11-08T03:03:10Z [load_day 2023-11-21] 2023-11-21 hour 7: 187640 events in 4 files, snapshot 3821584131881210595 (task 19198 ms: download 11109 ms overlapping gunzip 1222 ms, parse 5323 ms, sort 155 ms, write 11793 ms, index 506 ms: 13 candidates, 76376 repo-hours, 927 releases; waiting 15 ms; commit 3163 ms)
2026-11-08T03:03:25Z [load_day 2023-11-21] 2023-11-21 hour 8: 201867 events in 5 files, snapshot 8141349632898135737 (task 21246 ms: download 11579 ms overlapping gunzip 1295 ms, parse 5814 ms, sort 168 ms, write 13089 ms, index 660 ms: 7 candidates, 85441 repo-hours, 1007 releases; waiting 27 ms; commit 2775 ms)
2026-11-08T03:03:32Z [load_day 2023-11-21] 2023-11-21 hour 9: 177559 events in 4 files, snapshot 8350714776946838278 (task 20428 ms: download 12413 ms overlapping gunzip 1261 ms, parse 5452 ms, sort 144 ms, write 12793 ms, index 556 ms: 7 candidates, 78039 repo-hours, 1094 releases; waiting 24 ms; commit 2959 ms)
2026-11-08T03:03:46Z [load_day 2023-11-21] 2023-11-21 hour 10: 212757 events in 5 files, snapshot 4870756579168042427 (task 19498 ms: download 13820 ms overlapping gunzip 1127 ms, parse 5008 ms, sort 175 ms, write 12400 ms, index 584 ms: 17 candidates, 93884 repo-hours, 1232 releases; waiting 23 ms; commit 2556 ms)
2026-11-08T03:03:50Z [load_day 2023-11-21] 2023-11-21 hour 11: 209092 events in 5 files, snapshot 352462844026975315 (task 17724 ms: download 9513 ms overlapping gunzip 1149 ms, parse 5022 ms, sort 136 ms, write 10598 ms, index 594 ms: 4 candidates, 86991 repo-hours, 1267 releases; waiting 19 ms; commit 2568 ms)
2026-11-08T03:04:09Z [load_day 2023-11-21] 2023-11-21 hour 12: 249476 events in 5 files, snapshot 6763820605597714747 (task 21624 ms: download 12565 ms overlapping gunzip 1105 ms, parse 5066 ms, sort 179 ms, write 14233 ms, index 833 ms: 5 candidates, 116112 repo-hours, 1456 releases; waiting 19 ms; commit 3078 ms)
2026-11-08T03:04:17Z [load_day 2023-11-21] 2023-11-21 hour 13: 236002 events in 5 files, snapshot 7050103942516525261 (task 22719 ms: download 14950 ms overlapping gunzip 1506 ms, parse 7265 ms, sort 184 ms, write 12730 ms, index 821 ms: 8 candidates, 110195 repo-hours, 1302 releases; waiting 16 ms; commit 5703 ms)
2026-11-08T03:04:35Z [load_day 2023-11-21] 2023-11-21 hour 14: 241616 events in 5 files, snapshot 849000820690247165 (task 24951 ms: download 15769 ms overlapping gunzip 1411 ms, parse 6310 ms, sort 228 ms, write 15731 ms, index 1062 ms: 7 candidates, 100020 repo-hours, 1397 releases; waiting 18 ms; commit 2967 ms)
2026-11-08T03:04:41Z [load_day 2023-11-21] 2023-11-21 hour 15: 232023 events in 5 files, snapshot 821802732537121217 (task 25154 ms: download 16150 ms overlapping gunzip 1546 ms, parse 6918 ms, sort 203 ms, write 15512 ms, index 764 ms: 6 candidates, 100808 repo-hours, 1471 releases; waiting 17 ms; commit 2903 ms)
2026-11-08T03:05:02Z [load_day 2023-11-21] 2023-11-21 hour 16: 223041 events in 5 files, snapshot 2155365957028289930 (task 23933 ms: download 16464 ms overlapping gunzip 1296 ms, parse 5885 ms, sort 179 ms, write 15503 ms, index 855 ms: 16 candidates, 96225 repo-hours, 1338 releases; waiting 17 ms; commit 4718 ms)
2026-11-08T03:05:05Z [load_day 2023-11-21] 2023-11-21 hour 17: 203599 events in 5 files, snapshot 4626861941530348242 (task 23150 ms: download 13501 ms overlapping gunzip 1536 ms, parse 7002 ms, sort 173 ms, write 12812 ms, index 848 ms: 10 candidates, 86764 repo-hours, 1098 releases; waiting 497 ms; commit 3211 ms)
2026-11-08T03:05:25Z [load_day 2023-11-21] 2023-11-21 hour 18: 210617 events in 5 files, snapshot 6893831119724934206 (task 22948 ms: download 12654 ms overlapping gunzip 1466 ms, parse 6943 ms, sort 213 ms, write 12807 ms, index 741 ms: 12 candidates, 92514 repo-hours, 1144 releases; waiting 526 ms; commit 2890 ms)
2026-11-08T03:05:28Z [load_day 2023-11-21] 2023-11-21 hour 19: 191329 events in 4 files, snapshot 6215752525451384597 (task 21577 ms: download 12324 ms overlapping gunzip 1334 ms, parse 6362 ms, sort 171 ms, write 11977 ms, index 938 ms: 6 candidates, 80365 repo-hours, 1100 releases; waiting 515 ms; commit 2966 ms)
2026-11-08T03:05:44Z [load_day 2023-11-21] 2023-11-21 hour 20: 188406 events in 4 files, snapshot 6795205707338355806 (task 17593 ms: download 10035 ms overlapping gunzip 1107 ms, parse 5008 ms, sort 127 ms, write 10583 ms, index 549 ms: 10 candidates, 77832 repo-hours, 1004 releases; waiting 21 ms; commit 2986 ms)
2026-11-08T03:05:48Z [load_day 2023-11-21] 2023-11-21 hour 21: 182070 events in 4 files, snapshot 5133627884171375001 (task 18117 ms: download 10233 ms overlapping gunzip 1014 ms, parse 4531 ms, sort 138 ms, write 11426 ms, index 791 ms: 14 candidates, 72083 repo-hours, 1221 releases; waiting 13 ms; commit 3906 ms)
2026-11-08T03:06:02Z [load_day 2023-11-21] 2023-11-21 hour 22: 169775 events in 4 files, snapshot 6693828183340248704 (task 17887 ms: download 10532 ms overlapping gunzip 1258 ms, parse 5585 ms, sort 104 ms, write 10039 ms, index 677 ms: 8 candidates, 67093 repo-hours, 1026 releases; waiting 20 ms; commit 2558 ms)
2026-11-08T03:06:05Z [load_day 2023-11-21] 2023-11-21 hour 23: 153457 events in 4 files, snapshot 632579539815339245 (task 15241 ms: download 6370 ms overlapping gunzip 789 ms, parse 3474 ms, sort 112 ms, write 9621 ms, index 481 ms: 13 candidates, 59825 repo-hours, 817 releases; waiting 543 ms; commit 2358 ms)
2026-11-08T03:06:13Z [load_day 2023-11-21] 2023-11-21: repo_hour merged from 24 files to 1 (2004778 rows, 8736 ms)
2026-11-08T03:06:16Z [load_day 2023-11-21] 2023-11-21: index committed in 11068 ms (24 + 1 + 24 files)
2026-11-08T03:06:16Z [load_day 2023-11-21] 2023-11-21: 4688124 events in 24 hours
2026-11-08T03:06:16Z [load_day 2023-11-21] attempt 1 succeeded