orcaLog in

Runs · run 1789

load_day 2023-11-28, attempt 1 succeeded

Host
3c2b29713ce8:604843
Started
2026-11-08T03:14:39Z
Ended
2026-11-08T03:19:28Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1789/load_day/2023-11-28/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:14:39Z [load_day 2023-11-28] attempt 1 started
2026-11-08T03:14:40Z [load_day 2023-11-28] 2023-11-28: 2 hours in flight, about 1200 MB
2026-11-08T03:15:07Z [load_day 2023-11-28] 2023-11-28 hour 0: 241976 events in 5 files, snapshot 391203225413155061 (task 21959 ms: download 14170 ms overlapping gunzip 1363 ms, parse 6088 ms, sort 221 ms, write 12900 ms, index 601 ms: 10 candidates, 88664 repo-hours, 1017 releases; waiting 539 ms; commit 2861 ms)
2026-11-08T03:15:10Z [load_day 2023-11-28] 2023-11-28 hour 1: 275876 events in 6 files, snapshot 5638884748694553416 (task 21191 ms: download 12776 ms overlapping gunzip 1369 ms, parse 6022 ms, sort 335 ms, write 12053 ms, index 695 ms: 7 candidates, 94808 repo-hours, 833 releases; waiting 464 ms; commit 3133 ms)
2026-11-08T03:15:26Z [load_day 2023-11-28] 2023-11-28 hour 2: 207406 events in 5 files, snapshot 77316085626224875 (task 17965 ms: download 9272 ms overlapping gunzip 1110 ms, parse 4761 ms, sort 131 ms, write 9764 ms, index 1355 ms: 15 candidates, 68756 repo-hours, 833 releases; waiting 544 ms; commit 2552 ms)
2026-11-08T03:15:31Z [load_day 2023-11-28] 2023-11-28 hour 3: 214008 events in 5 files, snapshot 1834667000908859869 (task 15904 ms: download 8469 ms overlapping gunzip 948 ms, parse 4044 ms, sort 134 ms, write 10071 ms, index 481 ms: 8 candidates, 64754 repo-hours, 839 releases; waiting 20 ms; commit 5439 ms)
2026-11-08T03:15:45Z [load_day 2023-11-28] 2023-11-28 hour 4: 209685 events in 5 files, snapshot 3687698105082994317 (task 16558 ms: download 8900 ms overlapping gunzip 1081 ms, parse 4693 ms, sort 233 ms, write 9358 ms, index 457 ms: 12 candidates, 59682 repo-hours, 665 releases; waiting 502 ms; commit 4540 ms)
2026-11-08T03:15:48Z [load_day 2023-11-28] 2023-11-28 hour 5: 216763 events in 5 files, snapshot 5998849924277294753 (task 18007 ms: download 9298 ms overlapping gunzip 1277 ms, parse 5717 ms, sort 205 ms, write 10003 ms, index 574 ms: 8 candidates, 63754 repo-hours, 725 releases; waiting 10 ms; commit 3088 ms)
2026-11-08T03:16:03Z [load_day 2023-11-28] 2023-11-28 hour 6: 232319 events in 5 files, snapshot 7154145963462529821 (task 17238 ms: download 8580 ms overlapping gunzip 1052 ms, parse 4971 ms, sort 199 ms, write 10264 ms, index 517 ms: 7 candidates, 75587 repo-hours, 902 releases; waiting 16 ms; commit 3370 ms)
2026-11-08T03:16:09Z [load_day 2023-11-28] 2023-11-28 hour 7: 233643 events in 5 files, snapshot 9155607167042176027 (task 19120 ms: download 12226 ms overlapping gunzip 1152 ms, parse 5538 ms, sort 244 ms, write 11383 ms, index 541 ms: 11 candidates, 74145 repo-hours, 766 releases; waiting 27 ms; commit 3183 ms)
2026-11-08T03:16:33Z [load_day 2023-11-28] 2023-11-28 hour 8: 247235 events in 5 files, snapshot 304587526902771390 (task 21837 ms: download 13792 ms overlapping gunzip 1398 ms, parse 6141 ms, sort 191 ms, write 12913 ms, index 978 ms: 9 candidates, 83296 repo-hours, 970 releases; waiting 20 ms; commit 10521 ms)
2026-11-08T03:16:40Z [load_day 2023-11-28] 2023-11-28 hour 9: 266634 events in 6 files, snapshot 6943535378719417272 (task 24005 ms: download 13626 ms overlapping gunzip 1470 ms, parse 6828 ms, sort 211 ms, write 14726 ms, index 566 ms: 11 candidates, 85759 repo-hours, 1175 releases; waiting 16 ms; commit 6918 ms)
2026-11-08T03:16:50Z [load_day 2023-11-28] 2023-11-28 hour 10: 259042 events in 6 files, snapshot 5411696871552423965 (task 23459 ms: download 14011 ms overlapping gunzip 1461 ms, parse 6358 ms, sort 246 ms, write 14432 ms, index 737 ms: 9 candidates, 85814 repo-hours, 1051 releases; waiting 17 ms; commit 3028 ms)
2026-11-08T03:17:00Z [load_day 2023-11-28] 2023-11-28 hour 11: 248283 events in 5 files, snapshot 8007825382206304343 (task 22376 ms: download 14097 ms overlapping gunzip 1392 ms, parse 6436 ms, sort 308 ms, write 13120 ms, index 656 ms: 9 candidates, 83570 repo-hours, 1029 releases; waiting 189 ms; commit 3297 ms)
2026-11-08T03:17:14Z [load_day 2023-11-28] 2023-11-28 hour 12: 281790 events in 6 files, snapshot 2734562164341623956 (task 23205 ms: download 13005 ms overlapping gunzip 1455 ms, parse 6451 ms, sort 206 ms, write 13849 ms, index 1031 ms: 15 candidates, 112245 repo-hours, 1051 releases; waiting 16 ms; commit 2901 ms)
2026-11-08T03:17:22Z [load_day 2023-11-28] 2023-11-28 hour 13: 263875 events in 6 files, snapshot 4226761647133229283 (task 22014 ms: download 13612 ms overlapping gunzip 1391 ms, parse 6175 ms, sort 191 ms, write 13363 ms, index 669 ms: 6 candidates, 104436 repo-hours, 1079 releases; waiting 23 ms; commit 2667 ms)
2026-11-08T03:17:42Z [load_day 2023-11-28] 2023-11-28 hour 14: 273253 events in 6 files, snapshot 4720823293744604570 (task 22554 ms: download 16511 ms overlapping gunzip 1461 ms, parse 6392 ms, sort 185 ms, write 13522 ms, index 779 ms: 5 candidates, 99738 repo-hours, 1151 releases; waiting 18 ms; commit 6608 ms)
2026-11-08T03:17:46Z [load_day 2023-11-28] 2023-11-28 hour 15: 272492 events in 6 files, snapshot 8092849133721255252 (task 22574 ms: download 14971 ms overlapping gunzip 1241 ms, parse 5339 ms, sort 171 ms, write 14246 ms, index 781 ms: 12 candidates, 99815 repo-hours, 1219 releases; waiting 542 ms; commit 2977 ms)
2026-11-08T03:18:05Z [load_day 2023-11-28] 2023-11-28 hour 16: 266485 events in 6 files, snapshot 7638061722362845457 (task 24638 ms: download 14978 ms overlapping gunzip 1313 ms, parse 5993 ms, sort 231 ms, write 15565 ms, index 607 ms: 13 candidates, 94973 repo-hours, 1143 releases; waiting 633 ms; commit 2482 ms)
2026-11-08T03:18:14Z [load_day 2023-11-28] 2023-11-28 hour 17: 260771 events in 6 files, snapshot 3319126646096639155 (task 23991 ms: download 15487 ms overlapping gunzip 1494 ms, parse 6586 ms, sort 185 ms, write 14133 ms, index 748 ms: 18 candidates, 88367 repo-hours, 936 releases; waiting 514 ms; commit 5046 ms)
2026-11-08T03:18:31Z [load_day 2023-11-28] 2023-11-28 hour 18: 258345 events in 6 files, snapshot 1337650143578486188 (task 22916 ms: download 14706 ms overlapping gunzip 1194 ms, parse 5271 ms, sort 232 ms, write 14738 ms, index 648 ms: 10 candidates, 91148 repo-hours, 1061 releases; waiting 530 ms; commit 3921 ms)
2026-11-08T03:18:34Z [load_day 2023-11-28] 2023-11-28 hour 19: 247254 events in 5 files, snapshot 1230622078830697123 (task 22126 ms: download 13801 ms overlapping gunzip 1410 ms, parse 6167 ms, sort 178 ms, write 12897 ms, index 690 ms: 15 candidates, 80460 repo-hours, 1013 releases; waiting 489 ms; commit 2541 ms)
2026-11-08T03:18:53Z [load_day 2023-11-28] 2023-11-28 hour 20: 245588 events in 5 files, snapshot 2979249103722263173 (task 21975 ms: download 13374 ms overlapping gunzip 1280 ms, parse 5697 ms, sort 260 ms, write 13296 ms, index 669 ms: 17 candidates, 79038 repo-hours, 962 releases; waiting 526 ms; commit 3212 ms)
2026-11-08T03:18:57Z [load_day 2023-11-28] 2023-11-28 hour 21: 238317 events in 5 files, snapshot 7165956232064436309 (task 20965 ms: download 12929 ms overlapping gunzip 1323 ms, parse 5661 ms, sort 173 ms, write 12240 ms, index 695 ms: 13 candidates, 74257 repo-hours, 1020 releases; waiting 544 ms; commit 3203 ms)
2026-11-08T03:19:14Z [load_day 2023-11-28] 2023-11-28 hour 22: 226095 events in 5 files, snapshot 4642334391725781517 (task 19756 ms: download 12903 ms overlapping gunzip 1147 ms, parse 4950 ms, sort 136 ms, write 11815 ms, index 426 ms: 5 candidates, 66906 repo-hours, 917 releases; waiting 936 ms; commit 3047 ms)
2026-11-08T03:19:17Z [load_day 2023-11-28] 2023-11-28 hour 23: 222722 events in 5 files, snapshot 6973458484443328418 (task 17678 ms: download 9159 ms overlapping gunzip 1085 ms, parse 4574 ms, sort 194 ms, write 11188 ms, index 422 ms: 21 candidates, 63590 repo-hours, 814 releases; waiting 19 ms; commit 2834 ms)
2026-11-08T03:19:26Z [load_day 2023-11-28] 2023-11-28: repo_hour merged from 24 files to 1 (1983562 rows, 8858 ms)
2026-11-08T03:19:28Z [load_day 2023-11-28] 2023-11-28: index committed in 11714 ms (24 + 1 + 24 files)
2026-11-08T03:19:28Z [load_day 2023-11-28] 2023-11-28: 5909857 events in 24 hours
2026-11-08T03:19:28Z [load_day 2023-11-28] attempt 1 succeeded