orcaLog in

Runs · run 1821

load_day 2023-12-05, attempt 1 succeeded

Host
3c2b29713ce8:633635
Started
2026-11-08T03:30:34Z
Ended
2026-11-08T03:35:16Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1821/load_day/2023-12-05/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:30:34Z [load_day 2023-12-05] attempt 1 started
2026-11-08T03:30:36Z [load_day 2023-12-05] 2023-12-05: 2 hours in flight, about 1200 MB
2026-11-08T03:31:00Z [load_day 2023-12-05] 2023-12-05 hour 0: 187038 events in 4 files, snapshot 759548271913163634 (task 19684 ms: download 11720 ms overlapping gunzip 1127 ms, parse 5126 ms, sort 163 ms, write 11787 ms, index 766 ms: 12 candidates, 91587 repo-hours, 1019 releases; waiting 501 ms; commit 3387 ms)
2026-11-08T03:31:03Z [load_day 2023-12-05] 2023-12-05 hour 1: 188053 events in 4 files, snapshot 6585398640802792560 (task 19264 ms: download 11166 ms overlapping gunzip 1149 ms, parse 5191 ms, sort 166 ms, write 11183 ms, index 808 ms: 8 candidates, 98254 repo-hours, 912 releases; waiting 516 ms; commit 2960 ms)
2026-11-08T03:31:18Z [load_day 2023-12-05] 2023-12-05 hour 2: 165836 events in 4 files, snapshot 8591682376257091079 (task 16357 ms: download 6760 ms overlapping gunzip 988 ms, parse 4113 ms, sort 108 ms, write 9610 ms, index 756 ms: 10 candidates, 72474 repo-hours, 769 releases; waiting 473 ms; commit 3043 ms)
2026-11-08T03:31:25Z [load_day 2023-12-05] 2023-12-05 hour 3: 163958 events in 4 files, snapshot 6543779927161145407 (task 16010 ms: download 9677 ms overlapping gunzip 1107 ms, parse 4541 ms, sort 130 ms, write 8923 ms, index 518 ms: 2 candidates, 66171 repo-hours, 933 releases; waiting 519 ms; commit 6762 ms)
2026-11-08T03:31:38Z [load_day 2023-12-05] 2023-12-05 hour 4: 156031 events in 4 files, snapshot 893845899027770776 (task 15557 ms: download 6946 ms overlapping gunzip 1065 ms, parse 4268 ms, sort 105 ms, write 8868 ms, index 474 ms: 8 candidates, 61917 repo-hours, 661 releases; waiting 554 ms; commit 5500 ms)
2026-11-08T03:31:41Z [load_day 2023-12-05] 2023-12-05 hour 5: 173695 events in 4 files, snapshot 4640424484367684937 (task 17859 ms: download 10894 ms overlapping gunzip 1361 ms, parse 5282 ms, sort 164 ms, write 9529 ms, index 689 ms: 10 candidates, 67497 repo-hours, 688 releases; waiting 563 ms; commit 2877 ms)
2026-11-08T03:31:54Z [load_day 2023-12-05] 2023-12-05 hour 6: 196653 events in 4 files, snapshot 3621940322945062338 (task 17336 ms: download 9213 ms overlapping gunzip 1020 ms, parse 4594 ms, sort 236 ms, write 10061 ms, index 517 ms: 5 candidates, 78045 repo-hours, 795 releases; waiting 553 ms; commit 3362 ms)
2026-11-08T03:32:00Z [load_day 2023-12-05] 2023-12-05 hour 7: 200468 events in 5 files, snapshot 5985665649527854982 (task 18178 ms: download 9835 ms overlapping gunzip 1236 ms, parse 5385 ms, sort 182 ms, write 10068 ms, index 589 ms: 14 candidates, 77596 repo-hours, 837 releases; waiting 478 ms; commit 3116 ms)
2026-11-08T03:32:15Z [load_day 2023-12-05] 2023-12-05 hour 8: 219026 events in 5 files, snapshot 366265665617762223 (task 20308 ms: download 14140 ms overlapping gunzip 1140 ms, parse 5144 ms, sort 161 ms, write 12433 ms, index 622 ms: 11 candidates, 87214 repo-hours, 1046 releases; waiting 514 ms; commit 2329 ms)
2026-11-08T03:32:23Z [load_day 2023-12-05] 2023-12-05 hour 9: 221224 events in 5 files, snapshot 6434494558632538318 (task 22056 ms: download 15005 ms overlapping gunzip 1245 ms, parse 5387 ms, sort 267 ms, write 13527 ms, index 673 ms: 11 candidates, 88661 repo-hours, 1212 releases; waiting 542 ms; commit 2546 ms)
2026-11-08T03:32:35Z [load_day 2023-12-05] 2023-12-05 hour 10: 234725 events in 5 files, snapshot 7745235279423168239 (task 18736 ms: download 12676 ms overlapping gunzip 1196 ms, parse 5281 ms, sort 167 ms, write 10703 ms, index 595 ms: 11 candidates, 88104 repo-hours, 1161 releases; waiting 532 ms; commit 2567 ms)
2026-11-08T03:32:44Z [load_day 2023-12-05] 2023-12-05 hour 11: 249192 events in 5 files, snapshot 5244839943213925595 (task 19376 ms: download 11053 ms overlapping gunzip 1102 ms, parse 4952 ms, sort 170 ms, write 11630 ms, index 743 ms: 6 candidates, 82281 repo-hours, 1056 releases; waiting 518 ms; commit 3540 ms)
2026-11-08T03:33:01Z [load_day 2023-12-05] 2023-12-05 hour 12: 279352 events in 6 files, snapshot 1574913886941664769 (task 22443 ms: download 13115 ms overlapping gunzip 1253 ms, parse 6267 ms, sort 209 ms, write 13092 ms, index 796 ms: 13 candidates, 110202 repo-hours, 1092 releases; waiting 532 ms; commit 5519 ms)
2026-11-08T03:33:10Z [load_day 2023-12-05] 2023-12-05 hour 13: 265102 events in 6 files, snapshot 1899747213627866505 (task 24559 ms: download 13924 ms overlapping gunzip 1500 ms, parse 6818 ms, sort 212 ms, write 14163 ms, index 1070 ms: 18 candidates, 106701 repo-hours, 996 releases; waiting 500 ms; commit 3191 ms)
2026-11-08T03:33:26Z [load_day 2023-12-05] 2023-12-05 hour 14: 259252 events in 6 files, snapshot 3484393285978782876 (task 25514 ms: download 15556 ms overlapping gunzip 1444 ms, parse 6478 ms, sort 317 ms, write 15831 ms, index 978 ms: 7 candidates, 97902 repo-hours, 1156 releases; waiting 201 ms; commit 3245 ms)
2026-11-08T03:33:36Z [load_day 2023-12-05] 2023-12-05 hour 15: 270401 events in 6 files, snapshot 577582813095450800 (task 24497 ms: download 15050 ms overlapping gunzip 1508 ms, parse 6972 ms, sort 251 ms, write 13993 ms, index 1031 ms: 20 candidates, 95839 repo-hours, 1160 releases; waiting 512 ms; commit 3949 ms)
2026-11-08T03:33:50Z [load_day 2023-12-05] 2023-12-05 hour 16: 261325 events in 6 files, snapshot 30162852971414858 (task 23596 ms: download 14142 ms overlapping gunzip 1415 ms, parse 6097 ms, sort 197 ms, write 14460 ms, index 680 ms: 19 candidates, 91725 repo-hours, 1066 releases; waiting 431 ms; commit 2950 ms)
2026-11-08T03:33:59Z [load_day 2023-12-05] 2023-12-05 hour 17: 241830 events in 5 files, snapshot 2963171719527784859 (task 22389 ms: download 14186 ms overlapping gunzip 1268 ms, parse 5662 ms, sort 173 ms, write 14094 ms, index 630 ms: 9 candidates, 87479 repo-hours, 984 releases; waiting 298 ms; commit 2968 ms)
2026-11-08T03:34:13Z [load_day 2023-12-05] 2023-12-05 hour 18: 238393 events in 5 files, snapshot 636038922828586453 (task 21732 ms: download 13583 ms overlapping gunzip 1222 ms, parse 5414 ms, sort 168 ms, write 13274 ms, index 857 ms: 7 candidates, 91089 repo-hours, 1082 releases; waiting 557 ms; commit 3115 ms)
2026-11-08T03:34:22Z [load_day 2023-12-05] 2023-12-05 hour 19: 226375 events in 5 files, snapshot 1059998250342356037 (task 20648 ms: download 12922 ms overlapping gunzip 1242 ms, parse 5678 ms, sort 159 ms, write 12007 ms, index 805 ms: 12 candidates, 80314 repo-hours, 1189 releases; waiting 216 ms; commit 4132 ms)
2026-11-08T03:34:36Z [load_day 2023-12-05] 2023-12-05 hour 20: 220242 events in 5 files, snapshot 116006399713057867 (task 21467 ms: download 11333 ms overlapping gunzip 1292 ms, parse 5890 ms, sort 199 ms, write 12841 ms, index 535 ms: 15 candidates, 78171 repo-hours, 968 releases; waiting 500 ms; commit 3266 ms)
2026-11-08T03:34:53Z [load_day 2023-12-05] 2023-12-05 hour 21: 218707 events in 5 files, snapshot 2602435216255654053 (task 21468 ms: download 13610 ms overlapping gunzip 1254 ms, parse 5501 ms, sort 248 ms, write 13377 ms, index 604 ms: 9 candidates, 76073 repo-hours, 1051 releases; waiting 209 ms; commit 12534 ms)
2026-11-08T03:35:01Z [load_day 2023-12-05] 2023-12-05 hour 22: 226623 events in 5 files, snapshot 5926429229301092231 (task 23990 ms: download 17279 ms overlapping gunzip 1396 ms, parse 6174 ms, sort 148 ms, write 14934 ms, index 495 ms: 8 candidates, 72901 repo-hours, 959 releases; waiting 522 ms; commit 3034 ms)
2026-11-08T03:35:05Z [load_day 2023-12-05] 2023-12-05 hour 23: 204080 events in 5 files, snapshot 2772824781611360485 (task 19305 ms: download 11320 ms overlapping gunzip 1270 ms, parse 5605 ms, sort 185 ms, write 10878 ms, index 586 ms: 4 candidates, 67838 repo-hours, 810 releases; waiting 514 ms; commit 3725 ms)
2026-11-08T03:35:14Z [load_day 2023-12-05] 2023-12-05: repo_hour merged from 24 files to 1 (2016035 rows, 9175 ms)
2026-11-08T03:35:16Z [load_day 2023-12-05] 2023-12-05: index committed in 11458 ms (24 + 1 + 24 files)
2026-11-08T03:35:16Z [load_day 2023-12-05] 2023-12-05: 5267581 events in 24 hours
2026-11-08T03:35:16Z [load_day 2023-12-05] attempt 1 succeeded