orcaLog in

Runs · run 1452

load_day 2023-09-01, attempt 1 succeeded

Host
3c2b29713ce8:285109
Started
2026-11-08T00:25:27Z
Ended
2026-11-08T00:29:24Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1452/load_day/2023-09-01/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:25:27Z [load_day 2023-09-01] attempt 1 started
2026-11-08T00:25:29Z [load_day 2023-09-01] 2023-09-01: 2 hours in flight, about 1200 MB
2026-11-08T00:25:50Z [load_day 2023-09-01] 2023-09-01 hour 0: 167927 events in 4 files, snapshot 7956619038444948401 (task 16938 ms: download 7377 ms overlapping gunzip 965 ms, parse 4526 ms, sort 119 ms, write 9921 ms, index 687 ms: 10 candidates, 83647 repo-hours, 1005 releases; waiting 486 ms; commit 2209 ms)
2026-11-08T00:25:53Z [load_day 2023-09-01] 2023-09-01 hour 1: 168063 events in 4 files, snapshot 3463145188465612188 (task 16099 ms: download 6818 ms overlapping gunzip 963 ms, parse 4106 ms, sort 146 ms, write 9263 ms, index 742 ms: 7 candidates, 89959 repo-hours, 1087 releases; waiting 593 ms; commit 3329 ms)
2026-11-08T00:26:05Z [load_day 2023-09-01] 2023-09-01 hour 2: 144347 events in 3 files, snapshot 2239770244750528920 (task 14870 ms: download 6719 ms overlapping gunzip 798 ms, parse 3344 ms, sort 125 ms, write 9309 ms, index 479 ms: 3 candidates, 58211 repo-hours, 742 releases; waiting 517 ms; commit 2138 ms)
2026-11-08T00:26:10Z [load_day 2023-09-01] 2023-09-01 hour 3: 134515 events in 3 files, snapshot 7569887462233176369 (task 12972 ms: download 5686 ms overlapping gunzip 859 ms, parse 3773 ms, sort 88 ms, write 7094 ms, index 431 ms: 2 candidates, 54306 repo-hours, 818 releases; waiting 510 ms; commit 5051 ms)
2026-11-08T00:26:19Z [load_day 2023-09-01] 2023-09-01 hour 4: 126780 events in 3 files, snapshot 1753363930166224185 (task 13030 ms: download 6441 ms overlapping gunzip 754 ms, parse 3093 ms, sort 80 ms, write 7927 ms, index 338 ms: 3 candidates, 50813 repo-hours, 654 releases; waiting 536 ms; commit 2257 ms)
2026-11-08T00:26:24Z [load_day 2023-09-01] 2023-09-01 hour 5: 138074 events in 3 files, snapshot 3471444247780585675 (task 14114 ms: download 6405 ms overlapping gunzip 988 ms, parse 4217 ms, sort 95 ms, write 7565 ms, index 467 ms: 5 candidates, 58227 repo-hours, 801 releases; waiting 494 ms; commit 3059 ms)
2026-11-08T00:26:38Z [load_day 2023-09-01] 2023-09-01 hour 6: 164900 events in 4 files, snapshot 784043475666818392 (task 16857 ms: download 9450 ms overlapping gunzip 1070 ms, parse 4571 ms, sort 147 ms, write 9583 ms, index 624 ms: 5 candidates, 73952 repo-hours, 889 releases; waiting 515 ms; commit 2723 ms)
2026-11-08T00:26:45Z [load_day 2023-09-01] 2023-09-01 hour 7: 170968 events in 4 files, snapshot 2328920134067565182 (task 18187 ms: download 11332 ms overlapping gunzip 1171 ms, parse 5257 ms, sort 116 ms, write 10289 ms, index 533 ms: 3 candidates, 72402 repo-hours, 956 releases; waiting 524 ms; commit 3890 ms)
2026-11-08T00:26:58Z [load_day 2023-09-01] 2023-09-01 hour 8: 181646 events in 4 files, snapshot 261133886800003792 (task 20056 ms: download 11610 ms overlapping gunzip 1260 ms, parse 6111 ms, sort 162 ms, write 11280 ms, index 569 ms: 5 candidates, 78342 repo-hours, 1124 releases; waiting 373 ms; commit 2184 ms)
2026-11-08T00:27:06Z [load_day 2023-09-01] 2023-09-01 hour 9: 183653 events in 4 files, snapshot 1491564616894356274 (task 19255 ms: download 11429 ms overlapping gunzip 1235 ms, parse 5478 ms, sort 196 ms, write 10814 ms, index 802 ms: 3 candidates, 77557 repo-hours, 1283 releases; waiting 435 ms; commit 4916 ms)
2026-11-08T00:27:19Z [load_day 2023-09-01] 2023-09-01 hour 10: 174994 events in 4 files, snapshot 2294043839866743242 (task 18724 ms: download 11427 ms overlapping gunzip 1079 ms, parse 4733 ms, sort 119 ms, write 11496 ms, index 524 ms: 3 candidates, 73861 repo-hours, 1029 releases; waiting 510 ms; commit 3223 ms)
2026-11-08T00:27:23Z [load_day 2023-09-01] 2023-09-01 hour 11: 180727 events in 4 files, snapshot 2920629038425151518 (task 17516 ms: download 11064 ms overlapping gunzip 1079 ms, parse 4693 ms, sort 118 ms, write 10372 ms, index 521 ms: 5 candidates, 72993 repo-hours, 998 releases; waiting 518 ms; commit 2504 ms)
2026-11-08T00:27:37Z [load_day 2023-09-01] 2023-09-01 hour 12: 230194 events in 5 files, snapshot 2117226362893646192 (task 17601 ms: download 8985 ms overlapping gunzip 999 ms, parse 4488 ms, sort 137 ms, write 10294 ms, index 924 ms: 5 candidates, 105242 repo-hours, 1116 releases; waiting 535 ms; commit 2936 ms)
2026-11-08T00:27:43Z [load_day 2023-09-01] 2023-09-01 hour 13: 201118 events in 5 files, snapshot 219774886177605591 (task 19095 ms: download 9073 ms overlapping gunzip 1187 ms, parse 5085 ms, sort 146 ms, write 10901 ms, index 1015 ms: 6 candidates, 96674 repo-hours, 1164 releases; waiting 503 ms; commit 2650 ms)
2026-11-08T00:27:59Z [load_day 2023-09-01] 2023-09-01 hour 14: 193475 events in 4 files, snapshot 5871198095939020394 (task 20132 ms: download 10940 ms overlapping gunzip 1133 ms, parse 4930 ms, sort 159 ms, write 12453 ms, index 674 ms: 3 candidates, 84425 repo-hours, 1152 releases; waiting 522 ms; commit 2914 ms)
2026-11-08T00:28:05Z [load_day 2023-09-01] 2023-09-01 hour 15: 185958 events in 4 files, snapshot 2998807944914538605 (task 20728 ms: download 11442 ms overlapping gunzip 1183 ms, parse 5159 ms, sort 237 ms, write 12724 ms, index 609 ms: 4 candidates, 81931 repo-hours, 1237 releases; waiting 508 ms; commit 2669 ms)
2026-11-08T00:28:20Z [load_day 2023-09-01] 2023-09-01 hour 16: 179492 events in 4 files, snapshot 8151524779477057467 (task 20378 ms: download 11682 ms overlapping gunzip 1186 ms, parse 5037 ms, sort 172 ms, write 12904 ms, index 593 ms: 4 candidates, 76525 repo-hours, 1075 releases; waiting 194 ms; commit 2875 ms)
2026-11-08T00:28:26Z [load_day 2023-09-01] 2023-09-01 hour 17: 174876 events in 4 files, snapshot 6686157168330958768 (task 19973 ms: download 11335 ms overlapping gunzip 1173 ms, parse 5086 ms, sort 178 ms, write 11973 ms, index 748 ms: 3 candidates, 71438 repo-hours, 1015 releases; waiting 524 ms; commit 2804 ms)
2026-11-08T00:28:40Z [load_day 2023-09-01] 2023-09-01 hour 18: 174810 events in 4 files, snapshot 5641568339428564883 (task 18745 ms: download 10480 ms overlapping gunzip 931 ms, parse 4185 ms, sort 228 ms, write 11894 ms, index 693 ms: 7 candidates, 75709 repo-hours, 1110 releases; waiting 529 ms; commit 2681 ms)
2026-11-08T00:28:44Z [load_day 2023-09-01] 2023-09-01 hour 19: 157351 events in 4 files, snapshot 3440900732313493390 (task 16619 ms: download 7345 ms overlapping gunzip 1071 ms, parse 4902 ms, sort 152 ms, write 8927 ms, index 649 ms: 2 candidates, 63491 repo-hours, 1003 releases; waiting 485 ms; commit 3412 ms)
2026-11-08T00:28:59Z [load_day 2023-09-01] 2023-09-01 hour 20: 160159 events in 4 files, snapshot 7051111186738678580 (task 16289 ms: download 10294 ms overlapping gunzip 1060 ms, parse 4510 ms, sort 194 ms, write 9466 ms, index 522 ms: 2 candidates, 64229 repo-hours, 907 releases; waiting 222 ms; commit 4647 ms)
2026-11-08T00:29:02Z [load_day 2023-09-01] 2023-09-01 hour 21: 140400 events in 3 files, snapshot 1544398360011090225 (task 15490 ms: download 7034 ms overlapping gunzip 864 ms, parse 3805 ms, sort 197 ms, write 9070 ms, index 645 ms: 3 candidates, 56337 repo-hours, 925 releases; waiting 525 ms; commit 2459 ms)
2026-11-08T00:29:13Z [load_day 2023-09-01] 2023-09-01 hour 22: 130089 events in 3 files, snapshot 1152429771339117979 (task 13954 ms: download 6335 ms overlapping gunzip 872 ms, parse 3934 ms, sort 87 ms, write 7815 ms, index 369 ms: 4 candidates, 50308 repo-hours, 869 releases; waiting 530 ms; commit 2453 ms)
2026-11-08T00:29:15Z [load_day 2023-09-01] 2023-09-01 hour 23: 120375 events in 3 files, snapshot 4779472270148327765 (task 11927 ms: download 5341 ms overlapping gunzip 702 ms, parse 3037 ms, sort 73 ms, write 6994 ms, index 322 ms: 2 candidates, 46380 repo-hours, 677 releases; waiting 516 ms; commit 2158 ms)
2026-11-08T00:29:22Z [load_day 2023-09-01] 2023-09-01: repo_hour merged from 24 files to 1 (1716959 rows, 7484 ms)
2026-11-08T00:29:24Z [load_day 2023-09-01] 2023-09-01: index committed in 9627 ms (24 + 1 + 24 files)
2026-11-08T00:29:24Z [load_day 2023-09-01] 2023-09-01: 3984891 events in 24 hours
2026-11-08T00:29:24Z [load_day 2023-09-01] attempt 1 succeeded