orcaLog in

Runs · run 1642

load_day 2023-10-19, attempt 1 succeeded

Host
3c2b29713ce8:455654
Started
2026-11-08T01:56:13Z
Ended
2026-11-08T02:00:59Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1642/load_day/2023-10-19/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:56:13Z [load_day 2023-10-19] attempt 1 started
2026-11-08T01:56:15Z [load_day 2023-10-19] 2023-10-19: 2 hours in flight, about 1200 MB
2026-11-08T01:56:37Z [load_day 2023-10-19] 2023-10-19 hour 0: 174050 events in 4 files, snapshot 1864550320637782153 (task 18076 ms: download 8157 ms overlapping gunzip 1170 ms, parse 5116 ms, sort 156 ms, write 10285 ms, index 639 ms: 12 candidates, 88034 repo-hours, 1007 releases; waiting 494 ms; commit 2409 ms)
2026-11-08T01:56:40Z [load_day 2023-10-19] 2023-10-19 hour 1: 175140 events in 4 files, snapshot 6221487535572946884 (task 18736 ms: download 10864 ms overlapping gunzip 1068 ms, parse 5142 ms, sort 201 ms, write 10738 ms, index 734 ms: 4 candidates, 93210 repo-hours, 732 releases; waiting 568 ms; commit 2725 ms)
2026-11-08T01:56:54Z [load_day 2023-10-19] 2023-10-19 hour 2: 157081 events in 4 files, snapshot 1617304122472043098 (task 15680 ms: download 6595 ms overlapping gunzip 1002 ms, parse 4201 ms, sort 157 ms, write 9174 ms, index 596 ms: 7 candidates, 65254 repo-hours, 734 releases; waiting 221 ms; commit 2742 ms)
2026-11-08T01:57:04Z [load_day 2023-10-19] 2023-10-19 hour 3: 149960 events in 3 files, snapshot 8327180828529554253 (task 15256 ms: download 6304 ms overlapping gunzip 927 ms, parse 4017 ms, sort 130 ms, write 8798 ms, index 577 ms: 10 candidates, 62892 repo-hours, 862 releases; waiting 521 ms; commit 10075 ms)
2026-11-08T01:57:11Z [load_day 2023-10-19] 2023-10-19 hour 4: 144044 events in 3 files, snapshot 4985744517723371604 (task 15884 ms: download 7417 ms overlapping gunzip 974 ms, parse 4014 ms, sort 116 ms, write 9209 ms, index 684 ms: 6 candidates, 60031 repo-hours, 675 releases; waiting 546 ms; commit 2748 ms)
2026-11-08T01:57:24Z [load_day 2023-10-19] 2023-10-19 hour 5: 152758 events in 4 files, snapshot 1702250602491235498 (task 17694 ms: download 8119 ms overlapping gunzip 1105 ms, parse 4845 ms, sort 156 ms, write 9974 ms, index 797 ms: 7 candidates, 66004 repo-hours, 736 releases; waiting 523 ms; commit 11421 ms)
2026-11-08T01:57:53Z [load_day 2023-10-19] 2023-10-19 hour 6: 181455 events in 4 files, snapshot 3716893512121689062 (task 18564 ms: download 10724 ms overlapping gunzip 1080 ms, parse 4700 ms, sort 157 ms, write 11207 ms, index 609 ms: 10 candidates, 84275 repo-hours, 941 releases; waiting 541 ms; commit 24265 ms)
2026-11-08T01:57:57Z [load_day 2023-10-19] 2023-10-19 hour 7: 184659 events in 4 files, snapshot 4344584583185012375 (task 40949 ms: download 11044 ms overlapping gunzip 1130 ms, parse 5080 ms, sort 142 ms, write 33097 ms, index 654 ms: 4 candidates, 82380 repo-hours, 881 releases; waiting 530 ms; commit 2315 ms)
2026-11-08T01:58:00Z [load_day 2023-10-19] 2023-10-19 hour 8: 191800 events in 4 files, snapshot 8156514952979834027 (task 18206 ms: download 10510 ms overlapping gunzip 1166 ms, parse 5074 ms, sort 115 ms, write 10496 ms, index 617 ms: 13 candidates, 86955 repo-hours, 1074 releases; waiting 505 ms; commit 2696 ms)
2026-11-08T01:58:20Z [load_day 2023-10-19] 2023-10-19 hour 9: 196280 events in 4 files, snapshot 3401723112628731836 (task 21093 ms: download 12823 ms overlapping gunzip 1195 ms, parse 5198 ms, sort 143 ms, write 13178 ms, index 567 ms: 10 candidates, 88159 repo-hours, 1212 releases; waiting 520 ms; commit 3084 ms)
2026-11-08T01:58:23Z [load_day 2023-10-19] 2023-10-19 hour 10: 201678 events in 5 files, snapshot 3942406146055755908 (task 20514 ms: download 12677 ms overlapping gunzip 1346 ms, parse 6126 ms, sort 137 ms, write 11524 ms, index 611 ms: 8 candidates, 88933 repo-hours, 1165 releases; waiting 490 ms; commit 2473 ms)
2026-11-08T01:58:41Z [load_day 2023-10-19] 2023-10-19 hour 11: 204316 events in 5 files, snapshot 329299666250237562 (task 20159 ms: download 11028 ms overlapping gunzip 1235 ms, parse 5363 ms, sort 137 ms, write 11927 ms, index 692 ms: 9 candidates, 88316 repo-hours, 1155 releases; waiting 524 ms; commit 2488 ms)
2026-11-08T01:58:45Z [load_day 2023-10-19] 2023-10-19 hour 12: 249189 events in 5 files, snapshot 4240105472244123476 (task 21296 ms: download 12106 ms overlapping gunzip 1183 ms, parse 5255 ms, sort 174 ms, write 12946 ms, index 1167 ms: 7 candidates, 117742 repo-hours, 1228 releases; waiting 205 ms; commit 2387 ms)
2026-11-08T01:59:07Z [load_day 2023-10-19] 2023-10-19 hour 13: 228196 events in 5 files, snapshot 2130986526845701718 (task 24859 ms: download 15622 ms overlapping gunzip 1570 ms, parse 6780 ms, sort 136 ms, write 14785 ms, index 809 ms: 18 candidates, 113516 repo-hours, 1190 releases; waiting 535 ms; commit 2674 ms)
2026-11-08T01:59:12Z [load_day 2023-10-19] 2023-10-19 hour 14: 207023 events in 5 files, snapshot 6907771474424510006 (task 22844 ms: download 12542 ms overlapping gunzip 1365 ms, parse 6259 ms, sort 154 ms, write 13592 ms, index 772 ms: 12 candidates, 96274 repo-hours, 1192 releases; waiting 476 ms; commit 5688 ms)
2026-11-08T01:59:31Z [load_day 2023-10-19] 2023-10-19 hour 15: 177179 events in 4 files, snapshot 4413422255316645596 (task 20626 ms: download 12173 ms overlapping gunzip 1196 ms, parse 5351 ms, sort 117 ms, write 12548 ms, index 694 ms: 13 candidates, 86960 repo-hours, 1163 releases; waiting 509 ms; commit 4753 ms)
2026-11-08T01:59:33Z [load_day 2023-10-19] 2023-10-19 hour 16: 181478 events in 4 files, snapshot 5863315516997882784 (task 19691 ms: download 11774 ms overlapping gunzip 1228 ms, parse 5375 ms, sort 161 ms, write 11411 ms, index 757 ms: 16 candidates, 83719 repo-hours, 974 releases; waiting 470 ms; commit 2029 ms)
2026-11-08T01:59:51Z [load_day 2023-10-19] 2023-10-19 hour 17: 189553 events in 4 files, snapshot 5087609432835496544 (task 19522 ms: download 11145 ms overlapping gunzip 1298 ms, parse 5661 ms, sort 219 ms, write 10827 ms, index 655 ms: 19 candidates, 83120 repo-hours, 1099 releases; waiting 525 ms; commit 4856 ms)
2026-11-08T01:59:55Z [load_day 2023-10-19] 2023-10-19 hour 18: 193599 events in 4 files, snapshot 9221810987800400379 (task 20517 ms: download 11194 ms overlapping gunzip 1193 ms, parse 5343 ms, sort 130 ms, write 12246 ms, index 868 ms: 15 candidates, 87101 repo-hours, 1004 releases; waiting 485 ms; commit 2669 ms)
2026-11-08T02:00:11Z [load_day 2023-10-19] 2023-10-19 hour 19: 185623 events in 4 files, snapshot 8406650570250963961 (task 21280 ms: download 11831 ms overlapping gunzip 1202 ms, parse 5565 ms, sort 149 ms, write 12940 ms, index 604 ms: 9 candidates, 76664 repo-hours, 985 releases; waiting 509 ms; commit 2494 ms)
2026-11-08T02:00:18Z [load_day 2023-10-19] 2023-10-19 hour 20: 185863 events in 4 files, snapshot 597600734926827230 (task 21254 ms: download 11703 ms overlapping gunzip 1292 ms, parse 5489 ms, sort 140 ms, write 12660 ms, index 897 ms: 15 candidates, 77489 repo-hours, 1004 releases; waiting 503 ms; commit 3264 ms)
2026-11-08T02:00:33Z [load_day 2023-10-19] 2023-10-19 hour 21: 169564 events in 4 files, snapshot 6383916241596873652 (task 20540 ms: download 12959 ms overlapping gunzip 1229 ms, parse 5902 ms, sort 221 ms, write 11452 ms, index 532 ms: 14 candidates, 69123 repo-hours, 1083 releases; waiting 602 ms; commit 2542 ms)
2026-11-08T02:00:37Z [load_day 2023-10-19] 2023-10-19 hour 22: 147182 events in 3 files, snapshot 2048179568981833184 (task 16052 ms: download 7330 ms overlapping gunzip 902 ms, parse 3813 ms, sort 182 ms, write 9920 ms, index 403 ms: 14 candidates, 58778 repo-hours, 942 releases; waiting 555 ms; commit 4239 ms)
2026-11-08T02:00:47Z [load_day 2023-10-19] 2023-10-19 hour 23: 130791 events in 3 files, snapshot 2538959903601616714 (task 12656 ms: download 5484 ms overlapping gunzip 734 ms, parse 3183 ms, sort 105 ms, write 7632 ms, index 444 ms: 9 candidates, 54543 repo-hours, 739 releases; waiting 184 ms; commit 2436 ms)
2026-11-08T02:00:56Z [load_day 2023-10-19] 2023-10-19: repo_hour merged from 24 files to 1 (1959472 rows, 9338 ms)
2026-11-08T02:00:59Z [load_day 2023-10-19] 2023-10-19: index committed in 11945 ms (24 + 1 + 24 files)
2026-11-08T02:00:59Z [load_day 2023-10-19] 2023-10-19: 4358461 events in 24 hours
2026-11-08T02:00:59Z [load_day 2023-10-19] attempt 1 succeeded