orcaLog in

Runs · run 1647

load_day 2023-10-20, attempt 1 succeeded

Host
3c2b29713ce8:460113
Started
2026-11-08T01:58:31Z
Ended
2026-11-08T02:02:38Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1647/load_day/2023-10-20/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:58:31Z [load_day 2023-10-20] attempt 1 started
2026-11-08T01:58:32Z [load_day 2023-10-20] 2023-10-20: 2 hours in flight, about 1200 MB
2026-11-08T01:58:53Z [load_day 2023-10-20] 2023-10-20 hour 0: 166574 events in 4 files, snapshot 2132589843427838203 (task 16837 ms: download 7494 ms overlapping gunzip 1129 ms, parse 5080 ms, sort 154 ms, write 9041 ms, index 571 ms: 11 candidates, 87576 repo-hours, 929 releases; waiting 549 ms; commit 2574 ms)
2026-11-08T01:58:56Z [load_day 2023-10-20] 2023-10-20 hour 1: 163561 events in 4 files, snapshot 960631330454868344 (task 15945 ms: download 5906 ms overlapping gunzip 846 ms, parse 3718 ms, sort 108 ms, write 9709 ms, index 661 ms: 14 candidates, 90460 repo-hours, 762 releases; waiting 558 ms; commit 2478 ms)
2026-11-08T01:59:10Z [load_day 2023-10-20] 2023-10-20 hour 2: 137889 events in 3 files, snapshot 6257228669648388524 (task 12771 ms: download 5726 ms overlapping gunzip 764 ms, parse 3401 ms, sort 120 ms, write 7062 ms, index 547 ms: 20 candidates, 60557 repo-hours, 808 releases; waiting 556 ms; commit 4692 ms)
2026-11-08T01:59:15Z [load_day 2023-10-20] 2023-10-20 hour 3: 135005 events in 3 files, snapshot 7751187775320404275 (task 11995 ms: download 5434 ms overlapping gunzip 641 ms, parse 2924 ms, sort 85 ms, write 7030 ms, index 518 ms: 6 candidates, 58812 repo-hours, 953 releases; waiting 494 ms; commit 5298 ms)
2026-11-08T01:59:23Z [load_day 2023-10-20] 2023-10-20 hour 4: 130321 events in 3 files, snapshot 4255096781888833314 (task 14448 ms: download 6109 ms overlapping gunzip 938 ms, parse 4071 ms, sort 96 ms, write 8060 ms, index 511 ms: 9 candidates, 54776 repo-hours, 700 releases; waiting 526 ms; commit 2611 ms)
2026-11-08T01:59:28Z [load_day 2023-10-20] 2023-10-20 hour 5: 138101 events in 3 files, snapshot 6383587326568895894 (task 14307 ms: download 6449 ms overlapping gunzip 829 ms, parse 3700 ms, sort 82 ms, write 8440 ms, index 431 ms: 7 candidates, 59849 repo-hours, 728 releases; waiting 511 ms; commit 2537 ms)
2026-11-08T01:59:41Z [load_day 2023-10-20] 2023-10-20 hour 6: 175879 events in 4 files, snapshot 4643780729985739407 (task 16697 ms: download 9103 ms overlapping gunzip 919 ms, parse 4074 ms, sort 113 ms, write 10324 ms, index 765 ms: 9 candidates, 78281 repo-hours, 981 releases; waiting 188 ms; commit 2971 ms)
2026-11-08T01:59:49Z [load_day 2023-10-20] 2023-10-20 hour 7: 197682 events in 4 files, snapshot 8639826186686540423 (task 19766 ms: download 10476 ms overlapping gunzip 1206 ms, parse 5341 ms, sort 120 ms, write 11683 ms, index 622 ms: 10 candidates, 77515 repo-hours, 943 releases; waiting 534 ms; commit 2768 ms)
2026-11-08T02:00:04Z [load_day 2023-10-20] 2023-10-20 hour 8: 191603 events in 4 files, snapshot 8493170523054142093 (task 20649 ms: download 11554 ms overlapping gunzip 1296 ms, parse 5671 ms, sort 115 ms, write 12077 ms, index 844 ms: 14 candidates, 83516 repo-hours, 1137 releases; waiting 353 ms; commit 3444 ms)
2026-11-08T02:00:14Z [load_day 2023-10-20] 2023-10-20 hour 9: 194190 events in 4 files, snapshot 6299491673842345339 (task 22324 ms: download 11962 ms overlapping gunzip 1272 ms, parse 5410 ms, sort 175 ms, write 13874 ms, index 659 ms: 11 candidates, 84854 repo-hours, 1257 releases; waiting 500 ms; commit 5266 ms)
2026-11-08T02:00:26Z [load_day 2023-10-20] 2023-10-20 hour 10: 183090 events in 4 files, snapshot 6853940151631681121 (task 20882 ms: download 11019 ms overlapping gunzip 1356 ms, parse 6026 ms, sort 145 ms, write 12078 ms, index 556 ms: 10 candidates, 80011 repo-hours, 1064 releases; waiting 491 ms; commit 3493 ms)
2026-11-08T02:00:35Z [load_day 2023-10-20] 2023-10-20 hour 11: 179546 events in 4 files, snapshot 8071743020788125349 (task 20655 ms: download 12376 ms overlapping gunzip 1132 ms, parse 5174 ms, sort 115 ms, write 12780 ms, index 619 ms: 11 candidates, 77112 repo-hours, 1045 releases; waiting 495 ms; commit 4143 ms)
2026-11-08T02:00:49Z [load_day 2023-10-20] 2023-10-20 hour 12: 237341 events in 5 files, snapshot 5238451496037860725 (task 20778 ms: download 13723 ms overlapping gunzip 1354 ms, parse 6012 ms, sort 187 ms, write 11710 ms, index 731 ms: 12 candidates, 110715 repo-hours, 1066 releases; waiting 509 ms; commit 4875 ms)
2026-11-08T02:00:56Z [load_day 2023-10-20] 2023-10-20 hour 13: 217927 events in 5 files, snapshot 8646802856020392057 (task 20733 ms: download 13832 ms overlapping gunzip 1335 ms, parse 5945 ms, sort 154 ms, write 11693 ms, index 815 ms: 6 candidates, 103908 repo-hours, 1242 releases; waiting 570 ms; commit 2957 ms)
2026-11-08T02:01:08Z [load_day 2023-10-20] 2023-10-20 hour 14: 207921 events in 5 files, snapshot 4460407723137550445 (task 18411 ms: download 10245 ms overlapping gunzip 1153 ms, parse 4986 ms, sort 138 ms, write 10750 ms, index 622 ms: 20 candidates, 92108 repo-hours, 1207 releases; waiting 527 ms; commit 3708 ms)
2026-11-08T02:01:15Z [load_day 2023-10-20] 2023-10-20 hour 15: 186165 events in 4 files, snapshot 3119601193136318680 (task 18848 ms: download 9212 ms overlapping gunzip 1022 ms, parse 4459 ms, sort 114 ms, write 11773 ms, index 630 ms: 23 candidates, 86141 repo-hours, 1326 releases; waiting 521 ms; commit 2920 ms)
2026-11-08T02:01:28Z [load_day 2023-10-20] 2023-10-20 hour 16: 192240 events in 4 files, snapshot 7348851306101435289 (task 19792 ms: download 11258 ms overlapping gunzip 1196 ms, parse 5103 ms, sort 169 ms, write 11934 ms, index 707 ms: 5 candidates, 84095 repo-hours, 1262 releases; waiting 373 ms; commit 2515 ms)
2026-11-08T02:01:36Z [load_day 2023-10-20] 2023-10-20 hour 17: 180943 events in 4 files, snapshot 6300123218247317595 (task 17732 ms: download 10228 ms overlapping gunzip 1004 ms, parse 4408 ms, sort 150 ms, write 10605 ms, index 711 ms: 7 candidates, 77648 repo-hours, 1079 releases; waiting 524 ms; commit 4384 ms)
2026-11-08T02:01:48Z [load_day 2023-10-20] 2023-10-20 hour 18: 187608 events in 4 files, snapshot 5942109156857477309 (task 18517 ms: download 10267 ms overlapping gunzip 1066 ms, parse 4849 ms, sort 195 ms, write 10651 ms, index 958 ms: 11 candidates, 80746 repo-hours, 1219 releases; waiting 525 ms; commit 2979 ms)
2026-11-08T02:01:54Z [load_day 2023-10-20] 2023-10-20 hour 19: 171967 events in 4 files, snapshot 6602946040247483525 (task 18318 ms: download 10802 ms overlapping gunzip 1137 ms, parse 4864 ms, sort 181 ms, write 10761 ms, index 518 ms: 8 candidates, 69046 repo-hours, 1018 releases; waiting 502 ms; commit 3699 ms)
2026-11-08T02:02:13Z [load_day 2023-10-20] 2023-10-20 hour 20: 158548 events in 4 files, snapshot 5522819793367984193 (task 17756 ms: download 6611 ms overlapping gunzip 988 ms, parse 4318 ms, sort 93 ms, write 9288 ms, index 2186 ms: 16 candidates, 65698 repo-hours, 895 releases; waiting 520 ms; commit 8640 ms)
2026-11-08T02:02:15Z [load_day 2023-10-20] 2023-10-20 hour 21: 152790 events in 4 files, snapshot 7122195434115129998 (task 14459 ms: download 9583 ms overlapping gunzip 995 ms, parse 3873 ms, sort 84 ms, write 8254 ms, index 455 ms: 16 candidates, 61348 repo-hours, 941 releases; waiting 490 ms; commit 2844 ms)
2026-11-08T02:02:25Z [load_day 2023-10-20] 2023-10-20 hour 22: 135438 events in 3 files, snapshot 5837525308380663771 (task 13736 ms: download 6326 ms overlapping gunzip 830 ms, parse 3547 ms, sort 104 ms, write 7935 ms, index 576 ms: 17 candidates, 55126 repo-hours, 798 releases; waiting 523 ms; commit 4266 ms)
2026-11-08T02:02:27Z [load_day 2023-10-20] 2023-10-20 hour 23: 119167 events in 3 files, snapshot 7047162793960671680 (task 11783 ms: download 5563 ms overlapping gunzip 816 ms, parse 3386 ms, sort 67 ms, write 6383 ms, index 363 ms: 8 candidates, 50066 repo-hours, 721 releases; waiting 519 ms; commit 2074 ms)
2026-11-08T02:02:36Z [load_day 2023-10-20] 2023-10-20: repo_hour merged from 24 files to 1 (1829964 rows, 8154 ms)
2026-11-08T02:02:38Z [load_day 2023-10-20] 2023-10-20: index committed in 10435 ms (24 + 1 + 24 files)
2026-11-08T02:02:38Z [load_day 2023-10-20] 2023-10-20: 4141496 events in 24 hours
2026-11-08T02:02:38Z [load_day 2023-10-20] attempt 1 succeeded