orcaLog in

Runs · run 1590

load_day 2023-10-05, attempt 1 succeeded

Host
3c2b29713ce8:403586
Started
2026-11-08T01:28:49Z
Ended
2026-11-08T01:32:59Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1590/load_day/2023-10-05/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:28:49Z [load_day 2023-10-05] attempt 1 started
2026-11-08T01:28:51Z [load_day 2023-10-05] 2023-10-05: 2 hours in flight, about 1200 MB
2026-11-08T01:29:11Z [load_day 2023-10-05] 2023-10-05 hour 0: 161142 events in 4 files, snapshot 6752703847815535146 (task 15908 ms: download 7216 ms overlapping gunzip 1084 ms, parse 4756 ms, sort 146 ms, write 8487 ms, index 611 ms: 7 candidates, 84437 repo-hours, 939 releases; waiting 573 ms; commit 2725 ms)
2026-11-08T01:29:14Z [load_day 2023-10-05] 2023-10-05 hour 1: 146854 events in 3 files, snapshot 9005669128734616744 (task 14444 ms: download 5559 ms overlapping gunzip 823 ms, parse 3729 ms, sort 161 ms, write 8335 ms, index 635 ms: 6 candidates, 81436 repo-hours, 719 releases; waiting 436 ms; commit 2837 ms)
2026-11-08T01:29:25Z [load_day 2023-10-05] 2023-10-05 hour 2: 127473 events in 3 files, snapshot 5843214767633323022 (task 11597 ms: download 5018 ms overlapping gunzip 782 ms, parse 3129 ms, sort 92 ms, write 6478 ms, index 387 ms: 6 candidates, 54249 repo-hours, 626 releases; waiting 507 ms; commit 3812 ms)
2026-11-08T01:29:27Z [load_day 2023-10-05] 2023-10-05 hour 3: 129903 events in 3 files, snapshot 7809264936507932409 (task 11266 ms: download 4695 ms overlapping gunzip 616 ms, parse 2642 ms, sort 104 ms, write 6518 ms, index 575 ms: 8 candidates, 54299 repo-hours, 724 releases; waiting 522 ms; commit 2728 ms)
2026-11-08T01:29:38Z [load_day 2023-10-05] 2023-10-05 hour 4: 134787 events in 3 files, snapshot 8803979734379822650 (task 12737 ms: download 5083 ms overlapping gunzip 851 ms, parse 3812 ms, sort 110 ms, write 6901 ms, index 521 ms: 6 candidates, 55009 repo-hours, 654 releases; waiting 219 ms; commit 2998 ms)
2026-11-08T01:29:44Z [load_day 2023-10-05] 2023-10-05 hour 5: 143433 events in 3 files, snapshot 3402326865167093302 (task 13679 ms: download 5639 ms overlapping gunzip 741 ms, parse 3115 ms, sort 99 ms, write 8294 ms, index 643 ms: 7 candidates, 59984 repo-hours, 762 releases; waiting 527 ms; commit 4426 ms)
2026-11-08T01:29:55Z [load_day 2023-10-05] 2023-10-05 hour 6: 170725 events in 4 files, snapshot 5575347896111287160 (task 16496 ms: download 9283 ms overlapping gunzip 909 ms, parse 3976 ms, sort 154 ms, write 10085 ms, index 669 ms: 3 candidates, 74218 repo-hours, 806 releases; waiting 482 ms; commit 3096 ms)
2026-11-08T01:30:03Z [load_day 2023-10-05] 2023-10-05 hour 7: 172958 events in 4 files, snapshot 8928323935799443372 (task 19289 ms: download 11065 ms overlapping gunzip 1232 ms, parse 5386 ms, sort 157 ms, write 11264 ms, index 546 ms: 7 candidates, 74735 repo-hours, 864 releases; waiting 472 ms; commit 2979 ms)
2026-11-08T01:30:18Z [load_day 2023-10-05] 2023-10-05 hour 8: 185406 events in 4 files, snapshot 2257143801628998659 (task 22136 ms: download 12414 ms overlapping gunzip 1427 ms, parse 5794 ms, sort 143 ms, write 13311 ms, index 644 ms: 9 candidates, 80548 repo-hours, 923 releases; waiting 491 ms; commit 2781 ms)
2026-11-08T01:30:26Z [load_day 2023-10-05] 2023-10-05 hour 9: 195901 events in 4 files, snapshot 3437558361809905253 (task 22359 ms: download 12778 ms overlapping gunzip 1408 ms, parse 6356 ms, sort 185 ms, write 12834 ms, index 786 ms: 11 candidates, 83568 repo-hours, 1102 releases; waiting 444 ms; commit 2445 ms)
2026-11-08T01:30:39Z [load_day 2023-10-05] 2023-10-05 hour 10: 193734 events in 4 files, snapshot 1987317254433653789 (task 19585 ms: download 10498 ms overlapping gunzip 1134 ms, parse 5041 ms, sort 184 ms, write 11452 ms, index 828 ms: 4 candidates, 82682 repo-hours, 886 releases; waiting 630 ms; commit 2847 ms)
2026-11-08T01:30:47Z [load_day 2023-10-05] 2023-10-05 hour 11: 189107 events in 4 files, snapshot 3935107128755443923 (task 19977 ms: download 11189 ms overlapping gunzip 1106 ms, parse 4850 ms, sort 119 ms, write 12192 ms, index 572 ms: 8 candidates, 79940 repo-hours, 948 releases; waiting 541 ms; commit 2382 ms)
2026-11-08T01:31:02Z [load_day 2023-10-05] 2023-10-05 hour 12: 238268 events in 5 files, snapshot 1285612792303142785 (task 21936 ms: download 13472 ms overlapping gunzip 1203 ms, parse 5659 ms, sort 180 ms, write 13101 ms, index 955 ms: 6 candidates, 113692 repo-hours, 1115 releases; waiting 512 ms; commit 2591 ms)
2026-11-08T01:31:10Z [load_day 2023-10-05] 2023-10-05 hour 13: 220330 events in 5 files, snapshot 7282628010544613891 (task 22208 ms: download 15391 ms overlapping gunzip 1174 ms, parse 5411 ms, sort 135 ms, write 13967 ms, index 706 ms: 9 candidates, 108627 repo-hours, 1018 releases; waiting 513 ms; commit 2188 ms)
2026-11-08T01:31:22Z [load_day 2023-10-05] 2023-10-05 hour 14: 216348 events in 5 files, snapshot 79989625708114291 (task 19433 ms: download 13575 ms overlapping gunzip 1174 ms, parse 5021 ms, sort 129 ms, write 11697 ms, index 659 ms: 16 candidates, 97776 repo-hours, 1239 releases; waiting 511 ms; commit 2305 ms)
2026-11-08T01:31:31Z [load_day 2023-10-05] 2023-10-05 hour 15: 195221 events in 4 files, snapshot 816303383512007439 (task 20052 ms: download 10468 ms overlapping gunzip 1190 ms, parse 5012 ms, sort 186 ms, write 11821 ms, index 667 ms: 9 candidates, 92254 repo-hours, 1182 releases; waiting 781 ms; commit 2841 ms)
2026-11-08T01:31:46Z [load_day 2023-10-05] 2023-10-05 hour 16: 193071 events in 4 files, snapshot 4129174950361398371 (task 22649 ms: download 13184 ms overlapping gunzip 1152 ms, parse 5215 ms, sort 178 ms, write 14121 ms, index 1163 ms: 5 candidates, 88033 repo-hours, 988 releases; waiting 524 ms; commit 2547 ms)
2026-11-08T01:31:52Z [load_day 2023-10-05] 2023-10-05 hour 17: 193107 events in 4 files, snapshot 2914848556768819478 (task 19720 ms: download 10508 ms overlapping gunzip 1163 ms, parse 5072 ms, sort 202 ms, write 11851 ms, index 644 ms: 5 candidates, 84298 repo-hours, 898 releases; waiting 477 ms; commit 2352 ms)
2026-11-08T01:32:10Z [load_day 2023-10-05] 2023-10-05 hour 18: 201083 events in 5 files, snapshot 6928460438639689001 (task 21647 ms: download 11346 ms overlapping gunzip 1262 ms, parse 5611 ms, sort 172 ms, write 13143 ms, index 639 ms: 17 candidates, 90072 repo-hours, 1156 releases; waiting 520 ms; commit 3040 ms)
2026-11-08T01:32:15Z [load_day 2023-10-05] 2023-10-05 hour 19: 185285 events in 4 files, snapshot 6258071538130206440 (task 20443 ms: download 12053 ms overlapping gunzip 1323 ms, parse 5508 ms, sort 136 ms, write 12099 ms, index 606 ms: 9 candidates, 78208 repo-hours, 966 releases; waiting 538 ms; commit 3674 ms)
2026-11-08T01:32:30Z [load_day 2023-10-05] 2023-10-05 hour 20: 174145 events in 4 files, snapshot 4766678734977902565 (task 18675 ms: download 10869 ms overlapping gunzip 1184 ms, parse 5444 ms, sort 130 ms, write 10438 ms, index 667 ms: 6 candidates, 74317 repo-hours, 1110 releases; waiting 507 ms; commit 4161 ms)
2026-11-08T01:32:36Z [load_day 2023-10-05] 2023-10-05 hour 21: 159857 events in 4 files, snapshot 3317775944641165706 (task 16797 ms: download 10266 ms overlapping gunzip 1023 ms, parse 4452 ms, sort 146 ms, write 9922 ms, index 423 ms: 2 candidates, 64857 repo-hours, 1070 releases; waiting 512 ms; commit 5544 ms)
2026-11-08T01:32:47Z [load_day 2023-10-05] 2023-10-05 hour 22: 145051 events in 3 files, snapshot 3465858199130860623 (task 15395 ms: download 7407 ms overlapping gunzip 859 ms, parse 3623 ms, sort 95 ms, write 9454 ms, index 438 ms: 12 candidates, 57702 repo-hours, 880 releases; waiting 649 ms; commit 2356 ms)
2026-11-08T01:32:49Z [load_day 2023-10-05] 2023-10-05 hour 23: 130626 events in 3 files, snapshot 3251978644800276238 (task 12844 ms: download 6135 ms overlapping gunzip 805 ms, parse 3554 ms, sort 101 ms, write 7142 ms, index 411 ms: 7 candidates, 53282 repo-hours, 788 releases; waiting 490 ms; commit 2058 ms)
2026-11-08T01:32:57Z [load_day 2023-10-05] 2023-10-05: repo_hour merged from 24 files to 1 (1868223 rows, 8294 ms)
2026-11-08T01:32:59Z [load_day 2023-10-05] 2023-10-05: index committed in 10510 ms (24 + 1 + 24 files)
2026-11-08T01:32:59Z [load_day 2023-10-05] 2023-10-05: 4203815 events in 24 hours
2026-11-08T01:32:59Z [load_day 2023-10-05] attempt 1 succeeded