orcaLog in

Runs · run 1609

load_day 2023-10-10, attempt 1 succeeded

Host
3c2b29713ce8:420759
Started
2026-11-08T01:38:01Z
Ended
2026-11-08T01:42:16Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1609/load_day/2023-10-10/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:38:01Z [load_day 2023-10-10] attempt 1 started
2026-11-08T01:38:03Z [load_day 2023-10-10] 2023-10-10: 2 hours in flight, about 1200 MB
2026-11-08T01:38:24Z [load_day 2023-10-10] 2023-10-10 hour 0: 170436 events in 4 files, snapshot 2487557675595296505 (task 16728 ms: download 6963 ms overlapping gunzip 1020 ms, parse 4843 ms, sort 121 ms, write 9421 ms, index 561 ms: 5 candidates, 91262 repo-hours, 1153 releases; waiting 517 ms; commit 2804 ms)
2026-11-08T01:38:27Z [load_day 2023-10-10] 2023-10-10 hour 1: 171364 events in 4 files, snapshot 1689325889739657295 (task 16142 ms: download 6103 ms overlapping gunzip 846 ms, parse 3926 ms, sort 120 ms, write 9645 ms, index 785 ms: 4 candidates, 92765 repo-hours, 951 releases; waiting 506 ms; commit 2883 ms)
2026-11-08T01:38:38Z [load_day 2023-10-10] 2023-10-10 hour 2: 143226 events in 3 files, snapshot 5344764637340353041 (task 13157 ms: download 5459 ms overlapping gunzip 803 ms, parse 3530 ms, sort 136 ms, write 7364 ms, index 479 ms: 10 candidates, 62440 repo-hours, 857 releases; waiting 550 ms; commit 2538 ms)
2026-11-08T01:38:44Z [load_day 2023-10-10] 2023-10-10 hour 3: 137695 events in 3 files, snapshot 7167090766608683694 (task 13280 ms: download 5761 ms overlapping gunzip 692 ms, parse 3083 ms, sort 153 ms, write 7950 ms, index 626 ms: 4 candidates, 59770 repo-hours, 1013 releases; waiting 527 ms; commit 5096 ms)
2026-11-08T01:38:53Z [load_day 2023-10-10] 2023-10-10 hour 4: 136403 events in 3 files, snapshot 5625180627203533405 (task 13480 ms: download 5157 ms overlapping gunzip 814 ms, parse 3546 ms, sort 166 ms, write 7708 ms, index 458 ms: 6 candidates, 56287 repo-hours, 721 releases; waiting 476 ms; commit 2660 ms)
2026-11-08T01:38:57Z [load_day 2023-10-10] 2023-10-10 hour 5: 147778 events in 3 files, snapshot 529203473799323686 (task 14431 ms: download 6048 ms overlapping gunzip 836 ms, parse 3549 ms, sort 101 ms, write 8529 ms, index 677 ms: 4 candidates, 62121 repo-hours, 666 releases; waiting 513 ms; commit 2620 ms)
2026-11-08T01:39:12Z [load_day 2023-10-10] 2023-10-10 hour 6: 178743 events in 4 files, snapshot 8477115907222149206 (task 17885 ms: download 10281 ms overlapping gunzip 1011 ms, parse 4465 ms, sort 167 ms, write 10853 ms, index 576 ms: 9 candidates, 80667 repo-hours, 933 releases; waiting 522 ms; commit 2863 ms)
2026-11-08T01:39:19Z [load_day 2023-10-10] 2023-10-10 hour 7: 175393 events in 4 files, snapshot 646753536112059722 (task 19736 ms: download 11495 ms overlapping gunzip 1245 ms, parse 5342 ms, sort 111 ms, write 11626 ms, index 647 ms: 4 candidates, 79089 repo-hours, 1127 releases; waiting 481 ms; commit 3037 ms)
2026-11-08T01:39:34Z [load_day 2023-10-10] 2023-10-10 hour 8: 187524 events in 4 files, snapshot 5807598481763049304 (task 20395 ms: download 11266 ms overlapping gunzip 1111 ms, parse 5031 ms, sort 142 ms, write 12741 ms, index 590 ms: 12 candidates, 85213 repo-hours, 1228 releases; waiting 513 ms; commit 2997 ms)
2026-11-08T01:39:41Z [load_day 2023-10-10] 2023-10-10 hour 9: 195141 events in 4 files, snapshot 7184636499346816750 (task 21292 ms: download 11702 ms overlapping gunzip 1422 ms, parse 6434 ms, sort 150 ms, write 11747 ms, index 651 ms: 7 candidates, 87046 repo-hours, 1243 releases; waiting 510 ms; commit 2765 ms)
2026-11-08T01:39:54Z [load_day 2023-10-10] 2023-10-10 hour 10: 187931 events in 4 files, snapshot 8562391549001219921 (task 19966 ms: download 11140 ms overlapping gunzip 1151 ms, parse 5337 ms, sort 164 ms, write 11825 ms, index 706 ms: 10 candidates, 83306 repo-hours, 1012 releases; waiting 515 ms; commit 2226 ms)
2026-11-08T01:40:01Z [load_day 2023-10-10] 2023-10-10 hour 11: 188368 events in 4 files, snapshot 7081373866476303066 (task 19515 ms: download 10657 ms overlapping gunzip 1105 ms, parse 4986 ms, sort 118 ms, write 11939 ms, index 569 ms: 11 candidates, 82235 repo-hours, 1128 releases; waiting 499 ms; commit 2218 ms)
2026-11-08T01:40:14Z [load_day 2023-10-10] 2023-10-10 hour 12: 234727 events in 5 files, snapshot 4011666746783687566 (task 19006 ms: download 12410 ms overlapping gunzip 1036 ms, parse 4694 ms, sort 145 ms, write 11498 ms, index 749 ms: 6 candidates, 112928 repo-hours, 1207 releases; waiting 512 ms; commit 2432 ms)
2026-11-08T01:40:28Z [load_day 2023-10-10] 2023-10-10 hour 13: 222993 events in 5 files, snapshot 7518078307483071354 (task 18937 ms: download 12092 ms overlapping gunzip 1135 ms, parse 4916 ms, sort 146 ms, write 11399 ms, index 803 ms: 11 candidates, 108393 repo-hours, 1111 releases; waiting 297 ms; commit 9245 ms)
2026-11-08T01:40:38Z [load_day 2023-10-10] 2023-10-10 hour 14: 209395 events in 5 files, snapshot 8650988059861172556 (task 23473 ms: download 16259 ms overlapping gunzip 1507 ms, parse 6686 ms, sort 146 ms, write 13463 ms, index 845 ms: 12 candidates, 94319 repo-hours, 1136 releases; waiting 483 ms; commit 2547 ms)
2026-11-08T01:40:46Z [load_day 2023-10-10] 2023-10-10 hour 15: 188322 events in 4 files, snapshot 7179065056533410490 (task 22661 ms: download 14058 ms overlapping gunzip 1414 ms, parse 6257 ms, sort 157 ms, write 13349 ms, index 768 ms: 4 candidates, 88128 repo-hours, 1164 releases; waiting 467 ms; commit 2924 ms)
2026-11-08T01:41:02Z [load_day 2023-10-10] 2023-10-10 hour 16: 190020 events in 4 files, snapshot 3749217225332777884 (task 19395 ms: download 11095 ms overlapping gunzip 1114 ms, parse 5145 ms, sort 141 ms, write 11587 ms, index 782 ms: 11 candidates, 86429 repo-hours, 993 releases; waiting 333 ms; commit 6150 ms)
2026-11-08T01:41:08Z [load_day 2023-10-10] 2023-10-10 hour 17: 193546 events in 4 files, snapshot 4376279381528715147 (task 21578 ms: download 11171 ms overlapping gunzip 1345 ms, parse 5797 ms, sort 209 ms, write 12503 ms, index 898 ms: 6 candidates, 83906 repo-hours, 1016 releases; waiting 465 ms; commit 2749 ms)
2026-11-08T01:41:22Z [load_day 2023-10-10] 2023-10-10 hour 18: 201623 events in 5 files, snapshot 788162796758772444 (task 21252 ms: download 11761 ms overlapping gunzip 1296 ms, parse 6177 ms, sort 195 ms, write 11966 ms, index 861 ms: 11 candidates, 89634 repo-hours, 1050 releases; waiting 495 ms; commit 2870 ms)
2026-11-08T01:41:30Z [load_day 2023-10-10] 2023-10-10 hour 19: 192703 events in 4 files, snapshot 7577481981645958487 (task 20478 ms: download 10964 ms overlapping gunzip 1206 ms, parse 5175 ms, sort 169 ms, write 12287 ms, index 794 ms: 10 candidates, 79123 repo-hours, 1118 releases; waiting 567 ms; commit 2942 ms)
2026-11-08T01:41:45Z [load_day 2023-10-10] 2023-10-10 hour 20: 191406 events in 4 files, snapshot 8652170342519337551 (task 21281 ms: download 13064 ms overlapping gunzip 1318 ms, parse 5868 ms, sort 143 ms, write 12461 ms, index 596 ms: 8 candidates, 78636 repo-hours, 1015 releases; waiting 574 ms; commit 4141 ms)
2026-11-08T01:41:50Z [load_day 2023-10-10] 2023-10-10 hour 21: 176494 events in 4 files, snapshot 8283772381905467693 (task 18806 ms: download 10753 ms overlapping gunzip 1079 ms, parse 4530 ms, sort 187 ms, write 11256 ms, index 888 ms: 8 candidates, 69734 repo-hours, 1032 releases; waiting 513 ms; commit 2427 ms)
2026-11-08T01:42:02Z [load_day 2023-10-10] 2023-10-10 hour 22: 149762 events in 3 files, snapshot 7798477435849636234 (task 16944 ms: download 8035 ms overlapping gunzip 1136 ms, parse 4908 ms, sort 164 ms, write 9486 ms, index 469 ms: 9 candidates, 60908 repo-hours, 966 releases; waiting 478 ms; commit 2920 ms)
2026-11-08T01:42:05Z [load_day 2023-10-10] 2023-10-10 hour 23: 140810 events in 3 files, snapshot 1551630810050117628 (task 13920 ms: download 6211 ms overlapping gunzip 794 ms, parse 3400 ms, sort 150 ms, write 8297 ms, index 469 ms: 3 candidates, 56109 repo-hours, 748 releases; waiting 493 ms; commit 3015 ms)
2026-11-08T01:42:14Z [load_day 2023-10-10] 2023-10-10: repo_hour merged from 24 files to 1 (1930448 rows, 8668 ms)
2026-11-08T01:42:16Z [load_day 2023-10-10] 2023-10-10: index committed in 11117 ms (24 + 1 + 24 files)
2026-11-08T01:42:16Z [load_day 2023-10-10] 2023-10-10: 4311803 events in 24 hours
2026-11-08T01:42:16Z [load_day 2023-10-10] attempt 1 succeeded