orcaLog in

Runs · run 1750

load_day 2023-11-18, attempt 1 succeeded

Host
3c2b29713ce8:570137
Started
2026-11-08T02:55:54Z
Ended
2026-11-08T02:58:38Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1750/load_day/2023-11-18/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:55:54Z [load_day 2023-11-18] attempt 1 started
2026-11-08T02:55:56Z [load_day 2023-11-18] 2023-11-18: 2 hours in flight, about 1200 MB
2026-11-08T02:56:14Z [load_day 2023-11-18] 2023-11-18 hour 0: 158164 events in 4 files, snapshot 7511390173431228352 (task 13788 ms: download 5915 ms overlapping gunzip 871 ms, parse 3903 ms, sort 110 ms, write 8168 ms, index 515 ms: 13 candidates, 78867 repo-hours, 991 releases; waiting 18 ms; commit 2583 ms)
2026-11-08T02:56:19Z [load_day 2023-11-18] 2023-11-18 hour 1: 148691 events in 3 files, snapshot 3330959899631718097 (task 12858 ms: download 5610 ms overlapping gunzip 849 ms, parse 3881 ms, sort 110 ms, write 7275 ms, index 555 ms: 8 candidates, 80066 repo-hours, 791 releases; waiting 17 ms; commit 5239 ms)
2026-11-08T02:56:26Z [load_day 2023-11-18] 2023-11-18 hour 2: 113745 events in 3 files, snapshot 1621082683257620379 (task 10458 ms: download 1390 ms overlapping gunzip 634 ms, parse 2822 ms, sort 175 ms, write 6070 ms, index 548 ms: 10 candidates, 45379 repo-hours, 606 releases; waiting 16 ms; commit 3070 ms)
2026-11-08T02:56:29Z [load_day 2023-11-18] 2023-11-18 hour 3: 111460 events in 3 files, snapshot 3623286581371506134 (task 9504 ms: download 1761 ms overlapping gunzip 676 ms, parse 2899 ms, sort 78 ms, write 5218 ms, index 374 ms: 8 candidates, 44278 repo-hours, 715 releases; waiting 62 ms; commit 3186 ms)
2026-11-08T02:56:36Z [load_day 2023-11-18] 2023-11-18 hour 4: 112863 events in 3 files, snapshot 1496233909992650981 (task 8495 ms: download 1007 ms overlapping gunzip 582 ms, parse 2456 ms, sort 94 ms, write 4686 ms, index 452 ms: 4 candidates, 44146 repo-hours, 531 releases; waiting 23 ms; commit 2671 ms)
2026-11-08T02:56:40Z [load_day 2023-11-18] 2023-11-18 hour 5: 116480 events in 3 files, snapshot 5820708059295983118 (task 9296 ms: download 1171 ms overlapping gunzip 558 ms, parse 2472 ms, sort 141 ms, write 5515 ms, index 381 ms: 5 candidates, 46235 repo-hours, 586 releases; waiting 25 ms; commit 4228 ms)
2026-11-08T02:56:46Z [load_day 2023-11-18] 2023-11-18 hour 6: 124657 events in 3 files, snapshot 2996108644837029713 (task 9013 ms: download 969 ms overlapping gunzip 588 ms, parse 2557 ms, sort 75 ms, write 5080 ms, index 515 ms: 6 candidates, 53335 repo-hours, 616 releases; waiting 16 ms; commit 2777 ms)
2026-11-08T02:56:52Z [load_day 2023-11-18] 2023-11-18 hour 7: 122823 events in 3 files, snapshot 9146256162132032686 (task 10394 ms: download 1326 ms overlapping gunzip 699 ms, parse 3019 ms, sort 74 ms, write 5747 ms, index 626 ms: 9 candidates, 47572 repo-hours, 549 releases; waiting 22 ms; commit 4765 ms)
2026-11-08T02:56:58Z [load_day 2023-11-18] 2023-11-18 hour 8: 131734 events in 3 files, snapshot 1884699172221291545 (task 11325 ms: download 1257 ms overlapping gunzip 758 ms, parse 3399 ms, sort 157 ms, write 6408 ms, index 388 ms: 12 candidates, 50910 repo-hours, 649 releases; waiting 23 ms; commit 2852 ms)
2026-11-08T02:57:03Z [load_day 2023-11-18] 2023-11-18 hour 9: 135259 events in 3 files, snapshot 6037287988306348870 (task 11344 ms: download 1995 ms overlapping gunzip 759 ms, parse 3325 ms, sort 140 ms, write 6477 ms, index 432 ms: 6 candidates, 52046 repo-hours, 741 releases; waiting 26 ms; commit 2694 ms)
2026-11-08T02:57:11Z [load_day 2023-11-18] 2023-11-18 hour 10: 135939 events in 3 files, snapshot 4146064353742675412 (task 10425 ms: download 3490 ms overlapping gunzip 542 ms, parse 2624 ms, sort 156 ms, write 6450 ms, index 438 ms: 9 candidates, 56540 repo-hours, 673 releases; waiting 14 ms; commit 3390 ms)
2026-11-08T02:57:15Z [load_day 2023-11-18] 2023-11-18 hour 11: 136964 events in 3 files, snapshot 3316966873568886869 (task 11011 ms: download 1411 ms overlapping gunzip 680 ms, parse 3183 ms, sort 93 ms, write 6407 ms, index 430 ms: 5 candidates, 53684 repo-hours, 678 releases; waiting 24 ms; commit 3227 ms)
2026-11-08T02:57:26Z [load_day 2023-11-18] 2023-11-18 hour 12: 208200 events in 5 files, snapshot 5144172528027613061 (task 13133 ms: download 4523 ms overlapping gunzip 692 ms, parse 3536 ms, sort 212 ms, write 7840 ms, index 628 ms: 6 candidates, 86576 repo-hours, 719 releases; waiting 19 ms; commit 4608 ms)
2026-11-08T02:57:32Z [load_day 2023-11-18] 2023-11-18 hour 13: 173670 events in 4 files, snapshot 929753939855606289 (task 12708 ms: download 3885 ms overlapping gunzip 736 ms, parse 3571 ms, sort 120 ms, write 6907 ms, index 1153 ms: 14 candidates, 80412 repo-hours, 731 releases; waiting 24 ms; commit 6344 ms)
2026-11-08T02:57:39Z [load_day 2023-11-18] 2023-11-18 hour 14: 160456 events in 4 files, snapshot 8912037004773104661 (task 13456 ms: download 4771 ms overlapping gunzip 825 ms, parse 3660 ms, sort 161 ms, write 8099 ms, index 493 ms: 8 candidates, 64618 repo-hours, 806 releases; waiting 17 ms; commit 2799 ms)
2026-11-08T02:57:44Z [load_day 2023-11-18] 2023-11-18 hour 15: 164978 events in 4 files, snapshot 6934787542678754116 (task 13487 ms: download 5535 ms overlapping gunzip 796 ms, parse 3569 ms, sort 138 ms, write 7769 ms, index 468 ms: 3 candidates, 66848 repo-hours, 929 releases; waiting 500 ms; commit 3165 ms)
2026-11-08T02:57:53Z [load_day 2023-11-18] 2023-11-18 hour 16: 164087 events in 4 files, snapshot 114232513754785906 (task 12799 ms: download 4020 ms overlapping gunzip 756 ms, parse 3351 ms, sort 104 ms, write 7864 ms, index 513 ms: 9 candidates, 65426 repo-hours, 900 releases; waiting 19 ms; commit 2884 ms)
2026-11-08T02:58:00Z [load_day 2023-11-18] 2023-11-18 hour 17: 152451 events in 4 files, snapshot 1449287459018686492 (task 13059 ms: download 4744 ms overlapping gunzip 758 ms, parse 3449 ms, sort 130 ms, write 7316 ms, index 838 ms: 11 candidates, 59942 repo-hours, 803 releases; waiting 20 ms; commit 4806 ms)
2026-11-08T02:58:05Z [load_day 2023-11-18] 2023-11-18 hour 18: 152767 events in 4 files, snapshot 5031538881554151414 (task 10556 ms: download 3237 ms overlapping gunzip 660 ms, parse 3006 ms, sort 97 ms, write 5940 ms, index 642 ms: 11 candidates, 65894 repo-hours, 831 releases; waiting 20 ms; commit 3063 ms)
2026-11-08T02:58:11Z [load_day 2023-11-18] 2023-11-18 hour 19: 143496 events in 3 files, snapshot 8614838619607589701 (task 10621 ms: download 4004 ms overlapping gunzip 619 ms, parse 2815 ms, sort 94 ms, write 6509 ms, index 370 ms: 6 candidates, 54860 repo-hours, 903 releases; waiting 19 ms; commit 2575 ms)
2026-11-08T02:58:15Z [load_day 2023-11-18] 2023-11-18 hour 20: 139808 events in 3 files, snapshot 2085459610440342783 (task 9198 ms: download 1619 ms overlapping gunzip 583 ms, parse 2603 ms, sort 111 ms, write 5307 ms, index 363 ms: 10 candidates, 54558 repo-hours, 940 releases; waiting 27 ms; commit 2230 ms)
2026-11-08T02:58:20Z [load_day 2023-11-18] 2023-11-18 hour 21: 130444 events in 3 files, snapshot 8979111416218951060 (task 8437 ms: download 1234 ms overlapping gunzip 494 ms, parse 2252 ms, sort 118 ms, write 5026 ms, index 327 ms: 7 candidates, 51286 repo-hours, 841 releases; waiting 16 ms; commit 2364 ms)
2026-11-08T02:58:24Z [load_day 2023-11-18] 2023-11-18 hour 22: 126256 events in 3 files, snapshot 7794265198331378723 (task 8324 ms: download 1893 ms overlapping gunzip 456 ms, parse 2122 ms, sort 66 ms, write 4525 ms, index 386 ms: 6 candidates, 48294 repo-hours, 831 releases; waiting 512 ms; commit 2106 ms)
2026-11-08T02:58:29Z [load_day 2023-11-18] 2023-11-18 hour 23: 119599 events in 3 files, snapshot 1056303501342389748 (task 7653 ms: download 1722 ms overlapping gunzip 461 ms, parse 2167 ms, sort 74 ms, write 3893 ms, index 310 ms: 4 candidates, 45866 repo-hours, 678 releases; waiting 517 ms; commit 2681 ms)
2026-11-08T02:58:36Z [load_day 2023-11-18] 2023-11-18: repo_hour merged from 24 files to 1 (1397638 rows, 6548 ms)
2026-11-08T02:58:38Z [load_day 2023-11-18] 2023-11-18: index committed in 8756 ms (24 + 1 + 24 files)
2026-11-08T02:58:38Z [load_day 2023-11-18] 2023-11-18: 3384991 events in 24 hours
2026-11-08T02:58:38Z [load_day 2023-11-18] attempt 1 succeeded