orcaLog in

Runs · run 1574

load_day 2023-10-01, attempt 1 succeeded

Host
3c2b29713ce8:388651
Started
2026-11-08T01:21:05Z
Ended
2026-11-08T01:24:22Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1574/load_day/2023-10-01/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:21:05Z [load_day 2023-10-01] attempt 1 started
2026-11-08T01:21:07Z [load_day 2023-10-01] 2023-10-01: 2 hours in flight, about 1200 MB
2026-11-08T01:21:24Z [load_day 2023-10-01] 2023-10-01 hour 0: 132653 events in 3 files, snapshot 7694074779093503789 (task 12479 ms: download 5633 ms overlapping gunzip 818 ms, parse 3531 ms, sort 95 ms, write 6756 ms, index 501 ms: 10 candidates, 70745 repo-hours, 1005 releases; waiting 506 ms; commit 3311 ms)
2026-11-08T01:21:28Z [load_day 2023-10-01] 2023-10-01 hour 1: 139418 events in 3 files, snapshot 5919814852953258299 (task 12437 ms: download 5066 ms overlapping gunzip 748 ms, parse 3128 ms, sort 142 ms, write 7046 ms, index 594 ms: 4 candidates, 79460 repo-hours, 984 releases; waiting 489 ms; commit 4356 ms)
2026-11-08T01:21:37Z [load_day 2023-10-01] 2023-10-01 hour 2: 113284 events in 3 files, snapshot 5985551017596680293 (task 10880 ms: download 4946 ms overlapping gunzip 675 ms, parse 2870 ms, sort 73 ms, write 6112 ms, index 325 ms: 6 candidates, 42233 repo-hours, 657 releases; waiting 525 ms; commit 4323 ms)
2026-11-08T01:21:40Z [load_day 2023-10-01] 2023-10-01 hour 3: 107540 events in 3 files, snapshot 2396602089704574903 (task 10801 ms: download 1937 ms overlapping gunzip 662 ms, parse 2601 ms, sort 82 ms, write 6037 ms, index 641 ms: 4 candidates, 42040 repo-hours, 723 releases; waiting 491 ms; commit 2777 ms)
2026-11-08T01:21:49Z [load_day 2023-10-01] 2023-10-01 hour 4: 109172 events in 3 files, snapshot 709932633637969941 (task 10748 ms: download 2155 ms overlapping gunzip 655 ms, parse 2774 ms, sort 90 ms, write 6032 ms, index 391 ms: 4 candidates, 42842 repo-hours, 549 releases; waiting 529 ms; commit 4776 ms)
2026-11-08T01:21:52Z [load_day 2023-10-01] 2023-10-01 hour 5: 111290 events in 3 files, snapshot 4231977071825642858 (task 10523 ms: download 2436 ms overlapping gunzip 604 ms, parse 2571 ms, sort 76 ms, write 5999 ms, index 426 ms: 7 candidates, 45244 repo-hours, 662 releases; waiting 513 ms; commit 2399 ms)
2026-11-08T01:22:02Z [load_day 2023-10-01] 2023-10-01 hour 6: 125093 events in 3 files, snapshot 1475387782185796685 (task 11086 ms: download 4639 ms overlapping gunzip 785 ms, parse 3349 ms, sort 105 ms, write 5641 ms, index 379 ms: 8 candidates, 54352 repo-hours, 610 releases; waiting 507 ms; commit 4939 ms)
2026-11-08T01:22:05Z [load_day 2023-10-01] 2023-10-01 hour 7: 123986 events in 3 files, snapshot 2080519367296874156 (task 12269 ms: download 5573 ms overlapping gunzip 780 ms, parse 3302 ms, sort 78 ms, write 6904 ms, index 404 ms: 5 candidates, 50036 repo-hours, 571 releases; waiting 545 ms; commit 2425 ms)
2026-11-08T01:22:14Z [load_day 2023-10-01] 2023-10-01 hour 8: 134845 events in 3 files, snapshot 1234617852916642160 (task 13511 ms: download 5681 ms overlapping gunzip 813 ms, parse 3578 ms, sort 121 ms, write 7649 ms, index 576 ms: 11 candidates, 54170 repo-hours, 732 releases; waiting 511 ms; commit 2328 ms)
2026-11-08T01:22:20Z [load_day 2023-10-01] 2023-10-01 hour 9: 138068 events in 3 files, snapshot 3639120269734623418 (task 13419 ms: download 5914 ms overlapping gunzip 786 ms, parse 3250 ms, sort 108 ms, write 7938 ms, index 472 ms: 12 candidates, 55027 repo-hours, 910 releases; waiting 523 ms; commit 2752 ms)
2026-11-08T01:22:28Z [load_day 2023-10-01] 2023-10-01 hour 10: 141037 events in 3 files, snapshot 4621934733315376868 (task 12663 ms: download 5739 ms overlapping gunzip 750 ms, parse 3224 ms, sort 121 ms, write 7390 ms, index 378 ms: 7 candidates, 57791 repo-hours, 834 releases; waiting 503 ms; commit 2316 ms)
2026-11-08T01:22:31Z [load_day 2023-10-01] 2023-10-01 hour 11: 136028 events in 3 files, snapshot 1514678963364370987 (task 11020 ms: download 4901 ms overlapping gunzip 711 ms, parse 2943 ms, sort 84 ms, write 6104 ms, index 373 ms: 4 candidates, 54893 repo-hours, 729 releases; waiting 541 ms; commit 2266 ms)
2026-11-08T01:22:43Z [load_day 2023-10-01] 2023-10-01 hour 12: 208382 events in 5 files, snapshot 196193394893249406 (task 13337 ms: download 6186 ms overlapping gunzip 736 ms, parse 3434 ms, sort 113 ms, write 7364 ms, index 921 ms: 5 candidates, 90001 repo-hours, 783 releases; waiting 515 ms; commit 2976 ms)
2026-11-08T01:22:47Z [load_day 2023-10-01] 2023-10-01 hour 13: 173584 events in 4 files, snapshot 9114356888895959190 (task 15018 ms: download 6718 ms overlapping gunzip 835 ms, parse 3680 ms, sort 128 ms, write 8644 ms, index 776 ms: 5 candidates, 83436 repo-hours, 757 releases; waiting 657 ms; commit 2508 ms)
2026-11-08T01:23:01Z [load_day 2023-10-01] 2023-10-01 hour 14: 166503 events in 4 files, snapshot 7239484940274238547 (task 17267 ms: download 7079 ms overlapping gunzip 1159 ms, parse 4626 ms, sort 133 ms, write 9878 ms, index 677 ms: 6 candidates, 68597 repo-hours, 855 releases; waiting 501 ms; commit 2399 ms)
2026-11-08T01:23:05Z [load_day 2023-10-01] 2023-10-01 hour 15: 169883 events in 4 files, snapshot 7313621768804793431 (task 16223 ms: download 8918 ms overlapping gunzip 977 ms, parse 3839 ms, sort 157 ms, write 9620 ms, index 805 ms: 10 candidates, 71081 repo-hours, 997 releases; waiting 505 ms; commit 2618 ms)
2026-11-08T01:23:20Z [load_day 2023-10-01] 2023-10-01 hour 16: 164977 events in 4 files, snapshot 6919926086884118252 (task 17408 ms: download 7176 ms overlapping gunzip 1192 ms, parse 5018 ms, sort 170 ms, write 9546 ms, index 802 ms: 12 candidates, 69303 repo-hours, 869 releases; waiting 419 ms; commit 3038 ms)
2026-11-08T01:23:25Z [load_day 2023-10-01] 2023-10-01 hour 17: 160557 events in 4 files, snapshot 5285504440093973741 (task 16881 ms: download 6983 ms overlapping gunzip 1020 ms, parse 4164 ms, sort 127 ms, write 10017 ms, index 761 ms: 13 candidates, 65952 repo-hours, 782 releases; waiting 537 ms; commit 4412 ms)
2026-11-08T01:23:36Z [load_day 2023-10-01] 2023-10-01 hour 18: 163007 events in 4 files, snapshot 4241749314495307995 (task 15264 ms: download 6695 ms overlapping gunzip 876 ms, parse 3696 ms, sort 184 ms, write 8984 ms, index 699 ms: 9 candidates, 71785 repo-hours, 891 releases; waiting 536 ms; commit 2369 ms)
2026-11-08T01:23:39Z [load_day 2023-10-01] 2023-10-01 hour 19: 147669 events in 3 files, snapshot 412210427212923737 (task 14586 ms: download 6233 ms overlapping gunzip 854 ms, parse 3649 ms, sort 146 ms, write 8452 ms, index 655 ms: 6 candidates, 60522 repo-hours, 816 releases; waiting 575 ms; commit 2538 ms)
2026-11-08T01:23:51Z [load_day 2023-10-01] 2023-10-01 hour 20: 141375 events in 3 files, snapshot 929004851345813136 (task 13219 ms: download 5615 ms overlapping gunzip 784 ms, parse 3363 ms, sort 95 ms, write 7661 ms, index 589 ms: 11 candidates, 58751 repo-hours, 729 releases; waiting 486 ms; commit 2428 ms)
2026-11-08T01:23:54Z [load_day 2023-10-01] 2023-10-01 hour 21: 129831 events in 3 files, snapshot 383436845245818296 (task 12589 ms: download 5743 ms overlapping gunzip 677 ms, parse 2827 ms, sort 112 ms, write 7753 ms, index 411 ms: 8 candidates, 52025 repo-hours, 778 releases; waiting 569 ms; commit 2923 ms)
2026-11-08T01:24:10Z [load_day 2023-10-01] 2023-10-01 hour 22: 117381 events in 3 files, snapshot 1855899820808632 (task 11721 ms: download 5405 ms overlapping gunzip 648 ms, parse 2674 ms, sort 78 ms, write 7011 ms, index 327 ms: 4 candidates, 46137 repo-hours, 700 releases; waiting 600 ms; commit 9014 ms)
2026-11-08T01:24:12Z [load_day 2023-10-01] 2023-10-01 hour 23: 112156 events in 3 files, snapshot 1403133700281073476 (task 11385 ms: download 2631 ms overlapping gunzip 784 ms, parse 3308 ms, sort 97 ms, write 5960 ms, index 373 ms: 4 candidates, 45219 repo-hours, 567 releases; waiting 527 ms; commit 2123 ms)
2026-11-08T01:24:20Z [load_day 2023-10-01] 2023-10-01: repo_hour merged from 24 files to 1 (1431642 rows, 7116 ms)
2026-11-08T01:24:22Z [load_day 2023-10-01] 2023-10-01: index committed in 9553 ms (24 + 1 + 24 files)
2026-11-08T01:24:22Z [load_day 2023-10-01] 2023-10-01: 3367719 events in 24 hours
2026-11-08T01:24:22Z [load_day 2023-10-01] attempt 1 succeeded