orcaLog in

Runs · run 1342

load_day 2023-07-29, attempt 1 succeeded

Host
3c2b29713ce8:173803
Started
2026-11-07T23:24:16Z
Ended
2026-11-07T23:26:52Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1342/load_day/2023-07-29/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T23:24:16Z [load_day 2023-07-29] attempt 1 started
2026-11-07T23:24:18Z [load_day 2023-07-29] 2023-07-29: 2 hours in flight, about 1200 MB
2026-11-07T23:24:33Z [load_day 2023-07-29] 2023-07-29 hour 0: 123041 events in 3 files, snapshot 4464903301519458003 (task 11515 ms: download 2551 ms overlapping gunzip 686 ms, parse 3335 ms, sort 80 ms, write 6099 ms, index 476 ms: 5 candidates, 64346 repo-hours, 877 releases; waiting 515 ms; commit 2070 ms)
2026-11-07T23:24:38Z [load_day 2023-07-29] 2023-07-29 hour 1: 124014 events in 3 files, snapshot 1837342720834348430 (task 10504 ms: download 2207 ms overlapping gunzip 582 ms, parse 2697 ms, sort 87 ms, write 5780 ms, index 708 ms: 4 candidates, 72535 repo-hours, 715 releases; waiting 365 ms; commit 4899 ms)
2026-11-07T23:24:43Z [load_day 2023-07-29] 2023-07-29 hour 2: 93616 events in 2 files, snapshot 7157285612251671547 (task 8838 ms: download 2063 ms overlapping gunzip 421 ms, parse 2030 ms, sort 64 ms, write 5089 ms, index 386 ms: 8 candidates, 40808 repo-hours, 550 releases; waiting 514 ms; commit 3055 ms)
2026-11-07T23:24:46Z [load_day 2023-07-29] 2023-07-29 hour 3: 94997 events in 2 files, snapshot 1019792564664004015 (task 7668 ms: download 1912 ms overlapping gunzip 492 ms, parse 2086 ms, sort 82 ms, write 3781 ms, index 378 ms: 2 candidates, 40034 repo-hours, 633 releases; waiting 501 ms; commit 2209 ms)
2026-11-07T23:24:52Z [load_day 2023-07-29] 2023-07-29 hour 4: 94853 events in 2 files, snapshot 6469362333557106298 (task 8523 ms: download 1906 ms overlapping gunzip 475 ms, parse 2078 ms, sort 130 ms, write 4532 ms, index 477 ms: 2 candidates, 40597 repo-hours, 523 releases; waiting 504 ms; commit 2512 ms)
2026-11-07T23:24:56Z [load_day 2023-07-29] 2023-07-29 hour 5: 97613 events in 2 files, snapshot 2903051895708658851 (task 8858 ms: download 4550 ms overlapping gunzip 461 ms, parse 1814 ms, sort 66 ms, write 5248 ms, index 407 ms: 1 candidates, 42286 repo-hours, 605 releases; waiting 504 ms; commit 2503 ms)
2026-11-07T23:25:04Z [load_day 2023-07-29] 2023-07-29 hour 6: 112253 events in 3 files, snapshot 2005727001313069548 (task 10632 ms: download 4100 ms overlapping gunzip 739 ms, parse 2753 ms, sort 90 ms, write 5632 ms, index 394 ms: 3 candidates, 50814 repo-hours, 630 releases; waiting 713 ms; commit 2218 ms)
2026-11-07T23:25:09Z [load_day 2023-07-29] 2023-07-29 hour 7: 109469 events in 3 files, snapshot 8522701806464990092 (task 12096 ms: download 6308 ms overlapping gunzip 887 ms, parse 3048 ms, sort 106 ms, write 6625 ms, index 530 ms: 4 candidates, 46632 repo-hours, 561 releases; waiting 608 ms; commit 2150 ms)
2026-11-07T23:25:15Z [load_day 2023-07-29] 2023-07-29 hour 8: 114227 events in 3 files, snapshot 4872176662690462408 (task 9983 ms: download 4992 ms overlapping gunzip 560 ms, parse 2286 ms, sort 69 ms, write 5962 ms, index 336 ms: 2 candidates, 49138 repo-hours, 935 releases; waiting 487 ms; commit 2102 ms)
2026-11-07T23:25:22Z [load_day 2023-07-29] 2023-07-29 hour 9: 115538 events in 3 files, snapshot 2708715504558083830 (task 11820 ms: download 5828 ms overlapping gunzip 903 ms, parse 2735 ms, sort 95 ms, write 6687 ms, index 386 ms: 1 candidates, 49560 repo-hours, 665 releases; waiting 720 ms; commit 2288 ms)
2026-11-07T23:25:27Z [load_day 2023-07-29] 2023-07-29 hour 10: 115574 events in 3 files, snapshot 15340237299332713 (task 10386 ms: download 4929 ms overlapping gunzip 752 ms, parse 2806 ms, sort 74 ms, write 5600 ms, index 412 ms: 3 candidates, 50653 repo-hours, 629 releases; waiting 503 ms; commit 3087 ms)
2026-11-07T23:25:34Z [load_day 2023-07-29] 2023-07-29 hour 11: 113964 events in 3 files, snapshot 7808483489859933377 (task 10981 ms: download 5150 ms overlapping gunzip 822 ms, parse 2798 ms, sort 118 ms, write 6216 ms, index 363 ms: 3 candidates, 47292 repo-hours, 567 releases; waiting 375 ms; commit 2339 ms)
2026-11-07T23:25:41Z [load_day 2023-07-29] 2023-07-29 hour 12: 197137 events in 4 files, snapshot 6139540583323199858 (task 13336 ms: download 7129 ms overlapping gunzip 752 ms, parse 3544 ms, sort 196 ms, write 7213 ms, index 807 ms: 5 candidates, 82540 repo-hours, 687 releases; waiting 527 ms; commit 2456 ms)
2026-11-07T23:25:47Z [load_day 2023-07-29] 2023-07-29 hour 13: 150538 events in 4 files, snapshot 5726919441279396112 (task 12617 ms: download 4341 ms overlapping gunzip 761 ms, parse 3046 ms, sort 134 ms, write 7236 ms, index 623 ms: 3 candidates, 77521 repo-hours, 708 releases; waiting 497 ms; commit 2213 ms)
2026-11-07T23:25:55Z [load_day 2023-07-29] 2023-07-29 hour 14: 136063 events in 3 files, snapshot 243141634691164607 (task 12788 ms: download 5786 ms overlapping gunzip 792 ms, parse 3137 ms, sort 110 ms, write 7082 ms, index 591 ms: 1 candidates, 57942 repo-hours, 767 releases; waiting 718 ms; commit 2322 ms)
2026-11-07T23:26:01Z [load_day 2023-07-29] 2023-07-29 hour 15: 140108 events in 3 files, snapshot 2630701578389084596 (task 12356 ms: download 4600 ms overlapping gunzip 685 ms, parse 2709 ms, sort 103 ms, write 7416 ms, index 678 ms: 3 candidates, 59699 repo-hours, 832 releases; waiting 481 ms; commit 2988 ms)
2026-11-07T23:26:09Z [load_day 2023-07-29] 2023-07-29 hour 16: 134516 events in 3 files, snapshot 5782238107943309137 (task 12602 ms: download 6266 ms overlapping gunzip 782 ms, parse 3037 ms, sort 124 ms, write 7373 ms, index 386 ms: 3 candidates, 57478 repo-hours, 687 releases; waiting 612 ms; commit 2546 ms)
2026-11-07T23:26:18Z [load_day 2023-07-29] 2023-07-29 hour 17: 127812 events in 3 files, snapshot 1161541945350055572 (task 10594 ms: download 5083 ms overlapping gunzip 636 ms, parse 2620 ms, sort 135 ms, write 6041 ms, index 353 ms: 4 candidates, 52354 repo-hours, 711 releases; waiting 523 ms; commit 5922 ms)
2026-11-07T23:26:22Z [load_day 2023-07-29] 2023-07-29 hour 18: 129568 events in 3 files, snapshot 6090212151890652286 (task 11965 ms: download 5238 ms overlapping gunzip 749 ms, parse 2687 ms, sort 75 ms, write 7230 ms, index 449 ms: 6 candidates, 58514 repo-hours, 703 releases; waiting 496 ms; commit 3398 ms)
2026-11-07T23:26:27Z [load_day 2023-07-29] 2023-07-29 hour 19: 111215 events in 3 files, snapshot 3049749947017912481 (task 10691 ms: download 5151 ms overlapping gunzip 782 ms, parse 3024 ms, sort 96 ms, write 5521 ms, index 352 ms: 3 candidates, 46380 repo-hours, 632 releases; waiting 554 ms; commit 2135 ms)
2026-11-07T23:26:32Z [load_day 2023-07-29] 2023-07-29 hour 20: 109211 events in 3 files, snapshot 2209331607829498766 (task 9043 ms: download 2915 ms overlapping gunzip 625 ms, parse 2256 ms, sort 69 ms, write 4882 ms, index 299 ms: 2 candidates, 45611 repo-hours, 612 releases; waiting 613 ms; commit 2548 ms)
2026-11-07T23:26:35Z [load_day 2023-07-29] 2023-07-29 hour 21: 98518 events in 2 files, snapshot 3148356768380620462 (task 7955 ms: download 4149 ms overlapping gunzip 533 ms, parse 2032 ms, sort 71 ms, write 4163 ms, index 284 ms: 6 candidates, 40312 repo-hours, 661 releases; waiting 603 ms; commit 1864 ms)
2026-11-07T23:26:40Z [load_day 2023-07-29] 2023-07-29 hour 22: 89900 events in 2 files, snapshot 2656852113586358855 (task 7342 ms: download 2580 ms overlapping gunzip 461 ms, parse 1701 ms, sort 56 ms, write 3747 ms, index 337 ms: 2 candidates, 37551 repo-hours, 607 releases; waiting 729 ms; commit 2063 ms)
2026-11-07T23:26:43Z [load_day 2023-07-29] 2023-07-29 hour 23: 83534 events in 2 files, snapshot 2534892329719447752 (task 6835 ms: download 2508 ms overlapping gunzip 471 ms, parse 1683 ms, sort 61 ms, write 3416 ms, index 240 ms: 0 candidates, 34945 repo-hours, 558 releases; waiting 695 ms; commit 2091 ms)
2026-11-07T23:26:50Z [load_day 2023-07-29] 2023-07-29: repo_hour merged from 24 files to 1 (1245542 rows, 6374 ms)
2026-11-07T23:26:52Z [load_day 2023-07-29] 2023-07-29: index committed in 8526 ms (23 + 1 + 24 files)
2026-11-07T23:26:52Z [load_day 2023-07-29] 2023-07-29: 2817279 events in 24 hours
2026-11-07T23:26:52Z [load_day 2023-07-29] attempt 1 succeeded