orcaLog in

Runs · run 1455

load_day 2023-09-02, attempt 1 succeeded

Host
3c2b29713ce8:288923
Started
2026-11-08T00:27:29Z
Ended
2026-11-08T00:29:58Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1455/load_day/2023-09-02/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:27:29Z [load_day 2023-09-02] attempt 1 started
2026-11-08T00:27:31Z [load_day 2023-09-02] 2023-09-02: 2 hours in flight, about 1200 MB
2026-11-08T00:27:47Z [load_day 2023-09-02] 2023-09-02 hour 0: 143206 events in 3 files, snapshot 8757959037683598488 (task 12339 ms: download 2558 ms overlapping gunzip 698 ms, parse 3252 ms, sort 145 ms, write 6782 ms, index 649 ms: 6 candidates, 76536 repo-hours, 890 releases; waiting 504 ms; commit 2279 ms)
2026-11-08T00:27:50Z [load_day 2023-09-02] 2023-09-02 hour 1: 128967 events in 3 files, snapshot 1596600527809709339 (task 10892 ms: download 2740 ms overlapping gunzip 524 ms, parse 2548 ms, sort 104 ms, write 6091 ms, index 784 ms: 6 candidates, 70532 repo-hours, 736 releases; waiting 514 ms; commit 3134 ms)
2026-11-08T00:27:56Z [load_day 2023-09-02] 2023-09-02 hour 2: 102883 events in 3 files, snapshot 5558318754483332744 (task 8473 ms: download 2013 ms overlapping gunzip 467 ms, parse 1991 ms, sort 98 ms, write 4869 ms, index 417 ms: 2 candidates, 41472 repo-hours, 531 releases; waiting 305 ms; commit 2178 ms)
2026-11-08T00:28:01Z [load_day 2023-09-02] 2023-09-02 hour 3: 98201 events in 2 files, snapshot 7954402056585269863 (task 7625 ms: download 1771 ms overlapping gunzip 444 ms, parse 1974 ms, sort 63 ms, write 4028 ms, index 299 ms: 2 candidates, 40924 repo-hours, 670 releases; waiting 519 ms; commit 5041 ms)
2026-11-08T00:28:08Z [load_day 2023-09-02] 2023-09-02 hour 4: 97385 events in 2 files, snapshot 7897822957892995010 (task 8084 ms: download 1987 ms overlapping gunzip 405 ms, parse 1809 ms, sort 79 ms, write 4702 ms, index 300 ms: 2 candidates, 39899 repo-hours, 443 releases; waiting 511 ms; commit 4292 ms)
2026-11-08T00:28:10Z [load_day 2023-09-02] 2023-09-02 hour 5: 105081 events in 3 files, snapshot 9056658691338330093 (task 9428 ms: download 2059 ms overlapping gunzip 511 ms, parse 2289 ms, sort 117 ms, write 5244 ms, index 489 ms: 4 candidates, 43412 repo-hours, 568 releases; waiting 508 ms; commit 2468 ms)
2026-11-08T00:28:18Z [load_day 2023-09-02] 2023-09-02 hour 6: 116465 events in 3 files, snapshot 5376389965442902773 (task 9929 ms: download 2074 ms overlapping gunzip 598 ms, parse 2775 ms, sort 120 ms, write 5258 ms, index 403 ms: 5 candidates, 51198 repo-hours, 611 releases; waiting 481 ms; commit 2838 ms)
2026-11-08T00:28:23Z [load_day 2023-09-02] 2023-09-02 hour 7: 113413 events in 3 files, snapshot 6871723764712271903 (task 10141 ms: download 1874 ms overlapping gunzip 531 ms, parse 2443 ms, sort 71 ms, write 5902 ms, index 364 ms: 2 candidates, 46112 repo-hours, 584 releases; waiting 515 ms; commit 4222 ms)
2026-11-08T00:28:29Z [load_day 2023-09-02] 2023-09-02 hour 8: 119318 events in 3 files, snapshot 9195087901574970400 (task 10474 ms: download 2216 ms overlapping gunzip 672 ms, parse 3022 ms, sort 71 ms, write 5368 ms, index 617 ms: 3 candidates, 49896 repo-hours, 631 releases; waiting 455 ms; commit 2640 ms)
2026-11-08T00:28:33Z [load_day 2023-09-02] 2023-09-02 hour 9: 117863 events in 3 files, snapshot 5832525061730645510 (task 10182 ms: download 1849 ms overlapping gunzip 610 ms, parse 2681 ms, sort 71 ms, write 5678 ms, index 319 ms: 2 candidates, 49287 repo-hours, 792 releases; waiting 521 ms; commit 3296 ms)
2026-11-08T00:28:42Z [load_day 2023-09-02] 2023-09-02 hour 10: 120524 events in 3 files, snapshot 5459340754739108576 (task 10817 ms: download 2960 ms overlapping gunzip 535 ms, parse 2329 ms, sort 83 ms, write 6025 ms, index 364 ms: 1 candidates, 50786 repo-hours, 664 releases; waiting 1215 ms; commit 3687 ms)
2026-11-08T00:28:47Z [load_day 2023-09-02] 2023-09-02 hour 11: 117547 events in 3 files, snapshot 6387404413952563104 (task 9087 ms: download 1731 ms overlapping gunzip 565 ms, parse 2491 ms, sort 88 ms, write 5084 ms, index 387 ms: 3 candidates, 48596 repo-hours, 654 releases; waiting 229 ms; commit 4878 ms)
2026-11-08T00:28:55Z [load_day 2023-09-02] 2023-09-02 hour 12: 199176 events in 4 files, snapshot 6658866350415384460 (task 13281 ms: download 3937 ms overlapping gunzip 641 ms, parse 3310 ms, sort 134 ms, write 7733 ms, index 631 ms: 3 candidates, 82129 repo-hours, 756 releases; waiting 530 ms; commit 2167 ms)
2026-11-08T00:28:57Z [load_day 2023-09-02] 2023-09-02 hour 13: 152253 events in 4 files, snapshot 1056907033676818925 (task 11493 ms: download 4479 ms overlapping gunzip 634 ms, parse 3113 ms, sort 140 ms, write 6358 ms, index 559 ms: 4 candidates, 75925 repo-hours, 705 releases; waiting 471 ms; commit 2723 ms)
2026-11-08T00:29:07Z [load_day 2023-09-02] 2023-09-02 hour 14: 136782 events in 3 files, snapshot 1007503004331454031 (task 11445 ms: download 2606 ms overlapping gunzip 692 ms, parse 2958 ms, sort 114 ms, write 6239 ms, index 544 ms: 3 candidates, 59462 repo-hours, 711 releases; waiting 554 ms; commit 2274 ms)
2026-11-08T00:29:10Z [load_day 2023-09-02] 2023-09-02 hour 15: 138262 events in 3 files, snapshot 2288296991992216871 (task 11080 ms: download 4223 ms overlapping gunzip 567 ms, parse 2465 ms, sort 150 ms, write 6692 ms, index 418 ms: 5 candidates, 59039 repo-hours, 885 releases; waiting 499 ms; commit 2566 ms)
2026-11-08T00:29:18Z [load_day 2023-09-02] 2023-09-02 hour 16: 133226 events in 3 files, snapshot 785197874003206499 (task 9581 ms: download 2482 ms overlapping gunzip 559 ms, parse 2473 ms, sort 82 ms, write 5260 ms, index 385 ms: 4 candidates, 57459 repo-hours, 853 releases; waiting 521 ms; commit 2105 ms)
2026-11-08T00:29:20Z [load_day 2023-09-02] 2023-09-02 hour 17: 128450 events in 3 files, snapshot 589153690529213110 (task 8905 ms: download 2487 ms overlapping gunzip 500 ms, parse 2256 ms, sort 75 ms, write 4964 ms, index 371 ms: 7 candidates, 53322 repo-hours, 616 releases; waiting 511 ms; commit 2531 ms)
2026-11-08T00:29:27Z [load_day 2023-09-02] 2023-09-02 hour 18: 130609 events in 3 files, snapshot 7101303444061228106 (task 8310 ms: download 1765 ms overlapping gunzip 478 ms, parse 2086 ms, sort 86 ms, write 4512 ms, index 388 ms: 3 candidates, 58634 repo-hours, 800 releases; waiting 508 ms; commit 2191 ms)
2026-11-08T00:29:30Z [load_day 2023-09-02] 2023-09-02 hour 19: 116621 events in 3 files, snapshot 6929951092266805985 (task 7933 ms: download 1758 ms overlapping gunzip 433 ms, parse 1933 ms, sort 74 ms, write 4388 ms, index 320 ms: 2 candidates, 46615 repo-hours, 682 releases; waiting 529 ms; commit 2416 ms)
2026-11-08T00:29:37Z [load_day 2023-09-02] 2023-09-02 hour 20: 113912 events in 3 files, snapshot 675536111448382111 (task 9198 ms: download 1820 ms overlapping gunzip 548 ms, parse 2535 ms, sort 119 ms, write 4725 ms, index 469 ms: 7 candidates, 47003 repo-hours, 688 releases; waiting 510 ms; commit 2308 ms)
2026-11-08T00:29:40Z [load_day 2023-09-02] 2023-09-02 hour 21: 105635 events in 3 files, snapshot 2236239818311651867 (task 9156 ms: download 2093 ms overlapping gunzip 417 ms, parse 2011 ms, sort 123 ms, write 5455 ms, index 348 ms: 7 candidates, 42511 repo-hours, 779 releases; waiting 525 ms; commit 2198 ms)
2026-11-08T00:29:47Z [load_day 2023-09-02] 2023-09-02 hour 22: 94960 events in 2 files, snapshot 4700604534347859806 (task 7225 ms: download 2023 ms overlapping gunzip 400 ms, parse 1865 ms, sort 61 ms, write 3747 ms, index 291 ms: 5 candidates, 38062 repo-hours, 646 releases; waiting 535 ms; commit 3039 ms)
2026-11-08T00:29:50Z [load_day 2023-09-02] 2023-09-02 hour 23: 91314 events in 2 files, snapshot 4576206780469501602 (task 7622 ms: download 2136 ms overlapping gunzip 363 ms, parse 1727 ms, sort 82 ms, write 4253 ms, index 289 ms: 3 candidates, 37165 repo-hours, 628 releases; waiting 564 ms; commit 3103 ms)
2026-11-08T00:29:56Z [load_day 2023-09-02] 2023-09-02: repo_hour merged from 24 files to 1 (1265976 rows, 6071 ms)
2026-11-08T00:29:58Z [load_day 2023-09-02] 2023-09-02: index committed in 8084 ms (24 + 1 + 24 files)
2026-11-08T00:29:58Z [load_day 2023-09-02] 2023-09-02: 2922053 events in 24 hours
2026-11-08T00:29:58Z [load_day 2023-09-02] attempt 1 succeeded