orcaLog in

Runs · run 1532

load_day 2023-09-21, attempt 1 succeeded

Host
3c2b29713ce8:352978
Started
2026-11-08T01:01:47Z
Ended
2026-11-08T01:05:53Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1532/load_day/2023-09-21/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:01:47Z [load_day 2023-09-21] attempt 1 started
2026-11-08T01:01:48Z [load_day 2023-09-21] 2023-09-21: 2 hours in flight, about 1200 MB
2026-11-08T01:02:08Z [load_day 2023-09-21] 2023-09-21 hour 0: 170034 events in 4 files, snapshot 641442754005835364 (task 15415 ms: download 6320 ms overlapping gunzip 887 ms, parse 4114 ms, sort 161 ms, write 8756 ms, index 646 ms: 11 candidates, 93862 repo-hours, 962 releases; waiting 517 ms; commit 2706 ms)
2026-11-08T01:02:12Z [load_day 2023-09-21] 2023-09-21 hour 1: 162233 events in 4 files, snapshot 6758103628720884753 (task 14803 ms: download 5547 ms overlapping gunzip 752 ms, parse 3425 ms, sort 156 ms, write 9001 ms, index 721 ms: 11 candidates, 91380 repo-hours, 729 releases; waiting 527 ms; commit 3520 ms)
2026-11-08T01:02:23Z [load_day 2023-09-21] 2023-09-21 hour 2: 143994 events in 3 files, snapshot 7274519460159447904 (task 13234 ms: download 5553 ms overlapping gunzip 774 ms, parse 3367 ms, sort 140 ms, write 7622 ms, index 462 ms: 14 candidates, 62097 repo-hours, 760 releases; waiting 565 ms; commit 2733 ms)
2026-11-08T01:02:25Z [load_day 2023-09-21] 2023-09-21 hour 3: 137567 events in 3 files, snapshot 3659663706054758930 (task 13385 ms: download 5490 ms overlapping gunzip 791 ms, parse 3501 ms, sort 137 ms, write 7711 ms, index 472 ms: 11 candidates, 60215 repo-hours, 916 releases; waiting 462 ms; commit 2357 ms)
2026-11-08T01:02:36Z [load_day 2023-09-21] 2023-09-21 hour 4: 133593 events in 3 files, snapshot 698156328942698601 (task 12435 ms: download 4568 ms overlapping gunzip 766 ms, parse 3484 ms, sort 105 ms, write 6836 ms, index 378 ms: 14 candidates, 55934 repo-hours, 739 releases; waiting 524 ms; commit 2480 ms)
2026-11-08T01:02:40Z [load_day 2023-09-21] 2023-09-21 hour 5: 141471 events in 3 files, snapshot 2994379064415821788 (task 12833 ms: download 4797 ms overlapping gunzip 678 ms, parse 2997 ms, sort 141 ms, write 7945 ms, index 423 ms: 8 candidates, 61350 repo-hours, 762 releases; waiting 436 ms; commit 2906 ms)
2026-11-08T01:02:54Z [load_day 2023-09-21] 2023-09-21 hour 6: 172581 events in 4 files, snapshot 5376329083713757433 (task 17202 ms: download 9952 ms overlapping gunzip 1097 ms, parse 4804 ms, sort 147 ms, write 9651 ms, index 713 ms: 5 candidates, 79732 repo-hours, 955 releases; waiting 490 ms; commit 2738 ms)
2026-11-08T01:03:01Z [load_day 2023-09-21] 2023-09-21 hour 7: 182460 events in 4 files, snapshot 314973592967510282 (task 19987 ms: download 10972 ms overlapping gunzip 1044 ms, parse 4746 ms, sort 123 ms, write 12194 ms, index 1094 ms: 11 candidates, 81625 repo-hours, 1044 releases; waiting 507 ms; commit 3123 ms)
2026-11-08T01:03:18Z [load_day 2023-09-21] 2023-09-21 hour 8: 192799 events in 4 files, snapshot 5953602704154109260 (task 22877 ms: download 13109 ms overlapping gunzip 1455 ms, parse 6056 ms, sort 227 ms, write 13793 ms, index 593 ms: 16 candidates, 86136 repo-hours, 1243 releases; waiting 518 ms; commit 2608 ms)
2026-11-08T01:03:24Z [load_day 2023-09-21] 2023-09-21 hour 9: 200469 events in 5 files, snapshot 1219360434976958038 (task 21798 ms: download 11979 ms overlapping gunzip 1272 ms, parse 5699 ms, sort 156 ms, write 13140 ms, index 686 ms: 15 candidates, 89339 repo-hours, 1181 releases; waiting 506 ms; commit 2721 ms)
2026-11-08T01:03:40Z [load_day 2023-09-21] 2023-09-21 hour 10: 195114 events in 4 files, snapshot 6075106672048499041 (task 20649 ms: download 11512 ms overlapping gunzip 1188 ms, parse 5442 ms, sort 187 ms, write 12461 ms, index 562 ms: 8 candidates, 84238 repo-hours, 1106 releases; waiting 495 ms; commit 2502 ms)
2026-11-08T01:03:46Z [load_day 2023-09-21] 2023-09-21 hour 11: 190573 events in 4 files, snapshot 7582399337624609165 (task 19998 ms: download 11089 ms overlapping gunzip 1191 ms, parse 5658 ms, sort 150 ms, write 11582 ms, index 702 ms: 9 candidates, 81954 repo-hours, 1367 releases; waiting 497 ms; commit 3124 ms)
2026-11-08T01:04:05Z [load_day 2023-09-21] 2023-09-21 hour 12: 238167 events in 5 files, snapshot 4550426009706603751 (task 23841 ms: download 15336 ms overlapping gunzip 1364 ms, parse 6558 ms, sort 242 ms, write 13777 ms, index 1048 ms: 13 candidates, 115259 repo-hours, 1279 releases; waiting 514 ms; commit 2694 ms)
2026-11-08T01:04:09Z [load_day 2023-09-21] 2023-09-21 hour 13: 218362 events in 5 files, snapshot 6517679932671337049 (task 22602 ms: download 14741 ms overlapping gunzip 1219 ms, parse 5545 ms, sort 131 ms, write 14325 ms, index 917 ms: 11 candidates, 109487 repo-hours, 1116 releases; waiting 211 ms; commit 2641 ms)
2026-11-08T01:04:28Z [load_day 2023-09-21] 2023-09-21 hour 14: 208130 events in 5 files, snapshot 2001638260420041144 (task 22130 ms: download 12622 ms overlapping gunzip 1422 ms, parse 6368 ms, sort 207 ms, write 12657 ms, index 690 ms: 13 candidates, 96001 repo-hours, 1211 releases; waiting 522 ms; commit 2214 ms)
2026-11-08T01:04:30Z [load_day 2023-09-21] 2023-09-21 hour 15: 193953 events in 4 files, snapshot 2606364589969967386 (task 19946 ms: download 11352 ms overlapping gunzip 1094 ms, parse 4897 ms, sort 123 ms, write 12479 ms, index 588 ms: 14 candidates, 92527 repo-hours, 1149 releases; waiting 522 ms; commit 2496 ms)
2026-11-08T01:04:45Z [load_day 2023-09-21] 2023-09-21 hour 16: 189267 events in 4 files, snapshot 699717782523286394 (task 15721 ms: download 8948 ms overlapping gunzip 900 ms, parse 3993 ms, sort 126 ms, write 9370 ms, index 563 ms: 14 candidates, 85740 repo-hours, 1038 releases; waiting 515 ms; commit 2346 ms)
2026-11-08T01:04:47Z [load_day 2023-09-21] 2023-09-21 hour 17: 193386 events in 4 files, snapshot 3078214774015132387 (task 16562 ms: download 9362 ms overlapping gunzip 998 ms, parse 4274 ms, sort 132 ms, write 9604 ms, index 729 ms: 13 candidates, 83422 repo-hours, 989 releases; waiting 524 ms; commit 2122 ms)
2026-11-08T01:05:08Z [load_day 2023-09-21] 2023-09-21 hour 18: 197887 events in 4 files, snapshot 6635563999099490885 (task 19948 ms: download 10582 ms overlapping gunzip 1140 ms, parse 5031 ms, sort 228 ms, write 12092 ms, index 604 ms: 9 candidates, 88135 repo-hours, 1038 releases; waiting 512 ms; commit 4418 ms)
2026-11-08T01:05:13Z [load_day 2023-09-21] 2023-09-21 hour 19: 178329 events in 4 files, snapshot 3235412036182201830 (task 17456 ms: download 10222 ms overlapping gunzip 949 ms, parse 4304 ms, sort 106 ms, write 11120 ms, index 533 ms: 10 candidates, 73803 repo-hours, 914 releases; waiting 187 ms; commit 5296 ms)
2026-11-08T01:05:26Z [load_day 2023-09-21] 2023-09-21 hour 20: 174247 events in 4 files, snapshot 5549382437771758996 (task 18675 ms: download 11048 ms overlapping gunzip 1062 ms, parse 4876 ms, sort 137 ms, write 10524 ms, index 979 ms: 14 candidates, 73822 repo-hours, 988 releases; waiting 548 ms; commit 2600 ms)
2026-11-08T01:05:29Z [load_day 2023-09-21] 2023-09-21 hour 21: 157178 events in 4 files, snapshot 1503591081254211997 (task 15651 ms: download 6468 ms overlapping gunzip 878 ms, parse 4062 ms, sort 141 ms, write 9559 ms, index 434 ms: 11 candidates, 65117 repo-hours, 1026 releases; waiting 267 ms; commit 2962 ms)
2026-11-08T01:05:40Z [load_day 2023-09-21] 2023-09-21 hour 22: 137535 events in 3 files, snapshot 8647234089170326883 (task 13451 ms: download 5521 ms overlapping gunzip 820 ms, parse 3549 ms, sort 100 ms, write 7636 ms, index 541 ms: 18 candidates, 55697 repo-hours, 782 releases; waiting 525 ms; commit 2270 ms)
2026-11-08T01:05:42Z [load_day 2023-09-21] 2023-09-21 hour 23: 125744 events in 3 files, snapshot 878630521720281744 (task 11233 ms: download 4910 ms overlapping gunzip 632 ms, parse 2760 ms, sort 72 ms, write 6603 ms, index 386 ms: 12 candidates, 52514 repo-hours, 688 releases; waiting 496 ms; commit 2045 ms)
2026-11-08T01:05:50Z [load_day 2023-09-21] 2023-09-21: repo_hour merged from 24 files to 1 (1919386 rows, 8323 ms)
2026-11-08T01:05:53Z [load_day 2023-09-21] 2023-09-21: index committed in 10874 ms (24 + 1 + 24 files)
2026-11-08T01:05:53Z [load_day 2023-09-21] 2023-09-21: 4235073 events in 24 hours
2026-11-08T01:05:53Z [load_day 2023-09-21] attempt 1 succeeded