orcaLog in

Runs · run 1424

load_day 2023-08-24, attempt 1 succeeded

Host
3c2b29713ce8:257547
Started
2026-11-08T00:10:25Z
Ended
2026-11-08T00:14:30Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1424/load_day/2023-08-24/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:10:25Z [load_day 2023-08-24] attempt 1 started
2026-11-08T00:10:27Z [load_day 2023-08-24] 2023-08-24: 2 hours in flight, about 1200 MB
2026-11-08T00:10:47Z [load_day 2023-08-24] 2023-08-24 hour 0: 172710 events in 4 files, snapshot 1196990649816340307 (task 15981 ms: download 7156 ms overlapping gunzip 915 ms, parse 4400 ms, sort 165 ms, write 9116 ms, index 597 ms: 5 candidates, 94944 repo-hours, 956 releases; waiting 496 ms; commit 2432 ms)
2026-11-08T00:10:55Z [load_day 2023-08-24] 2023-08-24 hour 1: 163985 events in 4 files, snapshot 4876094262383577406 (task 15313 ms: download 6239 ms overlapping gunzip 807 ms, parse 3878 ms, sort 198 ms, write 8962 ms, index 689 ms: 5 candidates, 90713 repo-hours, 897 releases; waiting 461 ms; commit 8247 ms)
2026-11-08T00:11:03Z [load_day 2023-08-24] 2023-08-24 hour 2: 144032 events in 3 files, snapshot 1756754441942434916 (task 15876 ms: download 7279 ms overlapping gunzip 984 ms, parse 4162 ms, sort 148 ms, write 9244 ms, index 492 ms: 7 candidates, 62966 repo-hours, 776 releases; waiting 506 ms; commit 2138 ms)
2026-11-08T00:11:06Z [load_day 2023-08-24] 2023-08-24 hour 3: 135896 events in 3 files, snapshot 3629144717960655192 (task 13079 ms: download 5821 ms overlapping gunzip 802 ms, parse 3509 ms, sort 130 ms, write 7402 ms, index 399 ms: 5 candidates, 57953 repo-hours, 836 releases; waiting 471 ms; commit 2351 ms)
2026-11-08T00:11:17Z [load_day 2023-08-24] 2023-08-24 hour 4: 131141 events in 3 files, snapshot 3842279363163267401 (task 11677 ms: download 4105 ms overlapping gunzip 727 ms, parse 3274 ms, sort 141 ms, write 6694 ms, index 372 ms: 2 candidates, 54782 repo-hours, 660 releases; waiting 186 ms; commit 3136 ms)
2026-11-08T00:11:22Z [load_day 2023-08-24] 2023-08-24 hour 5: 143198 events in 3 files, snapshot 2263567824544110454 (task 14827 ms: download 5487 ms overlapping gunzip 823 ms, parse 3541 ms, sort 117 ms, write 8882 ms, index 622 ms: 5 candidates, 61294 repo-hours, 762 releases; waiting 507 ms; commit 2940 ms)
2026-11-08T00:11:35Z [load_day 2023-08-24] 2023-08-24 hour 6: 166259 events in 4 files, snapshot 5960159771064383048 (task 17466 ms: download 10264 ms overlapping gunzip 1106 ms, parse 4890 ms, sort 179 ms, write 9704 ms, index 760 ms: 6 candidates, 78733 repo-hours, 874 releases; waiting 505 ms; commit 2371 ms)
2026-11-08T00:11:41Z [load_day 2023-08-24] 2023-08-24 hour 7: 169715 events in 4 files, snapshot 3984000677865932427 (task 18363 ms: download 9899 ms overlapping gunzip 1084 ms, parse 4809 ms, sort 119 ms, write 10774 ms, index 755 ms: 1 candidates, 74724 repo-hours, 970 releases; waiting 487 ms; commit 2628 ms)
2026-11-08T00:11:56Z [load_day 2023-08-24] 2023-08-24 hour 8: 178537 events in 4 files, snapshot 991852366072706433 (task 19880 ms: download 11862 ms overlapping gunzip 1132 ms, parse 5084 ms, sort 125 ms, write 12134 ms, index 574 ms: 5 candidates, 80280 repo-hours, 997 releases; waiting 487 ms; commit 2098 ms)
2026-11-08T00:12:03Z [load_day 2023-08-24] 2023-08-24 hour 9: 184493 events in 4 files, snapshot 4729395040145042683 (task 20686 ms: download 11068 ms overlapping gunzip 1249 ms, parse 5581 ms, sort 223 ms, write 12201 ms, index 674 ms: 5 candidates, 81936 repo-hours, 1188 releases; waiting 521 ms; commit 2374 ms)
2026-11-08T00:12:16Z [load_day 2023-08-24] 2023-08-24 hour 10: 180530 events in 4 files, snapshot 8228140665863679950 (task 19231 ms: download 11765 ms overlapping gunzip 1022 ms, parse 4583 ms, sort 214 ms, write 12084 ms, index 556 ms: 8 candidates, 79611 repo-hours, 1003 releases; waiting 486 ms; commit 2407 ms)
2026-11-08T00:12:23Z [load_day 2023-08-24] 2023-08-24 hour 11: 186711 events in 4 files, snapshot 4790296631954587589 (task 19327 ms: download 10521 ms overlapping gunzip 1131 ms, parse 5047 ms, sort 187 ms, write 11281 ms, index 874 ms: 6 candidates, 81401 repo-hours, 1039 releases; waiting 498 ms; commit 2516 ms)
2026-11-08T00:12:40Z [load_day 2023-08-24] 2023-08-24 hour 12: 222596 events in 5 files, snapshot 422370050445320526 (task 21802 ms: download 12052 ms overlapping gunzip 1221 ms, parse 5646 ms, sort 149 ms, write 13234 ms, index 756 ms: 4 candidates, 108668 repo-hours, 1063 releases; waiting 491 ms; commit 2619 ms)
2026-11-08T00:12:47Z [load_day 2023-08-24] 2023-08-24 hour 13: 200024 events in 5 files, snapshot 7640751795220599135 (task 22253 ms: download 12002 ms overlapping gunzip 1341 ms, parse 5882 ms, sort 223 ms, write 13325 ms, index 682 ms: 5 candidates, 101954 repo-hours, 1066 releases; waiting 486 ms; commit 2569 ms)
2026-11-08T00:13:00Z [load_day 2023-08-24] 2023-08-24 hour 14: 195996 events in 4 files, snapshot 6236463313254127100 (task 19315 ms: download 11655 ms overlapping gunzip 1126 ms, parse 5018 ms, sort 120 ms, write 11609 ms, index 624 ms: 5 candidates, 91823 repo-hours, 1100 releases; waiting 506 ms; commit 1889 ms)
2026-11-08T00:13:03Z [load_day 2023-08-24] 2023-08-24 hour 15: 178341 events in 4 files, snapshot 3142193408950889903 (task 16480 ms: download 9673 ms overlapping gunzip 1067 ms, parse 4569 ms, sort 105 ms, write 9290 ms, index 603 ms: 9 candidates, 84834 repo-hours, 1121 releases; waiting 490 ms; commit 1906 ms)
2026-11-08T00:13:18Z [load_day 2023-08-24] 2023-08-24 hour 16: 174703 events in 4 files, snapshot 8112603184536426898 (task 17033 ms: download 9157 ms overlapping gunzip 951 ms, parse 4055 ms, sort 123 ms, write 10578 ms, index 558 ms: 6 candidates, 79841 repo-hours, 920 releases; waiting 515 ms; commit 2818 ms)
2026-11-08T00:13:27Z [load_day 2023-08-24] 2023-08-24 hour 17: 173433 events in 4 files, snapshot 6881706658660756967 (task 19839 ms: download 11684 ms overlapping gunzip 1080 ms, parse 4821 ms, sort 134 ms, write 12089 ms, index 540 ms: 3 candidates, 76740 repo-hours, 876 releases; waiting 857 ms; commit 4353 ms)
2026-11-08T00:13:38Z [load_day 2023-08-24] 2023-08-24 hour 18: 185853 events in 4 files, snapshot 272315587981986687 (task 19645 ms: download 11436 ms overlapping gunzip 1164 ms, parse 5168 ms, sort 115 ms, write 11731 ms, index 632 ms: 12 candidates, 84017 repo-hours, 992 releases; waiting 489 ms; commit 2259 ms)
2026-11-08T00:13:45Z [load_day 2023-08-24] 2023-08-24 hour 19: 171238 events in 4 files, snapshot 2212426893963649851 (task 18286 ms: download 11188 ms overlapping gunzip 1030 ms, parse 4790 ms, sort 134 ms, write 10658 ms, index 897 ms: 6 candidates, 71585 repo-hours, 914 releases; waiting 467 ms; commit 3508 ms)
2026-11-08T00:14:00Z [load_day 2023-08-24] 2023-08-24 hour 20: 160864 events in 4 files, snapshot 185796009446564894 (task 18732 ms: download 11655 ms overlapping gunzip 1070 ms, parse 5040 ms, sort 199 ms, write 10563 ms, index 579 ms: 5 candidates, 70163 repo-hours, 799 releases; waiting 984 ms; commit 4547 ms)
2026-11-08T00:14:03Z [load_day 2023-08-24] 2023-08-24 hour 21: 151614 events in 4 files, snapshot 9159469972155193833 (task 16491 ms: download 7482 ms overlapping gunzip 1074 ms, parse 4656 ms, sort 123 ms, write 9213 ms, index 624 ms: 7 candidates, 62769 repo-hours, 945 releases; waiting 515 ms; commit 2150 ms)
2026-11-08T00:14:14Z [load_day 2023-08-24] 2023-08-24 hour 22: 142877 events in 3 files, snapshot 7783254913241302548 (task 15105 ms: download 6436 ms overlapping gunzip 919 ms, parse 4013 ms, sort 115 ms, write 8871 ms, index 394 ms: 8 candidates, 57925 repo-hours, 729 releases; waiting 525 ms; commit 2289 ms)
2026-11-08T00:14:20Z [load_day 2023-08-24] 2023-08-24 hour 23: 135322 events in 3 files, snapshot 6735324985881839314 (task 13582 ms: download 6090 ms overlapping gunzip 718 ms, parse 3193 ms, sort 113 ms, write 8389 ms, index 384 ms: 4 candidates, 56308 repo-hours, 651 releases; waiting 523 ms; commit 4809 ms)
2026-11-08T00:14:28Z [load_day 2023-08-24] 2023-08-24: repo_hour merged from 24 files to 1 (1845964 rows, 8102 ms)
2026-11-08T00:14:30Z [load_day 2023-08-24] 2023-08-24: index committed in 10162 ms (24 + 1 + 24 files)
2026-11-08T00:14:30Z [load_day 2023-08-24] 2023-08-24: 4050068 events in 24 hours
2026-11-08T00:14:30Z [load_day 2023-08-24] attempt 1 succeeded