orcaLog in

Runs · run 1612

load_day 2023-10-11, attempt 1 succeeded

Host
3c2b29713ce8:425141
Started
2026-11-08T01:40:16Z
Ended
2026-11-08T01:44:50Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1612/load_day/2023-10-11/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:40:16Z [load_day 2023-10-11] attempt 1 started
2026-11-08T01:40:18Z [load_day 2023-10-11] 2023-10-11: 2 hours in flight, about 1200 MB
2026-11-08T01:40:41Z [load_day 2023-10-11] 2023-10-11 hour 0: 182681 events in 4 files, snapshot 7346650974882694252 (task 18753 ms: download 11775 ms overlapping gunzip 1244 ms, parse 5535 ms, sort 175 ms, write 10283 ms, index 735 ms: 6 candidates, 95099 repo-hours, 1124 releases; waiting 507 ms; commit 2215 ms)
2026-11-08T01:40:43Z [load_day 2023-10-11] 2023-10-11 hour 1: 180482 events in 4 files, snapshot 8852915726170757354 (task 18203 ms: download 10742 ms overlapping gunzip 1031 ms, parse 4739 ms, sort 147 ms, write 10684 ms, index 686 ms: 5 candidates, 96815 repo-hours, 813 releases; waiting 480 ms; commit 2585 ms)
2026-11-08T01:40:56Z [load_day 2023-10-11] 2023-10-11 hour 2: 149483 events in 3 files, snapshot 7850360519924043228 (task 14247 ms: download 6190 ms overlapping gunzip 894 ms, parse 3863 ms, sort 94 ms, write 7880 ms, index 676 ms: 6 candidates, 65014 repo-hours, 829 releases; waiting 548 ms; commit 2215 ms)
2026-11-08T01:40:59Z [load_day 2023-10-11] 2023-10-11 hour 3: 144012 events in 3 files, snapshot 6519859045952964587 (task 13576 ms: download 6216 ms overlapping gunzip 773 ms, parse 3387 ms, sort 86 ms, write 7957 ms, index 462 ms: 7 candidates, 62002 repo-hours, 897 releases; waiting 517 ms; commit 3438 ms)
2026-11-08T01:41:16Z [load_day 2023-10-11] 2023-10-11 hour 4: 157120 events in 4 files, snapshot 8139756408324523815 (task 18147 ms: download 11657 ms overlapping gunzip 1243 ms, parse 5179 ms, sort 110 ms, write 10284 ms, index 539 ms: 5 candidates, 60860 repo-hours, 626 releases; waiting 534 ms; commit 2943 ms)
2026-11-08T01:41:19Z [load_day 2023-10-11] 2023-10-11 hour 5: 144201 events in 3 files, snapshot 4121762194794804595 (task 14623 ms: download 6147 ms overlapping gunzip 873 ms, parse 3715 ms, sort 129 ms, write 8633 ms, index 459 ms: 8 candidates, 62971 repo-hours, 672 releases; waiting 492 ms; commit 3102 ms)
2026-11-08T01:41:43Z [load_day 2023-10-11] 2023-10-11 hour 6: 209104 events in 5 files, snapshot 5534031788210253032 (task 26150 ms: download 18010 ms overlapping gunzip 1679 ms, parse 7325 ms, sort 133 ms, write 15587 ms, index 639 ms: 12 candidates, 89563 repo-hours, 781 releases; waiting 484 ms; commit 2373 ms)
2026-11-08T01:41:47Z [load_day 2023-10-11] 2023-10-11 hour 7: 194031 events in 4 files, snapshot 8029786188877555829 (task 22879 ms: download 13724 ms overlapping gunzip 1357 ms, parse 5888 ms, sort 137 ms, write 13904 ms, index 660 ms: 9 candidates, 83967 repo-hours, 983 releases; waiting 534 ms; commit 4571 ms)
2026-11-08T01:42:04Z [load_day 2023-10-11] 2023-10-11 hour 8: 184193 events in 4 files, snapshot 5629374966130424833 (task 19797 ms: download 11757 ms overlapping gunzip 1068 ms, parse 4490 ms, sort 173 ms, write 12592 ms, index 689 ms: 9 candidates, 83232 repo-hours, 1094 releases; waiting 520 ms; commit 2435 ms)
2026-11-08T01:42:08Z [load_day 2023-10-11] 2023-10-11 hour 9: 201469 events in 5 files, snapshot 7799342966152598057 (task 19828 ms: download 11790 ms overlapping gunzip 1336 ms, parse 6082 ms, sort 143 ms, write 11039 ms, index 687 ms: 6 candidates, 88025 repo-hours, 1205 releases; waiting 234 ms; commit 3235 ms)
2026-11-08T01:42:23Z [load_day 2023-10-11] 2023-10-11 hour 10: 192911 events in 4 files, snapshot 4351188718584876423 (task 16816 ms: download 9717 ms overlapping gunzip 968 ms, parse 4127 ms, sort 138 ms, write 9566 ms, index 603 ms: 9 candidates, 86383 repo-hours, 1054 releases; waiting 1062 ms; commit 2888 ms)
2026-11-08T01:42:26Z [load_day 2023-10-11] 2023-10-11 hour 11: 190752 events in 4 files, snapshot 8585386278751010758 (task 16102 ms: download 9074 ms overlapping gunzip 924 ms, parse 4152 ms, sort 126 ms, write 9472 ms, index 630 ms: 5 candidates, 83589 repo-hours, 1034 releases; waiting 506 ms; commit 3201 ms)
2026-11-08T01:42:47Z [load_day 2023-10-11] 2023-10-11 hour 12: 243567 events in 5 files, snapshot 668674132252480155 (task 23575 ms: download 15504 ms overlapping gunzip 1294 ms, parse 5983 ms, sort 191 ms, write 14421 ms, index 852 ms: 6 candidates, 115681 repo-hours, 1142 releases; waiting 539 ms; commit 3021 ms)
2026-11-08T01:42:51Z [load_day 2023-10-11] 2023-10-11 hour 13: 215003 events in 5 files, snapshot 7007626554989170884 (task 22535 ms: download 15043 ms overlapping gunzip 1222 ms, parse 5656 ms, sort 136 ms, write 14057 ms, index 705 ms: 11 candidates, 107123 repo-hours, 1202 releases; waiting 494 ms; commit 3089 ms)
2026-11-08T01:43:11Z [load_day 2023-10-11] 2023-10-11 hour 14: 212770 events in 5 files, snapshot 8844735007039179300 (task 22662 ms: download 15871 ms overlapping gunzip 1316 ms, parse 5700 ms, sort 120 ms, write 14028 ms, index 701 ms: 4 candidates, 97668 repo-hours, 1345 releases; waiting 535 ms; commit 2404 ms)
2026-11-08T01:43:16Z [load_day 2023-10-11] 2023-10-11 hour 15: 183277 events in 4 files, snapshot 7298523865155213399 (task 20104 ms: download 11952 ms overlapping gunzip 1125 ms, parse 5285 ms, sort 117 ms, write 12174 ms, index 665 ms: 7 candidates, 88781 repo-hours, 1117 releases; waiting 516 ms; commit 5120 ms)
2026-11-08T01:43:36Z [load_day 2023-10-11] 2023-10-11 hour 16: 186797 events in 4 files, snapshot 1452669410057603482 (task 21531 ms: download 13014 ms overlapping gunzip 1098 ms, parse 4746 ms, sort 195 ms, write 14045 ms, index 642 ms: 12 candidates, 86865 repo-hours, 1033 releases; waiting 526 ms; commit 4984 ms)
2026-11-08T01:43:39Z [load_day 2023-10-11] 2023-10-11 hour 17: 188806 events in 4 files, snapshot 1492739901185266287 (task 19748 ms: download 11497 ms overlapping gunzip 1328 ms, parse 6184 ms, sort 152 ms, write 10745 ms, index 570 ms: 8 candidates, 83406 repo-hours, 1008 releases; waiting 477 ms; commit 2721 ms)
2026-11-08T01:43:55Z [load_day 2023-10-11] 2023-10-11 hour 18: 191182 events in 4 files, snapshot 3210000873709264819 (task 20458 ms: download 11383 ms overlapping gunzip 1306 ms, parse 5757 ms, sort 192 ms, write 11778 ms, index 645 ms: 13 candidates, 87843 repo-hours, 1442 releases; waiting 511 ms; commit 2754 ms)
2026-11-08T01:44:00Z [load_day 2023-10-11] 2023-10-11 hour 19: 181952 events in 4 files, snapshot 7393888718430519411 (task 18916 ms: download 10951 ms overlapping gunzip 1043 ms, parse 5052 ms, sort 157 ms, write 11132 ms, index 682 ms: 10 candidates, 76702 repo-hours, 1127 releases; waiting 402 ms; commit 3854 ms)
2026-11-08T01:44:16Z [load_day 2023-10-11] 2023-10-11 hour 20: 178675 events in 4 files, snapshot 7836256595749829529 (task 19340 ms: download 11004 ms overlapping gunzip 1110 ms, parse 5065 ms, sort 116 ms, write 11385 ms, index 801 ms: 9 candidates, 75265 repo-hours, 1121 releases; waiting 440 ms; commit 2735 ms)
2026-11-08T01:44:20Z [load_day 2023-10-11] 2023-10-11 hour 21: 173302 events in 4 files, snapshot 5199555891697708986 (task 19887 ms: download 11728 ms overlapping gunzip 1041 ms, parse 4619 ms, sort 119 ms, write 12313 ms, index 993 ms: 10 candidates, 67755 repo-hours, 990 releases; waiting 523 ms; commit 3039 ms)
2026-11-08T01:44:34Z [load_day 2023-10-11] 2023-10-11 hour 22: 157365 events in 4 files, snapshot 3269189457129729126 (task 17189 ms: download 10686 ms overlapping gunzip 917 ms, parse 4191 ms, sort 143 ms, write 10756 ms, index 558 ms: 3 candidates, 61951 repo-hours, 853 releases; waiting 358 ms; commit 2601 ms)
2026-11-08T01:44:39Z [load_day 2023-10-11] 2023-10-11 hour 23: 159149 events in 4 files, snapshot 4323580239057291024 (task 17800 ms: download 12048 ms overlapping gunzip 1069 ms, parse 4810 ms, sort 116 ms, write 10480 ms, index 451 ms: 5 candidates, 63468 repo-hours, 679 releases; waiting 533 ms; commit 2313 ms)
2026-11-08T01:44:48Z [load_day 2023-10-11] 2023-10-11: repo_hour merged from 24 files to 1 (1970028 rows, 8882 ms)
2026-11-08T01:44:50Z [load_day 2023-10-11] 2023-10-11: index committed in 11392 ms (24 + 1 + 24 files)
2026-11-08T01:44:50Z [load_day 2023-10-11] 2023-10-11: 4402284 events in 24 hours
2026-11-08T01:44:50Z [load_day 2023-10-11] attempt 1 succeeded