orcaLog in

Runs · run 1744

load_day 2023-11-16, attempt 1 succeeded

Host
3c2b29713ce8:561517
Started
2026-11-08T02:51:25Z
Ended
2026-11-08T02:55:53Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1744/load_day/2023-11-16/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:51:25Z [load_day 2023-11-16] attempt 1 started
2026-11-08T02:51:27Z [load_day 2023-11-16] 2023-11-16: 2 hours in flight, about 1200 MB
2026-11-08T02:51:50Z [load_day 2023-11-16] 2023-11-16 hour 0: 181618 events in 4 files, snapshot 4975582110664930362 (task 16607 ms: download 9396 ms overlapping gunzip 994 ms, parse 4544 ms, sort 146 ms, write 9585 ms, index 707 ms: 4 candidates, 92623 repo-hours, 1246 releases; waiting 364 ms; commit 4213 ms)
2026-11-08T02:51:53Z [load_day 2023-11-16] 2023-11-16 hour 1: 185493 events in 4 files, snapshot 8821432564758689245 (task 16863 ms: download 8765 ms overlapping gunzip 933 ms, parse 4387 ms, sort 121 ms, write 10272 ms, index 928 ms: 9 candidates, 99632 repo-hours, 1028 releases; waiting 26 ms; commit 3013 ms)
2026-11-08T02:52:05Z [load_day 2023-11-16] 2023-11-16 hour 2: 160143 events in 4 files, snapshot 5590073275902473152 (task 14922 ms: download 6012 ms overlapping gunzip 861 ms, parse 3764 ms, sort 149 ms, write 8542 ms, index 812 ms: 7 candidates, 67403 repo-hours, 873 releases; waiting 509 ms; commit 3133 ms)
2026-11-08T02:52:10Z [load_day 2023-11-16] 2023-11-16 hour 3: 157310 events in 4 files, snapshot 3382149953221369068 (task 14799 ms: download 5594 ms overlapping gunzip 882 ms, parse 3908 ms, sort 166 ms, write 9082 ms, index 532 ms: 5 candidates, 64812 repo-hours, 999 releases; waiting 15 ms; commit 3682 ms)
2026-11-08T02:52:19Z [load_day 2023-11-16] 2023-11-16 hour 4: 146658 events in 3 files, snapshot 2674023846281435952 (task 12832 ms: download 4784 ms overlapping gunzip 855 ms, parse 3652 ms, sort 148 ms, write 7546 ms, index 399 ms: 6 candidates, 59102 repo-hours, 773 releases; waiting 25 ms; commit 3030 ms)
2026-11-08T02:52:24Z [load_day 2023-11-16] 2023-11-16 hour 5: 150789 events in 4 files, snapshot 4098182280410006926 (task 14471 ms: download 5838 ms overlapping gunzip 895 ms, parse 3755 ms, sort 158 ms, write 8385 ms, index 560 ms: 4 candidates, 61634 repo-hours, 794 releases; waiting 488 ms; commit 2738 ms)
2026-11-08T02:52:37Z [load_day 2023-11-16] 2023-11-16 hour 6: 171483 events in 4 files, snapshot 2769943083767937175 (task 16969 ms: download 9508 ms overlapping gunzip 1062 ms, parse 4624 ms, sort 153 ms, write 9671 ms, index 664 ms: 2 candidates, 75431 repo-hours, 931 releases; waiting 484 ms; commit 2958 ms)
2026-11-08T02:52:43Z [load_day 2023-11-16] 2023-11-16 hour 7: 177572 events in 4 files, snapshot 1704878344150244264 (task 18120 ms: download 9985 ms overlapping gunzip 1205 ms, parse 5449 ms, sort 171 ms, write 10234 ms, index 854 ms: 7 candidates, 75286 repo-hours, 854 releases; waiting 21 ms; commit 2839 ms)
2026-11-08T02:52:59Z [load_day 2023-11-16] 2023-11-16 hour 8: 196153 events in 4 files, snapshot 1354643712145105902 (task 21105 ms: download 11794 ms overlapping gunzip 1190 ms, parse 5366 ms, sort 199 ms, write 13450 ms, index 655 ms: 9 candidates, 82772 repo-hours, 1165 releases; waiting 28 ms; commit 2835 ms)
2026-11-08T02:53:07Z [load_day 2023-11-16] 2023-11-16 hour 9: 203314 events in 5 files, snapshot 542142133594748795 (task 21667 ms: download 11291 ms overlapping gunzip 1361 ms, parse 5950 ms, sort 225 ms, write 13280 ms, index 621 ms: 3 candidates, 85756 repo-hours, 1296 releases; waiting 18 ms; commit 3415 ms)
2026-11-08T02:53:22Z [load_day 2023-11-16] 2023-11-16 hour 10: 199150 events in 4 files, snapshot 1834201886787341452 (task 21431 ms: download 11758 ms overlapping gunzip 1245 ms, parse 5336 ms, sort 157 ms, write 13747 ms, index 717 ms: 14 candidates, 86769 repo-hours, 1137 releases; waiting 28 ms; commit 3342 ms)
2026-11-08T02:53:35Z [load_day 2023-11-16] 2023-11-16 hour 11: 196390 events in 4 files, snapshot 8696567717655707451 (task 20336 ms: download 10537 ms overlapping gunzip 1221 ms, parse 5452 ms, sort 216 ms, write 12507 ms, index 724 ms: 8 candidates, 83834 repo-hours, 1179 releases; waiting 19 ms; commit 10061 ms)
2026-11-08T02:53:50Z [load_day 2023-11-16] 2023-11-16 hour 12: 242799 events in 5 files, snapshot 4578680137714669652 (task 26310 ms: download 14055 ms overlapping gunzip 1383 ms, parse 6134 ms, sort 201 ms, write 17656 ms, index 702 ms: 6 candidates, 115396 repo-hours, 1156 releases; waiting 14 ms; commit 3075 ms)
2026-11-08T02:53:59Z [load_day 2023-11-16] 2023-11-16 hour 13: 228203 events in 5 files, snapshot 3955598056435593932 (task 29620 ms: download 20937 ms overlapping gunzip 1322 ms, parse 5774 ms, sort 213 ms, write 21253 ms, index 850 ms: 13 candidates, 110157 repo-hours, 1282 releases; waiting 16 ms; commit 3341 ms)
2026-11-08T02:54:12Z [load_day 2023-11-16] 2023-11-16 hour 14: 221502 events in 5 files, snapshot 3634952239584509026 (task 21562 ms: download 14994 ms overlapping gunzip 1347 ms, parse 5938 ms, sort 136 ms, write 12713 ms, index 672 ms: 7 candidates, 98455 repo-hours, 1293 releases; waiting 504 ms; commit 2328 ms)
2026-11-08T02:54:19Z [load_day 2023-11-16] 2023-11-16 hour 15: 217940 events in 5 files, snapshot 2283097573918831631 (task 19563 ms: download 12930 ms overlapping gunzip 1136 ms, parse 4980 ms, sort 142 ms, write 12312 ms, index 770 ms: 7 candidates, 97949 repo-hours, 1473 releases; waiting 23 ms; commit 2388 ms)
2026-11-08T02:54:42Z [load_day 2023-11-16] 2023-11-16 hour 16: 211404 events in 5 files, snapshot 6223491359394681082 (task 22108 ms: download 14294 ms overlapping gunzip 1434 ms, parse 6137 ms, sort 144 ms, write 13515 ms, index 661 ms: 7 candidates, 94474 repo-hours, 1298 releases; waiting 21 ms; commit 9263 ms)
2026-11-08T02:54:44Z [load_day 2023-11-16] 2023-11-16 hour 17: 203272 events in 5 files, snapshot 1626168851292729439 (task 20921 ms: download 11198 ms overlapping gunzip 1141 ms, parse 5370 ms, sort 126 ms, write 13418 ms, index 652 ms: 19 candidates, 85766 repo-hours, 1095 releases; waiting 21 ms; commit 2473 ms)
2026-11-08T02:54:58Z [load_day 2023-11-16] 2023-11-16 hour 18: 204080 events in 5 files, snapshot 6223839923663059664 (task 21395 ms: download 11403 ms overlapping gunzip 1328 ms, parse 6120 ms, sort 137 ms, write 12053 ms, index 988 ms: 5 candidates, 89329 repo-hours, 1100 releases; waiting 504 ms; commit 2930 ms)
2026-11-08T02:55:08Z [load_day 2023-11-16] 2023-11-16 hour 19: 198894 events in 4 files, snapshot 8577678749345327177 (task 21078 ms: download 11340 ms overlapping gunzip 1206 ms, parse 5336 ms, sort 197 ms, write 13434 ms, index 693 ms: 3 candidates, 80927 repo-hours, 1100 releases; waiting 15 ms; commit 3988 ms)
2026-11-08T02:55:19Z [load_day 2023-11-16] 2023-11-16 hour 20: 190060 events in 4 files, snapshot 8465820308663136686 (task 19356 ms: download 10478 ms overlapping gunzip 1064 ms, parse 5016 ms, sort 129 ms, write 12059 ms, index 850 ms: 4 candidates, 78328 repo-hours, 1003 releases; waiting 26 ms; commit 2868 ms)
2026-11-08T02:55:29Z [load_day 2023-11-16] 2023-11-16 hour 21: 181595 events in 4 files, snapshot 7907858375901203035 (task 18186 ms: download 10180 ms overlapping gunzip 1071 ms, parse 4657 ms, sort 123 ms, write 11623 ms, index 507 ms: 5 candidates, 73269 repo-hours, 1198 releases; waiting 18 ms; commit 5935 ms)
2026-11-08T02:55:39Z [load_day 2023-11-16] 2023-11-16 hour 22: 172283 events in 4 files, snapshot 5473533464993082124 (task 19038 ms: download 11757 ms overlapping gunzip 1094 ms, parse 4651 ms, sort 113 ms, write 11753 ms, index 606 ms: 5 candidates, 69611 repo-hours, 1101 releases; waiting 508 ms; commit 3760 ms)
2026-11-08T02:55:42Z [load_day 2023-11-16] 2023-11-16 hour 23: 157410 events in 4 files, snapshot 9070844876934736419 (task 14951 ms: download 6584 ms overlapping gunzip 1033 ms, parse 4675 ms, sort 93 ms, write 8400 ms, index 508 ms: 6 candidates, 63150 repo-hours, 892 releases; waiting 22 ms; commit 2244 ms)
2026-11-08T02:55:51Z [load_day 2023-11-16] 2023-11-16: repo_hour merged from 24 files to 1 (1991865 rows, 8987 ms)
2026-11-08T02:55:53Z [load_day 2023-11-16] 2023-11-16: index committed in 11292 ms (24 + 1 + 24 files)
2026-11-08T02:55:53Z [load_day 2023-11-16] 2023-11-16: 4555515 events in 24 hours
2026-11-08T02:55:53Z [load_day 2023-11-16] attempt 1 succeeded