orcaLog in

Runs · run 1794

load_day 2023-11-29, attempt 1 succeeded

Host
3c2b29713ce8:610635
Started
2026-11-08T03:17:39Z
Ended
2026-11-08T03:22:25Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1794/load_day/2023-11-29/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:17:39Z [load_day 2023-11-29] attempt 1 started
2026-11-08T03:17:42Z [load_day 2023-11-29] 2023-11-29: 2 hours in flight, about 1200 MB
2026-11-08T03:18:08Z [load_day 2023-11-29] 2023-11-29 hour 0: 241984 events in 5 files, snapshot 9199263594162734980 (task 19670 ms: download 13286 ms overlapping gunzip 1351 ms, parse 5984 ms, sort 250 ms, write 11277 ms, index 564 ms: 13 candidates, 88754 repo-hours, 884 releases; waiting 17 ms; commit 4079 ms)
2026-11-08T03:18:11Z [load_day 2023-11-29] 2023-11-29 hour 1: 236616 events in 5 files, snapshot 2530177348054771525 (task 18096 ms: download 9216 ms overlapping gunzip 1153 ms, parse 5062 ms, sort 227 ms, write 10483 ms, index 959 ms: 7 candidates, 94773 repo-hours, 760 releases; waiting 22 ms; commit 3107 ms)
2026-11-08T03:18:25Z [load_day 2023-11-29] 2023-11-29 hour 2: 217875 events in 5 files, snapshot 7508659868952145440 (task 16813 ms: download 8999 ms overlapping gunzip 967 ms, parse 4176 ms, sort 245 ms, write 10193 ms, index 512 ms: 7 candidates, 65847 repo-hours, 754 releases; waiting 481 ms; commit 3002 ms)
2026-11-08T03:18:28Z [load_day 2023-11-29] 2023-11-29 hour 3: 209789 events in 5 files, snapshot 4267359853202222851 (task 16427 ms: download 8978 ms overlapping gunzip 978 ms, parse 4616 ms, sort 200 ms, write 9250 ms, index 557 ms: 10 candidates, 63221 repo-hours, 915 releases; waiting 515 ms; commit 2631 ms)
2026-11-08T03:18:41Z [load_day 2023-11-29] 2023-11-29 hour 4: 216331 events in 5 files, snapshot 1350960769072270417 (task 15279 ms: download 7940 ms overlapping gunzip 972 ms, parse 4116 ms, sort 154 ms, write 8801 ms, index 388 ms: 7 candidates, 58154 repo-hours, 624 releases; waiting 500 ms; commit 3014 ms)
2026-11-08T03:18:47Z [load_day 2023-11-29] 2023-11-29 hour 5: 215623 events in 5 files, snapshot 6213905348755669480 (task 17342 ms: download 8691 ms overlapping gunzip 1196 ms, parse 5346 ms, sort 191 ms, write 8987 ms, index 481 ms: 5 candidates, 62586 repo-hours, 660 releases; waiting 481 ms; commit 2764 ms)
2026-11-08T03:19:00Z [load_day 2023-11-29] 2023-11-29 hour 6: 228455 events in 5 files, snapshot 3849493066625821793 (task 16491 ms: download 8134 ms overlapping gunzip 1089 ms, parse 4686 ms, sort 134 ms, write 9039 ms, index 688 ms: 9 candidates, 74626 repo-hours, 806 releases; waiting 496 ms; commit 4040 ms)
2026-11-08T03:19:07Z [load_day 2023-11-29] 2023-11-29 hour 7: 231018 events in 5 files, snapshot 1424894748937187671 (task 18661 ms: download 11248 ms overlapping gunzip 1013 ms, parse 4778 ms, sort 215 ms, write 11303 ms, index 545 ms: 8 candidates, 73352 repo-hours, 927 releases; waiting 520 ms; commit 3289 ms)
2026-11-08T03:19:19Z [load_day 2023-11-29] 2023-11-29 hour 8: 221504 events in 5 files, snapshot 1976048932773178534 (task 19080 ms: download 10690 ms overlapping gunzip 1236 ms, parse 5283 ms, sort 180 ms, write 10897 ms, index 684 ms: 9 candidates, 82455 repo-hours, 1027 releases; waiting 505 ms; commit 3080 ms)
2026-11-08T03:19:27Z [load_day 2023-11-29] 2023-11-29 hour 9: 239428 events in 5 files, snapshot 7196718327368843319 (task 19024 ms: download 12014 ms overlapping gunzip 1095 ms, parse 4893 ms, sort 161 ms, write 11563 ms, index 578 ms: 13 candidates, 90274 repo-hours, 1166 releases; waiting 493 ms; commit 2623 ms)
2026-11-08T03:19:39Z [load_day 2023-11-29] 2023-11-29 hour 10: 229195 events in 5 files, snapshot 8698537599449855957 (task 17846 ms: download 11092 ms overlapping gunzip 1023 ms, parse 4585 ms, sort 139 ms, write 10292 ms, index 1042 ms: 9 candidates, 85200 repo-hours, 1227 releases; waiting 499 ms; commit 3197 ms)
2026-11-08T03:19:48Z [load_day 2023-11-29] 2023-11-29 hour 11: 238842 events in 5 files, snapshot 7160253452162562764 (task 20493 ms: download 11799 ms overlapping gunzip 1183 ms, parse 5398 ms, sort 162 ms, write 12356 ms, index 611 ms: 9 candidates, 79998 repo-hours, 1184 releases; waiting 495 ms; commit 2821 ms)
2026-11-08T03:20:02Z [load_day 2023-11-29] 2023-11-29 hour 12: 283101 events in 6 files, snapshot 990124299898539553 (task 21933 ms: download 12950 ms overlapping gunzip 1274 ms, parse 5962 ms, sort 208 ms, write 12495 ms, index 1253 ms: 9 candidates, 116517 repo-hours, 1050 releases; waiting 485 ms; commit 3781 ms)
2026-11-08T03:20:14Z [load_day 2023-11-29] 2023-11-29 hour 13: 266721 events in 6 files, snapshot 5038927762334075609 (task 24005 ms: download 13092 ms overlapping gunzip 1384 ms, parse 6337 ms, sort 222 ms, write 14597 ms, index 724 ms: 12 candidates, 107017 repo-hours, 1041 releases; waiting 477 ms; commit 2794 ms)
2026-11-08T03:20:30Z [load_day 2023-11-29] 2023-11-29 hour 14: 266280 events in 6 files, snapshot 2658928926559929655 (task 27534 ms: download 20394 ms overlapping gunzip 1736 ms, parse 8188 ms, sort 201 ms, write 15781 ms, index 831 ms: 8 candidates, 99106 repo-hours, 1205 releases; waiting 494 ms; commit 3037 ms)
2026-11-08T03:20:43Z [load_day 2023-11-29] 2023-11-29 hour 15: 263407 events in 6 files, snapshot 7280485868976926396 (task 27668 ms: download 16610 ms overlapping gunzip 1599 ms, parse 7181 ms, sort 184 ms, write 16816 ms, index 1036 ms: 10 candidates, 100747 repo-hours, 1357 releases; waiting 484 ms; commit 2786 ms)
2026-11-08T03:21:00Z [load_day 2023-11-29] 2023-11-29 hour 16: 260242 events in 6 files, snapshot 8733368153913495395 (task 25726 ms: download 16026 ms overlapping gunzip 1509 ms, parse 6795 ms, sort 270 ms, write 15665 ms, index 787 ms: 20 candidates, 96522 repo-hours, 1221 releases; waiting 495 ms; commit 5912 ms)
2026-11-08T03:21:10Z [load_day 2023-11-29] 2023-11-29 hour 17: 246699 events in 5 files, snapshot 2829520102855294883 (task 25236 ms: download 15241 ms overlapping gunzip 1538 ms, parse 6681 ms, sort 224 ms, write 15141 ms, index 883 ms: 9 candidates, 90125 repo-hours, 1004 releases; waiting 489 ms; commit 3349 ms)
2026-11-08T03:21:23Z [load_day 2023-11-29] 2023-11-29 hour 18: 243762 events in 5 files, snapshot 2194410346512478469 (task 23376 ms: download 15025 ms overlapping gunzip 1472 ms, parse 6741 ms, sort 213 ms, write 13315 ms, index 842 ms: 10 candidates, 90247 repo-hours, 1124 releases; waiting 503 ms; commit 4442 ms)
2026-11-08T03:21:33Z [load_day 2023-11-29] 2023-11-29 hour 19: 225922 events in 5 files, snapshot 2242572138776667926 (task 21700 ms: download 14769 ms overlapping gunzip 1301 ms, parse 5503 ms, sort 208 ms, write 13318 ms, index 603 ms: 9 candidates, 82283 repo-hours, 978 releases; waiting 485 ms; commit 3134 ms)
2026-11-08T03:21:46Z [load_day 2023-11-29] 2023-11-29 hour 20: 217370 events in 5 files, snapshot 758577210835960278 (task 21564 ms: download 11332 ms overlapping gunzip 1225 ms, parse 5151 ms, sort 168 ms, write 13278 ms, index 958 ms: 19 candidates, 79701 repo-hours, 981 releases; waiting 477 ms; commit 4727 ms)
2026-11-08T03:21:56Z [load_day 2023-11-29] 2023-11-29 hour 21: 211581 events in 5 files, snapshot 7883536225573541489 (task 21992 ms: download 14648 ms overlapping gunzip 1362 ms, parse 6078 ms, sort 164 ms, write 12918 ms, index 701 ms: 14 candidates, 76473 repo-hours, 1094 releases; waiting 510 ms; commit 3203 ms)
2026-11-08T03:22:10Z [load_day 2023-11-29] 2023-11-29 hour 22: 215544 events in 5 files, snapshot 1207793514837268866 (task 22514 ms: download 12461 ms overlapping gunzip 1295 ms, parse 5773 ms, sort 185 ms, write 13896 ms, index 647 ms: 6 candidates, 75105 repo-hours, 1047 releases; waiting 495 ms; commit 5250 ms)
2026-11-08T03:22:14Z [load_day 2023-11-29] 2023-11-29 hour 23: 207930 events in 5 files, snapshot 5127554638074289157 (task 17283 ms: download 9384 ms overlapping gunzip 1016 ms, parse 4585 ms, sort 248 ms, write 10041 ms, index 554 ms: 43 candidates, 60851 repo-hours, 734 releases; waiting 581 ms; commit 2389 ms)
2026-11-08T03:22:23Z [load_day 2023-11-29] 2023-11-29: repo_hour merged from 24 files to 1 (1993934 rows, 9034 ms)
2026-11-08T03:22:25Z [load_day 2023-11-29] 2023-11-29: index committed in 11721 ms (24 + 1 + 24 files)
2026-11-08T03:22:25Z [load_day 2023-11-29] 2023-11-29: 5635219 events in 24 hours
2026-11-08T03:22:25Z [load_day 2023-11-29] attempt 1 succeeded