orcaLog in

Runs · run 1784

load_day 2023-11-27, attempt 1 succeeded

Host
3c2b29713ce8:601565
Started
2026-11-08T03:12:51Z
Ended
2026-11-08T03:17:38Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1784/load_day/2023-11-27/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:12:51Z [load_day 2023-11-27] attempt 1 started
2026-11-08T03:12:52Z [load_day 2023-11-27] 2023-11-27: 2 hours in flight, about 1200 MB
2026-11-08T03:13:16Z [load_day 2023-11-27] 2023-11-27 hour 0: 239228 events in 5 files, snapshot 130259047416186080 (task 17341 ms: download 9216 ms overlapping gunzip 1058 ms, parse 4766 ms, sort 255 ms, write 10518 ms, index 536 ms: 10 candidates, 78334 repo-hours, 834 releases; waiting 21 ms; commit 4860 ms)
2026-11-08T03:13:19Z [load_day 2023-11-27] 2023-11-27 hour 1: 272078 events in 6 files, snapshot 423851315498893664 (task 20244 ms: download 11888 ms overlapping gunzip 1300 ms, parse 5848 ms, sort 261 ms, write 11352 ms, index 1266 ms: 10 candidates, 86427 repo-hours, 666 releases; waiting 26 ms; commit 2865 ms)
2026-11-08T03:13:32Z [load_day 2023-11-27] 2023-11-27 hour 2: 259364 events in 6 files, snapshot 4493710931893327450 (task 16312 ms: download 9130 ms overlapping gunzip 1105 ms, parse 5135 ms, sort 268 ms, write 8800 ms, index 786 ms: 10 candidates, 61371 repo-hours, 689 releases; waiting 23 ms; commit 3123 ms)
2026-11-08T03:13:37Z [load_day 2023-11-27] 2023-11-27 hour 3: 292158 events in 6 files, snapshot 974474401776594237 (task 15869 ms: download 8821 ms overlapping gunzip 1081 ms, parse 4928 ms, sort 261 ms, write 8375 ms, index 428 ms: 5 candidates, 51473 repo-hours, 622 releases; waiting 520 ms; commit 3058 ms)
2026-11-08T03:13:48Z [load_day 2023-11-27] 2023-11-27 hour 4: 293398 events in 6 files, snapshot 146525898821126942 (task 14776 ms: download 7279 ms overlapping gunzip 921 ms, parse 4342 ms, sort 320 ms, write 8568 ms, index 397 ms: 10 candidates, 46457 repo-hours, 431 releases; waiting 19 ms; commit 2982 ms)
2026-11-08T03:13:57Z [load_day 2023-11-27] 2023-11-27 hour 5: 257947 events in 6 files, snapshot 6564093130353112834 (task 16320 ms: download 8930 ms overlapping gunzip 1017 ms, parse 4671 ms, sort 194 ms, write 9015 ms, index 521 ms: 9 candidates, 57535 repo-hours, 631 releases; waiting 534 ms; commit 5331 ms)
2026-11-08T03:14:08Z [load_day 2023-11-27] 2023-11-27 hour 6: 263178 events in 6 files, snapshot 2226399439749525143 (task 18343 ms: download 10621 ms overlapping gunzip 1137 ms, parse 4974 ms, sort 265 ms, write 11018 ms, index 703 ms: 11 candidates, 71683 repo-hours, 746 releases; waiting 31 ms; commit 2931 ms)
2026-11-08T03:14:19Z [load_day 2023-11-27] 2023-11-27 hour 7: 258102 events in 6 files, snapshot 6835105539360880101 (task 20532 ms: download 11915 ms overlapping gunzip 1318 ms, parse 6248 ms, sort 250 ms, write 11933 ms, index 575 ms: 8 candidates, 70791 repo-hours, 772 releases; waiting 19 ms; commit 5524 ms)
2026-11-08T03:14:30Z [load_day 2023-11-27] 2023-11-27 hour 8: 265687 events in 6 files, snapshot 6130271335503141785 (task 21127 ms: download 13258 ms overlapping gunzip 1285 ms, parse 5675 ms, sort 266 ms, write 12462 ms, index 628 ms: 17 candidates, 80188 repo-hours, 1023 releases; waiting 487 ms; commit 2996 ms)
2026-11-08T03:14:37Z [load_day 2023-11-27] 2023-11-27 hour 9: 269354 events in 6 files, snapshot 6181048288055650428 (task 20235 ms: download 12763 ms overlapping gunzip 1332 ms, parse 5840 ms, sort 163 ms, write 11606 ms, index 527 ms: 18 candidates, 82545 repo-hours, 1012 releases; waiting 496 ms; commit 2429 ms)
2026-11-08T03:14:51Z [load_day 2023-11-27] 2023-11-27 hour 10: 271191 events in 6 files, snapshot 6636457197540063553 (task 19629 ms: download 11004 ms overlapping gunzip 1228 ms, parse 5142 ms, sort 195 ms, write 12277 ms, index 580 ms: 19 candidates, 83648 repo-hours, 1051 releases; waiting 19 ms; commit 3364 ms)
2026-11-08T03:15:00Z [load_day 2023-11-27] 2023-11-27 hour 11: 266604 events in 6 files, snapshot 4573738985446439693 (task 21161 ms: download 11583 ms overlapping gunzip 1239 ms, parse 5425 ms, sort 238 ms, write 13360 ms, index 699 ms: 12 candidates, 79666 repo-hours, 959 releases; waiting 14 ms; commit 3102 ms)
2026-11-08T03:15:15Z [load_day 2023-11-27] 2023-11-27 hour 12: 294030 events in 6 files, snapshot 4543343019401995601 (task 22446 ms: download 13078 ms overlapping gunzip 1338 ms, parse 6238 ms, sort 246 ms, write 13528 ms, index 879 ms: 11 candidates, 106220 repo-hours, 938 releases; waiting 22 ms; commit 3357 ms)
2026-11-08T03:15:29Z [load_day 2023-11-27] 2023-11-27 hour 13: 287339 events in 6 files, snapshot 1596486387283437120 (task 23203 ms: download 15747 ms overlapping gunzip 1445 ms, parse 6227 ms, sort 269 ms, write 14321 ms, index 711 ms: 16 candidates, 99224 repo-hours, 885 releases; waiting 17 ms; commit 7228 ms)
2026-11-08T03:15:43Z [load_day 2023-11-27] 2023-11-27 hour 14: 282119 events in 6 files, snapshot 2100839930826359402 (task 27248 ms: download 18992 ms overlapping gunzip 1568 ms, parse 6859 ms, sort 270 ms, write 16943 ms, index 896 ms: 21 candidates, 91581 repo-hours, 939 releases; waiting 499 ms; commit 3117 ms)
2026-11-08T03:15:56Z [load_day 2023-11-27] 2023-11-27 hour 15: 282795 events in 6 files, snapshot 2620932962416666085 (task 28607 ms: download 20746 ms overlapping gunzip 1547 ms, parse 6939 ms, sort 298 ms, write 18260 ms, index 701 ms: 21 candidates, 91337 repo-hours, 1072 releases; waiting 536 ms; commit 3077 ms)
2026-11-08T03:16:11Z [load_day 2023-11-27] 2023-11-27 hour 16: 279527 events in 6 files, snapshot 9023000601385005160 (task 27007 ms: download 18659 ms overlapping gunzip 1790 ms, parse 7848 ms, sort 308 ms, write 15376 ms, index 923 ms: 22 candidates, 88666 repo-hours, 893 releases; waiting 529 ms; commit 3423 ms)
2026-11-08T03:16:30Z [load_day 2023-11-27] 2023-11-27 hour 17: 273194 events in 6 files, snapshot 7260394971475700019 (task 25179 ms: download 15359 ms overlapping gunzip 1413 ms, parse 6351 ms, sort 206 ms, write 15644 ms, index 807 ms: 29 candidates, 83955 repo-hours, 987 releases; waiting 510 ms; commit 10792 ms)
2026-11-08T03:16:37Z [load_day 2023-11-27] 2023-11-27 hour 18: 272020 events in 6 files, snapshot 2819884605922671188 (task 23924 ms: download 16935 ms overlapping gunzip 1476 ms, parse 6591 ms, sort 259 ms, write 14280 ms, index 562 ms: 19 candidates, 87474 repo-hours, 964 releases; waiting 505 ms; commit 4235 ms)
2026-11-08T03:16:43Z [load_day 2023-11-27] 2023-11-27 hour 19: 70838 events in 2 files, snapshot 7087042735903389497 (task 8473 ms: download 713 ms overlapping gunzip 561 ms, parse 2481 ms, sort 89 ms, write 4864 ms, index 252 ms: 5 candidates, 30002 repo-hours, 280 releases; waiting 22 ms; commit 5980 ms)
2026-11-08T03:16:57Z [load_day 2023-11-27] 2023-11-27 hour 20: 200491 events in 5 files, snapshot 7228553289184495857 (task 20185 ms: download 11307 ms overlapping gunzip 1250 ms, parse 5688 ms, sort 185 ms, write 12008 ms, index 842 ms: 10 candidates, 72020 repo-hours, 711 releases; waiting 20 ms; commit 3273 ms)
2026-11-08T03:17:05Z [load_day 2023-11-27] 2023-11-27 hour 21: 250306 events in 6 files, snapshot 7355963831179394883 (task 23743 ms: download 14546 ms overlapping gunzip 1502 ms, parse 6497 ms, sort 229 ms, write 14522 ms, index 761 ms: 14 candidates, 74243 repo-hours, 1021 releases; waiting 19 ms; commit 2952 ms)
2026-11-08T03:17:18Z [load_day 2023-11-27] 2023-11-27 hour 22: 231117 events in 5 files, snapshot 1943165933155111721 (task 20430 ms: download 13215 ms overlapping gunzip 1228 ms, parse 5186 ms, sort 172 ms, write 12951 ms, index 681 ms: 7 candidates, 67346 repo-hours, 1025 releases; waiting 15 ms; commit 2794 ms)
2026-11-08T03:17:25Z [load_day 2023-11-27] 2023-11-27 hour 23: 213684 events in 5 files, snapshot 7911288896555715900 (task 18961 ms: download 11471 ms overlapping gunzip 1323 ms, parse 5558 ms, sort 153 ms, write 10729 ms, index 448 ms: 7 candidates, 62433 repo-hours, 739 releases; waiting 504 ms; commit 2529 ms)
2026-11-08T03:17:33Z [load_day 2023-11-27] 2023-11-27: repo_hour merged from 24 files to 1 (1804619 rows, 7971 ms)
2026-11-08T03:17:38Z [load_day 2023-11-27] 2023-11-27: index committed in 13009 ms (24 + 1 + 24 files)
2026-11-08T03:17:38Z [load_day 2023-11-27] 2023-11-27: 6145749 events in 24 hours
2026-11-08T03:17:38Z [load_day 2023-11-27] attempt 1 succeeded