orcaLog in

Runs · run 1650

load_day 2023-10-21, attempt 1 succeeded

Host
3c2b29713ce8:464834
Started
2026-11-08T02:01:00Z
Ended
2026-11-08T02:03:47Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1650/load_day/2023-10-21/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:01:00Z [load_day 2023-10-21] attempt 1 started
2026-11-08T02:01:02Z [load_day 2023-10-21] 2023-10-21: 2 hours in flight, about 1200 MB
2026-11-08T02:01:19Z [load_day 2023-10-21] 2023-10-21 hour 0: 144043 events in 3 files, snapshot 8605614482697828239 (task 13703 ms: download 6181 ms overlapping gunzip 794 ms, parse 3821 ms, sort 173 ms, write 7470 ms, index 650 ms: 11 candidates, 77945 repo-hours, 1005 releases; waiting 513 ms; commit 2486 ms)
2026-11-08T02:01:23Z [load_day 2023-10-21] 2023-10-21 hour 1: 139561 events in 3 files, snapshot 3446791815421722367 (task 12568 ms: download 5366 ms overlapping gunzip 679 ms, parse 2977 ms, sort 89 ms, write 7535 ms, index 572 ms: 9 candidates, 76673 repo-hours, 735 releases; waiting 502 ms; commit 3209 ms)
2026-11-08T02:01:30Z [load_day 2023-10-21] 2023-10-21 hour 2: 105735 events in 3 files, snapshot 4582912515126613956 (task 9834 ms: download 1919 ms overlapping gunzip 588 ms, parse 2457 ms, sort 74 ms, write 5669 ms, index 310 ms: 12 candidates, 44602 repo-hours, 617 releases; waiting 492 ms; commit 2450 ms)
2026-11-08T02:01:33Z [load_day 2023-10-21] 2023-10-21 hour 3: 104600 events in 3 files, snapshot 7992754725545957549 (task 9238 ms: download 2178 ms overlapping gunzip 632 ms, parse 2758 ms, sort 83 ms, write 4589 ms, index 363 ms: 7 candidates, 43746 repo-hours, 665 releases; waiting 502 ms; commit 3065 ms)
2026-11-08T02:01:41Z [load_day 2023-10-21] 2023-10-21 hour 4: 102954 events in 3 files, snapshot 9001191109192008010 (task 8861 ms: download 1636 ms overlapping gunzip 527 ms, parse 2533 ms, sort 108 ms, write 4602 ms, index 337 ms: 13 candidates, 42942 repo-hours, 537 releases; waiting 511 ms; commit 2753 ms)
2026-11-08T02:01:44Z [load_day 2023-10-21] 2023-10-21 hour 5: 104400 events in 3 files, snapshot 5707890816555392114 (task 9752 ms: download 1942 ms overlapping gunzip 536 ms, parse 2449 ms, sort 88 ms, write 5458 ms, index 337 ms: 5 candidates, 44896 repo-hours, 523 releases; waiting 587 ms; commit 2567 ms)
2026-11-08T02:01:52Z [load_day 2023-10-21] 2023-10-21 hour 6: 119223 events in 3 files, snapshot 5038090441965073806 (task 10054 ms: download 1864 ms overlapping gunzip 573 ms, parse 2511 ms, sort 85 ms, write 5580 ms, index 490 ms: 9 candidates, 54085 repo-hours, 633 releases; waiting 523 ms; commit 2740 ms)
2026-11-08T02:01:57Z [load_day 2023-10-21] 2023-10-21 hour 7: 119195 events in 3 files, snapshot 9084667996114947231 (task 10098 ms: download 1959 ms overlapping gunzip 508 ms, parse 2238 ms, sort 65 ms, write 6261 ms, index 495 ms: 7 candidates, 49028 repo-hours, 587 releases; waiting 205 ms; commit 4419 ms)
2026-11-08T02:02:08Z [load_day 2023-10-21] 2023-10-21 hour 8: 121754 events in 3 files, snapshot 7834530762812730974 (task 10022 ms: download 2146 ms overlapping gunzip 592 ms, parse 2541 ms, sort 77 ms, write 5690 ms, index 375 ms: 8 candidates, 51707 repo-hours, 593 releases; waiting 495 ms; commit 7790 ms)
2026-11-08T02:02:10Z [load_day 2023-10-21] 2023-10-21 hour 9: 127752 events in 3 files, snapshot 5426911502013441897 (task 10444 ms: download 2008 ms overlapping gunzip 545 ms, parse 2583 ms, sort 74 ms, write 6368 ms, index 353 ms: 5 candidates, 53136 repo-hours, 727 releases; waiting 205 ms; commit 2491 ms)
2026-11-08T02:02:20Z [load_day 2023-10-21] 2023-10-21 hour 10: 127848 events in 3 files, snapshot 8787406963921172987 (task 10735 ms: download 2418 ms overlapping gunzip 700 ms, parse 3064 ms, sort 80 ms, write 5607 ms, index 449 ms: 6 candidates, 55068 repo-hours, 663 releases; waiting 508 ms; commit 2413 ms)
2026-11-08T02:02:23Z [load_day 2023-10-21] 2023-10-21 hour 11: 125353 events in 3 files, snapshot 6519850775126993620 (task 9928 ms: download 2093 ms overlapping gunzip 498 ms, parse 2270 ms, sort 77 ms, write 5880 ms, index 383 ms: 7 candidates, 52933 repo-hours, 646 releases; waiting 521 ms; commit 2673 ms)
2026-11-08T02:02:33Z [load_day 2023-10-21] 2023-10-21 hour 12: 200855 events in 5 files, snapshot 7060114721552218466 (task 11715 ms: download 4265 ms overlapping gunzip 595 ms, parse 3075 ms, sort 109 ms, write 6634 ms, index 529 ms: 9 candidates, 87467 repo-hours, 765 releases; waiting 508 ms; commit 2255 ms)
2026-11-08T02:02:35Z [load_day 2023-10-21] 2023-10-21 hour 13: 159603 events in 4 files, snapshot 1448065189551283518 (task 10402 ms: download 3771 ms overlapping gunzip 564 ms, parse 2712 ms, sort 85 ms, write 5689 ms, index 557 ms: 9 candidates, 80496 repo-hours, 744 releases; waiting 483 ms; commit 2295 ms)
2026-11-08T02:02:44Z [load_day 2023-10-21] 2023-10-21 hour 14: 150410 events in 4 files, snapshot 1319461340737398222 (task 10443 ms: download 4113 ms overlapping gunzip 592 ms, parse 2627 ms, sort 101 ms, write 5538 ms, index 526 ms: 6 candidates, 64541 repo-hours, 1065 releases; waiting 694 ms; commit 2396 ms)
2026-11-08T02:02:47Z [load_day 2023-10-21] 2023-10-21 hour 15: 150189 events in 4 files, snapshot 419730046728889363 (task 10748 ms: download 3784 ms overlapping gunzip 554 ms, parse 2478 ms, sort 91 ms, write 5739 ms, index 1020 ms: 13 candidates, 66429 repo-hours, 872 releases; waiting 555 ms; commit 2764 ms)
2026-11-08T02:02:59Z [load_day 2023-10-21] 2023-10-21 hour 16: 145825 events in 3 files, snapshot 3162480547926492154 (task 11887 ms: download 5463 ms overlapping gunzip 788 ms, parse 3451 ms, sort 145 ms, write 6229 ms, index 503 ms: 8 candidates, 64200 repo-hours, 784 releases; waiting 489 ms; commit 5063 ms)
2026-11-08T02:03:04Z [load_day 2023-10-21] 2023-10-21 hour 17: 138663 events in 3 files, snapshot 6756212855178692149 (task 10408 ms: download 4127 ms overlapping gunzip 533 ms, parse 2367 ms, sort 80 ms, write 6219 ms, index 481 ms: 7 candidates, 59199 repo-hours, 823 releases; waiting 486 ms; commit 4589 ms)
2026-11-08T02:03:09Z [load_day 2023-10-21] 2023-10-21 hour 18: 140108 events in 3 files, snapshot 6027198603201252545 (task 10861 ms: download 4115 ms overlapping gunzip 553 ms, parse 2481 ms, sort 75 ms, write 6625 ms, index 410 ms: 11 candidates, 64459 repo-hours, 784 releases; waiting 352 ms; commit 2802 ms)
2026-11-08T02:03:16Z [load_day 2023-10-21] 2023-10-21 hour 19: 127393 events in 3 files, snapshot 5251241990370071924 (task 10992 ms: download 2318 ms overlapping gunzip 767 ms, parse 3301 ms, sort 87 ms, write 5659 ms, index 377 ms: 7 candidates, 52039 repo-hours, 769 releases; waiting 570 ms; commit 4904 ms)
2026-11-08T02:03:23Z [load_day 2023-10-21] 2023-10-21 hour 20: 123700 events in 3 files, snapshot 6541917466323586771 (task 10051 ms: download 2193 ms overlapping gunzip 545 ms, parse 2580 ms, sort 143 ms, write 5964 ms, index 399 ms: 8 candidates, 51469 repo-hours, 668 releases; waiting 188 ms; commit 5302 ms)
2026-11-08T02:03:27Z [load_day 2023-10-21] 2023-10-21 hour 21: 111741 events in 3 files, snapshot 2280676430492609341 (task 10844 ms: download 2404 ms overlapping gunzip 584 ms, parse 2677 ms, sort 65 ms, write 6255 ms, index 347 ms: 6 candidates, 45983 repo-hours, 788 releases; waiting 553 ms; commit 3713 ms)
2026-11-08T02:03:33Z [load_day 2023-10-21] 2023-10-21 hour 22: 102108 events in 3 files, snapshot 615084955982161295 (task 9070 ms: download 2275 ms overlapping gunzip 518 ms, parse 2415 ms, sort 109 ms, write 4807 ms, index 345 ms: 10 candidates, 41608 repo-hours, 684 releases; waiting 510 ms; commit 4999 ms)
2026-11-08T02:03:38Z [load_day 2023-10-21] 2023-10-21 hour 23: 94273 events in 2 files, snapshot 417781606886273975 (task 7039 ms: download 1752 ms overlapping gunzip 409 ms, parse 1910 ms, sort 51 ms, write 3583 ms, index 284 ms: 18 candidates, 39882 repo-hours, 553 releases; waiting 518 ms; commit 4727 ms)
2026-11-08T02:03:45Z [load_day 2023-10-21] 2023-10-21: repo_hour merged from 24 files to 1 (1364533 rows, 6686 ms)
2026-11-08T02:03:47Z [load_day 2023-10-21] 2023-10-21: index committed in 9302 ms (24 + 1 + 24 files)
2026-11-08T02:03:47Z [load_day 2023-10-21] 2023-10-21: 3087286 events in 24 hours
2026-11-08T02:03:47Z [load_day 2023-10-21] attempt 1 succeeded