orcaLog in

Runs · run 1547

load_day 2023-09-24, attempt 1 succeeded

Host
3c2b29713ce8:365637
Started
2026-11-08T01:08:37Z
Ended
2026-11-08T01:11:16Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1547/load_day/2023-09-24/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:08:37Z [load_day 2023-09-24] attempt 1 started
2026-11-08T01:08:38Z [load_day 2023-09-24] 2023-09-24: 2 hours in flight, about 1200 MB
2026-11-08T01:08:52Z [load_day 2023-09-24] 2023-09-24 hour 0: 123328 events in 3 files, snapshot 3782229748574489620 (task 9422 ms: download 1825 ms overlapping gunzip 441 ms, parse 2100 ms, sort 108 ms, write 5565 ms, index 429 ms: 13 candidates, 70154 repo-hours, 881 releases; waiting 519 ms; commit 2310 ms)
2026-11-08T01:09:01Z [load_day 2023-09-24] 2023-09-24 hour 1: 122715 events in 3 files, snapshot 9035064945250412992 (task 8582 ms: download 1747 ms overlapping gunzip 385 ms, parse 1953 ms, sort 62 ms, write 5028 ms, index 518 ms: 9 candidates, 72341 repo-hours, 672 releases; waiting 326 ms; commit 9118 ms)
2026-11-08T01:09:03Z [load_day 2023-09-24] 2023-09-24 hour 2: 94020 events in 2 files, snapshot 2207122982819319271 (task 7751 ms: download 1690 ms overlapping gunzip 481 ms, parse 2216 ms, sort 63 ms, write 3968 ms, index 298 ms: 10 candidates, 40170 repo-hours, 568 releases; waiting 456 ms; commit 2730 ms)
2026-11-08T01:09:06Z [load_day 2023-09-24] 2023-09-24 hour 3: 92994 events in 2 files, snapshot 152369902857077520 (task 7710 ms: download 1646 ms overlapping gunzip 401 ms, parse 1817 ms, sort 56 ms, write 4362 ms, index 261 ms: 7 candidates, 39342 repo-hours, 627 releases; waiting 546 ms; commit 2268 ms)
2026-11-08T01:09:13Z [load_day 2023-09-24] 2023-09-24 hour 4: 96151 events in 2 files, snapshot 6793627846138470012 (task 7048 ms: download 1616 ms overlapping gunzip 410 ms, parse 1937 ms, sort 83 ms, write 3590 ms, index 264 ms: 2 candidates, 40677 repo-hours, 470 releases; waiting 539 ms; commit 3902 ms)
2026-11-08T01:09:18Z [load_day 2023-09-24] 2023-09-24 hour 5: 104602 events in 3 files, snapshot 150897068478767195 (task 8448 ms: download 1994 ms overlapping gunzip 394 ms, parse 1802 ms, sort 68 ms, write 5025 ms, index 588 ms: 1 candidates, 43956 repo-hours, 618 releases; waiting 204 ms; commit 4707 ms)
2026-11-08T01:09:23Z [load_day 2023-09-24] 2023-09-24 hour 6: 121771 events in 3 files, snapshot 6814824691344794043 (task 9698 ms: download 2478 ms overlapping gunzip 622 ms, parse 2470 ms, sort 72 ms, write 5506 ms, index 362 ms: 5 candidates, 53482 repo-hours, 648 releases; waiting 183 ms; commit 2770 ms)
2026-11-08T01:09:28Z [load_day 2023-09-24] 2023-09-24 hour 7: 115720 events in 3 files, snapshot 1313977897594036808 (task 10319 ms: download 2915 ms overlapping gunzip 599 ms, parse 2724 ms, sort 99 ms, write 5728 ms, index 366 ms: 4 candidates, 50621 repo-hours, 609 releases; waiting 473 ms; commit 3914 ms)
2026-11-08T01:09:35Z [load_day 2023-09-24] 2023-09-24 hour 8: 119939 events in 3 files, snapshot 4457928959552441 (task 10661 ms: download 2173 ms overlapping gunzip 592 ms, parse 2681 ms, sort 162 ms, write 5867 ms, index 456 ms: 6 candidates, 52274 repo-hours, 650 releases; waiting 546 ms; commit 3069 ms)
2026-11-08T01:09:39Z [load_day 2023-09-24] 2023-09-24 hour 9: 124671 events in 3 files, snapshot 3954095487533237009 (task 10855 ms: download 2310 ms overlapping gunzip 650 ms, parse 2878 ms, sort 156 ms, write 5992 ms, index 385 ms: 6 candidates, 52824 repo-hours, 783 releases; waiting 477 ms; commit 3210 ms)
2026-11-08T01:09:47Z [load_day 2023-09-24] 2023-09-24 hour 10: 129262 events in 3 files, snapshot 5937688962551996243 (task 11078 ms: download 2057 ms overlapping gunzip 557 ms, parse 2611 ms, sort 92 ms, write 6511 ms, index 594 ms: 6 candidates, 56359 repo-hours, 704 releases; waiting 467 ms; commit 2749 ms)
2026-11-08T01:09:50Z [load_day 2023-09-24] 2023-09-24 hour 11: 125132 events in 3 files, snapshot 5393092290891802039 (task 10146 ms: download 2201 ms overlapping gunzip 554 ms, parse 2638 ms, sort 153 ms, write 5384 ms, index 562 ms: 10 candidates, 53214 repo-hours, 648 releases; waiting 523 ms; commit 2792 ms)
2026-11-08T01:10:01Z [load_day 2023-09-24] 2023-09-24 hour 12: 203548 events in 5 files, snapshot 4192135008712465290 (task 13174 ms: download 3969 ms overlapping gunzip 647 ms, parse 3419 ms, sort 147 ms, write 7495 ms, index 689 ms: 7 candidates, 87778 repo-hours, 785 releases; waiting 515 ms; commit 2514 ms)
2026-11-08T01:10:04Z [load_day 2023-09-24] 2023-09-24 hour 13: 160333 events in 4 files, snapshot 5867608518064654063 (task 12474 ms: download 4381 ms overlapping gunzip 752 ms, parse 3479 ms, sort 112 ms, write 6691 ms, index 662 ms: 6 candidates, 82501 repo-hours, 676 releases; waiting 529 ms; commit 3103 ms)
2026-11-08T01:10:14Z [load_day 2023-09-24] 2023-09-24 hour 14: 148145 events in 3 files, snapshot 4534331296054828459 (task 12571 ms: download 4677 ms overlapping gunzip 784 ms, parse 3388 ms, sort 129 ms, write 6981 ms, index 512 ms: 12 candidates, 65105 repo-hours, 786 releases; waiting 524 ms; commit 2302 ms)
2026-11-08T01:10:20Z [load_day 2023-09-24] 2023-09-24 hour 15: 153448 events in 4 files, snapshot 5796665395444528576 (task 11833 ms: download 4448 ms overlapping gunzip 651 ms, parse 2848 ms, sort 165 ms, write 7205 ms, index 464 ms: 9 candidates, 68915 repo-hours, 864 releases; waiting 179 ms; commit 5903 ms)
2026-11-08T01:10:32Z [load_day 2023-09-24] 2023-09-24 hour 16: 152860 events in 4 files, snapshot 5914360851010601143 (task 13104 ms: download 5523 ms overlapping gunzip 705 ms, parse 3191 ms, sort 286 ms, write 7633 ms, index 523 ms: 7 candidates, 67876 repo-hours, 847 releases; waiting 430 ms; commit 5143 ms)
2026-11-08T01:10:34Z [load_day 2023-09-24] 2023-09-24 hour 17: 143835 events in 3 files, snapshot 3916501152977796966 (task 12053 ms: download 4863 ms overlapping gunzip 703 ms, parse 3188 ms, sort 110 ms, write 6792 ms, index 454 ms: 11 candidates, 61074 repo-hours, 755 releases; waiting 479 ms; commit 2432 ms)
2026-11-08T01:10:43Z [load_day 2023-09-24] 2023-09-24 hour 18: 151781 events in 4 files, snapshot 8966558178994995731 (task 12452 ms: download 4598 ms overlapping gunzip 654 ms, parse 3219 ms, sort 142 ms, write 6798 ms, index 826 ms: 12 candidates, 68715 repo-hours, 863 releases; waiting 508 ms; commit 2568 ms)
2026-11-08T01:10:46Z [load_day 2023-09-24] 2023-09-24 hour 19: 134121 events in 3 files, snapshot 7097786512038414471 (task 10420 ms: download 2753 ms overlapping gunzip 600 ms, parse 2814 ms, sort 113 ms, write 5728 ms, index 392 ms: 5 candidates, 57216 repo-hours, 751 releases; waiting 506 ms; commit 2401 ms)
2026-11-08T01:10:55Z [load_day 2023-09-24] 2023-09-24 hour 20: 127441 events in 3 files, snapshot 2676950123634939733 (task 10060 ms: download 2092 ms overlapping gunzip 593 ms, parse 2641 ms, sort 104 ms, write 5428 ms, index 404 ms: 9 candidates, 54778 repo-hours, 696 releases; waiting 574 ms; commit 2818 ms)
2026-11-08T01:10:57Z [load_day 2023-09-24] 2023-09-24 hour 21: 118392 events in 3 files, snapshot 6247762455641078982 (task 9596 ms: download 2159 ms overlapping gunzip 531 ms, parse 2315 ms, sort 82 ms, write 5490 ms, index 547 ms: 10 candidates, 49104 repo-hours, 712 releases; waiting 325 ms; commit 2179 ms)
2026-11-08T01:11:05Z [load_day 2023-09-24] 2023-09-24 hour 22: 110584 events in 3 files, snapshot 793473206958357290 (task 8963 ms: download 1917 ms overlapping gunzip 547 ms, parse 2529 ms, sort 86 ms, write 4656 ms, index 368 ms: 6 candidates, 44029 repo-hours, 646 releases; waiting 508 ms; commit 3360 ms)
2026-11-08T01:11:08Z [load_day 2023-09-24] 2023-09-24 hour 23: 100533 events in 3 files, snapshot 4890124048729787168 (task 8633 ms: download 1727 ms overlapping gunzip 442 ms, parse 2102 ms, sort 117 ms, write 4740 ms, index 398 ms: 7 candidates, 42609 repo-hours, 570 releases; waiting 554 ms; commit 2389 ms)
2026-11-08T01:11:14Z [load_day 2023-09-24] 2023-09-24: repo_hour merged from 24 files to 1 (1375114 rows, 6452 ms)
2026-11-08T01:11:16Z [load_day 2023-09-24] 2023-09-24: index committed in 8520 ms (24 + 1 + 24 files)
2026-11-08T01:11:16Z [load_day 2023-09-24] 2023-09-24: 3075326 events in 24 hours
2026-11-08T01:11:16Z [load_day 2023-09-24] attempt 1 succeeded