orcaLog in

Runs · run 1412

load_day 2023-08-20, attempt 1 succeeded

Host
3c2b29713ce8:245671
Started
2026-11-08T00:03:54Z
Ended
2026-11-08T00:06:16Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1412/load_day/2023-08-20/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:03:54Z [load_day 2023-08-20] attempt 1 started
2026-11-08T00:03:56Z [load_day 2023-08-20] 2023-08-20: 2 hours in flight, about 1200 MB
2026-11-08T00:04:10Z [load_day 2023-08-20] 2023-08-20 hour 0: 127650 events in 3 files, snapshot 7697953158881614538 (task 9672 ms: download 2488 ms overlapping gunzip 500 ms, parse 2419 ms, sort 111 ms, write 5336 ms, index 506 ms: 2 candidates, 75313 repo-hours, 876 releases; waiting 511 ms; commit 2828 ms)
2026-11-08T00:04:14Z [load_day 2023-08-20] 2023-08-20 hour 1: 117433 events in 3 files, snapshot 1952252008945458201 (task 9193 ms: download 1884 ms overlapping gunzip 485 ms, parse 2417 ms, sort 70 ms, write 4938 ms, index 448 ms: 5 candidates, 68340 repo-hours, 604 releases; waiting 487 ms; commit 4724 ms)
2026-11-08T00:04:18Z [load_day 2023-08-20] 2023-08-20 hour 2: 89775 events in 2 files, snapshot 910653189734437436 (task 7354 ms: download 2245 ms overlapping gunzip 394 ms, parse 1805 ms, sort 79 ms, write 3879 ms, index 344 ms: 3 candidates, 37014 repo-hours, 564 releases; waiting 547 ms; commit 2744 ms)
2026-11-08T00:04:21Z [load_day 2023-08-20] 2023-08-20 hour 3: 89477 events in 2 files, snapshot 1772521297019097745 (task 6944 ms: download 1497 ms overlapping gunzip 471 ms, parse 2135 ms, sort 58 ms, write 3369 ms, index 351 ms: 4 candidates, 37314 repo-hours, 685 releases; waiting 218 ms; commit 3621 ms)
2026-11-08T00:04:26Z [load_day 2023-08-20] 2023-08-20 hour 4: 93081 events in 2 files, snapshot 204320842905126289 (task 7549 ms: download 1442 ms overlapping gunzip 379 ms, parse 1743 ms, sort 72 ms, write 4133 ms, index 418 ms: 3 candidates, 37778 repo-hours, 482 releases; waiting 508 ms; commit 2367 ms)
2026-11-08T00:04:30Z [load_day 2023-08-20] 2023-08-20 hour 5: 99408 events in 2 files, snapshot 602812481750172556 (task 8024 ms: download 1895 ms overlapping gunzip 449 ms, parse 2031 ms, sort 72 ms, write 4341 ms, index 304 ms: 1 candidates, 40474 repo-hours, 649 releases; waiting 490 ms; commit 3482 ms)
2026-11-08T00:04:35Z [load_day 2023-08-20] 2023-08-20 hour 6: 110694 events in 3 files, snapshot 8795972655008649544 (task 7658 ms: download 1708 ms overlapping gunzip 394 ms, parse 1828 ms, sort 71 ms, write 4182 ms, index 428 ms: 2 candidates, 48996 repo-hours, 681 releases; waiting 468 ms; commit 2495 ms)
2026-11-08T00:04:39Z [load_day 2023-08-20] 2023-08-20 hour 7: 111125 events in 3 files, snapshot 365515711799694083 (task 8630 ms: download 2062 ms overlapping gunzip 525 ms, parse 2337 ms, sort 76 ms, write 4544 ms, index 330 ms: 1 candidates, 45283 repo-hours, 546 releases; waiting 500 ms; commit 1992 ms)
2026-11-08T00:04:43Z [load_day 2023-08-20] 2023-08-20 hour 8: 115199 events in 3 files, snapshot 6852755120100672950 (task 7674 ms: download 2442 ms overlapping gunzip 420 ms, parse 1842 ms, sort 70 ms, write 4011 ms, index 312 ms: 3 candidates, 48439 repo-hours, 615 releases; waiting 743 ms; commit 2136 ms)
2026-11-08T00:04:48Z [load_day 2023-08-20] 2023-08-20 hour 9: 119211 events in 3 files, snapshot 7679399296122356671 (task 7872 ms: download 1870 ms overlapping gunzip 435 ms, parse 1939 ms, sort 70 ms, write 4296 ms, index 315 ms: 2 candidates, 48140 repo-hours, 756 releases; waiting 490 ms; commit 2192 ms)
2026-11-08T00:04:52Z [load_day 2023-08-20] 2023-08-20 hour 10: 120447 events in 3 files, snapshot 7763364796355287215 (task 7940 ms: download 2395 ms overlapping gunzip 431 ms, parse 1907 ms, sort 75 ms, write 4051 ms, index 340 ms: 7 candidates, 50745 repo-hours, 633 releases; waiting 803 ms; commit 2045 ms)
2026-11-08T00:04:57Z [load_day 2023-08-20] 2023-08-20 hour 11: 121176 events in 3 files, snapshot 5176459082614876363 (task 8593 ms: download 1807 ms overlapping gunzip 445 ms, parse 2035 ms, sort 74 ms, write 4654 ms, index 569 ms: 4 candidates, 50426 repo-hours, 710 releases; waiting 535 ms; commit 2310 ms)
2026-11-08T00:05:07Z [load_day 2023-08-20] 2023-08-20 hour 12: 206759 events in 5 files, snapshot 968811942591332190 (task 13833 ms: download 4570 ms overlapping gunzip 684 ms, parse 3480 ms, sort 195 ms, write 7945 ms, index 677 ms: 4 candidates, 85335 repo-hours, 834 releases; waiting 517 ms; commit 2823 ms)
2026-11-08T00:05:12Z [load_day 2023-08-20] 2023-08-20 hour 13: 154135 events in 4 files, snapshot 6640374762296855479 (task 11215 ms: download 3919 ms overlapping gunzip 595 ms, parse 2505 ms, sort 90 ms, write 6446 ms, index 614 ms: 4 candidates, 76816 repo-hours, 872 releases; waiting 615 ms; commit 4763 ms)
2026-11-08T00:05:20Z [load_day 2023-08-20] 2023-08-20 hour 14: 139418 events in 3 files, snapshot 8333802327170817668 (task 11227 ms: download 1977 ms overlapping gunzip 582 ms, parse 2720 ms, sort 90 ms, write 6731 ms, index 556 ms: 3 candidates, 60667 repo-hours, 817 releases; waiting 143 ms; commit 2835 ms)
2026-11-08T00:05:25Z [load_day 2023-08-20] 2023-08-20 hour 15: 144109 events in 3 files, snapshot 208308744847703484 (task 12152 ms: download 4091 ms overlapping gunzip 693 ms, parse 3203 ms, sort 136 ms, write 6951 ms, index 541 ms: 4 candidates, 62555 repo-hours, 901 releases; waiting 330 ms; commit 4298 ms)
2026-11-08T00:05:32Z [load_day 2023-08-20] 2023-08-20 hour 16: 139660 events in 3 files, snapshot 6705085201930487390 (task 11276 ms: download 2176 ms overlapping gunzip 599 ms, parse 2727 ms, sort 90 ms, write 6483 ms, index 526 ms: 5 candidates, 62268 repo-hours, 733 releases; waiting 514 ms; commit 2273 ms)
2026-11-08T00:05:35Z [load_day 2023-08-20] 2023-08-20 hour 17: 132252 events in 3 files, snapshot 76959445024661176 (task 11020 ms: download 2732 ms overlapping gunzip 702 ms, parse 3177 ms, sort 171 ms, write 5448 ms, index 723 ms: 2 candidates, 54859 repo-hours, 702 releases; waiting 500 ms; commit 2260 ms)
2026-11-08T00:05:45Z [load_day 2023-08-20] 2023-08-20 hour 18: 140932 events in 3 files, snapshot 2173462998955062429 (task 12039 ms: download 4219 ms overlapping gunzip 669 ms, parse 2914 ms, sort 125 ms, write 6582 ms, index 793 ms: 3 candidates, 62766 repo-hours, 779 releases; waiting 648 ms; commit 2455 ms)
2026-11-08T00:05:47Z [load_day 2023-08-20] 2023-08-20 hour 19: 123041 events in 3 files, snapshot 7035010288433704984 (task 10404 ms: download 2885 ms overlapping gunzip 526 ms, parse 2228 ms, sort 105 ms, write 6051 ms, index 398 ms: 4 candidates, 50078 repo-hours, 836 releases; waiting 749 ms; commit 2106 ms)
2026-11-08T00:05:56Z [load_day 2023-08-20] 2023-08-20 hour 20: 120806 events in 3 files, snapshot 5792411374904777970 (task 9470 ms: download 1603 ms overlapping gunzip 608 ms, parse 2746 ms, sort 123 ms, write 5160 ms, index 387 ms: 3 candidates, 49582 repo-hours, 730 releases; waiting 184 ms; commit 2168 ms)
2026-11-08T00:06:01Z [load_day 2023-08-20] 2023-08-20 hour 21: 107379 events in 3 files, snapshot 47436815239662226 (task 8415 ms: download 2106 ms overlapping gunzip 396 ms, parse 1802 ms, sort 63 ms, write 4976 ms, index 386 ms: 3 candidates, 43234 repo-hours, 752 releases; waiting 498 ms; commit 4982 ms)
2026-11-08T00:06:06Z [load_day 2023-08-20] 2023-08-20 hour 22: 99502 events in 2 files, snapshot 1660736368757326057 (task 9537 ms: download 2226 ms overlapping gunzip 532 ms, parse 2466 ms, sort 87 ms, write 5205 ms, index 419 ms: 2 candidates, 40190 repo-hours, 649 releases; waiting 507 ms; commit 2104 ms)
2026-11-08T00:06:08Z [load_day 2023-08-20] 2023-08-20 hour 23: 95554 events in 2 files, snapshot 3418929907911915552 (task 8631 ms: download 1921 ms overlapping gunzip 537 ms, parse 2345 ms, sort 77 ms, write 4636 ms, index 282 ms: 5 candidates, 39581 repo-hours, 611 releases; waiting 435 ms; commit 1962 ms)
2026-11-08T00:06:14Z [load_day 2023-08-20] 2023-08-20: repo_hour merged from 24 files to 1 (1276193 rows, 5948 ms)
2026-11-08T00:06:16Z [load_day 2023-08-20] 2023-08-20: index committed in 7640 ms (24 + 1 + 24 files)
2026-11-08T00:06:16Z [load_day 2023-08-20] 2023-08-20: 2918223 events in 24 hours
2026-11-08T00:06:16Z [load_day 2023-08-20] attempt 1 succeeded