orcaLog in

Runs · run 1360

load_day 2023-08-04, attempt 1 succeeded

Host
3c2b29713ce8:193850
Started
2026-11-07T23:35:20Z
Ended
2026-11-07T23:39:00Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1360/load_day/2023-08-04/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T23:35:21Z [load_day 2023-08-04] attempt 1 started
2026-11-07T23:35:22Z [load_day 2023-08-04] 2023-08-04: 2 hours in flight, about 1200 MB
2026-11-07T23:35:41Z [load_day 2023-08-04] 2023-08-04 hour 0: 154163 events in 4 files, snapshot 3406949271313824381 (task 15529 ms: download 7426 ms overlapping gunzip 949 ms, parse 4452 ms, sort 126 ms, write 8573 ms, index 650 ms: 4 candidates, 76918 repo-hours, 888 releases; waiting 475 ms; commit 2135 ms)
2026-11-07T23:35:44Z [load_day 2023-08-04] 2023-08-04 hour 1: 162878 events in 4 files, snapshot 867928817040946003 (task 15859 ms: download 7209 ms overlapping gunzip 990 ms, parse 4383 ms, sort 126 ms, write 8774 ms, index 712 ms: 4 candidates, 88207 repo-hours, 836 releases; waiting 505 ms; commit 2387 ms)
2026-11-07T23:35:57Z [load_day 2023-08-04] 2023-08-04 hour 2: 146861 events in 3 files, snapshot 887321622178878575 (task 14803 ms: download 6474 ms overlapping gunzip 798 ms, parse 3232 ms, sort 105 ms, write 8932 ms, index 461 ms: 3 candidates, 61034 repo-hours, 735 releases; waiting 825 ms; commit 2432 ms)
2026-11-07T23:36:00Z [load_day 2023-08-04] 2023-08-04 hour 3: 139159 events in 3 files, snapshot 6970679070356125928 (task 12940 ms: download 5569 ms overlapping gunzip 777 ms, parse 3359 ms, sort 174 ms, write 7423 ms, index 434 ms: 2 candidates, 57072 repo-hours, 835 releases; waiting 488 ms; commit 2673 ms)
2026-11-07T23:36:12Z [load_day 2023-08-04] 2023-08-04 hour 4: 129641 events in 3 files, snapshot 5795307055494166060 (task 12243 ms: download 5277 ms overlapping gunzip 675 ms, parse 3024 ms, sort 82 ms, write 7456 ms, index 352 ms: 3 candidates, 53201 repo-hours, 654 releases; waiting 372 ms; commit 3668 ms)
2026-11-07T23:36:15Z [load_day 2023-08-04] 2023-08-04 hour 5: 138239 events in 3 files, snapshot 5846531162991075036 (task 13719 ms: download 5660 ms overlapping gunzip 786 ms, parse 3366 ms, sort 80 ms, write 7887 ms, index 798 ms: 1 candidates, 57546 repo-hours, 704 releases; waiting 510 ms; commit 2694 ms)
2026-11-07T23:36:27Z [load_day 2023-08-04] 2023-08-04 hour 6: 161819 events in 4 files, snapshot 2451702848412268251 (task 15417 ms: download 6197 ms overlapping gunzip 836 ms, parse 3735 ms, sort 108 ms, write 9320 ms, index 559 ms: 4 candidates, 73602 repo-hours, 898 releases; waiting 521 ms; commit 2520 ms)
2026-11-07T23:36:34Z [load_day 2023-08-04] 2023-08-04 hour 7: 165369 events in 4 files, snapshot 2602410582255899598 (task 16846 ms: download 9863 ms overlapping gunzip 970 ms, parse 4424 ms, sort 119 ms, write 10016 ms, index 511 ms: 3 candidates, 72600 repo-hours, 923 releases; waiting 496 ms; commit 4054 ms)
2026-11-07T23:36:45Z [load_day 2023-08-04] 2023-08-04 hour 8: 164620 events in 4 files, snapshot 8344236747914770465 (task 17830 ms: download 10565 ms overlapping gunzip 823 ms, parse 3935 ms, sort 119 ms, write 11478 ms, index 662 ms: 5 candidates, 74729 repo-hours, 955 releases; waiting 514 ms; commit 2006 ms)
2026-11-07T23:36:53Z [load_day 2023-08-04] 2023-08-04 hour 9: 176996 events in 4 files, snapshot 3152083976501846505 (task 17847 ms: download 10133 ms overlapping gunzip 1153 ms, parse 4985 ms, sort 119 ms, write 10529 ms, index 545 ms: 9 candidates, 77613 repo-hours, 1136 releases; waiting 197 ms; commit 3435 ms)
2026-11-07T23:37:04Z [load_day 2023-08-04] 2023-08-04 hour 10: 178233 events in 4 files, snapshot 2735519750117404887 (task 17655 ms: download 9870 ms overlapping gunzip 963 ms, parse 4172 ms, sort 116 ms, write 11008 ms, index 564 ms: 8 candidates, 82755 repo-hours, 1017 releases; waiting 517 ms; commit 2365 ms)
2026-11-07T23:37:12Z [load_day 2023-08-04] 2023-08-04 hour 11: 166078 events in 4 files, snapshot 1320596459496222500 (task 18552 ms: download 7530 ms overlapping gunzip 1142 ms, parse 5295 ms, sort 150 ms, write 10499 ms, index 576 ms: 4 candidates, 69976 repo-hours, 951 releases; waiting 496 ms; commit 3287 ms)
2026-11-07T23:37:25Z [load_day 2023-08-04] 2023-08-04 hour 12: 213228 events in 5 files, snapshot 2911880295217624504 (task 19259 ms: download 10510 ms overlapping gunzip 1119 ms, parse 5297 ms, sort 164 ms, write 11027 ms, index 786 ms: 12 candidates, 100243 repo-hours, 1025 releases; waiting 508 ms; commit 2856 ms)
2026-11-07T23:37:32Z [load_day 2023-08-04] 2023-08-04 hour 13: 182793 events in 4 files, snapshot 526745881276019167 (task 19147 ms: download 10357 ms overlapping gunzip 1100 ms, parse 5013 ms, sort 165 ms, write 11368 ms, index 706 ms: 7 candidates, 90588 repo-hours, 868 releases; waiting 503 ms; commit 3007 ms)
2026-11-07T23:37:44Z [load_day 2023-08-04] 2023-08-04 hour 14: 177708 events in 4 files, snapshot 5029597162899655915 (task 18730 ms: download 11970 ms overlapping gunzip 1246 ms, parse 5749 ms, sort 107 ms, write 10267 ms, index 548 ms: 6 candidates, 79940 repo-hours, 980 releases; waiting 509 ms; commit 1956 ms)
2026-11-07T23:37:47Z [load_day 2023-08-04] 2023-08-04 hour 15: 153766 events in 4 files, snapshot 7468868221715585557 (task 14987 ms: download 6905 ms overlapping gunzip 1001 ms, parse 4329 ms, sort 94 ms, write 8528 ms, index 515 ms: 3 candidates, 72445 repo-hours, 889 releases; waiting 216 ms; commit 2579 ms)
2026-11-07T23:38:02Z [load_day 2023-08-04] 2023-08-04 hour 16: 170809 events in 4 files, snapshot 1554905350710206528 (task 15525 ms: download 9124 ms overlapping gunzip 977 ms, parse 4146 ms, sort 117 ms, write 8787 ms, index 752 ms: 4 candidates, 73167 repo-hours, 891 releases; waiting 520 ms; commit 3032 ms)
2026-11-07T23:38:04Z [load_day 2023-08-04] 2023-08-04 hour 17: 170357 events in 4 files, snapshot 955255534670164276 (task 15737 ms: download 8201 ms overlapping gunzip 956 ms, parse 4230 ms, sort 110 ms, write 8801 ms, index 862 ms: 3 candidates, 69499 repo-hours, 999 releases; waiting 518 ms; commit 2989 ms)
2026-11-07T23:38:19Z [load_day 2023-08-04] 2023-08-04 hour 18: 169522 events in 4 files, snapshot 6736566635300731305 (task 16574 ms: download 9517 ms overlapping gunzip 925 ms, parse 4044 ms, sort 113 ms, write 10148 ms, index 567 ms: 2 candidates, 74479 repo-hours, 1009 releases; waiting 490 ms; commit 2492 ms)
2026-11-07T23:38:23Z [load_day 2023-08-04] 2023-08-04 hour 19: 155542 events in 4 files, snapshot 8888706330501821042 (task 15629 ms: download 6628 ms overlapping gunzip 888 ms, parse 3863 ms, sort 125 ms, write 9548 ms, index 474 ms: 3 candidates, 63690 repo-hours, 974 releases; waiting 453 ms; commit 3772 ms)
2026-11-07T23:38:35Z [load_day 2023-08-04] 2023-08-04 hour 20: 151044 events in 4 files, snapshot 5659264466222083371 (task 14694 ms: download 5970 ms overlapping gunzip 820 ms, parse 3629 ms, sort 143 ms, write 8865 ms, index 459 ms: 6 candidates, 62235 repo-hours, 839 releases; waiting 520 ms; commit 3576 ms)
2026-11-07T23:38:38Z [load_day 2023-08-04] 2023-08-04 hour 21: 134843 events in 3 files, snapshot 2735614341530936918 (task 13549 ms: download 6287 ms overlapping gunzip 739 ms, parse 3272 ms, sort 133 ms, write 8218 ms, index 417 ms: 1 candidates, 53734 repo-hours, 867 releases; waiting 506 ms; commit 2859 ms)
2026-11-07T23:38:47Z [load_day 2023-08-04] 2023-08-04 hour 22: 119947 events in 3 files, snapshot 8569834631566576289 (task 11629 ms: download 4411 ms overlapping gunzip 689 ms, parse 3070 ms, sort 69 ms, write 6598 ms, index 407 ms: 0 candidates, 48251 repo-hours, 737 releases; waiting 486 ms; commit 2258 ms)
2026-11-07T23:38:50Z [load_day 2023-08-04] 2023-08-04 hour 23: 107088 events in 3 files, snapshot 1341874647882384320 (task 10091 ms: download 2269 ms overlapping gunzip 632 ms, parse 2820 ms, sort 119 ms, write 5427 ms, index 308 ms: 2 candidates, 44348 repo-hours, 685 releases; waiting 519 ms; commit 3114 ms)
2026-11-07T23:38:57Z [load_day 2023-08-04] 2023-08-04: repo_hour merged from 24 files to 1 (1677872 rows, 7468 ms)
2026-11-07T23:39:00Z [load_day 2023-08-04] 2023-08-04: index committed in 9747 ms (23 + 1 + 24 files)
2026-11-07T23:39:00Z [load_day 2023-08-04] 2023-08-04: 3790703 events in 24 hours
2026-11-07T23:39:00Z [load_day 2023-08-04] attempt 1 succeeded