orcaLog in

Runs · run 1587

load_day 2023-10-04, attempt 1 succeeded

Host
3c2b29713ce8:400201
Started
2026-11-08T01:27:04Z
Ended
2026-11-08T01:31:18Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1587/load_day/2023-10-04/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:27:04Z [load_day 2023-10-04] attempt 1 started
2026-11-08T01:27:05Z [load_day 2023-10-04] 2023-10-04: 2 hours in flight, about 1200 MB
2026-11-08T01:27:27Z [load_day 2023-10-04] 2023-10-04 hour 0: 177730 events in 4 files, snapshot 1770525849404621467 (task 18212 ms: download 11190 ms overlapping gunzip 1284 ms, parse 5556 ms, sort 121 ms, write 9836 ms, index 671 ms: 14 candidates, 91679 repo-hours, 1069 releases; waiting 467 ms; commit 2247 ms)
2026-11-08T01:27:30Z [load_day 2023-10-04] 2023-10-04 hour 1: 161821 events in 4 files, snapshot 2417585581759820164 (task 16786 ms: download 7036 ms overlapping gunzip 999 ms, parse 4554 ms, sort 151 ms, write 9496 ms, index 718 ms: 7 candidates, 87337 repo-hours, 784 releases; waiting 526 ms; commit 3138 ms)
2026-11-08T01:27:42Z [load_day 2023-10-04] 2023-10-04 hour 2: 141914 events in 3 files, snapshot 5919250107323393007 (task 13931 ms: download 6224 ms overlapping gunzip 885 ms, parse 3736 ms, sort 132 ms, write 7922 ms, index 469 ms: 12 candidates, 59031 repo-hours, 757 releases; waiting 496 ms; commit 2710 ms)
2026-11-08T01:27:45Z [load_day 2023-10-04] 2023-10-04 hour 3: 141248 events in 3 files, snapshot 8814983557561606238 (task 13209 ms: download 5732 ms overlapping gunzip 836 ms, parse 3693 ms, sort 110 ms, write 7269 ms, index 556 ms: 4 candidates, 57525 repo-hours, 852 releases; waiting 474 ms; commit 2855 ms)
2026-11-08T01:27:56Z [load_day 2023-10-04] 2023-10-04 hour 4: 143948 events in 3 files, snapshot 4131321072388862260 (task 12771 ms: download 4769 ms overlapping gunzip 792 ms, parse 3420 ms, sort 171 ms, write 6994 ms, index 583 ms: 6 candidates, 58317 repo-hours, 701 releases; waiting 522 ms; commit 2497 ms)
2026-11-08T01:28:01Z [load_day 2023-10-04] 2023-10-04 hour 5: 149501 events in 3 files, snapshot 3706495726528222199 (task 13872 ms: download 6050 ms overlapping gunzip 827 ms, parse 3608 ms, sort 101 ms, write 7871 ms, index 536 ms: 4 candidates, 61999 repo-hours, 889 releases; waiting 654 ms; commit 3083 ms)
2026-11-08T01:28:17Z [load_day 2023-10-04] 2023-10-04 hour 6: 172155 events in 4 files, snapshot 1548388139967875514 (task 17022 ms: download 9988 ms overlapping gunzip 997 ms, parse 4376 ms, sort 126 ms, write 10181 ms, index 540 ms: 4 candidates, 75255 repo-hours, 1007 releases; waiting 498 ms; commit 4607 ms)
2026-11-08T01:28:21Z [load_day 2023-10-04] 2023-10-04 hour 7: 184033 events in 4 files, snapshot 222627069908629804 (task 19678 ms: download 10972 ms overlapping gunzip 1187 ms, parse 5324 ms, sort 113 ms, write 11514 ms, index 605 ms: 5 candidates, 76641 repo-hours, 974 releases; waiting 510 ms; commit 2119 ms)
2026-11-08T01:28:36Z [load_day 2023-10-04] 2023-10-04 hour 8: 193875 events in 4 files, snapshot 8530706874763401481 (task 20913 ms: download 12288 ms overlapping gunzip 1231 ms, parse 5456 ms, sort 256 ms, write 12839 ms, index 610 ms: 11 candidates, 83308 repo-hours, 1131 releases; waiting 302 ms; commit 2135 ms)
2026-11-08T01:28:41Z [load_day 2023-10-04] 2023-10-04 hour 9: 193333 events in 4 files, snapshot 9003558259661028103 (task 18927 ms: download 11063 ms overlapping gunzip 1067 ms, parse 4721 ms, sort 132 ms, write 11572 ms, index 618 ms: 12 candidates, 83251 repo-hours, 1337 releases; waiting 510 ms; commit 2770 ms)
2026-11-08T01:28:53Z [load_day 2023-10-04] 2023-10-04 hour 10: 185161 events in 4 files, snapshot 4499136338045775719 (task 15588 ms: download 8891 ms overlapping gunzip 936 ms, parse 4136 ms, sort 138 ms, write 9284 ms, index 544 ms: 9 candidates, 81857 repo-hours, 1083 releases; waiting 213 ms; commit 2617 ms)
2026-11-08T01:29:00Z [load_day 2023-10-04] 2023-10-04 hour 11: 186229 events in 4 files, snapshot 4893611806215545643 (task 16558 ms: download 8509 ms overlapping gunzip 939 ms, parse 4099 ms, sort 109 ms, write 9619 ms, index 1002 ms: 7 candidates, 80678 repo-hours, 1113 releases; waiting 508 ms; commit 3928 ms)
2026-11-08T01:29:17Z [load_day 2023-10-04] 2023-10-04 hour 12: 240405 events in 5 files, snapshot 1217985695977266779 (task 23262 ms: download 14837 ms overlapping gunzip 1110 ms, parse 5212 ms, sort 143 ms, write 14772 ms, index 1183 ms: 11 candidates, 115752 repo-hours, 1068 releases; waiting 531 ms; commit 2565 ms)
2026-11-08T01:29:23Z [load_day 2023-10-04] 2023-10-04 hour 13: 218642 events in 5 files, snapshot 1256934811290949997 (task 22275 ms: download 14189 ms overlapping gunzip 1402 ms, parse 6493 ms, sort 146 ms, write 12892 ms, index 891 ms: 11 candidates, 108751 repo-hours, 1087 releases; waiting 178 ms; commit 2727 ms)
2026-11-08T01:29:42Z [load_day 2023-10-04] 2023-10-04 hour 14: 207288 events in 5 files, snapshot 6243853218267045198 (task 23141 ms: download 12586 ms overlapping gunzip 1245 ms, parse 5784 ms, sort 125 ms, write 14438 ms, index 804 ms: 8 candidates, 95879 repo-hours, 976 releases; waiting 523 ms; commit 2781 ms)
2026-11-08T01:29:46Z [load_day 2023-10-04] 2023-10-04 hour 15: 195568 events in 4 files, snapshot 2526568104479717324 (task 21823 ms: download 11920 ms overlapping gunzip 1308 ms, parse 5813 ms, sort 129 ms, write 12557 ms, index 1120 ms: 5 candidates, 92248 repo-hours, 1153 releases; waiting 503 ms; commit 3413 ms)
2026-11-08T01:30:06Z [load_day 2023-10-04] 2023-10-04 hour 16: 192948 events in 4 files, snapshot 1156073854896200472 (task 23028 ms: download 12867 ms overlapping gunzip 1356 ms, parse 6426 ms, sort 174 ms, write 13133 ms, index 1088 ms: 3 candidates, 89065 repo-hours, 950 releases; waiting 530 ms; commit 3083 ms)
2026-11-08T01:30:09Z [load_day 2023-10-04] 2023-10-04 hour 17: 188110 events in 4 files, snapshot 877295109219005312 (task 21451 ms: download 11983 ms overlapping gunzip 1071 ms, parse 4666 ms, sort 119 ms, write 13863 ms, index 920 ms: 3 candidates, 84852 repo-hours, 911 releases; waiting 541 ms; commit 3480 ms)
2026-11-08T01:30:27Z [load_day 2023-10-04] 2023-10-04 hour 18: 191390 events in 4 files, snapshot 4293873746588691791 (task 20530 ms: download 12249 ms overlapping gunzip 1233 ms, parse 5362 ms, sort 122 ms, write 12551 ms, index 634 ms: 12 candidates, 88949 repo-hours, 1061 releases; waiting 368 ms; commit 2519 ms)
2026-11-08T01:30:31Z [load_day 2023-10-04] 2023-10-04 hour 19: 182212 events in 4 files, snapshot 8353721467386222765 (task 20396 ms: download 12140 ms overlapping gunzip 1258 ms, parse 5725 ms, sort 120 ms, write 11598 ms, index 947 ms: 6 candidates, 78005 repo-hours, 1003 releases; waiting 470 ms; commit 3066 ms)
2026-11-08T01:30:49Z [load_day 2023-10-04] 2023-10-04 hour 20: 174179 events in 4 files, snapshot 1058420396867692762 (task 17970 ms: download 10770 ms overlapping gunzip 1066 ms, parse 5084 ms, sort 116 ms, write 10247 ms, index 652 ms: 6 candidates, 74671 repo-hours, 825 releases; waiting 519 ms; commit 4444 ms)
2026-11-08T01:30:51Z [load_day 2023-10-04] 2023-10-04 hour 21: 162771 events in 4 files, snapshot 3713279603022141229 (task 17239 ms: download 10903 ms overlapping gunzip 966 ms, parse 4312 ms, sort 99 ms, write 10534 ms, index 507 ms: 7 candidates, 67603 repo-hours, 907 releases; waiting 519 ms; commit 2477 ms)
2026-11-08T01:31:04Z [load_day 2023-10-04] 2023-10-04 hour 22: 145677 events in 3 files, snapshot 285431453371697312 (task 15529 ms: download 7211 ms overlapping gunzip 992 ms, parse 4235 ms, sort 115 ms, write 8648 ms, index 444 ms: 10 candidates, 59612 repo-hours, 934 releases; waiting 788 ms; commit 3449 ms)
2026-11-08T01:31:07Z [load_day 2023-10-04] 2023-10-04 hour 23: 134167 events in 3 files, snapshot 4727323722469787859 (task 13723 ms: download 6133 ms overlapping gunzip 746 ms, parse 3439 ms, sort 100 ms, write 8187 ms, index 410 ms: 5 candidates, 55432 repo-hours, 741 releases; waiting 494 ms; commit 2214 ms)
2026-11-08T01:31:15Z [load_day 2023-10-04] 2023-10-04: repo_hour merged from 24 files to 1 (1907697 rows, 8697 ms)
2026-11-08T01:31:18Z [load_day 2023-10-04] 2023-10-04: index committed in 11130 ms (24 + 1 + 24 files)
2026-11-08T01:31:18Z [load_day 2023-10-04] 2023-10-04: 4264305 events in 24 hours
2026-11-08T01:31:18Z [load_day 2023-10-04] attempt 1 succeeded