load_day 2024-06-04, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:131434

Started

2026-11-08T06:23:22Z

Ended

2026-11-08T06:25:16Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

726 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

188.7 s, up to 2.9 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2544/load_day/2024-06-04/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T06:23:22Z [load_day 2024-06-04] attempt 1 started
2026-11-08T06:23:23Z [load_day 2024-06-04] 2024-06-04: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T06:23:30Z [load_day 2024-06-04] 2024-06-04 hour 0: 200115 events in 0 files, snapshot none (task 6882 ms: download 3895 ms overlapping gunzip 1103 ms, parse 4601 ms, sort 0 ms, write 0 ms, index 967 ms: 10 candidates, 97708 repo-hours, 1201 releases; waiting 2 ms; commit 0 ms)
2026-11-08T06:23:30Z [load_day 2024-06-04] 2024-06-04 hour 1: 208970 events in 0 files, snapshot none (task 7092 ms: download 3986 ms overlapping gunzip 1098 ms, parse 4938 ms, sort 0 ms, write 0 ms, index 863 ms: 12 candidates, 110804 repo-hours, 949 releases; waiting 0 ms; commit 0 ms)
2026-11-08T06:23:37Z [load_day 2024-06-04] 2024-06-04 hour 2: 176113 events in 0 files, snapshot none (task 6602 ms: download 3257 ms overlapping gunzip 1024 ms, parse 4457 ms, sort 0 ms, write 0 ms, index 901 ms: 9 candidates, 72315 repo-hours, 925 releases; waiting 14 ms; commit 0 ms)
2026-11-08T06:23:38Z [load_day 2024-06-04] 2024-06-04 hour 3: 189085 events in 0 files, snapshot none (task 7489 ms: download 4826 ms overlapping gunzip 1279 ms, parse 5327 ms, sort 0 ms, write 0 ms, index 679 ms: 13 candidates, 77677 repo-hours, 4322 releases; waiting 0 ms; commit 0 ms)
2026-11-08T06:23:44Z [load_day 2024-06-04] 2024-06-04 hour 4: 166478 events in 0 files, snapshot none (task 6219 ms: download 3249 ms overlapping gunzip 1074 ms, parse 4431 ms, sort 0 ms, write 0 ms, index 493 ms: 15 candidates, 64609 repo-hours, 781 releases; waiting 20 ms; commit 0 ms)
2026-11-08T06:23:46Z [load_day 2024-06-04] 2024-06-04 hour 5: 177941 events in 0 files, snapshot none (task 7150 ms: download 4216 ms overlapping gunzip 1190 ms, parse 5028 ms, sort 0 ms, write 0 ms, index 711 ms: 9 candidates, 69945 repo-hours, 914 releases; waiting 6 ms; commit 0 ms)
2026-11-08T06:23:52Z [load_day 2024-06-04] 2024-06-04 hour 6: 213047 events in 0 files, snapshot none (task 7680 ms: download 4311 ms overlapping gunzip 1207 ms, parse 5259 ms, sort 0 ms, write 0 ms, index 957 ms: 9 candidates, 94933 repo-hours, 1098 releases; waiting 21 ms; commit 0 ms)
2026-11-08T06:23:54Z [load_day 2024-06-04] 2024-06-04 hour 7: 216561 events in 0 files, snapshot none (task 7304 ms: download 4440 ms overlapping gunzip 1144 ms, parse 5152 ms, sort 0 ms, write 0 ms, index 801 ms: 4 candidates, 87472 repo-hours, 1250 releases; waiting 21 ms; commit 0 ms)
2026-11-08T06:24:00Z [load_day 2024-06-04] 2024-06-04 hour 8: 234273 events in 0 files, snapshot none (task 7006 ms: download 4584 ms overlapping gunzip 1196 ms, parse 4952 ms, sort 0 ms, write 0 ms, index 640 ms: 18 candidates, 93729 repo-hours, 1377 releases; waiting 18 ms; commit 0 ms)
2026-11-08T06:24:02Z [load_day 2024-06-04] 2024-06-04 hour 9: 238308 events in 0 files, snapshot none (task 7034 ms: download 4473 ms overlapping gunzip 1170 ms, parse 4812 ms, sort 0 ms, write 0 ms, index 838 ms: 11 candidates, 99667 repo-hours, 1370 releases; waiting 20 ms; commit 0 ms)
2026-11-08T06:24:07Z [load_day 2024-06-04] 2024-06-04 hour 10: 230595 events in 0 files, snapshot none (task 6576 ms: download 4122 ms overlapping gunzip 1101 ms, parse 4617 ms, sort 0 ms, write 0 ms, index 651 ms: 8 candidates, 90515 repo-hours, 1175 releases; waiting 15 ms; commit 0 ms)
2026-11-08T06:24:09Z [load_day 2024-06-04] 2024-06-04 hour 11: 242251 events in 0 files, snapshot none (task 6465 ms: download 3988 ms overlapping gunzip 1076 ms, parse 4447 ms, sort 0 ms, write 0 ms, index 731 ms: 8 candidates, 90190 repo-hours, 1108 releases; waiting 19 ms; commit 0 ms)
2026-11-08T06:24:17Z [load_day 2024-06-04] 2024-06-04 hour 12: 291129 events in 0 files, snapshot none (task 9542 ms: download 5712 ms overlapping gunzip 1544 ms, parse 6573 ms, sort 0 ms, write 0 ms, index 1207 ms: 12 candidates, 133241 repo-hours, 1379 releases; waiting 20 ms; commit 0 ms)
2026-11-08T06:24:18Z [load_day 2024-06-04] 2024-06-04 hour 13: 268494 events in 0 files, snapshot none (task 8971 ms: download 6195 ms overlapping gunzip 1458 ms, parse 6429 ms, sort 0 ms, write 0 ms, index 875 ms: 11 candidates, 118371 repo-hours, 1254 releases; waiting 17 ms; commit 0 ms)
2026-11-08T06:24:29Z [load_day 2024-06-04] 2024-06-04 hour 14: 255483 events in 0 files, snapshot none (task 11168 ms: download 7769 ms overlapping gunzip 1902 ms, parse 7588 ms, sort 0 ms, write 0 ms, index 1468 ms: 14 candidates, 108807 repo-hours, 1369 releases; waiting 10 ms; commit 0 ms)
2026-11-08T06:24:30Z [load_day 2024-06-04] 2024-06-04 hour 15: 257146 events in 0 files, snapshot none (task 11930 ms: download 7996 ms overlapping gunzip 1866 ms, parse 8406 ms, sort 0 ms, write 0 ms, index 1424 ms: 14 candidates, 108346 repo-hours, 1301 releases; waiting 29 ms; commit 0 ms)
2026-11-08T06:24:39Z [load_day 2024-06-04] 2024-06-04 hour 16: 233081 events in 0 files, snapshot none (task 10428 ms: download 6438 ms overlapping gunzip 1654 ms, parse 7602 ms, sort 0 ms, write 0 ms, index 943 ms: 13 candidates, 93588 repo-hours, 1366 releases; waiting 28 ms; commit 0 ms)
2026-11-08T06:24:40Z [load_day 2024-06-04] 2024-06-04 hour 17: 229941 events in 0 files, snapshot none (task 8680 ms: download 5376 ms overlapping gunzip 1413 ms, parse 6145 ms, sort 0 ms, write 0 ms, index 907 ms: 7 candidates, 89197 repo-hours, 1224 releases; waiting 16 ms; commit 0 ms)
2026-11-08T06:24:48Z [load_day 2024-06-04] 2024-06-04 hour 18: 242828 events in 0 files, snapshot none (task 8552 ms: download 6017 ms overlapping gunzip 1437 ms, parse 6202 ms, sort 0 ms, write 0 ms, index 693 ms: 10 candidates, 102667 repo-hours, 1304 releases; waiting 18 ms; commit 0 ms)
2026-11-08T06:24:48Z [load_day 2024-06-04] 2024-06-04 hour 19: 228203 events in 0 files, snapshot none (task 7905 ms: download 5081 ms overlapping gunzip 1361 ms, parse 5652 ms, sort 0 ms, write 0 ms, index 679 ms: 13 candidates, 82071 repo-hours, 1194 releases; waiting 10 ms; commit 0 ms)
2026-11-08T06:24:57Z [load_day 2024-06-04] 2024-06-04 hour 20: 214848 events in 0 files, snapshot none (task 7807 ms: download 5035 ms overlapping gunzip 1347 ms, parse 5675 ms, sort 0 ms, write 0 ms, index 599 ms: 12 candidates, 81839 repo-hours, 1094 releases; waiting 0 ms; commit 0 ms)
2026-11-08T06:24:57Z [load_day 2024-06-04] 2024-06-04 hour 21: 201601 events in 0 files, snapshot none (task 6537 ms: download 4107 ms overlapping gunzip 1137 ms, parse 4500 ms, sort 0 ms, write 0 ms, index 687 ms: 9 candidates, 77787 repo-hours, 1103 releases; waiting 16 ms; commit 0 ms)
2026-11-08T06:25:04Z [load_day 2024-06-04] 2024-06-04 hour 22: 188226 events in 0 files, snapshot none (task 6643 ms: download 4332 ms overlapping gunzip 1172 ms, parse 4802 ms, sort 0 ms, write 0 ms, index 443 ms: 10 candidates, 64456 repo-hours, 989 releases; waiting 19 ms; commit 0 ms)
2026-11-08T06:25:04Z [load_day 2024-06-04] 2024-06-04 hour 23: 168461 events in 0 files, snapshot none (task 6202 ms: download 3370 ms overlapping gunzip 1105 ms, parse 4347 ms, sort 0 ms, write 0 ms, index 544 ms: 4 candidates, 60263 repo-hours, 720 releases; waiting 14 ms; commit 0 ms)
2026-11-08T06:25:14Z [load_day 2024-06-04] 2024-06-04: repo_hour merged from 24 files to 1 (2170197 rows, 9870 ms)
2026-11-08T06:25:16Z [load_day 2024-06-04] 2024-06-04: index committed in 12300 ms (24 + 1 + 24 files)
2026-11-08T06:25:16Z [load_day 2024-06-04] 2024-06-04: 5273178 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T06:25:16Z [load_day 2024-06-04] attempt 1 succeeded