orcaLog in

Runs · run 2435

load_day 2024-05-10, attempt 1 succeeded

Host
73ccef5e07c2:120599
Started
2026-11-08T06:01:37Z
Ended
2026-11-08T06:03:27Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
686 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
167.8 s, up to 2.9 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/2435/load_day/2024-05-10/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T06:01:37Z [load_day 2024-05-10] attempt 1 started
2026-11-08T06:01:38Z [load_day 2024-05-10] 2024-05-10: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T06:01:45Z [load_day 2024-05-10] 2024-05-10 hour 0: 188542 events in 0 files, snapshot none (task 7026 ms: download 4176 ms overlapping gunzip 1111 ms, parse 4357 ms, sort 0 ms, write 0 ms, index 717 ms: 11 candidates, 95327 repo-hours, 1127 releases; waiting 511 ms; commit 0 ms)
2026-11-08T06:01:45Z [load_day 2024-05-10] 2024-05-10 hour 1: 193249 events in 0 files, snapshot none (task 6971 ms: download 4108 ms overlapping gunzip 1072 ms, parse 4223 ms, sort 0 ms, write 0 ms, index 754 ms: 8 candidates, 103869 repo-hours, 842 releases; waiting 458 ms; commit 0 ms)
2026-11-08T06:01:53Z [load_day 2024-05-10] 2024-05-10 hour 2: 179120 events in 0 files, snapshot none (task 7093 ms: download 4236 ms overlapping gunzip 1193 ms, parse 4599 ms, sort 0 ms, write 0 ms, index 534 ms: 7 candidates, 68942 repo-hours, 762 releases; waiting 481 ms; commit 0 ms)
2026-11-08T06:01:53Z [load_day 2024-05-10] 2024-05-10 hour 3: 183186 events in 0 files, snapshot none (task 6878 ms: download 3845 ms overlapping gunzip 1081 ms, parse 4453 ms, sort 0 ms, write 0 ms, index 752 ms: 10 candidates, 73999 repo-hours, 2855 releases; waiting 353 ms; commit 0 ms)
2026-11-08T06:02:00Z [load_day 2024-05-10] 2024-05-10 hour 4: 176141 events in 0 files, snapshot none (task 7173 ms: download 3849 ms overlapping gunzip 1102 ms, parse 4372 ms, sort 0 ms, write 0 ms, index 643 ms: 7 candidates, 61348 repo-hours, 797 releases; waiting 722 ms; commit 0 ms)
2026-11-08T06:02:01Z [load_day 2024-05-10] 2024-05-10 hour 5: 173693 events in 0 files, snapshot none (task 7409 ms: download 4700 ms overlapping gunzip 1240 ms, parse 4768 ms, sort 0 ms, write 0 ms, index 632 ms: 5 candidates, 66820 repo-hours, 896 releases; waiting 530 ms; commit 0 ms)
2026-11-08T06:02:09Z [load_day 2024-05-10] 2024-05-10 hour 6: 196346 events in 0 files, snapshot none (task 8038 ms: download 4788 ms overlapping gunzip 1157 ms, parse 4777 ms, sort 0 ms, write 0 ms, index 861 ms: 3 candidates, 87060 repo-hours, 944 releases; waiting 804 ms; commit 0 ms)
2026-11-08T06:02:09Z [load_day 2024-05-10] 2024-05-10 hour 7: 205043 events in 0 files, snapshot none (task 7600 ms: download 4438 ms overlapping gunzip 1185 ms, parse 4664 ms, sort 0 ms, write 0 ms, index 841 ms: 5 candidates, 79790 repo-hours, 957 releases; waiting 599 ms; commit 0 ms)
2026-11-08T06:02:18Z [load_day 2024-05-10] 2024-05-10 hour 8: 220137 events in 0 files, snapshot none (task 8491 ms: download 5069 ms overlapping gunzip 1346 ms, parse 5711 ms, sort 0 ms, write 0 ms, index 729 ms: 5 candidates, 87303 repo-hours, 1108 releases; waiting 381 ms; commit 0 ms)
2026-11-08T06:02:18Z [load_day 2024-05-10] 2024-05-10 hour 9: 219001 events in 0 files, snapshot none (task 8096 ms: download 4939 ms overlapping gunzip 1264 ms, parse 5347 ms, sort 0 ms, write 0 ms, index 674 ms: 11 candidates, 90441 repo-hours, 1183 releases; waiting 489 ms; commit 0 ms)
2026-11-08T06:02:28Z [load_day 2024-05-10] 2024-05-10 hour 10: 215516 events in 0 files, snapshot none (task 8960 ms: download 6002 ms overlapping gunzip 1418 ms, parse 6125 ms, sort 0 ms, write 0 ms, index 677 ms: 15 candidates, 95517 repo-hours, 1108 releases; waiting 528 ms; commit 0 ms)
2026-11-08T06:02:28Z [load_day 2024-05-10] 2024-05-10 hour 11: 206373 events in 0 files, snapshot none (task 8383 ms: download 4708 ms overlapping gunzip 1280 ms, parse 5642 ms, sort 0 ms, write 0 ms, index 791 ms: 9 candidates, 80378 repo-hours, 1025 releases; waiting 386 ms; commit 0 ms)
2026-11-08T06:02:35Z [load_day 2024-05-10] 2024-05-10 hour 12: 269269 events in 0 files, snapshot none (task 7221 ms: download 4444 ms overlapping gunzip 1058 ms, parse 4775 ms, sort 0 ms, write 0 ms, index 834 ms: 17 candidates, 121175 repo-hours, 1256 releases; waiting 319 ms; commit 0 ms)
2026-11-08T06:02:36Z [load_day 2024-05-10] 2024-05-10 hour 13: 242490 events in 0 files, snapshot none (task 7699 ms: download 5120 ms overlapping gunzip 1166 ms, parse 5004 ms, sort 0 ms, write 0 ms, index 744 ms: 15 candidates, 109570 repo-hours, 1293 releases; waiting 494 ms; commit 0 ms)
2026-11-08T06:02:44Z [load_day 2024-05-10] 2024-05-10 hour 14: 241477 events in 0 files, snapshot none (task 8572 ms: download 4936 ms overlapping gunzip 1315 ms, parse 5277 ms, sort 0 ms, write 0 ms, index 798 ms: 8 candidates, 94105 repo-hours, 1204 releases; waiting 737 ms; commit 0 ms)
2026-11-08T06:02:44Z [load_day 2024-05-10] 2024-05-10 hour 15: 238421 events in 0 files, snapshot none (task 8043 ms: download 4487 ms overlapping gunzip 1170 ms, parse 5016 ms, sort 0 ms, write 0 ms, index 1066 ms: 9 candidates, 96170 repo-hours, 1329 releases; waiting 490 ms; commit 0 ms)
2026-11-08T06:02:53Z [load_day 2024-05-10] 2024-05-10 hour 16: 215731 events in 0 files, snapshot none (task 7820 ms: download 5001 ms overlapping gunzip 1270 ms, parse 5162 ms, sort 0 ms, write 0 ms, index 789 ms: 6 candidates, 87777 repo-hours, 1183 releases; waiting 358 ms; commit 0 ms)
2026-11-08T06:02:53Z [load_day 2024-05-10] 2024-05-10 hour 17: 202912 events in 0 files, snapshot none (task 8469 ms: download 5958 ms overlapping gunzip 1295 ms, parse 5730 ms, sort 0 ms, write 0 ms, index 624 ms: 13 candidates, 79909 repo-hours, 1091 releases; waiting 532 ms; commit 0 ms)
2026-11-08T06:03:01Z [load_day 2024-05-10] 2024-05-10 hour 18: 210105 events in 0 files, snapshot none (task 7726 ms: download 4727 ms overlapping gunzip 1152 ms, parse 4860 ms, sort 0 ms, write 0 ms, index 940 ms: 6 candidates, 91345 repo-hours, 1193 releases; waiting 530 ms; commit 0 ms)
2026-11-08T06:03:02Z [load_day 2024-05-10] 2024-05-10 hour 19: 199653 events in 0 files, snapshot none (task 7716 ms: download 4411 ms overlapping gunzip 1215 ms, parse 5183 ms, sort 0 ms, write 0 ms, index 587 ms: 6 candidates, 72952 repo-hours, 1038 releases; waiting 481 ms; commit 0 ms)
2026-11-08T06:03:09Z [load_day 2024-05-10] 2024-05-10 hour 20: 191089 events in 0 files, snapshot none (task 7002 ms: download 4104 ms overlapping gunzip 1071 ms, parse 4474 ms, sort 0 ms, write 0 ms, index 572 ms: 6 candidates, 73124 repo-hours, 930 releases; waiting 506 ms; commit 0 ms)
2026-11-08T06:03:09Z [load_day 2024-05-10] 2024-05-10 hour 21: 175671 events in 0 files, snapshot none (task 6251 ms: download 3511 ms overlapping gunzip 971 ms, parse 4027 ms, sort 0 ms, write 0 ms, index 485 ms: 12 candidates, 67359 repo-hours, 1042 releases; waiting 245 ms; commit 0 ms)
2026-11-08T06:03:15Z [load_day 2024-05-10] 2024-05-10 hour 22: 150010 events in 0 files, snapshot none (task 6010 ms: download 3592 ms overlapping gunzip 929 ms, parse 3758 ms, sort 0 ms, write 0 ms, index 396 ms: 11 candidates, 54778 repo-hours, 974 releases; waiting 532 ms; commit 0 ms)
2026-11-08T06:03:15Z [load_day 2024-05-10] 2024-05-10 hour 23: 137860 events in 0 files, snapshot none (task 5394 ms: download 3163 ms overlapping gunzip 835 ms, parse 3311 ms, sort 0 ms, write 0 ms, index 421 ms: 9 candidates, 50990 repo-hours, 703 releases; waiting 552 ms; commit 0 ms)
2026-11-08T06:03:24Z [load_day 2024-05-10] 2024-05-10: repo_hour merged from 24 files to 1 (1990048 rows, 8892 ms)
2026-11-08T06:03:27Z [load_day 2024-05-10] 2024-05-10: index committed in 11527 ms (24 + 1 + 24 files)
2026-11-08T06:03:27Z [load_day 2024-05-10] 2024-05-10: 4831035 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T06:03:27Z [load_day 2024-05-10] attempt 1 succeeded