orcaLog in

Runs · run 1802

load_day 2023-12-01, attempt 1 succeeded

Host
3c2b29713ce8:619710
Started
2026-11-08T03:22:26Z
Ended
2026-11-08T03:27:08Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1802/load_day/2023-12-01/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:22:26Z [load_day 2023-12-01] attempt 1 started
2026-11-08T03:22:27Z [load_day 2023-12-01] 2023-12-01: 2 hours in flight, about 1200 MB
2026-11-08T03:22:54Z [load_day 2023-12-01] 2023-12-01 hour 0: 265638 events in 6 files, snapshot 2517884441750887800 (task 21382 ms: download 13524 ms overlapping gunzip 1351 ms, parse 6402 ms, sort 197 ms, write 12045 ms, index 577 ms: 5 candidates, 82649 repo-hours, 886 releases; waiting 451 ms; commit 2959 ms)
2026-11-08T03:22:57Z [load_day 2023-12-01] 2023-12-01 hour 1: 289255 events in 6 files, snapshot 8193587864140966015 (task 20760 ms: download 12000 ms overlapping gunzip 1246 ms, parse 5811 ms, sort 266 ms, write 12231 ms, index 698 ms: 6 candidates, 90884 repo-hours, 1094 releases; waiting 241 ms; commit 3147 ms)
2026-11-08T03:23:14Z [load_day 2023-12-01] 2023-12-01 hour 2: 288275 events in 6 files, snapshot 1933897775206507005 (task 18804 ms: download 10554 ms overlapping gunzip 1113 ms, parse 5191 ms, sort 251 ms, write 10845 ms, index 582 ms: 4 candidates, 61250 repo-hours, 722 releases; waiting 496 ms; commit 3222 ms)
2026-11-08T03:23:19Z [load_day 2023-12-01] 2023-12-01 hour 3: 282457 events in 6 files, snapshot 1514843083112825463 (task 19694 ms: download 11086 ms overlapping gunzip 1231 ms, parse 5670 ms, sort 330 ms, write 10960 ms, index 681 ms: 8 candidates, 56960 repo-hours, 817 releases; waiting 532 ms; commit 4344 ms)
2026-11-08T03:23:33Z [load_day 2023-12-01] 2023-12-01 hour 4: 277389 events in 6 files, snapshot 3592488660953954903 (task 17447 ms: download 10450 ms overlapping gunzip 1134 ms, parse 5264 ms, sort 325 ms, write 9490 ms, index 471 ms: 4 candidates, 53721 repo-hours, 598 releases; waiting 490 ms; commit 3205 ms)
2026-11-08T03:23:39Z [load_day 2023-12-01] 2023-12-01 hour 5: 288026 events in 6 files, snapshot 2418091950892977629 (task 19063 ms: download 11211 ms overlapping gunzip 1288 ms, parse 5679 ms, sort 328 ms, write 10544 ms, index 396 ms: 9 candidates, 61005 repo-hours, 640 releases; waiting 491 ms; commit 4259 ms)
2026-11-08T03:23:52Z [load_day 2023-12-01] 2023-12-01 hour 6: 288844 events in 6 files, snapshot 770337138629242308 (task 18481 ms: download 10781 ms overlapping gunzip 1240 ms, parse 5601 ms, sort 237 ms, write 10090 ms, index 547 ms: 4 candidates, 68314 repo-hours, 715 releases; waiting 511 ms; commit 2703 ms)
2026-11-08T03:23:57Z [load_day 2023-12-01] 2023-12-01 hour 7: 287873 events in 6 files, snapshot 7829346707016186936 (task 17265 ms: download 10363 ms overlapping gunzip 1055 ms, parse 4827 ms, sort 286 ms, write 9973 ms, index 455 ms: 5 candidates, 65628 repo-hours, 701 releases; waiting 439 ms; commit 3518 ms)
2026-11-08T03:24:11Z [load_day 2023-12-01] 2023-12-01 hour 8: 295537 events in 6 files, snapshot 5474828403391640586 (task 17032 ms: download 11365 ms overlapping gunzip 1064 ms, parse 4777 ms, sort 235 ms, write 9859 ms, index 495 ms: 8 candidates, 73808 repo-hours, 834 releases; waiting 354 ms; commit 2591 ms)
2026-11-08T03:24:17Z [load_day 2023-12-01] 2023-12-01 hour 9: 293586 events in 6 files, snapshot 3349482951810524239 (task 18971 ms: download 12168 ms overlapping gunzip 1090 ms, parse 5005 ms, sort 213 ms, write 11045 ms, index 908 ms: 8 candidates, 74733 repo-hours, 1088 releases; waiting 485 ms; commit 3106 ms)
2026-11-08T03:24:37Z [load_day 2023-12-01] 2023-12-01 hour 10: 293593 events in 6 files, snapshot 8110962931912437430 (task 21925 ms: download 15482 ms overlapping gunzip 1237 ms, parse 5711 ms, sort 249 ms, write 13299 ms, index 634 ms: 8 candidates, 76014 repo-hours, 925 releases; waiting 513 ms; commit 6023 ms)
2026-11-08T03:24:40Z [load_day 2023-12-01] 2023-12-01 hour 11: 294641 events in 6 files, snapshot 7991745293884545351 (task 22210 ms: download 14055 ms overlapping gunzip 1307 ms, parse 6060 ms, sort 252 ms, write 12980 ms, index 839 ms: 7 candidates, 72856 repo-hours, 795 releases; waiting 491 ms; commit 2601 ms)
2026-11-08T03:24:58Z [load_day 2023-12-01] 2023-12-01 hour 12: 306836 events in 7 files, snapshot 597371376189583794 (task 22146 ms: download 14571 ms overlapping gunzip 1388 ms, parse 6334 ms, sort 282 ms, write 12682 ms, index 721 ms: 14 candidates, 95218 repo-hours, 785 releases; waiting 523 ms; commit 3153 ms)
2026-11-08T03:25:06Z [load_day 2023-12-01] 2023-12-01 hour 13: 300888 events in 7 files, snapshot 824147079989628176 (task 23383 ms: download 15099 ms overlapping gunzip 1266 ms, parse 5958 ms, sort 370 ms, write 14024 ms, index 932 ms: 13 candidates, 89671 repo-hours, 954 releases; waiting 493 ms; commit 4408 ms)
2026-11-08T03:25:25Z [load_day 2023-12-01] 2023-12-01 hour 14: 303316 events in 7 files, snapshot 2403021491607721552 (task 27162 ms: download 17895 ms overlapping gunzip 1586 ms, parse 7503 ms, sort 281 ms, write 16140 ms, index 817 ms: 16 candidates, 85011 repo-hours, 917 releases; waiting 505 ms; commit 2677 ms)
2026-11-08T03:25:33Z [load_day 2023-12-01] 2023-12-01 hour 15: 300391 events in 7 files, snapshot 2094207420863646944 (task 24795 ms: download 17404 ms overlapping gunzip 1485 ms, parse 6563 ms, sort 348 ms, write 15271 ms, index 607 ms: 14 candidates, 82804 repo-hours, 1062 releases; waiting 201 ms; commit 4971 ms)
2026-11-08T03:25:50Z [load_day 2023-12-01] 2023-12-01 hour 16: 294956 events in 6 files, snapshot 507436328097650065 (task 21903 ms: download 14781 ms overlapping gunzip 1386 ms, parse 6017 ms, sort 270 ms, write 12741 ms, index 670 ms: 17 candidates, 80089 repo-hours, 918 releases; waiting 520 ms; commit 4371 ms)
2026-11-08T03:26:03Z [load_day 2023-12-01] 2023-12-01 hour 17: 293468 events in 6 files, snapshot 4686027887652150015 (task 21425 ms: download 14348 ms overlapping gunzip 1261 ms, parse 5756 ms, sort 212 ms, write 12705 ms, index 713 ms: 14 candidates, 75516 repo-hours, 901 releases; waiting 520 ms; commit 12157 ms)
2026-11-08T03:26:13Z [load_day 2023-12-01] 2023-12-01 hour 18: 297281 events in 6 files, snapshot 7127758337285103219 (task 22757 ms: download 16126 ms overlapping gunzip 1499 ms, parse 6661 ms, sort 327 ms, write 12731 ms, index 801 ms: 10 candidates, 78363 repo-hours, 856 releases; waiting 502 ms; commit 2994 ms)
2026-11-08T03:26:18Z [load_day 2023-12-01] 2023-12-01 hour 19: 286439 events in 6 files, snapshot 1921638084251727578 (task 23007 ms: download 14047 ms overlapping gunzip 1267 ms, parse 6204 ms, sort 285 ms, write 13734 ms, index 672 ms: 5 candidates, 67007 repo-hours, 795 releases; waiting 546 ms; commit 3226 ms)
2026-11-08T03:26:34Z [load_day 2023-12-01] 2023-12-01 hour 20: 287174 events in 6 files, snapshot 4148109128837125975 (task 19403 ms: download 11739 ms overlapping gunzip 1163 ms, parse 5282 ms, sort 271 ms, write 11381 ms, index 468 ms: 8 candidates, 64067 repo-hours, 700 releases; waiting 499 ms; commit 3454 ms)
2026-11-08T03:26:39Z [load_day 2023-12-01] 2023-12-01 hour 21: 283123 events in 6 files, snapshot 4890605172267988482 (task 19953 ms: download 11023 ms overlapping gunzip 1175 ms, parse 5337 ms, sort 349 ms, write 11888 ms, index 457 ms: 9 candidates, 61988 repo-hours, 785 releases; waiting 487 ms; commit 3132 ms)
2026-11-08T03:26:54Z [load_day 2023-12-01] 2023-12-01 hour 22: 277027 events in 6 files, snapshot 3935246015172963532 (task 19706 ms: download 11500 ms overlapping gunzip 1252 ms, parse 5704 ms, sort 207 ms, write 11172 ms, index 500 ms: 10 candidates, 58042 repo-hours, 762 releases; waiting 538 ms; commit 2947 ms)
2026-11-08T03:26:57Z [load_day 2023-12-01] 2023-12-01 hour 23: 269109 events in 6 files, snapshot 8948049576762457526 (task 16729 ms: download 10329 ms overlapping gunzip 1127 ms, parse 5033 ms, sort 200 ms, write 9200 ms, index 388 ms: 8 candidates, 54571 repo-hours, 714 releases; waiting 532 ms; commit 2434 ms)
2026-11-08T03:27:05Z [load_day 2023-12-01] 2023-12-01: repo_hour merged from 24 files to 1 (1730169 rows, 8240 ms)
2026-11-08T03:27:08Z [load_day 2023-12-01] 2023-12-01: index committed in 10625 ms (24 + 1 + 24 files)
2026-11-08T03:27:08Z [load_day 2023-12-01] 2023-12-01: 6945122 events in 24 hours
2026-11-08T03:27:08Z [load_day 2023-12-01] attempt 1 succeeded