orcaLog in

Runs · run 1813

load_day 2023-12-04, attempt 1 succeeded

Host
3c2b29713ce8:628747
Started
2026-11-08T03:27:42Z
Ended
2026-11-08T03:32:31Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1813/load_day/2023-12-04/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:27:42Z [load_day 2023-12-04] attempt 1 started
2026-11-08T03:27:44Z [load_day 2023-12-04] 2023-12-04: 2 hours in flight, about 1200 MB
2026-11-08T03:28:07Z [load_day 2023-12-04] 2023-12-04 hour 0: 322317 events in 7 files, snapshot 527669444424954596 (task 16372 ms: download 8972 ms overlapping gunzip 845 ms, parse 4414 ms, sort 323 ms, write 9652 ms, index 342 ms: 6 candidates, 54136 repo-hours, 533 releases; waiting 518 ms; commit 2659 ms)
2026-11-08T03:28:12Z [load_day 2023-12-04] 2023-12-04 hour 1: 317341 events in 7 files, snapshot 687250659110794014 (task 16523 ms: download 9121 ms overlapping gunzip 944 ms, parse 4640 ms, sort 340 ms, write 9500 ms, index 362 ms: 1 candidates, 54298 repo-hours, 419 releases; waiting 486 ms; commit 5182 ms)
2026-11-08T03:28:27Z [load_day 2023-12-04] 2023-12-04 hour 2: 324549 events in 7 files, snapshot 8007258874905290311 (task 16763 ms: download 8959 ms overlapping gunzip 952 ms, parse 4605 ms, sort 333 ms, write 9494 ms, index 572 ms: 5 candidates, 39465 repo-hours, 383 releases; waiting 514 ms; commit 5096 ms)
2026-11-08T03:28:33Z [load_day 2023-12-04] 2023-12-04 hour 3: 322351 events in 7 files, snapshot 1633520243902505948 (task 16074 ms: download 9455 ms overlapping gunzip 1072 ms, parse 5186 ms, sort 412 ms, write 8274 ms, index 345 ms: 3 candidates, 38698 repo-hours, 359 releases; waiting 518 ms; commit 6270 ms)
2026-11-08T03:28:44Z [load_day 2023-12-04] 2023-12-04 hour 4: 322869 events in 7 files, snapshot 3217335437184582546 (task 17552 ms: download 9624 ms overlapping gunzip 988 ms, parse 5451 ms, sort 482 ms, write 9269 ms, index 324 ms: 3 candidates, 38120 repo-hours, 304 releases; waiting 565 ms; commit 3624 ms)
2026-11-08T03:28:51Z [load_day 2023-12-04] 2023-12-04 hour 5: 321315 events in 7 files, snapshot 2467760870655321105 (task 18830 ms: download 9409 ms overlapping gunzip 1139 ms, parse 5604 ms, sort 501 ms, write 10483 ms, index 383 ms: 5 candidates, 42041 repo-hours, 347 releases; waiting 484 ms; commit 3486 ms)
2026-11-08T03:29:04Z [load_day 2023-12-04] 2023-12-04 hour 6: 318648 events in 7 files, snapshot 8983908731685587060 (task 17232 ms: download 9334 ms overlapping gunzip 1016 ms, parse 4994 ms, sort 282 ms, write 9703 ms, index 423 ms: 5 candidates, 48143 repo-hours, 398 releases; waiting 517 ms; commit 4790 ms)
2026-11-08T03:29:13Z [load_day 2023-12-04] 2023-12-04 hour 7: 314529 events in 7 files, snapshot 4122222548650466161 (task 20819 ms: download 12815 ms overlapping gunzip 1368 ms, parse 6943 ms, sort 385 ms, write 11173 ms, index 475 ms: 4 candidates, 59952 repo-hours, 606 releases; waiting 191 ms; commit 3417 ms)
2026-11-08T03:29:24Z [load_day 2023-12-04] 2023-12-04 hour 8: 314397 events in 7 files, snapshot 9212888453504300043 (task 19914 ms: download 12384 ms overlapping gunzip 1206 ms, parse 5254 ms, sort 344 ms, write 11813 ms, index 537 ms: 8 candidates, 58815 repo-hours, 519 releases; waiting 497 ms; commit 2844 ms)
2026-11-08T03:29:38Z [load_day 2023-12-04] 2023-12-04 hour 9: 302215 events in 7 files, snapshot 6793642978051202158 (task 19532 ms: download 12566 ms overlapping gunzip 1258 ms, parse 5669 ms, sort 310 ms, write 11134 ms, index 468 ms: 5 candidates, 60418 repo-hours, 622 releases; waiting 473 ms; commit 7755 ms)
2026-11-08T03:29:45Z [load_day 2023-12-04] 2023-12-04 hour 10: 309684 events in 7 files, snapshot 1896837997301851872 (task 20705 ms: download 12921 ms overlapping gunzip 1250 ms, parse 5636 ms, sort 323 ms, write 12273 ms, index 399 ms: 7 candidates, 62647 repo-hours, 612 releases; waiting 559 ms; commit 2856 ms)
2026-11-08T03:29:57Z [load_day 2023-12-04] 2023-12-04 hour 11: 312769 events in 7 files, snapshot 8736106344224958958 (task 22325 ms: download 14358 ms overlapping gunzip 1357 ms, parse 6232 ms, sort 296 ms, write 13133 ms, index 501 ms: 2 candidates, 63723 repo-hours, 750 releases; waiting 502 ms; commit 3336 ms)
2026-11-08T03:30:10Z [load_day 2023-12-04] 2023-12-04 hour 12: 316568 events in 7 files, snapshot 2662121898052568541 (task 19512 ms: download 12311 ms overlapping gunzip 1207 ms, parse 5686 ms, sort 286 ms, write 10867 ms, index 683 ms: 4 candidates, 77035 repo-hours, 570 releases; waiting 513 ms; commit 6294 ms)
2026-11-08T03:30:19Z [load_day 2023-12-04] 2023-12-04 hour 13: 314126 events in 7 files, snapshot 3319101889391388273 (task 20897 ms: download 12641 ms overlapping gunzip 1168 ms, parse 5791 ms, sort 257 ms, write 12228 ms, index 613 ms: 7 candidates, 66329 repo-hours, 537 releases; waiting 548 ms; commit 3149 ms)
2026-11-08T03:30:28Z [load_day 2023-12-04] 2023-12-04 hour 14: 311868 events in 7 files, snapshot 1885791057006569770 (task 20218 ms: download 13982 ms overlapping gunzip 1234 ms, parse 6046 ms, sort 257 ms, write 11535 ms, index 507 ms: 6 candidates, 66453 repo-hours, 578 releases; waiting 421 ms; commit 3036 ms)
2026-11-08T03:30:42Z [load_day 2023-12-04] 2023-12-04 hour 15: 304764 events in 7 files, snapshot 6896831276377445741 (task 20541 ms: download 13734 ms overlapping gunzip 1268 ms, parse 5546 ms, sort 246 ms, write 12067 ms, index 594 ms: 9 candidates, 84915 repo-hours, 1006 releases; waiting 561 ms; commit 3949 ms)
2026-11-08T03:30:55Z [load_day 2023-12-04] 2023-12-04 hour 16: 294460 events in 6 files, snapshot 3704235616211734654 (task 25741 ms: download 16979 ms overlapping gunzip 1564 ms, parse 6477 ms, sort 263 ms, write 15707 ms, index 926 ms: 5 candidates, 91239 repo-hours, 990 releases; waiting 531 ms; commit 3303 ms)
2026-11-08T03:31:09Z [load_day 2023-12-04] 2023-12-04 hour 17: 293340 events in 6 files, snapshot 1737118919146178385 (task 26732 ms: download 18783 ms overlapping gunzip 1730 ms, parse 7530 ms, sort 348 ms, write 15484 ms, index 813 ms: 13 candidates, 86429 repo-hours, 946 releases; waiting 542 ms; commit 3019 ms)
2026-11-08T03:31:22Z [load_day 2023-12-04] 2023-12-04 hour 18: 294960 events in 6 files, snapshot 3013711348157749378 (task 25349 ms: download 17728 ms overlapping gunzip 1489 ms, parse 6811 ms, sort 286 ms, write 15171 ms, index 747 ms: 9 candidates, 90215 repo-hours, 1015 releases; waiting 529 ms; commit 3461 ms)
2026-11-08T03:31:35Z [load_day 2023-12-04] 2023-12-04 hour 19: 291837 events in 6 files, snapshot 3128151430729091966 (task 24688 ms: download 17146 ms overlapping gunzip 1464 ms, parse 6529 ms, sort 271 ms, write 15248 ms, index 609 ms: 10 candidates, 81301 repo-hours, 926 releases; waiting 259 ms; commit 3176 ms)
2026-11-08T03:31:47Z [load_day 2023-12-04] 2023-12-04 hour 20: 285147 events in 6 files, snapshot 5056141812548197465 (task 24559 ms: download 17682 ms overlapping gunzip 1509 ms, parse 6460 ms, sort 356 ms, write 14802 ms, index 643 ms: 13 candidates, 80958 repo-hours, 959 releases; waiting 545 ms; commit 2949 ms)
2026-11-08T03:32:03Z [load_day 2023-12-04] 2023-12-04 hour 21: 285354 events in 6 files, snapshot 5410593594416085021 (task 25031 ms: download 18051 ms overlapping gunzip 1547 ms, parse 6874 ms, sort 303 ms, write 14869 ms, index 642 ms: 11 candidates, 75225 repo-hours, 985 releases; waiting 583 ms; commit 5228 ms)
2026-11-08T03:32:11Z [load_day 2023-12-04] 2023-12-04 hour 22: 241992 events in 5 files, snapshot 7502802713408493135 (task 22484 ms: download 13636 ms overlapping gunzip 1392 ms, parse 5743 ms, sort 206 ms, write 13674 ms, index 652 ms: 17 candidates, 70697 repo-hours, 1085 releases; waiting 501 ms; commit 3004 ms)
2026-11-08T03:32:21Z [load_day 2023-12-04] 2023-12-04 hour 23: 182214 events in 4 files, snapshot 1260533353803108281 (task 18923 ms: download 11676 ms overlapping gunzip 1295 ms, parse 5227 ms, sort 124 ms, write 10941 ms, index 486 ms: 15 candidates, 67719 repo-hours, 771 releases; waiting 520 ms; commit 2514 ms)
2026-11-08T03:32:28Z [load_day 2023-12-04] 2023-12-04: repo_hour merged from 24 files to 1 (1558971 rows, 7724 ms)
2026-11-08T03:32:31Z [load_day 2023-12-04] 2023-12-04: index committed in 10192 ms (24 + 1 + 24 files)
2026-11-08T03:32:31Z [load_day 2023-12-04] 2023-12-04: 7219614 events in 24 hours
2026-11-08T03:32:31Z [load_day 2023-12-04] attempt 1 succeeded