orcaLog in

Runs · run 1595

load_day 2023-10-06, attempt 1 succeeded

Host
3c2b29713ce8:408457
Started
2026-11-08T01:31:19Z
Ended
2026-11-08T01:35:15Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1595/load_day/2023-10-06/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:31:19Z [load_day 2023-10-06] attempt 1 started
2026-11-08T01:31:21Z [load_day 2023-10-06] 2023-10-06: 2 hours in flight, about 1200 MB
2026-11-08T01:31:41Z [load_day 2023-10-06] 2023-10-06 hour 0: 158534 events in 4 files, snapshot 6155282940411783347 (task 15806 ms: download 6871 ms overlapping gunzip 1016 ms, parse 4603 ms, sort 135 ms, write 8578 ms, index 602 ms: 10 candidates, 83090 repo-hours, 935 releases; waiting 445 ms; commit 2579 ms)
2026-11-08T01:31:44Z [load_day 2023-10-06] 2023-10-06 hour 1: 147347 events in 3 files, snapshot 7933808012217709487 (task 14509 ms: download 5898 ms overlapping gunzip 771 ms, parse 3536 ms, sort 105 ms, write 8604 ms, index 736 ms: 7 candidates, 80198 repo-hours, 706 releases; waiting 480 ms; commit 3100 ms)
2026-11-08T01:31:55Z [load_day 2023-10-06] 2023-10-06 hour 2: 128392 events in 3 files, snapshot 973996823394545860 (task 11752 ms: download 4993 ms overlapping gunzip 720 ms, parse 3021 ms, sort 104 ms, write 6710 ms, index 389 ms: 9 candidates, 52359 repo-hours, 690 releases; waiting 513 ms; commit 3807 ms)
2026-11-08T01:31:57Z [load_day 2023-10-06] 2023-10-06 hour 3: 127464 events in 3 files, snapshot 7354074927956940253 (task 10946 ms: download 2589 ms overlapping gunzip 634 ms, parse 2695 ms, sort 76 ms, write 6220 ms, index 481 ms: 11 candidates, 52096 repo-hours, 1120 releases; waiting 541 ms; commit 2709 ms)
2026-11-08T01:32:07Z [load_day 2023-10-06] 2023-10-06 hour 4: 129761 events in 3 files, snapshot 360407966175018957 (task 12204 ms: download 4613 ms overlapping gunzip 678 ms, parse 2987 ms, sort 79 ms, write 7048 ms, index 573 ms: 9 candidates, 53789 repo-hours, 637 releases; waiting 527 ms; commit 2487 ms)
2026-11-08T01:32:12Z [load_day 2023-10-06] 2023-10-06 hour 5: 138377 events in 3 files, snapshot 4196880066000882800 (task 13573 ms: download 5567 ms overlapping gunzip 788 ms, parse 3496 ms, sort 120 ms, write 7549 ms, index 869 ms: 7 candidates, 57571 repo-hours, 681 releases; waiting 486 ms; commit 2656 ms)
2026-11-08T01:32:25Z [load_day 2023-10-06] 2023-10-06 hour 6: 164836 events in 4 files, snapshot 3120750698757671308 (task 17072 ms: download 6757 ms overlapping gunzip 961 ms, parse 4285 ms, sort 121 ms, write 9774 ms, index 566 ms: 7 candidates, 72308 repo-hours, 938 releases; waiting 580 ms; commit 2625 ms)
2026-11-08T01:32:33Z [load_day 2023-10-06] 2023-10-06 hour 7: 169515 events in 4 files, snapshot 9011737135789171806 (task 18492 ms: download 10653 ms overlapping gunzip 1034 ms, parse 4603 ms, sort 189 ms, write 11543 ms, index 537 ms: 11 candidates, 72961 repo-hours, 904 releases; waiting 302 ms; commit 4207 ms)
2026-11-08T01:32:45Z [load_day 2023-10-06] 2023-10-06 hour 8: 180024 events in 4 files, snapshot 5391638423553805210 (task 18684 ms: download 11085 ms overlapping gunzip 1147 ms, parse 4900 ms, sort 130 ms, write 11217 ms, index 558 ms: 9 candidates, 78842 repo-hours, 1094 releases; waiting 527 ms; commit 3053 ms)
2026-11-08T01:32:53Z [load_day 2023-10-06] 2023-10-06 hour 9: 183466 events in 4 files, snapshot 2880705936070479701 (task 19324 ms: download 11916 ms overlapping gunzip 1188 ms, parse 5459 ms, sort 122 ms, write 11108 ms, index 567 ms: 6 candidates, 79570 repo-hours, 1187 releases; waiting 574 ms; commit 2677 ms)
2026-11-08T01:33:07Z [load_day 2023-10-06] 2023-10-06 hour 10: 183569 events in 4 files, snapshot 107881135007232355 (task 15487 ms: download 8489 ms overlapping gunzip 964 ms, parse 4098 ms, sort 121 ms, write 8958 ms, index 562 ms: 5 candidates, 78722 repo-hours, 1130 releases; waiting 532 ms; commit 8665 ms)
2026-11-08T01:33:11Z [load_day 2023-10-06] 2023-10-06 hour 11: 183391 events in 4 files, snapshot 282409121902450303 (task 17417 ms: download 8351 ms overlapping gunzip 893 ms, parse 3974 ms, sort 133 ms, write 10885 ms, index 729 ms: 14 candidates, 78689 repo-hours, 1109 releases; waiting 531 ms; commit 2914 ms)
2026-11-08T01:33:27Z [load_day 2023-10-06] 2023-10-06 hour 12: 232358 events in 5 files, snapshot 3696838639361953782 (task 21424 ms: download 14322 ms overlapping gunzip 1176 ms, parse 5210 ms, sort 242 ms, write 13639 ms, index 733 ms: 10 candidates, 111159 repo-hours, 1126 releases; waiting 219 ms; commit 5249 ms)
2026-11-08T01:33:35Z [load_day 2023-10-06] 2023-10-06 hour 13: 214119 events in 5 files, snapshot 5468461054119768519 (task 22637 ms: download 14529 ms overlapping gunzip 1445 ms, parse 6603 ms, sort 201 ms, write 12756 ms, index 768 ms: 11 candidates, 106984 repo-hours, 1226 releases; waiting 520 ms; commit 2862 ms)
2026-11-08T01:33:48Z [load_day 2023-10-06] 2023-10-06 hour 14: 209259 events in 5 files, snapshot 4301240191390380357 (task 21245 ms: download 11380 ms overlapping gunzip 1087 ms, parse 4807 ms, sort 227 ms, write 13574 ms, index 765 ms: 11 candidates, 98339 repo-hours, 1171 releases; waiting 466 ms; commit 3761 ms)
2026-11-08T01:33:56Z [load_day 2023-10-06] 2023-10-06 hour 15: 188655 events in 4 files, snapshot 2741973757535434928 (task 19775 ms: download 11152 ms overlapping gunzip 1213 ms, parse 5681 ms, sort 150 ms, write 11322 ms, index 644 ms: 15 candidates, 86254 repo-hours, 1332 releases; waiting 518 ms; commit 2879 ms)
2026-11-08T01:34:08Z [load_day 2023-10-06] 2023-10-06 hour 16: 192281 events in 4 files, snapshot 5341938276593346407 (task 19577 ms: download 11083 ms overlapping gunzip 1245 ms, parse 5240 ms, sort 136 ms, write 11550 ms, index 601 ms: 13 candidates, 84439 repo-hours, 1090 releases; waiting 510 ms; commit 2424 ms)
2026-11-08T01:34:15Z [load_day 2023-10-06] 2023-10-06 hour 17: 182263 events in 4 files, snapshot 7104609884536810758 (task 18842 ms: download 10014 ms overlapping gunzip 1144 ms, parse 5147 ms, sort 136 ms, write 10969 ms, index 749 ms: 14 candidates, 76808 repo-hours, 1009 releases; waiting 480 ms; commit 2557 ms)
2026-11-08T01:34:28Z [load_day 2023-10-06] 2023-10-06 hour 18: 185551 events in 4 files, snapshot 5717839241642129307 (task 18913 ms: download 10457 ms overlapping gunzip 989 ms, parse 4226 ms, sort 228 ms, write 12005 ms, index 602 ms: 8 candidates, 83258 repo-hours, 1141 releases; waiting 519 ms; commit 2663 ms)
2026-11-08T01:34:35Z [load_day 2023-10-06] 2023-10-06 hour 19: 171193 events in 4 files, snapshot 5454113378742026024 (task 18455 ms: download 10549 ms overlapping gunzip 988 ms, parse 4382 ms, sort 156 ms, write 11116 ms, index 536 ms: 8 candidates, 70955 repo-hours, 975 releases; waiting 467 ms; commit 2702 ms)
2026-11-08T01:34:47Z [load_day 2023-10-06] 2023-10-06 hour 20: 163075 events in 4 files, snapshot 4716229879888934281 (task 17003 ms: download 6965 ms overlapping gunzip 826 ms, parse 3768 ms, sort 114 ms, write 10581 ms, index 531 ms: 11 candidates, 69355 repo-hours, 903 releases; waiting 770 ms; commit 3345 ms)
2026-11-08T01:34:53Z [load_day 2023-10-06] 2023-10-06 hour 21: 150048 events in 4 files, snapshot 8407260313639477186 (task 15891 ms: download 6955 ms overlapping gunzip 1045 ms, parse 4462 ms, sort 115 ms, write 8900 ms, index 563 ms: 10 candidates, 64999 repo-hours, 880 releases; waiting 531 ms; commit 3718 ms)
2026-11-08T01:35:02Z [load_day 2023-10-06] 2023-10-06 hour 22: 137439 events in 3 files, snapshot 2029938446199457938 (task 14579 ms: download 6229 ms overlapping gunzip 925 ms, parse 3529 ms, sort 179 ms, write 8763 ms, index 399 ms: 7 candidates, 55036 repo-hours, 853 releases; waiting 517 ms; commit 2488 ms)
2026-11-08T01:35:05Z [load_day 2023-10-06] 2023-10-06 hour 23: 118974 events in 3 files, snapshot 2244612186812950250 (task 11863 ms: download 5859 ms overlapping gunzip 867 ms, parse 3632 ms, sort 116 ms, write 6192 ms, index 326 ms: 2 candidates, 50131 repo-hours, 747 releases; waiting 495 ms; commit 2332 ms)
2026-11-08T01:35:13Z [load_day 2023-10-06] 2023-10-06: repo_hour merged from 24 files to 1 (1797912 rows, 8300 ms)
2026-11-08T01:35:15Z [load_day 2023-10-06] 2023-10-06: index committed in 10264 ms (24 + 1 + 24 files)
2026-11-08T01:35:15Z [load_day 2023-10-06] 2023-10-06: 4039891 events in 24 hours
2026-11-08T01:35:15Z [load_day 2023-10-06] attempt 1 succeeded