orcaLog in

Runs · run 3250

load_day 2024-11-27, attempt 1 succeeded

Host
73ccef5e07c2:213483
Started
2026-11-08T09:04:14Z
Ended
2026-11-08T09:06:38Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
814 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
240.6 s, up to 2.7 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3250/load_day/2024-11-27/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T09:04:14Z [load_day 2024-11-27] attempt 1 started
2026-11-08T09:04:15Z [load_day 2024-11-27] 2024-11-27: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T09:04:24Z [load_day 2024-11-27] 2024-11-27 hour 0: 192231 events in 0 files, snapshot none (task 8491 ms: download 5254 ms overlapping gunzip 1245 ms, parse 5580 ms, sort 0 ms, write 0 ms, index 847 ms: 9 candidates, 89656 repo-hours, 1210 releases; waiting 527 ms; commit 0 ms)
2026-11-08T09:04:25Z [load_day 2024-11-27] 2024-11-27 hour 1: 217218 events in 0 files, snapshot none (task 9669 ms: download 5942 ms overlapping gunzip 1578 ms, parse 6411 ms, sort 0 ms, write 0 ms, index 953 ms: 7 candidates, 114436 repo-hours, 939 releases; waiting 407 ms; commit 0 ms)
2026-11-08T09:04:33Z [load_day 2024-11-27] 2024-11-27 hour 2: 186662 events in 0 files, snapshot none (task 8156 ms: download 4540 ms overlapping gunzip 1204 ms, parse 4888 ms, sort 0 ms, write 0 ms, index 1213 ms: 4 candidates, 81398 repo-hours, 879 releases; waiting 552 ms; commit 0 ms)
2026-11-08T09:04:35Z [load_day 2024-11-27] 2024-11-27 hour 3: 220099 events in 0 files, snapshot none (task 9822 ms: download 6872 ms overlapping gunzip 1661 ms, parse 6597 ms, sort 0 ms, write 0 ms, index 793 ms: 1 candidates, 81663 repo-hours, 3967 releases; waiting 483 ms; commit 0 ms)
2026-11-08T09:04:43Z [load_day 2024-11-27] 2024-11-27 hour 4: 201983 events in 0 files, snapshot none (task 9652 ms: download 6783 ms overlapping gunzip 1608 ms, parse 6166 ms, sort 0 ms, write 0 ms, index 586 ms: 0 candidates, 67263 repo-hours, 686 releases; waiting 836 ms; commit 0 ms)
2026-11-08T09:04:46Z [load_day 2024-11-27] 2024-11-27 hour 5: 211971 events in 0 files, snapshot none (task 9800 ms: download 7060 ms overlapping gunzip 1733 ms, parse 6302 ms, sort 0 ms, write 0 ms, index 856 ms: 2 candidates, 71351 repo-hours, 841 releases; waiting 562 ms; commit 0 ms)
2026-11-08T09:04:54Z [load_day 2024-11-27] 2024-11-27 hour 6: 230901 events in 0 files, snapshot none (task 10341 ms: download 7042 ms overlapping gunzip 1635 ms, parse 6641 ms, sort 0 ms, write 0 ms, index 926 ms: 5 candidates, 93429 repo-hours, 999 releases; waiting 455 ms; commit 0 ms)
2026-11-08T09:04:56Z [load_day 2024-11-27] 2024-11-27 hour 7: 231842 events in 0 files, snapshot none (task 9513 ms: download 7025 ms overlapping gunzip 1684 ms, parse 6390 ms, sort 0 ms, write 0 ms, index 739 ms: 3 candidates, 83728 repo-hours, 941 releases; waiting 434 ms; commit 0 ms)
2026-11-08T09:05:05Z [load_day 2024-11-27] 2024-11-27 hour 8: 244504 events in 0 files, snapshot none (task 10931 ms: download 8285 ms overlapping gunzip 1820 ms, parse 7370 ms, sort 0 ms, write 0 ms, index 943 ms: 7 candidates, 93581 repo-hours, 1039 releases; waiting 530 ms; commit 0 ms)
2026-11-08T09:05:08Z [load_day 2024-11-27] 2024-11-27 hour 9: 259126 events in 0 files, snapshot none (task 11629 ms: download 8895 ms overlapping gunzip 2076 ms, parse 7862 ms, sort 0 ms, write 0 ms, index 889 ms: 7 candidates, 101467 repo-hours, 1237 releases; waiting 507 ms; commit 0 ms)
2026-11-08T09:05:17Z [load_day 2024-11-27] 2024-11-27 hour 10: 254326 events in 0 files, snapshot none (task 11026 ms: download 8594 ms overlapping gunzip 1873 ms, parse 6978 ms, sort 0 ms, write 0 ms, index 727 ms: 8 candidates, 97418 repo-hours, 1208 releases; waiting 1130 ms; commit 0 ms)
2026-11-08T09:05:18Z [load_day 2024-11-27] 2024-11-27 hour 11: 253105 events in 0 files, snapshot none (task 9992 ms: download 7493 ms overlapping gunzip 1805 ms, parse 6907 ms, sort 0 ms, write 0 ms, index 676 ms: 6 candidates, 92579 repo-hours, 1162 releases; waiting 243 ms; commit 0 ms)
2026-11-08T09:05:26Z [load_day 2024-11-27] 2024-11-27 hour 12: 277029 events in 0 files, snapshot none (task 8784 ms: download 5776 ms overlapping gunzip 1304 ms, parse 5496 ms, sort 0 ms, write 0 ms, index 1081 ms: 6 candidates, 136410 repo-hours, 1332 releases; waiting 546 ms; commit 0 ms)
2026-11-08T09:05:29Z [load_day 2024-11-27] 2024-11-27 hour 13: 280762 events in 0 files, snapshot none (task 9793 ms: download 7145 ms overlapping gunzip 1717 ms, parse 6370 ms, sort 0 ms, write 0 ms, index 874 ms: 7 candidates, 118292 repo-hours, 1189 releases; waiting 499 ms; commit 0 ms)
2026-11-08T09:05:37Z [load_day 2024-11-27] 2024-11-27 hour 14: 270805 events in 0 files, snapshot none (task 11081 ms: download 7881 ms overlapping gunzip 1915 ms, parse 7695 ms, sort 0 ms, write 0 ms, index 958 ms: 3 candidates, 106054 repo-hours, 1433 releases; waiting 233 ms; commit 0 ms)
2026-11-08T09:05:42Z [load_day 2024-11-27] 2024-11-27 hour 15: 276746 events in 0 files, snapshot none (task 12718 ms: download 9589 ms overlapping gunzip 2216 ms, parse 8513 ms, sort 0 ms, write 0 ms, index 1122 ms: 11 candidates, 110757 repo-hours, 1474 releases; waiting 503 ms; commit 0 ms)
2026-11-08T09:05:52Z [load_day 2024-11-27] 2024-11-27 hour 16: 258429 events in 0 files, snapshot none (task 14444 ms: download 10961 ms overlapping gunzip 2322 ms, parse 9505 ms, sort 0 ms, write 0 ms, index 1117 ms: 6 candidates, 101640 repo-hours, 1239 releases; waiting 1184 ms; commit 0 ms)
2026-11-08T09:05:54Z [load_day 2024-11-27] 2024-11-27 hour 17: 256241 events in 0 files, snapshot none (task 11616 ms: download 8845 ms overlapping gunzip 2025 ms, parse 7994 ms, sort 0 ms, write 0 ms, index 837 ms: 12 candidates, 95422 repo-hours, 1019 releases; waiting 452 ms; commit 0 ms)
2026-11-08T09:06:04Z [load_day 2024-11-27] 2024-11-27 hour 18: 254715 events in 0 files, snapshot none (task 11278 ms: download 7676 ms overlapping gunzip 1939 ms, parse 7447 ms, sort 0 ms, write 0 ms, index 997 ms: 5 candidates, 103045 repo-hours, 1109 releases; waiting 507 ms; commit 0 ms)
2026-11-08T09:06:05Z [load_day 2024-11-27] 2024-11-27 hour 19: 236790 events in 0 files, snapshot none (task 10541 ms: download 7950 ms overlapping gunzip 1886 ms, parse 7179 ms, sort 0 ms, write 0 ms, index 674 ms: 3 candidates, 82127 repo-hours, 974 releases; waiting 511 ms; commit 0 ms)
2026-11-08T09:06:15Z [load_day 2024-11-27] 2024-11-27 hour 20: 230440 events in 0 files, snapshot none (task 10360 ms: download 7083 ms overlapping gunzip 1794 ms, parse 7034 ms, sort 0 ms, write 0 ms, index 796 ms: 3 candidates, 80442 repo-hours, 972 releases; waiting 345 ms; commit 0 ms)
2026-11-08T09:06:16Z [load_day 2024-11-27] 2024-11-27 hour 21: 223648 events in 0 files, snapshot none (task 10392 ms: download 7591 ms overlapping gunzip 1880 ms, parse 7161 ms, sort 0 ms, write 0 ms, index 654 ms: 6 candidates, 79680 repo-hours, 1127 releases; waiting 490 ms; commit 0 ms)
2026-11-08T09:06:26Z [load_day 2024-11-27] 2024-11-27 hour 22: 210367 events in 0 files, snapshot none (task 10103 ms: download 7246 ms overlapping gunzip 1731 ms, parse 6672 ms, sort 0 ms, write 0 ms, index 825 ms: 5 candidates, 69913 repo-hours, 1087 releases; waiting 488 ms; commit 0 ms)
2026-11-08T09:06:26Z [load_day 2024-11-27] 2024-11-27 hour 23: 198542 events in 0 files, snapshot none (task 9305 ms: download 6313 ms overlapping gunzip 1673 ms, parse 6328 ms, sort 0 ms, write 0 ms, index 773 ms: 2 candidates, 64664 repo-hours, 896 releases; waiting 182 ms; commit 0 ms)
2026-11-08T09:06:35Z [load_day 2024-11-27] 2024-11-27: repo_hour merged from 24 files to 1 (2216415 rows, 9631 ms)
2026-11-08T09:06:38Z [load_day 2024-11-27] 2024-11-27: index committed in 12252 ms (23 + 1 + 24 files)
2026-11-08T09:06:38Z [load_day 2024-11-27] 2024-11-27: 5678482 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T09:06:38Z [load_day 2024-11-27] attempt 1 succeeded