load_day 2024-01-28, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:75028

Started

2026-11-08T04:22:32Z

Ended

2026-11-08T04:24:00Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

596 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

112.9 s, up to 2.9 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2016/load_day/2024-01-28/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:22:32Z [load_day 2024-01-28] attempt 1 started
2026-11-08T04:22:33Z [load_day 2024-01-28] 2024-01-28: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:22:39Z [load_day 2024-01-28] 2024-01-28 hour 0: 164035 events in 0 files, snapshot none (task 4896 ms: download 2105 ms overlapping gunzip 637 ms, parse 2897 ms, sort 0 ms, write 0 ms, index 841 ms: 7 candidates, 76972 repo-hours, 1123 releases; waiting 310 ms; commit 0 ms)
2026-11-08T04:22:39Z [load_day 2024-01-28] 2024-01-28 hour 1: 164398 events in 0 files, snapshot none (task 4861 ms: download 2164 ms overlapping gunzip 603 ms, parse 2719 ms, sort 0 ms, write 0 ms, index 728 ms: 6 candidates, 79700 repo-hours, 745 releases; waiting 488 ms; commit 0 ms)
2026-11-08T04:22:44Z [load_day 2024-01-28] 2024-01-28 hour 2: 133726 events in 0 files, snapshot none (task 4356 ms: download 2739 ms overlapping gunzip 646 ms, parse 2623 ms, sort 0 ms, write 0 ms, index 315 ms: 4 candidates, 44010 repo-hours, 670 releases; waiting 468 ms; commit 0 ms)
2026-11-08T04:22:44Z [load_day 2024-01-28] 2024-01-28 hour 3: 129907 events in 0 files, snapshot none (task 3944 ms: download 2312 ms overlapping gunzip 491 ms, parse 2278 ms, sort 0 ms, write 0 ms, index 414 ms: 8 candidates, 42958 repo-hours, 623 releases; waiting 495 ms; commit 0 ms)
2026-11-08T04:22:49Z [load_day 2024-01-28] 2024-01-28 hour 4: 130134 events in 0 files, snapshot none (task 4665 ms: download 2230 ms overlapping gunzip 660 ms, parse 2862 ms, sort 0 ms, write 0 ms, index 359 ms: 4 candidates, 43441 repo-hours, 489 releases; waiting 555 ms; commit 0 ms)
2026-11-08T04:22:49Z [load_day 2024-01-28] 2024-01-28 hour 5: 139485 events in 0 files, snapshot none (task 4822 ms: download 2261 ms overlapping gunzip 746 ms, parse 3218 ms, sort 0 ms, write 0 ms, index 428 ms: 4 candidates, 46766 repo-hours, 549 releases; waiting 181 ms; commit 0 ms)
2026-11-08T04:22:54Z [load_day 2024-01-28] 2024-01-28 hour 6: 149148 events in 0 files, snapshot none (task 4712 ms: download 2795 ms overlapping gunzip 612 ms, parse 2771 ms, sort 0 ms, write 0 ms, index 531 ms: 3 candidates, 54630 repo-hours, 595 releases; waiting 557 ms; commit 0 ms)
2026-11-08T04:22:54Z [load_day 2024-01-28] 2024-01-28 hour 7: 145159 events in 0 files, snapshot none (task 4499 ms: download 2258 ms overlapping gunzip 593 ms, parse 2653 ms, sort 0 ms, write 0 ms, index 495 ms: 1 candidates, 48913 repo-hours, 513 releases; waiting 194 ms; commit 0 ms)
2026-11-08T04:22:59Z [load_day 2024-01-28] 2024-01-28 hour 8: 151437 events in 0 files, snapshot none (task 4422 ms: download 2265 ms overlapping gunzip 612 ms, parse 2602 ms, sort 0 ms, write 0 ms, index 379 ms: 3 candidates, 52946 repo-hours, 619 releases; waiting 532 ms; commit 0 ms)
2026-11-08T04:22:59Z [load_day 2024-01-28] 2024-01-28 hour 9: 157963 events in 0 files, snapshot none (task 4860 ms: download 2225 ms overlapping gunzip 654 ms, parse 2814 ms, sort 0 ms, write 0 ms, index 570 ms: 6 candidates, 55623 repo-hours, 828 releases; waiting 510 ms; commit 0 ms)
2026-11-08T04:23:04Z [load_day 2024-01-28] 2024-01-28 hour 10: 164234 events in 0 files, snapshot none (task 5028 ms: download 3095 ms overlapping gunzip 703 ms, parse 3079 ms, sort 0 ms, write 0 ms, index 459 ms: 6 candidates, 60556 repo-hours, 608 releases; waiting 486 ms; commit 0 ms)
2026-11-08T04:23:05Z [load_day 2024-01-28] 2024-01-28 hour 11: 168022 events in 0 files, snapshot none (task 5398 ms: download 2813 ms overlapping gunzip 741 ms, parse 3362 ms, sort 0 ms, write 0 ms, index 423 ms: 4 candidates, 59048 repo-hours, 687 releases; waiting 466 ms; commit 0 ms)
2026-11-08T04:23:14Z [load_day 2024-01-28] 2024-01-28 hour 12: 264689 events in 0 files, snapshot none (task 9230 ms: download 4424 ms overlapping gunzip 1238 ms, parse 5947 ms, sort 0 ms, write 0 ms, index 1136 ms: 6 candidates, 94012 repo-hours, 904 releases; waiting 508 ms; commit 0 ms)
2026-11-08T04:23:14Z [load_day 2024-01-28] 2024-01-28 hour 13: 209692 events in 0 files, snapshot none (task 7955 ms: download 3899 ms overlapping gunzip 1171 ms, parse 5178 ms, sort 0 ms, write 0 ms, index 883 ms: 7 candidates, 85615 repo-hours, 781 releases; waiting 339 ms; commit 0 ms)
2026-11-08T04:23:22Z [load_day 2024-01-28] 2024-01-28 hour 14: 194114 events in 0 files, snapshot none (task 7180 ms: download 3645 ms overlapping gunzip 1061 ms, parse 4397 ms, sort 0 ms, write 0 ms, index 960 ms: 1 candidates, 71306 repo-hours, 742 releases; waiting 504 ms; commit 0 ms)
2026-11-08T04:23:22Z [load_day 2024-01-28] 2024-01-28 hour 15: 199781 events in 0 files, snapshot none (task 7606 ms: download 3705 ms overlapping gunzip 1065 ms, parse 4766 ms, sort 0 ms, write 0 ms, index 985 ms: 3 candidates, 74095 repo-hours, 1037 releases; waiting 508 ms; commit 0 ms)
2026-11-08T04:23:28Z [load_day 2024-01-28] 2024-01-28 hour 16: 198326 events in 0 files, snapshot none (task 6172 ms: download 3446 ms overlapping gunzip 898 ms, parse 3830 ms, sort 0 ms, write 0 ms, index 586 ms: 6 candidates, 71053 repo-hours, 1005 releases; waiting 508 ms; commit 0 ms)
2026-11-08T04:23:28Z [load_day 2024-01-28] 2024-01-28 hour 17: 185695 events in 0 files, snapshot none (task 5604 ms: download 3033 ms overlapping gunzip 800 ms, parse 3503 ms, sort 0 ms, write 0 ms, index 620 ms: 5 candidates, 68959 repo-hours, 823 releases; waiting 465 ms; commit 0 ms)
2026-11-08T04:23:36Z [load_day 2024-01-28] 2024-01-28 hour 18: 203095 events in 0 files, snapshot none (task 7414 ms: download 4121 ms overlapping gunzip 1110 ms, parse 4802 ms, sort 0 ms, write 0 ms, index 717 ms: 11 candidates, 74651 repo-hours, 890 releases; waiting 473 ms; commit 0 ms)
2026-11-08T04:23:36Z [load_day 2024-01-28] 2024-01-28 hour 19: 192354 events in 0 files, snapshot none (task 7103 ms: download 3943 ms overlapping gunzip 1020 ms, parse 4512 ms, sort 0 ms, write 0 ms, index 803 ms: 8 candidates, 62392 repo-hours, 827 releases; waiting 502 ms; commit 0 ms)
2026-11-08T04:23:43Z [load_day 2024-01-28] 2024-01-28 hour 20: 192866 events in 0 files, snapshot none (task 6024 ms: download 3075 ms overlapping gunzip 807 ms, parse 3628 ms, sort 0 ms, write 0 ms, index 797 ms: 15 candidates, 63703 repo-hours, 882 releases; waiting 544 ms; commit 0 ms)
2026-11-08T04:23:43Z [load_day 2024-01-28] 2024-01-28 hour 21: 176582 events in 0 files, snapshot none (task 5621 ms: download 2724 ms overlapping gunzip 752 ms, parse 3160 ms, sort 0 ms, write 0 ms, index 835 ms: 6 candidates, 56982 repo-hours, 897 releases; waiting 521 ms; commit 0 ms)
2026-11-08T04:23:49Z [load_day 2024-01-28] 2024-01-28 hour 22: 166003 events in 0 files, snapshot none (task 5905 ms: download 2805 ms overlapping gunzip 803 ms, parse 3796 ms, sort 0 ms, write 0 ms, index 509 ms: 3 candidates, 52421 repo-hours, 847 releases; waiting 500 ms; commit 0 ms)
2026-11-08T04:23:50Z [load_day 2024-01-28] 2024-01-28 hour 23: 151641 events in 0 files, snapshot none (task 6162 ms: download 2987 ms overlapping gunzip 900 ms, parse 4035 ms, sort 0 ms, write 0 ms, index 431 ms: 6 candidates, 49373 repo-hours, 742 releases; waiting 514 ms; commit 0 ms)
2026-11-08T04:23:57Z [load_day 2024-01-28] 2024-01-28: repo_hour merged from 24 files to 1 (1490125 rows, 7864 ms)
2026-11-08T04:24:00Z [load_day 2024-01-28] 2024-01-28: index committed in 10468 ms (24 + 1 + 24 files)
2026-11-08T04:24:00Z [load_day 2024-01-28] 2024-01-28: 4132486 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:24:00Z [load_day 2024-01-28] attempt 1 succeeded