load_day 2024-01-27, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:73169

Started

2026-11-08T04:21:06Z

Ended

2026-11-08T04:22:32Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

606 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

110.6 s, up to 2.2 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2014/load_day/2024-01-27/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:21:06Z [load_day 2024-01-27] attempt 1 started
2026-11-08T04:21:07Z [load_day 2024-01-27] 2024-01-27: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:21:14Z [load_day 2024-01-27] 2024-01-27 hour 0: 165774 events in 0 files, snapshot none (task 6504 ms: download 3325 ms overlapping gunzip 895 ms, parse 3972 ms, sort 0 ms, write 0 ms, index 812 ms: 8 candidates, 84031 repo-hours, 1089 releases; waiting 494 ms; commit 0 ms)
2026-11-08T04:21:15Z [load_day 2024-01-27] 2024-01-27 hour 1: 147178 events in 0 files, snapshot none (task 6778 ms: download 3579 ms overlapping gunzip 819 ms, parse 3923 ms, sort 0 ms, write 0 ms, index 985 ms: 7 candidates, 81539 repo-hours, 825 releases; waiting 480 ms; commit 0 ms)
2026-11-08T04:21:19Z [load_day 2024-01-27] 2024-01-27 hour 2: 116232 events in 0 files, snapshot none (task 4030 ms: download 1705 ms overlapping gunzip 641 ms, parse 2551 ms, sort 0 ms, write 0 ms, index 385 ms: 3 candidates, 47181 repo-hours, 655 releases; waiting 179 ms; commit 0 ms)
2026-11-08T04:21:19Z [load_day 2024-01-27] 2024-01-27 hour 3: 112589 events in 0 files, snapshot none (task 3806 ms: download 1977 ms overlapping gunzip 493 ms, parse 2102 ms, sort 0 ms, write 0 ms, index 336 ms: 4 candidates, 45503 repo-hours, 744 releases; waiting 614 ms; commit 0 ms)
2026-11-08T04:21:24Z [load_day 2024-01-27] 2024-01-27 hour 4: 124325 events in 0 files, snapshot none (task 5092 ms: download 2050 ms overlapping gunzip 677 ms, parse 3151 ms, sort 0 ms, write 0 ms, index 483 ms: 2 candidates, 45074 repo-hours, 547 releases; waiting 464 ms; commit 0 ms)
2026-11-08T04:21:25Z [load_day 2024-01-27] 2024-01-27 hour 5: 147470 events in 0 files, snapshot none (task 5775 ms: download 2983 ms overlapping gunzip 888 ms, parse 3496 ms, sort 0 ms, write 0 ms, index 483 ms: 3 candidates, 47922 repo-hours, 501 releases; waiting 591 ms; commit 0 ms)
2026-11-08T04:21:30Z [load_day 2024-01-27] 2024-01-27 hour 6: 160410 events in 0 files, snapshot none (task 5232 ms: download 2382 ms overlapping gunzip 687 ms, parse 2953 ms, sort 0 ms, write 0 ms, index 769 ms: 5 candidates, 57957 repo-hours, 872 releases; waiting 516 ms; commit 0 ms)
2026-11-08T04:21:31Z [load_day 2024-01-27] 2024-01-27 hour 7: 150631 events in 0 files, snapshot none (task 5529 ms: download 1820 ms overlapping gunzip 787 ms, parse 3408 ms, sort 0 ms, write 0 ms, index 874 ms: 2 candidates, 50773 repo-hours, 727 releases; waiting 188 ms; commit 0 ms)
2026-11-08T04:21:37Z [load_day 2024-01-27] 2024-01-27 hour 8: 163588 events in 0 files, snapshot none (task 6196 ms: download 3106 ms overlapping gunzip 936 ms, parse 4125 ms, sort 0 ms, write 0 ms, index 418 ms: 7 candidates, 54194 repo-hours, 794 releases; waiting 351 ms; commit 0 ms)
2026-11-08T04:21:38Z [load_day 2024-01-27] 2024-01-27 hour 9: 166635 events in 0 files, snapshot none (task 5785 ms: download 3085 ms overlapping gunzip 765 ms, parse 3578 ms, sort 0 ms, write 0 ms, index 480 ms: 6 candidates, 56766 repo-hours, 863 releases; waiting 480 ms; commit 0 ms)
2026-11-08T04:21:43Z [load_day 2024-01-27] 2024-01-27 hour 10: 165578 events in 0 files, snapshot none (task 5576 ms: download 2893 ms overlapping gunzip 758 ms, parse 3412 ms, sort 0 ms, write 0 ms, index 543 ms: 8 candidates, 58379 repo-hours, 585 releases; waiting 572 ms; commit 0 ms)
2026-11-08T04:21:44Z [load_day 2024-01-27] 2024-01-27 hour 11: 172974 events in 0 files, snapshot none (task 6061 ms: download 3160 ms overlapping gunzip 867 ms, parse 3746 ms, sort 0 ms, write 0 ms, index 642 ms: 2 candidates, 57488 repo-hours, 681 releases; waiting 528 ms; commit 0 ms)
2026-11-08T04:21:50Z [load_day 2024-01-27] 2024-01-27 hour 12: 251624 events in 0 files, snapshot none (task 6338 ms: download 2913 ms overlapping gunzip 819 ms, parse 3991 ms, sort 0 ms, write 0 ms, index 663 ms: 7 candidates, 90699 repo-hours, 802 releases; waiting 536 ms; commit 0 ms)
2026-11-08T04:21:51Z [load_day 2024-01-27] 2024-01-27 hour 13: 196555 events in 0 files, snapshot none (task 5866 ms: download 2923 ms overlapping gunzip 729 ms, parse 3304 ms, sort 0 ms, write 0 ms, index 747 ms: 4 candidates, 81547 repo-hours, 751 releases; waiting 516 ms; commit 0 ms)
2026-11-08T04:21:57Z [load_day 2024-01-27] 2024-01-27 hour 14: 185486 events in 0 files, snapshot none (task 7137 ms: download 3788 ms overlapping gunzip 973 ms, parse 4580 ms, sort 0 ms, write 0 ms, index 722 ms: 6 candidates, 67998 repo-hours, 745 releases; waiting 525 ms; commit 0 ms)
2026-11-08T04:21:58Z [load_day 2024-01-27] 2024-01-27 hour 15: 189506 events in 0 files, snapshot none (task 6464 ms: download 3284 ms overlapping gunzip 912 ms, parse 4069 ms, sort 0 ms, write 0 ms, index 672 ms: 5 candidates, 70033 repo-hours, 986 releases; waiting 448 ms; commit 0 ms)
2026-11-08T04:22:04Z [load_day 2024-01-27] 2024-01-27 hour 16: 186048 events in 0 files, snapshot none (task 6062 ms: download 2676 ms overlapping gunzip 899 ms, parse 3806 ms, sort 0 ms, write 0 ms, index 762 ms: 5 candidates, 70641 repo-hours, 941 releases; waiting 314 ms; commit 0 ms)
2026-11-08T04:22:06Z [load_day 2024-01-27] 2024-01-27 hour 17: 186256 events in 0 files, snapshot none (task 7701 ms: download 4271 ms overlapping gunzip 1088 ms, parse 4711 ms, sort 0 ms, write 0 ms, index 1055 ms: 10 candidates, 67733 repo-hours, 850 releases; waiting 549 ms; commit 0 ms)
2026-11-08T04:22:09Z [load_day 2024-01-27] 2024-01-27 hour 18: 190927 events in 0 files, snapshot none (task 5260 ms: download 2492 ms overlapping gunzip 681 ms, parse 2940 ms, sort 0 ms, write 0 ms, index 800 ms: 9 candidates, 73339 repo-hours, 899 releases; waiting 529 ms; commit 0 ms)
2026-11-08T04:22:13Z [load_day 2024-01-27] 2024-01-27 hour 19: 177211 events in 0 files, snapshot none (task 6394 ms: download 3287 ms overlapping gunzip 913 ms, parse 3910 ms, sort 0 ms, write 0 ms, index 713 ms: 10 candidates, 62382 repo-hours, 855 releases; waiting 532 ms; commit 0 ms)
2026-11-08T04:22:16Z [load_day 2024-01-27] 2024-01-27 hour 20: 170248 events in 0 files, snapshot none (task 5596 ms: download 2733 ms overlapping gunzip 767 ms, parse 3440 ms, sort 0 ms, write 0 ms, index 544 ms: 7 candidates, 60078 repo-hours, 836 releases; waiting 521 ms; commit 0 ms)
2026-11-08T04:22:18Z [load_day 2024-01-27] 2024-01-27 hour 21: 153860 events in 0 files, snapshot none (task 4677 ms: download 2318 ms overlapping gunzip 606 ms, parse 2668 ms, sort 0 ms, write 0 ms, index 831 ms: 7 candidates, 53215 repo-hours, 865 releases; waiting 266 ms; commit 0 ms)
2026-11-08T04:22:21Z [load_day 2024-01-27] 2024-01-27 hour 22: 145502 events in 0 files, snapshot none (task 4859 ms: download 2330 ms overlapping gunzip 569 ms, parse 2766 ms, sort 0 ms, write 0 ms, index 649 ms: 9 candidates, 49249 repo-hours, 871 releases; waiting 503 ms; commit 0 ms)
2026-11-08T04:22:23Z [load_day 2024-01-27] 2024-01-27 hour 23: 139002 events in 0 files, snapshot none (task 3967 ms: download 2274 ms overlapping gunzip 542 ms, parse 2301 ms, sort 0 ms, write 0 ms, index 360 ms: 4 candidates, 47473 repo-hours, 649 releases; waiting 456 ms; commit 0 ms)
2026-11-08T04:22:30Z [load_day 2024-01-27] 2024-01-27: repo_hour merged from 24 files to 1 (1481194 rows, 6666 ms)
2026-11-08T04:22:32Z [load_day 2024-01-27] 2024-01-27: index committed in 8882 ms (24 + 1 + 24 files)
2026-11-08T04:22:32Z [load_day 2024-01-27] 2024-01-27: 3965609 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:22:32Z [load_day 2024-01-27] attempt 1 succeeded