load_day 2023-12-29, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:13895

Started

2026-11-08T03:56:24Z

Ended

2026-11-08T03:57:47Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

568 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

120.8 s, up to 3.0 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1909/load_day/2023-12-29/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T03:56:24Z [load_day 2023-12-29] attempt 1 started
2026-11-08T03:56:25Z [load_day 2023-12-29] 2023-12-29: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T03:56:31Z [load_day 2023-12-29] 2023-12-29 hour 0: 178531 events in 0 files, snapshot none (task 5135 ms: download 2686 ms overlapping gunzip 684 ms, parse 2981 ms, sort 0 ms, write 0 ms, index 648 ms: 8 candidates, 85327 repo-hours, 1076 releases; waiting 518 ms; commit 0 ms)
2026-11-08T03:56:31Z [load_day 2023-12-29] 2023-12-29 hour 1: 164967 events in 0 files, snapshot none (task 5235 ms: download 2735 ms overlapping gunzip 711 ms, parse 3168 ms, sort 0 ms, write 0 ms, index 563 ms: 5 candidates, 75700 repo-hours, 749 releases; waiting 467 ms; commit 0 ms)
2026-11-08T03:56:36Z [load_day 2023-12-29] 2023-12-29 hour 2: 157194 events in 0 files, snapshot none (task 4709 ms: download 2292 ms overlapping gunzip 706 ms, parse 2825 ms, sort 0 ms, write 0 ms, index 440 ms: 2 candidates, 50585 repo-hours, 892 releases; waiting 484 ms; commit 0 ms)
2026-11-08T03:56:36Z [load_day 2023-12-29] 2023-12-29 hour 3: 148297 events in 0 files, snapshot none (task 4632 ms: download 2256 ms overlapping gunzip 642 ms, parse 2823 ms, sort 0 ms, write 0 ms, index 383 ms: 5 candidates, 50116 repo-hours, 941 releases; waiting 465 ms; commit 0 ms)
2026-11-08T03:56:41Z [load_day 2023-12-29] 2023-12-29 hour 4: 144926 events in 0 files, snapshot none (task 4723 ms: download 2667 ms overlapping gunzip 686 ms, parse 2850 ms, sort 0 ms, write 0 ms, index 385 ms: 4 candidates, 46852 repo-hours, 690 releases; waiting 488 ms; commit 0 ms)
2026-11-08T03:56:41Z [load_day 2023-12-29] 2023-12-29 hour 5: 152227 events in 0 files, snapshot none (task 4899 ms: download 2588 ms overlapping gunzip 731 ms, parse 2981 ms, sort 0 ms, write 0 ms, index 353 ms: 3 candidates, 51071 repo-hours, 802 releases; waiting 506 ms; commit 0 ms)
2026-11-08T03:56:47Z [load_day 2023-12-29] 2023-12-29 hour 6: 173290 events in 0 files, snapshot none (task 5399 ms: download 2918 ms overlapping gunzip 792 ms, parse 3291 ms, sort 0 ms, write 0 ms, index 504 ms: 4 candidates, 63957 repo-hours, 735 releases; waiting 519 ms; commit 0 ms)
2026-11-08T03:56:47Z [load_day 2023-12-29] 2023-12-29 hour 7: 165555 events in 0 files, snapshot none (task 5237 ms: download 2847 ms overlapping gunzip 788 ms, parse 3289 ms, sort 0 ms, write 0 ms, index 402 ms: 3 candidates, 59328 repo-hours, 714 releases; waiting 527 ms; commit 0 ms)
2026-11-08T03:56:52Z [load_day 2023-12-29] 2023-12-29 hour 8: 163442 events in 0 files, snapshot none (task 4567 ms: download 2728 ms overlapping gunzip 644 ms, parse 2655 ms, sort 0 ms, write 0 ms, index 479 ms: 8 candidates, 62882 repo-hours, 818 releases; waiting 500 ms; commit 0 ms)
2026-11-08T03:56:52Z [load_day 2023-12-29] 2023-12-29 hour 9: 170122 events in 0 files, snapshot none (task 4891 ms: download 2587 ms overlapping gunzip 689 ms, parse 2990 ms, sort 0 ms, write 0 ms, index 464 ms: 5 candidates, 65277 repo-hours, 1061 releases; waiting 529 ms; commit 0 ms)
2026-11-08T03:56:57Z [load_day 2023-12-29] 2023-12-29 hour 10: 166812 events in 0 files, snapshot none (task 4947 ms: download 2866 ms overlapping gunzip 707 ms, parse 2965 ms, sort 0 ms, write 0 ms, index 438 ms: 6 candidates, 63503 repo-hours, 879 releases; waiting 518 ms; commit 0 ms)
2026-11-08T03:56:59Z [load_day 2023-12-29] 2023-12-29 hour 11: 237528 events in 0 files, snapshot none (task 5858 ms: download 3521 ms overlapping gunzip 729 ms, parse 3426 ms, sort 0 ms, write 0 ms, index 402 ms: 11 candidates, 59971 repo-hours, 762 releases; waiting 1018 ms; commit 0 ms)
2026-11-08T03:57:07Z [load_day 2023-12-29] 2023-12-29 hour 12: 322915 events in 0 files, snapshot none (task 8818 ms: download 4910 ms overlapping gunzip 1014 ms, parse 5675 ms, sort 0 ms, write 0 ms, index 1289 ms: 8 candidates, 85061 repo-hours, 729 releases; waiting 513 ms; commit 0 ms)
2026-11-08T03:57:07Z [load_day 2023-12-29] 2023-12-29 hour 13: 262085 events in 0 files, snapshot none (task 7529 ms: download 4251 ms overlapping gunzip 966 ms, parse 4870 ms, sort 0 ms, write 0 ms, index 873 ms: 10 candidates, 80932 repo-hours, 869 releases; waiting 476 ms; commit 0 ms)
2026-11-08T03:57:14Z [load_day 2023-12-29] 2023-12-29 hour 14: 214457 events in 0 files, snapshot none (task 6922 ms: download 4388 ms overlapping gunzip 1087 ms, parse 4643 ms, sort 0 ms, write 0 ms, index 503 ms: 7 candidates, 69057 repo-hours, 898 releases; waiting 487 ms; commit 0 ms)
2026-11-08T03:57:14Z [load_day 2023-12-29] 2023-12-29 hour 15: 170643 events in 0 files, snapshot none (task 6145 ms: download 3406 ms overlapping gunzip 928 ms, parse 3872 ms, sort 0 ms, write 0 ms, index 518 ms: 8 candidates, 67624 repo-hours, 1053 releases; waiting 511 ms; commit 0 ms)
2026-11-08T03:57:21Z [load_day 2023-12-29] 2023-12-29 hour 16: 169612 events in 0 files, snapshot none (task 6221 ms: download 3237 ms overlapping gunzip 870 ms, parse 3633 ms, sort 0 ms, write 0 ms, index 899 ms: 9 candidates, 66279 repo-hours, 1074 releases; waiting 503 ms; commit 0 ms)
2026-11-08T03:57:21Z [load_day 2023-12-29] 2023-12-29 hour 17: 152201 events in 0 files, snapshot none (task 6072 ms: download 3544 ms overlapping gunzip 800 ms, parse 3509 ms, sort 0 ms, write 0 ms, index 885 ms: 5 candidates, 62112 repo-hours, 886 releases; waiting 501 ms; commit 0 ms)
2026-11-08T03:57:27Z [load_day 2023-12-29] 2023-12-29 hour 18: 172348 events in 0 files, snapshot none (task 5501 ms: download 2817 ms overlapping gunzip 752 ms, parse 3248 ms, sort 0 ms, write 0 ms, index 489 ms: 8 candidates, 67598 repo-hours, 961 releases; waiting 753 ms; commit 0 ms)
2026-11-08T03:57:27Z [load_day 2023-12-29] 2023-12-29 hour 19: 163100 events in 0 files, snapshot none (task 4869 ms: download 2836 ms overlapping gunzip 675 ms, parse 2900 ms, sort 0 ms, write 0 ms, index 407 ms: 5 candidates, 54309 repo-hours, 834 releases; waiting 538 ms; commit 0 ms)
2026-11-08T03:57:32Z [load_day 2023-12-29] 2023-12-29 hour 20: 155816 events in 0 files, snapshot none (task 5290 ms: download 2976 ms overlapping gunzip 773 ms, parse 3335 ms, sort 0 ms, write 0 ms, index 365 ms: 5 candidates, 52182 repo-hours, 703 releases; waiting 556 ms; commit 0 ms)
2026-11-08T03:57:32Z [load_day 2023-12-29] 2023-12-29 hour 21: 149214 events in 0 files, snapshot none (task 4944 ms: download 2589 ms overlapping gunzip 716 ms, parse 2984 ms, sort 0 ms, write 0 ms, index 386 ms: 9 candidates, 48633 repo-hours, 858 releases; waiting 556 ms; commit 0 ms)
2026-11-08T03:57:38Z [load_day 2023-12-29] 2023-12-29 hour 22: 145222 events in 0 files, snapshot none (task 5259 ms: download 2976 ms overlapping gunzip 778 ms, parse 3270 ms, sort 0 ms, write 0 ms, index 392 ms: 10 candidates, 45343 repo-hours, 829 releases; waiting 531 ms; commit 0 ms)
2026-11-08T03:57:38Z [load_day 2023-12-29] 2023-12-29 hour 23: 133529 events in 0 files, snapshot none (task 4249 ms: download 2410 ms overlapping gunzip 648 ms, parse 2536 ms, sort 0 ms, write 0 ms, index 301 ms: 1 candidates, 42137 repo-hours, 661 releases; waiting 501 ms; commit 0 ms)
2026-11-08T03:57:45Z [load_day 2023-12-29] 2023-12-29: repo_hour merged from 24 files to 1 (1475836 rows, 6517 ms)
2026-11-08T03:57:47Z [load_day 2023-12-29] 2023-12-29: index committed in 8857 ms (24 + 1 + 24 files)
2026-11-08T03:57:47Z [load_day 2023-12-29] 2023-12-29: 4234033 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T03:57:47Z [load_day 2023-12-29] attempt 1 succeeded