load_day 2024-01-26, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:72285

Started

2026-11-08T04:20:39Z

Ended

2026-11-08T04:22:34Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

687 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

166.1 s, up to 2.9 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2010/load_day/2024-01-26/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:20:39Z [load_day 2024-01-26] attempt 1 started
2026-11-08T04:20:41Z [load_day 2024-01-26] 2024-01-26: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:20:49Z [load_day 2024-01-26] 2024-01-26 hour 0: 203085 events in 0 files, snapshot none (task 8049 ms: download 4759 ms overlapping gunzip 1229 ms, parse 5326 ms, sort 0 ms, write 0 ms, index 727 ms: 12 candidates, 89924 repo-hours, 965 releases; waiting 473 ms; commit 0 ms)
2026-11-08T04:20:49Z [load_day 2024-01-26] 2024-01-26 hour 1: 201625 events in 0 files, snapshot none (task 7964 ms: download 4286 ms overlapping gunzip 1121 ms, parse 4730 ms, sort 0 ms, write 0 ms, index 1272 ms: 3 candidates, 95081 repo-hours, 934 releases; waiting 547 ms; commit 0 ms)
2026-11-08T04:20:56Z [load_day 2024-01-26] 2024-01-26 hour 2: 178374 events in 0 files, snapshot none (task 6033 ms: download 3447 ms overlapping gunzip 869 ms, parse 3742 ms, sort 0 ms, write 0 ms, index 589 ms: 5 candidates, 66321 repo-hours, 743 releases; waiting 527 ms; commit 0 ms)
2026-11-08T04:20:56Z [load_day 2024-01-26] 2024-01-26 hour 3: 167021 events in 0 files, snapshot none (task 5460 ms: download 3043 ms overlapping gunzip 789 ms, parse 3376 ms, sort 0 ms, write 0 ms, index 449 ms: 8 candidates, 62517 repo-hours, 1016 releases; waiting 513 ms; commit 0 ms)
2026-11-08T04:21:01Z [load_day 2024-01-26] 2024-01-26 hour 4: 163894 events in 0 files, snapshot none (task 4953 ms: download 2597 ms overlapping gunzip 741 ms, parse 3023 ms, sort 0 ms, write 0 ms, index 401 ms: 3 candidates, 57126 repo-hours, 662 releases; waiting 505 ms; commit 0 ms)
2026-11-08T04:21:02Z [load_day 2024-01-26] 2024-01-26 hour 5: 177675 events in 0 files, snapshot none (task 5244 ms: download 3555 ms overlapping gunzip 817 ms, parse 3185 ms, sort 0 ms, write 0 ms, index 443 ms: 7 candidates, 62251 repo-hours, 619 releases; waiting 503 ms; commit 0 ms)
2026-11-08T04:21:08Z [load_day 2024-01-26] 2024-01-26 hour 6: 192007 events in 0 files, snapshot none (task 5655 ms: download 3041 ms overlapping gunzip 812 ms, parse 3436 ms, sort 0 ms, write 0 ms, index 594 ms: 6 candidates, 73656 repo-hours, 796 releases; waiting 536 ms; commit 0 ms)
2026-11-08T04:21:08Z [load_day 2024-01-26] 2024-01-26 hour 7: 201530 events in 0 files, snapshot none (task 5974 ms: download 3283 ms overlapping gunzip 901 ms, parse 3613 ms, sort 0 ms, write 0 ms, index 567 ms: 10 candidates, 72516 repo-hours, 789 releases; waiting 512 ms; commit 0 ms)
2026-11-08T04:21:18Z [load_day 2024-01-26] 2024-01-26 hour 8: 212982 events in 0 files, snapshot none (task 9506 ms: download 6268 ms overlapping gunzip 1433 ms, parse 6092 ms, sort 0 ms, write 0 ms, index 1053 ms: 9 candidates, 81333 repo-hours, 915 releases; waiting 584 ms; commit 0 ms)
2026-11-08T04:21:19Z [load_day 2024-01-26] 2024-01-26 hour 9: 224390 events in 0 files, snapshot none (task 10004 ms: download 6397 ms overlapping gunzip 1661 ms, parse 6712 ms, sort 0 ms, write 0 ms, index 736 ms: 9 candidates, 87025 repo-hours, 1190 releases; waiting 489 ms; commit 0 ms)
2026-11-08T04:21:26Z [load_day 2024-01-26] 2024-01-26 hour 10: 215989 events in 0 files, snapshot none (task 8391 ms: download 5596 ms overlapping gunzip 1319 ms, parse 5462 ms, sort 0 ms, write 0 ms, index 740 ms: 10 candidates, 80743 repo-hours, 1079 releases; waiting 512 ms; commit 0 ms)
2026-11-08T04:21:28Z [load_day 2024-01-26] 2024-01-26 hour 11: 196268 events in 0 files, snapshot none (task 9331 ms: download 6091 ms overlapping gunzip 1409 ms, parse 6139 ms, sort 0 ms, write 0 ms, index 1012 ms: 5 candidates, 78259 repo-hours, 1038 releases; waiting 486 ms; commit 0 ms)
2026-11-08T04:21:37Z [load_day 2024-01-26] 2024-01-26 hour 12: 260761 events in 0 files, snapshot none (task 10360 ms: download 5847 ms overlapping gunzip 1609 ms, parse 7087 ms, sort 0 ms, write 0 ms, index 889 ms: 9 candidates, 108914 repo-hours, 1208 releases; waiting 453 ms; commit 0 ms)
2026-11-08T04:21:38Z [load_day 2024-01-26] 2024-01-26 hour 13: 226997 events in 0 files, snapshot none (task 9251 ms: download 6434 ms overlapping gunzip 1456 ms, parse 6194 ms, sort 0 ms, write 0 ms, index 759 ms: 10 candidates, 105013 repo-hours, 1235 releases; waiting 586 ms; commit 0 ms)
2026-11-08T04:21:48Z [load_day 2024-01-26] 2024-01-26 hour 14: 220580 events in 0 files, snapshot none (task 10534 ms: download 7258 ms overlapping gunzip 1665 ms, parse 7209 ms, sort 0 ms, write 0 ms, index 944 ms: 12 candidates, 93232 repo-hours, 1191 releases; waiting 487 ms; commit 0 ms)
2026-11-08T04:21:48Z [load_day 2024-01-26] 2024-01-26 hour 15: 224277 events in 0 files, snapshot none (task 9868 ms: download 6490 ms overlapping gunzip 1530 ms, parse 6565 ms, sort 0 ms, write 0 ms, index 997 ms: 9 candidates, 95497 repo-hours, 1381 releases; waiting 502 ms; commit 0 ms)
2026-11-08T04:21:58Z [load_day 2024-01-26] 2024-01-26 hour 16: 217446 events in 0 files, snapshot none (task 9349 ms: download 6519 ms overlapping gunzip 1494 ms, parse 6321 ms, sort 0 ms, write 0 ms, index 705 ms: 8 candidates, 91210 repo-hours, 1135 releases; waiting 550 ms; commit 0 ms)
2026-11-08T04:21:58Z [load_day 2024-01-26] 2024-01-26 hour 17: 205349 events in 0 files, snapshot none (task 8993 ms: download 6466 ms overlapping gunzip 1458 ms, parse 6077 ms, sort 0 ms, write 0 ms, index 619 ms: 3 candidates, 83303 repo-hours, 1201 releases; waiting 537 ms; commit 0 ms)
2026-11-08T04:22:07Z [load_day 2024-01-26] 2024-01-26 hour 18: 204489 events in 0 files, snapshot none (task 8705 ms: download 5538 ms overlapping gunzip 1414 ms, parse 5606 ms, sort 0 ms, write 0 ms, index 724 ms: 11 candidates, 86181 repo-hours, 1098 releases; waiting 483 ms; commit 0 ms)
2026-11-08T04:22:07Z [load_day 2024-01-26] 2024-01-26 hour 19: 208313 events in 0 files, snapshot none (task 8621 ms: download 5665 ms overlapping gunzip 1331 ms, parse 5787 ms, sort 0 ms, write 0 ms, index 630 ms: 8 candidates, 77441 repo-hours, 966 releases; waiting 560 ms; commit 0 ms)
2026-11-08T04:22:16Z [load_day 2024-01-26] 2024-01-26 hour 20: 210142 events in 0 files, snapshot none (task 7812 ms: download 4851 ms overlapping gunzip 1205 ms, parse 4856 ms, sort 0 ms, write 0 ms, index 917 ms: 11 candidates, 72016 repo-hours, 946 releases; waiting 520 ms; commit 0 ms)
2026-11-08T04:22:16Z [load_day 2024-01-26] 2024-01-26 hour 21: 196002 events in 0 files, snapshot none (task 7952 ms: download 5155 ms overlapping gunzip 1289 ms, parse 5193 ms, sort 0 ms, write 0 ms, index 674 ms: 12 candidates, 65710 repo-hours, 1007 releases; waiting 498 ms; commit 0 ms)
2026-11-08T04:22:23Z [load_day 2024-01-26] 2024-01-26 hour 22: 153505 events in 0 files, snapshot none (task 6975 ms: download 4543 ms overlapping gunzip 1157 ms, parse 4496 ms, sort 0 ms, write 0 ms, index 478 ms: 11 candidates, 59779 repo-hours, 1719 releases; waiting 528 ms; commit 0 ms)
2026-11-08T04:22:23Z [load_day 2024-01-26] 2024-01-26 hour 23: 141364 events in 0 files, snapshot none (task 6229 ms: download 3685 ms overlapping gunzip 978 ms, parse 4059 ms, sort 0 ms, write 0 ms, index 384 ms: 9 candidates, 54900 repo-hours, 745 releases; waiting 499 ms; commit 0 ms)
2026-11-08T04:22:32Z [load_day 2024-01-26] 2024-01-26: repo_hour merged from 24 files to 1 (1899948 rows, 8308 ms)
2026-11-08T04:22:34Z [load_day 2024-01-26] 2024-01-26: index committed in 11028 ms (24 + 1 + 24 files)
2026-11-08T04:22:34Z [load_day 2024-01-26] 2024-01-26: 4804065 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:22:34Z [load_day 2024-01-26] attempt 1 succeeded