load_day 2023-12-28, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:13466

Started

2026-11-08T03:55:32Z

Ended

2026-11-08T03:56:56Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

601 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

123.8 s, up to 3.1 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1905/load_day/2023-12-28/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T03:55:32Z [load_day 2023-12-28] attempt 1 started
2026-11-08T03:55:33Z [load_day 2023-12-28] 2023-12-28: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T03:55:40Z [load_day 2023-12-28] 2023-12-28 hour 0: 164620 events in 0 files, snapshot none (task 5683 ms: download 2822 ms overlapping gunzip 883 ms, parse 3810 ms, sort 0 ms, write 0 ms, index 527 ms: 7 candidates, 78600 repo-hours, 948 releases; waiting 182 ms; commit 0 ms)
2026-11-08T03:55:40Z [load_day 2023-12-28] 2023-12-28 hour 1: 157409 events in 0 files, snapshot none (task 5424 ms: download 2833 ms overlapping gunzip 776 ms, parse 3186 ms, sort 0 ms, write 0 ms, index 633 ms: 4 candidates, 77691 repo-hours, 775 releases; waiting 505 ms; commit 0 ms)
2026-11-08T03:55:45Z [load_day 2023-12-28] 2023-12-28 hour 2: 138240 events in 0 files, snapshot none (task 4958 ms: download 2421 ms overlapping gunzip 711 ms, parse 2908 ms, sort 0 ms, write 0 ms, index 490 ms: 5 candidates, 51970 repo-hours, 711 releases; waiting 538 ms; commit 0 ms)
2026-11-08T03:55:45Z [load_day 2023-12-28] 2023-12-28 hour 3: 140000 events in 0 files, snapshot none (task 5273 ms: download 2406 ms overlapping gunzip 748 ms, parse 3295 ms, sort 0 ms, write 0 ms, index 471 ms: 5 candidates, 51082 repo-hours, 835 releases; waiting 507 ms; commit 0 ms)
2026-11-08T03:55:50Z [load_day 2023-12-28] 2023-12-28 hour 4: 135413 events in 0 files, snapshot none (task 4418 ms: download 2468 ms overlapping gunzip 600 ms, parse 2424 ms, sort 0 ms, write 0 ms, index 546 ms: 9 candidates, 47409 repo-hours, 583 releases; waiting 540 ms; commit 0 ms)
2026-11-08T03:55:51Z [load_day 2023-12-28] 2023-12-28 hour 5: 149386 events in 0 files, snapshot none (task 4727 ms: download 2575 ms overlapping gunzip 655 ms, parse 2683 ms, sort 0 ms, write 0 ms, index 623 ms: 4 candidates, 54560 repo-hours, 628 releases; waiting 476 ms; commit 0 ms)
2026-11-08T03:55:56Z [load_day 2023-12-28] 2023-12-28 hour 6: 164039 events in 0 files, snapshot none (task 5321 ms: download 3452 ms overlapping gunzip 807 ms, parse 3223 ms, sort 0 ms, write 0 ms, index 508 ms: 2 candidates, 65661 repo-hours, 770 releases; waiting 488 ms; commit 0 ms)
2026-11-08T03:55:56Z [load_day 2023-12-28] 2023-12-28 hour 7: 162264 events in 0 files, snapshot none (task 5350 ms: download 3201 ms overlapping gunzip 771 ms, parse 3220 ms, sort 0 ms, write 0 ms, index 478 ms: 5 candidates, 61336 repo-hours, 687 releases; waiting 513 ms; commit 0 ms)
2026-11-08T03:56:03Z [load_day 2023-12-28] 2023-12-28 hour 8: 169920 events in 0 files, snapshot none (task 6045 ms: download 3521 ms overlapping gunzip 923 ms, parse 3809 ms, sort 0 ms, write 0 ms, index 472 ms: 8 candidates, 66515 repo-hours, 953 releases; waiting 497 ms; commit 0 ms)
2026-11-08T03:56:03Z [load_day 2023-12-28] 2023-12-28 hour 9: 176218 events in 0 files, snapshot none (task 6160 ms: download 3313 ms overlapping gunzip 846 ms, parse 3564 ms, sort 0 ms, write 0 ms, index 501 ms: 7 candidates, 67050 repo-hours, 1122 releases; waiting 124 ms; commit 0 ms)
2026-11-08T03:56:09Z [load_day 2023-12-28] 2023-12-28 hour 10: 174510 events in 0 files, snapshot none (task 5706 ms: download 4088 ms overlapping gunzip 836 ms, parse 3653 ms, sort 0 ms, write 0 ms, index 488 ms: 4 candidates, 65830 repo-hours, 958 releases; waiting 480 ms; commit 0 ms)
2026-11-08T03:56:09Z [load_day 2023-12-28] 2023-12-28 hour 11: 168265 events in 0 files, snapshot none (task 5528 ms: download 3140 ms overlapping gunzip 818 ms, parse 3465 ms, sort 0 ms, write 0 ms, index 450 ms: 6 candidates, 63290 repo-hours, 945 releases; waiting 481 ms; commit 0 ms)
2026-11-08T03:56:16Z [load_day 2023-12-28] 2023-12-28 hour 12: 232828 events in 0 files, snapshot none (task 6845 ms: download 4080 ms overlapping gunzip 1017 ms, parse 4469 ms, sort 0 ms, write 0 ms, index 817 ms: 6 candidates, 96988 repo-hours, 1003 releases; waiting 328 ms; commit 0 ms)
2026-11-08T03:56:16Z [load_day 2023-12-28] 2023-12-28 hour 13: 195954 events in 0 files, snapshot none (task 6662 ms: download 4109 ms overlapping gunzip 942 ms, parse 4283 ms, sort 0 ms, write 0 ms, index 650 ms: 8 candidates, 88020 repo-hours, 933 releases; waiting 488 ms; commit 0 ms)
2026-11-08T03:56:22Z [load_day 2023-12-28] 2023-12-28 hour 14: 191565 events in 0 files, snapshot none (task 5944 ms: download 4506 ms overlapping gunzip 811 ms, parse 3426 ms, sort 0 ms, write 0 ms, index 546 ms: 5 candidates, 73191 repo-hours, 1121 releases; waiting 795 ms; commit 0 ms)
2026-11-08T03:56:22Z [load_day 2023-12-28] 2023-12-28 hour 15: 203164 events in 0 files, snapshot none (task 5759 ms: download 3111 ms overlapping gunzip 809 ms, parse 3342 ms, sort 0 ms, write 0 ms, index 573 ms: 6 candidates, 74750 repo-hours, 1266 releases; waiting 539 ms; commit 0 ms)
2026-11-08T03:56:29Z [load_day 2023-12-28] 2023-12-28 hour 16: 199166 events in 0 files, snapshot none (task 6118 ms: download 3134 ms overlapping gunzip 861 ms, parse 3726 ms, sort 0 ms, write 0 ms, index 763 ms: 1 candidates, 71784 repo-hours, 1021 releases; waiting 505 ms; commit 0 ms)
2026-11-08T03:56:29Z [load_day 2023-12-28] 2023-12-28 hour 17: 189036 events in 0 files, snapshot none (task 5769 ms: download 2711 ms overlapping gunzip 833 ms, parse 3438 ms, sort 0 ms, write 0 ms, index 700 ms: 5 candidates, 67039 repo-hours, 987 releases; waiting 515 ms; commit 0 ms)
2026-11-08T03:56:36Z [load_day 2023-12-28] 2023-12-28 hour 18: 190155 events in 0 files, snapshot none (task 6213 ms: download 3363 ms overlapping gunzip 859 ms, parse 3788 ms, sort 0 ms, write 0 ms, index 526 ms: 4 candidates, 68535 repo-hours, 1026 releases; waiting 725 ms; commit 0 ms)
2026-11-08T03:56:36Z [load_day 2023-12-28] 2023-12-28 hour 19: 171982 events in 0 files, snapshot none (task 5659 ms: download 2586 ms overlapping gunzip 839 ms, parse 3632 ms, sort 0 ms, write 0 ms, index 482 ms: 7 candidates, 56432 repo-hours, 968 releases; waiting 494 ms; commit 0 ms)
2026-11-08T03:56:42Z [load_day 2023-12-28] 2023-12-28 hour 20: 171199 events in 0 files, snapshot none (task 5737 ms: download 3455 ms overlapping gunzip 882 ms, parse 3651 ms, sort 0 ms, write 0 ms, index 392 ms: 7 candidates, 55701 repo-hours, 893 releases; waiting 529 ms; commit 0 ms)
2026-11-08T03:56:42Z [load_day 2023-12-28] 2023-12-28 hour 21: 165739 events in 0 files, snapshot none (task 5500 ms: download 3253 ms overlapping gunzip 831 ms, parse 3479 ms, sort 0 ms, write 0 ms, index 396 ms: 6 candidates, 52903 repo-hours, 986 releases; waiting 530 ms; commit 0 ms)
2026-11-08T03:56:47Z [load_day 2023-12-28] 2023-12-28 hour 22: 153624 events in 0 files, snapshot none (task 4954 ms: download 2946 ms overlapping gunzip 742 ms, parse 3027 ms, sort 0 ms, write 0 ms, index 372 ms: 9 candidates, 49996 repo-hours, 1136 releases; waiting 540 ms; commit 0 ms)
2026-11-08T03:56:47Z [load_day 2023-12-28] 2023-12-28 hour 23: 141423 events in 0 files, snapshot none (task 4783 ms: download 2732 ms overlapping gunzip 712 ms, parse 2957 ms, sort 0 ms, write 0 ms, index 346 ms: 10 candidates, 45135 repo-hours, 868 releases; waiting 523 ms; commit 0 ms)
2026-11-08T03:56:54Z [load_day 2023-12-28] 2023-12-28: repo_hour merged from 24 files to 1 (1551468 rows, 6791 ms)
2026-11-08T03:56:56Z [load_day 2023-12-28] 2023-12-28: index committed in 9082 ms (24 + 1 + 24 files)
2026-11-08T03:56:56Z [load_day 2023-12-28] 2023-12-28: 4106119 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T03:56:56Z [load_day 2023-12-28] attempt 1 succeeded