load_day 2024-01-04, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:15646

Started

2026-11-08T04:00:26Z

Ended

2026-11-08T04:02:08Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

668 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

157.1 s, up to 3.0 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1932/load_day/2024-01-04/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:00:26Z [load_day 2024-01-04] attempt 1 started
2026-11-08T04:00:27Z [load_day 2024-01-04] 2024-01-04: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:00:35Z [load_day 2024-01-04] 2024-01-04 hour 0: 203997 events in 0 files, snapshot none (task 7096 ms: download 4255 ms overlapping gunzip 1051 ms, parse 4570 ms, sort 0 ms, write 0 ms, index 684 ms: 4 candidates, 81892 repo-hours, 1043 releases; waiting 521 ms; commit 0 ms)
2026-11-08T04:00:35Z [load_day 2024-01-04] 2024-01-04 hour 1: 204315 events in 0 files, snapshot none (task 6867 ms: download 4160 ms overlapping gunzip 1044 ms, parse 4388 ms, sort 0 ms, write 0 ms, index 620 ms: 5 candidates, 90544 repo-hours, 967 releases; waiting 469 ms; commit 0 ms)
2026-11-08T04:00:42Z [load_day 2024-01-04] 2024-01-04 hour 2: 182305 events in 0 files, snapshot none (task 6697 ms: download 3456 ms overlapping gunzip 1007 ms, parse 4277 ms, sort 0 ms, write 0 ms, index 576 ms: 2 candidates, 60435 repo-hours, 798 releases; waiting 565 ms; commit 0 ms)
2026-11-08T04:00:42Z [load_day 2024-01-04] 2024-01-04 hour 3: 180565 events in 0 files, snapshot none (task 6489 ms: download 3394 ms overlapping gunzip 1004 ms, parse 4095 ms, sort 0 ms, write 0 ms, index 517 ms: 7 candidates, 56662 repo-hours, 917 releases; waiting 535 ms; commit 0 ms)
2026-11-08T04:00:48Z [load_day 2024-01-04] 2024-01-04 hour 4: 173077 events in 0 files, snapshot none (task 5374 ms: download 2557 ms overlapping gunzip 730 ms, parse 3288 ms, sort 0 ms, write 0 ms, index 523 ms: 5 candidates, 53101 repo-hours, 662 releases; waiting 493 ms; commit 0 ms)
2026-11-08T04:00:48Z [load_day 2024-01-04] 2024-01-04 hour 5: 182018 events in 0 files, snapshot none (task 5826 ms: download 2932 ms overlapping gunzip 917 ms, parse 3710 ms, sort 0 ms, write 0 ms, index 492 ms: 2 candidates, 57908 repo-hours, 730 releases; waiting 502 ms; commit 0 ms)
2026-11-08T04:00:54Z [load_day 2024-01-04] 2024-01-04 hour 6: 199531 events in 0 files, snapshot none (task 5457 ms: download 3195 ms overlapping gunzip 745 ms, parse 3292 ms, sort 0 ms, write 0 ms, index 549 ms: 7 candidates, 69938 repo-hours, 791 releases; waiting 535 ms; commit 0 ms)
2026-11-08T04:00:54Z [load_day 2024-01-04] 2024-01-04 hour 7: 196264 events in 0 files, snapshot none (task 5496 ms: download 3132 ms overlapping gunzip 815 ms, parse 3469 ms, sort 0 ms, write 0 ms, index 527 ms: 9 candidates, 68370 repo-hours, 838 releases; waiting 439 ms; commit 0 ms)
2026-11-08T04:01:01Z [load_day 2024-01-04] 2024-01-04 hour 8: 200555 events in 0 files, snapshot none (task 7107 ms: download 4609 ms overlapping gunzip 1096 ms, parse 4626 ms, sort 0 ms, write 0 ms, index 556 ms: 7 candidates, 76189 repo-hours, 925 releases; waiting 497 ms; commit 0 ms)
2026-11-08T04:01:02Z [load_day 2024-01-04] 2024-01-04 hour 9: 207223 events in 0 files, snapshot none (task 7042 ms: download 4613 ms overlapping gunzip 1087 ms, parse 4565 ms, sort 0 ms, write 0 ms, index 586 ms: 3 candidates, 79404 repo-hours, 1066 releases; waiting 494 ms; commit 0 ms)
2026-11-08T04:01:09Z [load_day 2024-01-04] 2024-01-04 hour 10: 209987 events in 0 files, snapshot none (task 6949 ms: download 4475 ms overlapping gunzip 1049 ms, parse 4459 ms, sort 0 ms, write 0 ms, index 566 ms: 3 candidates, 80109 repo-hours, 927 releases; waiting 547 ms; commit 0 ms)
2026-11-08T04:01:09Z [load_day 2024-01-04] 2024-01-04 hour 11: 205742 events in 0 files, snapshot none (task 6665 ms: download 4203 ms overlapping gunzip 1011 ms, parse 4284 ms, sort 0 ms, write 0 ms, index 562 ms: 4 candidates, 76117 repo-hours, 881 releases; waiting 529 ms; commit 0 ms)
2026-11-08T04:01:17Z [load_day 2024-01-04] 2024-01-04 hour 12: 253263 events in 0 files, snapshot none (task 7461 ms: download 4109 ms overlapping gunzip 1004 ms, parse 4485 ms, sort 0 ms, write 0 ms, index 1128 ms: 10 candidates, 108252 repo-hours, 1022 releases; waiting 513 ms; commit 0 ms)
2026-11-08T04:01:17Z [load_day 2024-01-04] 2024-01-04 hour 13: 219766 events in 0 files, snapshot none (task 7200 ms: download 3848 ms overlapping gunzip 999 ms, parse 4184 ms, sort 0 ms, write 0 ms, index 1250 ms: 7 candidates, 102499 repo-hours, 1255 releases; waiting 505 ms; commit 0 ms)
2026-11-08T04:01:26Z [load_day 2024-01-04] 2024-01-04 hour 14: 213614 events in 0 files, snapshot none (task 8775 ms: download 5950 ms overlapping gunzip 1450 ms, parse 5827 ms, sort 0 ms, write 0 ms, index 667 ms: 5 candidates, 89615 repo-hours, 1001 releases; waiting 519 ms; commit 0 ms)
2026-11-08T04:01:26Z [load_day 2024-01-04] 2024-01-04 hour 15: 217098 events in 0 files, snapshot none (task 8358 ms: download 5489 ms overlapping gunzip 1335 ms, parse 5430 ms, sort 0 ms, write 0 ms, index 748 ms: 7 candidates, 91949 repo-hours, 1309 releases; waiting 530 ms; commit 0 ms)
2026-11-08T04:01:35Z [load_day 2024-01-04] 2024-01-04 hour 16: 210930 events in 0 files, snapshot none (task 8501 ms: download 5876 ms overlapping gunzip 1285 ms, parse 5277 ms, sort 0 ms, write 0 ms, index 672 ms: 7 candidates, 91463 repo-hours, 1189 releases; waiting 889 ms; commit 0 ms)
2026-11-08T04:01:35Z [load_day 2024-01-04] 2024-01-04 hour 17: 196852 events in 0 files, snapshot none (task 7892 ms: download 5307 ms overlapping gunzip 1194 ms, parse 5269 ms, sort 0 ms, write 0 ms, index 593 ms: 8 candidates, 82146 repo-hours, 1081 releases; waiting 551 ms; commit 0 ms)
2026-11-08T04:01:42Z [load_day 2024-01-04] 2024-01-04 hour 18: 197001 events in 0 files, snapshot none (task 6650 ms: download 4178 ms overlapping gunzip 1015 ms, parse 4171 ms, sort 0 ms, write 0 ms, index 667 ms: 6 candidates, 84427 repo-hours, 1076 releases; waiting 502 ms; commit 0 ms)
2026-11-08T04:01:42Z [load_day 2024-01-04] 2024-01-04 hour 19: 178803 events in 0 files, snapshot none (task 7114 ms: download 4424 ms overlapping gunzip 983 ms, parse 4078 ms, sort 0 ms, write 0 ms, index 679 ms: 4 candidates, 71924 repo-hours, 953 releases; waiting 622 ms; commit 0 ms)
2026-11-08T04:01:50Z [load_day 2024-01-04] 2024-01-04 hour 20: 175704 events in 0 files, snapshot none (task 7237 ms: download 4584 ms overlapping gunzip 1102 ms, parse 4825 ms, sort 0 ms, write 0 ms, index 854 ms: 9 candidates, 71095 repo-hours, 1042 releases; waiting 209 ms; commit 0 ms)
2026-11-08T04:01:50Z [load_day 2024-01-04] 2024-01-04 hour 21: 175908 events in 0 files, snapshot none (task 6402 ms: download 4195 ms overlapping gunzip 996 ms, parse 4155 ms, sort 0 ms, write 0 ms, index 517 ms: 7 candidates, 64897 repo-hours, 1000 releases; waiting 512 ms; commit 0 ms)
2026-11-08T04:01:57Z [load_day 2024-01-04] 2024-01-04 hour 22: 186065 events in 0 files, snapshot none (task 6480 ms: download 3533 ms overlapping gunzip 875 ms, parse 3625 ms, sort 0 ms, write 0 ms, index 741 ms: 5 candidates, 59471 repo-hours, 945 releases; waiting 566 ms; commit 0 ms)
2026-11-08T04:01:57Z [load_day 2024-01-04] 2024-01-04 hour 23: 173082 events in 0 files, snapshot none (task 5239 ms: download 3859 ms overlapping gunzip 794 ms, parse 3317 ms, sort 0 ms, write 0 ms, index 394 ms: 6 candidates, 55342 repo-hours, 871 releases; waiting 384 ms; commit 0 ms)
2026-11-08T04:02:05Z [load_day 2024-01-04] 2024-01-04: repo_hour merged from 24 files to 1 (1823749 rows, 8104 ms)
2026-11-08T04:02:08Z [load_day 2024-01-04] 2024-01-04: index committed in 10234 ms (24 + 1 + 24 files)
2026-11-08T04:02:08Z [load_day 2024-01-04] 2024-01-04: 4743665 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:02:08Z [load_day 2024-01-04] attempt 1 succeeded