load_day 2024-10-07, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:188475

Started

2026-11-08T08:14:26Z

Ended

2026-11-08T08:16:23Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

804 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

181.4 s, up to 2.8 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/3051/load_day/2024-10-07/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T08:14:26Z [load_day 2024-10-07] attempt 1 started
2026-11-08T08:14:27Z [load_day 2024-10-07] 2024-10-07: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:14:34Z [load_day 2024-10-07] 2024-10-07 hour 0: 175507 events in 0 files, snapshot none (task 5890 ms: download 3667 ms overlapping gunzip 1011 ms, parse 3544 ms, sort 0 ms, write 0 ms, index 554 ms: 5 candidates, 80798 repo-hours, 1085 releases; waiting 530 ms; commit 0 ms)
2026-11-08T08:14:34Z [load_day 2024-10-07] 2024-10-07 hour 1: 192853 events in 0 files, snapshot none (task 6151 ms: download 3777 ms overlapping gunzip 969 ms, parse 3743 ms, sort 0 ms, write 0 ms, index 686 ms: 3 candidates, 100937 repo-hours, 793 releases; waiting 493 ms; commit 0 ms)
2026-11-08T08:14:40Z [load_day 2024-10-07] 2024-10-07 hour 2: 171032 events in 0 files, snapshot none (task 5830 ms: download 3563 ms overlapping gunzip 944 ms, parse 3577 ms, sort 0 ms, write 0 ms, index 546 ms: 17 candidates, 70332 repo-hours, 731 releases; waiting 499 ms; commit 0 ms)
2026-11-08T08:14:41Z [load_day 2024-10-07] 2024-10-07 hour 3: 195633 events in 0 files, snapshot none (task 6193 ms: download 3951 ms overlapping gunzip 987 ms, parse 3901 ms, sort 0 ms, write 0 ms, index 521 ms: 6 candidates, 71579 repo-hours, 3864 releases; waiting 506 ms; commit 0 ms)
2026-11-08T08:14:47Z [load_day 2024-10-07] 2024-10-07 hour 4: 180662 events in 0 files, snapshot none (task 6615 ms: download 4390 ms overlapping gunzip 1223 ms, parse 4131 ms, sort 0 ms, write 0 ms, index 471 ms: 4 candidates, 63833 repo-hours, 649 releases; waiting 534 ms; commit 0 ms)
2026-11-08T08:14:47Z [load_day 2024-10-07] 2024-10-07 hour 5: 189125 events in 0 files, snapshot none (task 6296 ms: download 3906 ms overlapping gunzip 1093 ms, parse 3843 ms, sort 0 ms, write 0 ms, index 602 ms: 3 candidates, 67595 repo-hours, 791 releases; waiting 515 ms; commit 0 ms)
2026-11-08T08:14:54Z [load_day 2024-10-07] 2024-10-07 hour 6: 221111 events in 0 files, snapshot none (task 6749 ms: download 4408 ms overlapping gunzip 1224 ms, parse 4111 ms, sort 0 ms, write 0 ms, index 643 ms: 3 candidates, 90440 repo-hours, 989 releases; waiting 512 ms; commit 0 ms)
2026-11-08T08:14:56Z [load_day 2024-10-07] 2024-10-07 hour 7: 230617 events in 0 files, snapshot none (task 8383 ms: download 6156 ms overlapping gunzip 1202 ms, parse 4327 ms, sort 0 ms, write 0 ms, index 724 ms: 7 candidates, 86040 repo-hours, 1028 releases; waiting 1918 ms; commit 0 ms)
2026-11-08T08:15:01Z [load_day 2024-10-07] 2024-10-07 hour 8: 240932 events in 0 files, snapshot none (task 6443 ms: download 4043 ms overlapping gunzip 1135 ms, parse 3940 ms, sort 0 ms, write 0 ms, index 750 ms: 8 candidates, 95814 repo-hours, 1100 releases; waiting 292 ms; commit 0 ms)
2026-11-08T08:15:04Z [load_day 2024-10-07] 2024-10-07 hour 9: 247435 events in 0 files, snapshot none (task 6840 ms: download 4483 ms overlapping gunzip 1150 ms, parse 4122 ms, sort 0 ms, write 0 ms, index 700 ms: 7 candidates, 98340 repo-hours, 1323 releases; waiting 557 ms; commit 0 ms)
2026-11-08T08:15:10Z [load_day 2024-10-07] 2024-10-07 hour 10: 240211 events in 0 files, snapshot none (task 8176 ms: download 5060 ms overlapping gunzip 1358 ms, parse 5017 ms, sort 0 ms, write 0 ms, index 969 ms: 6 candidates, 93691 repo-hours, 1079 releases; waiting 525 ms; commit 0 ms)
2026-11-08T08:15:12Z [load_day 2024-10-07] 2024-10-07 hour 11: 241278 events in 0 files, snapshot none (task 8336 ms: download 6050 ms overlapping gunzip 1560 ms, parse 5285 ms, sort 0 ms, write 0 ms, index 695 ms: 6 candidates, 90601 repo-hours, 1103 releases; waiting 515 ms; commit 0 ms)
2026-11-08T08:15:19Z [load_day 2024-10-07] 2024-10-07 hour 12: 281542 events in 0 files, snapshot none (task 8512 ms: download 5741 ms overlapping gunzip 1415 ms, parse 5315 ms, sort 0 ms, write 0 ms, index 979 ms: 10 candidates, 137955 repo-hours, 1366 releases; waiting 543 ms; commit 0 ms)
2026-11-08T08:15:22Z [load_day 2024-10-07] 2024-10-07 hour 13: 281662 events in 0 files, snapshot none (task 8935 ms: download 5688 ms overlapping gunzip 1517 ms, parse 5430 ms, sort 0 ms, write 0 ms, index 1431 ms: 11 candidates, 124348 repo-hours, 1279 releases; waiting 267 ms; commit 0 ms)
2026-11-08T08:15:29Z [load_day 2024-10-07] 2024-10-07 hour 14: 270773 events in 0 files, snapshot none (task 9424 ms: download 5663 ms overlapping gunzip 1682 ms, parse 5955 ms, sort 0 ms, write 0 ms, index 1011 ms: 8 candidates, 108720 repo-hours, 1317 releases; waiting 491 ms; commit 0 ms)
2026-11-08T08:15:33Z [load_day 2024-10-07] 2024-10-07 hour 15: 269726 events in 0 files, snapshot none (task 10325 ms: download 6876 ms overlapping gunzip 1762 ms, parse 6183 ms, sort 0 ms, write 0 ms, index 1239 ms: 11 candidates, 111561 repo-hours, 1426 releases; waiting 769 ms; commit 0 ms)
2026-11-08T08:15:39Z [load_day 2024-10-07] 2024-10-07 hour 16: 258821 events in 0 files, snapshot none (task 9056 ms: download 5423 ms overlapping gunzip 1559 ms, parse 5659 ms, sort 0 ms, write 0 ms, index 1078 ms: 10 candidates, 102563 repo-hours, 1190 releases; waiting 509 ms; commit 0 ms)
2026-11-08T08:15:42Z [load_day 2024-10-07] 2024-10-07 hour 17: 257976 events in 0 files, snapshot none (task 9079 ms: download 6182 ms overlapping gunzip 1676 ms, parse 5759 ms, sort 0 ms, write 0 ms, index 853 ms: 16 candidates, 96750 repo-hours, 1082 releases; waiting 463 ms; commit 0 ms)
2026-11-08T08:15:49Z [load_day 2024-10-07] 2024-10-07 hour 18: 268919 events in 0 files, snapshot none (task 9636 ms: download 5693 ms overlapping gunzip 1665 ms, parse 5838 ms, sort 0 ms, write 0 ms, index 1315 ms: 9 candidates, 112272 repo-hours, 1281 releases; waiting 527 ms; commit 0 ms)
2026-11-08T08:15:53Z [load_day 2024-10-07] 2024-10-07 hour 19: 251184 events in 0 files, snapshot none (task 9785 ms: download 7592 ms overlapping gunzip 1915 ms, parse 6421 ms, sort 0 ms, write 0 ms, index 652 ms: 13 candidates, 91856 repo-hours, 1119 releases; waiting 497 ms; commit 0 ms)
2026-11-08T08:15:59Z [load_day 2024-10-07] 2024-10-07 hour 20: 232497 events in 0 files, snapshot none (task 9205 ms: download 5949 ms overlapping gunzip 1682 ms, parse 5798 ms, sort 0 ms, write 0 ms, index 816 ms: 10 candidates, 85164 repo-hours, 1029 releases; waiting 633 ms; commit 0 ms)
2026-11-08T08:16:02Z [load_day 2024-10-07] 2024-10-07 hour 21: 231777 events in 0 files, snapshot none (task 9051 ms: download 6107 ms overlapping gunzip 1432 ms, parse 5577 ms, sort 0 ms, write 0 ms, index 1031 ms: 11 candidates, 83444 repo-hours, 1177 releases; waiting 550 ms; commit 0 ms)
2026-11-08T08:16:06Z [load_day 2024-10-07] 2024-10-07 hour 22: 208088 events in 0 files, snapshot none (task 7134 ms: download 4067 ms overlapping gunzip 1246 ms, parse 4344 ms, sort 0 ms, write 0 ms, index 717 ms: 6 candidates, 70698 repo-hours, 1279 releases; waiting 427 ms; commit 0 ms)
2026-11-08T08:16:10Z [load_day 2024-10-07] 2024-10-07 hour 23: 192147 events in 0 files, snapshot none (task 6727 ms: download 4572 ms overlapping gunzip 1103 ms, parse 4544 ms, sort 0 ms, write 0 ms, index 507 ms: 6 candidates, 64018 repo-hours, 905 releases; waiting 313 ms; commit 0 ms)
2026-11-08T08:16:20Z [load_day 2024-10-07] 2024-10-07: repo_hour merged from 24 files to 1 (2199349 rows, 10146 ms)
2026-11-08T08:16:23Z [load_day 2024-10-07] 2024-10-07: index committed in 12954 ms (24 + 1 + 24 files)
2026-11-08T08:16:23Z [load_day 2024-10-07] 2024-10-07: 5531508 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:16:23Z [load_day 2024-10-07] attempt 1 succeeded