load_day 2024-03-04, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:94307

Started

2026-11-08T04:57:12Z

Ended

2026-11-08T04:59:17Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

717 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

204.3 s, up to 3.1 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2158/load_day/2024-03-04/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:57:12Z [load_day 2024-03-04] attempt 1 started
2026-11-08T04:57:12Z [load_day 2024-03-04] 2024-03-04: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:57:19Z [load_day 2024-03-04] 2024-03-04 hour 0: 213898 events in 0 files, snapshot none (task 5893 ms: download 2506 ms overlapping gunzip 911 ms, parse 3901 ms, sort 0 ms, write 0 ms, index 664 ms: 8 candidates, 88868 repo-hours, 1089 releases; waiting 201 ms; commit 0 ms)
2026-11-08T04:57:20Z [load_day 2024-03-04] 2024-03-04 hour 1: 222409 events in 0 files, snapshot none (task 6769 ms: download 4442 ms overlapping gunzip 1084 ms, parse 4275 ms, sort 0 ms, write 0 ms, index 625 ms: 7 candidates, 90981 repo-hours, 766 releases; waiting 504 ms; commit 0 ms)
2026-11-08T04:57:26Z [load_day 2024-03-04] 2024-03-04 hour 2: 188821 events in 0 files, snapshot none (task 6518 ms: download 3594 ms overlapping gunzip 1014 ms, parse 3992 ms, sort 0 ms, write 0 ms, index 512 ms: 7 candidates, 64697 repo-hours, 653 releases; waiting 522 ms; commit 0 ms)
2026-11-08T04:57:27Z [load_day 2024-03-04] 2024-03-04 hour 3: 195757 events in 0 files, snapshot none (task 6615 ms: download 4187 ms overlapping gunzip 1023 ms, parse 4256 ms, sort 0 ms, write 0 ms, index 558 ms: 5 candidates, 66658 repo-hours, 827 releases; waiting 529 ms; commit 0 ms)
2026-11-08T04:57:33Z [load_day 2024-03-04] 2024-03-04 hour 4: 188461 events in 0 files, snapshot none (task 7271 ms: download 4562 ms overlapping gunzip 1211 ms, parse 4569 ms, sort 0 ms, write 0 ms, index 445 ms: 6 candidates, 60420 repo-hours, 654 releases; waiting 528 ms; commit 0 ms)
2026-11-08T04:57:34Z [load_day 2024-03-04] 2024-03-04 hour 5: 192466 events in 0 files, snapshot none (task 6816 ms: download 4522 ms overlapping gunzip 1092 ms, parse 4555 ms, sort 0 ms, write 0 ms, index 468 ms: 5 candidates, 65260 repo-hours, 618 releases; waiting 511 ms; commit 0 ms)
2026-11-08T04:57:41Z [load_day 2024-03-04] 2024-03-04 hour 6: 216165 events in 0 files, snapshot none (task 6700 ms: download 4460 ms overlapping gunzip 1152 ms, parse 4284 ms, sort 0 ms, write 0 ms, index 532 ms: 7 candidates, 75136 repo-hours, 775 releases; waiting 508 ms; commit 0 ms)
2026-11-08T04:57:41Z [load_day 2024-03-04] 2024-03-04 hour 7: 230683 events in 0 files, snapshot none (task 6957 ms: download 4584 ms overlapping gunzip 1162 ms, parse 4509 ms, sort 0 ms, write 0 ms, index 612 ms: 12 candidates, 84602 repo-hours, 892 releases; waiting 373 ms; commit 0 ms)
2026-11-08T04:57:50Z [load_day 2024-03-04] 2024-03-04 hour 8: 221801 events in 0 files, snapshot none (task 8162 ms: download 4965 ms overlapping gunzip 1290 ms, parse 5212 ms, sort 0 ms, write 0 ms, index 914 ms: 5 candidates, 80367 repo-hours, 940 releases; waiting 505 ms; commit 0 ms)
2026-11-08T04:57:52Z [load_day 2024-03-04] 2024-03-04 hour 9: 250526 events in 0 files, snapshot none (task 10018 ms: download 7322 ms overlapping gunzip 1658 ms, parse 6675 ms, sort 0 ms, write 0 ms, index 829 ms: 14 candidates, 90228 repo-hours, 1215 releases; waiting 524 ms; commit 0 ms)
2026-11-08T04:58:00Z [load_day 2024-03-04] 2024-03-04 hour 10: 251405 events in 0 files, snapshot none (task 9623 ms: download 6682 ms overlapping gunzip 1622 ms, parse 6499 ms, sort 0 ms, write 0 ms, index 730 ms: 11 candidates, 90120 repo-hours, 1165 releases; waiting 485 ms; commit 0 ms)
2026-11-08T04:58:02Z [load_day 2024-03-04] 2024-03-04 hour 11: 250784 events in 0 files, snapshot none (task 9425 ms: download 6633 ms overlapping gunzip 1576 ms, parse 6054 ms, sort 0 ms, write 0 ms, index 950 ms: 11 candidates, 89793 repo-hours, 1083 releases; waiting 496 ms; commit 0 ms)
2026-11-08T04:58:10Z [load_day 2024-03-04] 2024-03-04 hour 12: 294773 events in 0 files, snapshot none (task 9603 ms: download 6682 ms overlapping gunzip 1419 ms, parse 6273 ms, sort 0 ms, write 0 ms, index 1069 ms: 11 candidates, 119900 repo-hours, 1139 releases; waiting 579 ms; commit 0 ms)
2026-11-08T04:58:13Z [load_day 2024-03-04] 2024-03-04 hour 13: 284204 events in 0 files, snapshot none (task 10805 ms: download 7128 ms overlapping gunzip 1704 ms, parse 6738 ms, sort 0 ms, write 0 ms, index 1511 ms: 11 candidates, 114208 repo-hours, 1299 releases; waiting 555 ms; commit 0 ms)
2026-11-08T04:58:23Z [load_day 2024-03-04] 2024-03-04 hour 14: 276346 events in 0 files, snapshot none (task 12209 ms: download 8662 ms overlapping gunzip 2037 ms, parse 8465 ms, sort 0 ms, write 0 ms, index 973 ms: 6 candidates, 103325 repo-hours, 1266 releases; waiting 501 ms; commit 0 ms)
2026-11-08T04:58:26Z [load_day 2024-03-04] 2024-03-04 hour 15: 279130 events in 0 files, snapshot none (task 11860 ms: download 8651 ms overlapping gunzip 1941 ms, parse 8223 ms, sort 0 ms, write 0 ms, index 1015 ms: 11 candidates, 105037 repo-hours, 1284 releases; waiting 442 ms; commit 0 ms)
2026-11-08T04:58:34Z [load_day 2024-03-04] 2024-03-04 hour 16: 276019 events in 0 files, snapshot none (task 11177 ms: download 8004 ms overlapping gunzip 1853 ms, parse 7823 ms, sort 0 ms, write 0 ms, index 758 ms: 9 candidates, 102025 repo-hours, 1193 releases; waiting 489 ms; commit 0 ms)
2026-11-08T04:58:37Z [load_day 2024-03-04] 2024-03-04 hour 17: 264663 events in 0 files, snapshot none (task 10621 ms: download 8057 ms overlapping gunzip 1790 ms, parse 7286 ms, sort 0 ms, write 0 ms, index 810 ms: 10 candidates, 93681 repo-hours, 1138 releases; waiting 471 ms; commit 0 ms)
2026-11-08T04:58:45Z [load_day 2024-03-04] 2024-03-04 hour 18: 268392 events in 0 files, snapshot none (task 9905 ms: download 7010 ms overlapping gunzip 1611 ms, parse 6598 ms, sort 0 ms, write 0 ms, index 878 ms: 12 candidates, 99533 repo-hours, 1156 releases; waiting 526 ms; commit 0 ms)
2026-11-08T04:58:47Z [load_day 2024-03-04] 2024-03-04 hour 19: 258463 events in 0 files, snapshot none (task 9659 ms: download 7078 ms overlapping gunzip 1573 ms, parse 6508 ms, sort 0 ms, write 0 ms, index 789 ms: 10 candidates, 89817 repo-hours, 1061 releases; waiting 543 ms; commit 0 ms)
2026-11-08T04:58:55Z [load_day 2024-03-04] 2024-03-04 hour 20: 242599 events in 0 files, snapshot none (task 9580 ms: download 6600 ms overlapping gunzip 1544 ms, parse 6572 ms, sort 0 ms, write 0 ms, index 661 ms: 11 candidates, 84278 repo-hours, 977 releases; waiting 527 ms; commit 0 ms)
2026-11-08T04:58:57Z [load_day 2024-03-04] 2024-03-04 hour 21: 245743 events in 0 files, snapshot none (task 9768 ms: download 7485 ms overlapping gunzip 1644 ms, parse 6773 ms, sort 0 ms, write 0 ms, index 592 ms: 8 candidates, 81221 repo-hours, 1102 releases; waiting 480 ms; commit 0 ms)
2026-11-08T04:59:04Z [load_day 2024-03-04] 2024-03-04 hour 22: 230050 events in 0 files, snapshot none (task 8932 ms: download 5957 ms overlapping gunzip 1401 ms, parse 6143 ms, sort 0 ms, write 0 ms, index 662 ms: 10 candidates, 74163 repo-hours, 1098 releases; waiting 473 ms; commit 0 ms)
2026-11-08T04:59:06Z [load_day 2024-03-04] 2024-03-04 hour 23: 208400 events in 0 files, snapshot none (task 7634 ms: download 5166 ms overlapping gunzip 1267 ms, parse 5099 ms, sort 0 ms, write 0 ms, index 530 ms: 12 candidates, 66978 repo-hours, 871 releases; waiting 522 ms; commit 0 ms)
2026-11-08T04:59:14Z [load_day 2024-03-04] 2024-03-04: repo_hour merged from 24 files to 1 (2081296 rows, 8942 ms)
2026-11-08T04:59:17Z [load_day 2024-03-04] 2024-03-04: index committed in 11264 ms (24 + 1 + 24 files)
2026-11-08T04:59:17Z [load_day 2024-03-04] 2024-03-04: 5751958 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:59:17Z [load_day 2024-03-04] attempt 1 succeeded