load_day 2024-01-12, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:18172

Started

2026-11-08T04:07:14Z

Ended

2026-11-08T04:09:03Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

691 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

162.2 s, up to 3.1 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1962/load_day/2024-01-12/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:07:14Z [load_day 2024-01-12] attempt 1 started
2026-11-08T04:07:15Z [load_day 2024-01-12] 2024-01-12: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:07:23Z [load_day 2024-01-12] 2024-01-12 hour 0: 187914 events in 0 files, snapshot none (task 7425 ms: download 4241 ms overlapping gunzip 1102 ms, parse 4631 ms, sort 0 ms, write 0 ms, index 928 ms: 5 candidates, 83069 repo-hours, 1028 releases; waiting 475 ms; commit 0 ms)
2026-11-08T04:07:23Z [load_day 2024-01-12] 2024-01-12 hour 1: 186313 events in 0 files, snapshot none (task 6406 ms: download 3497 ms overlapping gunzip 932 ms, parse 3840 ms, sort 0 ms, write 0 ms, index 812 ms: 3 candidates, 93259 repo-hours, 882 releases; waiting 519 ms; commit 0 ms)
2026-11-08T04:07:29Z [load_day 2024-01-12] 2024-01-12 hour 2: 162816 events in 0 files, snapshot none (task 5944 ms: download 3476 ms overlapping gunzip 923 ms, parse 3796 ms, sort 0 ms, write 0 ms, index 500 ms: 3 candidates, 62552 repo-hours, 718 releases; waiting 472 ms; commit 0 ms)
2026-11-08T04:07:29Z [load_day 2024-01-12] 2024-01-12 hour 3: 157225 events in 0 files, snapshot none (task 5734 ms: download 3252 ms overlapping gunzip 900 ms, parse 3592 ms, sort 0 ms, write 0 ms, index 430 ms: 2 candidates, 58360 repo-hours, 943 releases; waiting 434 ms; commit 0 ms)
2026-11-08T04:07:35Z [load_day 2024-01-12] 2024-01-12 hour 4: 155832 events in 0 files, snapshot none (task 5516 ms: download 3221 ms overlapping gunzip 831 ms, parse 3477 ms, sort 0 ms, write 0 ms, index 366 ms: 4 candidates, 53886 repo-hours, 593 releases; waiting 522 ms; commit 0 ms)
2026-11-08T04:07:35Z [load_day 2024-01-12] 2024-01-12 hour 5: 168611 events in 0 files, snapshot none (task 5440 ms: download 3102 ms overlapping gunzip 802 ms, parse 3404 ms, sort 0 ms, write 0 ms, index 405 ms: 3 candidates, 58916 repo-hours, 623 releases; waiting 472 ms; commit 0 ms)
2026-11-08T04:07:41Z [load_day 2024-01-12] 2024-01-12 hour 6: 186382 events in 0 files, snapshot none (task 5275 ms: download 2845 ms overlapping gunzip 762 ms, parse 3126 ms, sort 0 ms, write 0 ms, index 565 ms: 3 candidates, 71875 repo-hours, 828 releases; waiting 580 ms; commit 0 ms)
2026-11-08T04:07:41Z [load_day 2024-01-12] 2024-01-12 hour 7: 193242 events in 0 files, snapshot none (task 5676 ms: download 3324 ms overlapping gunzip 865 ms, parse 3497 ms, sort 0 ms, write 0 ms, index 529 ms: 3 candidates, 70790 repo-hours, 1066 releases; waiting 522 ms; commit 0 ms)
2026-11-08T04:07:47Z [load_day 2024-01-12] 2024-01-12 hour 8: 202196 events in 0 files, snapshot none (task 5825 ms: download 3360 ms overlapping gunzip 866 ms, parse 3633 ms, sort 0 ms, write 0 ms, index 561 ms: 6 candidates, 77554 repo-hours, 1043 releases; waiting 506 ms; commit 0 ms)
2026-11-08T04:07:48Z [load_day 2024-01-12] 2024-01-12 hour 9: 214061 events in 0 files, snapshot none (task 6277 ms: download 3861 ms overlapping gunzip 946 ms, parse 3950 ms, sort 0 ms, write 0 ms, index 556 ms: 6 candidates, 83259 repo-hours, 1331 releases; waiting 563 ms; commit 0 ms)
2026-11-08T04:07:57Z [load_day 2024-01-12] 2024-01-12 hour 10: 218666 events in 0 files, snapshot none (task 8692 ms: download 6055 ms overlapping gunzip 1446 ms, parse 5714 ms, sort 0 ms, write 0 ms, index 608 ms: 9 candidates, 82162 repo-hours, 1120 releases; waiting 529 ms; commit 0 ms)
2026-11-08T04:07:57Z [load_day 2024-01-12] 2024-01-12 hour 11: 206257 events in 0 files, snapshot none (task 7823 ms: download 5385 ms overlapping gunzip 1208 ms, parse 5326 ms, sort 0 ms, write 0 ms, index 568 ms: 9 candidates, 76434 repo-hours, 964 releases; waiting 504 ms; commit 0 ms)
2026-11-08T04:08:05Z [load_day 2024-01-12] 2024-01-12 hour 12: 255318 events in 0 files, snapshot none (task 8268 ms: download 4610 ms overlapping gunzip 1283 ms, parse 5417 ms, sort 0 ms, write 0 ms, index 958 ms: 9 candidates, 108195 repo-hours, 1088 releases; waiting 321 ms; commit 0 ms)
2026-11-08T04:08:06Z [load_day 2024-01-12] 2024-01-12 hour 13: 234972 events in 0 files, snapshot none (task 8788 ms: download 5451 ms overlapping gunzip 1350 ms, parse 5828 ms, sort 0 ms, write 0 ms, index 768 ms: 9 candidates, 103164 repo-hours, 1091 releases; waiting 487 ms; commit 0 ms)
2026-11-08T04:08:15Z [load_day 2024-01-12] 2024-01-12 hour 14: 226841 events in 0 files, snapshot none (task 9104 ms: download 5793 ms overlapping gunzip 1348 ms, parse 5567 ms, sort 0 ms, write 0 ms, index 1070 ms: 5 candidates, 88933 repo-hours, 1130 releases; waiting 845 ms; commit 0 ms)
2026-11-08T04:08:16Z [load_day 2024-01-12] 2024-01-12 hour 15: 234801 events in 0 files, snapshot none (task 10191 ms: download 7032 ms overlapping gunzip 1609 ms, parse 6965 ms, sort 0 ms, write 0 ms, index 844 ms: 10 candidates, 93520 repo-hours, 1213 releases; waiting 496 ms; commit 0 ms)
2026-11-08T04:08:26Z [load_day 2024-01-12] 2024-01-12 hour 16: 229333 events in 0 files, snapshot none (task 10320 ms: download 6992 ms overlapping gunzip 1608 ms, parse 6674 ms, sort 0 ms, write 0 ms, index 850 ms: 9 candidates, 90766 repo-hours, 1066 releases; waiting 842 ms; commit 0 ms)
2026-11-08T04:08:28Z [load_day 2024-01-12] 2024-01-12 hour 17: 222559 events in 0 files, snapshot none (task 11124 ms: download 7306 ms overlapping gunzip 1802 ms, parse 7403 ms, sort 0 ms, write 0 ms, index 1114 ms: 3 candidates, 84163 repo-hours, 1023 releases; waiting 506 ms; commit 0 ms)
2026-11-08T04:08:35Z [load_day 2024-01-12] 2024-01-12 hour 18: 218835 events in 0 files, snapshot none (task 9148 ms: download 5208 ms overlapping gunzip 1454 ms, parse 6123 ms, sort 0 ms, write 0 ms, index 792 ms: 7 candidates, 86076 repo-hours, 1115 releases; waiting 521 ms; commit 0 ms)
2026-11-08T04:08:36Z [load_day 2024-01-12] 2024-01-12 hour 19: 196546 events in 0 files, snapshot none (task 7551 ms: download 4529 ms overlapping gunzip 1156 ms, parse 4892 ms, sort 0 ms, write 0 ms, index 661 ms: 7 candidates, 70578 repo-hours, 1013 releases; waiting 567 ms; commit 0 ms)
2026-11-08T04:08:44Z [load_day 2024-01-12] 2024-01-12 hour 20: 193997 events in 0 files, snapshot none (task 7720 ms: download 5033 ms overlapping gunzip 1163 ms, parse 4993 ms, sort 0 ms, write 0 ms, index 598 ms: 4 candidates, 69998 repo-hours, 898 releases; waiting 675 ms; commit 0 ms)
2026-11-08T04:08:44Z [load_day 2024-01-12] 2024-01-12 hour 21: 183381 events in 0 files, snapshot none (task 7275 ms: download 4684 ms overlapping gunzip 1125 ms, parse 4740 ms, sort 0 ms, write 0 ms, index 467 ms: 9 candidates, 64053 repo-hours, 973 releases; waiting 671 ms; commit 0 ms)
2026-11-08T04:08:51Z [load_day 2024-01-12] 2024-01-12 hour 22: 173185 events in 0 files, snapshot none (task 6829 ms: download 4466 ms overlapping gunzip 1040 ms, parse 4676 ms, sort 0 ms, write 0 ms, index 430 ms: 6 candidates, 58368 repo-hours, 1050 releases; waiting 407 ms; commit 0 ms)
2026-11-08T04:08:51Z [load_day 2024-01-12] 2024-01-12 hour 23: 156964 events in 0 files, snapshot none (task 6356 ms: download 3913 ms overlapping gunzip 1082 ms, parse 4462 ms, sort 0 ms, write 0 ms, index 401 ms: 11 candidates, 52288 repo-hours, 726 releases; waiting 214 ms; commit 0 ms)
2026-11-08T04:09:00Z [load_day 2024-01-12] 2024-01-12: repo_hour merged from 24 files to 1 (1842218 rows, 8927 ms)
2026-11-08T04:09:03Z [load_day 2024-01-12] 2024-01-12: index committed in 11654 ms (24 + 1 + 24 files)
2026-11-08T04:09:03Z [load_day 2024-01-12] 2024-01-12: 4766247 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:09:03Z [load_day 2024-01-12] attempt 1 succeeded