load_day 2024-09-17, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:178905

Started

2026-11-08T07:57:19Z

Ended

2026-11-08T07:59:23Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

756 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

188.2 s, up to 2.5 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2971/load_day/2024-09-17/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T07:57:19Z [load_day 2024-09-17] attempt 1 started
2026-11-08T07:57:19Z [load_day 2024-09-17] 2024-09-17: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T07:57:27Z [load_day 2024-09-17] 2024-09-17 hour 0: 191486 events in 0 files, snapshot none (task 6776 ms: download 4050 ms overlapping gunzip 1112 ms, parse 4408 ms, sort 0 ms, write 0 ms, index 765 ms: 5 candidates, 100052 repo-hours, 1128 releases; waiting 191 ms; commit 0 ms)
2026-11-08T07:57:27Z [load_day 2024-09-17] 2024-09-17 hour 1: 186780 events in 0 files, snapshot none (task 6663 ms: download 3813 ms overlapping gunzip 954 ms, parse 4130 ms, sort 0 ms, write 0 ms, index 683 ms: 7 candidates, 100431 repo-hours, 778 releases; waiting 517 ms; commit 0 ms)
2026-11-08T07:57:35Z [load_day 2024-09-17] 2024-09-17 hour 2: 154175 events in 0 files, snapshot none (task 7335 ms: download 5111 ms overlapping gunzip 1003 ms, parse 4027 ms, sort 0 ms, write 0 ms, index 749 ms: 9 candidates, 58142 repo-hours, 700 releases; waiting 1225 ms; commit 0 ms)
2026-11-08T07:57:35Z [load_day 2024-09-17] 2024-09-17 hour 3: 170534 events in 0 files, snapshot none (task 7626 ms: download 4058 ms overlapping gunzip 1175 ms, parse 5086 ms, sort 0 ms, write 0 ms, index 569 ms: 5 candidates, 68007 repo-hours, 3792 releases; waiting 514 ms; commit 0 ms)
2026-11-08T07:57:42Z [load_day 2024-09-17] 2024-09-17 hour 4: 157266 events in 0 files, snapshot none (task 6742 ms: download 3666 ms overlapping gunzip 1043 ms, parse 4202 ms, sort 0 ms, write 0 ms, index 659 ms: 2 candidates, 58756 repo-hours, 689 releases; waiting 504 ms; commit 0 ms)
2026-11-08T07:57:43Z [load_day 2024-09-17] 2024-09-17 hour 5: 165757 events in 0 files, snapshot none (task 7951 ms: download 4388 ms overlapping gunzip 1312 ms, parse 4913 ms, sort 0 ms, write 0 ms, index 976 ms: 5 candidates, 62546 repo-hours, 802 releases; waiting 453 ms; commit 0 ms)
2026-11-08T07:57:51Z [load_day 2024-09-17] 2024-09-17 hour 6: 190953 events in 0 files, snapshot none (task 8298 ms: download 5096 ms overlapping gunzip 1293 ms, parse 5357 ms, sort 0 ms, write 0 ms, index 783 ms: 7 candidates, 83306 repo-hours, 983 releases; waiting 521 ms; commit 0 ms)
2026-11-08T07:57:52Z [load_day 2024-09-17] 2024-09-17 hour 7: 199871 events in 0 files, snapshot none (task 7767 ms: download 5203 ms overlapping gunzip 1172 ms, parse 5097 ms, sort 0 ms, write 0 ms, index 694 ms: 4 candidates, 77880 repo-hours, 1221 releases; waiting 468 ms; commit 0 ms)
2026-11-08T07:58:01Z [load_day 2024-09-17] 2024-09-17 hour 8: 210888 events in 0 files, snapshot none (task 10233 ms: download 7092 ms overlapping gunzip 1655 ms, parse 6975 ms, sort 0 ms, write 0 ms, index 837 ms: 3 candidates, 84861 repo-hours, 1126 releases; waiting 491 ms; commit 0 ms)
2026-11-08T07:58:02Z [load_day 2024-09-17] 2024-09-17 hour 9: 218849 events in 0 files, snapshot none (task 9135 ms: download 6641 ms overlapping gunzip 1471 ms, parse 6204 ms, sort 0 ms, write 0 ms, index 647 ms: 6 candidates, 90878 repo-hours, 1269 releases; waiting 480 ms; commit 0 ms)
2026-11-08T07:58:11Z [load_day 2024-09-17] 2024-09-17 hour 10: 217578 events in 0 files, snapshot none (task 9703 ms: download 7010 ms overlapping gunzip 1545 ms, parse 6578 ms, sort 0 ms, write 0 ms, index 697 ms: 5 candidates, 87829 repo-hours, 1103 releases; waiting 525 ms; commit 0 ms)
2026-11-08T07:58:12Z [load_day 2024-09-17] 2024-09-17 hour 11: 230478 events in 0 files, snapshot none (task 9512 ms: download 7089 ms overlapping gunzip 1646 ms, parse 6403 ms, sort 0 ms, write 0 ms, index 651 ms: 0 candidates, 88987 repo-hours, 1068 releases; waiting 510 ms; commit 0 ms)
2026-11-08T07:58:23Z [load_day 2024-09-17] 2024-09-17 hour 12: 277979 events in 0 files, snapshot none (task 10926 ms: download 7837 ms overlapping gunzip 1647 ms, parse 7427 ms, sort 0 ms, write 0 ms, index 1054 ms: 11 candidates, 135663 repo-hours, 1423 releases; waiting 489 ms; commit 0 ms)
2026-11-08T07:58:23Z [load_day 2024-09-17] 2024-09-17 hour 13: 265697 events in 0 files, snapshot none (task 11060 ms: download 8285 ms overlapping gunzip 1755 ms, parse 7576 ms, sort 0 ms, write 0 ms, index 901 ms: 8 candidates, 121811 repo-hours, 1343 releases; waiting 532 ms; commit 0 ms)
2026-11-08T07:58:33Z [load_day 2024-09-17] 2024-09-17 hour 14: 254808 events in 0 files, snapshot none (task 9573 ms: download 7006 ms overlapping gunzip 1528 ms, parse 6478 ms, sort 0 ms, write 0 ms, index 759 ms: 6 candidates, 106309 repo-hours, 1445 releases; waiting 502 ms; commit 0 ms)
2026-11-08T07:58:33Z [load_day 2024-09-17] 2024-09-17 hour 15: 256565 events in 0 files, snapshot none (task 8963 ms: download 6300 ms overlapping gunzip 1386 ms, parse 5993 ms, sort 0 ms, write 0 ms, index 740 ms: 3 candidates, 109105 repo-hours, 1401 releases; waiting 487 ms; commit 0 ms)
2026-11-08T07:58:44Z [load_day 2024-09-17] 2024-09-17 hour 16: 234683 events in 0 files, snapshot none (task 10361 ms: download 7704 ms overlapping gunzip 1556 ms, parse 6781 ms, sort 0 ms, write 0 ms, index 745 ms: 4 candidates, 97984 repo-hours, 1200 releases; waiting 972 ms; commit 0 ms)
2026-11-08T07:58:44Z [load_day 2024-09-17] 2024-09-17 hour 17: 228419 events in 0 files, snapshot none (task 9132 ms: download 6101 ms overlapping gunzip 1425 ms, parse 6029 ms, sort 0 ms, write 0 ms, index 703 ms: 9 candidates, 91123 repo-hours, 1113 releases; waiting 690 ms; commit 0 ms)
2026-11-08T07:58:54Z [load_day 2024-09-17] 2024-09-17 hour 18: 234580 events in 0 files, snapshot none (task 9798 ms: download 7170 ms overlapping gunzip 1602 ms, parse 6558 ms, sort 0 ms, write 0 ms, index 848 ms: 2 candidates, 104562 repo-hours, 1246 releases; waiting 508 ms; commit 0 ms)
2026-11-08T07:58:54Z [load_day 2024-09-17] 2024-09-17 hour 19: 218549 events in 0 files, snapshot none (task 9842 ms: download 7381 ms overlapping gunzip 1657 ms, parse 6958 ms, sort 0 ms, write 0 ms, index 641 ms: 8 candidates, 86518 repo-hours, 1115 releases; waiting 314 ms; commit 0 ms)
2026-11-08T07:59:04Z [load_day 2024-09-17] 2024-09-17 hour 20: 205688 events in 0 files, snapshot none (task 9342 ms: download 6590 ms overlapping gunzip 1606 ms, parse 6254 ms, sort 0 ms, write 0 ms, index 701 ms: 5 candidates, 84013 repo-hours, 945 releases; waiting 514 ms; commit 0 ms)
2026-11-08T07:59:04Z [load_day 2024-09-17] 2024-09-17 hour 21: 196611 events in 0 files, snapshot none (task 8497 ms: download 5862 ms overlapping gunzip 1347 ms, parse 5455 ms, sort 0 ms, write 0 ms, index 827 ms: 9 candidates, 77927 repo-hours, 1042 releases; waiting 541 ms; commit 0 ms)
2026-11-08T07:59:11Z [load_day 2024-09-17] 2024-09-17 hour 22: 171268 events in 0 files, snapshot none (task 6412 ms: download 4199 ms overlapping gunzip 1094 ms, parse 4285 ms, sort 0 ms, write 0 ms, index 447 ms: 5 candidates, 65380 repo-hours, 997 releases; waiting 282 ms; commit 0 ms)
2026-11-08T07:59:11Z [load_day 2024-09-17] 2024-09-17 hour 23: 164303 events in 0 files, snapshot none (task 6596 ms: download 4250 ms overlapping gunzip 1031 ms, parse 4175 ms, sort 0 ms, write 0 ms, index 469 ms: 5 candidates, 62100 repo-hours, 850 releases; waiting 586 ms; commit 0 ms)
2026-11-08T07:59:20Z [load_day 2024-09-17] 2024-09-17: repo_hour merged from 24 files to 1 (2104170 rows, 9198 ms)
2026-11-08T07:59:23Z [load_day 2024-09-17] 2024-09-17: index committed in 12008 ms (23 + 1 + 24 files)
2026-11-08T07:59:23Z [load_day 2024-09-17] 2024-09-17: 5003765 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T07:59:23Z [load_day 2024-09-17] attempt 1 succeeded