load_day 2024-01-09, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:17240

Started

2026-11-08T04:04:02Z

Ended

2026-11-08T04:05:55Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

697 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

180.6 s, up to 3.4 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1951/load_day/2024-01-09/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:04:02Z [load_day 2024-01-09] attempt 1 started
2026-11-08T04:04:03Z [load_day 2024-01-09] 2024-01-09: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:04:11Z [load_day 2024-01-09] 2024-01-09 hour 0: 211981 events in 0 files, snapshot none (task 7562 ms: download 4166 ms overlapping gunzip 1114 ms, parse 4558 ms, sort 0 ms, write 0 ms, index 1109 ms: 6 candidates, 85605 repo-hours, 945 releases; waiting 485 ms; commit 0 ms)
2026-11-08T04:04:11Z [load_day 2024-01-09] 2024-01-09 hour 1: 226771 events in 0 files, snapshot none (task 7306 ms: download 4161 ms overlapping gunzip 1185 ms, parse 4453 ms, sort 0 ms, write 0 ms, index 909 ms: 5 candidates, 94686 repo-hours, 887 releases; waiting 503 ms; commit 0 ms)
2026-11-08T04:04:18Z [load_day 2024-01-09] 2024-01-09 hour 2: 198496 events in 0 files, snapshot none (task 6174 ms: download 4039 ms overlapping gunzip 959 ms, parse 3988 ms, sort 0 ms, write 0 ms, index 475 ms: 9 candidates, 63624 repo-hours, 847 releases; waiting 450 ms; commit 0 ms)
2026-11-08T04:04:18Z [load_day 2024-01-09] 2024-01-09 hour 3: 194713 events in 0 files, snapshot none (task 6056 ms: download 3628 ms overlapping gunzip 924 ms, parse 3911 ms, sort 0 ms, write 0 ms, index 536 ms: 4 candidates, 60501 repo-hours, 874 releases; waiting 491 ms; commit 0 ms)
2026-11-08T04:04:25Z [load_day 2024-01-09] 2024-01-09 hour 4: 190623 events in 0 files, snapshot none (task 6630 ms: download 4098 ms overlapping gunzip 1030 ms, parse 4384 ms, sort 0 ms, write 0 ms, index 405 ms: 8 candidates, 57367 repo-hours, 657 releases; waiting 508 ms; commit 0 ms)
2026-11-08T04:04:26Z [load_day 2024-01-09] 2024-01-09 hour 5: 196910 events in 0 files, snapshot none (task 6940 ms: download 4135 ms overlapping gunzip 1102 ms, parse 4610 ms, sort 0 ms, write 0 ms, index 455 ms: 2 candidates, 61566 repo-hours, 874 releases; waiting 495 ms; commit 0 ms)
2026-11-08T04:04:34Z [load_day 2024-01-09] 2024-01-09 hour 6: 219800 events in 0 files, snapshot none (task 8440 ms: download 5347 ms overlapping gunzip 1123 ms, parse 4976 ms, sort 0 ms, write 0 ms, index 748 ms: 3 candidates, 75770 repo-hours, 1023 releases; waiting 1309 ms; commit 0 ms)
2026-11-08T04:04:34Z [load_day 2024-01-09] 2024-01-09 hour 7: 223394 events in 0 files, snapshot none (task 7966 ms: download 4910 ms overlapping gunzip 1254 ms, parse 5129 ms, sort 0 ms, write 0 ms, index 701 ms: 11 candidates, 75303 repo-hours, 1151 releases; waiting 569 ms; commit 0 ms)
2026-11-08T04:04:43Z [load_day 2024-01-09] 2024-01-09 hour 8: 231629 events in 0 files, snapshot none (task 8429 ms: download 5108 ms overlapping gunzip 1360 ms, parse 5575 ms, sort 0 ms, write 0 ms, index 697 ms: 9 candidates, 83108 repo-hours, 1085 releases; waiting 523 ms; commit 0 ms)
2026-11-08T04:04:43Z [load_day 2024-01-09] 2024-01-09 hour 9: 238701 events in 0 files, snapshot none (task 8121 ms: download 5172 ms overlapping gunzip 1256 ms, parse 5306 ms, sort 0 ms, write 0 ms, index 812 ms: 3 candidates, 85403 repo-hours, 1265 releases; waiting 487 ms; commit 0 ms)
2026-11-08T04:04:52Z [load_day 2024-01-09] 2024-01-09 hour 10: 244036 events in 0 files, snapshot none (task 8708 ms: download 5449 ms overlapping gunzip 1369 ms, parse 5915 ms, sort 0 ms, write 0 ms, index 669 ms: 11 candidates, 86574 repo-hours, 1176 releases; waiting 505 ms; commit 0 ms)
2026-11-08T04:04:52Z [load_day 2024-01-09] 2024-01-09 hour 11: 228580 events in 0 files, snapshot none (task 8726 ms: download 4851 ms overlapping gunzip 1430 ms, parse 5902 ms, sort 0 ms, write 0 ms, index 773 ms: 6 candidates, 80680 repo-hours, 1220 releases; waiting 347 ms; commit 0 ms)
2026-11-08T04:05:00Z [load_day 2024-01-09] 2024-01-09 hour 12: 253340 events in 0 files, snapshot none (task 8136 ms: download 5023 ms overlapping gunzip 1205 ms, parse 5305 ms, sort 0 ms, write 0 ms, index 760 ms: 10 candidates, 110214 repo-hours, 1179 releases; waiting 442 ms; commit 0 ms)
2026-11-08T04:05:00Z [load_day 2024-01-09] 2024-01-09 hour 13: 183668 events in 0 files, snapshot none (task 6637 ms: download 3660 ms overlapping gunzip 955 ms, parse 4028 ms, sort 0 ms, write 0 ms, index 848 ms: 12 candidates, 84991 repo-hours, 846 releases; waiting 513 ms; commit 0 ms)
2026-11-08T04:05:09Z [load_day 2024-01-09] 2024-01-09 hour 14: 247992 events in 0 files, snapshot none (task 7656 ms: download 5288 ms overlapping gunzip 1225 ms, parse 5033 ms, sort 0 ms, write 0 ms, index 651 ms: 5 candidates, 96521 repo-hours, 1213 releases; waiting 531 ms; commit 0 ms)
2026-11-08T04:05:09Z [load_day 2024-01-09] 2024-01-09 hour 15: 247440 events in 0 files, snapshot none (task 7676 ms: download 5175 ms overlapping gunzip 1196 ms, parse 5133 ms, sort 0 ms, write 0 ms, index 736 ms: 6 candidates, 98899 repo-hours, 1356 releases; waiting 355 ms; commit 0 ms)
2026-11-08T04:05:16Z [load_day 2024-01-09] 2024-01-09 hour 16: 244942 events in 0 files, snapshot none (task 7323 ms: download 4887 ms overlapping gunzip 1169 ms, parse 4695 ms, sort 0 ms, write 0 ms, index 669 ms: 2 candidates, 97691 repo-hours, 1159 releases; waiting 524 ms; commit 0 ms)
2026-11-08T04:05:17Z [load_day 2024-01-09] 2024-01-09 hour 17: 246703 events in 0 files, snapshot none (task 7611 ms: download 5201 ms overlapping gunzip 1174 ms, parse 4872 ms, sort 0 ms, write 0 ms, index 632 ms: 6 candidates, 92629 repo-hours, 1049 releases; waiting 637 ms; commit 0 ms)
2026-11-08T04:05:28Z [load_day 2024-01-09] 2024-01-09 hour 18: 246865 events in 0 files, snapshot none (task 8922 ms: download 5922 ms overlapping gunzip 1395 ms, parse 6078 ms, sort 0 ms, write 0 ms, index 699 ms: 6 candidates, 91261 repo-hours, 1169 releases; waiting 487 ms; commit 0 ms)
2026-11-08T04:05:28Z [load_day 2024-01-09] 2024-01-09 hour 19: 231943 events in 0 files, snapshot none (task 8781 ms: download 5689 ms overlapping gunzip 1487 ms, parse 5876 ms, sort 0 ms, write 0 ms, index 648 ms: 7 candidates, 79775 repo-hours, 974 releases; waiting 515 ms; commit 0 ms)
2026-11-08T04:05:36Z [load_day 2024-01-09] 2024-01-09 hour 20: 223812 events in 0 files, snapshot none (task 8379 ms: download 5658 ms overlapping gunzip 1272 ms, parse 5335 ms, sort 0 ms, write 0 ms, index 547 ms: 9 candidates, 75236 repo-hours, 1038 releases; waiting 774 ms; commit 0 ms)
2026-11-08T04:05:36Z [load_day 2024-01-09] 2024-01-09 hour 21: 211476 events in 0 files, snapshot none (task 7756 ms: download 5023 ms overlapping gunzip 1206 ms, parse 4909 ms, sort 0 ms, write 0 ms, index 551 ms: 9 candidates, 70394 repo-hours, 985 releases; waiting 543 ms; commit 0 ms)
2026-11-08T04:05:44Z [load_day 2024-01-09] 2024-01-09 hour 22: 197307 events in 0 files, snapshot none (task 7074 ms: download 4767 ms overlapping gunzip 1082 ms, parse 4700 ms, sort 0 ms, write 0 ms, index 486 ms: 7 candidates, 63579 repo-hours, 967 releases; waiting 538 ms; commit 0 ms)
2026-11-08T04:05:44Z [load_day 2024-01-09] 2024-01-09 hour 23: 187687 events in 0 files, snapshot none (task 7192 ms: download 4971 ms overlapping gunzip 1135 ms, parse 4902 ms, sort 0 ms, write 0 ms, index 409 ms: 7 candidates, 59545 repo-hours, 723 releases; waiting 500 ms; commit 0 ms)
2026-11-08T04:05:53Z [load_day 2024-01-09] 2024-01-09: repo_hour merged from 24 files to 1 (1930922 rows, 8483 ms)
2026-11-08T04:05:55Z [load_day 2024-01-09] 2024-01-09: index committed in 10709 ms (24 + 1 + 24 files)
2026-11-08T04:05:55Z [load_day 2024-01-09] 2024-01-09: 5328809 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:05:55Z [load_day 2024-01-09] attempt 1 succeeded