load_day 2024-10-29, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:199769

Started

2026-11-08T08:33:32Z

Ended

2026-11-08T08:35:23Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

741 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

178.1 s, up to 2.9 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/3143/load_day/2024-10-29/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T08:33:32Z [load_day 2024-10-29] attempt 1 started
2026-11-08T08:33:33Z [load_day 2024-10-29] 2024-10-29: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:33:41Z [load_day 2024-10-29] 2024-10-29 hour 0: 208163 events in 0 files, snapshot none (task 7436 ms: download 4153 ms overlapping gunzip 1160 ms, parse 4679 ms, sort 0 ms, write 0 ms, index 807 ms: 13 candidates, 94844 repo-hours, 1256 releases; waiting 474 ms; commit 0 ms)
2026-11-08T08:33:41Z [load_day 2024-10-29] 2024-10-29 hour 1: 221650 events in 0 files, snapshot none (task 7159 ms: download 3930 ms overlapping gunzip 1076 ms, parse 4320 ms, sort 0 ms, write 0 ms, index 995 ms: 8 candidates, 115013 repo-hours, 1027 releases; waiting 486 ms; commit 0 ms)
2026-11-08T08:33:49Z [load_day 2024-10-29] 2024-10-29 hour 2: 199736 events in 0 files, snapshot none (task 7030 ms: download 3802 ms overlapping gunzip 1052 ms, parse 4115 ms, sort 0 ms, write 0 ms, index 988 ms: 4 candidates, 86876 repo-hours, 1042 releases; waiting 511 ms; commit 0 ms)
2026-11-08T08:33:49Z [load_day 2024-10-29] 2024-10-29 hour 3: 207851 events in 0 files, snapshot none (task 7439 ms: download 4201 ms overlapping gunzip 1238 ms, parse 4724 ms, sort 0 ms, write 0 ms, index 729 ms: 5 candidates, 81753 repo-hours, 4034 releases; waiting 517 ms; commit 0 ms)
2026-11-08T08:33:55Z [load_day 2024-10-29] 2024-10-29 hour 4: 187909 events in 0 files, snapshot none (task 5572 ms: download 3057 ms overlapping gunzip 881 ms, parse 3377 ms, sort 0 ms, write 0 ms, index 780 ms: 6 candidates, 65601 repo-hours, 783 releases; waiting 270 ms; commit 0 ms)
2026-11-08T08:33:56Z [load_day 2024-10-29] 2024-10-29 hour 5: 195867 events in 0 files, snapshot none (task 6509 ms: download 4078 ms overlapping gunzip 1001 ms, parse 3723 ms, sort 0 ms, write 0 ms, index 866 ms: 9 candidates, 69209 repo-hours, 927 releases; waiting 589 ms; commit 0 ms)
2026-11-08T08:34:02Z [load_day 2024-10-29] 2024-10-29 hour 6: 218764 events in 0 files, snapshot none (task 6807 ms: download 4271 ms overlapping gunzip 1125 ms, parse 4138 ms, sort 0 ms, write 0 ms, index 713 ms: 7 candidates, 90916 repo-hours, 1014 releases; waiting 521 ms; commit 0 ms)
2026-11-08T08:34:04Z [load_day 2024-10-29] 2024-10-29 hour 7: 218968 events in 0 files, snapshot none (task 7064 ms: download 4609 ms overlapping gunzip 1208 ms, parse 4247 ms, sort 0 ms, write 0 ms, index 772 ms: 6 candidates, 82914 repo-hours, 1037 releases; waiting 517 ms; commit 0 ms)
2026-11-08T08:34:11Z [load_day 2024-10-29] 2024-10-29 hour 8: 234352 events in 0 files, snapshot none (task 8132 ms: download 5321 ms overlapping gunzip 1348 ms, parse 5247 ms, sort 0 ms, write 0 ms, index 742 ms: 8 candidates, 92470 repo-hours, 1155 releases; waiting 500 ms; commit 0 ms)
2026-11-08T08:34:12Z [load_day 2024-10-29] 2024-10-29 hour 9: 250310 events in 0 files, snapshot none (task 7892 ms: download 5620 ms overlapping gunzip 1349 ms, parse 5031 ms, sort 0 ms, write 0 ms, index 733 ms: 8 candidates, 103577 repo-hours, 1401 releases; waiting 506 ms; commit 0 ms)
2026-11-08T08:34:18Z [load_day 2024-10-29] 2024-10-29 hour 10: 238896 events in 0 files, snapshot none (task 6403 ms: download 4101 ms overlapping gunzip 1057 ms, parse 3845 ms, sort 0 ms, write 0 ms, index 713 ms: 8 candidates, 94948 repo-hours, 1236 releases; waiting 525 ms; commit 0 ms)
2026-11-08T08:34:20Z [load_day 2024-10-29] 2024-10-29 hour 11: 239179 events in 0 files, snapshot none (task 6592 ms: download 4276 ms overlapping gunzip 1064 ms, parse 4012 ms, sort 0 ms, write 0 ms, index 677 ms: 8 candidates, 90840 repo-hours, 1239 releases; waiting 514 ms; commit 0 ms)
2026-11-08T08:34:25Z [load_day 2024-10-29] 2024-10-29 hour 12: 280357 events in 0 files, snapshot none (task 7090 ms: download 4263 ms overlapping gunzip 1034 ms, parse 4184 ms, sort 0 ms, write 0 ms, index 1117 ms: 13 candidates, 137947 repo-hours, 1401 releases; waiting 535 ms; commit 0 ms)
2026-11-08T08:34:27Z [load_day 2024-10-29] 2024-10-29 hour 13: 274158 events in 0 files, snapshot none (task 7253 ms: download 4658 ms overlapping gunzip 1142 ms, parse 4265 ms, sort 0 ms, write 0 ms, index 1054 ms: 10 candidates, 121220 repo-hours, 1274 releases; waiting 564 ms; commit 0 ms)
2026-11-08T08:34:34Z [load_day 2024-10-29] 2024-10-29 hour 14: 257997 events in 0 files, snapshot none (task 8711 ms: download 6236 ms overlapping gunzip 1549 ms, parse 5795 ms, sort 0 ms, write 0 ms, index 857 ms: 16 candidates, 108278 repo-hours, 1344 releases; waiting 315 ms; commit 0 ms)
2026-11-08T08:34:38Z [load_day 2024-10-29] 2024-10-29 hour 15: 262570 events in 0 files, snapshot none (task 10411 ms: download 6222 ms overlapping gunzip 1761 ms, parse 6156 ms, sort 0 ms, write 0 ms, index 1650 ms: 11 candidates, 113007 repo-hours, 1551 releases; waiting 534 ms; commit 0 ms)
2026-11-08T08:34:44Z [load_day 2024-10-29] 2024-10-29 hour 16: 259609 events in 0 files, snapshot none (task 9809 ms: download 6091 ms overlapping gunzip 1761 ms, parse 5958 ms, sort 0 ms, write 0 ms, index 1238 ms: 11 candidates, 107380 repo-hours, 1360 releases; waiting 249 ms; commit 0 ms)
2026-11-08T08:34:47Z [load_day 2024-10-29] 2024-10-29 hour 17: 252254 events in 0 files, snapshot none (task 8391 ms: download 5871 ms overlapping gunzip 1477 ms, parse 5275 ms, sort 0 ms, write 0 ms, index 752 ms: 9 candidates, 98258 repo-hours, 1285 releases; waiting 570 ms; commit 0 ms)
2026-11-08T08:34:54Z [load_day 2024-10-29] 2024-10-29 hour 18: 252187 events in 0 files, snapshot none (task 8600 ms: download 5715 ms overlapping gunzip 1413 ms, parse 5445 ms, sort 0 ms, write 0 ms, index 899 ms: 8 candidates, 111019 repo-hours, 1167 releases; waiting 507 ms; commit 0 ms)
2026-11-08T08:34:56Z [load_day 2024-10-29] 2024-10-29 hour 19: 231056 events in 0 files, snapshot none (task 7736 ms: download 5455 ms overlapping gunzip 1364 ms, parse 4817 ms, sort 0 ms, write 0 ms, index 772 ms: 7 candidates, 88686 repo-hours, 1186 releases; waiting 523 ms; commit 0 ms)
2026-11-08T08:35:03Z [load_day 2024-10-29] 2024-10-29 hour 20: 226813 events in 0 files, snapshot none (task 8884 ms: download 6426 ms overlapping gunzip 1604 ms, parse 5897 ms, sort 0 ms, write 0 ms, index 734 ms: 10 candidates, 87523 repo-hours, 1140 releases; waiting 348 ms; commit 0 ms)
2026-11-08T08:35:04Z [load_day 2024-10-29] 2024-10-29 hour 21: 224194 events in 0 files, snapshot none (task 7705 ms: download 5516 ms overlapping gunzip 1302 ms, parse 4976 ms, sort 0 ms, write 0 ms, index 608 ms: 13 candidates, 84806 repo-hours, 1437 releases; waiting 528 ms; commit 0 ms)
2026-11-08T08:35:10Z [load_day 2024-10-29] 2024-10-29 hour 22: 204405 events in 0 files, snapshot none (task 7012 ms: download 4587 ms overlapping gunzip 1226 ms, parse 4399 ms, sort 0 ms, write 0 ms, index 559 ms: 10 candidates, 73880 repo-hours, 1483 releases; waiting 513 ms; commit 0 ms)
2026-11-08T08:35:11Z [load_day 2024-10-29] 2024-10-29 hour 23: 189209 events in 0 files, snapshot none (task 6791 ms: download 4517 ms overlapping gunzip 1147 ms, parse 4206 ms, sort 0 ms, write 0 ms, index 525 ms: 11 candidates, 67145 repo-hours, 946 releases; waiting 535 ms; commit 0 ms)
2026-11-08T08:35:21Z [load_day 2024-10-29] 2024-10-29: repo_hour merged from 24 files to 1 (2268110 rows, 9529 ms)
2026-11-08T08:35:23Z [load_day 2024-10-29] 2024-10-29: index committed in 11981 ms (24 + 1 + 24 files)
2026-11-08T08:35:23Z [load_day 2024-10-29] 2024-10-29: 5536454 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:35:23Z [load_day 2024-10-29] attempt 1 succeeded