orcaLog in

Runs · run 3298

load_day 2024-12-09, attempt 1 succeeded

Host
73ccef5e07c2:219659
Started
2026-11-08T09:16:46Z
Ended
2026-11-08T09:19:06Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
832 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
225.7 s, up to 3.2 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3298/load_day/2024-12-09/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T09:16:46Z [load_day 2024-12-09] attempt 1 started
2026-11-08T09:16:47Z [load_day 2024-12-09] 2024-12-09: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T09:16:55Z [load_day 2024-12-09] 2024-12-09 hour 0: 186911 events in 0 files, snapshot none (task 7856 ms: download 5383 ms overlapping gunzip 1409 ms, parse 5202 ms, sort 0 ms, write 0 ms, index 617 ms: 3 candidates, 81226 repo-hours, 1178 releases; waiting 338 ms; commit 0 ms)
2026-11-08T09:16:56Z [load_day 2024-12-09] 2024-12-09 hour 1: 216901 events in 0 files, snapshot none (task 9037 ms: download 6185 ms overlapping gunzip 1530 ms, parse 5707 ms, sort 0 ms, write 0 ms, index 934 ms: 4 candidates, 112663 repo-hours, 922 releases; waiting 558 ms; commit 0 ms)
2026-11-08T09:17:03Z [load_day 2024-12-09] 2024-12-09 hour 2: 190487 events in 0 files, snapshot none (task 7745 ms: download 4440 ms overlapping gunzip 1294 ms, parse 4756 ms, sort 0 ms, write 0 ms, index 1075 ms: 5 candidates, 80990 repo-hours, 1063 releases; waiting 301 ms; commit 0 ms)
2026-11-08T09:17:05Z [load_day 2024-12-09] 2024-12-09 hour 3: 214664 events in 0 files, snapshot none (task 8154 ms: download 5533 ms overlapping gunzip 1411 ms, parse 5201 ms, sort 0 ms, write 0 ms, index 730 ms: 12 candidates, 94695 repo-hours, 4004 releases; waiting 457 ms; commit 0 ms)
2026-11-08T09:17:11Z [load_day 2024-12-09] 2024-12-09 hour 4: 193750 events in 0 files, snapshot none (task 7426 ms: download 4208 ms overlapping gunzip 1252 ms, parse 4858 ms, sort 0 ms, write 0 ms, index 616 ms: 2 candidates, 74394 repo-hours, 810 releases; waiting 332 ms; commit 0 ms)
2026-11-08T09:17:12Z [load_day 2024-12-09] 2024-12-09 hour 5: 192802 events in 0 files, snapshot none (task 7034 ms: download 4523 ms overlapping gunzip 1123 ms, parse 4761 ms, sort 0 ms, write 0 ms, index 617 ms: 9 candidates, 74294 repo-hours, 790 releases; waiting 256 ms; commit 0 ms)
2026-11-08T09:17:21Z [load_day 2024-12-09] 2024-12-09 hour 6: 219648 events in 0 files, snapshot none (task 8776 ms: download 6136 ms overlapping gunzip 1533 ms, parse 5665 ms, sort 0 ms, write 0 ms, index 816 ms: 4 candidates, 95370 repo-hours, 963 releases; waiting 466 ms; commit 0 ms)
2026-11-08T09:17:22Z [load_day 2024-12-09] 2024-12-09 hour 7: 214670 events in 0 files, snapshot none (task 9465 ms: download 6877 ms overlapping gunzip 1542 ms, parse 6261 ms, sort 0 ms, write 0 ms, index 755 ms: 6 candidates, 85656 repo-hours, 1045 releases; waiting 535 ms; commit 0 ms)
2026-11-08T09:17:34Z [load_day 2024-12-09] 2024-12-09 hour 8: 231580 events in 0 files, snapshot none (task 13267 ms: download 9674 ms overlapping gunzip 2258 ms, parse 9485 ms, sort 0 ms, write 0 ms, index 923 ms: 9 candidates, 95380 repo-hours, 1167 releases; waiting 332 ms; commit 0 ms)
2026-11-08T09:17:36Z [load_day 2024-12-09] 2024-12-09 hour 9: 243331 events in 0 files, snapshot none (task 13201 ms: download 10611 ms overlapping gunzip 2128 ms, parse 8115 ms, sort 0 ms, write 0 ms, index 742 ms: 8 candidates, 102605 repo-hours, 1363 releases; waiting 1875 ms; commit 0 ms)
2026-11-08T09:17:45Z [load_day 2024-12-09] 2024-12-09 hour 10: 236504 events in 0 files, snapshot none (task 10061 ms: download 7032 ms overlapping gunzip 1679 ms, parse 6716 ms, sort 0 ms, write 0 ms, index 797 ms: 19 candidates, 97346 repo-hours, 1338 releases; waiting 559 ms; commit 0 ms)
2026-11-08T09:17:48Z [load_day 2024-12-09] 2024-12-09 hour 11: 239074 events in 0 files, snapshot none (task 10964 ms: download 7986 ms overlapping gunzip 1744 ms, parse 6870 ms, sort 0 ms, write 0 ms, index 1083 ms: 10 candidates, 92755 repo-hours, 1141 releases; waiting 937 ms; commit 0 ms)
2026-11-08T09:17:56Z [load_day 2024-12-09] 2024-12-09 hour 12: 274371 events in 0 files, snapshot none (task 10844 ms: download 7946 ms overlapping gunzip 1802 ms, parse 7420 ms, sort 0 ms, write 0 ms, index 1022 ms: 7 candidates, 137649 repo-hours, 1391 releases; waiting 345 ms; commit 0 ms)
2026-11-08T09:17:59Z [load_day 2024-12-09] 2024-12-09 hour 13: 262146 events in 0 files, snapshot none (task 9805 ms: download 7131 ms overlapping gunzip 1617 ms, parse 6578 ms, sort 0 ms, write 0 ms, index 838 ms: 9 candidates, 121205 repo-hours, 1204 releases; waiting 425 ms; commit 0 ms)
2026-11-08T09:18:05Z [load_day 2024-12-09] 2024-12-09 hour 14: 243510 events in 0 files, snapshot none (task 8331 ms: download 5804 ms overlapping gunzip 1352 ms, parse 5336 ms, sort 0 ms, write 0 ms, index 758 ms: 9 candidates, 103365 repo-hours, 1402 releases; waiting 587 ms; commit 0 ms)
2026-11-08T09:18:10Z [load_day 2024-12-09] 2024-12-09 hour 15: 270032 events in 0 files, snapshot none (task 11107 ms: download 6813 ms overlapping gunzip 1636 ms, parse 6665 ms, sort 0 ms, write 0 ms, index 1943 ms: 6 candidates, 116277 repo-hours, 1560 releases; waiting 532 ms; commit 0 ms)
2026-11-08T09:18:19Z [load_day 2024-12-09] 2024-12-09 hour 16: 257260 events in 0 files, snapshot none (task 12661 ms: download 8721 ms overlapping gunzip 2019 ms, parse 8275 ms, sort 0 ms, write 0 ms, index 1560 ms: 11 candidates, 108442 repo-hours, 1435 releases; waiting 510 ms; commit 0 ms)
2026-11-08T09:18:22Z [load_day 2024-12-09] 2024-12-09 hour 17: 253475 events in 0 files, snapshot none (task 11062 ms: download 8167 ms overlapping gunzip 1913 ms, parse 7453 ms, sort 0 ms, write 0 ms, index 880 ms: 14 candidates, 101108 repo-hours, 1086 releases; waiting 502 ms; commit 0 ms)
2026-11-08T09:18:31Z [load_day 2024-12-09] 2024-12-09 hour 18: 250050 events in 0 files, snapshot none (task 11901 ms: download 8644 ms overlapping gunzip 1945 ms, parse 7990 ms, sort 0 ms, write 0 ms, index 826 ms: 8 candidates, 109477 repo-hours, 1205 releases; waiting 823 ms; commit 0 ms)
2026-11-08T09:18:33Z [load_day 2024-12-09] 2024-12-09 hour 19: 234796 events in 0 files, snapshot none (task 10542 ms: download 8024 ms overlapping gunzip 1847 ms, parse 7171 ms, sort 0 ms, write 0 ms, index 718 ms: 4 candidates, 90245 repo-hours, 1201 releases; waiting 528 ms; commit 0 ms)
2026-11-08T09:18:42Z [load_day 2024-12-09] 2024-12-09 hour 20: 231906 events in 0 files, snapshot none (task 10392 ms: download 7722 ms overlapping gunzip 1772 ms, parse 6896 ms, sort 0 ms, write 0 ms, index 857 ms: 9 candidates, 89863 repo-hours, 1042 releases; waiting 524 ms; commit 0 ms)
2026-11-08T09:18:43Z [load_day 2024-12-09] 2024-12-09 hour 21: 223723 events in 0 files, snapshot none (task 9349 ms: download 7015 ms overlapping gunzip 1530 ms, parse 6381 ms, sort 0 ms, write 0 ms, index 622 ms: 20 candidates, 87560 repo-hours, 1329 releases; waiting 536 ms; commit 0 ms)
2026-11-08T09:18:52Z [load_day 2024-12-09] 2024-12-09 hour 22: 208902 events in 0 files, snapshot none (task 9247 ms: download 6843 ms overlapping gunzip 1481 ms, parse 6284 ms, sort 0 ms, write 0 ms, index 639 ms: 7 candidates, 77818 repo-hours, 1424 releases; waiting 511 ms; commit 0 ms)
2026-11-08T09:18:52Z [load_day 2024-12-09] 2024-12-09 hour 23: 193561 events in 0 files, snapshot none (task 8510 ms: download 6226 ms overlapping gunzip 1350 ms, parse 5739 ms, sort 0 ms, write 0 ms, index 541 ms: 9 candidates, 73109 repo-hours, 837 releases; waiting 564 ms; commit 0 ms)
2026-11-08T09:19:03Z [load_day 2024-12-09] 2024-12-09: repo_hour merged from 24 files to 1 (2303492 rows, 11189 ms)
2026-11-08T09:19:06Z [load_day 2024-12-09] 2024-12-09: index committed in 14118 ms (24 + 1 + 24 files)
2026-11-08T09:19:06Z [load_day 2024-12-09] 2024-12-09: 5484054 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T09:19:06Z [load_day 2024-12-09] attempt 1 succeeded