orcaLog in

Runs · run 3209

load_day 2024-11-16, attempt 1 succeeded

Host
73ccef5e07c2:208186
Started
2026-11-08T08:51:57Z
Ended
2026-11-08T08:53:41Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
651 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
163.3 s, up to 2.9 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3209/load_day/2024-11-16/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T08:51:57Z [load_day 2024-11-16] attempt 1 started
2026-11-08T08:51:58Z [load_day 2024-11-16] 2024-11-16: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:52:05Z [load_day 2024-11-16] 2024-11-16 hour 0: 172371 events in 0 files, snapshot none (task 6629 ms: download 3493 ms overlapping gunzip 1002 ms, parse 4261 ms, sort 0 ms, write 0 ms, index 693 ms: 3 candidates, 80807 repo-hours, 1182 releases; waiting 395 ms; commit 0 ms)
2026-11-08T08:52:05Z [load_day 2024-11-16] 2024-11-16 hour 1: 181676 events in 0 files, snapshot none (task 6718 ms: download 3314 ms overlapping gunzip 893 ms, parse 4095 ms, sort 0 ms, write 0 ms, index 942 ms: 5 candidates, 98828 repo-hours, 844 releases; waiting 512 ms; commit 0 ms)
2026-11-08T08:52:12Z [load_day 2024-11-16] 2024-11-16 hour 2: 149719 events in 0 files, snapshot none (task 6574 ms: download 2897 ms overlapping gunzip 935 ms, parse 4232 ms, sort 0 ms, write 0 ms, index 593 ms: 5 candidates, 63386 repo-hours, 810 releases; waiting 517 ms; commit 0 ms)
2026-11-08T08:52:14Z [load_day 2024-11-16] 2024-11-16 hour 3: 192224 events in 0 files, snapshot none (task 8540 ms: download 5753 ms overlapping gunzip 1318 ms, parse 5489 ms, sort 0 ms, write 0 ms, index 670 ms: 8 candidates, 63966 repo-hours, 3980 releases; waiting 508 ms; commit 0 ms)
2026-11-08T08:52:20Z [load_day 2024-11-16] 2024-11-16 hour 4: 166595 events in 0 files, snapshot none (task 7245 ms: download 4155 ms overlapping gunzip 1179 ms, parse 4519 ms, sort 0 ms, write 0 ms, index 680 ms: 6 candidates, 53378 repo-hours, 588 releases; waiting 518 ms; commit 0 ms)
2026-11-08T08:52:23Z [load_day 2024-11-16] 2024-11-16 hour 5: 174189 events in 0 files, snapshot none (task 8085 ms: download 5256 ms overlapping gunzip 1411 ms, parse 5334 ms, sort 0 ms, write 0 ms, index 556 ms: 3 candidates, 54138 repo-hours, 652 releases; waiting 472 ms; commit 0 ms)
2026-11-08T08:52:28Z [load_day 2024-11-16] 2024-11-16 hour 6: 193085 events in 0 files, snapshot none (task 7349 ms: download 4213 ms overlapping gunzip 1211 ms, parse 5070 ms, sort 0 ms, write 0 ms, index 595 ms: 3 candidates, 74623 repo-hours, 901 releases; waiting 184 ms; commit 0 ms)
2026-11-08T08:52:30Z [load_day 2024-11-16] 2024-11-16 hour 7: 177330 events in 0 files, snapshot none (task 6044 ms: download 3832 ms overlapping gunzip 987 ms, parse 3849 ms, sort 0 ms, write 0 ms, index 449 ms: 7 candidates, 57431 repo-hours, 789 releases; waiting 509 ms; commit 0 ms)
2026-11-08T08:52:35Z [load_day 2024-11-16] 2024-11-16 hour 8: 175867 events in 0 files, snapshot none (task 6011 ms: download 3670 ms overlapping gunzip 929 ms, parse 3807 ms, sort 0 ms, write 0 ms, index 443 ms: 17 candidates, 61421 repo-hours, 778 releases; waiting 520 ms; commit 0 ms)
2026-11-08T08:52:36Z [load_day 2024-11-16] 2024-11-16 hour 9: 187167 events in 0 files, snapshot none (task 6125 ms: download 3810 ms overlapping gunzip 937 ms, parse 3748 ms, sort 0 ms, write 0 ms, index 560 ms: 4 candidates, 68038 repo-hours, 1000 releases; waiting 565 ms; commit 0 ms)
2026-11-08T08:52:42Z [load_day 2024-11-16] 2024-11-16 hour 10: 184667 events in 0 files, snapshot none (task 6645 ms: download 4405 ms overlapping gunzip 997 ms, parse 3885 ms, sort 0 ms, write 0 ms, index 528 ms: 6 candidates, 64684 repo-hours, 728 releases; waiting 887 ms; commit 0 ms)
2026-11-08T08:52:43Z [load_day 2024-11-16] 2024-11-16 hour 11: 187143 events in 0 files, snapshot none (task 5810 ms: download 3643 ms overlapping gunzip 954 ms, parse 3702 ms, sort 0 ms, write 0 ms, index 449 ms: 5 candidates, 60620 repo-hours, 811 releases; waiting 417 ms; commit 0 ms)
2026-11-08T08:52:48Z [load_day 2024-11-16] 2024-11-16 hour 12: 230347 events in 0 files, snapshot none (task 5298 ms: download 2491 ms overlapping gunzip 660 ms, parse 3105 ms, sort 0 ms, write 0 ms, index 788 ms: 6 candidates, 118350 repo-hours, 866 releases; waiting 509 ms; commit 0 ms)
2026-11-08T08:52:51Z [load_day 2024-11-16] 2024-11-16 hour 13: 220394 events in 0 files, snapshot none (task 7403 ms: download 5388 ms overlapping gunzip 1047 ms, parse 4270 ms, sort 0 ms, write 0 ms, index 879 ms: 7 candidates, 90193 repo-hours, 740 releases; waiting 742 ms; commit 0 ms)
2026-11-08T08:52:56Z [load_day 2024-11-16] 2024-11-16 hour 14: 206909 events in 0 files, snapshot none (task 7255 ms: download 4637 ms overlapping gunzip 1176 ms, parse 4672 ms, sort 0 ms, write 0 ms, index 588 ms: 3 candidates, 74509 repo-hours, 806 releases; waiting 520 ms; commit 0 ms)
2026-11-08T08:53:00Z [load_day 2024-11-16] 2024-11-16 hour 15: 211879 events in 0 files, snapshot none (task 8553 ms: download 5591 ms overlapping gunzip 1380 ms, parse 5412 ms, sort 0 ms, write 0 ms, index 861 ms: 8 candidates, 82354 repo-hours, 911 releases; waiting 543 ms; commit 0 ms)
2026-11-08T08:53:04Z [load_day 2024-11-16] 2024-11-16 hour 16: 205449 events in 0 files, snapshot none (task 8122 ms: download 4902 ms overlapping gunzip 1371 ms, parse 5517 ms, sort 0 ms, write 0 ms, index 705 ms: 5 candidates, 75569 repo-hours, 820 releases; waiting 173 ms; commit 0 ms)
2026-11-08T08:53:08Z [load_day 2024-11-16] 2024-11-16 hour 17: 200481 events in 0 files, snapshot none (task 7621 ms: download 4555 ms overlapping gunzip 1222 ms, parse 4979 ms, sort 0 ms, write 0 ms, index 775 ms: 4 candidates, 69034 repo-hours, 819 releases; waiting 349 ms; commit 0 ms)
2026-11-08T08:53:14Z [load_day 2024-11-16] 2024-11-16 hour 18: 212846 events in 0 files, snapshot none (task 9406 ms: download 5038 ms overlapping gunzip 1467 ms, parse 6101 ms, sort 0 ms, write 0 ms, index 890 ms: 3 candidates, 86326 repo-hours, 965 releases; waiting 558 ms; commit 0 ms)
2026-11-08T08:53:16Z [load_day 2024-11-16] 2024-11-16 hour 19: 186904 events in 0 files, snapshot none (task 7489 ms: download 5054 ms overlapping gunzip 1325 ms, parse 5229 ms, sort 0 ms, write 0 ms, index 462 ms: 4 candidates, 62010 repo-hours, 823 releases; waiting 189 ms; commit 0 ms)
2026-11-08T08:53:22Z [load_day 2024-11-16] 2024-11-16 hour 20: 190449 events in 0 files, snapshot none (task 7516 ms: download 4882 ms overlapping gunzip 1165 ms, parse 5038 ms, sort 0 ms, write 0 ms, index 537 ms: 5 candidates, 62496 repo-hours, 759 releases; waiting 496 ms; commit 0 ms)
2026-11-08T08:53:24Z [load_day 2024-11-16] 2024-11-16 hour 21: 192496 events in 0 files, snapshot none (task 7531 ms: download 5311 ms overlapping gunzip 1303 ms, parse 4886 ms, sort 0 ms, write 0 ms, index 485 ms: 2 candidates, 63969 repo-hours, 1110 releases; waiting 563 ms; commit 0 ms)
2026-11-08T08:53:30Z [load_day 2024-11-16] 2024-11-16 hour 22: 175808 events in 0 files, snapshot none (task 7095 ms: download 4223 ms overlapping gunzip 1166 ms, parse 4668 ms, sort 0 ms, write 0 ms, index 455 ms: 3 candidates, 54236 repo-hours, 1048 releases; waiting 505 ms; commit 0 ms)
2026-11-08T08:53:31Z [load_day 2024-11-16] 2024-11-16 hour 23: 165059 events in 0 files, snapshot none (task 6313 ms: download 4194 ms overlapping gunzip 1014 ms, parse 4168 ms, sort 0 ms, write 0 ms, index 360 ms: 3 candidates, 51544 repo-hours, 695 releases; waiting 564 ms; commit 0 ms)
2026-11-08T08:53:39Z [load_day 2024-11-16] 2024-11-16: repo_hour merged from 24 files to 1 (1691910 rows, 7857 ms)
2026-11-08T08:53:41Z [load_day 2024-11-16] 2024-11-16: index committed in 10620 ms (24 + 1 + 24 files)
2026-11-08T08:53:41Z [load_day 2024-11-16] 2024-11-16: 4541054 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:53:41Z [load_day 2024-11-16] attempt 1 succeeded