orcaLog in

Runs · run 3111

load_day 2024-10-21, attempt 1 succeeded

Host
73ccef5e07c2:195803
Started
2026-11-08T08:26:22Z
Ended
2026-11-08T08:28:19Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
809 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
185.7 s, up to 2.8 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3111/load_day/2024-10-21/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T08:26:22Z [load_day 2024-10-21] attempt 1 started
2026-11-08T08:26:23Z [load_day 2024-10-21] 2024-10-21: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:26:29Z [load_day 2024-10-21] 2024-10-21 hour 0: 184933 events in 0 files, snapshot none (task 5780 ms: download 3217 ms overlapping gunzip 907 ms, parse 3275 ms, sort 0 ms, write 0 ms, index 891 ms: 8 candidates, 84925 repo-hours, 999 releases; waiting 486 ms; commit 0 ms)
2026-11-08T08:26:29Z [load_day 2024-10-21] 2024-10-21 hour 1: 204924 events in 0 files, snapshot none (task 6009 ms: download 3243 ms overlapping gunzip 900 ms, parse 3287 ms, sort 0 ms, write 0 ms, index 1059 ms: 7 candidates, 106792 repo-hours, 915 releases; waiting 486 ms; commit 0 ms)
2026-11-08T08:26:35Z [load_day 2024-10-21] 2024-10-21 hour 2: 177958 events in 0 files, snapshot none (task 5417 ms: download 2896 ms overlapping gunzip 810 ms, parse 2993 ms, sort 0 ms, write 0 ms, index 846 ms: 6 candidates, 77594 repo-hours, 906 releases; waiting 546 ms; commit 0 ms)
2026-11-08T08:26:36Z [load_day 2024-10-21] 2024-10-21 hour 3: 207367 events in 0 files, snapshot none (task 6467 ms: download 3372 ms overlapping gunzip 1016 ms, parse 3636 ms, sort 0 ms, write 0 ms, index 796 ms: 9 candidates, 78181 repo-hours, 3943 releases; waiting 493 ms; commit 0 ms)
2026-11-08T08:26:42Z [load_day 2024-10-21] 2024-10-21 hour 4: 192839 events in 0 files, snapshot none (task 6645 ms: download 3626 ms overlapping gunzip 1143 ms, parse 3890 ms, sort 0 ms, write 0 ms, index 678 ms: 2 candidates, 65806 repo-hours, 672 releases; waiting 611 ms; commit 0 ms)
2026-11-08T08:26:44Z [load_day 2024-10-21] 2024-10-21 hour 5: 202113 events in 0 files, snapshot none (task 7068 ms: download 4582 ms overlapping gunzip 1227 ms, parse 4361 ms, sort 0 ms, write 0 ms, index 664 ms: 4 candidates, 71023 repo-hours, 793 releases; waiting 525 ms; commit 0 ms)
2026-11-08T08:26:51Z [load_day 2024-10-21] 2024-10-21 hour 6: 233045 events in 0 files, snapshot none (task 8409 ms: download 5455 ms overlapping gunzip 1446 ms, parse 5316 ms, sort 0 ms, write 0 ms, index 832 ms: 8 candidates, 97550 repo-hours, 990 releases; waiting 565 ms; commit 0 ms)
2026-11-08T08:26:53Z [load_day 2024-10-21] 2024-10-21 hour 7: 240846 events in 0 files, snapshot none (task 8566 ms: download 6182 ms overlapping gunzip 1577 ms, parse 5721 ms, sort 0 ms, write 0 ms, index 759 ms: 5 candidates, 91722 repo-hours, 1123 releases; waiting 277 ms; commit 0 ms)
2026-11-08T08:27:00Z [load_day 2024-10-21] 2024-10-21 hour 8: 253255 events in 0 files, snapshot none (task 7905 ms: download 5549 ms overlapping gunzip 1364 ms, parse 5013 ms, sort 0 ms, write 0 ms, index 697 ms: 11 candidates, 98980 repo-hours, 1179 releases; waiting 562 ms; commit 0 ms)
2026-11-08T08:27:01Z [load_day 2024-10-21] 2024-10-21 hour 9: 258828 events in 0 files, snapshot none (task 7819 ms: download 5432 ms overlapping gunzip 1340 ms, parse 4865 ms, sort 0 ms, write 0 ms, index 774 ms: 10 candidates, 104426 repo-hours, 1356 releases; waiting 558 ms; commit 0 ms)
2026-11-08T08:27:09Z [load_day 2024-10-21] 2024-10-21 hour 10: 242450 events in 0 files, snapshot none (task 8469 ms: download 4686 ms overlapping gunzip 1489 ms, parse 5266 ms, sort 0 ms, write 0 ms, index 826 ms: 11 candidates, 96676 repo-hours, 1182 releases; waiting 590 ms; commit 0 ms)
2026-11-08T08:27:10Z [load_day 2024-10-21] 2024-10-21 hour 11: 243634 events in 0 files, snapshot none (task 8010 ms: download 5486 ms overlapping gunzip 1347 ms, parse 5036 ms, sort 0 ms, write 0 ms, index 704 ms: 8 candidates, 93565 repo-hours, 1128 releases; waiting 540 ms; commit 0 ms)
2026-11-08T08:27:18Z [load_day 2024-10-21] 2024-10-21 hour 12: 280216 events in 0 files, snapshot none (task 9048 ms: download 6026 ms overlapping gunzip 1421 ms, parse 5597 ms, sort 0 ms, write 0 ms, index 1139 ms: 15 candidates, 139288 repo-hours, 1371 releases; waiting 546 ms; commit 0 ms)
2026-11-08T08:27:21Z [load_day 2024-10-21] 2024-10-21 hour 13: 281024 events in 0 files, snapshot none (task 10457 ms: download 7020 ms overlapping gunzip 1606 ms, parse 6193 ms, sort 0 ms, write 0 ms, index 1789 ms: 9 candidates, 124685 repo-hours, 1287 releases; waiting 555 ms; commit 0 ms)
2026-11-08T08:27:28Z [load_day 2024-10-21] 2024-10-21 hour 14: 274454 events in 0 files, snapshot none (task 9760 ms: download 6547 ms overlapping gunzip 1639 ms, parse 5984 ms, sort 0 ms, write 0 ms, index 1445 ms: 11 candidates, 112048 repo-hours, 1444 releases; waiting 430 ms; commit 0 ms)
2026-11-08T08:27:31Z [load_day 2024-10-21] 2024-10-21 hour 15: 273583 events in 0 files, snapshot none (task 9522 ms: download 6565 ms overlapping gunzip 1669 ms, parse 6022 ms, sort 0 ms, write 0 ms, index 1075 ms: 13 candidates, 114175 repo-hours, 1407 releases; waiting 494 ms; commit 0 ms)
2026-11-08T08:27:38Z [load_day 2024-10-21] 2024-10-21 hour 16: 254635 events in 0 files, snapshot none (task 9064 ms: download 5471 ms overlapping gunzip 1469 ms, parse 5502 ms, sort 0 ms, write 0 ms, index 1207 ms: 10 candidates, 104202 repo-hours, 1229 releases; waiting 534 ms; commit 0 ms)
2026-11-08T08:27:42Z [load_day 2024-10-21] 2024-10-21 hour 17: 255139 events in 0 files, snapshot none (task 10077 ms: download 6490 ms overlapping gunzip 1631 ms, parse 5895 ms, sort 0 ms, write 0 ms, index 1128 ms: 5 candidates, 97897 repo-hours, 1194 releases; waiting 1080 ms; commit 0 ms)
2026-11-08T08:27:47Z [load_day 2024-10-21] 2024-10-21 hour 18: 263045 events in 0 files, snapshot none (task 8777 ms: download 5332 ms overlapping gunzip 1501 ms, parse 5533 ms, sort 0 ms, write 0 ms, index 1138 ms: 15 candidates, 111012 repo-hours, 1182 releases; waiting 312 ms; commit 0 ms)
2026-11-08T08:27:51Z [load_day 2024-10-21] 2024-10-21 hour 19: 243702 events in 0 files, snapshot none (task 9349 ms: download 6166 ms overlapping gunzip 1574 ms, parse 6097 ms, sort 0 ms, write 0 ms, index 911 ms: 8 candidates, 90075 repo-hours, 1177 releases; waiting 516 ms; commit 0 ms)
2026-11-08T08:27:57Z [load_day 2024-10-21] 2024-10-21 hour 20: 236926 events in 0 files, snapshot none (task 9265 ms: download 5178 ms overlapping gunzip 1568 ms, parse 5651 ms, sort 0 ms, write 0 ms, index 1136 ms: 9 candidates, 86729 repo-hours, 1147 releases; waiting 574 ms; commit 0 ms)
2026-11-08T08:28:00Z [load_day 2024-10-21] 2024-10-21 hour 21: 221257 events in 0 files, snapshot none (task 7796 ms: download 5312 ms overlapping gunzip 1345 ms, parse 4915 ms, sort 0 ms, write 0 ms, index 818 ms: 10 candidates, 80308 repo-hours, 1124 releases; waiting 440 ms; commit 0 ms)
2026-11-08T08:28:05Z [load_day 2024-10-21] 2024-10-21 hour 22: 197380 events in 0 files, snapshot none (task 7109 ms: download 4121 ms overlapping gunzip 1138 ms, parse 4111 ms, sort 0 ms, write 0 ms, index 821 ms: 5 candidates, 68276 repo-hours, 1229 releases; waiting 746 ms; commit 0 ms)
2026-11-08T08:28:07Z [load_day 2024-10-21] 2024-10-21 hour 23: 192527 events in 0 files, snapshot none (task 6235 ms: download 3847 ms overlapping gunzip 1099 ms, parse 3947 ms, sort 0 ms, write 0 ms, index 561 ms: 5 candidates, 66086 repo-hours, 872 releases; waiting 318 ms; commit 0 ms)
2026-11-08T08:28:17Z [load_day 2024-10-21] 2024-10-21: repo_hour merged from 24 files to 1 (2262021 rows, 10101 ms)
2026-11-08T08:28:19Z [load_day 2024-10-21] 2024-10-21: index committed in 12316 ms (24 + 1 + 24 files)
2026-11-08T08:28:19Z [load_day 2024-10-21] 2024-10-21: 5616080 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:28:19Z [load_day 2024-10-21] attempt 1 succeeded