orcaLog in

Runs · run 3226

load_day 2024-11-21, attempt 1 succeeded

Host
73ccef5e07c2:210332
Started
2026-11-08T08:57:08Z
Ended
2026-11-08T08:59:29Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
736 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
233.2 s, up to 3.1 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3226/load_day/2024-11-21/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T08:57:08Z [load_day 2024-11-21] attempt 1 started
2026-11-08T08:57:09Z [load_day 2024-11-21] 2024-11-21: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:57:18Z [load_day 2024-11-21] 2024-11-21 hour 0: 203396 events in 0 files, snapshot none (task 8027 ms: download 4481 ms overlapping gunzip 1104 ms, parse 4825 ms, sort 0 ms, write 0 ms, index 1141 ms: 9 candidates, 93539 repo-hours, 1258 releases; waiting 599 ms; commit 0 ms)
2026-11-08T08:57:18Z [load_day 2024-11-21] 2024-11-21 hour 1: 219927 events in 0 files, snapshot none (task 8145 ms: download 4345 ms overlapping gunzip 1175 ms, parse 5138 ms, sort 0 ms, write 0 ms, index 1224 ms: 3 candidates, 114631 repo-hours, 989 releases; waiting 225 ms; commit 0 ms)
2026-11-08T08:57:26Z [load_day 2024-11-21] 2024-11-21 hour 2: 190464 events in 0 files, snapshot none (task 7776 ms: download 5053 ms overlapping gunzip 1203 ms, parse 5099 ms, sort 0 ms, write 0 ms, index 691 ms: 5 candidates, 84329 repo-hours, 988 releases; waiting 515 ms; commit 0 ms)
2026-11-08T08:57:28Z [load_day 2024-11-21] 2024-11-21 hour 3: 218574 events in 0 files, snapshot none (task 9527 ms: download 6777 ms overlapping gunzip 1611 ms, parse 6266 ms, sort 0 ms, write 0 ms, index 687 ms: 5 candidates, 80716 repo-hours, 3911 releases; waiting 549 ms; commit 0 ms)
2026-11-08T08:57:37Z [load_day 2024-11-21] 2024-11-21 hour 4: 199173 events in 0 files, snapshot none (task 9864 ms: download 6670 ms overlapping gunzip 1621 ms, parse 6622 ms, sort 0 ms, write 0 ms, index 723 ms: 3 candidates, 67591 repo-hours, 716 releases; waiting 521 ms; commit 0 ms)
2026-11-08T08:57:38Z [load_day 2024-11-21] 2024-11-21 hour 5: 208686 events in 0 files, snapshot none (task 9571 ms: download 7151 ms overlapping gunzip 1711 ms, parse 6455 ms, sort 0 ms, write 0 ms, index 568 ms: 8 candidates, 73076 repo-hours, 856 releases; waiting 483 ms; commit 0 ms)
2026-11-08T08:57:48Z [load_day 2024-11-21] 2024-11-21 hour 6: 231773 events in 0 files, snapshot none (task 10274 ms: download 7084 ms overlapping gunzip 1653 ms, parse 6887 ms, sort 0 ms, write 0 ms, index 827 ms: 3 candidates, 92051 repo-hours, 901 releases; waiting 545 ms; commit 0 ms)
2026-11-08T08:57:49Z [load_day 2024-11-21] 2024-11-21 hour 7: 227131 events in 0 files, snapshot none (task 9804 ms: download 7097 ms overlapping gunzip 1664 ms, parse 6693 ms, sort 0 ms, write 0 ms, index 639 ms: 8 candidates, 82704 repo-hours, 798 releases; waiting 527 ms; commit 0 ms)
2026-11-08T08:57:59Z [load_day 2024-11-21] 2024-11-21 hour 8: 239043 events in 0 files, snapshot none (task 10421 ms: download 7436 ms overlapping gunzip 1709 ms, parse 6991 ms, sort 0 ms, write 0 ms, index 885 ms: 10 candidates, 92309 repo-hours, 1278 releases; waiting 529 ms; commit 0 ms)
2026-11-08T08:58:00Z [load_day 2024-11-21] 2024-11-21 hour 9: 252550 events in 0 files, snapshot none (task 10793 ms: download 8029 ms overlapping gunzip 1836 ms, parse 7319 ms, sort 0 ms, write 0 ms, index 1072 ms: 5 candidates, 99725 repo-hours, 1234 releases; waiting 312 ms; commit 0 ms)
2026-11-08T08:58:11Z [load_day 2024-11-21] 2024-11-21 hour 10: 244470 events in 0 files, snapshot none (task 11956 ms: download 8174 ms overlapping gunzip 1977 ms, parse 7998 ms, sort 0 ms, write 0 ms, index 1010 ms: 7 candidates, 94960 repo-hours, 1214 releases; waiting 523 ms; commit 0 ms)
2026-11-08T08:58:13Z [load_day 2024-11-21] 2024-11-21 hour 11: 245275 events in 0 files, snapshot none (task 12407 ms: download 8775 ms overlapping gunzip 2160 ms, parse 8667 ms, sort 0 ms, write 0 ms, index 1078 ms: 5 candidates, 91162 repo-hours, 1075 releases; waiting 181 ms; commit 0 ms)
2026-11-08T08:58:22Z [load_day 2024-11-21] 2024-11-21 hour 12: 267364 events in 0 files, snapshot none (task 10026 ms: download 6099 ms overlapping gunzip 1462 ms, parse 6369 ms, sort 0 ms, write 0 ms, index 1393 ms: 7 candidates, 135174 repo-hours, 1301 releases; waiting 500 ms; commit 0 ms)
2026-11-08T08:58:25Z [load_day 2024-11-21] 2024-11-21 hour 13: 264735 events in 0 files, snapshot none (task 11546 ms: download 8195 ms overlapping gunzip 1885 ms, parse 7973 ms, sort 0 ms, write 0 ms, index 1205 ms: 12 candidates, 115290 repo-hours, 1182 releases; waiting 166 ms; commit 0 ms)
2026-11-08T08:58:32Z [load_day 2024-11-21] 2024-11-21 hour 14: 261442 events in 0 files, snapshot none (task 9348 ms: download 6712 ms overlapping gunzip 1599 ms, parse 6086 ms, sort 0 ms, write 0 ms, index 841 ms: 17 candidates, 104565 repo-hours, 1319 releases; waiting 523 ms; commit 0 ms)
2026-11-08T08:58:35Z [load_day 2024-11-21] 2024-11-21 hour 15: 266833 events in 0 files, snapshot none (task 9459 ms: download 6876 ms overlapping gunzip 1547 ms, parse 6112 ms, sort 0 ms, write 0 ms, index 812 ms: 8 candidates, 109659 repo-hours, 1488 releases; waiting 705 ms; commit 0 ms)
2026-11-08T08:58:42Z [load_day 2024-11-21] 2024-11-21 hour 16: 257988 events in 0 files, snapshot none (task 9766 ms: download 6987 ms overlapping gunzip 1581 ms, parse 6364 ms, sort 0 ms, write 0 ms, index 872 ms: 6 candidates, 103575 repo-hours, 1244 releases; waiting 526 ms; commit 0 ms)
2026-11-08T08:58:46Z [load_day 2024-11-21] 2024-11-21 hour 17: 248482 events in 0 files, snapshot none (task 9788 ms: download 6190 ms overlapping gunzip 1570 ms, parse 6228 ms, sort 0 ms, write 0 ms, index 1095 ms: 5 candidates, 95634 repo-hours, 1196 releases; waiting 612 ms; commit 0 ms)
2026-11-08T08:58:54Z [load_day 2024-11-21] 2024-11-21 hour 18: 251715 events in 0 files, snapshot none (task 11734 ms: download 8451 ms overlapping gunzip 1865 ms, parse 7785 ms, sort 0 ms, write 0 ms, index 1318 ms: 7 candidates, 104556 repo-hours, 1145 releases; waiting 499 ms; commit 0 ms)
2026-11-08T08:58:56Z [load_day 2024-11-21] 2024-11-21 hour 19: 233690 events in 0 files, snapshot none (task 10101 ms: download 7425 ms overlapping gunzip 1684 ms, parse 6881 ms, sort 0 ms, write 0 ms, index 666 ms: 12 candidates, 85190 repo-hours, 1072 releases; waiting 518 ms; commit 0 ms)
2026-11-08T08:59:06Z [load_day 2024-11-21] 2024-11-21 hour 20: 228886 events in 0 files, snapshot none (task 10973 ms: download 7833 ms overlapping gunzip 1948 ms, parse 7300 ms, sort 0 ms, write 0 ms, index 875 ms: 2 candidates, 85244 repo-hours, 955 releases; waiting 517 ms; commit 0 ms)
2026-11-08T08:59:07Z [load_day 2024-11-21] 2024-11-21 hour 21: 223900 events in 0 files, snapshot none (task 10316 ms: download 7733 ms overlapping gunzip 1695 ms, parse 7093 ms, sort 0 ms, write 0 ms, index 755 ms: 4 candidates, 82886 repo-hours, 1083 releases; waiting 468 ms; commit 0 ms)
2026-11-08T08:59:16Z [load_day 2024-11-21] 2024-11-21 hour 22: 209498 events in 0 files, snapshot none (task 9544 ms: download 6787 ms overlapping gunzip 1594 ms, parse 6512 ms, sort 0 ms, write 0 ms, index 712 ms: 5 candidates, 72743 repo-hours, 1154 releases; waiting 509 ms; commit 0 ms)
2026-11-08T08:59:17Z [load_day 2024-11-21] 2024-11-21 hour 23: 198460 events in 0 files, snapshot none (task 9333 ms: download 6806 ms overlapping gunzip 1522 ms, parse 6392 ms, sort 0 ms, write 0 ms, index 586 ms: 8 candidates, 69641 repo-hours, 806 releases; waiting 543 ms; commit 0 ms)
2026-11-08T08:59:27Z [load_day 2024-11-21] 2024-11-21: repo_hour merged from 24 files to 1 (2230950 rows, 9747 ms)
2026-11-08T08:59:29Z [load_day 2024-11-21] 2024-11-21: index committed in 12338 ms (24 + 1 + 24 files)
2026-11-08T08:59:29Z [load_day 2024-11-21] 2024-11-21: 5593455 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:59:29Z [load_day 2024-11-21] attempt 1 succeeded