orcaLog in

Runs · run 2987

load_day 2024-09-21, attempt 1 succeeded

Host
73ccef5e07c2:180901
Started
2026-11-08T08:01:21Z
Ended
2026-11-08T08:02:54Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
656 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
119.7 s, up to 2.8 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/2987/load_day/2024-09-21/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T08:01:21Z [load_day 2024-09-21] attempt 1 started
2026-11-08T08:01:22Z [load_day 2024-09-21] 2024-09-21: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:01:29Z [load_day 2024-09-21] 2024-09-21 hour 0: 156780 events in 0 files, snapshot none (task 6265 ms: download 2368 ms overlapping gunzip 893 ms, parse 4057 ms, sort 0 ms, write 0 ms, index 772 ms: 7 candidates, 77943 repo-hours, 996 releases; waiting 263 ms; commit 0 ms)
2026-11-08T08:01:29Z [load_day 2024-09-21] 2024-09-21 hour 1: 164617 events in 0 files, snapshot none (task 6604 ms: download 2505 ms overlapping gunzip 901 ms, parse 3701 ms, sort 0 ms, write 0 ms, index 1212 ms: 8 candidates, 88728 repo-hours, 678 releases; waiting 482 ms; commit 0 ms)
2026-11-08T08:01:37Z [load_day 2024-09-21] 2024-09-21 hour 2: 135380 events in 0 files, snapshot none (task 7334 ms: download 3165 ms overlapping gunzip 811 ms, parse 3320 ms, sort 0 ms, write 0 ms, index 2329 ms: 14 candidates, 59604 repo-hours, 666 releases; waiting 516 ms; commit 0 ms)
2026-11-08T08:01:37Z [load_day 2024-09-21] 2024-09-21 hour 3: 153330 events in 0 files, snapshot none (task 5309 ms: download 2390 ms overlapping gunzip 845 ms, parse 3385 ms, sort 0 ms, write 0 ms, index 433 ms: 2 candidates, 60710 repo-hours, 3822 releases; waiting 343 ms; commit 0 ms)
2026-11-08T08:01:49Z [load_day 2024-09-21] 2024-09-21 hour 4: 136936 events in 0 files, snapshot none (task 5294 ms: download 2761 ms overlapping gunzip 738 ms, parse 3149 ms, sort 0 ms, write 0 ms, index 623 ms: 9 candidates, 51120 repo-hours, 573 releases; waiting 523 ms; commit 0 ms)
2026-11-08T08:01:49Z [load_day 2024-09-21] 2024-09-21 hour 5: 143130 events in 0 files, snapshot none (task 5716 ms: download 3063 ms overlapping gunzip 781 ms, parse 3213 ms, sort 0 ms, write 0 ms, index 608 ms: 6 candidates, 53421 repo-hours, 691 releases; waiting 726 ms; commit 0 ms)
2026-11-08T08:01:56Z [load_day 2024-09-21] 2024-09-21 hour 6: 178046 events in 0 files, snapshot none (task 6236 ms: download 3852 ms overlapping gunzip 1005 ms, parse 3798 ms, sort 0 ms, write 0 ms, index 560 ms: 6 candidates, 76536 repo-hours, 1092 releases; waiting 507 ms; commit 0 ms)
2026-11-08T08:01:56Z [load_day 2024-09-21] 2024-09-21 hour 7: 160701 events in 0 files, snapshot none (task 5767 ms: download 3512 ms overlapping gunzip 932 ms, parse 3604 ms, sort 0 ms, write 0 ms, index 478 ms: 6 candidates, 60595 repo-hours, 824 releases; waiting 506 ms; commit 0 ms)
2026-11-08T08:02:02Z [load_day 2024-09-21] 2024-09-21 hour 8: 165823 events in 0 files, snapshot none (task 6054 ms: download 3470 ms overlapping gunzip 906 ms, parse 3659 ms, sort 0 ms, write 0 ms, index 515 ms: 17 candidates, 63873 repo-hours, 949 releases; waiting 706 ms; commit 0 ms)
2026-11-08T08:02:03Z [load_day 2024-09-21] 2024-09-21 hour 9: 179139 events in 0 files, snapshot none (task 6277 ms: download 3666 ms overlapping gunzip 919 ms, parse 3896 ms, sort 0 ms, write 0 ms, index 644 ms: 5 candidates, 69683 repo-hours, 1260 releases; waiting 497 ms; commit 0 ms)
2026-11-08T08:02:09Z [load_day 2024-09-21] 2024-09-21 hour 10: 166253 events in 0 files, snapshot none (task 6729 ms: download 3951 ms overlapping gunzip 978 ms, parse 4270 ms, sort 0 ms, write 0 ms, index 630 ms: 19 candidates, 64592 repo-hours, 871 releases; waiting 570 ms; commit 0 ms)
2026-11-08T08:02:09Z [load_day 2024-09-21] 2024-09-21 hour 11: 166288 events in 0 files, snapshot none (task 6391 ms: download 3594 ms overlapping gunzip 1062 ms, parse 4171 ms, sort 0 ms, write 0 ms, index 525 ms: 9 candidates, 62037 repo-hours, 725 releases; waiting 328 ms; commit 0 ms)
2026-11-08T08:02:16Z [load_day 2024-09-21] 2024-09-21 hour 12: 232626 events in 0 files, snapshot none (task 6556 ms: download 2920 ms overlapping gunzip 909 ms, parse 3995 ms, sort 0 ms, write 0 ms, index 847 ms: 7 candidates, 110998 repo-hours, 857 releases; waiting 531 ms; commit 0 ms)
2026-11-08T08:02:17Z [load_day 2024-09-21] 2024-09-21 hour 13: 199974 events in 0 files, snapshot none (task 7395 ms: download 4434 ms overlapping gunzip 1012 ms, parse 4127 ms, sort 0 ms, write 0 ms, index 651 ms: 11 candidates, 91099 repo-hours, 870 releases; waiting 1136 ms; commit 0 ms)
2026-11-08T08:02:23Z [load_day 2024-09-21] 2024-09-21 hour 14: 183413 events in 0 files, snapshot none (task 6692 ms: download 4352 ms overlapping gunzip 823 ms, parse 3404 ms, sort 0 ms, write 0 ms, index 846 ms: 11 candidates, 72490 repo-hours, 868 releases; waiting 1171 ms; commit 0 ms)
2026-11-08T08:02:23Z [load_day 2024-09-21] 2024-09-21 hour 15: 189147 events in 0 files, snapshot none (task 5552 ms: download 2479 ms overlapping gunzip 764 ms, parse 3288 ms, sort 0 ms, write 0 ms, index 763 ms: 10 candidates, 80720 repo-hours, 1100 releases; waiting 479 ms; commit 0 ms)
2026-11-08T08:02:29Z [load_day 2024-09-21] 2024-09-21 hour 16: 178750 events in 0 files, snapshot none (task 5120 ms: download 3318 ms overlapping gunzip 686 ms, parse 2853 ms, sort 0 ms, write 0 ms, index 555 ms: 6 candidates, 71345 repo-hours, 883 releases; waiting 752 ms; commit 0 ms)
2026-11-08T08:02:29Z [load_day 2024-09-21] 2024-09-21 hour 17: 171293 events in 0 files, snapshot none (task 4569 ms: download 2467 ms overlapping gunzip 621 ms, parse 2555 ms, sort 0 ms, write 0 ms, index 511 ms: 15 candidates, 66216 repo-hours, 780 releases; waiting 504 ms; commit 0 ms)
2026-11-08T08:02:34Z [load_day 2024-09-21] 2024-09-21 hour 18: 189783 events in 0 files, snapshot none (task 4815 ms: download 2821 ms overlapping gunzip 638 ms, parse 2770 ms, sort 0 ms, write 0 ms, index 589 ms: 9 candidates, 81974 repo-hours, 1000 releases; waiting 519 ms; commit 0 ms)
2026-11-08T08:02:34Z [load_day 2024-09-21] 2024-09-21 hour 19: 164981 events in 0 files, snapshot none (task 4327 ms: download 2255 ms overlapping gunzip 644 ms, parse 2644 ms, sort 0 ms, write 0 ms, index 423 ms: 10 candidates, 57886 repo-hours, 863 releases; waiting 366 ms; commit 0 ms)
2026-11-08T08:02:39Z [load_day 2024-09-21] 2024-09-21 hour 20: 160036 events in 0 files, snapshot none (task 4431 ms: download 2584 ms overlapping gunzip 586 ms, parse 2470 ms, sort 0 ms, write 0 ms, index 404 ms: 5 candidates, 57289 repo-hours, 823 releases; waiting 701 ms; commit 0 ms)
2026-11-08T08:02:39Z [load_day 2024-09-21] 2024-09-21 hour 21: 152097 events in 0 files, snapshot none (task 4117 ms: download 2283 ms overlapping gunzip 567 ms, parse 2342 ms, sort 0 ms, write 0 ms, index 432 ms: 9 candidates, 57575 repo-hours, 871 releases; waiting 522 ms; commit 0 ms)
2026-11-08T08:02:44Z [load_day 2024-09-21] 2024-09-21 hour 22: 133793 events in 0 files, snapshot none (task 4590 ms: download 2341 ms overlapping gunzip 643 ms, parse 2774 ms, sort 0 ms, write 0 ms, index 347 ms: 12 candidates, 45998 repo-hours, 942 releases; waiting 521 ms; commit 0 ms)
2026-11-08T08:02:44Z [load_day 2024-09-21] 2024-09-21 hour 23: 130003 events in 0 files, snapshot none (task 4351 ms: download 2179 ms overlapping gunzip 658 ms, parse 2662 ms, sort 0 ms, write 0 ms, index 325 ms: 6 candidates, 44383 repo-hours, 729 releases; waiting 447 ms; commit 0 ms)
2026-11-08T08:02:51Z [load_day 2024-09-21] 2024-09-21: repo_hour merged from 24 files to 1 (1626815 rows, 7296 ms)
2026-11-08T08:02:54Z [load_day 2024-09-21] 2024-09-21: index committed in 9613 ms (24 + 1 + 24 files)
2026-11-08T08:02:54Z [load_day 2024-09-21] 2024-09-21: 3992319 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:02:54Z [load_day 2024-09-21] attempt 1 succeeded