orcaLog in

Runs · run 3130

load_day 2024-10-26, attempt 1 succeeded

Host
73ccef5e07c2:198199
Started
2026-11-08T08:31:08Z
Ended
2026-11-08T08:32:26Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
664 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
110.9 s, up to 2.8 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3130/load_day/2024-10-26/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T08:31:08Z [load_day 2024-10-26] attempt 1 started
2026-11-08T08:31:09Z [load_day 2024-10-26] 2024-10-26: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:31:16Z [load_day 2024-10-26] 2024-10-26 hour 0: 176653 events in 0 files, snapshot none (task 6491 ms: download 3169 ms overlapping gunzip 980 ms, parse 3882 ms, sort 0 ms, write 0 ms, index 660 ms: 13 candidates, 87863 repo-hours, 1111 releases; waiting 708 ms; commit 0 ms)
2026-11-08T08:31:16Z [load_day 2024-10-26] 2024-10-26 hour 1: 177458 events in 0 files, snapshot none (task 5679 ms: download 2546 ms overlapping gunzip 766 ms, parse 3350 ms, sort 0 ms, write 0 ms, index 839 ms: 7 candidates, 97586 repo-hours, 749 releases; waiting 493 ms; commit 0 ms)
2026-11-08T08:31:21Z [load_day 2024-10-26] 2024-10-26 hour 2: 149068 events in 0 files, snapshot none (task 4442 ms: download 2537 ms overlapping gunzip 591 ms, parse 2295 ms, sort 0 ms, write 0 ms, index 691 ms: 3 candidates, 66169 repo-hours, 664 releases; waiting 530 ms; commit 0 ms)
2026-11-08T08:31:21Z [load_day 2024-10-26] 2024-10-26 hour 3: 173220 events in 0 files, snapshot none (task 4710 ms: download 2358 ms overlapping gunzip 694 ms, parse 2926 ms, sort 0 ms, write 0 ms, index 515 ms: 4 candidates, 66834 repo-hours, 3833 releases; waiting 295 ms; commit 0 ms)
2026-11-08T08:31:26Z [load_day 2024-10-26] 2024-10-26 hour 4: 152601 events in 0 files, snapshot none (task 5160 ms: download 2525 ms overlapping gunzip 797 ms, parse 3159 ms, sort 0 ms, write 0 ms, index 563 ms: 5 candidates, 55679 repo-hours, 591 releases; waiting 381 ms; commit 0 ms)
2026-11-08T08:31:27Z [load_day 2024-10-26] 2024-10-26 hour 5: 158778 events in 0 files, snapshot none (task 5270 ms: download 2290 ms overlapping gunzip 907 ms, parse 3245 ms, sort 0 ms, write 0 ms, index 670 ms: 3 candidates, 56787 repo-hours, 694 releases; waiting 205 ms; commit 0 ms)
2026-11-08T08:31:32Z [load_day 2024-10-26] 2024-10-26 hour 6: 179126 events in 0 files, snapshot none (task 5546 ms: download 2437 ms overlapping gunzip 727 ms, parse 2886 ms, sort 0 ms, write 0 ms, index 1029 ms: 2 candidates, 73001 repo-hours, 1005 releases; waiting 532 ms; commit 0 ms)
2026-11-08T08:31:33Z [load_day 2024-10-26] 2024-10-26 hour 7: 169769 events in 0 files, snapshot none (task 6077 ms: download 3243 ms overlapping gunzip 902 ms, parse 3392 ms, sort 0 ms, write 0 ms, index 503 ms: 2 candidates, 59077 repo-hours, 853 releases; waiting 1053 ms; commit 0 ms)
2026-11-08T08:31:38Z [load_day 2024-10-26] 2024-10-26 hour 8: 167194 events in 0 files, snapshot none (task 4647 ms: download 2319 ms overlapping gunzip 666 ms, parse 2680 ms, sort 0 ms, write 0 ms, index 436 ms: 6 candidates, 63157 repo-hours, 905 releases; waiting 534 ms; commit 0 ms)
2026-11-08T08:31:39Z [load_day 2024-10-26] 2024-10-26 hour 9: 183184 events in 0 files, snapshot none (task 4876 ms: download 2687 ms overlapping gunzip 717 ms, parse 2848 ms, sort 0 ms, write 0 ms, index 510 ms: 6 candidates, 69653 repo-hours, 1063 releases; waiting 510 ms; commit 0 ms)
2026-11-08T08:31:43Z [load_day 2024-10-26] 2024-10-26 hour 10: 173222 events in 0 files, snapshot none (task 5301 ms: download 2244 ms overlapping gunzip 803 ms, parse 3299 ms, sort 0 ms, write 0 ms, index 599 ms: 3 candidates, 64856 repo-hours, 782 releases; waiting 363 ms; commit 0 ms)
2026-11-08T08:31:44Z [load_day 2024-10-26] 2024-10-26 hour 11: 173815 events in 0 files, snapshot none (task 4884 ms: download 2551 ms overlapping gunzip 835 ms, parse 3154 ms, sort 0 ms, write 0 ms, index 452 ms: 7 candidates, 61296 repo-hours, 636 releases; waiting 222 ms; commit 0 ms)
2026-11-08T08:31:50Z [load_day 2024-10-26] 2024-10-26 hour 12: 227899 events in 0 files, snapshot none (task 5838 ms: download 2217 ms overlapping gunzip 687 ms, parse 3026 ms, sort 0 ms, write 0 ms, index 1269 ms: 10 candidates, 112362 repo-hours, 897 releases; waiting 539 ms; commit 0 ms)
2026-11-08T08:31:50Z [load_day 2024-10-26] 2024-10-26 hour 13: 210538 events in 0 files, snapshot none (task 5781 ms: download 3400 ms overlapping gunzip 780 ms, parse 3128 ms, sort 0 ms, write 0 ms, index 1010 ms: 7 candidates, 90771 repo-hours, 844 releases; waiting 558 ms; commit 0 ms)
2026-11-08T08:31:55Z [load_day 2024-10-26] 2024-10-26 hour 14: 201309 events in 0 files, snapshot none (task 5249 ms: download 2761 ms overlapping gunzip 795 ms, parse 3114 ms, sort 0 ms, write 0 ms, index 779 ms: 6 candidates, 77668 repo-hours, 880 releases; waiting 259 ms; commit 0 ms)
2026-11-08T08:31:57Z [load_day 2024-10-26] 2024-10-26 hour 15: 209192 events in 0 files, snapshot none (task 5582 ms: download 2769 ms overlapping gunzip 828 ms, parse 3148 ms, sort 0 ms, write 0 ms, index 778 ms: 3 candidates, 87294 repo-hours, 1142 releases; waiting 529 ms; commit 0 ms)
2026-11-08T08:32:01Z [load_day 2024-10-26] 2024-10-26 hour 16: 199795 events in 0 files, snapshot none (task 5175 ms: download 3437 ms overlapping gunzip 770 ms, parse 3045 ms, sort 0 ms, write 0 ms, index 605 ms: 3 candidates, 82414 repo-hours, 881 releases; waiting 533 ms; commit 0 ms)
2026-11-08T08:32:02Z [load_day 2024-10-26] 2024-10-26 hour 17: 183313 events in 0 files, snapshot none (task 4608 ms: download 2467 ms overlapping gunzip 662 ms, parse 2613 ms, sort 0 ms, write 0 ms, index 527 ms: 7 candidates, 67293 repo-hours, 770 releases; waiting 498 ms; commit 0 ms)
2026-11-08T08:32:07Z [load_day 2024-10-26] 2024-10-26 hour 18: 197528 events in 0 files, snapshot none (task 4811 ms: download 2866 ms overlapping gunzip 654 ms, parse 2651 ms, sort 0 ms, write 0 ms, index 602 ms: 7 candidates, 84383 repo-hours, 976 releases; waiting 493 ms; commit 0 ms)
2026-11-08T08:32:07Z [load_day 2024-10-26] 2024-10-26 hour 19: 169160 events in 0 files, snapshot none (task 3985 ms: download 2236 ms overlapping gunzip 596 ms, parse 2408 ms, sort 0 ms, write 0 ms, index 460 ms: 4 candidates, 58952 repo-hours, 827 releases; waiting 280 ms; commit 0 ms)
2026-11-08T08:32:11Z [load_day 2024-10-26] 2024-10-26 hour 20: 161841 events in 0 files, snapshot none (task 3798 ms: download 1861 ms overlapping gunzip 529 ms, parse 2205 ms, sort 0 ms, write 0 ms, index 411 ms: 10 candidates, 57946 repo-hours, 781 releases; waiting 368 ms; commit 0 ms)
2026-11-08T08:32:11Z [load_day 2024-10-26] 2024-10-26 hour 21: 161382 events in 0 files, snapshot none (task 3971 ms: download 2469 ms overlapping gunzip 529 ms, parse 2127 ms, sort 0 ms, write 0 ms, index 396 ms: 4 candidates, 58142 repo-hours, 992 releases; waiting 568 ms; commit 0 ms)
2026-11-08T08:32:16Z [load_day 2024-10-26] 2024-10-26 hour 22: 141044 events in 0 files, snapshot none (task 4508 ms: download 1883 ms overlapping gunzip 664 ms, parse 2631 ms, sort 0 ms, write 0 ms, index 495 ms: 17 candidates, 48173 repo-hours, 1115 releases; waiting 515 ms; commit 0 ms)
2026-11-08T08:32:16Z [load_day 2024-10-26] 2024-10-26 hour 23: 140133 events in 0 files, snapshot none (task 3878 ms: download 1635 ms overlapping gunzip 523 ms, parse 2213 ms, sort 0 ms, write 0 ms, index 619 ms: 0 candidates, 46542 repo-hours, 695 releases; waiting 262 ms; commit 0 ms)
2026-11-08T08:32:23Z [load_day 2024-10-26] 2024-10-26: repo_hour merged from 24 files to 1 (1693898 rows, 7625 ms)
2026-11-08T08:32:26Z [load_day 2024-10-26] 2024-10-26: index committed in 10597 ms (23 + 1 + 24 files)
2026-11-08T08:32:26Z [load_day 2024-10-26] 2024-10-26: 4237222 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:32:26Z [load_day 2024-10-26] attempt 1 succeeded