orcaLog in

Runs · run 2908

load_day 2024-08-31, attempt 1 succeeded

Host
73ccef5e07c2:171389
Started
2026-11-08T07:41:17Z
Ended
2026-11-08T07:42:40Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
655 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
112.8 s, up to 2.6 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/2908/load_day/2024-08-31/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T07:41:17Z [load_day 2024-08-31] attempt 1 started
2026-11-08T07:41:17Z [load_day 2024-08-31] 2024-08-31: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T07:41:23Z [load_day 2024-08-31] 2024-08-31 hour 0: 160005 events in 0 files, snapshot none (task 5155 ms: download 2507 ms overlapping gunzip 758 ms, parse 3129 ms, sort 0 ms, write 0 ms, index 696 ms: 8 candidates, 79024 repo-hours, 1120 releases; waiting 234 ms; commit 0 ms)
2026-11-08T07:41:24Z [load_day 2024-08-31] 2024-08-31 hour 1: 167377 events in 0 files, snapshot none (task 6203 ms: download 3703 ms overlapping gunzip 778 ms, parse 3324 ms, sort 0 ms, write 0 ms, index 1037 ms: 4 candidates, 91287 repo-hours, 860 releases; waiting 730 ms; commit 0 ms)
2026-11-08T07:41:29Z [load_day 2024-08-31] 2024-08-31 hour 2: 131428 events in 0 files, snapshot none (task 5196 ms: download 2465 ms overlapping gunzip 758 ms, parse 3237 ms, sort 0 ms, write 0 ms, index 388 ms: 4 candidates, 55423 repo-hours, 703 releases; waiting 498 ms; commit 0 ms)
2026-11-08T07:41:30Z [load_day 2024-08-31] 2024-08-31 hour 3: 154490 events in 0 files, snapshot none (task 4734 ms: download 2532 ms overlapping gunzip 633 ms, parse 2822 ms, sort 0 ms, write 0 ms, index 446 ms: 3 candidates, 61362 repo-hours, 5297 releases; waiting 510 ms; commit 0 ms)
2026-11-08T07:41:35Z [load_day 2024-08-31] 2024-08-31 hour 4: 133418 events in 0 files, snapshot none (task 5145 ms: download 2885 ms overlapping gunzip 812 ms, parse 2960 ms, sort 0 ms, write 0 ms, index 509 ms: 1 candidates, 51010 repo-hours, 584 releases; waiting 564 ms; commit 0 ms)
2026-11-08T07:41:35Z [load_day 2024-08-31] 2024-08-31 hour 5: 141599 events in 0 files, snapshot none (task 5113 ms: download 2669 ms overlapping gunzip 703 ms, parse 2999 ms, sort 0 ms, write 0 ms, index 573 ms: 3 candidates, 53623 repo-hours, 713 releases; waiting 538 ms; commit 0 ms)
2026-11-08T07:41:40Z [load_day 2024-08-31] 2024-08-31 hour 6: 165862 events in 0 files, snapshot none (task 5052 ms: download 2695 ms overlapping gunzip 723 ms, parse 3013 ms, sort 0 ms, write 0 ms, index 523 ms: 4 candidates, 73190 repo-hours, 953 releases; waiting 517 ms; commit 0 ms)
2026-11-08T07:41:41Z [load_day 2024-08-31] 2024-08-31 hour 7: 154230 events in 0 files, snapshot none (task 4838 ms: download 2542 ms overlapping gunzip 738 ms, parse 3165 ms, sort 0 ms, write 0 ms, index 439 ms: 3 candidates, 57719 repo-hours, 913 releases; waiting 197 ms; commit 0 ms)
2026-11-08T07:41:47Z [load_day 2024-08-31] 2024-08-31 hour 8: 158722 events in 0 files, snapshot none (task 5845 ms: download 2912 ms overlapping gunzip 848 ms, parse 3700 ms, sort 0 ms, write 0 ms, index 543 ms: 3 candidates, 61142 repo-hours, 902 releases; waiting 486 ms; commit 0 ms)
2026-11-08T07:41:47Z [load_day 2024-08-31] 2024-08-31 hour 9: 167902 events in 0 files, snapshot none (task 5974 ms: download 2917 ms overlapping gunzip 853 ms, parse 3767 ms, sort 0 ms, write 0 ms, index 528 ms: 3 candidates, 67676 repo-hours, 1029 releases; waiting 522 ms; commit 0 ms)
2026-11-08T07:41:53Z [load_day 2024-08-31] 2024-08-31 hour 10: 159643 events in 0 files, snapshot none (task 5758 ms: download 2642 ms overlapping gunzip 718 ms, parse 3140 ms, sort 0 ms, write 0 ms, index 814 ms: 18 candidates, 61633 repo-hours, 810 releases; waiting 681 ms; commit 0 ms)
2026-11-08T07:41:53Z [load_day 2024-08-31] 2024-08-31 hour 11: 158056 events in 0 files, snapshot none (task 5443 ms: download 2902 ms overlapping gunzip 689 ms, parse 3233 ms, sort 0 ms, write 0 ms, index 664 ms: 4 candidates, 58482 repo-hours, 723 releases; waiting 493 ms; commit 0 ms)
2026-11-08T07:41:59Z [load_day 2024-08-31] 2024-08-31 hour 12: 229922 events in 0 files, snapshot none (task 6325 ms: download 3427 ms overlapping gunzip 821 ms, parse 4050 ms, sort 0 ms, write 0 ms, index 694 ms: 6 candidates, 107937 repo-hours, 888 releases; waiting 538 ms; commit 0 ms)
2026-11-08T07:41:59Z [load_day 2024-08-31] 2024-08-31 hour 13: 193491 events in 0 files, snapshot none (task 5874 ms: download 3264 ms overlapping gunzip 858 ms, parse 3808 ms, sort 0 ms, write 0 ms, index 612 ms: 10 candidates, 87002 repo-hours, 830 releases; waiting 286 ms; commit 0 ms)
2026-11-08T07:42:07Z [load_day 2024-08-31] 2024-08-31 hour 14: 181538 events in 0 files, snapshot none (task 7138 ms: download 4340 ms overlapping gunzip 1066 ms, parse 4719 ms, sort 0 ms, write 0 ms, index 525 ms: 7 candidates, 71391 repo-hours, 956 releases; waiting 508 ms; commit 0 ms)
2026-11-08T07:42:07Z [load_day 2024-08-31] 2024-08-31 hour 15: 188400 events in 0 files, snapshot none (task 6514 ms: download 4289 ms overlapping gunzip 935 ms, parse 3916 ms, sort 0 ms, write 0 ms, index 665 ms: 15 candidates, 80328 repo-hours, 983 releases; waiting 504 ms; commit 0 ms)
2026-11-08T07:42:13Z [load_day 2024-08-31] 2024-08-31 hour 16: 173434 events in 0 files, snapshot none (task 5662 ms: download 3225 ms overlapping gunzip 778 ms, parse 3358 ms, sort 0 ms, write 0 ms, index 744 ms: 6 candidates, 72047 repo-hours, 835 releases; waiting 513 ms; commit 0 ms)
2026-11-08T07:42:14Z [load_day 2024-08-31] 2024-08-31 hour 17: 168380 events in 0 files, snapshot none (task 6009 ms: download 4079 ms overlapping gunzip 851 ms, parse 3687 ms, sort 0 ms, write 0 ms, index 563 ms: 18 candidates, 63565 repo-hours, 726 releases; waiting 599 ms; commit 0 ms)
2026-11-08T07:42:19Z [load_day 2024-08-31] 2024-08-31 hour 18: 181207 events in 0 files, snapshot none (task 5744 ms: download 2561 ms overlapping gunzip 707 ms, parse 3248 ms, sort 0 ms, write 0 ms, index 926 ms: 12 candidates, 79305 repo-hours, 912 releases; waiting 505 ms; commit 0 ms)
2026-11-08T07:42:19Z [load_day 2024-08-31] 2024-08-31 hour 19: 155298 events in 0 files, snapshot none (task 4777 ms: download 2598 ms overlapping gunzip 640 ms, parse 2677 ms, sort 0 ms, write 0 ms, index 537 ms: 9 candidates, 57863 repo-hours, 820 releases; waiting 607 ms; commit 0 ms)
2026-11-08T07:42:24Z [load_day 2024-08-31] 2024-08-31 hour 20: 150075 events in 0 files, snapshot none (task 4344 ms: download 1926 ms overlapping gunzip 616 ms, parse 2538 ms, sort 0 ms, write 0 ms, index 451 ms: 5 candidates, 56232 repo-hours, 868 releases; waiting 478 ms; commit 0 ms)
2026-11-08T07:42:25Z [load_day 2024-08-31] 2024-08-31 hour 21: 143909 events in 0 files, snapshot none (task 5244 ms: download 2176 ms overlapping gunzip 751 ms, parse 3220 ms, sort 0 ms, write 0 ms, index 387 ms: 1 candidates, 55904 repo-hours, 938 releases; waiting 495 ms; commit 0 ms)
2026-11-08T07:42:29Z [load_day 2024-08-31] 2024-08-31 hour 22: 125302 events in 0 files, snapshot none (task 4823 ms: download 3127 ms overlapping gunzip 650 ms, parse 2788 ms, sort 0 ms, write 0 ms, index 368 ms: 1 candidates, 44954 repo-hours, 942 releases; waiting 541 ms; commit 0 ms)
2026-11-08T07:42:30Z [load_day 2024-08-31] 2024-08-31 hour 23: 121048 events in 0 files, snapshot none (task 4859 ms: download 2275 ms overlapping gunzip 738 ms, parse 2995 ms, sort 0 ms, write 0 ms, index 367 ms: 2 candidates, 43071 repo-hours, 652 releases; waiting 475 ms; commit 0 ms)
2026-11-08T07:42:37Z [load_day 2024-08-31] 2024-08-31: repo_hour merged from 24 files to 1 (1591170 rows, 6938 ms)
2026-11-08T07:42:40Z [load_day 2024-08-31] 2024-08-31: index committed in 9259 ms (24 + 1 + 24 files)
2026-11-08T07:42:40Z [load_day 2024-08-31] 2024-08-31: 3864736 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T07:42:40Z [load_day 2024-08-31] attempt 1 succeeded