orcaLog in

Runs · run 1678

load_day 2023-10-29, attempt 1 succeeded

Host
3c2b29713ce8:495216
Started
2026-11-08T02:17:05Z
Ended
2026-11-08T02:19:44Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1678/load_day/2023-10-29/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:17:05Z [load_day 2023-10-29] attempt 1 started
2026-11-08T02:17:06Z [load_day 2023-10-29] 2023-10-29: 2 hours in flight, about 1200 MB
2026-11-08T02:17:18Z [load_day 2023-10-29] 2023-10-29 hour 0: 124665 events in 3 files, snapshot 5661668986161697919 (task 7800 ms: download 1023 ms overlapping gunzip 436 ms, parse 2034 ms, sort 69 ms, write 4589 ms, index 468 ms: 9 candidates, 68966 repo-hours, 935 releases; waiting 19 ms; commit 2350 ms)
2026-11-08T02:17:20Z [load_day 2023-10-29] 2023-10-29 hour 1: 125657 events in 3 files, snapshot 2286184157629534915 (task 7451 ms: download 1092 ms overlapping gunzip 394 ms, parse 1892 ms, sort 86 ms, write 4132 ms, index 519 ms: 7 candidates, 74688 repo-hours, 708 releases; waiting 225 ms; commit 2353 ms)
2026-11-08T02:17:25Z [load_day 2023-10-29] 2023-10-29 hour 2: 94243 events in 2 files, snapshot 479811352539598540 (task 5772 ms: download 616 ms overlapping gunzip 359 ms, parse 1539 ms, sort 73 ms, write 3288 ms, index 289 ms: 6 candidates, 39580 repo-hours, 642 releases; waiting 17 ms; commit 2624 ms)
2026-11-08T02:17:29Z [load_day 2023-10-29] 2023-10-29 hour 3: 96653 events in 2 files, snapshot 780919333866760311 (task 6964 ms: download 659 ms overlapping gunzip 328 ms, parse 1548 ms, sort 70 ms, write 3791 ms, index 576 ms: 4 candidates, 39969 repo-hours, 709 releases; waiting 12 ms; commit 2891 ms)
2026-11-08T02:17:34Z [load_day 2023-10-29] 2023-10-29 hour 4: 100218 events in 3 files, snapshot 1602170945346067401 (task 7364 ms: download 608 ms overlapping gunzip 413 ms, parse 1859 ms, sort 131 ms, write 4314 ms, index 429 ms: 10 candidates, 40591 repo-hours, 563 releases; waiting 22 ms; commit 3053 ms)
2026-11-08T02:17:39Z [load_day 2023-10-29] 2023-10-29 hour 5: 101991 events in 3 files, snapshot 1419923158504124695 (task 8621 ms: download 1338 ms overlapping gunzip 536 ms, parse 2447 ms, sort 84 ms, write 4918 ms, index 412 ms: 4 candidates, 42305 repo-hours, 534 releases; waiting 20 ms; commit 3043 ms)
2026-11-08T02:17:45Z [load_day 2023-10-29] 2023-10-29 hour 6: 115268 events in 3 files, snapshot 6526471295761457208 (task 8408 ms: download 1153 ms overlapping gunzip 517 ms, parse 2204 ms, sort 73 ms, write 4872 ms, index 522 ms: 5 candidates, 51148 repo-hours, 618 releases; waiting 23 ms; commit 4420 ms)
2026-11-08T02:17:50Z [load_day 2023-10-29] 2023-10-29 hour 7: 111863 events in 3 files, snapshot 5928676711997027152 (task 9907 ms: download 992 ms overlapping gunzip 577 ms, parse 2570 ms, sort 77 ms, write 6039 ms, index 426 ms: 7 candidates, 46571 repo-hours, 539 releases; waiting 29 ms; commit 2843 ms)
2026-11-08T02:17:55Z [load_day 2023-10-29] 2023-10-29 hour 8: 122263 events in 3 files, snapshot 1787559757910708217 (task 10377 ms: download 1317 ms overlapping gunzip 625 ms, parse 2832 ms, sort 79 ms, write 6166 ms, index 448 ms: 8 candidates, 50586 repo-hours, 619 releases; waiting 17 ms; commit 3225 ms)
2026-11-08T02:18:05Z [load_day 2023-10-29] 2023-10-29 hour 9: 129970 events in 3 files, snapshot 4420867535052673190 (task 10609 ms: download 1130 ms overlapping gunzip 660 ms, parse 2724 ms, sort 93 ms, write 6239 ms, index 682 ms: 6 candidates, 52294 repo-hours, 770 releases; waiting 16 ms; commit 5771 ms)
2026-11-08T02:18:08Z [load_day 2023-10-29] 2023-10-29 hour 10: 131266 events in 3 files, snapshot 2326859334681319876 (task 11493 ms: download 1432 ms overlapping gunzip 721 ms, parse 3059 ms, sort 92 ms, write 6838 ms, index 576 ms: 8 candidates, 56072 repo-hours, 705 releases; waiting 16 ms; commit 2750 ms)
2026-11-08T02:18:15Z [load_day 2023-10-29] 2023-10-29 hour 11: 132583 events in 3 files, snapshot 3370706781561516769 (task 11788 ms: download 2250 ms overlapping gunzip 748 ms, parse 3574 ms, sort 114 ms, write 6274 ms, index 673 ms: 9 candidates, 54028 repo-hours, 715 releases; waiting 17 ms; commit 2831 ms)
2026-11-08T02:18:23Z [load_day 2023-10-29] 2023-10-29 hour 12: 207495 events in 5 files, snapshot 2905745918275962910 (task 14183 ms: download 4451 ms overlapping gunzip 813 ms, parse 3984 ms, sort 139 ms, write 8428 ms, index 613 ms: 14 candidates, 88431 repo-hours, 842 releases; waiting 22 ms; commit 2734 ms)
2026-11-08T02:18:30Z [load_day 2023-10-29] 2023-10-29 hour 13: 166660 events in 4 files, snapshot 3471694308190336196 (task 13289 ms: download 4295 ms overlapping gunzip 778 ms, parse 3473 ms, sort 148 ms, write 7965 ms, index 689 ms: 8 candidates, 81934 repo-hours, 869 releases; waiting 26 ms; commit 3029 ms)
2026-11-08T02:18:38Z [load_day 2023-10-29] 2023-10-29 hour 14: 154410 events in 4 files, snapshot 7385271082423135120 (task 14086 ms: download 4517 ms overlapping gunzip 911 ms, parse 4130 ms, sort 165 ms, write 8104 ms, index 568 ms: 10 candidates, 66894 repo-hours, 944 releases; waiting 21 ms; commit 2997 ms)
2026-11-08T02:18:45Z [load_day 2023-10-29] 2023-10-29 hour 15: 158058 events in 4 files, snapshot 3394070115656683784 (task 13624 ms: download 5381 ms overlapping gunzip 740 ms, parse 3233 ms, sort 118 ms, write 8427 ms, index 870 ms: 9 candidates, 68680 repo-hours, 928 releases; waiting 19 ms; commit 3125 ms)
2026-11-08T02:18:52Z [load_day 2023-10-29] 2023-10-29 hour 16: 155037 events in 4 files, snapshot 7232374245147639747 (task 13261 ms: download 4634 ms overlapping gunzip 703 ms, parse 3278 ms, sort 138 ms, write 8383 ms, index 539 ms: 8 candidates, 67096 repo-hours, 835 releases; waiting 19 ms; commit 2667 ms)
2026-11-08T02:18:58Z [load_day 2023-10-29] 2023-10-29 hour 17: 150080 events in 4 files, snapshot 5941109511028724993 (task 11884 ms: download 3906 ms overlapping gunzip 625 ms, parse 2974 ms, sort 220 ms, write 7102 ms, index 715 ms: 7 candidates, 63402 repo-hours, 830 releases; waiting 20 ms; commit 2787 ms)
2026-11-08T02:19:06Z [load_day 2023-10-29] 2023-10-29 hour 18: 152926 events in 4 files, snapshot 1377449076527002097 (task 12142 ms: download 4354 ms overlapping gunzip 685 ms, parse 3144 ms, sort 161 ms, write 7390 ms, index 551 ms: 6 candidates, 69482 repo-hours, 925 releases; waiting 18 ms; commit 3093 ms)
2026-11-08T02:19:11Z [load_day 2023-10-29] 2023-10-29 hour 19: 136835 events in 3 files, snapshot 1598255649245251287 (task 11178 ms: download 3844 ms overlapping gunzip 620 ms, parse 2917 ms, sort 178 ms, write 6595 ms, index 664 ms: 3 candidates, 57842 repo-hours, 846 releases; waiting 12 ms; commit 4192 ms)
2026-11-08T02:19:19Z [load_day 2023-10-29] 2023-10-29 hour 20: 134936 events in 3 files, snapshot 6339743955741298325 (task 12177 ms: download 2583 ms overlapping gunzip 715 ms, parse 3323 ms, sort 120 ms, write 6796 ms, index 397 ms: 8 candidates, 57006 repo-hours, 1251 releases; waiting 507 ms; commit 3053 ms)
2026-11-08T02:19:25Z [load_day 2023-10-29] 2023-10-29 hour 21: 124107 events in 3 files, snapshot 7657322185463814880 (task 10027 ms: download 1106 ms overlapping gunzip 527 ms, parse 2464 ms, sort 96 ms, write 5812 ms, index 886 ms: 6 candidates, 50985 repo-hours, 852 releases; waiting 19 ms; commit 5697 ms)
2026-11-08T02:19:33Z [load_day 2023-10-29] 2023-10-29 hour 22: 115709 events in 3 files, snapshot 1024543625304474447 (task 9876 ms: download 1214 ms overlapping gunzip 560 ms, parse 2647 ms, sort 74 ms, write 6020 ms, index 368 ms: 9 candidates, 46815 repo-hours, 717 releases; waiting 23 ms; commit 5733 ms)
2026-11-08T02:19:35Z [load_day 2023-10-29] 2023-10-29 hour 23: 108781 events in 3 files, snapshot 3449110338943718014 (task 9768 ms: download 2086 ms overlapping gunzip 517 ms, parse 2528 ms, sort 66 ms, write 5501 ms, index 376 ms: 7 candidates, 46239 repo-hours, 676 releases; waiting 484 ms; commit 2497 ms)
2026-11-08T02:19:42Z [load_day 2023-10-29] 2023-10-29: repo_hour merged from 24 files to 1 (1381604 rows, 6578 ms)
2026-11-08T02:19:44Z [load_day 2023-10-29] 2023-10-29: index committed in 9165 ms (24 + 1 + 24 files)
2026-11-08T02:19:44Z [load_day 2023-10-29] 2023-10-29: 3151674 events in 24 hours
2026-11-08T02:19:44Z [load_day 2023-10-29] attempt 1 succeeded