orcaLog in

Runs · run 1516

load_day 2023-09-17, attempt 1 succeeded

Host
3c2b29713ce8:340672
Started
2026-11-08T00:55:10Z
Ended
2026-11-08T00:57:35Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1516/load_day/2023-09-17/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:55:10Z [load_day 2023-09-17] attempt 1 started
2026-11-08T00:55:12Z [load_day 2023-09-17] 2023-09-17: 2 hours in flight, about 1200 MB
2026-11-08T00:55:28Z [load_day 2023-09-17] 2023-09-17 hour 0: 126267 events in 3 files, snapshot 5026517696947325332 (task 10016 ms: download 2091 ms overlapping gunzip 615 ms, parse 3039 ms, sort 93 ms, write 4947 ms, index 520 ms: 10 candidates, 69887 repo-hours, 972 releases; waiting 494 ms; commit 4237 ms)
2026-11-08T00:55:32Z [load_day 2023-09-17] 2023-09-17 hour 1: 121793 events in 3 files, snapshot 939044792409847537 (task 9692 ms: download 2302 ms overlapping gunzip 526 ms, parse 2464 ms, sort 108 ms, write 5411 ms, index 470 ms: 8 candidates, 71667 repo-hours, 753 releases; waiting 501 ms; commit 3769 ms)
2026-11-08T00:55:36Z [load_day 2023-09-17] 2023-09-17 hour 2: 93418 events in 2 files, snapshot 5440033052042812157 (task 7539 ms: download 1911 ms overlapping gunzip 346 ms, parse 1641 ms, sort 81 ms, write 4380 ms, index 276 ms: 7 candidates, 38809 repo-hours, 656 releases; waiting 497 ms; commit 3695 ms)
2026-11-08T00:55:38Z [load_day 2023-09-17] 2023-09-17 hour 3: 91679 events in 2 files, snapshot 867997634961205662 (task 7084 ms: download 1796 ms overlapping gunzip 452 ms, parse 1944 ms, sort 84 ms, write 3525 ms, index 298 ms: 7 candidates, 38141 repo-hours, 798 releases; waiting 513 ms; commit 2070 ms)
2026-11-08T00:55:43Z [load_day 2023-09-17] 2023-09-17 hour 4: 96920 events in 2 files, snapshot 3743300919102453626 (task 6629 ms: download 2196 ms overlapping gunzip 364 ms, parse 1646 ms, sort 62 ms, write 3420 ms, index 341 ms: 4 candidates, 39514 repo-hours, 593 releases; waiting 522 ms; commit 2928 ms)
2026-11-08T00:55:46Z [load_day 2023-09-17] 2023-09-17 hour 5: 102890 events in 3 files, snapshot 2984228452597932748 (task 7009 ms: download 1972 ms overlapping gunzip 370 ms, parse 1669 ms, sort 64 ms, write 3635 ms, index 295 ms: 8 candidates, 42381 repo-hours, 751 releases; waiting 607 ms; commit 2070 ms)
2026-11-08T00:55:49Z [load_day 2023-09-17] 2023-09-17 hour 6: 114231 events in 3 files, snapshot 7189648587146662274 (task 6918 ms: download 2106 ms overlapping gunzip 354 ms, parse 1674 ms, sort 67 ms, write 3645 ms, index 368 ms: 6 candidates, 50370 repo-hours, 754 releases; waiting 551 ms; commit 2186 ms)
2026-11-08T00:55:55Z [load_day 2023-09-17] 2023-09-17 hour 7: 111547 events in 3 files, snapshot 6941833014958255898 (task 7524 ms: download 1590 ms overlapping gunzip 387 ms, parse 1746 ms, sort 68 ms, write 4165 ms, index 364 ms: 6 candidates, 46675 repo-hours, 677 releases; waiting 517 ms; commit 2014 ms)
2026-11-08T00:55:58Z [load_day 2023-09-17] 2023-09-17 hour 8: 121683 events in 3 files, snapshot 2784205482535359513 (task 8027 ms: download 2146 ms overlapping gunzip 451 ms, parse 1944 ms, sort 74 ms, write 4462 ms, index 371 ms: 7 candidates, 50753 repo-hours, 727 releases; waiting 502 ms; commit 1979 ms)
2026-11-08T00:56:05Z [load_day 2023-09-17] 2023-09-17 hour 9: 124828 events in 3 files, snapshot 3154861900099803095 (task 8585 ms: download 2023 ms overlapping gunzip 486 ms, parse 2163 ms, sort 114 ms, write 4481 ms, index 585 ms: 7 candidates, 52362 repo-hours, 945 releases; waiting 502 ms; commit 2818 ms)
2026-11-08T00:56:10Z [load_day 2023-09-17] 2023-09-17 hour 10: 125293 events in 3 files, snapshot 4825156560117970390 (task 9669 ms: download 2101 ms overlapping gunzip 504 ms, parse 2321 ms, sort 76 ms, write 5477 ms, index 452 ms: 14 candidates, 52746 repo-hours, 779 releases; waiting 498 ms; commit 2647 ms)
2026-11-08T00:56:17Z [load_day 2023-09-17] 2023-09-17 hour 11: 122313 events in 3 files, snapshot 2803259729038663205 (task 9860 ms: download 2217 ms overlapping gunzip 506 ms, parse 2249 ms, sort 76 ms, write 5834 ms, index 442 ms: 5 candidates, 51473 repo-hours, 826 releases; waiting 494 ms; commit 3108 ms)
2026-11-08T00:56:24Z [load_day 2023-09-17] 2023-09-17 hour 12: 208242 events in 5 files, snapshot 4035595173573027431 (task 13073 ms: download 4044 ms overlapping gunzip 675 ms, parse 3331 ms, sort 162 ms, write 7427 ms, index 822 ms: 6 candidates, 86577 repo-hours, 846 releases; waiting 371 ms; commit 2792 ms)
2026-11-08T00:56:29Z [load_day 2023-09-17] 2023-09-17 hour 13: 162075 events in 4 files, snapshot 8328667385157269687 (task 12198 ms: download 4090 ms overlapping gunzip 669 ms, parse 3128 ms, sort 140 ms, write 6888 ms, index 640 ms: 13 candidates, 79336 repo-hours, 1398 releases; waiting 508 ms; commit 2613 ms)
2026-11-08T00:56:38Z [load_day 2023-09-17] 2023-09-17 hour 14: 152503 events in 4 files, snapshot 7366855424946987290 (task 11821 ms: download 4339 ms overlapping gunzip 638 ms, parse 2987 ms, sort 109 ms, write 6715 ms, index 572 ms: 10 candidates, 64507 repo-hours, 837 releases; waiting 510 ms; commit 3353 ms)
2026-11-08T00:56:42Z [load_day 2023-09-17] 2023-09-17 hour 15: 150988 events in 4 files, snapshot 3025483387420392545 (task 11795 ms: download 4019 ms overlapping gunzip 644 ms, parse 3009 ms, sort 121 ms, write 6674 ms, index 573 ms: 9 candidates, 66819 repo-hours, 913 releases; waiting 505 ms; commit 2617 ms)
2026-11-08T00:56:50Z [load_day 2023-09-17] 2023-09-17 hour 16: 146780 events in 3 files, snapshot 6189775428192509141 (task 11230 ms: download 4058 ms overlapping gunzip 696 ms, parse 3015 ms, sort 97 ms, write 6345 ms, index 520 ms: 13 candidates, 64240 repo-hours, 785 releases; waiting 199 ms; commit 3588 ms)
2026-11-08T00:56:55Z [load_day 2023-09-17] 2023-09-17 hour 17: 136106 events in 3 files, snapshot 5267016604821252848 (task 10750 ms: download 2297 ms overlapping gunzip 593 ms, parse 2534 ms, sort 87 ms, write 6262 ms, index 461 ms: 13 candidates, 59042 repo-hours, 686 releases; waiting 517 ms; commit 4156 ms)
2026-11-08T00:57:03Z [load_day 2023-09-17] 2023-09-17 hour 18: 143288 events in 3 files, snapshot 2541138311243242171 (task 12302 ms: download 3866 ms overlapping gunzip 719 ms, parse 3289 ms, sort 199 ms, write 6698 ms, index 591 ms: 15 candidates, 65360 repo-hours, 793 releases; waiting 525 ms; commit 2259 ms)
2026-11-08T00:57:06Z [load_day 2023-09-17] 2023-09-17 hour 19: 130724 events in 3 files, snapshot 6063263870935737337 (task 11224 ms: download 2527 ms overlapping gunzip 642 ms, parse 2976 ms, sort 198 ms, write 5950 ms, index 597 ms: 9 candidates, 54102 repo-hours, 793 releases; waiting 489 ms; commit 2773 ms)
2026-11-08T00:57:14Z [load_day 2023-09-17] 2023-09-17 hour 20: 124713 events in 3 files, snapshot 1082124479446262377 (task 10282 ms: download 2428 ms overlapping gunzip 603 ms, parse 2782 ms, sort 113 ms, write 5600 ms, index 401 ms: 7 candidates, 53167 repo-hours, 674 releases; waiting 496 ms; commit 2102 ms)
2026-11-08T00:57:19Z [load_day 2023-09-17] 2023-09-17 hour 21: 112553 events in 3 files, snapshot 2417419287573827905 (task 8645 ms: download 2003 ms overlapping gunzip 421 ms, parse 1989 ms, sort 105 ms, write 4962 ms, index 391 ms: 4 candidates, 46378 repo-hours, 741 releases; waiting 497 ms; commit 5355 ms)
2026-11-08T00:57:24Z [load_day 2023-09-17] 2023-09-17 hour 22: 104554 events in 3 files, snapshot 8272031952576672747 (task 9682 ms: download 2406 ms overlapping gunzip 609 ms, parse 2499 ms, sort 114 ms, write 5283 ms, index 415 ms: 8 candidates, 41949 repo-hours, 612 releases; waiting 499 ms; commit 2073 ms)
2026-11-08T00:57:27Z [load_day 2023-09-17] 2023-09-17 hour 23: 100066 events in 3 files, snapshot 7004316453279848459 (task 8547 ms: download 2006 ms overlapping gunzip 546 ms, parse 2448 ms, sort 77 ms, write 4382 ms, index 298 ms: 5 candidates, 41415 repo-hours, 542 releases; waiting 466 ms; commit 2236 ms)
2026-11-08T00:57:33Z [load_day 2023-09-17] 2023-09-17: repo_hour merged from 24 files to 1 (1327670 rows, 6033 ms)
2026-11-08T00:57:35Z [load_day 2023-09-17] 2023-09-17: index committed in 8139 ms (24 + 1 + 24 files)
2026-11-08T00:57:35Z [load_day 2023-09-17] 2023-09-17: 3025454 events in 24 hours
2026-11-08T00:57:35Z [load_day 2023-09-17] attempt 1 succeeded