orcaLog in

Runs · run 1053

load_day 2023-05-03, attempt 1 succeeded

Host
caf251265124:330470
Started
2026-11-07T19:34:15Z
Ended
2026-11-07T19:37:20Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1053/load_day/2023-05-03/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T19:34:15Z [load_day 2023-05-03] attempt 1 started
2026-11-07T19:34:17Z [load_day 2023-05-03] 2023-05-03: 2 hours in flight, about 1200 MB
2026-11-07T19:34:31Z [load_day 2023-05-03] 2023-05-03 hour 0: 152241 events in 4 files, snapshot 4448383655653690295 (task 11211 ms: download 4934 ms overlapping gunzip 640 ms, parse 3000 ms, sort 98 ms, write 6157 ms, index 507 ms: 4 candidates, 76591 repo-hours, 991 releases; waiting 525 ms; commit 1540 ms)
2026-11-07T19:34:32Z [load_day 2023-05-03] 2023-05-03 hour 1: 159454 events in 4 files, snapshot 5838117167876770603 (task 11900 ms: download 5443 ms overlapping gunzip 645 ms, parse 2990 ms, sort 92 ms, write 5846 ms, index 502 ms: 5 candidates, 85262 repo-hours, 774 releases; waiting 1505 ms; commit 1629 ms)
2026-11-07T19:34:41Z [load_day 2023-05-03] 2023-05-03 hour 2: 129058 events in 3 files, snapshot 2834132991953318355 (task 9269 ms: download 3856 ms overlapping gunzip 536 ms, parse 2407 ms, sort 72 ms, write 5029 ms, index 364 ms: 9 candidates, 55393 repo-hours, 678 releases; waiting 539 ms; commit 1838 ms)
2026-11-07T19:34:43Z [load_day 2023-05-03] 2023-05-03 hour 3: 123544 events in 3 files, snapshot 5042852813065889263 (task 9141 ms: download 3542 ms overlapping gunzip 500 ms, parse 2249 ms, sort 63 ms, write 4686 ms, index 341 ms: 4 candidates, 52054 repo-hours, 810 releases; waiting 979 ms; commit 1645 ms)
2026-11-07T19:34:51Z [load_day 2023-05-03] 2023-05-03 hour 4: 125681 events in 3 files, snapshot 5344556600503942221 (task 9518 ms: download 3111 ms overlapping gunzip 576 ms, parse 2458 ms, sort 83 ms, write 5055 ms, index 346 ms: 4 candidates, 51732 repo-hours, 634 releases; waiting 651 ms; commit 1606 ms)
2026-11-07T19:34:53Z [load_day 2023-05-03] 2023-05-03 hour 5: 137601 events in 3 files, snapshot 6066092268650334014 (task 10105 ms: download 4272 ms overlapping gunzip 608 ms, parse 2684 ms, sort 95 ms, write 5425 ms, index 386 ms: 4 candidates, 57938 repo-hours, 749 releases; waiting 546 ms; commit 1818 ms)
2026-11-07T19:35:04Z [load_day 2023-05-03] 2023-05-03 hour 6: 155780 events in 4 files, snapshot 3636434761898283495 (task 11781 ms: download 4559 ms overlapping gunzip 671 ms, parse 3074 ms, sort 107 ms, write 6652 ms, index 480 ms: 8 candidates, 69256 repo-hours, 880 releases; waiting 491 ms; commit 1748 ms)
2026-11-07T19:35:08Z [load_day 2023-05-03] 2023-05-03 hour 7: 163999 events in 4 files, snapshot 5185364791577039957 (task 14069 ms: download 5860 ms overlapping gunzip 783 ms, parse 3442 ms, sort 121 ms, write 7589 ms, index 512 ms: 12 candidates, 70388 repo-hours, 809 releases; waiting 1097 ms; commit 1753 ms)
2026-11-07T19:35:18Z [load_day 2023-05-03] 2023-05-03 hour 8: 168909 events in 4 files, snapshot 9056202874438760509 (task 13871 ms: download 7998 ms overlapping gunzip 791 ms, parse 3620 ms, sort 107 ms, write 7938 ms, index 529 ms: 10 candidates, 76966 repo-hours, 1025 releases; waiting 518 ms; commit 1678 ms)
2026-11-07T19:35:25Z [load_day 2023-05-03] 2023-05-03 hour 9: 187189 events in 4 files, snapshot 3150317608126430387 (task 15624 ms: download 8742 ms overlapping gunzip 914 ms, parse 4049 ms, sort 124 ms, write 8678 ms, index 551 ms: 6 candidates, 84062 repo-hours, 1373 releases; waiting 974 ms; commit 1678 ms)
2026-11-07T19:35:33Z [load_day 2023-05-03] 2023-05-03 hour 10: 180827 events in 4 files, snapshot 8482778020419273223 (task 14126 ms: download 7962 ms overlapping gunzip 833 ms, parse 3800 ms, sort 101 ms, write 7981 ms, index 534 ms: 9 candidates, 77842 repo-hours, 1059 releases; waiting 578 ms; commit 1914 ms)
2026-11-07T19:35:40Z [load_day 2023-05-03] 2023-05-03 hour 11: 182426 events in 4 files, snapshot 6900193893287491908 (task 14946 ms: download 8351 ms overlapping gunzip 873 ms, parse 3857 ms, sort 119 ms, write 8632 ms, index 563 ms: 3 candidates, 79024 repo-hours, 1056 releases; waiting 600 ms; commit 1609 ms)
2026-11-07T19:35:50Z [load_day 2023-05-03] 2023-05-03 hour 12: 219668 events in 5 files, snapshot 2606499513645764745 (task 16050 ms: download 8380 ms overlapping gunzip 922 ms, parse 4209 ms, sort 127 ms, write 9342 ms, index 650 ms: 12 candidates, 106459 repo-hours, 1150 releases; waiting 527 ms; commit 1668 ms)
2026-11-07T19:35:57Z [load_day 2023-05-03] 2023-05-03 hour 13: 207236 events in 5 files, snapshot 4841081402226263559 (task 16135 ms: download 8283 ms overlapping gunzip 959 ms, parse 4277 ms, sort 118 ms, write 9568 ms, index 682 ms: 9 candidates, 104701 repo-hours, 1120 releases; waiting 237 ms; commit 1597 ms)
2026-11-07T19:36:08Z [load_day 2023-05-03] 2023-05-03 hour 14: 194836 events in 4 files, snapshot 1573218832083598051 (task 16450 ms: download 8971 ms overlapping gunzip 1010 ms, parse 4549 ms, sort 144 ms, write 9312 ms, index 658 ms: 8 candidates, 92220 repo-hours, 1089 releases; waiting 491 ms; commit 1846 ms)
2026-11-07T19:36:13Z [load_day 2023-05-03] 2023-05-03 hour 15: 175639 events in 4 files, snapshot 2270943033955667024 (task 15137 ms: download 8652 ms overlapping gunzip 904 ms, parse 3917 ms, sort 117 ms, write 8631 ms, index 551 ms: 5 candidates, 85192 repo-hours, 1160 releases; waiting 520 ms; commit 1697 ms)
2026-11-07T19:36:24Z [load_day 2023-05-03] 2023-05-03 hour 16: 178500 events in 4 files, snapshot 7135886162292592651 (task 15059 ms: download 8666 ms overlapping gunzip 875 ms, parse 3956 ms, sort 119 ms, write 8598 ms, index 567 ms: 5 candidates, 84418 repo-hours, 991 releases; waiting 589 ms; commit 1896 ms)
2026-11-07T19:36:29Z [load_day 2023-05-03] 2023-05-03 hour 17: 183026 events in 4 files, snapshot 7057145769610444200 (task 15216 ms: download 8568 ms overlapping gunzip 894 ms, parse 3999 ms, sort 125 ms, write 8808 ms, index 562 ms: 10 candidates, 81244 repo-hours, 828 releases; waiting 507 ms; commit 1649 ms)
2026-11-07T19:36:39Z [load_day 2023-05-03] 2023-05-03 hour 18: 185908 events in 4 files, snapshot 2572598293206324831 (task 14562 ms: download 7908 ms overlapping gunzip 860 ms, parse 3823 ms, sort 116 ms, write 8695 ms, index 573 ms: 6 candidates, 84177 repo-hours, 989 releases; waiting 186 ms; commit 1768 ms)
2026-11-07T19:36:45Z [load_day 2023-05-03] 2023-05-03 hour 19: 170758 events in 4 files, snapshot 6718348549171734902 (task 14745 ms: download 8771 ms overlapping gunzip 807 ms, parse 3569 ms, sort 106 ms, write 8013 ms, index 526 ms: 4 candidates, 76774 repo-hours, 935 releases; waiting 1285 ms; commit 1733 ms)
2026-11-07T19:36:54Z [load_day 2023-05-03] 2023-05-03 hour 20: 170428 events in 4 files, snapshot 3141292236108795920 (task 13847 ms: download 8158 ms overlapping gunzip 827 ms, parse 3718 ms, sort 97 ms, write 7946 ms, index 497 ms: 7 candidates, 75056 repo-hours, 973 releases; waiting 463 ms; commit 1856 ms)
2026-11-07T19:37:01Z [load_day 2023-05-03] 2023-05-03 hour 21: 160218 events in 4 files, snapshot 1471141774471458181 (task 13177 ms: download 5582 ms overlapping gunzip 783 ms, parse 3394 ms, sort 90 ms, write 7650 ms, index 487 ms: 6 candidates, 65623 repo-hours, 970 releases; waiting 509 ms; commit 3885 ms)
2026-11-07T19:37:06Z [load_day 2023-05-03] 2023-05-03 hour 22: 145066 events in 3 files, snapshot 666212379508968468 (task 11673 ms: download 5025 ms overlapping gunzip 710 ms, parse 3022 ms, sort 100 ms, write 6616 ms, index 382 ms: 9 candidates, 58366 repo-hours, 753 releases; waiting 519 ms; commit 1821 ms)
2026-11-07T19:37:10Z [load_day 2023-05-03] 2023-05-03 hour 23: 130381 events in 3 files, snapshot 6354961871877676045 (task 10528 ms: download 4820 ms overlapping gunzip 628 ms, parse 2730 ms, sort 79 ms, write 5764 ms, index 355 ms: 6 candidates, 54352 repo-hours, 690 releases; waiting 631 ms; commit 1687 ms)
2026-11-07T19:37:18Z [load_day 2023-05-03] 2023-05-03: repo_hour merged from 24 files to 1 (1805090 rows, 7698 ms)
2026-11-07T19:37:20Z [load_day 2023-05-03] 2023-05-03: index committed in 9833 ms (24 + 1 + 24 files)
2026-11-07T19:37:20Z [load_day 2023-05-03] 2023-05-03: 3988373 events in 24 hours
2026-11-07T19:37:20Z [load_day 2023-05-03] attempt 1 succeeded