orcaLog in

Runs · run 966

load_day 2023-04-04, attempt 1 succeeded

Host
caf251265124:232402
Started
2026-11-07T18:08:08Z
Ended
2026-11-07T18:11:40Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/966/load_day/2023-04-04/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T18:08:08Z [load_day 2023-04-04] attempt 1 started
2026-11-07T18:08:09Z [load_day 2023-04-04] 2023-04-04: 2 hours in flight, about 1200 MB
2026-11-07T18:08:37Z [load_day 2023-04-04] 2023-04-04 hour 0: 208705 events in 5 files, snapshot 2791142348953130804 (task 24567 ms: download 17819 ms overlapping gunzip 1577 ms, parse 6365 ms, sort 138 ms, write 15296 ms, index 658 ms: 9 candidates, 103684 repo-hours, 765 releases; waiting 243 ms; commit 1794 ms)
2026-11-07T18:08:39Z [load_day 2023-04-04] 2023-04-04 hour 1: 210413 events in 5 files, snapshot 2188130399246236806 (task 23431 ms: download 16555 ms overlapping gunzip 1420 ms, parse 5754 ms, sort 139 ms, write 14468 ms, index 773 ms: 5 candidates, 112855 repo-hours, 752 releases; waiting 525 ms; commit 1819 ms)
2026-11-07T18:08:50Z [load_day 2023-04-04] 2023-04-04 hour 2: 153104 events in 4 files, snapshot 6377447979376269515 (task 12560 ms: download 5457 ms overlapping gunzip 710 ms, parse 3061 ms, sort 116 ms, write 7252 ms, index 566 ms: 4 candidates, 72354 repo-hours, 718 releases; waiting 522 ms; commit 1847 ms)
2026-11-07T18:08:52Z [load_day 2023-04-04] 2023-04-04 hour 3: 144347 events in 3 files, snapshot 1896584300810052218 (task 11190 ms: download 4565 ms overlapping gunzip 637 ms, parse 2824 ms, sort 99 ms, write 6453 ms, index 408 ms: 3 candidates, 63183 repo-hours, 917 releases; waiting 498 ms; commit 1580 ms)
2026-11-07T18:09:02Z [load_day 2023-04-04] 2023-04-04 hour 4: 134883 events in 3 files, snapshot 2813674535233832427 (task 10575 ms: download 4620 ms overlapping gunzip 644 ms, parse 2808 ms, sort 103 ms, write 5728 ms, index 404 ms: 3 candidates, 58458 repo-hours, 639 releases; waiting 524 ms; commit 1579 ms)
2026-11-07T18:09:05Z [load_day 2023-04-04] 2023-04-04 hour 5: 144591 events in 3 files, snapshot 5170144469147154743 (task 11540 ms: download 4861 ms overlapping gunzip 679 ms, parse 3027 ms, sort 104 ms, write 6364 ms, index 460 ms: 1 candidates, 64829 repo-hours, 762 releases; waiting 591 ms; commit 1884 ms)
2026-11-07T18:09:16Z [load_day 2023-04-04] 2023-04-04 hour 6: 173113 events in 4 files, snapshot 1211295912013431649 (task 13770 ms: download 7859 ms overlapping gunzip 801 ms, parse 3511 ms, sort 106 ms, write 7547 ms, index 618 ms: 4 candidates, 81732 repo-hours, 936 releases; waiting 876 ms; commit 1662 ms)
2026-11-07T18:09:20Z [load_day 2023-04-04] 2023-04-04 hour 7: 173934 events in 4 files, snapshot 6692870405892016813 (task 14991 ms: download 8530 ms overlapping gunzip 864 ms, parse 3778 ms, sort 107 ms, write 8865 ms, index 581 ms: 5 candidates, 81022 repo-hours, 873 releases; waiting 497 ms; commit 1719 ms)
2026-11-07T18:09:34Z [load_day 2023-04-04] 2023-04-04 hour 8: 187262 events in 4 files, snapshot 4997393861056629020 (task 16774 ms: download 9691 ms overlapping gunzip 1005 ms, parse 4229 ms, sort 139 ms, write 9899 ms, index 599 ms: 9 candidates, 86525 repo-hours, 1075 releases; waiting 605 ms; commit 1779 ms)
2026-11-07T18:09:38Z [load_day 2023-04-04] 2023-04-04 hour 9: 185134 events in 4 files, snapshot 161711248651738981 (task 16184 ms: download 9395 ms overlapping gunzip 970 ms, parse 4072 ms, sort 121 ms, write 9288 ms, index 550 ms: 8 candidates, 87970 repo-hours, 1028 releases; waiting 826 ms; commit 1866 ms)
2026-11-07T18:09:50Z [load_day 2023-04-04] 2023-04-04 hour 10: 182627 events in 4 files, snapshot 1550266337404140732 (task 15383 ms: download 8827 ms overlapping gunzip 954 ms, parse 4060 ms, sort 110 ms, write 9130 ms, index 532 ms: 7 candidates, 82586 repo-hours, 915 releases; waiting 300 ms; commit 1781 ms)
2026-11-07T18:09:53Z [load_day 2023-04-04] 2023-04-04 hour 11: 181735 events in 4 files, snapshot 3626864406683193986 (task 14825 ms: download 8379 ms overlapping gunzip 874 ms, parse 3796 ms, sort 132 ms, write 8689 ms, index 615 ms: 8 candidates, 81320 repo-hours, 945 releases; waiting 484 ms; commit 1611 ms)
2026-11-07T18:10:07Z [load_day 2023-04-04] 2023-04-04 hour 12: 207530 events in 5 files, snapshot 7617036538988057893 (task 15725 ms: download 8191 ms overlapping gunzip 882 ms, parse 3942 ms, sort 128 ms, write 9246 ms, index 682 ms: 7 candidates, 105982 repo-hours, 992 releases; waiting 522 ms; commit 1726 ms)
2026-11-07T18:10:10Z [load_day 2023-04-04] 2023-04-04 hour 13: 198023 events in 4 files, snapshot 1971519030485272788 (task 16425 ms: download 8696 ms overlapping gunzip 946 ms, parse 4160 ms, sort 136 ms, write 9766 ms, index 717 ms: 5 candidates, 103483 repo-hours, 1113 releases; waiting 379 ms; commit 1567 ms)
2026-11-07T18:10:25Z [load_day 2023-04-04] 2023-04-04 hour 14: 191608 events in 4 files, snapshot 4675362079394671663 (task 17792 ms: download 10115 ms overlapping gunzip 1077 ms, parse 4603 ms, sort 129 ms, write 10191 ms, index 665 ms: 7 candidates, 93755 repo-hours, 975 releases; waiting 773 ms; commit 1598 ms)
2026-11-07T18:10:29Z [load_day 2023-04-04] 2023-04-04 hour 15: 190959 events in 4 files, snapshot 6440153949141126085 (task 17539 ms: download 10264 ms overlapping gunzip 995 ms, parse 4355 ms, sort 136 ms, write 10015 ms, index 612 ms: 5 candidates, 91973 repo-hours, 1073 releases; waiting 1134 ms; commit 1656 ms)
2026-11-07T18:10:43Z [load_day 2023-04-04] 2023-04-04 hour 16: 186808 events in 4 files, snapshot 8207992776445209793 (task 16406 ms: download 9508 ms overlapping gunzip 981 ms, parse 4264 ms, sort 119 ms, write 9515 ms, index 621 ms: 3 candidates, 88883 repo-hours, 962 releases; waiting 567 ms; commit 1603 ms)
2026-11-07T18:10:46Z [load_day 2023-04-04] 2023-04-04 hour 17: 190466 events in 4 files, snapshot 3128228047230926126 (task 17207 ms: download 10263 ms overlapping gunzip 989 ms, parse 4312 ms, sort 130 ms, write 9120 ms, index 609 ms: 11 candidates, 84644 repo-hours, 964 releases; waiting 1771 ms; commit 1503 ms)
2026-11-07T18:10:59Z [load_day 2023-04-04] 2023-04-04 hour 18: 190720 events in 4 files, snapshot 4677531949312550324 (task 15912 ms: download 8766 ms overlapping gunzip 938 ms, parse 4147 ms, sort 129 ms, write 9100 ms, index 586 ms: 5 candidates, 88774 repo-hours, 1058 releases; waiting 709 ms; commit 1643 ms)
2026-11-07T18:11:03Z [load_day 2023-04-04] 2023-04-04 hour 19: 182202 events in 4 files, snapshot 8837806675131880611 (task 16019 ms: download 9544 ms overlapping gunzip 918 ms, parse 4058 ms, sort 135 ms, write 8533 ms, index 564 ms: 7 candidates, 79809 repo-hours, 1022 releases; waiting 1563 ms; commit 1552 ms)
2026-11-07T18:11:15Z [load_day 2023-04-04] 2023-04-04 hour 20: 173900 events in 4 files, snapshot 2420365728516186594 (task 14444 ms: download 8361 ms overlapping gunzip 849 ms, parse 3744 ms, sort 109 ms, write 8362 ms, index 577 ms: 12 candidates, 76396 repo-hours, 932 releases; waiting 525 ms; commit 1582 ms)
2026-11-07T18:11:18Z [load_day 2023-04-04] 2023-04-04 hour 21: 161285 events in 4 files, snapshot 7291759793933008571 (task 14292 ms: download 8523 ms overlapping gunzip 894 ms, parse 3836 ms, sort 105 ms, write 7956 ms, index 568 ms: 10 candidates, 70226 repo-hours, 956 releases; waiting 635 ms; commit 1512 ms)
2026-11-07T18:11:28Z [load_day 2023-04-04] 2023-04-04 hour 22: 144564 events in 3 files, snapshot 318510781189209955 (task 11911 ms: download 5280 ms overlapping gunzip 730 ms, parse 3196 ms, sort 100 ms, write 6677 ms, index 415 ms: 4 candidates, 60761 repo-hours, 826 releases; waiting 556 ms; commit 1597 ms)
2026-11-07T18:11:30Z [load_day 2023-04-04] 2023-04-04 hour 23: 135483 events in 3 files, snapshot 5536938791625516074 (task 10560 ms: download 4412 ms overlapping gunzip 621 ms, parse 2733 ms, sort 86 ms, write 5953 ms, index 390 ms: 9 candidates, 58134 repo-hours, 697 releases; waiting 457 ms; commit 1552 ms)
2026-11-07T18:11:38Z [load_day 2023-04-04] 2023-04-04: repo_hour merged from 24 files to 1 (1979338 rows, 8315 ms)
2026-11-07T18:11:40Z [load_day 2023-04-04] 2023-04-04: index committed in 10117 ms (24 + 1 + 24 files)
2026-11-07T18:11:40Z [load_day 2023-04-04] 2023-04-04: 4233396 events in 24 hours
2026-11-07T18:11:40Z [load_day 2023-04-04] attempt 1 succeeded