orcaLog in

Runs · run 1113

load_day 2023-05-23, attempt 1 succeeded

Host
caf251265124:394202
Started
2026-11-07T20:30:53Z
Ended
2026-11-07T20:34:00Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1113/load_day/2023-05-23/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T20:30:53Z [load_day 2023-05-23] attempt 1 started
2026-11-07T20:30:55Z [load_day 2023-05-23] 2023-05-23: 2 hours in flight, about 1200 MB
2026-11-07T20:31:10Z [load_day 2023-05-23] 2023-05-23 hour 0: 157544 events in 4 files, snapshot 8957576335689746699 (task 12801 ms: download 5923 ms overlapping gunzip 772 ms, parse 3406 ms, sort 97 ms, write 7188 ms, index 531 ms: 6 candidates, 84204 repo-hours, 842 releases; waiting 530 ms; commit 1672 ms)
2026-11-07T20:31:12Z [load_day 2023-05-23] 2023-05-23 hour 1: 162314 events in 4 files, snapshot 3282959684724404810 (task 12355 ms: download 5342 ms overlapping gunzip 708 ms, parse 3203 ms, sort 95 ms, write 6738 ms, index 609 ms: 11 candidates, 91047 repo-hours, 739 releases; waiting 728 ms; commit 1761 ms)
2026-11-07T20:31:22Z [load_day 2023-05-23] 2023-05-23 hour 2: 147125 events in 3 files, snapshot 1270561757799797529 (task 11111 ms: download 4736 ms overlapping gunzip 691 ms, parse 2847 ms, sort 87 ms, write 6235 ms, index 460 ms: 6 candidates, 66787 repo-hours, 750 releases; waiting 496 ms; commit 1739 ms)
2026-11-07T20:31:24Z [load_day 2023-05-23] 2023-05-23 hour 3: 136343 events in 3 files, snapshot 4627960996409329888 (task 10663 ms: download 4778 ms overlapping gunzip 684 ms, parse 2937 ms, sort 84 ms, write 5761 ms, index 409 ms: 9 candidates, 60835 repo-hours, 837 releases; waiting 496 ms; commit 1885 ms)
2026-11-07T20:31:34Z [load_day 2023-05-23] 2023-05-23 hour 4: 130925 events in 3 files, snapshot 3493194604265141508 (task 10741 ms: download 5058 ms overlapping gunzip 676 ms, parse 2884 ms, sort 75 ms, write 5784 ms, index 374 ms: 5 candidates, 57084 repo-hours, 601 releases; waiting 683 ms; commit 1988 ms)
2026-11-07T20:31:37Z [load_day 2023-05-23] 2023-05-23 hour 5: 147772 events in 3 files, snapshot 4755107881351996198 (task 11963 ms: download 5094 ms overlapping gunzip 729 ms, parse 3106 ms, sort 87 ms, write 6641 ms, index 437 ms: 7 candidates, 68382 repo-hours, 764 releases; waiting 510 ms; commit 1723 ms)
2026-11-07T20:31:50Z [load_day 2023-05-23] 2023-05-23 hour 6: 176679 events in 4 files, snapshot 8447159084080755226 (task 14608 ms: download 8397 ms overlapping gunzip 938 ms, parse 3958 ms, sort 94 ms, write 8293 ms, index 533 ms: 12 candidates, 83523 repo-hours, 985 releases; waiting 529 ms; commit 1886 ms)
2026-11-07T20:31:58Z [load_day 2023-05-23] 2023-05-23 hour 7: 166905 events in 4 files, snapshot 4337572262689442332 (task 15977 ms: download 10008 ms overlapping gunzip 928 ms, parse 3964 ms, sort 95 ms, write 8160 ms, index 502 ms: 8 candidates, 79201 repo-hours, 858 releases; waiting 2007 ms; commit 5490 ms)
2026-11-07T20:32:06Z [load_day 2023-05-23] 2023-05-23 hour 8: 178932 events in 4 files, snapshot 8503597751095771686 (task 15458 ms: download 8970 ms overlapping gunzip 911 ms, parse 4152 ms, sort 103 ms, write 9052 ms, index 533 ms: 11 candidates, 84625 repo-hours, 992 releases; waiting 390 ms; commit 1821 ms)
2026-11-07T20:32:10Z [load_day 2023-05-23] 2023-05-23 hour 9: 177754 events in 4 files, snapshot 5729577083746446196 (task 15568 ms: download 9130 ms overlapping gunzip 940 ms, parse 4061 ms, sort 104 ms, write 8756 ms, index 539 ms: 9 candidates, 84052 repo-hours, 1195 releases; waiting 760 ms; commit 1624 ms)
2026-11-07T20:32:21Z [load_day 2023-05-23] 2023-05-23 hour 10: 175022 events in 4 files, snapshot 1746164947269156432 (task 14550 ms: download 8392 ms overlapping gunzip 861 ms, parse 3850 ms, sort 109 ms, write 8177 ms, index 574 ms: 7 candidates, 82067 repo-hours, 1040 releases; waiting 651 ms; commit 1770 ms)
2026-11-07T20:32:25Z [load_day 2023-05-23] 2023-05-23 hour 11: 161940 events in 4 files, snapshot 6043952391512235530 (task 14335 ms: download 6178 ms overlapping gunzip 817 ms, parse 3658 ms, sort 107 ms, write 7974 ms, index 532 ms: 2 candidates, 76222 repo-hours, 917 releases; waiting 930 ms; commit 1672 ms)
2026-11-07T20:32:36Z [load_day 2023-05-23] 2023-05-23 hour 12: 187666 events in 4 files, snapshot 8304923256636721154 (task 14435 ms: download 7912 ms overlapping gunzip 863 ms, parse 3781 ms, sort 118 ms, write 8312 ms, index 625 ms: 12 candidates, 100549 repo-hours, 866 releases; waiting 386 ms; commit 1666 ms)
2026-11-07T20:32:42Z [load_day 2023-05-23] 2023-05-23 hour 13: 182090 events in 4 files, snapshot 6479486458972483297 (task 15416 ms: download 8550 ms overlapping gunzip 882 ms, parse 4054 ms, sort 107 ms, write 8807 ms, index 610 ms: 6 candidates, 98025 repo-hours, 1000 releases; waiting 610 ms; commit 1663 ms)
2026-11-07T20:32:53Z [load_day 2023-05-23] 2023-05-23 hour 14: 178404 events in 4 files, snapshot 2466029106665059565 (task 15634 ms: download 9025 ms overlapping gunzip 940 ms, parse 4103 ms, sort 114 ms, write 9031 ms, index 617 ms: 13 candidates, 90375 repo-hours, 996 releases; waiting 535 ms; commit 1619 ms)
2026-11-07T20:32:58Z [load_day 2023-05-23] 2023-05-23 hour 15: 166809 events in 4 files, snapshot 717919774370904857 (task 14546 ms: download 8622 ms overlapping gunzip 826 ms, parse 3617 ms, sort 92 ms, write 8344 ms, index 590 ms: 9 candidates, 84055 repo-hours, 983 releases; waiting 735 ms; commit 2251 ms)
2026-11-07T20:33:08Z [load_day 2023-05-23] 2023-05-23 hour 16: 164082 events in 4 files, snapshot 7448210159127268654 (task 13799 ms: download 8139 ms overlapping gunzip 834 ms, parse 3743 ms, sort 94 ms, write 8014 ms, index 555 ms: 4 candidates, 80830 repo-hours, 825 releases; waiting 235 ms; commit 1806 ms)
2026-11-07T20:33:12Z [load_day 2023-05-23] 2023-05-23 hour 17: 169790 events in 4 files, snapshot 6381231722557678137 (task 14222 ms: download 8405 ms overlapping gunzip 845 ms, parse 3748 ms, sort 97 ms, write 8171 ms, index 555 ms: 10 candidates, 79262 repo-hours, 835 releases; waiting 520 ms; commit 1874 ms)
2026-11-07T20:33:23Z [load_day 2023-05-23] 2023-05-23 hour 18: 174798 events in 4 files, snapshot 3626704769640108856 (task 13983 ms: download 7751 ms overlapping gunzip 828 ms, parse 3666 ms, sort 104 ms, write 8192 ms, index 565 ms: 3 candidates, 82658 repo-hours, 951 releases; waiting 333 ms; commit 1947 ms)
2026-11-07T20:33:26Z [load_day 2023-05-23] 2023-05-23 hour 19: 162749 events in 4 files, snapshot 6535706315646120298 (task 13615 ms: download 5777 ms overlapping gunzip 781 ms, parse 3581 ms, sort 105 ms, write 7554 ms, index 496 ms: 6 candidates, 74346 repo-hours, 889 releases; waiting 751 ms; commit 1722 ms)
2026-11-07T20:33:37Z [load_day 2023-05-23] 2023-05-23 hour 20: 166140 events in 4 files, snapshot 6928157491388374740 (task 13582 ms: download 8031 ms overlapping gunzip 839 ms, parse 3755 ms, sort 93 ms, write 7788 ms, index 496 ms: 3 candidates, 75232 repo-hours, 854 releases; waiting 275 ms; commit 1661 ms)
2026-11-07T20:33:39Z [load_day 2023-05-23] 2023-05-23 hour 21: 147390 events in 3 files, snapshot 2180748635053110596 (task 11993 ms: download 5188 ms overlapping gunzip 748 ms, parse 3284 ms, sort 85 ms, write 6702 ms, index 400 ms: 5 candidates, 64339 repo-hours, 909 releases; waiting 503 ms; commit 1713 ms)
2026-11-07T20:33:49Z [load_day 2023-05-23] 2023-05-23 hour 22: 134010 events in 3 files, snapshot 6133158344170162074 (task 11214 ms: download 5056 ms overlapping gunzip 703 ms, parse 3030 ms, sort 79 ms, write 6251 ms, index 377 ms: 9 candidates, 57303 repo-hours, 773 releases; waiting 465 ms; commit 1748 ms)
2026-11-07T20:33:51Z [load_day 2023-05-23] 2023-05-23 hour 23: 123191 events in 3 files, snapshot 6686240816357870240 (task 10176 ms: download 4629 ms overlapping gunzip 616 ms, parse 2621 ms, sort 67 ms, write 5646 ms, index 360 ms: 5 candidates, 54620 repo-hours, 693 releases; waiting 546 ms; commit 1568 ms)
2026-11-07T20:33:58Z [load_day 2023-05-23] 2023-05-23: repo_hour merged from 24 files to 1 (1859623 rows, 7606 ms)
2026-11-07T20:34:00Z [load_day 2023-05-23] 2023-05-23: index committed in 9432 ms (24 + 1 + 24 files)
2026-11-07T20:34:00Z [load_day 2023-05-23] 2023-05-23: 3876374 events in 24 hours
2026-11-07T20:34:00Z [load_day 2023-05-23] attempt 1 succeeded