orcaLog in

Runs · run 1014

load_day 2023-04-20, attempt 1 succeeded

Host
caf251265124:287755
Started
2026-11-07T18:56:22Z
Ended
2026-11-07T18:59:42Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1014/load_day/2023-04-20/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T18:56:22Z [load_day 2023-04-20] attempt 1 started
2026-11-07T18:56:23Z [load_day 2023-04-20] 2023-04-20: 2 hours in flight, about 1200 MB
2026-11-07T18:56:37Z [load_day 2023-04-20] 2023-04-20 hour 0: 160744 events in 4 files, snapshot 5432026804851801140 (task 11265 ms: download 4581 ms overlapping gunzip 661 ms, parse 3059 ms, sort 98 ms, write 6258 ms, index 571 ms: 4 candidates, 80568 repo-hours, 825 releases; waiting 253 ms; commit 1546 ms)
2026-11-07T18:56:39Z [load_day 2023-04-20] 2023-04-20 hour 1: 180188 events in 4 files, snapshot 5042139838350532047 (task 12646 ms: download 7040 ms overlapping gunzip 794 ms, parse 3554 ms, sort 118 ms, write 7111 ms, index 551 ms: 5 candidates, 90136 repo-hours, 727 releases; waiting 209 ms; commit 1640 ms)
2026-11-07T18:56:49Z [load_day 2023-04-20] 2023-04-20 hour 2: 148020 events in 3 files, snapshot 51885127519043753 (task 11143 ms: download 4610 ms overlapping gunzip 660 ms, parse 2901 ms, sort 91 ms, write 6210 ms, index 474 ms: 3 candidates, 67356 repo-hours, 678 releases; waiting 522 ms; commit 1802 ms)
2026-11-07T18:56:51Z [load_day 2023-04-20] 2023-04-20 hour 3: 143726 events in 3 files, snapshot 4156689802967048075 (task 10451 ms: download 4358 ms overlapping gunzip 627 ms, parse 2621 ms, sort 91 ms, write 5818 ms, index 403 ms: 2 candidates, 62306 repo-hours, 783 releases; waiting 554 ms; commit 1606 ms)
2026-11-07T18:57:00Z [load_day 2023-04-20] 2023-04-20 hour 4: 133682 events in 3 files, snapshot 3411126907621965518 (task 9708 ms: download 4188 ms overlapping gunzip 574 ms, parse 2475 ms, sort 73 ms, write 5289 ms, index 375 ms: 2 candidates, 55760 repo-hours, 565 releases; waiting 579 ms; commit 1735 ms)
2026-11-07T18:57:03Z [load_day 2023-04-20] 2023-04-20 hour 5: 148612 events in 3 files, snapshot 6733351232957513230 (task 11219 ms: download 4879 ms overlapping gunzip 736 ms, parse 3102 ms, sort 102 ms, write 5984 ms, index 408 ms: 3 candidates, 62626 repo-hours, 755 releases; waiting 639 ms; commit 1683 ms)
2026-11-07T18:57:14Z [load_day 2023-04-20] 2023-04-20 hour 6: 172460 events in 4 files, snapshot 8094459428674386097 (task 13900 ms: download 7877 ms overlapping gunzip 805 ms, parse 3473 ms, sort 124 ms, write 7808 ms, index 556 ms: 3 candidates, 78474 repo-hours, 915 releases; waiting 806 ms; commit 1542 ms)
2026-11-07T18:57:22Z [load_day 2023-04-20] 2023-04-20 hour 7: 168854 events in 4 files, snapshot 3048214526281773690 (task 14059 ms: download 8111 ms overlapping gunzip 820 ms, parse 3560 ms, sort 111 ms, write 8106 ms, index 533 ms: 3 candidates, 77917 repo-hours, 812 releases; waiting 626 ms; commit 5176 ms)
2026-11-07T18:57:30Z [load_day 2023-04-20] 2023-04-20 hour 8: 179480 events in 4 files, snapshot 4340674582940122363 (task 14746 ms: download 8372 ms overlapping gunzip 874 ms, parse 3867 ms, sort 105 ms, write 8517 ms, index 536 ms: 4 candidates, 82639 repo-hours, 1028 releases; waiting 518 ms; commit 1534 ms)
2026-11-07T18:57:35Z [load_day 2023-04-20] 2023-04-20 hour 9: 185591 events in 4 files, snapshot 4893412206526843599 (task 15789 ms: download 8956 ms overlapping gunzip 946 ms, parse 4213 ms, sort 112 ms, write 8953 ms, index 594 ms: 6 candidates, 84703 repo-hours, 1142 releases; waiting 598 ms; commit 1733 ms)
2026-11-07T18:57:46Z [load_day 2023-04-20] 2023-04-20 hour 10: 177804 events in 4 files, snapshot 1994052928692932709 (task 15449 ms: download 9314 ms overlapping gunzip 906 ms, parse 3952 ms, sort 115 ms, write 9103 ms, index 534 ms: 6 candidates, 81221 repo-hours, 1108 releases; waiting 529 ms; commit 1447 ms)
2026-11-07T18:57:49Z [load_day 2023-04-20] 2023-04-20 hour 11: 179916 events in 4 files, snapshot 105543854647661121 (task 14391 ms: download 8219 ms overlapping gunzip 868 ms, parse 3739 ms, sort 102 ms, write 8356 ms, index 531 ms: 4 candidates, 79955 repo-hours, 999 releases; waiting 510 ms; commit 1482 ms)
2026-11-07T18:58:03Z [load_day 2023-04-20] 2023-04-20 hour 12: 203096 events in 5 files, snapshot 3243575433561649154 (task 15794 ms: download 8455 ms overlapping gunzip 860 ms, parse 3875 ms, sort 124 ms, write 9115 ms, index 667 ms: 5 candidates, 102337 repo-hours, 1010 releases; waiting 846 ms; commit 1747 ms)
2026-11-07T18:58:08Z [load_day 2023-04-20] 2023-04-20 hour 13: 200400 events in 5 files, snapshot 9119941126263899558 (task 17255 ms: download 9673 ms overlapping gunzip 971 ms, parse 4282 ms, sort 127 ms, write 9541 ms, index 695 ms: 10 candidates, 101387 repo-hours, 1057 releases; waiting 1252 ms; commit 1846 ms)
2026-11-07T18:58:20Z [load_day 2023-04-20] 2023-04-20 hour 14: 193219 events in 4 files, snapshot 6907772238627444942 (task 16002 ms: download 8582 ms overlapping gunzip 974 ms, parse 4194 ms, sort 132 ms, write 9627 ms, index 668 ms: 5 candidates, 91262 repo-hours, 1150 releases; waiting 184 ms; commit 1938 ms)
2026-11-07T18:58:24Z [load_day 2023-04-20] 2023-04-20 hour 15: 178531 events in 4 files, snapshot 5100213774883927762 (task 16006 ms: download 9245 ms overlapping gunzip 948 ms, parse 4054 ms, sort 115 ms, write 9385 ms, index 579 ms: 4 candidates, 86057 repo-hours, 982 releases; waiting 656 ms; commit 1772 ms)
2026-11-07T18:58:35Z [load_day 2023-04-20] 2023-04-20 hour 16: 177293 events in 4 files, snapshot 6328280727652769344 (task 15357 ms: download 9037 ms overlapping gunzip 907 ms, parse 4001 ms, sort 117 ms, write 8880 ms, index 588 ms: 6 candidates, 84692 repo-hours, 849 releases; waiting 560 ms; commit 1667 ms)
2026-11-07T18:58:41Z [load_day 2023-04-20] 2023-04-20 hour 17: 186279 events in 4 files, snapshot 6936981224818183400 (task 16062 ms: download 8988 ms overlapping gunzip 981 ms, parse 4226 ms, sort 125 ms, write 9338 ms, index 542 ms: 3 candidates, 79661 repo-hours, 898 releases; waiting 507 ms; commit 1848 ms)
2026-11-07T18:58:52Z [load_day 2023-04-20] 2023-04-20 hour 18: 189766 events in 4 files, snapshot 1068554391482353244 (task 15789 ms: download 8890 ms overlapping gunzip 941 ms, parse 4117 ms, sort 125 ms, write 9138 ms, index 549 ms: 6 candidates, 85217 repo-hours, 1083 releases; waiting 527 ms; commit 1524 ms)
2026-11-07T18:58:58Z [load_day 2023-04-20] 2023-04-20 hour 19: 182101 events in 4 files, snapshot 3841028091016326714 (task 15751 ms: download 8990 ms overlapping gunzip 985 ms, parse 4247 ms, sort 114 ms, write 9069 ms, index 540 ms: 2 candidates, 80666 repo-hours, 1015 releases; waiting 528 ms; commit 1676 ms)
2026-11-07T18:59:10Z [load_day 2023-04-20] 2023-04-20 hour 20: 187042 events in 4 files, snapshot 5636638277741879427 (task 17222 ms: download 10532 ms overlapping gunzip 1097 ms, parse 4682 ms, sort 117 ms, write 9928 ms, index 537 ms: 3 candidates, 83492 repo-hours, 895 releases; waiting 522 ms; commit 1740 ms)
2026-11-07T18:59:13Z [load_day 2023-04-20] 2023-04-20 hour 21: 162251 events in 4 files, snapshot 707270356808299330 (task 14089 ms: download 8489 ms overlapping gunzip 831 ms, parse 3619 ms, sort 113 ms, write 7834 ms, index 478 ms: 8 candidates, 68545 repo-hours, 969 releases; waiting 837 ms; commit 1708 ms)
2026-11-07T18:59:25Z [load_day 2023-04-20] 2023-04-20 hour 22: 148717 events in 3 files, snapshot 3402804215653061855 (task 12392 ms: download 5355 ms overlapping gunzip 789 ms, parse 3335 ms, sort 109 ms, write 7024 ms, index 458 ms: 2 candidates, 60849 repo-hours, 805 releases; waiting 377 ms; commit 3757 ms)
2026-11-07T18:59:32Z [load_day 2023-04-20] 2023-04-20 hour 23: 130692 events in 3 files, snapshot 550107379487899474 (task 13178 ms: download 4521 ms overlapping gunzip 620 ms, parse 2669 ms, sort 78 ms, write 6020 ms, index 2955 ms: 1 candidates, 54763 repo-hours, 719 releases; waiting 528 ms; commit 6657 ms)
2026-11-07T18:59:40Z [load_day 2023-04-20] 2023-04-20: repo_hour merged from 24 files to 1 (1882589 rows, 7894 ms)
2026-11-07T18:59:42Z [load_day 2023-04-20] 2023-04-20: index committed in 9440 ms (24 + 1 + 24 files)
2026-11-07T18:59:42Z [load_day 2023-04-20] 2023-04-20: 4118464 events in 24 hours
2026-11-07T18:59:42Z [load_day 2023-04-20] attempt 1 succeeded