orcaLog in

Runs · run 1098

load_day 2023-05-18, attempt 1 succeeded

Host
caf251265124:379668
Started
2026-11-07T20:17:43Z
Ended
2026-11-07T20:20:40Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1098/load_day/2023-05-18/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T20:17:43Z [load_day 2023-05-18] attempt 1 started
2026-11-07T20:17:45Z [load_day 2023-05-18] 2023-05-18: 2 hours in flight, about 1200 MB
2026-11-07T20:17:59Z [load_day 2023-05-18] 2023-05-18 hour 0: 144395 events in 3 files, snapshot 1389792003958533558 (task 10857 ms: download 4671 ms overlapping gunzip 630 ms, parse 2934 ms, sort 86 ms, write 5843 ms, index 481 ms: 5 candidates, 76448 repo-hours, 935 releases; waiting 527 ms; commit 1779 ms)
2026-11-07T20:18:01Z [load_day 2023-05-18] 2023-05-18 hour 1: 153400 events in 4 files, snapshot 6365037701663294172 (task 10693 ms: download 4239 ms overlapping gunzip 627 ms, parse 2964 ms, sort 87 ms, write 5954 ms, index 559 ms: 7 candidates, 88174 repo-hours, 724 releases; waiting 200 ms; commit 1697 ms)
2026-11-07T20:18:10Z [load_day 2023-05-18] 2023-05-18 hour 2: 131595 events in 3 files, snapshot 5888807866925917411 (task 9731 ms: download 4088 ms overlapping gunzip 567 ms, parse 2526 ms, sort 60 ms, write 5453 ms, index 441 ms: 5 candidates, 62249 repo-hours, 768 releases; waiting 453 ms; commit 1738 ms)
2026-11-07T20:18:11Z [load_day 2023-05-18] 2023-05-18 hour 3: 127086 events in 3 files, snapshot 6019596242577237644 (task 9501 ms: download 2578 ms overlapping gunzip 534 ms, parse 2418 ms, sort 64 ms, write 5088 ms, index 372 ms: 2 candidates, 57739 repo-hours, 787 releases; waiting 734 ms; commit 1682 ms)
2026-11-07T20:18:19Z [load_day 2023-05-18] 2023-05-18 hour 4: 120356 events in 3 files, snapshot 1137455463745170136 (task 8972 ms: download 2038 ms overlapping gunzip 508 ms, parse 2285 ms, sort 78 ms, write 4880 ms, index 384 ms: 3 candidates, 52606 repo-hours, 551 releases; waiting 518 ms; commit 1735 ms)
2026-11-07T20:18:22Z [load_day 2023-05-18] 2023-05-18 hour 5: 132346 events in 3 files, snapshot 8175755823112132786 (task 10058 ms: download 4152 ms overlapping gunzip 591 ms, parse 2600 ms, sort 83 ms, write 5359 ms, index 385 ms: 5 candidates, 58693 repo-hours, 735 releases; waiting 629 ms; commit 1869 ms)
2026-11-07T20:18:32Z [load_day 2023-05-18] 2023-05-18 hour 6: 156511 events in 4 files, snapshot 4079246751042949728 (task 11427 ms: download 4449 ms overlapping gunzip 682 ms, parse 2990 ms, sort 99 ms, write 6572 ms, index 494 ms: 7 candidates, 73420 repo-hours, 818 releases; waiting 280 ms; commit 1733 ms)
2026-11-07T20:18:35Z [load_day 2023-05-18] 2023-05-18 hour 7: 159500 events in 4 files, snapshot 1127312592926623486 (task 12405 ms: download 5132 ms overlapping gunzip 709 ms, parse 3157 ms, sort 93 ms, write 6822 ms, index 529 ms: 13 candidates, 72518 repo-hours, 868 releases; waiting 782 ms; commit 1767 ms)
2026-11-07T20:18:46Z [load_day 2023-05-18] 2023-05-18 hour 8: 169680 events in 4 files, snapshot 7463060704431290311 (task 13105 ms: download 7728 ms overlapping gunzip 784 ms, parse 3465 ms, sort 105 ms, write 7455 ms, index 528 ms: 6 candidates, 79444 repo-hours, 934 releases; waiting 508 ms; commit 1617 ms)
2026-11-07T20:18:50Z [load_day 2023-05-18] 2023-05-18 hour 9: 176891 events in 4 files, snapshot 3445278379421413034 (task 13933 ms: download 7812 ms overlapping gunzip 825 ms, parse 3712 ms, sort 107 ms, write 8038 ms, index 546 ms: 8 candidates, 81160 repo-hours, 1109 releases; waiting 498 ms; commit 1883 ms)
2026-11-07T20:19:00Z [load_day 2023-05-18] 2023-05-18 hour 10: 173524 events in 4 files, snapshot 4899746279248384951 (task 13053 ms: download 7368 ms overlapping gunzip 772 ms, parse 3483 ms, sort 100 ms, write 7652 ms, index 522 ms: 4 candidates, 77976 repo-hours, 1001 releases; waiting 270 ms; commit 1838 ms)
2026-11-07T20:19:04Z [load_day 2023-05-18] 2023-05-18 hour 11: 174934 events in 4 files, snapshot 4073245871830870566 (task 13308 ms: download 7317 ms overlapping gunzip 775 ms, parse 3487 ms, sort 107 ms, write 7549 ms, index 562 ms: 11 candidates, 79050 repo-hours, 943 releases; waiting 505 ms; commit 1673 ms)
2026-11-07T20:19:17Z [load_day 2023-05-18] 2023-05-18 hour 12: 221912 events in 5 files, snapshot 42296845746703828 (task 16621 ms: download 8974 ms overlapping gunzip 909 ms, parse 4163 ms, sort 128 ms, write 9272 ms, index 684 ms: 8 candidates, 108024 repo-hours, 1042 releases; waiting 1211 ms; commit 1788 ms)
2026-11-07T20:19:21Z [load_day 2023-05-18] 2023-05-18 hour 13: 196836 events in 4 files, snapshot 167718325420177641 (task 15702 ms: download 8501 ms overlapping gunzip 884 ms, parse 3963 ms, sort 121 ms, write 8879 ms, index 674 ms: 5 candidates, 102339 repo-hours, 921 releases; waiting 915 ms; commit 1812 ms)
2026-11-07T20:19:33Z [load_day 2023-05-18] 2023-05-18 hour 14: 184019 events in 4 files, snapshot 8079489154943792235 (task 14893 ms: download 8519 ms overlapping gunzip 882 ms, parse 3827 ms, sort 108 ms, write 8620 ms, index 566 ms: 13 candidates, 87752 repo-hours, 966 releases; waiting 517 ms; commit 1735 ms)
2026-11-07T20:19:35Z [load_day 2023-05-18] 2023-05-18 hour 15: 169843 events in 4 files, snapshot 5557044143702984450 (task 13811 ms: download 8073 ms overlapping gunzip 811 ms, parse 3587 ms, sort 95 ms, write 7864 ms, index 551 ms: 11 candidates, 82008 repo-hours, 1047 releases; waiting 601 ms; commit 1711 ms)
2026-11-07T20:19:48Z [load_day 2023-05-18] 2023-05-18 hour 16: 179621 events in 4 files, snapshot 6721576450649099545 (task 14551 ms: download 8190 ms overlapping gunzip 852 ms, parse 3789 ms, sort 112 ms, write 8398 ms, index 583 ms: 9 candidates, 81438 repo-hours, 997 releases; waiting 507 ms; commit 1799 ms)
2026-11-07T20:19:50Z [load_day 2023-05-18] 2023-05-18 hour 17: 170365 events in 4 files, snapshot 4241286232019150502 (task 14312 ms: download 8149 ms overlapping gunzip 844 ms, parse 3845 ms, sort 119 ms, write 8176 ms, index 521 ms: 9 candidates, 76241 repo-hours, 941 releases; waiting 513 ms; commit 1572 ms)
2026-11-07T20:20:03Z [load_day 2023-05-18] 2023-05-18 hour 18: 177177 events in 4 files, snapshot 8515305178382464291 (task 13762 ms: download 7728 ms overlapping gunzip 804 ms, parse 3595 ms, sort 96 ms, write 7941 ms, index 594 ms: 9 candidates, 79796 repo-hours, 960 releases; waiting 511 ms; commit 1747 ms)
2026-11-07T20:20:07Z [load_day 2023-05-18] 2023-05-18 hour 19: 160304 events in 4 files, snapshot 1701390763117479018 (task 15155 ms: download 7045 ms overlapping gunzip 798 ms, parse 3628 ms, sort 99 ms, write 7787 ms, index 503 ms: 10 candidates, 70846 repo-hours, 942 releases; waiting 1954 ms; commit 1842 ms)
2026-11-07T20:20:17Z [load_day 2023-05-18] 2023-05-18 hour 20: 156348 events in 4 files, snapshot 6464727190162459558 (task 13374 ms: download 6043 ms overlapping gunzip 788 ms, parse 3432 ms, sort 98 ms, write 7335 ms, index 478 ms: 6 candidates, 70034 repo-hours, 887 releases; waiting 962 ms; commit 1595 ms)
2026-11-07T20:20:19Z [load_day 2023-05-18] 2023-05-18 hour 21: 141623 events in 3 files, snapshot 4878426149938460922 (task 11707 ms: download 5098 ms overlapping gunzip 675 ms, parse 3046 ms, sort 83 ms, write 6620 ms, index 426 ms: 7 candidates, 61218 repo-hours, 989 releases; waiting 526 ms; commit 1810 ms)
2026-11-07T20:20:29Z [load_day 2023-05-18] 2023-05-18 hour 22: 128416 events in 3 files, snapshot 4272725681739282891 (task 11052 ms: download 4888 ms overlapping gunzip 626 ms, parse 2720 ms, sort 82 ms, write 6060 ms, index 370 ms: 12 candidates, 55069 repo-hours, 833 releases; waiting 620 ms; commit 1568 ms)
2026-11-07T20:20:30Z [load_day 2023-05-18] 2023-05-18 hour 23: 121764 events in 3 files, snapshot 3166720258024547039 (task 9705 ms: download 4428 ms overlapping gunzip 564 ms, parse 2581 ms, sort 78 ms, write 5285 ms, index 344 ms: 6 candidates, 52769 repo-hours, 643 releases; waiting 588 ms; commit 1710 ms)
2026-11-07T20:20:38Z [load_day 2023-05-18] 2023-05-18: repo_hour merged from 24 files to 1 (1787011 rows, 7342 ms)
2026-11-07T20:20:40Z [load_day 2023-05-18] 2023-05-18: index committed in 9586 ms (24 + 1 + 24 files)
2026-11-07T20:20:40Z [load_day 2023-05-18] 2023-05-18: 3828446 events in 24 hours
2026-11-07T20:20:40Z [load_day 2023-05-18] attempt 1 succeeded