orcaLog in

Runs · run 954

load_day 2023-03-31, attempt 1 succeeded

Host
caf251265124:219860
Started
2026-11-07T17:56:40Z
Ended
2026-11-07T17:59:56Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/954/load_day/2023-03-31/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T17:56:40Z [load_day 2023-03-31] attempt 1 started
2026-11-07T17:56:44Z [load_day 2023-03-31] 2023-03-31: 2 hours in flight, about 1200 MB
2026-11-07T17:56:58Z [load_day 2023-03-31] 2023-03-31 hour 0: 151474 events in 4 files, snapshot 4780699153662783896 (task 11537 ms: download 4955 ms overlapping gunzip 667 ms, parse 3052 ms, sort 102 ms, write 6362 ms, index 495 ms: 3 candidates, 75742 repo-hours, 905 releases; waiting 504 ms; commit 1522 ms)
2026-11-07T17:57:00Z [load_day 2023-03-31] 2023-03-31 hour 1: 162653 events in 4 files, snapshot 7235871468900320883 (task 11820 ms: download 4755 ms overlapping gunzip 667 ms, parse 2975 ms, sort 100 ms, write 6695 ms, index 577 ms: 8 candidates, 87355 repo-hours, 838 releases; waiting 503 ms; commit 1720 ms)
2026-11-07T17:57:10Z [load_day 2023-03-31] 2023-03-31 hour 2: 140968 events in 3 files, snapshot 1642253612875977736 (task 11101 ms: download 4645 ms overlapping gunzip 634 ms, parse 2669 ms, sort 89 ms, write 6106 ms, index 486 ms: 1 candidates, 67237 repo-hours, 707 releases; waiting 763 ms; commit 1398 ms)
2026-11-07T17:57:12Z [load_day 2023-03-31] 2023-03-31 hour 3: 139069 events in 3 files, snapshot 3946334663484357021 (task 10892 ms: download 4813 ms overlapping gunzip 638 ms, parse 2801 ms, sort 92 ms, write 5692 ms, index 408 ms: 6 candidates, 60670 repo-hours, 1100 releases; waiting 887 ms; commit 1446 ms)
2026-11-07T17:57:20Z [load_day 2023-03-31] 2023-03-31 hour 4: 129711 events in 3 files, snapshot 5739630382901853458 (task 9511 ms: download 3973 ms overlapping gunzip 563 ms, parse 2446 ms, sort 74 ms, write 5334 ms, index 363 ms: 3 candidates, 55767 repo-hours, 648 releases; waiting 399 ms; commit 1611 ms)
2026-11-07T17:57:24Z [load_day 2023-03-31] 2023-03-31 hour 5: 143586 events in 3 files, snapshot 2504048434642548100 (task 11039 ms: download 4685 ms overlapping gunzip 656 ms, parse 2888 ms, sort 94 ms, write 6168 ms, index 414 ms: 5 candidates, 62177 repo-hours, 754 releases; waiting 525 ms; commit 1673 ms)
2026-11-07T17:57:34Z [load_day 2023-03-31] 2023-03-31 hour 6: 170284 events in 4 files, snapshot 5285802034893450308 (task 12909 ms: download 5169 ms overlapping gunzip 758 ms, parse 3251 ms, sort 112 ms, write 7374 ms, index 516 ms: 2 candidates, 77252 repo-hours, 933 releases; waiting 553 ms; commit 1603 ms)
2026-11-07T17:57:39Z [load_day 2023-03-31] 2023-03-31 hour 7: 177197 events in 4 files, snapshot 1657061168593965665 (task 14225 ms: download 8045 ms overlapping gunzip 834 ms, parse 3724 ms, sort 108 ms, write 8173 ms, index 549 ms: 4 candidates, 79046 repo-hours, 987 releases; waiting 550 ms; commit 1677 ms)
2026-11-07T17:57:52Z [load_day 2023-03-31] 2023-03-31 hour 8: 183385 events in 4 files, snapshot 8702268252939368185 (task 16584 ms: download 9750 ms overlapping gunzip 915 ms, parse 4021 ms, sort 125 ms, write 8993 ms, index 650 ms: 7 candidates, 85668 repo-hours, 988 releases; waiting 1591 ms; commit 1837 ms)
2026-11-07T17:57:55Z [load_day 2023-03-31] 2023-03-31 hour 9: 181985 events in 4 files, snapshot 1251303620558330394 (task 15881 ms: download 8767 ms overlapping gunzip 945 ms, parse 4098 ms, sort 127 ms, write 9207 ms, index 639 ms: 10 candidates, 82818 repo-hours, 1131 releases; waiting 533 ms; commit 1466 ms)
2026-11-07T17:58:08Z [load_day 2023-03-31] 2023-03-31 hour 10: 183443 events in 4 files, snapshot 7088227597826945074 (task 14947 ms: download 8388 ms overlapping gunzip 893 ms, parse 3980 ms, sort 139 ms, write 8866 ms, index 538 ms: 5 candidates, 81080 repo-hours, 1139 releases; waiting 223 ms; commit 1687 ms)
2026-11-07T17:58:12Z [load_day 2023-03-31] 2023-03-31 hour 11: 183929 events in 4 files, snapshot 8717176061216694772 (task 15720 ms: download 8737 ms overlapping gunzip 915 ms, parse 4066 ms, sort 125 ms, write 9132 ms, index 559 ms: 0 candidates, 79504 repo-hours, 1338 releases; waiting 540 ms; commit 1504 ms)
2026-11-07T17:58:24Z [load_day 2023-03-31] 2023-03-31 hour 12: 205184 events in 5 files, snapshot 4805904422020090912 (task 15859 ms: download 8441 ms overlapping gunzip 871 ms, parse 4007 ms, sort 122 ms, write 9347 ms, index 670 ms: 7 candidates, 102008 repo-hours, 1293 releases; waiting 531 ms; commit 1565 ms)
2026-11-07T17:58:28Z [load_day 2023-03-31] 2023-03-31 hour 13: 189771 events in 4 files, snapshot 2311518262907630198 (task 15679 ms: download 8138 ms overlapping gunzip 895 ms, parse 3982 ms, sort 126 ms, write 9212 ms, index 633 ms: 7 candidates, 99302 repo-hours, 957 releases; waiting 283 ms; commit 1561 ms)
2026-11-07T17:58:43Z [load_day 2023-03-31] 2023-03-31 hour 14: 185946 events in 4 files, snapshot 4365578403776846113 (task 17764 ms: download 10247 ms overlapping gunzip 1126 ms, parse 4566 ms, sort 134 ms, write 10428 ms, index 633 ms: 9 candidates, 88268 repo-hours, 1074 releases; waiting 584 ms; commit 1526 ms)
2026-11-07T17:58:45Z [load_day 2023-03-31] 2023-03-31 hour 15: 179521 events in 4 files, snapshot 3208710232341147718 (task 16016 ms: download 9426 ms overlapping gunzip 1012 ms, parse 4303 ms, sort 116 ms, write 9438 ms, index 572 ms: 4 candidates, 85075 repo-hours, 1073 releases; waiting 316 ms; commit 1569 ms)
2026-11-07T17:58:58Z [load_day 2023-03-31] 2023-03-31 hour 16: 174270 events in 4 files, snapshot 6997670646229154134 (task 14658 ms: download 8420 ms overlapping gunzip 853 ms, parse 3703 ms, sort 115 ms, write 8484 ms, index 545 ms: 7 candidates, 84572 repo-hours, 907 releases; waiting 607 ms; commit 1613 ms)
2026-11-07T17:59:01Z [load_day 2023-03-31] 2023-03-31 hour 17: 176171 events in 4 files, snapshot 1511213851576114357 (task 14879 ms: download 8743 ms overlapping gunzip 868 ms, parse 3761 ms, sort 107 ms, write 8699 ms, index 554 ms: 8 candidates, 78579 repo-hours, 910 releases; waiting 556 ms; commit 1687 ms)
2026-11-07T17:59:16Z [load_day 2023-03-31] 2023-03-31 hour 18: 173374 events in 4 files, snapshot 3450592579727917385 (task 14199 ms: download 8169 ms overlapping gunzip 858 ms, parse 3771 ms, sort 121 ms, write 8130 ms, index 539 ms: 8 candidates, 80396 repo-hours, 971 releases; waiting 446 ms; commit 4335 ms)
2026-11-07T17:59:21Z [load_day 2023-03-31] 2023-03-31 hour 19: 172379 events in 4 files, snapshot 2049855068541750176 (task 17712 ms: download 8976 ms overlapping gunzip 887 ms, parse 3723 ms, sort 123 ms, write 9836 ms, index 1677 ms: 4 candidates, 78069 repo-hours, 1037 releases; waiting 1115 ms; commit 2719 ms)
2026-11-07T17:59:31Z [load_day 2023-03-31] 2023-03-31 hour 20: 168376 events in 4 files, snapshot 2726809104686108572 (task 13908 ms: download 8153 ms overlapping gunzip 801 ms, parse 3602 ms, sort 120 ms, write 7860 ms, index 657 ms: 8 candidates, 75003 repo-hours, 906 releases; waiting 541 ms; commit 1662 ms)
2026-11-07T17:59:34Z [load_day 2023-03-31] 2023-03-31 hour 21: 152130 events in 4 files, snapshot 5460042529244205437 (task 12324 ms: download 5212 ms overlapping gunzip 712 ms, parse 3167 ms, sort 109 ms, write 6984 ms, index 477 ms: 5 candidates, 64965 repo-hours, 903 releases; waiting 550 ms; commit 1432 ms)
2026-11-07T17:59:45Z [load_day 2023-03-31] 2023-03-31 hour 22: 133704 events in 3 files, snapshot 3913875719803063700 (task 13258 ms: download 7243 ms overlapping gunzip 659 ms, parse 2875 ms, sort 86 ms, write 5979 ms, index 355 ms: 6 candidates, 55883 repo-hours, 765 releases; waiting 3012 ms; commit 1433 ms)
2026-11-07T17:59:47Z [load_day 2023-03-31] 2023-03-31 hour 23: 121265 events in 3 files, snapshot 3687844661916083134 (task 9685 ms: download 4466 ms overlapping gunzip 593 ms, parse 2644 ms, sort 77 ms, write 5165 ms, index 333 ms: 3 candidates, 51601 repo-hours, 669 releases; waiting 525 ms; commit 1475 ms)
2026-11-07T17:59:54Z [load_day 2023-03-31] 2023-03-31: repo_hour merged from 24 files to 1 (1838037 rows, 7975 ms)
2026-11-07T17:59:56Z [load_day 2023-03-31] 2023-03-31: index committed in 9881 ms (23 + 1 + 24 files)
2026-11-07T17:59:56Z [load_day 2023-03-31] 2023-03-31: 3979775 events in 24 hours
2026-11-07T17:59:56Z [load_day 2023-03-31] attempt 1 succeeded