orcaLog in

Runs · run 1119

load_day 2023-05-25, attempt 1 succeeded

Host
caf251265124:401648
Started
2026-11-07T20:37:08Z
Ended
2026-11-07T20:40:12Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1119/load_day/2023-05-25/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T20:37:08Z [load_day 2023-05-25] attempt 1 started
2026-11-07T20:37:10Z [load_day 2023-05-25] 2023-05-25: 2 hours in flight, about 1200 MB
2026-11-07T20:37:24Z [load_day 2023-05-25] 2023-05-25 hour 0: 147858 events in 3 files, snapshot 2143440130763478768 (task 11268 ms: download 4999 ms overlapping gunzip 630 ms, parse 2961 ms, sort 85 ms, write 6243 ms, index 518 ms: 8 candidates, 77286 repo-hours, 891 releases; waiting 504 ms; commit 1762 ms)
2026-11-07T20:37:26Z [load_day 2023-05-25] 2023-05-25 hour 1: 151543 events in 4 files, snapshot 1357418146184206676 (task 11106 ms: download 4589 ms overlapping gunzip 603 ms, parse 2842 ms, sort 87 ms, write 6197 ms, index 573 ms: 5 candidates, 86222 repo-hours, 762 releases; waiting 499 ms; commit 1762 ms)
2026-11-07T20:37:36Z [load_day 2023-05-25] 2023-05-25 hour 2: 136235 events in 3 files, snapshot 8569088341197810455 (task 11201 ms: download 4888 ms overlapping gunzip 591 ms, parse 2569 ms, sort 88 ms, write 5637 ms, index 420 ms: 5 candidates, 63978 repo-hours, 720 releases; waiting 1170 ms; commit 1614 ms)
2026-11-07T20:37:38Z [load_day 2023-05-25] 2023-05-25 hour 3: 130748 events in 3 files, snapshot 4512200719013771366 (task 9896 ms: download 4125 ms overlapping gunzip 577 ms, parse 2562 ms, sort 75 ms, write 5429 ms, index 385 ms: 7 candidates, 57514 repo-hours, 912 releases; waiting 538 ms; commit 1691 ms)
2026-11-07T20:37:46Z [load_day 2023-05-25] 2023-05-25 hour 4: 123181 events in 3 files, snapshot 5828897889637527709 (task 8774 ms: download 1943 ms overlapping gunzip 533 ms, parse 2285 ms, sort 72 ms, write 4762 ms, index 344 ms: 1 candidates, 53689 repo-hours, 623 releases; waiting 518 ms; commit 1743 ms)
2026-11-07T20:37:49Z [load_day 2023-05-25] 2023-05-25 hour 5: 135063 events in 3 files, snapshot 1210392202189495007 (task 10330 ms: download 4272 ms overlapping gunzip 604 ms, parse 2647 ms, sort 70 ms, write 5616 ms, index 389 ms: 9 candidates, 60155 repo-hours, 768 releases; waiting 563 ms; commit 1584 ms)
2026-11-07T20:38:00Z [load_day 2023-05-25] 2023-05-25 hour 6: 166016 events in 4 files, snapshot 2096284789023606797 (task 12874 ms: download 5090 ms overlapping gunzip 728 ms, parse 3210 ms, sort 85 ms, write 7289 ms, index 533 ms: 9 candidates, 77684 repo-hours, 1028 releases; waiting 596 ms; commit 1913 ms)
2026-11-07T20:38:05Z [load_day 2023-05-25] 2023-05-25 hour 7: 165719 events in 4 files, snapshot 1669016728945685088 (task 14749 ms: download 9021 ms overlapping gunzip 801 ms, parse 3570 ms, sort 102 ms, write 7766 ms, index 544 ms: 6 candidates, 77779 repo-hours, 997 releases; waiting 1677 ms; commit 1778 ms)
2026-11-07T20:38:15Z [load_day 2023-05-25] 2023-05-25 hour 8: 177060 events in 4 files, snapshot 7834743093785639295 (task 14530 ms: download 8143 ms overlapping gunzip 858 ms, parse 3830 ms, sort 96 ms, write 8624 ms, index 595 ms: 7 candidates, 84082 repo-hours, 1112 releases; waiting 269 ms; commit 1657 ms)
2026-11-07T20:38:21Z [load_day 2023-05-25] 2023-05-25 hour 9: 169008 events in 4 files, snapshot 7005033470401226587 (task 14969 ms: download 8772 ms overlapping gunzip 901 ms, parse 3926 ms, sort 115 ms, write 8646 ms, index 595 ms: 2 candidates, 80202 repo-hours, 1096 releases; waiting 523 ms; commit 1596 ms)
2026-11-07T20:38:29Z [load_day 2023-05-25] 2023-05-25 hour 10: 161926 events in 4 files, snapshot 4401675943037898672 (task 13385 ms: download 5682 ms overlapping gunzip 774 ms, parse 3419 ms, sort 97 ms, write 7565 ms, index 525 ms: 6 candidates, 76112 repo-hours, 945 releases; waiting 677 ms; commit 1829 ms)
2026-11-07T20:38:35Z [load_day 2023-05-25] 2023-05-25 hour 11: 159329 events in 4 files, snapshot 5816423844484761633 (task 13472 ms: download 5579 ms overlapping gunzip 768 ms, parse 3448 ms, sort 98 ms, write 7559 ms, index 484 ms: 6 candidates, 74219 repo-hours, 966 releases; waiting 726 ms; commit 1802 ms)
2026-11-07T20:38:45Z [load_day 2023-05-25] 2023-05-25 hour 12: 188864 events in 4 files, snapshot 7495622617473082746 (task 14284 ms: download 7840 ms overlapping gunzip 800 ms, parse 3679 ms, sort 110 ms, write 8273 ms, index 635 ms: 4 candidates, 100007 repo-hours, 919 releases; waiting 477 ms; commit 1667 ms)
2026-11-07T20:38:51Z [load_day 2023-05-25] 2023-05-25 hour 13: 186909 events in 4 files, snapshot 974517173024414803 (task 15279 ms: download 8305 ms overlapping gunzip 857 ms, parse 3908 ms, sort 102 ms, write 8879 ms, index 628 ms: 5 candidates, 99774 repo-hours, 1059 releases; waiting 611 ms; commit 1631 ms)
2026-11-07T20:39:00Z [load_day 2023-05-25] 2023-05-25 hour 14: 174592 events in 4 files, snapshot 516365319453168301 (task 15022 ms: download 8685 ms overlapping gunzip 892 ms, parse 3938 ms, sort 100 ms, write 8640 ms, index 585 ms: 8 candidates, 86735 repo-hours, 1169 releases; waiting 522 ms; commit 1650 ms)
2026-11-07T20:39:07Z [load_day 2023-05-25] 2023-05-25 hour 15: 170204 events in 4 files, snapshot 361239673946941243 (task 14930 ms: download 8443 ms overlapping gunzip 845 ms, parse 3716 ms, sort 104 ms, write 8841 ms, index 616 ms: 6 candidates, 84612 repo-hours, 1078 releases; waiting 514 ms; commit 1809 ms)
2026-11-07T20:39:17Z [load_day 2023-05-25] 2023-05-25 hour 16: 162679 events in 4 files, snapshot 5946042655686451006 (task 14053 ms: download 5866 ms overlapping gunzip 809 ms, parse 3602 ms, sort 97 ms, write 8146 ms, index 554 ms: 7 candidates, 78566 repo-hours, 1058 releases; waiting 509 ms; commit 2897 ms)
2026-11-07T20:39:24Z [load_day 2023-05-25] 2023-05-25 hour 17: 177329 events in 4 files, snapshot 5547354774254286508 (task 15120 ms: download 8739 ms overlapping gunzip 916 ms, parse 4094 ms, sort 113 ms, write 8682 ms, index 522 ms: 10 candidates, 79055 repo-hours, 1000 releases; waiting 539 ms; commit 1534 ms)
2026-11-07T20:39:32Z [load_day 2023-05-25] 2023-05-25 hour 18: 171201 events in 4 files, snapshot 855866275413959745 (task 13728 ms: download 7581 ms overlapping gunzip 803 ms, parse 3556 ms, sort 104 ms, write 7908 ms, index 546 ms: 6 candidates, 80587 repo-hours, 1064 releases; waiting 212 ms; commit 1788 ms)
2026-11-07T20:39:38Z [load_day 2023-05-25] 2023-05-25 hour 19: 161004 events in 4 files, snapshot 146278987431363272 (task 13674 ms: download 6008 ms overlapping gunzip 837 ms, parse 3726 ms, sort 97 ms, write 7566 ms, index 482 ms: 4 candidates, 70721 repo-hours, 915 releases; waiting 680 ms; commit 1713 ms)
2026-11-07T20:39:45Z [load_day 2023-05-25] 2023-05-25 hour 20: 159001 events in 4 files, snapshot 6142868111298078828 (task 13120 ms: download 5154 ms overlapping gunzip 818 ms, parse 3624 ms, sort 103 ms, write 7589 ms, index 506 ms: 8 candidates, 70392 repo-hours, 921 releases; waiting 183 ms; commit 1757 ms)
2026-11-07T20:39:52Z [load_day 2023-05-25] 2023-05-25 hour 21: 147979 events in 3 files, snapshot 8033004381221659339 (task 12512 ms: download 5221 ms overlapping gunzip 767 ms, parse 3333 ms, sort 121 ms, write 7058 ms, index 436 ms: 7 candidates, 62765 repo-hours, 1062 releases; waiting 508 ms; commit 1757 ms)
2026-11-07T20:39:58Z [load_day 2023-05-25] 2023-05-25 hour 22: 136541 events in 3 files, snapshot 1840507293967244098 (task 11436 ms: download 5114 ms overlapping gunzip 721 ms, parse 3106 ms, sort 84 ms, write 6308 ms, index 375 ms: 5 candidates, 56832 repo-hours, 788 releases; waiting 518 ms; commit 1709 ms)
2026-11-07T20:40:03Z [load_day 2023-05-25] 2023-05-25 hour 23: 129788 events in 3 files, snapshot 4643064826006753705 (task 10406 ms: download 4583 ms overlapping gunzip 610 ms, parse 2776 ms, sort 69 ms, write 5732 ms, index 388 ms: 3 candidates, 54992 repo-hours, 673 releases; waiting 539 ms; commit 1746 ms)
2026-11-07T20:40:11Z [load_day 2023-05-25] 2023-05-25: repo_hour merged from 24 files to 1 (1793960 rows, 7695 ms)
2026-11-07T20:40:12Z [load_day 2023-05-25] 2023-05-25: index committed in 9383 ms (24 + 1 + 24 files)
2026-11-07T20:40:12Z [load_day 2023-05-25] 2023-05-25: 3789777 events in 24 hours
2026-11-07T20:40:12Z [load_day 2023-05-25] attempt 1 succeeded