orcaLog in

Runs · run 1160

load_day 2023-06-07, attempt 1 succeeded

Host
3c2b29713ce8:51
Started
2026-11-07T21:13:32Z
Ended
2026-11-07T21:16:34Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1160/load_day/2023-06-07/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T21:13:32Z [load_day 2023-06-07] attempt 1 started
2026-11-07T21:13:34Z [load_day 2023-06-07] 2023-06-07: 2 hours in flight, about 1200 MB
2026-11-07T21:13:48Z [load_day 2023-06-07] 2023-06-07 hour 0: 147017 events in 3 files, snapshot 446009281057650684 (task 11150 ms: download 4791 ms overlapping gunzip 624 ms, parse 2972 ms, sort 92 ms, write 6113 ms, index 493 ms: 19 candidates, 73518 repo-hours, 981 releases; waiting 522 ms; commit 1694 ms)
2026-11-07T21:13:50Z [load_day 2023-06-07] 2023-06-07 hour 1: 159771 events in 4 files, snapshot 7004443913214170770 (task 11866 ms: download 4881 ms overlapping gunzip 650 ms, parse 3070 ms, sort 132 ms, write 6387 ms, index 570 ms: 12 candidates, 87210 repo-hours, 1002 releases; waiting 704 ms; commit 1663 ms)
2026-11-07T21:14:00Z [load_day 2023-06-07] 2023-06-07 hour 2: 143669 events in 3 files, snapshot 7980123601668682316 (task 11113 ms: download 4890 ms overlapping gunzip 643 ms, parse 2843 ms, sort 88 ms, write 5750 ms, index 471 ms: 13 candidates, 72020 repo-hours, 716 releases; waiting 952 ms; commit 1646 ms)
2026-11-07T21:14:02Z [load_day 2023-06-07] 2023-06-07 hour 3: 132215 events in 3 files, snapshot 7610048879830561197 (task 9373 ms: download 4039 ms overlapping gunzip 546 ms, parse 2426 ms, sort 73 ms, write 4984 ms, index 391 ms: 14 candidates, 59976 repo-hours, 923 releases; waiting 705 ms; commit 1690 ms)
2026-11-07T21:14:11Z [load_day 2023-06-07] 2023-06-07 hour 4: 128649 events in 3 files, snapshot 908354113875933966 (task 9848 ms: download 4176 ms overlapping gunzip 540 ms, parse 2366 ms, sort 76 ms, write 5406 ms, index 382 ms: 11 candidates, 58972 repo-hours, 701 releases; waiting 760 ms; commit 1726 ms)
2026-11-07T21:14:14Z [load_day 2023-06-07] 2023-06-07 hour 5: 144475 events in 3 files, snapshot 2457758110129089384 (task 11251 ms: download 4840 ms overlapping gunzip 693 ms, parse 3000 ms, sort 93 ms, write 6248 ms, index 415 ms: 18 candidates, 62957 repo-hours, 839 releases; waiting 538 ms; commit 1685 ms)
2026-11-07T21:14:24Z [load_day 2023-06-07] 2023-06-07 hour 6: 167496 events in 4 files, snapshot 3637095430666872694 (task 12456 ms: download 4917 ms overlapping gunzip 726 ms, parse 3214 ms, sort 84 ms, write 7101 ms, index 518 ms: 18 candidates, 80321 repo-hours, 1152 releases; waiting 521 ms; commit 1735 ms)
2026-11-07T21:14:28Z [load_day 2023-06-07] 2023-06-07 hour 7: 162249 events in 4 files, snapshot 5690886176395406340 (task 13250 ms: download 5582 ms overlapping gunzip 773 ms, parse 3392 ms, sort 107 ms, write 7310 ms, index 508 ms: 11 candidates, 76910 repo-hours, 1480 releases; waiting 861 ms; commit 1827 ms)
2026-11-07T21:14:39Z [load_day 2023-06-07] 2023-06-07 hour 8: 169458 events in 4 files, snapshot 7919207110540760560 (task 14199 ms: download 8481 ms overlapping gunzip 844 ms, parse 3755 ms, sort 103 ms, write 8138 ms, index 530 ms: 11 candidates, 80394 repo-hours, 981 releases; waiting 534 ms; commit 1735 ms)
2026-11-07T21:14:44Z [load_day 2023-06-07] 2023-06-07 hour 9: 180189 events in 4 files, snapshot 586543122748858423 (task 14808 ms: download 8500 ms overlapping gunzip 886 ms, parse 3885 ms, sort 114 ms, write 8477 ms, index 599 ms: 19 candidates, 82373 repo-hours, 1108 releases; waiting 514 ms; commit 1731 ms)
2026-11-07T21:14:54Z [load_day 2023-06-07] 2023-06-07 hour 10: 169078 events in 4 files, snapshot 3041239872854511939 (task 13911 ms: download 8271 ms overlapping gunzip 831 ms, parse 3657 ms, sort 107 ms, write 7533 ms, index 498 ms: 9 candidates, 77815 repo-hours, 1241 releases; waiting 834 ms; commit 1994 ms)
2026-11-07T21:14:58Z [load_day 2023-06-07] 2023-06-07 hour 11: 164633 events in 4 files, snapshot 1709628556289262489 (task 14056 ms: download 8498 ms overlapping gunzip 796 ms, parse 3485 ms, sort 95 ms, write 7439 ms, index 499 ms: 11 candidates, 75571 repo-hours, 947 releases; waiting 1303 ms; commit 1801 ms)
2026-11-07T21:15:10Z [load_day 2023-06-07] 2023-06-07 hour 12: 195852 events in 4 files, snapshot 4262722116151385096 (task 14931 ms: download 7732 ms overlapping gunzip 822 ms, parse 3772 ms, sort 110 ms, write 8744 ms, index 683 ms: 11 candidates, 103309 repo-hours, 1009 releases; waiting 486 ms; commit 2134 ms)
2026-11-07T21:15:16Z [load_day 2023-06-07] 2023-06-07 hour 13: 175872 events in 4 files, snapshot 588525393720433770 (task 16349 ms: download 9830 ms overlapping gunzip 854 ms, parse 3756 ms, sort 117 ms, write 8645 ms, index 606 ms: 15 candidates, 94017 repo-hours, 957 releases; waiting 2019 ms; commit 1804 ms)
2026-11-07T21:15:25Z [load_day 2023-06-07] 2023-06-07 hour 14: 171297 events in 4 files, snapshot 1901678385972140756 (task 15142 ms: download 9205 ms overlapping gunzip 835 ms, parse 3713 ms, sort 106 ms, write 8449 ms, index 560 ms: 13 candidates, 84300 repo-hours, 962 releases; waiting 1192 ms; commit 1572 ms)
2026-11-07T21:15:31Z [load_day 2023-06-07] 2023-06-07 hour 15: 168156 events in 4 files, snapshot 1707302955075956927 (task 14565 ms: download 8197 ms overlapping gunzip 831 ms, parse 3648 ms, sort 104 ms, write 8149 ms, index 603 ms: 11 candidates, 83060 repo-hours, 1064 releases; waiting 512 ms; commit 1752 ms)
2026-11-07T21:15:40Z [load_day 2023-06-07] 2023-06-07 hour 16: 133290 events in 3 files, snapshot 4705983301113494651 (task 13903 ms: download 6011 ms overlapping gunzip 820 ms, parse 3664 ms, sort 88 ms, write 8015 ms, index 461 ms: 12 candidates, 70399 repo-hours, 900 releases; waiting 538 ms; commit 1876 ms)
2026-11-07T21:15:45Z [load_day 2023-06-07] 2023-06-07 hour 17: 139406 events in 3 files, snapshot 2790120172679528869 (task 12939 ms: download 6385 ms overlapping gunzip 757 ms, parse 3380 ms, sort 83 ms, write 7397 ms, index 496 ms: 7 candidates, 73368 repo-hours, 733 releases; waiting 526 ms; commit 1693 ms)
2026-11-07T21:15:56Z [load_day 2023-06-07] 2023-06-07 hour 18: 192846 events in 4 files, snapshot 2775215519237569659 (task 14557 ms: download 7710 ms overlapping gunzip 865 ms, parse 3792 ms, sort 124 ms, write 8329 ms, index 616 ms: 14 candidates, 91675 repo-hours, 965 releases; waiting 511 ms; commit 1781 ms)
2026-11-07T21:15:59Z [load_day 2023-06-07] 2023-06-07 hour 19: 172467 events in 4 files, snapshot 4251414406089538891 (task 13231 ms: download 7246 ms overlapping gunzip 782 ms, parse 3488 ms, sort 101 ms, write 7629 ms, index 603 ms: 13 candidates, 78712 repo-hours, 857 releases; waiting 341 ms; commit 1813 ms)
2026-11-07T21:16:10Z [load_day 2023-06-07] 2023-06-07 hour 20: 165495 events in 4 files, snapshot 504797392836682987 (task 13704 ms: download 8138 ms overlapping gunzip 805 ms, parse 3524 ms, sort 105 ms, write 7980 ms, index 472 ms: 15 candidates, 72609 repo-hours, 859 releases; waiting 514 ms; commit 1892 ms)
2026-11-07T21:16:13Z [load_day 2023-06-07] 2023-06-07 hour 21: 158238 events in 4 files, snapshot 2358636364018751028 (task 13212 ms: download 5816 ms overlapping gunzip 794 ms, parse 3463 ms, sort 102 ms, write 7548 ms, index 424 ms: 10 candidates, 64944 repo-hours, 898 releases; waiting 565 ms; commit 1763 ms)
2026-11-07T21:16:23Z [load_day 2023-06-07] 2023-06-07 hour 22: 160601 events in 4 files, snapshot 943285023169609408 (task 12096 ms: download 5153 ms overlapping gunzip 732 ms, parse 3171 ms, sort 101 ms, write 6765 ms, index 449 ms: 8 candidates, 70497 repo-hours, 810 releases; waiting 528 ms; commit 1628 ms)
2026-11-07T21:16:25Z [load_day 2023-06-07] 2023-06-07 hour 23: 134237 events in 3 files, snapshot 6421286333712932770 (task 10459 ms: download 4607 ms overlapping gunzip 626 ms, parse 2742 ms, sort 94 ms, write 5796 ms, index 386 ms: 7 candidates, 55289 repo-hours, 727 releases; waiting 496 ms; commit 1527 ms)
2026-11-07T21:16:32Z [load_day 2023-06-07] 2023-06-07: repo_hour merged from 24 files to 1 (1830216 rows, 7365 ms)
2026-11-07T21:16:34Z [load_day 2023-06-07] 2023-06-07: index committed in 9345 ms (24 + 1 + 24 files)
2026-11-07T21:16:34Z [load_day 2023-06-07] 2023-06-07: 3836656 events in 24 hours
2026-11-07T21:16:34Z [load_day 2023-06-07] attempt 1 succeeded