orcaLog in

Runs · run 970

load_day 2023-04-05, attempt 1 succeeded

Host
caf251265124:236586
Started
2026-11-07T18:11:41Z
Ended
2026-11-07T18:14:51Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/970/load_day/2023-04-05/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T18:11:41Z [load_day 2023-04-05] attempt 1 started
2026-11-07T18:11:42Z [load_day 2023-04-05] 2023-04-05: 2 hours in flight, about 1200 MB
2026-11-07T18:11:57Z [load_day 2023-04-05] 2023-04-05 hour 0: 159358 events in 4 files, snapshot 3867527825956958175 (task 11757 ms: download 4966 ms overlapping gunzip 648 ms, parse 2987 ms, sort 104 ms, write 6491 ms, index 543 ms: 11 candidates, 84311 repo-hours, 960 releases; waiting 652 ms; commit 1502 ms)
2026-11-07T18:11:58Z [load_day 2023-04-05] 2023-04-05 hour 1: 163146 events in 4 files, snapshot 7947298708595499824 (task 11940 ms: download 4865 ms overlapping gunzip 669 ms, parse 2987 ms, sort 114 ms, write 6798 ms, index 538 ms: 4 candidates, 85649 repo-hours, 748 releases; waiting 514 ms; commit 1422 ms)
2026-11-07T18:12:08Z [load_day 2023-04-05] 2023-04-05 hour 2: 133424 events in 3 files, snapshot 4979158309327530702 (task 10059 ms: download 4346 ms overlapping gunzip 590 ms, parse 2559 ms, sort 88 ms, write 5533 ms, index 433 ms: 2 candidates, 57952 repo-hours, 724 releases; waiting 600 ms; commit 1757 ms)
2026-11-07T18:12:11Z [load_day 2023-04-05] 2023-04-05 hour 3: 126505 events in 3 files, snapshot 7933149906981895925 (task 11533 ms: download 5921 ms overlapping gunzip 605 ms, parse 2591 ms, sort 88 ms, write 5465 ms, index 398 ms: 5 candidates, 55104 repo-hours, 828 releases; waiting 2097 ms; commit 1467 ms)
2026-11-07T18:12:18Z [load_day 2023-04-05] 2023-04-05 hour 4: 127580 events in 3 files, snapshot 3171519627463580973 (task 9597 ms: download 4077 ms overlapping gunzip 565 ms, parse 2451 ms, sort 80 ms, write 5320 ms, index 363 ms: 5 candidates, 56568 repo-hours, 624 releases; waiting 519 ms; commit 1600 ms)
2026-11-07T18:12:22Z [load_day 2023-04-05] 2023-04-05 hour 5: 137340 events in 3 files, snapshot 4001487711526105608 (task 10075 ms: download 4274 ms overlapping gunzip 605 ms, parse 2639 ms, sort 84 ms, write 5503 ms, index 373 ms: 5 candidates, 59668 repo-hours, 779 releases; waiting 541 ms; commit 1655 ms)
2026-11-07T18:12:32Z [load_day 2023-04-05] 2023-04-05 hour 6: 161403 events in 4 files, snapshot 8934771017369936762 (task 12441 ms: download 5272 ms overlapping gunzip 731 ms, parse 3169 ms, sort 100 ms, write 6833 ms, index 520 ms: 6 candidates, 71963 repo-hours, 857 releases; waiting 822 ms; commit 1615 ms)
2026-11-07T18:12:36Z [load_day 2023-04-05] 2023-04-05 hour 7: 170411 events in 4 files, snapshot 8900258903172617547 (task 13664 ms: download 7903 ms overlapping gunzip 830 ms, parse 3638 ms, sort 108 ms, write 7749 ms, index 493 ms: 5 candidates, 75203 repo-hours, 943 releases; waiting 522 ms; commit 1684 ms)
2026-11-07T18:12:47Z [load_day 2023-04-05] 2023-04-05 hour 8: 176851 events in 4 files, snapshot 1876370345998546396 (task 14550 ms: download 8268 ms overlapping gunzip 852 ms, parse 3782 ms, sort 108 ms, write 8447 ms, index 568 ms: 7 candidates, 80394 repo-hours, 861 releases; waiting 524 ms; commit 1556 ms)
2026-11-07T18:12:52Z [load_day 2023-04-05] 2023-04-05 hour 9: 179215 events in 4 files, snapshot 3327332141077779042 (task 14689 ms: download 8420 ms overlapping gunzip 870 ms, parse 3875 ms, sort 112 ms, write 8387 ms, index 540 ms: 7 candidates, 80206 repo-hours, 1095 releases; waiting 556 ms; commit 1757 ms)
2026-11-07T18:13:02Z [load_day 2023-04-05] 2023-04-05 hour 10: 178095 events in 4 files, snapshot 4133923703690334945 (task 14558 ms: download 8122 ms overlapping gunzip 837 ms, parse 3768 ms, sort 102 ms, write 8377 ms, index 618 ms: 7 candidates, 80845 repo-hours, 887 releases; waiting 524 ms; commit 1671 ms)
2026-11-07T18:13:07Z [load_day 2023-04-05] 2023-04-05 hour 11: 182703 events in 4 files, snapshot 7539196720739292366 (task 14895 ms: download 8532 ms overlapping gunzip 882 ms, parse 3827 ms, sort 118 ms, write 8821 ms, index 528 ms: 10 candidates, 79763 repo-hours, 1058 releases; waiting 517 ms; commit 1454 ms)
2026-11-07T18:13:19Z [load_day 2023-04-05] 2023-04-05 hour 12: 203014 events in 5 files, snapshot 4754753748440123492 (task 15753 ms: download 8265 ms overlapping gunzip 920 ms, parse 4108 ms, sort 130 ms, write 9072 ms, index 694 ms: 8 candidates, 105493 repo-hours, 971 releases; waiting 534 ms; commit 1651 ms)
2026-11-07T18:13:27Z [load_day 2023-04-05] 2023-04-05 hour 13: 198312 events in 4 files, snapshot 7209547945763292642 (task 18398 ms: download 10985 ms overlapping gunzip 972 ms, parse 4242 ms, sort 125 ms, write 9278 ms, index 646 ms: 5 candidates, 104425 repo-hours, 985 releases; waiting 2780 ms; commit 1582 ms)
2026-11-07T18:13:37Z [load_day 2023-04-05] 2023-04-05 hour 14: 197054 events in 4 files, snapshot 2664429016432544678 (task 17258 ms: download 9712 ms overlapping gunzip 1063 ms, parse 4633 ms, sort 126 ms, write 9991 ms, index 638 ms: 9 candidates, 94536 repo-hours, 1035 releases; waiting 519 ms; commit 1483 ms)
2026-11-07T18:13:44Z [load_day 2023-04-05] 2023-04-05 hour 15: 196650 events in 4 files, snapshot 1520386127912380377 (task 16593 ms: download 9045 ms overlapping gunzip 1007 ms, parse 4356 ms, sort 123 ms, write 9569 ms, index 646 ms: 3 candidates, 94039 repo-hours, 1056 releases; waiting 541 ms; commit 1600 ms)
2026-11-07T18:13:53Z [load_day 2023-04-05] 2023-04-05 hour 16: 184382 events in 4 files, snapshot 2052667524519757962 (task 15646 ms: download 8955 ms overlapping gunzip 907 ms, parse 4020 ms, sort 118 ms, write 9068 ms, index 610 ms: 7 candidates, 87922 repo-hours, 964 releases; waiting 635 ms; commit 1536 ms)
2026-11-07T18:14:00Z [load_day 2023-04-05] 2023-04-05 hour 17: 185459 events in 4 files, snapshot 6830741260621703771 (task 15135 ms: download 8458 ms overlapping gunzip 884 ms, parse 3888 ms, sort 108 ms, write 8866 ms, index 539 ms: 7 candidates, 83592 repo-hours, 931 releases; waiting 584 ms; commit 1866 ms)
2026-11-07T18:14:09Z [load_day 2023-04-05] 2023-04-05 hour 18: 185612 events in 4 files, snapshot 8345517916569963775 (task 15258 ms: download 8506 ms overlapping gunzip 877 ms, parse 3954 ms, sort 106 ms, write 8906 ms, index 571 ms: 5 candidates, 87478 repo-hours, 1013 releases; waiting 519 ms; commit 1355 ms)
2026-11-07T18:14:16Z [load_day 2023-04-05] 2023-04-05 hour 19: 173298 events in 4 files, snapshot 8646784078826973353 (task 15659 ms: download 9570 ms overlapping gunzip 882 ms, parse 3843 ms, sort 103 ms, write 8145 ms, index 497 ms: 6 candidates, 76211 repo-hours, 933 releases; waiting 1759 ms; commit 1489 ms)
2026-11-07T18:14:24Z [load_day 2023-04-05] 2023-04-05 hour 20: 170020 events in 4 files, snapshot 3686180171698870202 (task 13861 ms: download 8110 ms overlapping gunzip 816 ms, parse 3626 ms, sort 95 ms, write 8021 ms, index 528 ms: 5 candidates, 75763 repo-hours, 914 releases; waiting 507 ms; commit 1580 ms)
2026-11-07T18:14:30Z [load_day 2023-04-05] 2023-04-05 hour 21: 156256 events in 4 files, snapshot 6449022048245233766 (task 13004 ms: download 5565 ms overlapping gunzip 756 ms, parse 3368 ms, sort 99 ms, write 7405 ms, index 464 ms: 8 candidates, 66832 repo-hours, 904 releases; waiting 596 ms; commit 1596 ms)
2026-11-07T18:14:37Z [load_day 2023-04-05] 2023-04-05 hour 22: 146680 events in 3 files, snapshot 7292430990459470152 (task 11714 ms: download 5131 ms overlapping gunzip 713 ms, parse 3063 ms, sort 90 ms, write 6671 ms, index 395 ms: 4 candidates, 60718 repo-hours, 786 releases; waiting 530 ms; commit 1675 ms)
2026-11-07T18:14:41Z [load_day 2023-04-05] 2023-04-05 hour 23: 137495 events in 3 files, snapshot 1380438726844123793 (task 10895 ms: download 4843 ms overlapping gunzip 649 ms, parse 2811 ms, sort 76 ms, write 5912 ms, index 362 ms: 3 candidates, 56561 repo-hours, 650 releases; waiting 773 ms; commit 1455 ms)
2026-11-07T18:14:49Z [load_day 2023-04-05] 2023-04-05: repo_hour merged from 24 files to 1 (1861196 rows, 7680 ms)
2026-11-07T18:14:51Z [load_day 2023-04-05] 2023-04-05: index committed in 10050 ms (24 + 1 + 24 files)
2026-11-07T18:14:51Z [load_day 2023-04-05] 2023-04-05: 4030263 events in 24 hours
2026-11-07T18:14:51Z [load_day 2023-04-05] attempt 1 succeeded