orcaLog in

Runs · run 1673

load_day 2023-10-28, attempt 1 succeeded

Host
3c2b29713ce8:490023
Started
2026-11-08T02:14:17Z
Ended
2026-11-08T02:17:03Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1673/load_day/2023-10-28/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:14:17Z [load_day 2023-10-28] attempt 1 started
2026-11-08T02:14:19Z [load_day 2023-10-28] 2023-10-28: 2 hours in flight, about 1200 MB
2026-11-08T02:14:37Z [load_day 2023-10-28] 2023-10-28 hour 0: 143111 events in 3 files, snapshot 2165593414523157896 (task 13948 ms: download 5552 ms overlapping gunzip 908 ms, parse 4217 ms, sort 166 ms, write 7665 ms, index 760 ms: 8 candidates, 78633 repo-hours, 1029 releases; waiting 34 ms; commit 2503 ms)
2026-11-08T02:14:40Z [load_day 2023-10-28] 2023-10-28 hour 1: 136429 events in 3 files, snapshot 7637793656895979553 (task 12511 ms: download 4534 ms overlapping gunzip 777 ms, parse 3338 ms, sort 94 ms, write 7106 ms, index 964 ms: 5 candidates, 76534 repo-hours, 670 releases; waiting 32 ms; commit 2728 ms)
2026-11-08T02:14:48Z [load_day 2023-10-28] 2023-10-28 hour 2: 108035 events in 3 files, snapshot 6037410180154037033 (task 9503 ms: download 1027 ms overlapping gunzip 541 ms, parse 2111 ms, sort 84 ms, write 6226 ms, index 326 ms: 5 candidates, 45660 repo-hours, 612 releases; waiting 14 ms; commit 2532 ms)
2026-11-08T02:14:52Z [load_day 2023-10-28] 2023-10-28 hour 3: 111159 events in 3 files, snapshot 8783740656309597334 (task 8929 ms: download 1666 ms overlapping gunzip 652 ms, parse 2635 ms, sort 70 ms, write 4885 ms, index 446 ms: 7 candidates, 45262 repo-hours, 839 releases; waiting 21 ms; commit 4758 ms)
2026-11-08T02:14:59Z [load_day 2023-10-28] 2023-10-28 hour 4: 112037 events in 3 files, snapshot 4518868264732347031 (task 10624 ms: download 1300 ms overlapping gunzip 600 ms, parse 2406 ms, sort 79 ms, write 6900 ms, index 413 ms: 13 candidates, 47427 repo-hours, 515 releases; waiting 33 ms; commit 2304 ms)
2026-11-08T02:15:02Z [load_day 2023-10-28] 2023-10-28 hour 5: 108252 events in 3 files, snapshot 2067777870689897206 (task 9317 ms: download 1310 ms overlapping gunzip 713 ms, parse 2957 ms, sort 73 ms, write 4881 ms, index 466 ms: 8 candidates, 46468 repo-hours, 523 releases; waiting 16 ms; commit 2909 ms)
2026-11-08T02:15:13Z [load_day 2023-10-28] 2023-10-28 hour 6: 123381 events in 3 files, snapshot 6063931701742668034 (task 9980 ms: download 1199 ms overlapping gunzip 567 ms, parse 2441 ms, sort 110 ms, write 6266 ms, index 369 ms: 10 candidates, 55116 repo-hours, 716 releases; waiting 28 ms; commit 5145 ms)
2026-11-08T02:15:16Z [load_day 2023-10-28] 2023-10-28 hour 7: 118785 events in 3 files, snapshot 6868169170829698598 (task 9394 ms: download 1362 ms overlapping gunzip 597 ms, parse 2553 ms, sort 84 ms, write 5375 ms, index 516 ms: 10 candidates, 50739 repo-hours, 499 releases; waiting 29 ms; commit 2927 ms)
2026-11-08T02:15:23Z [load_day 2023-10-28] 2023-10-28 hour 8: 124117 events in 3 files, snapshot 1731426725175772994 (task 10746 ms: download 1581 ms overlapping gunzip 604 ms, parse 2739 ms, sort 84 ms, write 6480 ms, index 587 ms: 10 candidates, 53919 repo-hours, 622 releases; waiting 45 ms; commit 2859 ms)
2026-11-08T02:15:28Z [load_day 2023-10-28] 2023-10-28 hour 9: 130762 events in 3 files, snapshot 5013005409448754892 (task 11409 ms: download 1700 ms overlapping gunzip 664 ms, parse 2904 ms, sort 135 ms, write 7024 ms, index 438 ms: 10 candidates, 55070 repo-hours, 755 releases; waiting 25 ms; commit 3089 ms)
2026-11-08T02:15:35Z [load_day 2023-10-28] 2023-10-28 hour 10: 128897 events in 3 files, snapshot 1880751332274175467 (task 10698 ms: download 2483 ms overlapping gunzip 585 ms, parse 2637 ms, sort 84 ms, write 6029 ms, index 384 ms: 6 candidates, 55810 repo-hours, 701 releases; waiting 732 ms; commit 2823 ms)
2026-11-08T02:15:40Z [load_day 2023-10-28] 2023-10-28 hour 11: 126086 events in 3 files, snapshot 8174098010556739482 (task 10878 ms: download 1174 ms overlapping gunzip 737 ms, parse 3360 ms, sort 127 ms, write 5913 ms, index 535 ms: 7 candidates, 53032 repo-hours, 718 releases; waiting 11 ms; commit 2758 ms)
2026-11-08T02:15:50Z [load_day 2023-10-28] 2023-10-28 hour 12: 204887 events in 5 files, snapshot 7295513191027461264 (task 13826 ms: download 4818 ms overlapping gunzip 738 ms, parse 3868 ms, sort 172 ms, write 8263 ms, index 570 ms: 8 candidates, 86698 repo-hours, 776 releases; waiting 21 ms; commit 2924 ms)
2026-11-08T02:15:54Z [load_day 2023-10-28] 2023-10-28 hour 13: 162989 events in 4 files, snapshot 8843448826494066237 (task 12588 ms: download 3964 ms overlapping gunzip 753 ms, parse 3747 ms, sort 132 ms, write 7132 ms, index 611 ms: 8 candidates, 79799 repo-hours, 751 releases; waiting 19 ms; commit 2826 ms)
2026-11-08T02:16:03Z [load_day 2023-10-28] 2023-10-28 hour 14: 151040 events in 4 files, snapshot 7035729053861465060 (task 11882 ms: download 4139 ms overlapping gunzip 750 ms, parse 3274 ms, sort 92 ms, write 7052 ms, index 494 ms: 9 candidates, 64598 repo-hours, 821 releases; waiting 18 ms; commit 2648 ms)
2026-11-08T02:16:08Z [load_day 2023-10-28] 2023-10-28 hour 15: 154622 events in 4 files, snapshot 6452795899624711185 (task 11911 ms: download 4210 ms overlapping gunzip 697 ms, parse 3075 ms, sort 112 ms, write 7123 ms, index 695 ms: 12 candidates, 67045 repo-hours, 1020 releases; waiting 20 ms; commit 3990 ms)
2026-11-08T02:16:16Z [load_day 2023-10-28] 2023-10-28 hour 16: 149193 events in 3 files, snapshot 931322739848508399 (task 12044 ms: download 3904 ms overlapping gunzip 640 ms, parse 2887 ms, sort 190 ms, write 7685 ms, index 421 ms: 7 candidates, 64551 repo-hours, 887 releases; waiting 19 ms; commit 2526 ms)
2026-11-08T02:16:23Z [load_day 2023-10-28] 2023-10-28 hour 17: 141658 events in 3 files, snapshot 7693410471581650137 (task 11964 ms: download 4563 ms overlapping gunzip 686 ms, parse 3007 ms, sort 97 ms, write 6669 ms, index 760 ms: 4 candidates, 59617 repo-hours, 763 releases; waiting 515 ms; commit 5654 ms)
2026-11-08T02:16:32Z [load_day 2023-10-28] 2023-10-28 hour 18: 145927 events in 3 files, snapshot 804222921226787503 (task 11071 ms: download 3857 ms overlapping gunzip 625 ms, parse 2860 ms, sort 95 ms, write 5810 ms, index 499 ms: 6 candidates, 65674 repo-hours, 855 releases; waiting 487 ms; commit 5788 ms)
2026-11-08T02:16:39Z [load_day 2023-10-28] 2023-10-28 hour 19: 128133 events in 3 files, snapshot 7724863508220698463 (task 10722 ms: download 1254 ms overlapping gunzip 649 ms, parse 2798 ms, sort 74 ms, write 6612 ms, index 386 ms: 9 candidates, 53098 repo-hours, 748 releases; waiting 22 ms; commit 7855 ms)
2026-11-08T02:16:43Z [load_day 2023-10-28] 2023-10-28 hour 20: 122907 events in 3 files, snapshot 577068791360057727 (task 10940 ms: download 2039 ms overlapping gunzip 581 ms, parse 2630 ms, sort 159 ms, write 6962 ms, index 380 ms: 7 candidates, 51468 repo-hours, 667 releases; waiting 30 ms; commit 3190 ms)
2026-11-08T02:16:48Z [load_day 2023-10-28] 2023-10-28 hour 21: 113072 events in 3 files, snapshot 8306403756220049427 (task 10237 ms: download 780 ms overlapping gunzip 473 ms, parse 2077 ms, sort 69 ms, write 6827 ms, index 529 ms: 2 candidates, 45867 repo-hours, 761 releases; waiting 19 ms; commit 5494 ms)
2026-11-08T02:16:52Z [load_day 2023-10-28] 2023-10-28 hour 22: 103709 events in 3 files, snapshot 5471799083415293810 (task 8996 ms: download 1295 ms overlapping gunzip 535 ms, parse 2389 ms, sort 109 ms, write 5352 ms, index 406 ms: 6 candidates, 40755 repo-hours, 781 releases; waiting 19 ms; commit 2434 ms)
2026-11-08T02:16:54Z [load_day 2023-10-28] 2023-10-28 hour 23: 93517 events in 2 files, snapshot 4338064189646756908 (task 8153 ms: download 1782 ms overlapping gunzip 374 ms, parse 1854 ms, sort 104 ms, write 4337 ms, index 300 ms: 3 candidates, 39324 repo-hours, 635 releases; waiting 488 ms; commit 2470 ms)
2026-11-08T02:17:01Z [load_day 2023-10-28] 2023-10-28: repo_hour merged from 24 files to 1 (1382164 rows, 6462 ms)
2026-11-08T02:17:03Z [load_day 2023-10-28] 2023-10-28: index committed in 8983 ms (24 + 1 + 24 files)
2026-11-08T02:17:03Z [load_day 2023-10-28] 2023-10-28: 3142705 events in 24 hours
2026-11-08T02:17:03Z [load_day 2023-10-28] attempt 1 succeeded