orcaLog in

Runs · run 1601

load_day 2023-10-08, attempt 1 succeeded

Host
3c2b29713ce8:415692
Started
2026-11-08T01:35:16Z
Ended
2026-11-08T01:38:00Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1601/load_day/2023-10-08/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:35:16Z [load_day 2023-10-08] attempt 1 started
2026-11-08T01:35:18Z [load_day 2023-10-08] 2023-10-08: 2 hours in flight, about 1200 MB
2026-11-08T01:35:34Z [load_day 2023-10-08] 2023-10-08 hour 0: 133266 events in 3 files, snapshot 8056980673354069177 (task 11945 ms: download 2994 ms overlapping gunzip 793 ms, parse 3621 ms, sort 164 ms, write 6052 ms, index 533 ms: 10 candidates, 76449 repo-hours, 928 releases; waiting 491 ms; commit 2068 ms)
2026-11-08T01:35:36Z [load_day 2023-10-08] 2023-10-08 hour 1: 134774 events in 3 files, snapshot 1301109487018088947 (task 11312 ms: download 2604 ms overlapping gunzip 680 ms, parse 3198 ms, sort 110 ms, write 5686 ms, index 854 ms: 6 candidates, 81512 repo-hours, 727 releases; waiting 479 ms; commit 2452 ms)
2026-11-08T01:35:43Z [load_day 2023-10-08] 2023-10-08 hour 2: 111013 events in 3 files, snapshot 562031192033924390 (task 8243 ms: download 1973 ms overlapping gunzip 511 ms, parse 2130 ms, sort 68 ms, write 4330 ms, index 349 ms: 9 candidates, 49073 repo-hours, 586 releases; waiting 525 ms; commit 2384 ms)
2026-11-08T01:35:45Z [load_day 2023-10-08] 2023-10-08 hour 3: 109103 events in 3 files, snapshot 8985066222271190145 (task 7840 ms: download 2055 ms overlapping gunzip 441 ms, parse 1896 ms, sort 70 ms, write 4247 ms, index 306 ms: 7 candidates, 48387 repo-hours, 682 releases; waiting 530 ms; commit 2112 ms)
2026-11-08T01:35:51Z [load_day 2023-10-08] 2023-10-08 hour 4: 100531 events in 3 files, snapshot 2803249565430792982 (task 7038 ms: download 2120 ms overlapping gunzip 378 ms, parse 1722 ms, sort 69 ms, write 3716 ms, index 299 ms: 4 candidates, 42537 repo-hours, 515 releases; waiting 519 ms; commit 2396 ms)
2026-11-08T01:35:54Z [load_day 2023-10-08] 2023-10-08 hour 5: 105841 events in 3 files, snapshot 194196170326149920 (task 7636 ms: download 1854 ms overlapping gunzip 461 ms, parse 1945 ms, sort 79 ms, write 3876 ms, index 481 ms: 5 candidates, 46259 repo-hours, 530 releases; waiting 480 ms; commit 2176 ms)
2026-11-08T01:36:04Z [load_day 2023-10-08] 2023-10-08 hour 6: 125939 events in 3 files, snapshot 6705513501776920320 (task 10775 ms: download 1970 ms overlapping gunzip 547 ms, parse 2459 ms, sort 156 ms, write 6166 ms, index 602 ms: 4 candidates, 59422 repo-hours, 751 releases; waiting 561 ms; commit 3833 ms)
2026-11-08T01:36:09Z [load_day 2023-10-08] 2023-10-08 hour 7: 127072 events in 3 files, snapshot 6803174208896533868 (task 12928 ms: download 2461 ms overlapping gunzip 756 ms, parse 3135 ms, sort 118 ms, write 7677 ms, index 471 ms: 5 candidates, 56128 repo-hours, 723 releases; waiting 502 ms; commit 2872 ms)
2026-11-08T01:36:17Z [load_day 2023-10-08] 2023-10-08 hour 8: 136076 events in 3 files, snapshot 7418073093416121989 (task 12452 ms: download 4649 ms overlapping gunzip 598 ms, parse 2584 ms, sort 82 ms, write 7541 ms, index 868 ms: 8 candidates, 59584 repo-hours, 696 releases; waiting 507 ms; commit 2920 ms)
2026-11-08T01:36:21Z [load_day 2023-10-08] 2023-10-08 hour 9: 138696 events in 3 files, snapshot 3760905893022583394 (task 11831 ms: download 4200 ms overlapping gunzip 757 ms, parse 3421 ms, sort 135 ms, write 6634 ms, index 432 ms: 7 candidates, 59815 repo-hours, 824 releases; waiting 236 ms; commit 2124 ms)
2026-11-08T01:36:32Z [load_day 2023-10-08] 2023-10-08 hour 10: 135265 events in 3 files, snapshot 6345889010124443344 (task 11512 ms: download 2281 ms overlapping gunzip 597 ms, parse 2640 ms, sort 106 ms, write 6436 ms, index 921 ms: 6 candidates, 58602 repo-hours, 672 releases; waiting 532 ms; commit 4765 ms)
2026-11-08T01:36:35Z [load_day 2023-10-08] 2023-10-08 hour 11: 134352 events in 3 files, snapshot 6460129108332394517 (task 12005 ms: download 2527 ms overlapping gunzip 641 ms, parse 2985 ms, sort 176 ms, write 6693 ms, index 597 ms: 4 candidates, 56744 repo-hours, 784 releases; waiting 525 ms; commit 2922 ms)
2026-11-08T01:36:45Z [load_day 2023-10-08] 2023-10-08 hour 12: 203028 events in 5 files, snapshot 1544371886090925781 (task 14284 ms: download 5294 ms overlapping gunzip 774 ms, parse 3775 ms, sort 144 ms, write 8030 ms, index 797 ms: 6 candidates, 88564 repo-hours, 788 releases; waiting 494 ms; commit 2536 ms)
2026-11-08T01:36:51Z [load_day 2023-10-08] 2023-10-08 hour 13: 164174 events in 4 files, snapshot 2358911819069712106 (task 12071 ms: download 4467 ms overlapping gunzip 719 ms, parse 3294 ms, sort 148 ms, write 6476 ms, index 656 ms: 8 candidates, 81511 repo-hours, 718 releases; waiting 473 ms; commit 5685 ms)
2026-11-08T01:36:59Z [load_day 2023-10-08] 2023-10-08 hour 14: 155022 events in 4 files, snapshot 7300424910017455387 (task 13052 ms: download 5465 ms overlapping gunzip 854 ms, parse 3517 ms, sort 169 ms, write 7116 ms, index 594 ms: 5 candidates, 66450 repo-hours, 785 releases; waiting 495 ms; commit 2756 ms)
2026-11-08T01:37:05Z [load_day 2023-10-08] 2023-10-08 hour 15: 158407 events in 4 files, snapshot 3126710416314347474 (task 13093 ms: download 5029 ms overlapping gunzip 696 ms, parse 3196 ms, sort 104 ms, write 7693 ms, index 574 ms: 9 candidates, 68296 repo-hours, 889 releases; waiting 479 ms; commit 5081 ms)
2026-11-08T01:37:14Z [load_day 2023-10-08] 2023-10-08 hour 16: 158602 events in 4 files, snapshot 1385662579161474364 (task 13371 ms: download 4708 ms overlapping gunzip 862 ms, parse 3708 ms, sort 156 ms, write 7283 ms, index 572 ms: 8 candidates, 65445 repo-hours, 799 releases; waiting 535 ms; commit 2820 ms)
2026-11-08T01:37:17Z [load_day 2023-10-08] 2023-10-08 hour 17: 149107 events in 3 files, snapshot 2030434524999786124 (task 11983 ms: download 4581 ms overlapping gunzip 674 ms, parse 3014 ms, sort 99 ms, write 7021 ms, index 557 ms: 5 candidates, 62343 repo-hours, 720 releases; waiting 330 ms; commit 3315 ms)
2026-11-08T01:37:28Z [load_day 2023-10-08] 2023-10-08 hour 18: 154985 events in 4 files, snapshot 6524942088441960467 (task 12314 ms: download 4740 ms overlapping gunzip 632 ms, parse 2928 ms, sort 158 ms, write 7221 ms, index 540 ms: 3 candidates, 69722 repo-hours, 782 releases; waiting 504 ms; commit 3592 ms)
2026-11-08T01:37:30Z [load_day 2023-10-08] 2023-10-08 hour 19: 138745 events in 3 files, snapshot 6265591886447380755 (task 10915 ms: download 2319 ms overlapping gunzip 652 ms, parse 2901 ms, sort 87 ms, write 6094 ms, index 420 ms: 2 candidates, 56868 repo-hours, 779 releases; waiting 484 ms; commit 2348 ms)
2026-11-08T01:37:39Z [load_day 2023-10-08] 2023-10-08 hour 20: 134049 events in 3 files, snapshot 8900004114610707837 (task 10669 ms: download 2119 ms overlapping gunzip 551 ms, parse 2450 ms, sort 95 ms, write 6237 ms, index 417 ms: 8 candidates, 55220 repo-hours, 752 releases; waiting 526 ms; commit 2587 ms)
2026-11-08T01:37:42Z [load_day 2023-10-08] 2023-10-08 hour 21: 121420 events in 3 files, snapshot 5316392694966088909 (task 10279 ms: download 2463 ms overlapping gunzip 514 ms, parse 2388 ms, sort 128 ms, write 5802 ms, index 552 ms: 6 candidates, 48445 repo-hours, 736 releases; waiting 626 ms; commit 2573 ms)
2026-11-08T01:37:48Z [load_day 2023-10-08] 2023-10-08 hour 22: 105869 events in 3 files, snapshot 265317047679751650 (task 8589 ms: download 1759 ms overlapping gunzip 493 ms, parse 2172 ms, sort 69 ms, write 4777 ms, index 340 ms: 7 candidates, 42346 repo-hours, 683 releases; waiting 525 ms; commit 2286 ms)
2026-11-08T01:37:52Z [load_day 2023-10-08] 2023-10-08 hour 23: 100370 events in 3 files, snapshot 3001200018918301126 (task 8224 ms: download 2044 ms overlapping gunzip 459 ms, parse 2093 ms, sort 68 ms, write 4349 ms, index 445 ms: 9 candidates, 41487 repo-hours, 647 releases; waiting 519 ms; commit 2932 ms)
2026-11-08T01:37:58Z [load_day 2023-10-08] 2023-10-08: repo_hour merged from 24 files to 1 (1441209 rows, 6341 ms)
2026-11-08T01:38:00Z [load_day 2023-10-08] 2023-10-08: index committed in 8428 ms (24 + 1 + 24 files)
2026-11-08T01:38:00Z [load_day 2023-10-08] 2023-10-08: 3235706 events in 24 hours
2026-11-08T01:38:00Z [load_day 2023-10-08] attempt 1 succeeded