orcaLog in

Runs · run 1511

load_day 2023-09-16, attempt 1 succeeded

Host
3c2b29713ce8:337125
Started
2026-11-08T00:53:15Z
Ended
2026-11-08T00:55:56Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1511/load_day/2023-09-16/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:53:15Z [load_day 2023-09-16] attempt 1 started
2026-11-08T00:53:17Z [load_day 2023-09-16] 2023-09-16: 2 hours in flight, about 1200 MB
2026-11-08T00:53:33Z [load_day 2023-09-16] 2023-09-16 hour 0: 149032 events in 3 files, snapshot 6295463652106585852 (task 12500 ms: download 4993 ms overlapping gunzip 695 ms, parse 3286 ms, sort 102 ms, write 6959 ms, index 512 ms: 7 candidates, 81762 repo-hours, 1125 releases; waiting 664 ms; commit 2450 ms)
2026-11-08T00:53:38Z [load_day 2023-09-16] 2023-09-16 hour 1: 133577 events in 3 files, snapshot 685991398116559980 (task 11745 ms: download 1616 ms overlapping gunzip 644 ms, parse 3003 ms, sort 83 ms, write 6837 ms, index 702 ms: 4 candidates, 73857 repo-hours, 903 releases; waiting 187 ms; commit 4610 ms)
2026-11-08T00:53:43Z [load_day 2023-09-16] 2023-09-16 hour 2: 105640 events in 3 files, snapshot 3184187520661688135 (task 9058 ms: download 2846 ms overlapping gunzip 587 ms, parse 2573 ms, sort 79 ms, write 4734 ms, index 317 ms: 6 candidates, 40527 repo-hours, 778 releases; waiting 504 ms; commit 2301 ms)
2026-11-08T00:53:46Z [load_day 2023-09-16] 2023-09-16 hour 3: 98341 events in 2 files, snapshot 6491869524160519513 (task 8023 ms: download 2486 ms overlapping gunzip 450 ms, parse 1948 ms, sort 104 ms, write 4388 ms, index 328 ms: 3 candidates, 40308 repo-hours, 874 releases; waiting 501 ms; commit 2940 ms)
2026-11-08T00:54:00Z [load_day 2023-09-16] 2023-09-16 hour 4: 99610 events in 2 files, snapshot 1190114860340483588 (task 8329 ms: download 1877 ms overlapping gunzip 511 ms, parse 2321 ms, sort 126 ms, write 4219 ms, index 341 ms: 5 candidates, 41022 repo-hours, 756 releases; waiting 508 ms; commit 9465 ms)
2026-11-08T00:54:03Z [load_day 2023-09-16] 2023-09-16 hour 5: 108492 events in 3 files, snapshot 459801784456734480 (task 9215 ms: download 2106 ms overlapping gunzip 499 ms, parse 2135 ms, sort 66 ms, write 5230 ms, index 515 ms: 6 candidates, 43517 repo-hours, 835 releases; waiting 480 ms; commit 2641 ms)
2026-11-08T00:54:07Z [load_day 2023-09-16] 2023-09-16 hour 6: 120552 events in 3 files, snapshot 7779768516619929182 (task 10703 ms: download 2296 ms overlapping gunzip 530 ms, parse 2494 ms, sort 89 ms, write 6027 ms, index 706 ms: 13 candidates, 53179 repo-hours, 762 releases; waiting 502 ms; commit 3919 ms)
2026-11-08T00:54:14Z [load_day 2023-09-16] 2023-09-16 hour 7: 119823 events in 3 files, snapshot 2069131254709897277 (task 10082 ms: download 1957 ms overlapping gunzip 540 ms, parse 2548 ms, sort 119 ms, write 5633 ms, index 414 ms: 8 candidates, 49930 repo-hours, 695 releases; waiting 505 ms; commit 2872 ms)
2026-11-08T00:54:19Z [load_day 2023-09-16] 2023-09-16 hour 8: 128334 events in 3 files, snapshot 42830545788276366 (task 10366 ms: download 1924 ms overlapping gunzip 484 ms, parse 2166 ms, sort 105 ms, write 6336 ms, index 524 ms: 12 candidates, 54427 repo-hours, 800 releases; waiting 487 ms; commit 4093 ms)
2026-11-08T00:54:26Z [load_day 2023-09-16] 2023-09-16 hour 9: 124873 events in 3 files, snapshot 219673006831135391 (task 10837 ms: download 2020 ms overlapping gunzip 573 ms, parse 2569 ms, sort 108 ms, write 6381 ms, index 408 ms: 9 candidates, 52440 repo-hours, 1015 releases; waiting 486 ms; commit 2619 ms)
2026-11-08T00:54:31Z [load_day 2023-09-16] 2023-09-16 hour 10: 125606 events in 3 files, snapshot 5150233914132957073 (task 10795 ms: download 2596 ms overlapping gunzip 653 ms, parse 2736 ms, sort 103 ms, write 5935 ms, index 554 ms: 6 candidates, 53414 repo-hours, 858 releases; waiting 500 ms; commit 3745 ms)
2026-11-08T00:54:38Z [load_day 2023-09-16] 2023-09-16 hour 11: 125276 events in 3 files, snapshot 4243783337964322363 (task 10746 ms: download 1940 ms overlapping gunzip 621 ms, parse 2688 ms, sort 148 ms, write 6075 ms, index 416 ms: 7 candidates, 51852 repo-hours, 843 releases; waiting 499 ms; commit 3182 ms)
2026-11-08T00:54:43Z [load_day 2023-09-16] 2023-09-16 hour 12: 204668 events in 5 files, snapshot 7445623463096378876 (task 12699 ms: download 3883 ms overlapping gunzip 624 ms, parse 3062 ms, sort 107 ms, write 7605 ms, index 674 ms: 5 candidates, 86240 repo-hours, 1125 releases; waiting 220 ms; commit 2549 ms)
2026-11-08T00:54:53Z [load_day 2023-09-16] 2023-09-16 hour 13: 161169 events in 4 files, snapshot 747583258684087738 (task 13209 ms: download 4378 ms overlapping gunzip 711 ms, parse 3223 ms, sort 127 ms, write 7738 ms, index 643 ms: 7 candidates, 77787 repo-hours, 1074 releases; waiting 513 ms; commit 3196 ms)
2026-11-08T00:54:55Z [load_day 2023-09-16] 2023-09-16 hour 14: 145831 events in 3 files, snapshot 6363762205999390482 (task 11201 ms: download 3831 ms overlapping gunzip 627 ms, parse 2729 ms, sort 164 ms, write 6251 ms, index 531 ms: 8 candidates, 61303 repo-hours, 984 releases; waiting 541 ms; commit 2580 ms)
2026-11-08T00:55:03Z [load_day 2023-09-16] 2023-09-16 hour 15: 145630 events in 3 files, snapshot 6124974793863854821 (task 10160 ms: download 4066 ms overlapping gunzip 541 ms, parse 2479 ms, sort 86 ms, write 5808 ms, index 447 ms: 10 candidates, 63024 repo-hours, 1096 releases; waiting 503 ms; commit 2407 ms)
2026-11-08T00:55:07Z [load_day 2023-09-16] 2023-09-16 hour 16: 143241 events in 3 files, snapshot 7833649396841616397 (task 10673 ms: download 4583 ms overlapping gunzip 572 ms, parse 2534 ms, sort 92 ms, write 5619 ms, index 521 ms: 11 candidates, 61721 repo-hours, 882 releases; waiting 1052 ms; commit 2209 ms)
2026-11-08T00:55:14Z [load_day 2023-09-16] 2023-09-16 hour 17: 137910 events in 3 files, snapshot 7808070430205009031 (task 9562 ms: download 3763 ms overlapping gunzip 549 ms, parse 2368 ms, sort 89 ms, write 5393 ms, index 378 ms: 5 candidates, 57421 repo-hours, 788 releases; waiting 368 ms; commit 2056 ms)
2026-11-08T00:55:18Z [load_day 2023-09-16] 2023-09-16 hour 18: 139267 events in 3 files, snapshot 7159920171621474882 (task 9088 ms: download 3195 ms overlapping gunzip 468 ms, parse 2200 ms, sort 99 ms, write 4943 ms, index 552 ms: 6 candidates, 61991 repo-hours, 1019 releases; waiting 508 ms; commit 3332 ms)
2026-11-08T00:55:25Z [load_day 2023-09-16] 2023-09-16 hour 19: 122933 events in 3 files, snapshot 603605017583198231 (task 10629 ms: download 2596 ms overlapping gunzip 544 ms, parse 2362 ms, sort 75 ms, write 6431 ms, index 387 ms: 5 candidates, 50800 repo-hours, 853 releases; waiting 512 ms; commit 2436 ms)
2026-11-08T00:55:34Z [load_day 2023-09-16] 2023-09-16 hour 20: 122141 events in 3 files, snapshot 2667773122311193740 (task 8942 ms: download 1655 ms overlapping gunzip 621 ms, parse 2552 ms, sort 85 ms, write 4875 ms, index 332 ms: 5 candidates, 50792 repo-hours, 758 releases; waiting 187 ms; commit 8180 ms)
2026-11-08T00:55:41Z [load_day 2023-09-16] 2023-09-16 hour 21: 110509 events in 3 files, snapshot 2249565093055213541 (task 9504 ms: download 2327 ms overlapping gunzip 466 ms, parse 2136 ms, sort 123 ms, write 5538 ms, index 388 ms: 6 candidates, 43834 repo-hours, 892 releases; waiting 507 ms; commit 6828 ms)
2026-11-08T00:55:44Z [load_day 2023-09-16] 2023-09-16 hour 22: 98335 events in 2 files, snapshot 8598946365151924182 (task 7945 ms: download 1710 ms overlapping gunzip 443 ms, parse 1858 ms, sort 77 ms, write 4478 ms, index 367 ms: 4 candidates, 39177 repo-hours, 741 releases; waiting 513 ms; commit 3884 ms)
2026-11-08T00:55:48Z [load_day 2023-09-16] 2023-09-16 hour 23: 94427 events in 2 files, snapshot 1650537832695425950 (task 6346 ms: download 1587 ms overlapping gunzip 376 ms, parse 1692 ms, sort 53 ms, write 3352 ms, index 286 ms: 4 candidates, 38540 repo-hours, 619 releases; waiting 288 ms; commit 3307 ms)
2026-11-08T00:55:54Z [load_day 2023-09-16] 2023-09-16: repo_hour merged from 24 files to 1 (1328865 rows, 6390 ms)
2026-11-08T00:55:56Z [load_day 2023-09-16] 2023-09-16: index committed in 8347 ms (24 + 1 + 24 files)
2026-11-08T00:55:56Z [load_day 2023-09-16] 2023-09-16: 3065217 events in 24 hours
2026-11-08T00:55:56Z [load_day 2023-09-16] attempt 1 succeeded