orcaLog in

Runs · run 1778

load_day 2023-11-25, attempt 1 succeeded

Host
3c2b29713ce8:597185
Started
2026-11-08T03:10:17Z
Ended
2026-11-08T03:12:50Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1778/load_day/2023-11-25/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:10:17Z [load_day 2023-11-25] attempt 1 started
2026-11-08T03:10:19Z [load_day 2023-11-25] 2023-11-25: 2 hours in flight, about 1200 MB
2026-11-08T03:10:34Z [load_day 2023-11-25] 2023-11-25 hour 0: 151740 events in 4 files, snapshot 9071973474753938616 (task 11148 ms: download 1616 ms overlapping gunzip 671 ms, parse 3236 ms, sort 143 ms, write 6301 ms, index 533 ms: 7 candidates, 76298 repo-hours, 908 releases; waiting 75 ms; commit 2737 ms)
2026-11-08T03:10:40Z [load_day 2023-11-25] 2023-11-25 hour 1: 146393 events in 3 files, snapshot 8039138677181322987 (task 10318 ms: download 1372 ms overlapping gunzip 600 ms, parse 2772 ms, sort 96 ms, write 5995 ms, index 632 ms: 15 candidates, 75460 repo-hours, 716 releases; waiting 22 ms; commit 5868 ms)
2026-11-08T03:10:45Z [load_day 2023-11-25] 2023-11-25 hour 2: 115733 events in 3 files, snapshot 2016058445892092960 (task 8786 ms: download 906 ms overlapping gunzip 624 ms, parse 2647 ms, sort 109 ms, write 4839 ms, index 357 ms: 12 candidates, 42895 repo-hours, 577 releases; waiting 24 ms; commit 3142 ms)
2026-11-08T03:10:49Z [load_day 2023-11-25] 2023-11-25 hour 3: 113921 events in 3 files, snapshot 2276151870799556040 (task 8987 ms: download 1717 ms overlapping gunzip 537 ms, parse 2431 ms, sort 73 ms, write 5076 ms, index 639 ms: 14 candidates, 42979 repo-hours, 669 releases; waiting 32 ms; commit 4566 ms)
2026-11-08T03:10:56Z [load_day 2023-11-25] 2023-11-25 hour 4: 116750 events in 3 files, snapshot 4114596937384221308 (task 8912 ms: download 754 ms overlapping gunzip 490 ms, parse 2320 ms, sort 121 ms, write 5459 ms, index 309 ms: 12 candidates, 42868 repo-hours, 474 releases; waiting 18 ms; commit 4565 ms)
2026-11-08T03:10:59Z [load_day 2023-11-25] 2023-11-25 hour 5: 115938 events in 3 files, snapshot 964526847755797125 (task 8938 ms: download 1285 ms overlapping gunzip 534 ms, parse 2397 ms, sort 58 ms, write 5184 ms, index 543 ms: 15 candidates, 46068 repo-hours, 602 releases; waiting 17 ms; commit 2435 ms)
2026-11-08T03:11:05Z [load_day 2023-11-25] 2023-11-25 hour 6: 124990 events in 3 files, snapshot 7075594024167619548 (task 8717 ms: download 1550 ms overlapping gunzip 508 ms, parse 2315 ms, sort 143 ms, write 4946 ms, index 577 ms: 8 candidates, 53622 repo-hours, 626 releases; waiting 19 ms; commit 3038 ms)
2026-11-08T03:11:11Z [load_day 2023-11-25] 2023-11-25 hour 7: 121842 events in 3 files, snapshot 3301778066026262249 (task 9121 ms: download 1042 ms overlapping gunzip 585 ms, parse 2639 ms, sort 139 ms, write 5120 ms, index 422 ms: 11 candidates, 47649 repo-hours, 509 releases; waiting 17 ms; commit 4719 ms)
2026-11-08T03:11:17Z [load_day 2023-11-25] 2023-11-25 hour 8: 130511 events in 3 files, snapshot 8816167250041466121 (task 9846 ms: download 1120 ms overlapping gunzip 558 ms, parse 2403 ms, sort 84 ms, write 6144 ms, index 399 ms: 10 candidates, 51938 repo-hours, 598 releases; waiting 33 ms; commit 4654 ms)
2026-11-08T03:11:20Z [load_day 2023-11-25] 2023-11-25 hour 9: 122125 events in 3 files, snapshot 7302060654212852214 (task 9557 ms: download 1852 ms overlapping gunzip 573 ms, parse 2533 ms, sort 90 ms, write 5673 ms, index 454 ms: 10 candidates, 49835 repo-hours, 702 releases; waiting 21 ms; commit 2583 ms)
2026-11-08T03:11:28Z [load_day 2023-11-25] 2023-11-25 hour 10: 137587 events in 3 files, snapshot 1612584641190352978 (task 10097 ms: download 1302 ms overlapping gunzip 575 ms, parse 2570 ms, sort 109 ms, write 6227 ms, index 384 ms: 6 candidates, 54333 repo-hours, 709 releases; waiting 25 ms; commit 3627 ms)
2026-11-08T03:11:31Z [load_day 2023-11-25] 2023-11-25 hour 11: 139809 events in 3 files, snapshot 1040918350856801565 (task 9597 ms: download 1289 ms overlapping gunzip 632 ms, parse 2986 ms, sort 120 ms, write 5173 ms, index 468 ms: 6 candidates, 53720 repo-hours, 659 releases; waiting 19 ms; commit 3439 ms)
2026-11-08T03:11:38Z [load_day 2023-11-25] 2023-11-25 hour 12: 203492 events in 5 files, snapshot 7241576634866785027 (task 10579 ms: download 3676 ms overlapping gunzip 674 ms, parse 3267 ms, sort 118 ms, write 5798 ms, index 519 ms: 5 candidates, 85988 repo-hours, 778 releases; waiting 21 ms; commit 2628 ms)
2026-11-08T03:11:41Z [load_day 2023-11-25] 2023-11-25 hour 13: 169889 events in 4 files, snapshot 5615693888737678956 (task 9237 ms: download 2746 ms overlapping gunzip 540 ms, parse 2473 ms, sort 83 ms, write 5333 ms, index 616 ms: 9 candidates, 79428 repo-hours, 755 releases; waiting 16 ms; commit 2344 ms)
2026-11-08T03:11:49Z [load_day 2023-11-25] 2023-11-25 hour 14: 164952 events in 4 files, snapshot 2521783685260192008 (task 9729 ms: download 3121 ms overlapping gunzip 582 ms, parse 2569 ms, sort 86 ms, write 5845 ms, index 446 ms: 9 candidates, 64162 repo-hours, 750 releases; waiting 14 ms; commit 2923 ms)
2026-11-08T03:11:54Z [load_day 2023-11-25] 2023-11-25 hour 15: 160438 events in 4 files, snapshot 5056281433931730526 (task 11766 ms: download 3775 ms overlapping gunzip 638 ms, parse 2761 ms, sort 85 ms, write 6983 ms, index 483 ms: 5 candidates, 66152 repo-hours, 923 releases; waiting 516 ms; commit 2913 ms)
2026-11-08T03:12:04Z [load_day 2023-11-25] 2023-11-25 hour 16: 157753 events in 4 files, snapshot 4556943522541867831 (task 12825 ms: download 5066 ms overlapping gunzip 743 ms, parse 3469 ms, sort 156 ms, write 7129 ms, index 528 ms: 10 candidates, 64796 repo-hours, 768 releases; waiting 549 ms; commit 3561 ms)
2026-11-08T03:12:09Z [load_day 2023-11-25] 2023-11-25 hour 17: 157283 events in 4 files, snapshot 5753380307029691574 (task 12600 ms: download 4453 ms overlapping gunzip 791 ms, parse 3479 ms, sort 174 ms, write 7297 ms, index 616 ms: 7 candidates, 60355 repo-hours, 791 releases; waiting 26 ms; commit 4035 ms)
2026-11-08T03:12:18Z [load_day 2023-11-25] 2023-11-25 hour 18: 158017 events in 4 files, snapshot 3182762476939838232 (task 13015 ms: download 4878 ms overlapping gunzip 745 ms, parse 3761 ms, sort 220 ms, write 7553 ms, index 510 ms: 10 candidates, 65210 repo-hours, 824 releases; waiting 25 ms; commit 3931 ms)
2026-11-08T03:12:21Z [load_day 2023-11-25] 2023-11-25 hour 19: 144481 events in 3 files, snapshot 603031053916964544 (task 10727 ms: download 3798 ms overlapping gunzip 617 ms, parse 3054 ms, sort 210 ms, write 6063 ms, index 560 ms: 7 candidates, 53135 repo-hours, 874 releases; waiting 18 ms; commit 2894 ms)
2026-11-08T03:12:29Z [load_day 2023-11-25] 2023-11-25 hour 20: 143143 events in 3 files, snapshot 5455454891978623870 (task 9457 ms: download 1255 ms overlapping gunzip 562 ms, parse 2538 ms, sort 124 ms, write 5563 ms, index 417 ms: 7 candidates, 52776 repo-hours, 700 releases; waiting 24 ms; commit 4061 ms)
2026-11-08T03:12:32Z [load_day 2023-11-25] 2023-11-25 hour 21: 141089 events in 3 files, snapshot 2097576082342790478 (task 9617 ms: download 1266 ms overlapping gunzip 672 ms, parse 3008 ms, sort 102 ms, write 5216 ms, index 419 ms: 5 candidates, 49110 repo-hours, 872 releases; waiting 15 ms; commit 3213 ms)
2026-11-08T03:12:37Z [load_day 2023-11-25] 2023-11-25 hour 22: 127571 events in 3 files, snapshot 7143321451652022561 (task 8717 ms: download 1270 ms overlapping gunzip 493 ms, parse 2252 ms, sort 80 ms, write 5330 ms, index 333 ms: 9 candidates, 44835 repo-hours, 795 releases; waiting 31 ms; commit 2571 ms)
2026-11-08T03:12:41Z [load_day 2023-11-25] 2023-11-25 hour 23: 131486 events in 3 files, snapshot 554344913630094476 (task 8222 ms: download 787 ms overlapping gunzip 444 ms, parse 2093 ms, sort 109 ms, write 4919 ms, index 431 ms: 8 candidates, 43199 repo-hours, 726 releases; waiting 23 ms; commit 3202 ms)
2026-11-08T03:12:48Z [load_day 2023-11-25] 2023-11-25: repo_hour merged from 24 files to 1 (1366811 rows, 6431 ms)
2026-11-08T03:12:50Z [load_day 2023-11-25] 2023-11-25: index committed in 9033 ms (24 + 1 + 24 files)
2026-11-08T03:12:50Z [load_day 2023-11-25] 2023-11-25: 3396933 events in 24 hours
2026-11-08T03:12:50Z [load_day 2023-11-25] attempt 1 succeeded