orcaLog in

Runs · run 786

load_day 2023-02-03, attempt 1 succeeded

Host
caf251265124:14414
Started
2026-11-07T14:59:34Z
Ended
2026-11-07T15:02:50Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/786/load_day/2023-02-03/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T14:59:34Z [load_day 2023-02-03] attempt 1 started
2026-11-07T14:59:35Z [load_day 2023-02-03] 2023-02-03: 2 hours in flight, about 1200 MB
2026-11-07T14:59:50Z [load_day 2023-02-03] 2023-02-03 hour 0: 154224 events in 4 files, snapshot 539046852996304079 (task 12718 ms: download 5640 ms overlapping gunzip 818 ms, parse 3377 ms, sort 94 ms, write 7082 ms, index 516 ms: 5 candidates, 73057 repo-hours, 873 releases; waiting 495 ms; commit 1309 ms)
2026-11-07T14:59:52Z [load_day 2023-02-03] 2023-02-03 hour 1: 152938 events in 4 files, snapshot 9019055392316846585 (task 12027 ms: download 5229 ms overlapping gunzip 756 ms, parse 3366 ms, sort 107 ms, write 6379 ms, index 563 ms: 5 candidates, 79080 repo-hours, 869 releases; waiting 508 ms; commit 1344 ms)
2026-11-07T15:00:03Z [load_day 2023-02-03] 2023-02-03 hour 2: 133491 events in 3 files, snapshot 947359587640628443 (task 11488 ms: download 5182 ms overlapping gunzip 652 ms, parse 2738 ms, sort 84 ms, write 6448 ms, index 490 ms: 9 candidates, 65903 repo-hours, 657 releases; waiting 487 ms; commit 1569 ms)
2026-11-07T15:00:04Z [load_day 2023-02-03] 2023-02-03 hour 3: 127864 events in 3 files, snapshot 2833290237318250962 (task 10230 ms: download 4704 ms overlapping gunzip 649 ms, parse 2764 ms, sort 73 ms, write 5518 ms, index 389 ms: 2 candidates, 56468 repo-hours, 826 releases; waiting 524 ms; commit 1478 ms)
2026-11-07T15:00:16Z [load_day 2023-02-03] 2023-02-03 hour 4: 136981 events in 3 files, snapshot 8741119831743848469 (task 11696 ms: download 5556 ms overlapping gunzip 711 ms, parse 2964 ms, sort 88 ms, write 6464 ms, index 413 ms: 1 candidates, 56352 repo-hours, 671 releases; waiting 721 ms; commit 1549 ms)
2026-11-07T15:00:17Z [load_day 2023-02-03] 2023-02-03 hour 5: 136986 events in 3 files, snapshot 7753767974315447395 (task 11081 ms: download 5064 ms overlapping gunzip 648 ms, parse 2833 ms, sort 88 ms, write 5938 ms, index 460 ms: 1 candidates, 59116 repo-hours, 727 releases; waiting 791 ms; commit 1582 ms)
2026-11-07T15:00:31Z [load_day 2023-02-03] 2023-02-03 hour 6: 152322 events in 4 files, snapshot 3051000785363247525 (task 14269 ms: download 7781 ms overlapping gunzip 746 ms, parse 3107 ms, sort 93 ms, write 6206 ms, index 452 ms: 3 candidates, 68581 repo-hours, 782 releases; waiting 3315 ms; commit 1416 ms)
2026-11-07T15:00:32Z [load_day 2023-02-03] 2023-02-03 hour 7: 157572 events in 4 files, snapshot 2204854011608167714 (task 12570 ms: download 5425 ms overlapping gunzip 771 ms, parse 3304 ms, sort 84 ms, write 7083 ms, index 490 ms: 4 candidates, 70439 repo-hours, 704 releases; waiting 552 ms; commit 1391 ms)
2026-11-07T15:00:45Z [load_day 2023-02-03] 2023-02-03 hour 8: 175219 events in 4 files, snapshot 5669542793563418855 (task 13940 ms: download 7954 ms overlapping gunzip 857 ms, parse 3718 ms, sort 104 ms, write 8210 ms, index 521 ms: 3 candidates, 78719 repo-hours, 973 releases; waiting 205 ms; commit 1385 ms)
2026-11-07T15:00:48Z [load_day 2023-02-03] 2023-02-03 hour 9: 185725 events in 4 files, snapshot 4978321725843860158 (task 15545 ms: download 8952 ms overlapping gunzip 959 ms, parse 4133 ms, sort 101 ms, write 8860 ms, index 599 ms: 5 candidates, 82072 repo-hours, 1222 releases; waiting 562 ms; commit 1462 ms)
2026-11-07T15:01:02Z [load_day 2023-02-03] 2023-02-03 hour 10: 185030 events in 4 files, snapshot 4940517085527405257 (task 15539 ms: download 8834 ms overlapping gunzip 959 ms, parse 4117 ms, sort 112 ms, write 8901 ms, index 589 ms: 5 candidates, 81650 repo-hours, 1147 releases; waiting 524 ms; commit 1519 ms)
2026-11-07T15:01:04Z [load_day 2023-02-03] 2023-02-03 hour 11: 178421 events in 4 files, snapshot 2931643800458454245 (task 14900 ms: download 8805 ms overlapping gunzip 904 ms, parse 3916 ms, sort 120 ms, write 8253 ms, index 520 ms: 4 candidates, 77573 repo-hours, 940 releases; waiting 883 ms; commit 1545 ms)
2026-11-07T15:01:19Z [load_day 2023-02-03] 2023-02-03 hour 12: 215046 events in 5 files, snapshot 7588959486626759685 (task 16185 ms: download 8610 ms overlapping gunzip 963 ms, parse 4298 ms, sort 134 ms, write 9205 ms, index 682 ms: 7 candidates, 105999 repo-hours, 1020 releases; waiting 527 ms; commit 1541 ms)
2026-11-07T15:01:22Z [load_day 2023-02-03] 2023-02-03 hour 13: 206497 events in 5 files, snapshot 7639422497347411839 (task 16918 ms: download 9195 ms overlapping gunzip 972 ms, parse 4222 ms, sort 131 ms, write 9549 ms, index 664 ms: 7 candidates, 101521 repo-hours, 1061 releases; waiting 1057 ms; commit 1428 ms)
2026-11-07T15:01:37Z [load_day 2023-02-03] 2023-02-03 hour 14: 200037 events in 5 files, snapshot 6822070219260426930 (task 17132 ms: download 9772 ms overlapping gunzip 1017 ms, parse 4457 ms, sort 134 ms, write 9713 ms, index 616 ms: 5 candidates, 89474 repo-hours, 1102 releases; waiting 886 ms; commit 1555 ms)
2026-11-07T15:01:39Z [load_day 2023-02-03] 2023-02-03 hour 15: 200044 events in 5 files, snapshot 7861655113210955367 (task 16745 ms: download 9198 ms overlapping gunzip 1016 ms, parse 4419 ms, sort 135 ms, write 9738 ms, index 659 ms: 2 candidates, 90771 repo-hours, 1331 releases; waiting 519 ms; commit 1398 ms)
2026-11-07T15:01:54Z [load_day 2023-02-03] 2023-02-03 hour 16: 188717 events in 4 files, snapshot 5346829651977894256 (task 16508 ms: download 9608 ms overlapping gunzip 962 ms, parse 4192 ms, sort 114 ms, write 9116 ms, index 654 ms: 6 candidates, 85817 repo-hours, 1100 releases; waiting 1088 ms; commit 1458 ms)
2026-11-07T15:01:55Z [load_day 2023-02-03] 2023-02-03 hour 17: 187021 events in 4 files, snapshot 8011066587961424853 (task 15182 ms: download 8480 ms overlapping gunzip 945 ms, parse 4146 ms, sort 132 ms, write 8789 ms, index 597 ms: 3 candidates, 80725 repo-hours, 916 releases; waiting 221 ms; commit 1409 ms)
2026-11-07T15:02:10Z [load_day 2023-02-03] 2023-02-03 hour 18: 183096 events in 4 files, snapshot 6658676521598875720 (task 15206 ms: download 8643 ms overlapping gunzip 896 ms, parse 4008 ms, sort 120 ms, write 8764 ms, index 605 ms: 8 candidates, 81551 repo-hours, 996 releases; waiting 508 ms; commit 1274 ms)
2026-11-07T15:02:11Z [load_day 2023-02-03] 2023-02-03 hour 19: 170246 events in 4 files, snapshot 5289562411757026298 (task 14194 ms: download 8262 ms overlapping gunzip 853 ms, parse 3695 ms, sort 109 ms, write 8133 ms, index 526 ms: 7 candidates, 73530 repo-hours, 933 releases; waiting 529 ms; commit 1266 ms)
2026-11-07T15:02:25Z [load_day 2023-02-03] 2023-02-03 hour 20: 166228 events in 4 files, snapshot 914902291000636418 (task 13799 ms: download 8378 ms overlapping gunzip 842 ms, parse 3655 ms, sort 93 ms, write 7818 ms, index 481 ms: 4 candidates, 70813 repo-hours, 906 releases; waiting 539 ms; commit 1256 ms)
2026-11-07T15:02:26Z [load_day 2023-02-03] 2023-02-03 hour 21: 165301 events in 4 files, snapshot 3447447867321865415 (task 13872 ms: download 8106 ms overlapping gunzip 867 ms, parse 3755 ms, sort 110 ms, write 7801 ms, index 462 ms: 6 candidates, 70272 repo-hours, 880 releases; waiting 512 ms; commit 1567 ms)
2026-11-07T15:02:39Z [load_day 2023-02-03] 2023-02-03 hour 22: 150054 events in 4 files, snapshot 5624840254320109791 (task 13986 ms: download 6810 ms overlapping gunzip 848 ms, parse 3599 ms, sort 111 ms, write 7438 ms, index 407 ms: 9 candidates, 61808 repo-hours, 828 releases; waiting 1221 ms; commit 1488 ms)
2026-11-07T15:02:41Z [load_day 2023-02-03] 2023-02-03 hour 23: 133089 events in 3 files, snapshot 7318782603275733969 (task 11274 ms: download 5222 ms overlapping gunzip 680 ms, parse 2860 ms, sort 79 ms, write 6377 ms, index 351 ms: 0 candidates, 56069 repo-hours, 686 releases; waiting 595 ms; commit 1339 ms)
2026-11-07T15:02:48Z [load_day 2023-02-03] 2023-02-03: repo_hour merged from 24 files to 1 (1817360 rows, 7442 ms)
2026-11-07T15:02:50Z [load_day 2023-02-03] 2023-02-03: index committed in 9564 ms (23 + 1 + 24 files)
2026-11-07T15:02:50Z [load_day 2023-02-03] 2023-02-03: 4042149 events in 24 hours
2026-11-07T15:02:50Z [load_day 2023-02-03] attempt 1 succeeded