orcaLog in

Runs · run 942

load_day 2023-03-27, attempt 1 succeeded

Host
caf251265124:204692
Started
2026-11-07T17:43:36Z
Ended
2026-11-07T17:46:55Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/942/load_day/2023-03-27/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T17:43:36Z [load_day 2023-03-27] attempt 1 started
2026-11-07T17:43:38Z [load_day 2023-03-27] 2023-03-27: 2 hours in flight, about 1200 MB
2026-11-07T17:43:50Z [load_day 2023-03-27] 2023-03-27 hour 0: 141472 events in 3 files, snapshot 3991986193293687406 (task 9961 ms: download 4156 ms overlapping gunzip 565 ms, parse 2586 ms, sort 92 ms, write 5685 ms, index 495 ms: 7 candidates, 69899 repo-hours, 834 releases; waiting 223 ms; commit 1343 ms)
2026-11-07T17:43:54Z [load_day 2023-03-27] 2023-03-27 hour 1: 176029 events in 4 files, snapshot 4473168068911461931 (task 13886 ms: download 8155 ms overlapping gunzip 922 ms, parse 3882 ms, sort 115 ms, write 7465 ms, index 568 ms: 7 candidates, 84846 repo-hours, 816 releases; waiting 593 ms; commit 1669 ms)
2026-11-07T17:44:03Z [load_day 2023-03-27] 2023-03-27 hour 2: 150419 events in 4 files, snapshot 1859546473040518458 (task 12376 ms: download 5453 ms overlapping gunzip 797 ms, parse 3331 ms, sort 102 ms, write 6801 ms, index 509 ms: 4 candidates, 64363 repo-hours, 797 releases; waiting 520 ms; commit 1366 ms)
2026-11-07T17:44:07Z [load_day 2023-03-27] 2023-03-27 hour 3: 144816 events in 3 files, snapshot 2861765162596555231 (task 12604 ms: download 6429 ms overlapping gunzip 749 ms, parse 3096 ms, sort 102 ms, write 6231 ms, index 392 ms: 7 candidates, 59372 repo-hours, 864 releases; waiting 1755 ms; commit 1474 ms)
2026-11-07T17:44:15Z [load_day 2023-03-27] 2023-03-27 hour 4: 145801 events in 3 files, snapshot 8774435737636400178 (task 11444 ms: download 5060 ms overlapping gunzip 692 ms, parse 2969 ms, sort 105 ms, write 6344 ms, index 434 ms: 5 candidates, 64690 repo-hours, 631 releases; waiting 532 ms; commit 1434 ms)
2026-11-07T17:44:21Z [load_day 2023-03-27] 2023-03-27 hour 5: 154633 events in 4 files, snapshot 5531156747134884478 (task 13196 ms: download 5920 ms overlapping gunzip 854 ms, parse 3636 ms, sort 111 ms, write 7384 ms, index 395 ms: 4 candidates, 61795 repo-hours, 763 releases; waiting 473 ms; commit 1424 ms)
2026-11-07T17:44:31Z [load_day 2023-03-27] 2023-03-27 hour 6: 176937 events in 4 files, snapshot 7776438540046385708 (task 14337 ms: download 8294 ms overlapping gunzip 902 ms, parse 3845 ms, sort 115 ms, write 8048 ms, index 538 ms: 4 candidates, 78494 repo-hours, 933 releases; waiting 589 ms; commit 1588 ms)
2026-11-07T17:44:38Z [load_day 2023-03-27] 2023-03-27 hour 7: 182773 events in 4 files, snapshot 3819711753691827644 (task 15999 ms: download 9302 ms overlapping gunzip 983 ms, parse 4200 ms, sort 116 ms, write 9016 ms, index 518 ms: 5 candidates, 81369 repo-hours, 898 releases; waiting 706 ms; commit 1418 ms)
2026-11-07T17:44:46Z [load_day 2023-03-27] 2023-03-27 hour 8: 168717 events in 4 files, snapshot 5056999439288528390 (task 14442 ms: download 8375 ms overlapping gunzip 881 ms, parse 3734 ms, sort 102 ms, write 8603 ms, index 541 ms: 3 candidates, 79216 repo-hours, 900 releases; waiting 225 ms; commit 1517 ms)
2026-11-07T17:44:53Z [load_day 2023-03-27] 2023-03-27 hour 9: 171840 events in 4 files, snapshot 1523485796734757346 (task 14740 ms: download 8710 ms overlapping gunzip 909 ms, parse 3952 ms, sort 110 ms, write 8317 ms, index 531 ms: 5 candidates, 83130 repo-hours, 993 releases; waiting 537 ms; commit 1380 ms)
2026-11-07T17:45:02Z [load_day 2023-03-27] 2023-03-27 hour 10: 179052 events in 4 files, snapshot 8550823765995227679 (task 14871 ms: download 8497 ms overlapping gunzip 890 ms, parse 3817 ms, sort 118 ms, write 8606 ms, index 614 ms: 3 candidates, 85862 repo-hours, 877 releases; waiting 521 ms; commit 1827 ms)
2026-11-07T17:45:10Z [load_day 2023-03-27] 2023-03-27 hour 11: 174470 events in 4 files, snapshot 6896212960466231288 (task 15387 ms: download 8993 ms overlapping gunzip 943 ms, parse 3994 ms, sort 109 ms, write 8974 ms, index 547 ms: 4 candidates, 78518 repo-hours, 873 releases; waiting 543 ms; commit 1566 ms)
2026-11-07T17:45:11Z [load_day 2023-03-27] 2023-03-27 hour 12: 53770 events in 2 files, snapshot 2616592011587854280 (task 6157 ms: download 1780 ms overlapping gunzip 352 ms, parse 1420 ms, sort 38 ms, write 3230 ms, index 223 ms: 2 candidates, 30142 repo-hours, 205 releases; waiting 537 ms; commit 1499 ms)
2026-11-07T17:45:25Z [load_day 2023-03-27] 2023-03-27 hour 13: 151104 events in 4 files, snapshot 3190343273081821346 (task 13893 ms: download 5817 ms overlapping gunzip 778 ms, parse 3430 ms, sort 93 ms, write 7900 ms, index 598 ms: 4 candidates, 87926 repo-hours, 701 releases; waiting 749 ms; commit 1691 ms)
2026-11-07T17:45:29Z [load_day 2023-03-27] 2023-03-27 hour 14: 181448 events in 4 files, snapshot 9047543168463838288 (task 17245 ms: download 10307 ms overlapping gunzip 993 ms, parse 4353 ms, sort 113 ms, write 9933 ms, index 605 ms: 7 candidates, 92505 repo-hours, 1027 releases; waiting 862 ms; commit 1519 ms)
2026-11-07T17:45:42Z [load_day 2023-03-27] 2023-03-27 hour 15: 191349 events in 4 files, snapshot 2222423117933827737 (task 16933 ms: download 9366 ms overlapping gunzip 930 ms, parse 4063 ms, sort 117 ms, write 9627 ms, index 634 ms: 13 candidates, 104174 repo-hours, 1153 releases; waiting 942 ms; commit 1490 ms)
2026-11-07T17:45:47Z [load_day 2023-03-27] 2023-03-27 hour 16: 193449 events in 4 files, snapshot 551064094671193917 (task 17014 ms: download 9550 ms overlapping gunzip 989 ms, parse 4253 ms, sort 149 ms, write 9862 ms, index 623 ms: 3 candidates, 98039 repo-hours, 938 releases; waiting 583 ms; commit 1460 ms)
2026-11-07T17:45:59Z [load_day 2023-03-27] 2023-03-27 hour 17: 176535 events in 4 files, snapshot 8084975730136034119 (task 15925 ms: download 9500 ms overlapping gunzip 940 ms, parse 4030 ms, sort 107 ms, write 8993 ms, index 581 ms: 6 candidates, 80475 repo-hours, 797 releases; waiting 886 ms; commit 1571 ms)
2026-11-07T17:46:04Z [load_day 2023-03-27] 2023-03-27 hour 18: 188636 events in 4 files, snapshot 8798377858422182426 (task 16180 ms: download 9311 ms overlapping gunzip 984 ms, parse 4243 ms, sort 121 ms, write 9231 ms, index 586 ms: 10 candidates, 85953 repo-hours, 954 releases; waiting 713 ms; commit 1539 ms)
2026-11-07T17:46:15Z [load_day 2023-03-27] 2023-03-27 hour 19: 186721 events in 4 files, snapshot 1062451288635943386 (task 14872 ms: download 8120 ms overlapping gunzip 889 ms, parse 3813 ms, sort 113 ms, write 8836 ms, index 592 ms: 5 candidates, 93074 repo-hours, 935 releases; waiting 288 ms; commit 1782 ms)
2026-11-07T17:46:21Z [load_day 2023-03-27] 2023-03-27 hour 20: 191260 events in 4 files, snapshot 6357080945166542284 (task 15872 ms: download 8932 ms overlapping gunzip 956 ms, parse 3993 ms, sort 140 ms, write 9225 ms, index 604 ms: 5 candidates, 96048 repo-hours, 949 releases; waiting 585 ms; commit 1623 ms)
2026-11-07T17:46:31Z [load_day 2023-03-27] 2023-03-27 hour 21: 174883 events in 4 files, snapshot 1190927617144355957 (task 15620 ms: download 9343 ms overlapping gunzip 993 ms, parse 4103 ms, sort 109 ms, write 9006 ms, index 534 ms: 10 candidates, 74367 repo-hours, 955 releases; waiting 542 ms; commit 1646 ms)
2026-11-07T17:46:37Z [load_day 2023-03-27] 2023-03-27 hour 22: 165975 events in 4 files, snapshot 3680168283093194901 (task 15784 ms: download 10167 ms overlapping gunzip 984 ms, parse 4046 ms, sort 98 ms, write 9196 ms, index 518 ms: 11 candidates, 66177 repo-hours, 805 releases; waiting 567 ms; commit 1368 ms)
2026-11-07T17:46:44Z [load_day 2023-03-27] 2023-03-27 hour 23: 143919 events in 3 files, snapshot 2782733830036083254 (task 12962 ms: download 6171 ms overlapping gunzip 778 ms, parse 3301 ms, sort 87 ms, write 6886 ms, index 430 ms: 10 candidates, 59545 repo-hours, 686 releases; waiting 1053 ms; commit 1351 ms)
2026-11-07T17:46:52Z [load_day 2023-03-27] 2023-03-27: repo_hour merged from 24 files to 1 (1859979 rows, 8003 ms)
2026-11-07T17:46:55Z [load_day 2023-03-27] 2023-03-27: index committed in 10284 ms (24 + 1 + 24 files)
2026-11-07T17:46:55Z [load_day 2023-03-27] 2023-03-27: 3966008 events in 24 hours
2026-11-07T17:46:55Z [load_day 2023-03-27] attempt 1 succeeded