orcaLog in

Runs · run 873

load_day 2023-03-04, attempt 1 succeeded

Host
caf251265124:122472
Started
2026-11-07T16:31:24Z
Ended
2026-11-07T16:33:52Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/873/load_day/2023-03-04/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T16:31:24Z [load_day 2023-03-04] attempt 1 started
2026-11-07T16:31:25Z [load_day 2023-03-04] 2023-03-04: 2 hours in flight, about 1200 MB
2026-11-07T16:31:39Z [load_day 2023-03-04] 2023-03-04 hour 0: 164921 events in 4 files, snapshot 5353472951755823047 (task 11199 ms: download 4570 ms overlapping gunzip 668 ms, parse 2857 ms, sort 85 ms, write 6478 ms, index 560 ms: 15 candidates, 94446 repo-hours, 1176 releases; waiting 235 ms; commit 1246 ms)
2026-11-07T16:31:41Z [load_day 2023-03-04] 2023-03-04 hour 1: 168407 events in 4 files, snapshot 239751459718450087 (task 11434 ms: download 5133 ms overlapping gunzip 732 ms, parse 3026 ms, sort 97 ms, write 6122 ms, index 565 ms: 3 candidates, 95256 repo-hours, 1072 releases; waiting 600 ms; commit 1547 ms)
2026-11-07T16:31:51Z [load_day 2023-03-04] 2023-03-04 hour 2: 160247 events in 4 files, snapshot 5015961142722383108 (task 11261 ms: download 4857 ms overlapping gunzip 721 ms, parse 2858 ms, sort 78 ms, write 6001 ms, index 522 ms: 4 candidates, 87983 repo-hours, 894 releases; waiting 672 ms; commit 1367 ms)
2026-11-07T16:31:52Z [load_day 2023-03-04] 2023-03-04 hour 3: 159492 events in 4 files, snapshot 1196090945044002877 (task 10942 ms: download 4672 ms overlapping gunzip 716 ms, parse 2887 ms, sort 85 ms, write 5808 ms, index 487 ms: 1 candidates, 82786 repo-hours, 1017 releases; waiting 641 ms; commit 1384 ms)
2026-11-07T16:32:04Z [load_day 2023-03-04] 2023-03-04 hour 4: 163094 events in 4 files, snapshot 7729669275756239011 (task 11734 ms: download 4784 ms overlapping gunzip 749 ms, parse 3037 ms, sort 101 ms, write 5953 ms, index 551 ms: 6 candidates, 87154 repo-hours, 837 releases; waiting 545 ms; commit 1444 ms)
2026-11-07T16:32:05Z [load_day 2023-03-04] 2023-03-04 hour 5: 162094 events in 4 files, snapshot 8960068482958028208 (task 10699 ms: download 4097 ms overlapping gunzip 670 ms, parse 2692 ms, sort 92 ms, write 5772 ms, index 572 ms: 5 candidates, 87129 repo-hours, 954 releases; waiting 529 ms; commit 1439 ms)
2026-11-07T16:32:14Z [load_day 2023-03-04] 2023-03-04 hour 6: 146803 events in 3 files, snapshot 8089384451078371446 (task 9292 ms: download 3725 ms overlapping gunzip 575 ms, parse 2378 ms, sort 82 ms, write 4801 ms, index 499 ms: 4 candidates, 78005 repo-hours, 1016 releases; waiting 598 ms; commit 1431 ms)
2026-11-07T16:32:15Z [load_day 2023-03-04] 2023-03-04 hour 7: 132226 events in 3 files, snapshot 8381124909762345050 (task 9011 ms: download 2801 ms overlapping gunzip 541 ms, parse 2209 ms, sort 73 ms, write 4685 ms, index 408 ms: 1 candidates, 62737 repo-hours, 875 releases; waiting 738 ms; commit 1381 ms)
2026-11-07T16:32:24Z [load_day 2023-03-04] 2023-03-04 hour 8: 152304 events in 4 files, snapshot 6122806648695966594 (task 9196 ms: download 3384 ms overlapping gunzip 546 ms, parse 2298 ms, sort 90 ms, write 4968 ms, index 485 ms: 2 candidates, 80254 repo-hours, 873 releases; waiting 517 ms; commit 1443 ms)
2026-11-07T16:32:27Z [load_day 2023-03-04] 2023-03-04 hour 9: 157572 events in 4 files, snapshot 2360516529501274155 (task 11585 ms: download 5503 ms overlapping gunzip 602 ms, parse 2512 ms, sort 89 ms, write 5197 ms, index 530 ms: 6 candidates, 84444 repo-hours, 1035 releases; waiting 2337 ms; commit 1538 ms)
2026-11-07T16:32:34Z [load_day 2023-03-04] 2023-03-04 hour 10: 170125 events in 4 files, snapshot 1354756086332911323 (task 9646 ms: download 3213 ms overlapping gunzip 548 ms, parse 2287 ms, sort 89 ms, write 5205 ms, index 546 ms: 7 candidates, 99157 repo-hours, 918 releases; waiting 510 ms; commit 1579 ms)
2026-11-07T16:32:38Z [load_day 2023-03-04] 2023-03-04 hour 11: 157979 events in 4 files, snapshot 846326615851968759 (task 9791 ms: download 3716 ms overlapping gunzip 569 ms, parse 2373 ms, sort 97 ms, write 5019 ms, index 558 ms: 4 candidates, 82774 repo-hours, 987 releases; waiting 709 ms; commit 1460 ms)
2026-11-07T16:32:46Z [load_day 2023-03-04] 2023-03-04 hour 12: 184988 events in 4 files, snapshot 6805390815171153758 (task 10555 ms: download 3861 ms overlapping gunzip 642 ms, parse 2806 ms, sort 105 ms, write 5726 ms, index 521 ms: 4 candidates, 89736 repo-hours, 1000 releases; waiting 408 ms; commit 1506 ms)
2026-11-07T16:32:50Z [load_day 2023-03-04] 2023-03-04 hour 13: 166753 events in 4 files, snapshot 9097502898879473166 (task 11061 ms: download 4315 ms overlapping gunzip 623 ms, parse 2678 ms, sort 88 ms, write 5819 ms, index 510 ms: 3 candidates, 83603 repo-hours, 1034 releases; waiting 1045 ms; commit 1483 ms)
2026-11-07T16:32:57Z [load_day 2023-03-04] 2023-03-04 hour 14: 154150 events in 4 files, snapshot 6475813069722506849 (task 10665 ms: download 4223 ms overlapping gunzip 646 ms, parse 2681 ms, sort 95 ms, write 5855 ms, index 471 ms: 2 candidates, 68463 repo-hours, 1052 releases; waiting 575 ms; commit 1341 ms)
2026-11-07T16:33:02Z [load_day 2023-03-04] 2023-03-04 hour 15: 162200 events in 4 files, snapshot 1177496301659061119 (task 11281 ms: download 4724 ms overlapping gunzip 691 ms, parse 2881 ms, sort 83 ms, write 6146 ms, index 467 ms: 2 candidates, 71554 repo-hours, 1247 releases; waiting 639 ms; commit 1583 ms)
2026-11-07T16:33:08Z [load_day 2023-03-04] 2023-03-04 hour 16: 157797 events in 4 files, snapshot 8082233524854274604 (task 10484 ms: download 4193 ms overlapping gunzip 611 ms, parse 2617 ms, sort 93 ms, write 5755 ms, index 502 ms: 2 candidates, 73580 repo-hours, 1106 releases; waiting 597 ms; commit 1465 ms)
2026-11-07T16:33:13Z [load_day 2023-03-04] 2023-03-04 hour 17: 151462 events in 4 files, snapshot 1431022888630810117 (task 10748 ms: download 4403 ms overlapping gunzip 663 ms, parse 2834 ms, sort 107 ms, write 5902 ms, index 414 ms: 3 candidates, 63973 repo-hours, 1118 releases; waiting 510 ms; commit 1575 ms)
2026-11-07T16:33:19Z [load_day 2023-03-04] 2023-03-04 hour 18: 155258 events in 4 files, snapshot 1918767030016859980 (task 10177 ms: download 3896 ms overlapping gunzip 633 ms, parse 2733 ms, sort 77 ms, write 5655 ms, index 443 ms: 5 candidates, 69759 repo-hours, 1028 releases; waiting 360 ms; commit 1382 ms)
2026-11-07T16:33:24Z [load_day 2023-03-04] 2023-03-04 hour 19: 133902 events in 3 files, snapshot 6900788589866009432 (task 9903 ms: download 4295 ms overlapping gunzip 586 ms, parse 2549 ms, sort 86 ms, write 5113 ms, index 374 ms: 1 candidates, 55084 repo-hours, 964 releases; waiting 872 ms; commit 1340 ms)
2026-11-07T16:33:29Z [load_day 2023-03-04] 2023-03-04 hour 20: 131304 events in 3 files, snapshot 8259224509535949865 (task 9105 ms: download 3811 ms overlapping gunzip 547 ms, parse 2347 ms, sort 78 ms, write 4771 ms, index 385 ms: 6 candidates, 53992 repo-hours, 999 releases; waiting 611 ms; commit 1441 ms)
2026-11-07T16:33:34Z [load_day 2023-03-04] 2023-03-04 hour 21: 152368 events in 4 files, snapshot 1268954369103710446 (task 9290 ms: download 3475 ms overlapping gunzip 524 ms, parse 2274 ms, sort 101 ms, write 5100 ms, index 468 ms: 6 candidates, 80308 repo-hours, 1016 releases; waiting 517 ms; commit 1393 ms)
2026-11-07T16:33:38Z [load_day 2023-03-04] 2023-03-04 hour 22: 117881 events in 3 files, snapshot 2127181692062329016 (task 8378 ms: download 2289 ms overlapping gunzip 504 ms, parse 2210 ms, sort 68 ms, write 4416 ms, index 307 ms: 11 candidates, 46342 repo-hours, 1030 releases; waiting 591 ms; commit 1464 ms)
2026-11-07T16:33:43Z [load_day 2023-03-04] 2023-03-04 hour 23: 115856 events in 3 files, snapshot 4636455999129001700 (task 7875 ms: download 2275 ms overlapping gunzip 466 ms, parse 2021 ms, sort 63 ms, write 4021 ms, index 323 ms: 3 candidates, 52133 repo-hours, 910 releases; waiting 693 ms; commit 1407 ms)
2026-11-07T16:33:50Z [load_day 2023-03-04] 2023-03-04: repo_hour merged from 24 files to 1 (1830652 rows, 7058 ms)
2026-11-07T16:33:52Z [load_day 2023-03-04] 2023-03-04: index committed in 9007 ms (24 + 1 + 24 files)
2026-11-07T16:33:52Z [load_day 2023-03-04] 2023-03-04: 3679183 events in 24 hours
2026-11-07T16:33:52Z [load_day 2023-03-04] attempt 1 succeeded