orcaLog in

Runs · run 1670

load_day 2023-10-27, attempt 1 succeeded

Host
3c2b29713ce8:487898
Started
2026-11-08T02:13:08Z
Ended
2026-11-08T02:17:20Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1670/load_day/2023-10-27/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:13:08Z [load_day 2023-10-27] attempt 1 started
2026-11-08T02:13:10Z [load_day 2023-10-27] 2023-10-27: 2 hours in flight, about 1200 MB
2026-11-08T02:13:32Z [load_day 2023-10-27] 2023-10-27 hour 0: 170277 events in 4 files, snapshot 2826939662433267596 (task 16119 ms: download 7022 ms overlapping gunzip 981 ms, parse 4693 ms, sort 121 ms, write 9489 ms, index 746 ms: 10 candidates, 92219 repo-hours, 1269 releases; waiting 25 ms; commit 4335 ms)
2026-11-08T02:13:34Z [load_day 2023-10-27] 2023-10-27 hour 1: 168553 events in 4 files, snapshot 4280040128660021114 (task 16437 ms: download 6181 ms overlapping gunzip 911 ms, parse 4236 ms, sort 206 ms, write 10087 ms, index 913 ms: 6 candidates, 93871 repo-hours, 879 releases; waiting 20 ms; commit 2636 ms)
2026-11-08T02:13:44Z [load_day 2023-10-27] 2023-10-27 hour 2: 141646 events in 3 files, snapshot 2860658547697061323 (task 12942 ms: download 5424 ms overlapping gunzip 776 ms, parse 3394 ms, sort 99 ms, write 8036 ms, index 552 ms: 3 candidates, 63006 repo-hours, 836 releases; waiting 19 ms; commit 2617 ms)
2026-11-08T02:13:49Z [load_day 2023-10-27] 2023-10-27 hour 3: 135927 events in 3 files, snapshot 3704721306874549743 (task 13540 ms: download 5605 ms overlapping gunzip 846 ms, parse 3726 ms, sort 159 ms, write 8254 ms, index 487 ms: 1 candidates, 60520 repo-hours, 849 releases; waiting 18 ms; commit 2631 ms)
2026-11-08T02:14:07Z [load_day 2023-10-27] 2023-10-27 hour 4: 132055 events in 3 files, snapshot 624096742409316801 (task 12926 ms: download 4836 ms overlapping gunzip 774 ms, parse 3412 ms, sort 137 ms, write 8027 ms, index 471 ms: 4 candidates, 55328 repo-hours, 654 releases; waiting 16 ms; commit 11861 ms)
2026-11-08T02:14:09Z [load_day 2023-10-27] 2023-10-27 hour 5: 141084 events in 3 files, snapshot 8016804582805255811 (task 15441 ms: download 6534 ms overlapping gunzip 973 ms, parse 4228 ms, sort 154 ms, write 9431 ms, index 559 ms: 6 candidates, 61277 repo-hours, 743 releases; waiting 23 ms; commit 2339 ms)
2026-11-08T02:14:14Z [load_day 2023-10-27] 2023-10-27 hour 6: 174248 events in 4 files, snapshot 2844069048402024979 (task 14778 ms: download 8513 ms overlapping gunzip 1021 ms, parse 4458 ms, sort 125 ms, write 8522 ms, index 546 ms: 6 candidates, 78499 repo-hours, 962 releases; waiting 24 ms; commit 2655 ms)
2026-11-08T02:14:27Z [load_day 2023-10-27] 2023-10-27 hour 7: 181678 events in 4 files, snapshot 8153961840069379935 (task 16611 ms: download 8524 ms overlapping gunzip 990 ms, parse 4376 ms, sort 120 ms, write 10305 ms, index 738 ms: 10 candidates, 79163 repo-hours, 956 releases; waiting 19 ms; commit 2704 ms)
2026-11-08T02:14:34Z [load_day 2023-10-27] 2023-10-27 hour 8: 194728 events in 4 files, snapshot 4015620330990206516 (task 18653 ms: download 8626 ms overlapping gunzip 1176 ms, parse 4993 ms, sort 133 ms, write 11472 ms, index 813 ms: 4 candidates, 86644 repo-hours, 1122 releases; waiting 10 ms; commit 2953 ms)
2026-11-08T02:14:50Z [load_day 2023-10-27] 2023-10-27 hour 9: 194919 events in 4 files, snapshot 1420345119605266853 (task 20091 ms: download 11134 ms overlapping gunzip 1249 ms, parse 5367 ms, sort 157 ms, write 12599 ms, index 633 ms: 7 candidates, 85881 repo-hours, 1213 releases; waiting 10 ms; commit 4447 ms)
2026-11-08T02:14:55Z [load_day 2023-10-27] 2023-10-27 hour 10: 186307 events in 4 files, snapshot 379488962443621640 (task 19298 ms: download 10462 ms overlapping gunzip 1334 ms, parse 5885 ms, sort 157 ms, write 11164 ms, index 663 ms: 14 candidates, 81584 repo-hours, 1096 releases; waiting 23 ms; commit 2987 ms)
2026-11-08T02:15:10Z [load_day 2023-10-27] 2023-10-27 hour 11: 190897 events in 4 files, snapshot 5700056087235572281 (task 20282 ms: download 11262 ms overlapping gunzip 1249 ms, parse 5697 ms, sort 115 ms, write 12278 ms, index 704 ms: 17 candidates, 78365 repo-hours, 1008 releases; waiting 20 ms; commit 2956 ms)
2026-11-08T02:15:19Z [load_day 2023-10-27] 2023-10-27 hour 12: 244942 events in 5 files, snapshot 5381048930605262102 (task 22739 ms: download 14513 ms overlapping gunzip 1290 ms, parse 5948 ms, sort 163 ms, write 14034 ms, index 1071 ms: 9 candidates, 112417 repo-hours, 1157 releases; waiting 19 ms; commit 3399 ms)
2026-11-08T02:15:38Z [load_day 2023-10-27] 2023-10-27 hour 13: 221031 events in 5 files, snapshot 5862315215014308799 (task 23821 ms: download 15855 ms overlapping gunzip 1501 ms, parse 6744 ms, sort 185 ms, write 14477 ms, index 694 ms: 11 candidates, 108812 repo-hours, 1096 releases; waiting 24 ms; commit 5427 ms)
2026-11-08T02:15:43Z [load_day 2023-10-27] 2023-10-27 hour 14: 214076 events in 5 files, snapshot 720685357314063123 (task 23844 ms: download 15608 ms overlapping gunzip 1290 ms, parse 5914 ms, sort 225 ms, write 15221 ms, index 957 ms: 11 candidates, 94225 repo-hours, 1219 releases; waiting 24 ms; commit 2741 ms)
2026-11-08T02:15:59Z [load_day 2023-10-27] 2023-10-27 hour 15: 207742 events in 5 files, snapshot 4949181325105284151 (task 22483 ms: download 12020 ms overlapping gunzip 1426 ms, parse 6324 ms, sort 218 ms, write 13596 ms, index 697 ms: 7 candidates, 92651 repo-hours, 1384 releases; waiting 24 ms; commit 2644 ms)
2026-11-08T02:16:06Z [load_day 2023-10-27] 2023-10-27 hour 16: 195520 events in 4 files, snapshot 1512131132101970492 (task 20898 ms: download 11897 ms overlapping gunzip 1295 ms, parse 5831 ms, sort 168 ms, write 12555 ms, index 831 ms: 9 candidates, 85623 repo-hours, 1024 releases; waiting 21 ms; commit 2967 ms)
2026-11-08T02:16:25Z [load_day 2023-10-27] 2023-10-27 hour 17: 185785 events in 4 files, snapshot 1825691578784140085 (task 20074 ms: download 11488 ms overlapping gunzip 1045 ms, parse 4684 ms, sort 173 ms, write 12564 ms, index 761 ms: 6 candidates, 78200 repo-hours, 932 releases; waiting 522 ms; commit 8328 ms)
2026-11-08T02:16:28Z [load_day 2023-10-27] 2023-10-27 hour 18: 185347 events in 4 files, snapshot 6105886388901123379 (task 18305 ms: download 10407 ms overlapping gunzip 1028 ms, parse 4542 ms, sort 107 ms, write 11350 ms, index 1064 ms: 10 candidates, 82943 repo-hours, 995 releases; waiting 19 ms; commit 3021 ms)
2026-11-08T02:16:45Z [load_day 2023-10-27] 2023-10-27 hour 19: 169902 events in 4 files, snapshot 5552343014637631615 (task 19206 ms: download 10083 ms overlapping gunzip 927 ms, parse 4263 ms, sort 107 ms, write 12579 ms, index 640 ms: 10 candidates, 70339 repo-hours, 937 releases; waiting 22 ms; commit 5298 ms)
2026-11-08T02:16:50Z [load_day 2023-10-27] 2023-10-27 hour 20: 160986 events in 4 files, snapshot 2275928744007039814 (task 17667 ms: download 10951 ms overlapping gunzip 1083 ms, parse 4932 ms, sort 139 ms, write 10618 ms, index 667 ms: 23 candidates, 67310 repo-hours, 864 releases; waiting 16 ms; commit 4910 ms)
2026-11-08T02:16:58Z [load_day 2023-10-27] 2023-10-27 hour 21: 146117 events in 3 files, snapshot 6187418563154800731 (task 14920 ms: download 6717 ms overlapping gunzip 1005 ms, parse 4659 ms, sort 159 ms, write 8420 ms, index 462 ms: 11 candidates, 59662 repo-hours, 895 releases; waiting 21 ms; commit 2684 ms)
2026-11-08T02:17:01Z [load_day 2023-10-27] 2023-10-27 hour 22: 130763 events in 3 files, snapshot 896710511858838360 (task 12290 ms: download 6112 ms overlapping gunzip 893 ms, parse 3852 ms, sort 104 ms, write 6766 ms, index 401 ms: 6 candidates, 52707 repo-hours, 812 releases; waiting 31 ms; commit 2242 ms)
2026-11-08T02:17:08Z [load_day 2023-10-27] 2023-10-27 hour 23: 118009 events in 3 files, snapshot 6440898691111213577 (task 9348 ms: download 4252 ms overlapping gunzip 548 ms, parse 2380 ms, sort 78 ms, write 5201 ms, index 367 ms: 6 candidates, 48100 repo-hours, 703 releases; waiting 519 ms; commit 2555 ms)
2026-11-08T02:17:17Z [load_day 2023-10-27] 2023-10-27: repo_hour merged from 24 files to 1 (1869346 rows, 8314 ms)
2026-11-08T02:17:20Z [load_day 2023-10-27] 2023-10-27: index committed in 11145 ms (24 + 1 + 24 files)
2026-11-08T02:17:20Z [load_day 2023-10-27] 2023-10-27: 4192539 events in 24 hours
2026-11-08T02:17:20Z [load_day 2023-10-27] attempt 1 succeeded