orcaLog in

Runs · run 1797

load_day 2023-11-30, attempt 1 succeeded

Host
3c2b29713ce8:614098
Started
2026-11-08T03:19:29Z
Ended
2026-11-08T03:24:07Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1797/load_day/2023-11-30/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:19:29Z [load_day 2023-11-30] attempt 1 started
2026-11-08T03:19:31Z [load_day 2023-11-30] 2023-11-30: 2 hours in flight, about 1200 MB
2026-11-08T03:19:55Z [load_day 2023-11-30] 2023-11-30 hour 0: 275487 events in 6 files, snapshot 8115674170382261217 (task 20043 ms: download 12767 ms overlapping gunzip 1250 ms, parse 5880 ms, sort 221 ms, write 11310 ms, index 600 ms: 11 candidates, 86535 repo-hours, 981 releases; waiting 516 ms; commit 2623 ms)
2026-11-08T03:19:58Z [load_day 2023-11-30] 2023-11-30 hour 1: 271435 events in 6 files, snapshot 7443852970477337300 (task 17925 ms: download 10706 ms overlapping gunzip 1031 ms, parse 5047 ms, sort 260 ms, write 10223 ms, index 654 ms: 3 candidates, 85955 repo-hours, 681 releases; waiting 471 ms; commit 3059 ms)
2026-11-08T03:20:16Z [load_day 2023-11-30] 2023-11-30 hour 2: 271139 events in 6 files, snapshot 6095453168231992508 (task 18692 ms: download 11450 ms overlapping gunzip 1092 ms, parse 5133 ms, sort 234 ms, write 10864 ms, index 569 ms: 11 candidates, 66438 repo-hours, 767 releases; waiting 507 ms; commit 3991 ms)
2026-11-08T03:20:19Z [load_day 2023-11-30] 2023-11-30 hour 3: 269839 events in 6 files, snapshot 7680512357443302392 (task 17792 ms: download 10820 ms overlapping gunzip 1031 ms, parse 4854 ms, sort 277 ms, write 10103 ms, index 795 ms: 9 candidates, 58567 repo-hours, 858 releases; waiting 476 ms; commit 3482 ms)
2026-11-08T03:20:33Z [load_day 2023-11-30] 2023-11-30 hour 4: 259903 events in 6 files, snapshot 4199692663201298539 (task 15497 ms: download 9182 ms overlapping gunzip 1000 ms, parse 4686 ms, sort 198 ms, write 8365 ms, index 403 ms: 9 candidates, 54794 repo-hours, 637 releases; waiting 549 ms; commit 4258 ms)
2026-11-08T03:20:37Z [load_day 2023-11-30] 2023-11-30 hour 5: 219530 events in 5 files, snapshot 3706951213458417747 (task 16632 ms: download 8298 ms overlapping gunzip 1032 ms, parse 4593 ms, sort 241 ms, write 9534 ms, index 467 ms: 7 candidates, 62962 repo-hours, 966 releases; waiting 517 ms; commit 3228 ms)
2026-11-08T03:20:50Z [load_day 2023-11-30] 2023-11-30 hour 6: 233699 events in 5 files, snapshot 4317313886464349591 (task 17186 ms: download 8580 ms overlapping gunzip 1014 ms, parse 4313 ms, sort 167 ms, write 10371 ms, index 531 ms: 5 candidates, 74281 repo-hours, 1146 releases; waiting 504 ms; commit 3143 ms)
2026-11-08T03:20:58Z [load_day 2023-11-30] 2023-11-30 hour 7: 249805 events in 5 files, snapshot 7217607721756867284 (task 19079 ms: download 11700 ms overlapping gunzip 1198 ms, parse 5573 ms, sort 204 ms, write 10593 ms, index 713 ms: 4 candidates, 73455 repo-hours, 847 releases; waiting 504 ms; commit 3612 ms)
2026-11-08T03:21:14Z [load_day 2023-11-30] 2023-11-30 hour 8: 286413 events in 6 files, snapshot 1652849899861144149 (task 22662 ms: download 13178 ms overlapping gunzip 1387 ms, parse 6304 ms, sort 278 ms, write 13226 ms, index 653 ms: 11 candidates, 78187 repo-hours, 850 releases; waiting 525 ms; commit 3171 ms)
2026-11-08T03:21:20Z [load_day 2023-11-30] 2023-11-30 hour 9: 289059 events in 6 files, snapshot 5937689678624886940 (task 22296 ms: download 15441 ms overlapping gunzip 1311 ms, parse 6102 ms, sort 282 ms, write 13274 ms, index 606 ms: 7 candidates, 77580 repo-hours, 1042 releases; waiting 498 ms; commit 2756 ms)
2026-11-08T03:21:36Z [load_day 2023-11-30] 2023-11-30 hour 10: 312924 events in 7 files, snapshot 3871155513709924120 (task 20751 ms: download 13332 ms overlapping gunzip 1117 ms, parse 5111 ms, sort 238 ms, write 12762 ms, index 754 ms: 8 candidates, 69326 repo-hours, 719 releases; waiting 485 ms; commit 2926 ms)
2026-11-08T03:21:43Z [load_day 2023-11-30] 2023-11-30 hour 11: 309838 events in 7 files, snapshot 1812039197264298948 (task 21996 ms: download 14091 ms overlapping gunzip 1407 ms, parse 6827 ms, sort 372 ms, write 11954 ms, index 632 ms: 12 candidates, 71355 repo-hours, 834 releases; waiting 493 ms; commit 2565 ms)
2026-11-08T03:22:01Z [load_day 2023-11-30] 2023-11-30 hour 12: 294562 events in 6 files, snapshot 682118879553419990 (task 23432 ms: download 13647 ms overlapping gunzip 1317 ms, parse 6119 ms, sort 382 ms, write 14042 ms, index 794 ms: 10 candidates, 106721 repo-hours, 993 releases; waiting 514 ms; commit 2779 ms)
2026-11-08T03:22:08Z [load_day 2023-11-30] 2023-11-30 hour 13: 300399 events in 7 files, snapshot 2556306788098544128 (task 22723 ms: download 14831 ms overlapping gunzip 1275 ms, parse 5872 ms, sort 233 ms, write 13681 ms, index 751 ms: 10 candidates, 94959 repo-hours, 866 releases; waiting 482 ms; commit 3117 ms)
2026-11-08T03:22:23Z [load_day 2023-11-30] 2023-11-30 hour 14: 306654 events in 7 files, snapshot 5901530740635863547 (task 21040 ms: download 14405 ms overlapping gunzip 1146 ms, parse 5364 ms, sort 280 ms, write 12878 ms, index 616 ms: 13 candidates, 80607 repo-hours, 786 releases; waiting 503 ms; commit 2835 ms)
2026-11-08T03:22:28Z [load_day 2023-11-30] 2023-11-30 hour 15: 298321 events in 6 files, snapshot 2064630565344717556 (task 19376 ms: download 12679 ms overlapping gunzip 1138 ms, parse 5067 ms, sort 249 ms, write 11516 ms, index 575 ms: 9 candidates, 85491 repo-hours, 1011 releases; waiting 574 ms; commit 2678 ms)
2026-11-08T03:22:47Z [load_day 2023-11-30] 2023-11-30 hour 16: 293410 events in 6 files, snapshot 5678359423628966896 (task 22792 ms: download 14577 ms overlapping gunzip 1372 ms, parse 6389 ms, sort 248 ms, write 13205 ms, index 813 ms: 15 candidates, 88100 repo-hours, 1209 releases; waiting 493 ms; commit 3328 ms)
2026-11-08T03:22:51Z [load_day 2023-11-30] 2023-11-30 hour 17: 288621 events in 6 files, snapshot 8467224106064375294 (task 22352 ms: download 15168 ms overlapping gunzip 1258 ms, parse 5473 ms, sort 302 ms, write 13618 ms, index 878 ms: 7 candidates, 79992 repo-hours, 882 releases; waiting 528 ms; commit 2471 ms)
2026-11-08T03:23:10Z [load_day 2023-11-30] 2023-11-30 hour 18: 291703 events in 6 files, snapshot 8250466920855297050 (task 21833 ms: download 14654 ms overlapping gunzip 1391 ms, parse 6076 ms, sort 306 ms, write 12659 ms, index 616 ms: 12 candidates, 83821 repo-hours, 852 releases; waiting 498 ms; commit 2858 ms)
2026-11-08T03:23:19Z [load_day 2023-11-30] 2023-11-30 hour 19: 283279 events in 6 files, snapshot 8011604529982387344 (task 21928 ms: download 12710 ms overlapping gunzip 1169 ms, parse 5492 ms, sort 333 ms, write 13489 ms, index 539 ms: 16 candidates, 73077 repo-hours, 856 releases; waiting 481 ms; commit 6705 ms)
2026-11-08T03:23:30Z [load_day 2023-11-30] 2023-11-30 hour 20: 301728 events in 7 files, snapshot 1240216822836523634 (task 19096 ms: download 11108 ms overlapping gunzip 1206 ms, parse 5565 ms, sort 289 ms, write 10808 ms, index 424 ms: 5 candidates, 62856 repo-hours, 652 releases; waiting 517 ms; commit 3022 ms)
2026-11-08T03:23:36Z [load_day 2023-11-30] 2023-11-30 hour 21: 311290 events in 7 files, snapshot 6514193447625015353 (task 19593 ms: download 12750 ms overlapping gunzip 1152 ms, parse 5675 ms, sort 401 ms, write 11037 ms, index 611 ms: 8 candidates, 54651 repo-hours, 598 releases; waiting 470 ms; commit 2961 ms)
2026-11-08T03:23:50Z [load_day 2023-11-30] 2023-11-30 hour 22: 283842 events in 6 files, snapshot 8606823017897262318 (task 19132 ms: download 12209 ms overlapping gunzip 1160 ms, parse 5278 ms, sort 358 ms, write 11402 ms, index 384 ms: 6 candidates, 55622 repo-hours, 722 releases; waiting 315 ms; commit 2402 ms)
2026-11-08T03:23:55Z [load_day 2023-11-30] 2023-11-30 hour 23: 214174 events in 5 files, snapshot 5306493833973820148 (task 17790 ms: download 10317 ms overlapping gunzip 1078 ms, parse 4788 ms, sort 174 ms, write 10463 ms, index 581 ms: 4 candidates, 61845 repo-hours, 807 releases; waiting 491 ms; commit 2298 ms)
2026-11-08T03:24:03Z [load_day 2023-11-30] 2023-11-30: repo_hour merged from 24 files to 1 (1787177 rows, 8412 ms)
2026-11-08T03:24:06Z [load_day 2023-11-30] 2023-11-30: index committed in 11336 ms (24 + 1 + 24 files)
2026-11-08T03:24:06Z [load_day 2023-11-30] 2023-11-30: 6717054 events in 24 hours
2026-11-08T03:24:06Z [load_day 2023-11-30] attempt 1 succeeded