orcaLog in

Runs · run 1477

load_day 2023-09-08, attempt 1 succeeded

Host
3c2b29713ce8:308993
Started
2026-11-08T00:38:27Z
Ended
2026-11-08T00:42:18Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1477/load_day/2023-09-08/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:38:27Z [load_day 2023-09-08] attempt 1 started
2026-11-08T00:38:29Z [load_day 2023-09-08] 2023-09-08: 2 hours in flight, about 1200 MB
2026-11-08T00:38:47Z [load_day 2023-09-08] 2023-09-08 hour 0: 164400 events in 4 files, snapshot 5375029715130982025 (task 14387 ms: download 5893 ms overlapping gunzip 777 ms, parse 3664 ms, sort 102 ms, write 8317 ms, index 781 ms: 8 candidates, 89310 repo-hours, 961 releases; waiting 511 ms; commit 2402 ms)
2026-11-08T00:38:50Z [load_day 2023-09-08] 2023-09-08 hour 1: 153631 events in 4 files, snapshot 8085798601725280543 (task 13534 ms: download 5301 ms overlapping gunzip 648 ms, parse 3214 ms, sort 126 ms, write 8100 ms, index 645 ms: 11 candidates, 85635 repo-hours, 827 releases; waiting 509 ms; commit 2742 ms)
2026-11-08T00:39:06Z [load_day 2023-09-08] 2023-09-08 hour 2: 139112 events in 3 files, snapshot 5827201240117164965 (task 13272 ms: download 6329 ms overlapping gunzip 625 ms, parse 2808 ms, sort 104 ms, write 7577 ms, index 417 ms: 7 candidates, 58288 repo-hours, 771 releases; waiting 1469 ms; commit 6860 ms)
2026-11-08T00:39:09Z [load_day 2023-09-08] 2023-09-08 hour 3: 131189 events in 3 files, snapshot 3326211384140817477 (task 10827 ms: download 4659 ms overlapping gunzip 746 ms, parse 3140 ms, sort 143 ms, write 5529 ms, index 432 ms: 7 candidates, 55623 repo-hours, 870 releases; waiting 526 ms; commit 2445 ms)
2026-11-08T00:39:15Z [load_day 2023-09-08] 2023-09-08 hour 4: 129880 events in 3 files, snapshot 8831943420352420305 (task 11790 ms: download 5026 ms overlapping gunzip 716 ms, parse 2962 ms, sort 97 ms, write 6893 ms, index 369 ms: 10 candidates, 51809 repo-hours, 565 releases; waiting 537 ms; commit 2430 ms)
2026-11-08T00:39:25Z [load_day 2023-09-08] 2023-09-08 hour 5: 143097 events in 3 files, snapshot 8696293439635395464 (task 14076 ms: download 5580 ms overlapping gunzip 778 ms, parse 3336 ms, sort 149 ms, write 8493 ms, index 499 ms: 10 candidates, 59138 repo-hours, 652 releases; waiting 495 ms; commit 3739 ms)
2026-11-08T00:39:32Z [load_day 2023-09-08] 2023-09-08 hour 6: 169420 events in 4 files, snapshot 3610698559468667425 (task 16179 ms: download 8788 ms overlapping gunzip 856 ms, parse 3726 ms, sort 141 ms, write 9574 ms, index 742 ms: 9 candidates, 75236 repo-hours, 901 releases; waiting 843 ms; commit 2647 ms)
2026-11-08T00:39:43Z [load_day 2023-09-08] 2023-09-08 hour 7: 173442 events in 4 files, snapshot 4177179242481192337 (task 17884 ms: download 10572 ms overlapping gunzip 1120 ms, parse 4871 ms, sort 157 ms, write 10350 ms, index 669 ms: 5 candidates, 76069 repo-hours, 832 releases; waiting 497 ms; commit 3242 ms)
2026-11-08T00:39:52Z [load_day 2023-09-08] 2023-09-08 hour 8: 183439 events in 4 files, snapshot 4387880074248103943 (task 19166 ms: download 10921 ms overlapping gunzip 1060 ms, parse 4616 ms, sort 142 ms, write 11981 ms, index 562 ms: 12 candidates, 83194 repo-hours, 1000 releases; waiting 495 ms; commit 2216 ms)
2026-11-08T00:40:02Z [load_day 2023-09-08] 2023-09-08 hour 9: 194882 events in 4 files, snapshot 2208020585755610068 (task 17770 ms: download 10253 ms overlapping gunzip 1141 ms, parse 5025 ms, sort 141 ms, write 10041 ms, index 586 ms: 8 candidates, 86638 repo-hours, 1214 releases; waiting 497 ms; commit 2537 ms)
2026-11-08T00:40:08Z [load_day 2023-09-08] 2023-09-08 hour 10: 181654 events in 4 files, snapshot 2026309187223364305 (task 15609 ms: download 8414 ms overlapping gunzip 861 ms, parse 3790 ms, sort 119 ms, write 9126 ms, index 543 ms: 13 candidates, 77943 repo-hours, 1105 releases; waiting 654 ms; commit 2309 ms)
2026-11-08T00:40:19Z [load_day 2023-09-08] 2023-09-08 hour 11: 175906 events in 4 files, snapshot 8888693671326859958 (task 15705 ms: download 8293 ms overlapping gunzip 947 ms, parse 4176 ms, sort 119 ms, write 9158 ms, index 519 ms: 16 candidates, 73367 repo-hours, 1096 releases; waiting 510 ms; commit 2868 ms)
2026-11-08T00:40:31Z [load_day 2023-09-08] 2023-09-08 hour 12: 235632 events in 5 files, snapshot 6290380516173130942 (task 20067 ms: download 10049 ms overlapping gunzip 1040 ms, parse 4855 ms, sort 195 ms, write 12236 ms, index 890 ms: 8 candidates, 107516 repo-hours, 1394 releases; waiting 527 ms; commit 3462 ms)
2026-11-08T00:40:40Z [load_day 2023-09-08] 2023-09-08 hour 13: 212272 events in 5 files, snapshot 3215124615788494694 (task 20073 ms: download 12669 ms overlapping gunzip 1253 ms, parse 5682 ms, sort 138 ms, write 11419 ms, index 827 ms: 11 candidates, 102325 repo-hours, 1099 releases; waiting 478 ms; commit 3089 ms)
2026-11-08T00:40:54Z [load_day 2023-09-08] 2023-09-08 hour 14: 205561 events in 5 files, snapshot 6496181754221333705 (task 21847 ms: download 11800 ms overlapping gunzip 1385 ms, parse 6195 ms, sort 135 ms, write 12719 ms, index 595 ms: 16 candidates, 87817 repo-hours, 1225 releases; waiting 550 ms; commit 4016 ms)
2026-11-08T00:41:00Z [load_day 2023-09-08] 2023-09-08 hour 15: 192180 events in 4 files, snapshot 3081614295329838195 (task 19885 ms: download 11149 ms overlapping gunzip 1093 ms, parse 4713 ms, sort 150 ms, write 12601 ms, index 666 ms: 9 candidates, 83375 repo-hours, 1269 releases; waiting 344 ms; commit 2727 ms)
2026-11-08T00:41:15Z [load_day 2023-09-08] 2023-09-08 hour 16: 190164 events in 4 files, snapshot 7948657379021969551 (task 19674 ms: download 11147 ms overlapping gunzip 1203 ms, parse 5384 ms, sort 127 ms, write 11498 ms, index 632 ms: 12 candidates, 81458 repo-hours, 995 releases; waiting 544 ms; commit 3825 ms)
2026-11-08T00:41:20Z [load_day 2023-09-08] 2023-09-08 hour 17: 177955 events in 4 files, snapshot 2443628331878816821 (task 17828 ms: download 10402 ms overlapping gunzip 1073 ms, parse 4683 ms, sort 109 ms, write 10568 ms, index 582 ms: 4 candidates, 73746 repo-hours, 976 releases; waiting 545 ms; commit 3014 ms)
2026-11-08T00:41:36Z [load_day 2023-09-08] 2023-09-08 hour 18: 180968 events in 4 files, snapshot 2434614931377868784 (task 19123 ms: download 10983 ms overlapping gunzip 1119 ms, parse 5041 ms, sort 177 ms, write 11294 ms, index 644 ms: 6 candidates, 76768 repo-hours, 973 releases; waiting 533 ms; commit 4663 ms)
2026-11-08T00:41:38Z [load_day 2023-09-08] 2023-09-08 hour 19: 165202 events in 4 files, snapshot 1135519952141855560 (task 18127 ms: download 10814 ms overlapping gunzip 1149 ms, parse 4890 ms, sort 210 ms, write 10424 ms, index 689 ms: 11 candidates, 65433 repo-hours, 980 releases; waiting 552 ms; commit 2611 ms)
2026-11-08T00:41:53Z [load_day 2023-09-08] 2023-09-08 hour 20: 155110 events in 4 files, snapshot 1672678782083493436 (task 16505 ms: download 6933 ms overlapping gunzip 1068 ms, parse 4578 ms, sort 130 ms, write 9605 ms, index 576 ms: 10 candidates, 62989 repo-hours, 856 releases; waiting 232 ms; commit 3753 ms)
2026-11-08T00:41:56Z [load_day 2023-09-08] 2023-09-08 hour 21: 148722 events in 3 files, snapshot 419554317282769782 (task 15940 ms: download 7294 ms overlapping gunzip 1064 ms, parse 4267 ms, sort 136 ms, write 9099 ms, index 606 ms: 7 candidates, 58349 repo-hours, 888 releases; waiting 483 ms; commit 2684 ms)
2026-11-08T00:42:05Z [load_day 2023-09-08] 2023-09-08 hour 22: 132316 events in 3 files, snapshot 479124450943395314 (task 13170 ms: download 5865 ms overlapping gunzip 911 ms, parse 3746 ms, sort 93 ms, write 7246 ms, index 359 ms: 9 candidates, 51702 repo-hours, 780 releases; waiting 512 ms; commit 2047 ms)
2026-11-08T00:42:08Z [load_day 2023-09-08] 2023-09-08 hour 23: 121417 events in 3 files, snapshot 2260926632831035377 (task 12158 ms: download 5968 ms overlapping gunzip 711 ms, parse 3050 ms, sort 108 ms, write 6439 ms, index 310 ms: 5 candidates, 48764 repo-hours, 731 releases; waiting 1239 ms; commit 2108 ms)
2026-11-08T00:42:16Z [load_day 2023-09-08] 2023-09-08: repo_hour merged from 24 files to 1 (1772492 rows, 7658 ms)
2026-11-08T00:42:18Z [load_day 2023-09-08] 2023-09-08: index committed in 9718 ms (24 + 1 + 24 files)
2026-11-08T00:42:18Z [load_day 2023-09-08] 2023-09-08: 4057551 events in 24 hours
2026-11-08T00:42:18Z [load_day 2023-09-08] attempt 1 succeeded