orcaLog in

Runs · run 1522

load_day 2023-09-19, attempt 1 succeeded

Host
3c2b29713ce8:344855
Started
2026-11-08T00:57:35Z
Ended
2026-11-08T01:01:47Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1522/load_day/2023-09-19/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:57:35Z [load_day 2023-09-19] attempt 1 started
2026-11-08T00:57:37Z [load_day 2023-09-19] 2023-09-19: 2 hours in flight, about 1200 MB
2026-11-08T00:58:00Z [load_day 2023-09-19] 2023-09-19 hour 0: 174960 events in 4 files, snapshot 3684564148179349275 (task 17398 ms: download 8017 ms overlapping gunzip 1081 ms, parse 5293 ms, sort 140 ms, write 9468 ms, index 602 ms: 24 candidates, 93161 repo-hours, 897 releases; waiting 511 ms; commit 4622 ms)
2026-11-08T00:58:03Z [load_day 2023-09-19] 2023-09-19 hour 1: 169227 events in 4 files, snapshot 460211237927153397 (task 16471 ms: download 6298 ms overlapping gunzip 949 ms, parse 4353 ms, sort 181 ms, write 9501 ms, index 681 ms: 10 candidates, 94119 repo-hours, 944 releases; waiting 487 ms; commit 2837 ms)
2026-11-08T00:58:13Z [load_day 2023-09-19] 2023-09-19 hour 2: 141712 events in 3 files, snapshot 4844395229071899490 (task 13225 ms: download 5361 ms overlapping gunzip 815 ms, parse 3522 ms, sort 102 ms, write 7690 ms, index 563 ms: 10 candidates, 62714 repo-hours, 803 releases; waiting 286 ms; commit 2603 ms)
2026-11-08T00:58:19Z [load_day 2023-09-19] 2023-09-19 hour 3: 139732 events in 3 files, snapshot 6195564505854414200 (task 13266 ms: download 5506 ms overlapping gunzip 802 ms, parse 3467 ms, sort 105 ms, write 7443 ms, index 677 ms: 9 candidates, 60966 repo-hours, 830 releases; waiting 468 ms; commit 4026 ms)
2026-11-08T00:58:28Z [load_day 2023-09-19] 2023-09-19 hour 4: 131895 events in 3 files, snapshot 4222174822523200211 (task 12615 ms: download 5322 ms overlapping gunzip 718 ms, parse 3022 ms, sort 137 ms, write 6875 ms, index 391 ms: 9 candidates, 55802 repo-hours, 609 releases; waiting 1039 ms; commit 3872 ms)
2026-11-08T00:58:34Z [load_day 2023-09-19] 2023-09-19 hour 5: 146460 events in 3 files, snapshot 4779098462927681873 (task 15753 ms: download 6121 ms overlapping gunzip 1035 ms, parse 4474 ms, sort 139 ms, write 8713 ms, index 613 ms: 9 candidates, 62795 repo-hours, 731 releases; waiting 531 ms; commit 2670 ms)
2026-11-08T00:58:46Z [load_day 2023-09-19] 2023-09-19 hour 6: 171685 events in 4 files, snapshot 3542236053936285564 (task 16744 ms: download 9851 ms overlapping gunzip 996 ms, parse 4388 ms, sort 171 ms, write 9948 ms, index 576 ms: 12 candidates, 79664 repo-hours, 998 releases; waiting 434 ms; commit 3839 ms)
2026-11-08T00:58:56Z [load_day 2023-09-19] 2023-09-19 hour 7: 184958 events in 4 files, snapshot 145958330469157325 (task 20159 ms: download 10209 ms overlapping gunzip 1237 ms, parse 5378 ms, sort 211 ms, write 11475 ms, index 959 ms: 13 candidates, 82587 repo-hours, 1063 releases; waiting 473 ms; commit 3010 ms)
2026-11-08T00:59:08Z [load_day 2023-09-19] 2023-09-19 hour 8: 202065 events in 5 files, snapshot 7034200622755726128 (task 22924 ms: download 12772 ms overlapping gunzip 1484 ms, parse 6650 ms, sort 236 ms, write 13142 ms, index 607 ms: 15 candidates, 88761 repo-hours, 1031 releases; waiting 581 ms; commit 2739 ms)
2026-11-08T00:59:17Z [load_day 2023-09-19] 2023-09-19 hour 9: 200140 events in 5 files, snapshot 97364745102751440 (task 20295 ms: download 11077 ms overlapping gunzip 1091 ms, parse 4894 ms, sort 161 ms, write 12653 ms, index 826 ms: 7 candidates, 88015 repo-hours, 1255 releases; waiting 382 ms; commit 2856 ms)
2026-11-08T00:59:30Z [load_day 2023-09-19] 2023-09-19 hour 10: 192218 events in 4 files, snapshot 1669893461594818339 (task 19742 ms: download 10289 ms overlapping gunzip 1313 ms, parse 5682 ms, sort 214 ms, write 11030 ms, index 574 ms: 11 candidates, 83231 repo-hours, 1072 releases; waiting 573 ms; commit 3073 ms)
2026-11-08T00:59:39Z [load_day 2023-09-19] 2023-09-19 hour 11: 193282 events in 4 files, snapshot 8930633174082278867 (task 20512 ms: download 11316 ms overlapping gunzip 1164 ms, parse 5083 ms, sort 180 ms, write 12575 ms, index 734 ms: 13 candidates, 83451 repo-hours, 1134 releases; waiting 479 ms; commit 2943 ms)
2026-11-08T00:59:55Z [load_day 2023-09-19] 2023-09-19 hour 12: 225731 events in 5 files, snapshot 1416341767240915269 (task 21595 ms: download 11244 ms overlapping gunzip 1278 ms, parse 5761 ms, sort 227 ms, write 12678 ms, index 811 ms: 19 candidates, 111862 repo-hours, 1127 releases; waiting 507 ms; commit 5412 ms)
2026-11-08T00:59:57Z [load_day 2023-09-19] 2023-09-19 hour 13: 157359 events in 4 files, snapshot 1616143005685404871 (task 18004 ms: download 7452 ms overlapping gunzip 980 ms, parse 4218 ms, sort 107 ms, write 11051 ms, index 795 ms: 14 candidates, 83233 repo-hours, 777 releases; waiting 530 ms; commit 2332 ms)
2026-11-08T01:00:18Z [load_day 2023-09-19] 2023-09-19 hour 14: 206162 events in 5 files, snapshot 6462991851341319748 (task 24655 ms: download 14100 ms overlapping gunzip 1395 ms, parse 6328 ms, sort 132 ms, write 15630 ms, index 693 ms: 10 candidates, 95550 repo-hours, 1096 releases; waiting 192 ms; commit 2692 ms)
2026-11-08T01:00:22Z [load_day 2023-09-19] 2023-09-19 hour 15: 180171 events in 4 files, snapshot 8765748580569842397 (task 21075 ms: download 13596 ms overlapping gunzip 1134 ms, parse 5157 ms, sort 110 ms, write 13227 ms, index 604 ms: 7 candidates, 87530 repo-hours, 1204 releases; waiting 523 ms; commit 3361 ms)
2026-11-08T01:00:36Z [load_day 2023-09-19] 2023-09-19 hour 16: 182773 events in 4 files, snapshot 1411757304490261441 (task 16654 ms: download 9789 ms overlapping gunzip 1017 ms, parse 4441 ms, sort 102 ms, write 9665 ms, index 610 ms: 15 candidates, 85288 repo-hours, 1075 releases; waiting 500 ms; commit 2658 ms)
2026-11-08T01:00:39Z [load_day 2023-09-19] 2023-09-19 hour 17: 182278 events in 4 files, snapshot 2444803532058525507 (task 16530 ms: download 9394 ms overlapping gunzip 932 ms, parse 3941 ms, sort 108 ms, write 9236 ms, index 986 ms: 13 candidates, 81120 repo-hours, 1020 releases; waiting 979 ms; commit 3357 ms)
2026-11-08T01:00:58Z [load_day 2023-09-19] 2023-09-19 hour 18: 191522 events in 4 files, snapshot 8274445376908224606 (task 19058 ms: download 11423 ms overlapping gunzip 1140 ms, parse 5340 ms, sort 121 ms, write 10723 ms, index 699 ms: 11 candidates, 87825 repo-hours, 1067 releases; waiting 518 ms; commit 3978 ms)
2026-11-08T01:01:00Z [load_day 2023-09-19] 2023-09-19 hour 19: 179927 events in 4 files, snapshot 1677912993146433915 (task 18988 ms: download 11304 ms overlapping gunzip 1169 ms, parse 5076 ms, sort 105 ms, write 11320 ms, index 847 ms: 7 candidates, 76994 repo-hours, 1011 releases; waiting 196 ms; commit 2843 ms)
2026-11-08T01:01:17Z [load_day 2023-09-19] 2023-09-19 hour 20: 178369 events in 4 files, snapshot 992864493127818414 (task 18668 ms: download 10938 ms overlapping gunzip 1280 ms, parse 5569 ms, sort 137 ms, write 10262 ms, index 575 ms: 9 candidates, 75911 repo-hours, 1058 releases; waiting 523 ms; commit 3913 ms)
2026-11-08T01:01:23Z [load_day 2023-09-19] 2023-09-19 hour 21: 161228 events in 4 files, snapshot 956319300745913858 (task 16568 ms: download 6979 ms overlapping gunzip 949 ms, parse 4182 ms, sort 101 ms, write 9937 ms, index 649 ms: 22 candidates, 67120 repo-hours, 978 releases; waiting 514 ms; commit 5317 ms)
2026-11-08T01:01:34Z [load_day 2023-09-19] 2023-09-19 hour 22: 143442 events in 3 files, snapshot 3438848404555018713 (task 15393 ms: download 6313 ms overlapping gunzip 938 ms, parse 4030 ms, sort 156 ms, write 8840 ms, index 576 ms: 27 candidates, 58424 repo-hours, 895 releases; waiting 526 ms; commit 4222 ms)
2026-11-08T01:01:36Z [load_day 2023-09-19] 2023-09-19 hour 23: 133170 events in 3 files, snapshot 4320071617406900143 (task 14141 ms: download 6301 ms overlapping gunzip 843 ms, parse 3671 ms, sort 87 ms, write 8252 ms, index 438 ms: 14 candidates, 55822 repo-hours, 678 releases; waiting 490 ms; commit 2111 ms)
2026-11-08T01:01:44Z [load_day 2023-09-19] 2023-09-19: repo_hour merged from 24 files to 1 (1901945 rows, 8382 ms)
2026-11-08T01:01:47Z [load_day 2023-09-19] 2023-09-19: index committed in 10545 ms (24 + 1 + 24 files)
2026-11-08T01:01:47Z [load_day 2023-09-19] 2023-09-19: 4170466 events in 24 hours
2026-11-08T01:01:47Z [load_day 2023-09-19] attempt 1 succeeded