orcaLog in

Runs · run 1372

load_day 2023-08-08, attempt 1 succeeded

Host
3c2b29713ce8:204580
Started
2026-11-07T23:41:28Z
Ended
2026-11-07T23:45:26Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1372/load_day/2023-08-08/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T23:41:28Z [load_day 2023-08-08] attempt 1 started
2026-11-07T23:41:29Z [load_day 2023-08-08] 2023-08-08: 2 hours in flight, about 1200 MB
2026-11-07T23:41:48Z [load_day 2023-08-08] 2023-08-08 hour 0: 162013 events in 4 files, snapshot 4002789100631757105 (task 15769 ms: download 7336 ms overlapping gunzip 960 ms, parse 4357 ms, sort 178 ms, write 8892 ms, index 522 ms: 5 candidates, 78827 repo-hours, 917 releases; waiting 522 ms; commit 2067 ms)
2026-11-07T23:41:51Z [load_day 2023-08-08] 2023-08-08 hour 1: 167732 events in 4 files, snapshot 9177357454039235238 (task 15709 ms: download 6542 ms overlapping gunzip 986 ms, parse 4218 ms, sort 98 ms, write 8949 ms, index 604 ms: 3 candidates, 90295 repo-hours, 806 releases; waiting 498 ms; commit 3023 ms)
2026-11-07T23:42:03Z [load_day 2023-08-08] 2023-08-08 hour 2: 145555 events in 3 files, snapshot 781568836604645423 (task 13718 ms: download 6201 ms overlapping gunzip 759 ms, parse 3193 ms, sort 147 ms, write 8148 ms, index 485 ms: 4 candidates, 63961 repo-hours, 823 releases; waiting 632 ms; commit 2439 ms)
2026-11-07T23:42:08Z [load_day 2023-08-08] 2023-08-08 hour 3: 135555 events in 3 files, snapshot 951365861928216700 (task 12553 ms: download 5539 ms overlapping gunzip 866 ms, parse 3715 ms, sort 89 ms, write 6609 ms, index 427 ms: 1 candidates, 58057 repo-hours, 792 releases; waiting 510 ms; commit 4458 ms)
2026-11-07T23:42:17Z [load_day 2023-08-08] 2023-08-08 hour 4: 130794 events in 3 files, snapshot 6308373549632515958 (task 12659 ms: download 5393 ms overlapping gunzip 777 ms, parse 3240 ms, sort 82 ms, write 7289 ms, index 452 ms: 2 candidates, 55028 repo-hours, 636 releases; waiting 513 ms; commit 2462 ms)
2026-11-07T23:42:22Z [load_day 2023-08-08] 2023-08-08 hour 5: 149768 events in 3 files, snapshot 1748290615508550383 (task 15223 ms: download 6545 ms overlapping gunzip 981 ms, parse 4252 ms, sort 180 ms, write 8561 ms, index 492 ms: 4 candidates, 68168 repo-hours, 798 releases; waiting 464 ms; commit 2448 ms)
2026-11-07T23:42:34Z [load_day 2023-08-08] 2023-08-08 hour 6: 174648 events in 4 files, snapshot 7785385715539203906 (task 16358 ms: download 9299 ms overlapping gunzip 988 ms, parse 4368 ms, sort 116 ms, write 9390 ms, index 672 ms: 11 candidates, 83414 repo-hours, 940 releases; waiting 484 ms; commit 2168 ms)
2026-11-07T23:42:41Z [load_day 2023-08-08] 2023-08-08 hour 7: 173998 events in 4 files, snapshot 5099519698124396156 (task 18351 ms: download 10045 ms overlapping gunzip 1170 ms, parse 5119 ms, sort 136 ms, write 10353 ms, index 634 ms: 4 candidates, 76685 repo-hours, 887 releases; waiting 474 ms; commit 2100 ms)
2026-11-07T23:42:58Z [load_day 2023-08-08] 2023-08-08 hour 8: 187769 events in 4 files, snapshot 701334496274357099 (task 22028 ms: download 13083 ms overlapping gunzip 1423 ms, parse 6317 ms, sort 201 ms, write 12634 ms, index 713 ms: 6 candidates, 83487 repo-hours, 1015 releases; waiting 466 ms; commit 2744 ms)
2026-11-07T23:43:02Z [load_day 2023-08-08] 2023-08-08 hour 9: 187309 events in 4 files, snapshot 2644315948457595870 (task 20287 ms: download 11286 ms overlapping gunzip 1092 ms, parse 4838 ms, sort 157 ms, write 12488 ms, index 900 ms: 6 candidates, 82754 repo-hours, 1218 releases; waiting 489 ms; commit 2872 ms)
2026-11-07T23:43:18Z [load_day 2023-08-08] 2023-08-08 hour 10: 184500 events in 4 files, snapshot 9038059127290281596 (task 19215 ms: download 11437 ms overlapping gunzip 1096 ms, parse 4921 ms, sort 153 ms, write 11662 ms, index 578 ms: 6 candidates, 79775 repo-hours, 1072 releases; waiting 470 ms; commit 2273 ms)
2026-11-07T23:43:22Z [load_day 2023-08-08] 2023-08-08 hour 11: 179065 events in 4 files, snapshot 1565196097275540907 (task 17900 ms: download 10311 ms overlapping gunzip 1053 ms, parse 4607 ms, sort 111 ms, write 10463 ms, index 808 ms: 3 candidates, 76218 repo-hours, 1085 releases; waiting 510 ms; commit 2829 ms)
2026-11-07T23:43:41Z [load_day 2023-08-08] 2023-08-08 hour 12: 221540 events in 5 files, snapshot 6534480699598858656 (task 20679 ms: download 11045 ms overlapping gunzip 1169 ms, parse 5165 ms, sort 129 ms, write 12401 ms, index 956 ms: 5 candidates, 106609 repo-hours, 1064 releases; waiting 526 ms; commit 3560 ms)
2026-11-07T23:43:43Z [load_day 2023-08-08] 2023-08-08 hour 13: 197326 events in 4 files, snapshot 4031753264587531338 (task 20872 ms: download 10944 ms overlapping gunzip 1139 ms, parse 5305 ms, sort 127 ms, write 12229 ms, index 1240 ms: 6 candidates, 98477 repo-hours, 977 releases; waiting 503 ms; commit 2418 ms)
2026-11-07T23:44:02Z [load_day 2023-08-08] 2023-08-08 hour 14: 182976 events in 4 files, snapshot 7619478382199820041 (task 21331 ms: download 12913 ms overlapping gunzip 1317 ms, parse 6054 ms, sort 242 ms, write 12127 ms, index 768 ms: 3 candidates, 85475 repo-hours, 1020 releases; waiting 515 ms; commit 2474 ms)
2026-11-07T23:44:04Z [load_day 2023-08-08] 2023-08-08 hour 15: 170498 events in 4 files, snapshot 7020002522977818849 (task 17854 ms: download 10853 ms overlapping gunzip 950 ms, parse 4133 ms, sort 112 ms, write 11273 ms, index 590 ms: 2 candidates, 80365 repo-hours, 1029 releases; waiting 484 ms; commit 2731 ms)
2026-11-07T23:44:20Z [load_day 2023-08-08] 2023-08-08 hour 16: 174433 events in 4 files, snapshot 573459479033547503 (task 17545 ms: download 11357 ms overlapping gunzip 1115 ms, parse 4897 ms, sort 138 ms, write 10208 ms, index 547 ms: 10 candidates, 80942 repo-hours, 956 releases; waiting 347 ms; commit 2073 ms)
2026-11-07T23:44:22Z [load_day 2023-08-08] 2023-08-08 hour 17: 169431 events in 4 files, snapshot 281373990380376154 (task 15966 ms: download 10065 ms overlapping gunzip 1016 ms, parse 4450 ms, sort 123 ms, write 9130 ms, index 505 ms: 3 candidates, 75210 repo-hours, 904 releases; waiting 469 ms; commit 2238 ms)
2026-11-07T23:44:38Z [load_day 2023-08-08] 2023-08-08 hour 18: 185454 events in 4 files, snapshot 578918888676071726 (task 17050 ms: download 8464 ms overlapping gunzip 942 ms, parse 4067 ms, sort 122 ms, write 10328 ms, index 778 ms: 4 candidates, 82992 repo-hours, 1090 releases; waiting 505 ms; commit 2076 ms)
2026-11-07T23:44:40Z [load_day 2023-08-08] 2023-08-08 hour 19: 174135 events in 4 files, snapshot 3306756721670273356 (task 16626 ms: download 9354 ms overlapping gunzip 960 ms, parse 4259 ms, sort 160 ms, write 9909 ms, index 480 ms: 3 candidates, 72476 repo-hours, 974 releases; waiting 522 ms; commit 2746 ms)
2026-11-07T23:44:57Z [load_day 2023-08-08] 2023-08-08 hour 20: 175603 events in 4 files, snapshot 1000061083637413468 (task 18968 ms: download 11534 ms overlapping gunzip 1036 ms, parse 4546 ms, sort 164 ms, write 11476 ms, index 659 ms: 5 candidates, 72094 repo-hours, 870 releases; waiting 529 ms; commit 1991 ms)
2026-11-07T23:45:02Z [load_day 2023-08-08] 2023-08-08 hour 21: 160341 events in 4 files, snapshot 4607853928491065832 (task 16359 ms: download 9745 ms overlapping gunzip 1004 ms, parse 4202 ms, sort 132 ms, write 9851 ms, index 681 ms: 5 candidates, 64944 repo-hours, 994 releases; waiting 194 ms; commit 4509 ms)
2026-11-07T23:45:14Z [load_day 2023-08-08] 2023-08-08 hour 22: 145905 events in 3 files, snapshot 2790187519778928587 (task 14849 ms: download 6746 ms overlapping gunzip 813 ms, parse 3423 ms, sort 88 ms, write 9484 ms, index 512 ms: 7 candidates, 60729 repo-hours, 822 releases; waiting 294 ms; commit 3140 ms)
2026-11-07T23:45:16Z [load_day 2023-08-08] 2023-08-08 hour 23: 136531 events in 3 files, snapshot 327343969736784860 (task 13309 ms: download 6312 ms overlapping gunzip 821 ms, parse 3671 ms, sort 121 ms, write 7546 ms, index 410 ms: 4 candidates, 55727 repo-hours, 676 releases; waiting 469 ms; commit 2065 ms)
2026-11-07T23:45:25Z [load_day 2023-08-08] 2023-08-08: repo_hour merged from 24 files to 1 (1832709 rows, 8095 ms)
2026-11-07T23:45:26Z [load_day 2023-08-08] 2023-08-08: index committed in 10010 ms (24 + 1 + 24 files)
2026-11-07T23:45:26Z [load_day 2023-08-08] 2023-08-08: 4072879 events in 24 hours
2026-11-07T23:45:26Z [load_day 2023-08-08] attempt 1 succeeded