orcaLog in

Runs · run 1527

load_day 2023-09-20, attempt 1 succeeded

Host
3c2b29713ce8:350607
Started
2026-11-08T01:00:31Z
Ended
2026-11-08T01:04:43Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1527/load_day/2023-09-20/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:00:31Z [load_day 2023-09-20] attempt 1 started
2026-11-08T01:00:32Z [load_day 2023-09-20] 2023-09-20: 2 hours in flight, about 1200 MB
2026-11-08T01:00:52Z [load_day 2023-09-20] 2023-09-20 hour 0: 171561 events in 4 files, snapshot 5472119340876932097 (task 15765 ms: download 6863 ms overlapping gunzip 863 ms, parse 4273 ms, sort 135 ms, write 9119 ms, index 610 ms: 20 candidates, 93882 repo-hours, 972 releases; waiting 452 ms; commit 2312 ms)
2026-11-08T01:00:55Z [load_day 2023-09-20] 2023-09-20 hour 1: 160132 events in 4 files, snapshot 3593679591246907483 (task 14714 ms: download 6022 ms overlapping gunzip 724 ms, parse 3245 ms, sort 173 ms, write 8957 ms, index 622 ms: 15 candidates, 90666 repo-hours, 732 releases; waiting 623 ms; commit 2928 ms)
2026-11-08T01:01:13Z [load_day 2023-09-20] 2023-09-20 hour 2: 137701 events in 3 files, snapshot 1482213912992208642 (task 12775 ms: download 5213 ms overlapping gunzip 696 ms, parse 2960 ms, sort 126 ms, write 7737 ms, index 432 ms: 16 candidates, 61660 repo-hours, 688 releases; waiting 540 ms; commit 8776 ms)
2026-11-08T01:01:15Z [load_day 2023-09-20] 2023-09-20 hour 3: 140628 events in 3 files, snapshot 2785315205957144526 (task 13626 ms: download 5698 ms overlapping gunzip 678 ms, parse 3060 ms, sort 143 ms, write 8068 ms, index 821 ms: 17 candidates, 62407 repo-hours, 837 releases; waiting 516 ms; commit 2545 ms)
2026-11-08T01:01:19Z [load_day 2023-09-20] 2023-09-20 hour 4: 132041 events in 3 files, snapshot 2801597430656026809 (task 11899 ms: download 4947 ms overlapping gunzip 706 ms, parse 3115 ms, sort 99 ms, write 6725 ms, index 687 ms: 25 candidates, 57338 repo-hours, 632 releases; waiting 269 ms; commit 2451 ms)
2026-11-08T01:01:32Z [load_day 2023-09-20] 2023-09-20 hour 5: 142692 events in 3 files, snapshot 4887248017943428875 (task 15355 ms: download 6293 ms overlapping gunzip 904 ms, parse 3924 ms, sort 81 ms, write 9220 ms, index 402 ms: 19 candidates, 62108 repo-hours, 792 releases; waiting 525 ms; commit 3018 ms)
2026-11-08T01:01:38Z [load_day 2023-09-20] 2023-09-20 hour 6: 171975 events in 4 files, snapshot 4619601347123835881 (task 17079 ms: download 10348 ms overlapping gunzip 897 ms, parse 4150 ms, sort 137 ms, write 10579 ms, index 549 ms: 15 candidates, 80077 repo-hours, 976 releases; waiting 525 ms; commit 3227 ms)
2026-11-08T01:01:48Z [load_day 2023-09-20] 2023-09-20 hour 7: 181582 events in 4 files, snapshot 7730561139393315838 (task 15561 ms: download 9062 ms overlapping gunzip 989 ms, parse 4278 ms, sort 124 ms, write 8887 ms, index 523 ms: 9 candidates, 80151 repo-hours, 911 releases; waiting 482 ms; commit 2442 ms)
2026-11-08T01:01:57Z [load_day 2023-09-20] 2023-09-20 hour 8: 189262 events in 4 files, snapshot 6176457878571867343 (task 17049 ms: download 8982 ms overlapping gunzip 1006 ms, parse 4491 ms, sort 116 ms, write 9385 ms, index 1274 ms: 13 candidates, 85594 repo-hours, 1069 releases; waiting 522 ms; commit 3666 ms)
2026-11-08T01:02:10Z [load_day 2023-09-20] 2023-09-20 hour 9: 198082 events in 4 files, snapshot 4017780292875562201 (task 20493 ms: download 11635 ms overlapping gunzip 1307 ms, parse 5728 ms, sort 220 ms, write 11840 ms, index 563 ms: 9 candidates, 86401 repo-hours, 1243 releases; waiting 506 ms; commit 2529 ms)
2026-11-08T01:02:18Z [load_day 2023-09-20] 2023-09-20 hour 10: 188944 events in 4 files, snapshot 116818717647492536 (task 19776 ms: download 11185 ms overlapping gunzip 1194 ms, parse 5194 ms, sort 148 ms, write 11765 ms, index 666 ms: 14 candidates, 82626 repo-hours, 1065 releases; waiting 500 ms; commit 2922 ms)
2026-11-08T01:02:30Z [load_day 2023-09-20] 2023-09-20 hour 11: 190438 events in 4 files, snapshot 157055956982430289 (task 19447 ms: download 11035 ms overlapping gunzip 979 ms, parse 4397 ms, sort 169 ms, write 12272 ms, index 779 ms: 12 candidates, 82662 repo-hours, 1058 releases; waiting 546 ms; commit 2684 ms)
2026-11-08T01:02:43Z [load_day 2023-09-20] 2023-09-20 hour 12: 242309 events in 5 files, snapshot 9104562386817183508 (task 22176 ms: download 13517 ms overlapping gunzip 1253 ms, parse 5860 ms, sort 160 ms, write 13136 ms, index 944 ms: 15 candidates, 115295 repo-hours, 1198 releases; waiting 492 ms; commit 5047 ms)
2026-11-08T01:02:57Z [load_day 2023-09-20] 2023-09-20 hour 13: 215740 events in 5 files, snapshot 6371295568220369393 (task 22927 ms: download 15031 ms overlapping gunzip 1171 ms, parse 5347 ms, sort 263 ms, write 14683 ms, index 979 ms: 17 candidates, 107661 repo-hours, 1118 releases; waiting 183 ms; commit 4996 ms)
2026-11-08T01:03:07Z [load_day 2023-09-20] 2023-09-20 hour 14: 214978 events in 5 files, snapshot 5116876955885350052 (task 24833 ms: download 16170 ms overlapping gunzip 1806 ms, parse 7485 ms, sort 167 ms, write 13845 ms, index 760 ms: 23 candidates, 99722 repo-hours, 1221 releases; waiting 485 ms; commit 3099 ms)
2026-11-08T01:03:20Z [load_day 2023-09-20] 2023-09-20 hour 15: 189558 events in 4 files, snapshot 6749691582921102268 (task 22967 ms: download 13894 ms overlapping gunzip 1557 ms, parse 6942 ms, sort 164 ms, write 12936 ms, index 625 ms: 22 candidates, 89907 repo-hours, 1172 releases; waiting 438 ms; commit 4015 ms)
2026-11-08T01:03:28Z [load_day 2023-09-20] 2023-09-20 hour 16: 191684 events in 4 files, snapshot 3665355331528981279 (task 20788 ms: download 11289 ms overlapping gunzip 1106 ms, parse 4786 ms, sort 166 ms, write 13057 ms, index 828 ms: 22 candidates, 85761 repo-hours, 991 releases; waiting 516 ms; commit 2497 ms)
2026-11-08T01:03:43Z [load_day 2023-09-20] 2023-09-20 hour 17: 184285 events in 4 files, snapshot 5095426771944459964 (task 20422 ms: download 11485 ms overlapping gunzip 1261 ms, parse 5891 ms, sort 145 ms, write 11649 ms, index 619 ms: 20 candidates, 80790 repo-hours, 940 releases; waiting 307 ms; commit 5722 ms)
2026-11-08T01:03:50Z [load_day 2023-09-20] 2023-09-20 hour 18: 186640 events in 4 files, snapshot 8464329076156542465 (task 20056 ms: download 10992 ms overlapping gunzip 970 ms, parse 4546 ms, sort 125 ms, write 12342 ms, index 664 ms: 9 candidates, 85999 repo-hours, 1033 releases; waiting 943 ms; commit 2763 ms)
2026-11-08T01:04:02Z [load_day 2023-09-20] 2023-09-20 hour 19: 180095 events in 4 files, snapshot 8351112272518836862 (task 20046 ms: download 11563 ms overlapping gunzip 1337 ms, parse 5814 ms, sort 170 ms, write 11398 ms, index 555 ms: 11 candidates, 77155 repo-hours, 1108 releases; waiting 447 ms; commit 2946 ms)
2026-11-08T01:04:11Z [load_day 2023-09-20] 2023-09-20 hour 20: 171647 events in 4 files, snapshot 8328461688446600423 (task 18656 ms: download 10918 ms overlapping gunzip 1129 ms, parse 4966 ms, sort 181 ms, write 10819 ms, index 697 ms: 15 candidates, 73647 repo-hours, 877 releases; waiting 579 ms; commit 4543 ms)
2026-11-08T01:04:20Z [load_day 2023-09-20] 2023-09-20 hour 21: 160941 events in 4 files, snapshot 5484584331256601547 (task 17698 ms: download 10700 ms overlapping gunzip 964 ms, parse 4244 ms, sort 166 ms, write 10372 ms, index 547 ms: 14 candidates, 66523 repo-hours, 974 releases; waiting 1098 ms; commit 2716 ms)
2026-11-08T01:04:25Z [load_day 2023-09-20] 2023-09-20 hour 22: 141230 events in 3 files, snapshot 4843214676439655889 (task 14754 ms: download 6181 ms overlapping gunzip 870 ms, parse 3704 ms, sort 96 ms, write 9051 ms, index 483 ms: 18 candidates, 58471 repo-hours, 828 releases; waiting 293 ms; commit 2487 ms)
2026-11-08T01:04:33Z [load_day 2023-09-20] 2023-09-20 hour 23: 131034 events in 3 files, snapshot 1188618095670377176 (task 11620 ms: download 5288 ms overlapping gunzip 761 ms, parse 3366 ms, sort 95 ms, write 5981 ms, index 603 ms: 6 candidates, 55602 repo-hours, 724 releases; waiting 518 ms; commit 2602 ms)
2026-11-08T01:04:41Z [load_day 2023-09-20] 2023-09-20: repo_hour merged from 24 files to 1 (1922105 rows, 8160 ms)
2026-11-08T01:04:43Z [load_day 2023-09-20] 2023-09-20: index committed in 10345 ms (24 + 1 + 24 files)
2026-11-08T01:04:43Z [load_day 2023-09-20] 2023-09-20: 4215179 events in 24 hours
2026-11-08T01:04:43Z [load_day 2023-09-20] attempt 1 succeeded