orcaLog in

Runs · run 1711

load_day 2023-11-07, attempt 1 succeeded

Host
3c2b29713ce8:526001
Started
2026-11-08T02:33:08Z
Ended
2026-11-08T02:37:34Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1711/load_day/2023-11-07/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:33:08Z [load_day 2023-11-07] attempt 1 started
2026-11-08T02:33:10Z [load_day 2023-11-07] 2023-11-07: 2 hours in flight, about 1200 MB
2026-11-08T02:33:34Z [load_day 2023-11-07] 2023-11-07 hour 0: 176977 events in 4 files, snapshot 8522610605637105091 (task 19356 ms: download 11729 ms overlapping gunzip 1265 ms, parse 5909 ms, sort 170 ms, write 10481 ms, index 773 ms: 11 candidates, 91629 repo-hours, 1038 releases; waiting 504 ms; commit 3004 ms)
2026-11-08T02:33:37Z [load_day 2023-11-07] 2023-11-07 hour 1: 177580 events in 4 files, snapshot 3840205870305939196 (task 17276 ms: download 10248 ms overlapping gunzip 924 ms, parse 4411 ms, sort 131 ms, write 10936 ms, index 667 ms: 4 candidates, 97761 repo-hours, 874 releases; waiting 18 ms; commit 2865 ms)
2026-11-08T02:33:50Z [load_day 2023-11-07] 2023-11-07 hour 2: 153531 events in 4 files, snapshot 7008133169995410195 (task 15021 ms: download 6132 ms overlapping gunzip 996 ms, parse 4182 ms, sort 107 ms, write 8834 ms, index 691 ms: 8 candidates, 68084 repo-hours, 953 releases; waiting 19 ms; commit 2792 ms)
2026-11-08T02:33:54Z [load_day 2023-11-07] 2023-11-07 hour 3: 148885 events in 3 files, snapshot 2231344200860358195 (task 15379 ms: download 6108 ms overlapping gunzip 920 ms, parse 4372 ms, sort 161 ms, write 9043 ms, index 561 ms: 15 candidates, 64714 repo-hours, 1022 releases; waiting 31 ms; commit 3002 ms)
2026-11-08T02:34:05Z [load_day 2023-11-07] 2023-11-07 hour 4: 143462 events in 3 files, snapshot 6935939319544997480 (task 13534 ms: download 5133 ms overlapping gunzip 856 ms, parse 3702 ms, sort 110 ms, write 8189 ms, index 466 ms: 9 candidates, 59438 repo-hours, 773 releases; waiting 24 ms; commit 3062 ms)
2026-11-08T02:34:11Z [load_day 2023-11-07] 2023-11-07 hour 5: 145823 events in 3 files, snapshot 4364271600286997953 (task 15982 ms: download 6667 ms overlapping gunzip 964 ms, parse 4218 ms, sort 138 ms, write 9695 ms, index 741 ms: 5 candidates, 63390 repo-hours, 715 releases; waiting 15 ms; commit 3081 ms)
2026-11-08T02:34:24Z [load_day 2023-11-07] 2023-11-07 hour 6: 166187 events in 4 files, snapshot 1199663293592198083 (task 16447 ms: download 7005 ms overlapping gunzip 957 ms, parse 4258 ms, sort 171 ms, write 9727 ms, index 578 ms: 8 candidates, 76314 repo-hours, 842 releases; waiting 456 ms; commit 4546 ms)
2026-11-08T02:34:29Z [load_day 2023-11-07] 2023-11-07 hour 7: 179962 events in 4 files, snapshot 3278210912199994725 (task 17670 ms: download 9341 ms overlapping gunzip 1092 ms, parse 4489 ms, sort 134 ms, write 11168 ms, index 565 ms: 8 candidates, 76701 repo-hours, 940 releases; waiting 26 ms; commit 2824 ms)
2026-11-08T02:34:45Z [load_day 2023-11-07] 2023-11-07 hour 8: 195529 events in 4 files, snapshot 3564653740915819279 (task 21448 ms: download 12010 ms overlapping gunzip 1361 ms, parse 5844 ms, sort 166 ms, write 13261 ms, index 594 ms: 7 candidates, 84354 repo-hours, 963 releases; waiting 21 ms; commit 2557 ms)
2026-11-08T02:34:53Z [load_day 2023-11-07] 2023-11-07 hour 9: 200575 events in 5 files, snapshot 5026126279084172372 (task 21765 ms: download 11749 ms overlapping gunzip 1420 ms, parse 6384 ms, sort 169 ms, write 12623 ms, index 914 ms: 6 candidates, 87489 repo-hours, 1364 releases; waiting 45 ms; commit 2943 ms)
2026-11-08T02:35:07Z [load_day 2023-11-07] 2023-11-07 hour 10: 202279 events in 5 files, snapshot 70355651268852072 (task 21751 ms: download 11696 ms overlapping gunzip 1212 ms, parse 5814 ms, sort 189 ms, write 13697 ms, index 610 ms: 8 candidates, 87589 repo-hours, 1086 releases; waiting 30 ms; commit 2921 ms)
2026-11-08T02:35:14Z [load_day 2023-11-07] 2023-11-07 hour 11: 196851 events in 4 files, snapshot 633013502770519119 (task 20407 ms: download 11433 ms overlapping gunzip 1278 ms, parse 5795 ms, sort 133 ms, write 12384 ms, index 589 ms: 3 candidates, 84168 repo-hours, 1210 releases; waiting 18 ms; commit 2932 ms)
2026-11-08T02:35:31Z [load_day 2023-11-07] 2023-11-07 hour 12: 246903 events in 5 files, snapshot 4871517093302508350 (task 22311 ms: download 14406 ms overlapping gunzip 1231 ms, parse 5740 ms, sort 168 ms, write 14100 ms, index 850 ms: 11 candidates, 115648 repo-hours, 1206 releases; waiting 27 ms; commit 3045 ms)
2026-11-08T02:35:37Z [load_day 2023-11-07] 2023-11-07 hour 13: 229412 events in 5 files, snapshot 1656632485692808063 (task 22328 ms: download 14798 ms overlapping gunzip 1389 ms, parse 6439 ms, sort 214 ms, write 13311 ms, index 748 ms: 4 candidates, 112053 repo-hours, 1126 releases; waiting 21 ms; commit 2614 ms)
2026-11-08T02:35:52Z [load_day 2023-11-07] 2023-11-07 hour 14: 225571 events in 5 files, snapshot 7881117216825880174 (task 20467 ms: download 14010 ms overlapping gunzip 1233 ms, parse 5342 ms, sort 144 ms, write 12244 ms, index 622 ms: 5 candidates, 101534 repo-hours, 1217 releases; waiting 624 ms; commit 2863 ms)
2026-11-08T02:36:00Z [load_day 2023-11-07] 2023-11-07 hour 15: 222312 events in 5 files, snapshot 2261695434939026193 (task 21153 ms: download 12617 ms overlapping gunzip 1234 ms, parse 5459 ms, sort 150 ms, write 13305 ms, index 797 ms: 4 candidates, 101168 repo-hours, 1366 releases; waiting 22 ms; commit 3081 ms)
2026-11-08T02:36:18Z [load_day 2023-11-07] 2023-11-07 hour 16: 217185 events in 5 files, snapshot 8470464410975094917 (task 25091 ms: download 17916 ms overlapping gunzip 1523 ms, parse 6716 ms, sort 160 ms, write 14795 ms, index 1052 ms: 8 candidates, 98493 repo-hours, 1207 releases; waiting 554 ms; commit 2911 ms)
2026-11-08T02:36:23Z [load_day 2023-11-07] 2023-11-07 hour 17: 206528 events in 5 files, snapshot 1856388135994595228 (task 22390 ms: download 12068 ms overlapping gunzip 1459 ms, parse 6421 ms, sort 147 ms, write 13290 ms, index 828 ms: 7 candidates, 89862 repo-hours, 1193 releases; waiting 22 ms; commit 2550 ms)
2026-11-08T02:36:40Z [load_day 2023-11-07] 2023-11-07 hour 18: 202401 events in 5 files, snapshot 9053788442011157160 (task 20443 ms: download 10945 ms overlapping gunzip 1205 ms, parse 5360 ms, sort 133 ms, write 12850 ms, index 681 ms: 8 candidates, 91352 repo-hours, 1165 releases; waiting 18 ms; commit 2954 ms)
2026-11-08T02:36:46Z [load_day 2023-11-07] 2023-11-07 hour 19: 190270 events in 4 files, snapshot 5528478075628077589 (task 19723 ms: download 10961 ms overlapping gunzip 1188 ms, parse 5371 ms, sort 154 ms, write 12122 ms, index 667 ms: 9 candidates, 80240 repo-hours, 1078 releases; waiting 19 ms; commit 4547 ms)
2026-11-08T02:37:02Z [load_day 2023-11-07] 2023-11-07 hour 20: 182115 events in 4 files, snapshot 524917558456910318 (task 19372 ms: download 11430 ms overlapping gunzip 1249 ms, parse 5418 ms, sort 145 ms, write 11688 ms, index 658 ms: 7 candidates, 78000 repo-hours, 940 releases; waiting 22 ms; commit 4584 ms)
2026-11-08T02:37:05Z [load_day 2023-11-07] 2023-11-07 hour 21: 180590 events in 4 files, snapshot 7606262680331407376 (task 19411 ms: download 11232 ms overlapping gunzip 1188 ms, parse 5318 ms, sort 122 ms, write 11692 ms, index 861 ms: 12 candidates, 75259 repo-hours, 1079 releases; waiting 27 ms; commit 3022 ms)
2026-11-08T02:37:19Z [load_day 2023-11-07] 2023-11-07 hour 22: 168340 events in 4 files, snapshot 3662367435237881245 (task 17934 ms: download 11348 ms overlapping gunzip 1061 ms, parse 4705 ms, sort 146 ms, write 10635 ms, index 555 ms: 10 candidates, 69805 repo-hours, 1059 releases; waiting 495 ms; commit 2323 ms)
2026-11-08T02:37:23Z [load_day 2023-11-07] 2023-11-07 hour 23: 154095 events in 4 files, snapshot 7920325538891166322 (task 15119 ms: download 6658 ms overlapping gunzip 813 ms, parse 3583 ms, sort 102 ms, write 9935 ms, index 488 ms: 9 candidates, 65910 repo-hours, 867 releases; waiting 14 ms; commit 3707 ms)
2026-11-08T02:37:31Z [load_day 2023-11-07] 2023-11-07: repo_hour merged from 24 files to 1 (2020955 rows, 8531 ms)
2026-11-08T02:37:34Z [load_day 2023-11-07] 2023-11-07: index committed in 10903 ms (24 + 1 + 24 files)
2026-11-08T02:37:34Z [load_day 2023-11-07] 2023-11-07: 4513363 events in 24 hours
2026-11-08T02:37:34Z [load_day 2023-11-07] attempt 1 succeeded