orcaLog in

Runs · run 1582

load_day 2023-10-03, attempt 1 succeeded

Host
3c2b29713ce8:394864
Started
2026-11-08T01:24:23Z
Ended
2026-11-08T01:28:48Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1582/load_day/2023-10-03/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:24:23Z [load_day 2023-10-03] attempt 1 started
2026-11-08T01:24:24Z [load_day 2023-10-03] 2023-10-03: 2 hours in flight, about 1200 MB
2026-11-08T01:24:48Z [load_day 2023-10-03] 2023-10-03 hour 0: 180408 events in 4 files, snapshot 3551220704579032567 (task 19166 ms: download 12543 ms overlapping gunzip 1309 ms, parse 5817 ms, sort 159 ms, write 10511 ms, index 601 ms: 14 candidates, 91814 repo-hours, 891 releases; waiting 497 ms; commit 3293 ms)
2026-11-08T01:24:51Z [load_day 2023-10-03] 2023-10-03 hour 1: 166661 events in 4 files, snapshot 2959681768541164260 (task 17084 ms: download 6784 ms overlapping gunzip 1000 ms, parse 4365 ms, sort 161 ms, write 9885 ms, index 640 ms: 6 candidates, 88502 repo-hours, 683 releases; waiting 491 ms; commit 2904 ms)
2026-11-08T01:25:02Z [load_day 2023-10-03] 2023-10-03 hour 2: 132688 events in 3 files, snapshot 1242001475699297533 (task 13129 ms: download 5287 ms overlapping gunzip 745 ms, parse 3152 ms, sort 119 ms, write 7767 ms, index 513 ms: 6 candidates, 55596 repo-hours, 551 releases; waiting 539 ms; commit 2381 ms)
2026-11-08T01:25:07Z [load_day 2023-10-03] 2023-10-03 hour 3: 132912 events in 3 files, snapshot 405092783840651449 (task 14086 ms: download 5927 ms overlapping gunzip 820 ms, parse 3396 ms, sort 110 ms, write 8668 ms, index 444 ms: 3 candidates, 55808 repo-hours, 732 releases; waiting 401 ms; commit 3017 ms)
2026-11-08T01:25:22Z [load_day 2023-10-03] 2023-10-03 hour 4: 141419 events in 3 files, snapshot 1785494389319723507 (task 15986 ms: download 7250 ms overlapping gunzip 1126 ms, parse 4715 ms, sort 141 ms, write 8800 ms, index 439 ms: 11 candidates, 59163 repo-hours, 608 releases; waiting 501 ms; commit 5400 ms)
2026-11-08T01:25:25Z [load_day 2023-10-03] 2023-10-03 hour 5: 143873 events in 3 files, snapshot 1292623429998177047 (task 16188 ms: download 7305 ms overlapping gunzip 979 ms, parse 4034 ms, sort 97 ms, write 9719 ms, index 643 ms: 5 candidates, 59023 repo-hours, 665 releases; waiting 505 ms; commit 2817 ms)
2026-11-08T01:25:41Z [load_day 2023-10-03] 2023-10-03 hour 6: 168317 events in 4 files, snapshot 5519089817032209242 (task 18161 ms: download 11452 ms overlapping gunzip 1105 ms, parse 4919 ms, sort 128 ms, write 11093 ms, index 524 ms: 13 candidates, 73161 repo-hours, 897 releases; waiting 191 ms; commit 4583 ms)
2026-11-08T01:25:46Z [load_day 2023-10-03] 2023-10-03 hour 7: 177553 events in 4 files, snapshot 6405880285616341424 (task 19554 ms: download 11254 ms overlapping gunzip 1087 ms, parse 4769 ms, sort 132 ms, write 12091 ms, index 682 ms: 6 candidates, 74152 repo-hours, 952 releases; waiting 495 ms; commit 3295 ms)
2026-11-08T01:26:00Z [load_day 2023-10-03] 2023-10-03 hour 8: 185363 events in 4 files, snapshot 3347196904146535894 (task 21089 ms: download 12418 ms overlapping gunzip 1203 ms, parse 5295 ms, sort 225 ms, write 12922 ms, index 632 ms: 6 candidates, 78462 repo-hours, 998 releases; waiting 549 ms; commit 2253 ms)
2026-11-08T01:26:07Z [load_day 2023-10-03] 2023-10-03 hour 9: 192977 events in 4 files, snapshot 1595801547555925632 (task 20749 ms: download 11815 ms overlapping gunzip 1377 ms, parse 5919 ms, sort 142 ms, write 11805 ms, index 640 ms: 5 candidates, 80541 repo-hours, 1173 releases; waiting 542 ms; commit 2665 ms)
2026-11-08T01:26:22Z [load_day 2023-10-03] 2023-10-03 hour 10: 189730 events in 4 files, snapshot 5370691956613850885 (task 20378 ms: download 12079 ms overlapping gunzip 1188 ms, parse 5302 ms, sort 182 ms, write 12283 ms, index 650 ms: 8 candidates, 79977 repo-hours, 978 releases; waiting 543 ms; commit 2797 ms)
2026-11-08T01:26:30Z [load_day 2023-10-03] 2023-10-03 hour 11: 190196 events in 4 files, snapshot 8986182337486890777 (task 20719 ms: download 12229 ms overlapping gunzip 1276 ms, parse 5849 ms, sort 227 ms, write 11890 ms, index 724 ms: 8 candidates, 79299 repo-hours, 1094 releases; waiting 521 ms; commit 3333 ms)
2026-11-08T01:26:48Z [load_day 2023-10-03] 2023-10-03 hour 12: 246250 events in 5 files, snapshot 4681176624415920615 (task 23582 ms: download 15621 ms overlapping gunzip 1363 ms, parse 6340 ms, sort 197 ms, write 13995 ms, index 886 ms: 8 candidates, 114504 repo-hours, 1126 releases; waiting 531 ms; commit 4017 ms)
2026-11-08T01:26:56Z [load_day 2023-10-03] 2023-10-03 hour 13: 223937 events in 5 files, snapshot 6447692378762976919 (task 22907 ms: download 15752 ms overlapping gunzip 1377 ms, parse 5887 ms, sort 204 ms, write 13536 ms, index 1090 ms: 8 candidates, 108732 repo-hours, 1299 releases; waiting 500 ms; commit 4967 ms)
2026-11-08T01:27:06Z [load_day 2023-10-03] 2023-10-03 hour 14: 208332 events in 5 files, snapshot 7441279232346327218 (task 18881 ms: download 10469 ms overlapping gunzip 1102 ms, parse 4894 ms, sort 145 ms, write 11242 ms, index 648 ms: 7 candidates, 96167 repo-hours, 1118 releases; waiting 514 ms; commit 2174 ms)
2026-11-08T01:27:13Z [load_day 2023-10-03] 2023-10-03 hour 15: 187759 events in 4 files, snapshot 3614748796608544227 (task 18163 ms: download 9578 ms overlapping gunzip 1006 ms, parse 4288 ms, sort 128 ms, write 10499 ms, index 1400 ms: 4 candidates, 89187 repo-hours, 996 releases; waiting 530 ms; commit 2663 ms)
2026-11-08T01:27:33Z [load_day 2023-10-03] 2023-10-03 hour 16: 190588 events in 4 files, snapshot 4669116209485767368 (task 21948 ms: download 13376 ms overlapping gunzip 1208 ms, parse 5304 ms, sort 214 ms, write 13750 ms, index 612 ms: 12 candidates, 88473 repo-hours, 884 releases; waiting 561 ms; commit 5728 ms)
2026-11-08T01:27:36Z [load_day 2023-10-03] 2023-10-03 hour 17: 193662 events in 4 files, snapshot 3340929894643381608 (task 21579 ms: download 12011 ms overlapping gunzip 1151 ms, parse 5270 ms, sort 133 ms, write 13155 ms, index 1065 ms: 6 candidates, 86761 repo-hours, 902 releases; waiting 506 ms; commit 2688 ms)
2026-11-08T01:27:52Z [load_day 2023-10-03] 2023-10-03 hour 18: 205804 events in 5 files, snapshot 315118603399773467 (task 21603 ms: download 11854 ms overlapping gunzip 1299 ms, parse 5515 ms, sort 170 ms, write 13344 ms, index 744 ms: 9 candidates, 93775 repo-hours, 1117 releases; waiting 303 ms; commit 2614 ms)
2026-11-08T01:27:58Z [load_day 2023-10-03] 2023-10-03 hour 19: 194051 events in 4 files, snapshot 7235297393138068142 (task 20841 ms: download 11257 ms overlapping gunzip 1219 ms, parse 5194 ms, sort 160 ms, write 12470 ms, index 949 ms: 10 candidates, 81695 repo-hours, 1186 releases; waiting 506 ms; commit 3496 ms)
2026-11-08T01:28:14Z [load_day 2023-10-03] 2023-10-03 hour 20: 189305 events in 4 files, snapshot 318535730489260747 (task 20629 ms: download 11769 ms overlapping gunzip 1262 ms, parse 5153 ms, sort 142 ms, write 12339 ms, index 629 ms: 8 candidates, 80077 repo-hours, 1198 releases; waiting 526 ms; commit 2836 ms)
2026-11-08T01:28:24Z [load_day 2023-10-03] 2023-10-03 hour 21: 188340 events in 4 files, snapshot 1631998256955403245 (task 24330 ms: download 14909 ms overlapping gunzip 1772 ms, parse 7189 ms, sort 157 ms, write 13753 ms, index 689 ms: 11 candidates, 73616 repo-hours, 1113 releases; waiting 544 ms; commit 3435 ms)
2026-11-08T01:28:33Z [load_day 2023-10-03] 2023-10-03 hour 22: 163407 events in 4 files, snapshot 7889176433689525076 (task 18393 ms: download 11593 ms overlapping gunzip 1128 ms, parse 4464 ms, sort 121 ms, write 11488 ms, index 459 ms: 7 candidates, 64167 repo-hours, 1021 releases; waiting 521 ms; commit 2413 ms)
2026-11-08T01:28:37Z [load_day 2023-10-03] 2023-10-03 hour 23: 147772 events in 3 files, snapshot 7566498268415565238 (task 13692 ms: download 6685 ms overlapping gunzip 983 ms, parse 3994 ms, sort 106 ms, write 7505 ms, index 421 ms: 11 candidates, 60145 repo-hours, 868 releases; waiting 383 ms; commit 2125 ms)
2026-11-08T01:28:46Z [load_day 2023-10-03] 2023-10-03: repo_hour merged from 24 files to 1 (1912797 rows, 8631 ms)
2026-11-08T01:28:48Z [load_day 2023-10-03] 2023-10-03: index committed in 10809 ms (24 + 1 + 24 files)
2026-11-08T01:28:48Z [load_day 2023-10-03] 2023-10-03: 4341304 events in 24 hours
2026-11-08T01:28:48Z [load_day 2023-10-03] attempt 1 succeeded