orcaLog in

Runs · run 1697

load_day 2023-11-03, attempt 1 succeeded

Host
3c2b29713ce8:513690
Started
2026-11-08T02:26:28Z
Ended
2026-11-08T02:30:28Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1697/load_day/2023-11-03/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:26:28Z [load_day 2023-11-03] attempt 1 started
2026-11-08T02:26:30Z [load_day 2023-11-03] 2023-11-03: 2 hours in flight, about 1200 MB
2026-11-08T02:26:49Z [load_day 2023-11-03] 2023-11-03 hour 0: 164748 events in 4 files, snapshot 7321553262777249263 (task 14951 ms: download 6607 ms overlapping gunzip 993 ms, parse 4575 ms, sort 160 ms, write 8397 ms, index 592 ms: 6 candidates, 85944 repo-hours, 972 releases; waiting 25 ms; commit 2395 ms)
2026-11-08T02:26:54Z [load_day 2023-11-03] 2023-11-03 hour 1: 159149 events in 4 files, snapshot 5201399564051646998 (task 14522 ms: download 5332 ms overlapping gunzip 862 ms, parse 3949 ms, sort 102 ms, write 8574 ms, index 793 ms: 6 candidates, 89488 repo-hours, 786 releases; waiting 25 ms; commit 5265 ms)
2026-11-08T02:27:04Z [load_day 2023-11-03] 2023-11-03 hour 2: 140869 events in 3 files, snapshot 1077826425731582210 (task 12946 ms: download 5395 ms overlapping gunzip 778 ms, parse 3121 ms, sort 104 ms, write 8253 ms, index 474 ms: 13 candidates, 61888 repo-hours, 774 releases; waiting 17 ms; commit 2916 ms)
2026-11-08T02:27:06Z [load_day 2023-11-03] 2023-11-03 hour 3: 133014 events in 3 files, snapshot 727189788514034674 (task 12668 ms: download 5234 ms overlapping gunzip 767 ms, parse 3322 ms, sort 83 ms, write 7456 ms, index 797 ms: 5 candidates, 59449 repo-hours, 832 releases; waiting 23 ms; commit 2574 ms)
2026-11-08T02:27:17Z [load_day 2023-11-03] 2023-11-03 hour 4: 131850 events in 3 files, snapshot 7762969345929294106 (task 12129 ms: download 4569 ms overlapping gunzip 749 ms, parse 3311 ms, sort 110 ms, write 7160 ms, index 560 ms: 8 candidates, 55645 repo-hours, 684 releases; waiting 16 ms; commit 3135 ms)
2026-11-08T02:27:21Z [load_day 2023-11-03] 2023-11-03 hour 5: 135955 events in 3 files, snapshot 301691474552865751 (task 13060 ms: download 5141 ms overlapping gunzip 745 ms, parse 3116 ms, sort 113 ms, write 7794 ms, index 496 ms: 6 candidates, 58325 repo-hours, 689 releases; waiting 494 ms; commit 3003 ms)
2026-11-08T02:27:35Z [load_day 2023-11-03] 2023-11-03 hour 6: 159349 events in 4 files, snapshot 8848876490018384735 (task 14479 ms: download 5714 ms overlapping gunzip 920 ms, parse 4199 ms, sort 139 ms, write 8444 ms, index 545 ms: 10 candidates, 75522 repo-hours, 946 releases; waiting 24 ms; commit 5274 ms)
2026-11-08T02:27:41Z [load_day 2023-11-03] 2023-11-03 hour 7: 168484 events in 4 files, snapshot 6894187919681004828 (task 16902 ms: download 9370 ms overlapping gunzip 974 ms, parse 4095 ms, sort 133 ms, write 10555 ms, index 933 ms: 11 candidates, 74586 repo-hours, 858 releases; waiting 20 ms; commit 5146 ms)
2026-11-08T02:27:54Z [load_day 2023-11-03] 2023-11-03 hour 8: 190558 events in 4 files, snapshot 2621936598854597705 (task 20412 ms: download 12393 ms overlapping gunzip 1252 ms, parse 5139 ms, sort 197 ms, write 13018 ms, index 580 ms: 6 candidates, 81047 repo-hours, 995 releases; waiting 19 ms; commit 2472 ms)
2026-11-08T02:28:01Z [load_day 2023-11-03] 2023-11-03 hour 9: 195121 events in 4 files, snapshot 7064392744998795328 (task 19984 ms: download 11274 ms overlapping gunzip 1134 ms, parse 4956 ms, sort 151 ms, write 12845 ms, index 647 ms: 13 candidates, 83671 repo-hours, 1620 releases; waiting 20 ms; commit 4499 ms)
2026-11-08T02:28:16Z [load_day 2023-11-03] 2023-11-03 hour 10: 189573 events in 4 files, snapshot 1659083123124291123 (task 19850 ms: download 12185 ms overlapping gunzip 1309 ms, parse 5986 ms, sort 161 ms, write 11053 ms, index 563 ms: 15 candidates, 82374 repo-hours, 1139 releases; waiting 520 ms; commit 4102 ms)
2026-11-08T02:28:19Z [load_day 2023-11-03] 2023-11-03 hour 11: 180788 events in 4 files, snapshot 8074660392557971947 (task 17132 ms: download 10669 ms overlapping gunzip 1094 ms, parse 5241 ms, sort 155 ms, write 9838 ms, index 580 ms: 20 candidates, 76241 repo-hours, 1033 releases; waiting 26 ms; commit 2421 ms)
2026-11-08T02:28:33Z [load_day 2023-11-03] 2023-11-03 hour 12: 233966 events in 5 files, snapshot 4555724043915263296 (task 16784 ms: download 10738 ms overlapping gunzip 1013 ms, parse 4521 ms, sort 145 ms, write 10117 ms, index 759 ms: 19 candidates, 108088 repo-hours, 1103 releases; waiting 26 ms; commit 3126 ms)
2026-11-08T02:28:39Z [load_day 2023-11-03] 2023-11-03 hour 13: 219267 events in 5 files, snapshot 9005800753616292805 (task 19219 ms: download 11260 ms overlapping gunzip 1106 ms, parse 4749 ms, sort 127 ms, write 12170 ms, index 853 ms: 14 candidates, 105878 repo-hours, 1171 releases; waiting 22 ms; commit 2866 ms)
2026-11-08T02:28:59Z [load_day 2023-11-03] 2023-11-03 hour 14: 210863 events in 5 files, snapshot 9168400285864528897 (task 22658 ms: download 15811 ms overlapping gunzip 1303 ms, parse 5701 ms, sort 138 ms, write 14545 ms, index 756 ms: 18 candidates, 92765 repo-hours, 1274 releases; waiting 20 ms; commit 5411 ms)
2026-11-08T02:29:03Z [load_day 2023-11-03] 2023-11-03 hour 15: 213330 events in 5 files, snapshot 8780394773555724877 (task 22810 ms: download 15020 ms overlapping gunzip 1355 ms, parse 5914 ms, sort 150 ms, write 14341 ms, index 826 ms: 18 candidates, 93921 repo-hours, 1518 releases; waiting 25 ms; commit 2875 ms)
2026-11-08T02:29:24Z [load_day 2023-11-03] 2023-11-03 hour 16: 200881 events in 5 files, snapshot 1052966486252885582 (task 22437 ms: download 12383 ms overlapping gunzip 1331 ms, parse 5943 ms, sort 176 ms, write 13907 ms, index 861 ms: 10 candidates, 89092 repo-hours, 1169 releases; waiting 22 ms; commit 6743 ms)
2026-11-08T02:29:27Z [load_day 2023-11-03] 2023-11-03 hour 17: 188507 events in 4 files, snapshot 686240439875658051 (task 20356 ms: download 11153 ms overlapping gunzip 1176 ms, parse 5220 ms, sort 173 ms, write 12619 ms, index 935 ms: 5 candidates, 81169 repo-hours, 1160 releases; waiting 31 ms; commit 3355 ms)
2026-11-08T02:29:37Z [load_day 2023-11-03] 2023-11-03 hour 18: 160319 events in 4 files, snapshot 9130784152755023081 (task 15478 ms: download 6213 ms overlapping gunzip 921 ms, parse 4345 ms, sort 122 ms, write 9331 ms, index 552 ms: 6 candidates, 77023 repo-hours, 900 releases; waiting 18 ms; commit 2837 ms)
2026-11-08T02:29:53Z [load_day 2023-11-03] 2023-11-03 hour 19: 151088 events in 4 files, snapshot 5876005558492362808 (task 16575 ms: download 6612 ms overlapping gunzip 1020 ms, parse 4763 ms, sort 155 ms, write 9772 ms, index 651 ms: 7 candidates, 66496 repo-hours, 924 releases; waiting 30 ms; commit 11035 ms)
2026-11-08T02:29:56Z [load_day 2023-11-03] 2023-11-03 hour 20: 162515 events in 4 files, snapshot 7886245969742088083 (task 17916 ms: download 10300 ms overlapping gunzip 1054 ms, parse 4700 ms, sort 132 ms, write 11175 ms, index 620 ms: 7 candidates, 68553 repo-hours, 931 releases; waiting 23 ms; commit 3216 ms)
2026-11-08T02:30:10Z [load_day 2023-11-03] 2023-11-03 hour 21: 149153 events in 3 files, snapshot 6852231406778030604 (task 16753 ms: download 7075 ms overlapping gunzip 1042 ms, parse 4240 ms, sort 152 ms, write 9956 ms, index 549 ms: 8 candidates, 62637 repo-hours, 1022 releases; waiting 496 ms; commit 9727 ms)
2026-11-08T02:30:14Z [load_day 2023-11-03] 2023-11-03 hour 22: 134107 events in 3 files, snapshot 2234946789730398628 (task 14238 ms: download 5456 ms overlapping gunzip 831 ms, parse 3589 ms, sort 97 ms, write 8920 ms, index 581 ms: 5 candidates, 55402 repo-hours, 913 releases; waiting 15 ms; commit 4148 ms)
2026-11-08T02:30:17Z [load_day 2023-11-03] 2023-11-03 hour 23: 120095 events in 3 files, snapshot 1260322029325381948 (task 12232 ms: download 5699 ms overlapping gunzip 842 ms, parse 3884 ms, sort 102 ms, write 6823 ms, index 332 ms: 4 candidates, 50393 repo-hours, 842 releases; waiting 21 ms; commit 2432 ms)
2026-11-08T02:30:25Z [load_day 2023-11-03] 2023-11-03: repo_hour merged from 24 files to 1 (1835597 rows, 8526 ms)
2026-11-08T02:30:28Z [load_day 2023-11-03] 2023-11-03: index committed in 11114 ms (24 + 1 + 24 files)
2026-11-08T02:30:28Z [load_day 2023-11-03] 2023-11-03: 4093549 events in 24 hours
2026-11-08T02:30:28Z [load_day 2023-11-03] attempt 1 succeeded