orcaLog in

Runs · run 1689

load_day 2023-11-01, attempt 1 succeeded

Host
3c2b29713ce8:504857
Started
2026-11-08T02:22:04Z
Ended
2026-11-08T02:26:28Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1689/load_day/2023-11-01/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:22:04Z [load_day 2023-11-01] attempt 1 started
2026-11-08T02:22:05Z [load_day 2023-11-01] 2023-11-01: 2 hours in flight, about 1200 MB
2026-11-08T02:22:28Z [load_day 2023-11-01] 2023-11-01 hour 0: 171925 events in 4 files, snapshot 7972348894180197960 (task 18021 ms: download 10984 ms overlapping gunzip 1256 ms, parse 5633 ms, sort 171 ms, write 9503 ms, index 619 ms: 8 candidates, 86283 repo-hours, 1093 releases; waiting 501 ms; commit 2677 ms)
2026-11-08T02:22:31Z [load_day 2023-11-01] 2023-11-01 hour 1: 167911 events in 4 files, snapshot 650703844009211198 (task 16944 ms: download 10279 ms overlapping gunzip 1018 ms, parse 4406 ms, sort 119 ms, write 10217 ms, index 977 ms: 6 candidates, 90969 repo-hours, 1151 releases; waiting 27 ms; commit 3039 ms)
2026-11-08T02:22:45Z [load_day 2023-11-01] 2023-11-01 hour 2: 146571 events in 3 files, snapshot 1094658575498930484 (task 15094 ms: download 5906 ms overlapping gunzip 869 ms, parse 3687 ms, sort 149 ms, write 9532 ms, index 635 ms: 9 candidates, 62500 repo-hours, 888 releases; waiting 22 ms; commit 3180 ms)
2026-11-08T02:22:47Z [load_day 2023-11-01] 2023-11-01 hour 3: 147717 events in 3 files, snapshot 7876106043470719830 (task 15493 ms: download 6557 ms overlapping gunzip 932 ms, parse 3930 ms, sort 114 ms, write 9491 ms, index 825 ms: 13 candidates, 60359 repo-hours, 944 releases; waiting 20 ms; commit 2717 ms)
2026-11-08T02:23:00Z [load_day 2023-11-01] 2023-11-01 hour 4: 138095 events in 3 files, snapshot 6149518409640092961 (task 14290 ms: download 6000 ms overlapping gunzip 910 ms, parse 3969 ms, sort 92 ms, write 8403 ms, index 695 ms: 7 candidates, 56528 repo-hours, 705 releases; waiting 19 ms; commit 3464 ms)
2026-11-08T02:23:05Z [load_day 2023-11-01] 2023-11-01 hour 5: 145131 events in 3 files, snapshot 7113507829403193402 (task 15832 ms: download 6457 ms overlapping gunzip 1154 ms, parse 4785 ms, sort 133 ms, write 9111 ms, index 444 ms: 3 candidates, 60110 repo-hours, 661 releases; waiting 18 ms; commit 3364 ms)
2026-11-08T02:23:18Z [load_day 2023-11-01] 2023-11-01 hour 6: 162798 events in 4 files, snapshot 403292703544775130 (task 15720 ms: download 6978 ms overlapping gunzip 1146 ms, parse 4989 ms, sort 143 ms, write 8660 ms, index 562 ms: 3 candidates, 73654 repo-hours, 777 releases; waiting 18 ms; commit 4010 ms)
2026-11-08T02:23:24Z [load_day 2023-11-01] 2023-11-01 hour 7: 170074 events in 4 files, snapshot 2044219646657204473 (task 16806 ms: download 9227 ms overlapping gunzip 1056 ms, parse 4364 ms, sort 100 ms, write 10309 ms, index 741 ms: 7 candidates, 71858 repo-hours, 778 releases; waiting 20 ms; commit 4532 ms)
2026-11-08T02:23:38Z [load_day 2023-11-01] 2023-11-01 hour 8: 182057 events in 4 files, snapshot 6841285898858061775 (task 18336 ms: download 10639 ms overlapping gunzip 1127 ms, parse 4885 ms, sort 220 ms, write 11339 ms, index 538 ms: 5 candidates, 79172 repo-hours, 985 releases; waiting 24 ms; commit 4162 ms)
2026-11-08T02:23:45Z [load_day 2023-11-01] 2023-11-01 hour 9: 188712 events in 4 files, snapshot 86923611276693091 (task 21025 ms: download 11460 ms overlapping gunzip 1396 ms, parse 6073 ms, sort 169 ms, write 12443 ms, index 720 ms: 6 candidates, 80685 repo-hours, 1198 releases; waiting 22 ms; commit 2761 ms)
2026-11-08T02:23:58Z [load_day 2023-11-01] 2023-11-01 hour 10: 193230 events in 4 files, snapshot 3610374411802534689 (task 19478 ms: download 11843 ms overlapping gunzip 1176 ms, parse 4979 ms, sort 127 ms, write 12407 ms, index 546 ms: 4 candidates, 82584 repo-hours, 1048 releases; waiting 19 ms; commit 3387 ms)
2026-11-08T02:24:03Z [load_day 2023-11-01] 2023-11-01 hour 11: 188100 events in 4 files, snapshot 206659800355814966 (task 16764 ms: download 9815 ms overlapping gunzip 1103 ms, parse 4831 ms, sort 171 ms, write 9895 ms, index 523 ms: 4 candidates, 79970 repo-hours, 1123 releases; waiting 25 ms; commit 2852 ms)
2026-11-08T02:24:18Z [load_day 2023-11-01] 2023-11-01 hour 12: 243101 events in 5 files, snapshot 4411113284525899740 (task 19015 ms: download 11260 ms overlapping gunzip 1034 ms, parse 4717 ms, sort 146 ms, write 11734 ms, index 1126 ms: 6 candidates, 112072 repo-hours, 1160 releases; waiting 21 ms; commit 2954 ms)
2026-11-08T02:24:25Z [load_day 2023-11-01] 2023-11-01 hour 13: 227680 events in 5 files, snapshot 1937412390900927261 (task 20700 ms: download 11678 ms overlapping gunzip 1276 ms, parse 5598 ms, sort 198 ms, write 12590 ms, index 827 ms: 10 candidates, 108819 repo-hours, 1266 releases; waiting 22 ms; commit 3036 ms)
2026-11-08T02:24:43Z [load_day 2023-11-01] 2023-11-01 hour 14: 219427 events in 5 files, snapshot 7953775147510819739 (task 24251 ms: download 16008 ms overlapping gunzip 1484 ms, parse 6163 ms, sort 171 ms, write 15570 ms, index 641 ms: 5 candidates, 96074 repo-hours, 1751 releases; waiting 17 ms; commit 2774 ms)
2026-11-08T02:24:52Z [load_day 2023-11-01] 2023-11-01 hour 15: 221313 events in 5 files, snapshot 4677054807044455865 (task 24297 ms: download 16737 ms overlapping gunzip 1468 ms, parse 6400 ms, sort 170 ms, write 15306 ms, index 722 ms: 6 candidates, 96492 repo-hours, 1410 releases; waiting 26 ms; commit 4383 ms)
2026-11-08T02:25:10Z [load_day 2023-11-01] 2023-11-01 hour 16: 215866 events in 5 files, snapshot 6049847862356520498 (task 25279 ms: download 17222 ms overlapping gunzip 1522 ms, parse 6588 ms, sort 140 ms, write 15326 ms, index 645 ms: 9 candidates, 93548 repo-hours, 1427 releases; waiting 665 ms; commit 2881 ms)
2026-11-08T02:25:15Z [load_day 2023-11-01] 2023-11-01 hour 17: 207432 events in 5 files, snapshot 7802461282308187611 (task 23747 ms: download 17160 ms overlapping gunzip 1654 ms, parse 7147 ms, sort 217 ms, write 13847 ms, index 633 ms: 11 candidates, 87687 repo-hours, 1184 releases; waiting 22 ms; commit 2669 ms)
2026-11-08T02:25:33Z [load_day 2023-11-01] 2023-11-01 hour 18: 201568 events in 5 files, snapshot 5014452694558831336 (task 22620 ms: download 12539 ms overlapping gunzip 1417 ms, parse 6010 ms, sort 151 ms, write 14192 ms, index 637 ms: 9 candidates, 90918 repo-hours, 1168 releases; waiting 22 ms; commit 2949 ms)
2026-11-08T02:25:36Z [load_day 2023-11-01] 2023-11-01 hour 19: 191208 events in 4 files, snapshot 5456949841854992283 (task 20543 ms: download 11751 ms overlapping gunzip 1211 ms, parse 5177 ms, sort 122 ms, write 12879 ms, index 928 ms: 9 candidates, 78065 repo-hours, 1196 releases; waiting 31 ms; commit 2580 ms)
2026-11-08T02:25:56Z [load_day 2023-11-01] 2023-11-01 hour 20: 182154 events in 4 files, snapshot 6275348018175957130 (task 21342 ms: download 12930 ms overlapping gunzip 1258 ms, parse 5587 ms, sort 133 ms, write 12892 ms, index 629 ms: 15 candidates, 77275 repo-hours, 1151 releases; waiting 566 ms; commit 2477 ms)
2026-11-08T02:25:59Z [load_day 2023-11-01] 2023-11-01 hour 21: 168149 events in 4 files, snapshot 1117033708301710213 (task 19186 ms: download 12205 ms overlapping gunzip 1189 ms, parse 5347 ms, sort 109 ms, write 11765 ms, index 550 ms: 14 candidates, 68308 repo-hours, 1071 releases; waiting 21 ms; commit 3110 ms)
2026-11-08T02:26:14Z [load_day 2023-11-01] 2023-11-01 hour 22: 150957 events in 4 files, snapshot 7114278315164843224 (task 17609 ms: download 7787 ms overlapping gunzip 913 ms, parse 4020 ms, sort 87 ms, write 11194 ms, index 557 ms: 16 candidates, 62082 repo-hours, 966 releases; waiting 532 ms; commit 2216 ms)
2026-11-08T02:26:16Z [load_day 2023-11-01] 2023-11-01 hour 23: 141332 events in 3 files, snapshot 277185130317393161 (task 15378 ms: download 7144 ms overlapping gunzip 1135 ms, parse 4812 ms, sort 96 ms, write 8723 ms, index 400 ms: 7 candidates, 59308 repo-hours, 787 releases; waiting 25 ms; commit 2392 ms)
2026-11-08T02:26:25Z [load_day 2023-11-01] 2023-11-01: repo_hour merged from 24 files to 1 (1915320 rows, 8963 ms)
2026-11-08T02:26:28Z [load_day 2023-11-01] 2023-11-01: index committed in 11098 ms (24 + 1 + 24 files)
2026-11-08T02:26:28Z [load_day 2023-11-01] 2023-11-01: 4372508 events in 24 hours
2026-11-08T02:26:28Z [load_day 2023-11-01] attempt 1 succeeded