orcaLog in

Runs · run 1686

load_day 2023-10-31, attempt 1 succeeded

Host
3c2b29713ce8:500211
Started
2026-11-08T02:19:45Z
Ended
2026-11-08T02:24:08Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1686/load_day/2023-10-31/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:19:45Z [load_day 2023-10-31] attempt 1 started
2026-11-08T02:19:46Z [load_day 2023-10-31] 2023-10-31: 2 hours in flight, about 1200 MB
2026-11-08T02:20:07Z [load_day 2023-10-31] 2023-10-31 hour 0: 174251 events in 4 files, snapshot 1434604404311407559 (task 16629 ms: download 7053 ms overlapping gunzip 1016 ms, parse 4567 ms, sort 172 ms, write 9924 ms, index 740 ms: 9 candidates, 90367 repo-hours, 963 releases; waiting 29 ms; commit 2633 ms)
2026-11-08T02:20:12Z [load_day 2023-10-31] 2023-10-31 hour 1: 175950 events in 4 files, snapshot 7615224084593550812 (task 16314 ms: download 6011 ms overlapping gunzip 919 ms, parse 4263 ms, sort 147 ms, write 9914 ms, index 866 ms: 5 candidates, 95145 repo-hours, 923 releases; waiting 14 ms; commit 4917 ms)
2026-11-08T02:20:23Z [load_day 2023-10-31] 2023-10-31 hour 2: 149359 events in 3 files, snapshot 922158450797582290 (task 14115 ms: download 5542 ms overlapping gunzip 873 ms, parse 3982 ms, sort 98 ms, write 8476 ms, index 453 ms: 6 candidates, 65052 repo-hours, 800 releases; waiting 26 ms; commit 3095 ms)
2026-11-08T02:20:28Z [load_day 2023-10-31] 2023-10-31 hour 3: 141596 events in 3 files, snapshot 3148734611934963543 (task 13152 ms: download 5902 ms overlapping gunzip 817 ms, parse 3736 ms, sort 139 ms, write 7732 ms, index 481 ms: 5 candidates, 62430 repo-hours, 969 releases; waiting 19 ms; commit 5042 ms)
2026-11-08T02:20:37Z [load_day 2023-10-31] 2023-10-31 hour 4: 139349 events in 3 files, snapshot 1668454148435684495 (task 13180 ms: download 5824 ms overlapping gunzip 792 ms, parse 3316 ms, sort 161 ms, write 7894 ms, index 533 ms: 7 candidates, 57769 repo-hours, 751 releases; waiting 182 ms; commit 3130 ms)
2026-11-08T02:20:42Z [load_day 2023-10-31] 2023-10-31 hour 5: 149386 events in 3 files, snapshot 5043850371100812480 (task 14542 ms: download 5663 ms overlapping gunzip 963 ms, parse 4170 ms, sort 98 ms, write 8527 ms, index 546 ms: 7 candidates, 61909 repo-hours, 670 releases; waiting 21 ms; commit 2734 ms)
2026-11-08T02:20:54Z [load_day 2023-10-31] 2023-10-31 hour 6: 168293 events in 4 files, snapshot 4500623708138626443 (task 15795 ms: download 6514 ms overlapping gunzip 1036 ms, parse 4617 ms, sort 150 ms, write 9150 ms, index 621 ms: 6 candidates, 76556 repo-hours, 898 releases; waiting 16 ms; commit 2785 ms)
2026-11-08T02:21:01Z [load_day 2023-10-31] 2023-10-31 hour 7: 173363 events in 4 files, snapshot 3797111437273783446 (task 17785 ms: download 10097 ms overlapping gunzip 955 ms, parse 4454 ms, sort 137 ms, write 11472 ms, index 544 ms: 5 candidates, 76169 repo-hours, 877 releases; waiting 15 ms; commit 2951 ms)
2026-11-08T02:21:23Z [load_day 2023-10-31] 2023-10-31 hour 8: 189624 events in 4 files, snapshot 460527187945004063 (task 20556 ms: download 11213 ms overlapping gunzip 1187 ms, parse 5350 ms, sort 129 ms, write 12880 ms, index 776 ms: 5 candidates, 83701 repo-hours, 1095 releases; waiting 16 ms; commit 9923 ms)
2026-11-08T02:21:27Z [load_day 2023-10-31] 2023-10-31 hour 9: 202627 events in 5 files, snapshot 2201097869798104345 (task 22957 ms: download 12354 ms overlapping gunzip 1453 ms, parse 6575 ms, sort 194 ms, write 13602 ms, index 904 ms: 7 candidates, 88105 repo-hours, 1285 releases; waiting 24 ms; commit 3876 ms)
2026-11-08T02:21:39Z [load_day 2023-10-31] 2023-10-31 hour 10: 200781 events in 5 files, snapshot 2712422530289737263 (task 21691 ms: download 12196 ms overlapping gunzip 1298 ms, parse 5980 ms, sort 167 ms, write 13262 ms, index 750 ms: 10 candidates, 87535 repo-hours, 1107 releases; waiting 27 ms; commit 2933 ms)
2026-11-08T02:21:48Z [load_day 2023-10-31] 2023-10-31 hour 11: 198045 events in 4 files, snapshot 145050917699146613 (task 20592 ms: download 11338 ms overlapping gunzip 1226 ms, parse 5639 ms, sort 134 ms, write 12702 ms, index 680 ms: 5 candidates, 85668 repo-hours, 1113 releases; waiting 21 ms; commit 3008 ms)
2026-11-08T02:22:00Z [load_day 2023-10-31] 2023-10-31 hour 12: 253417 events in 6 files, snapshot 5210483246067922441 (task 20475 ms: download 13262 ms overlapping gunzip 1214 ms, parse 5519 ms, sort 166 ms, write 12526 ms, index 824 ms: 8 candidates, 117184 repo-hours, 1174 releases; waiting 19 ms; commit 2259 ms)
2026-11-08T02:22:08Z [load_day 2023-10-31] 2023-10-31 hour 13: 234811 events in 5 files, snapshot 4098199313299292720 (task 19749 ms: download 12205 ms overlapping gunzip 1191 ms, parse 5271 ms, sort 187 ms, write 12123 ms, index 759 ms: 6 candidates, 113595 repo-hours, 1258 releases; waiting 17 ms; commit 2299 ms)
2026-11-08T02:22:26Z [load_day 2023-10-31] 2023-10-31 hour 14: 230547 events in 5 files, snapshot 5839408295131576112 (task 24019 ms: download 15008 ms overlapping gunzip 1459 ms, parse 6580 ms, sort 197 ms, write 14691 ms, index 878 ms: 10 candidates, 102938 repo-hours, 1235 releases; waiting 19 ms; commit 3109 ms)
2026-11-08T02:22:34Z [load_day 2023-10-31] 2023-10-31 hour 15: 226314 events in 5 files, snapshot 9093703906815831633 (task 24659 ms: download 16856 ms overlapping gunzip 1390 ms, parse 6396 ms, sort 187 ms, write 15587 ms, index 894 ms: 8 candidates, 101211 repo-hours, 1400 releases; waiting 16 ms; commit 3014 ms)
2026-11-08T02:22:52Z [load_day 2023-10-31] 2023-10-31 hour 16: 218623 events in 5 files, snapshot 5163258390923777631 (task 25195 ms: download 16902 ms overlapping gunzip 1427 ms, parse 6416 ms, sort 177 ms, write 15519 ms, index 791 ms: 5 candidates, 96198 repo-hours, 1292 releases; waiting 581 ms; commit 3150 ms)
2026-11-08T02:22:57Z [load_day 2023-10-31] 2023-10-31 hour 17: 202861 events in 5 files, snapshot 8936016540431900490 (task 21864 ms: download 11448 ms overlapping gunzip 1264 ms, parse 5749 ms, sort 145 ms, write 13561 ms, index 932 ms: 9 candidates, 87985 repo-hours, 1217 releases; waiting 21 ms; commit 2920 ms)
2026-11-08T02:23:15Z [load_day 2023-10-31] 2023-10-31 hour 18: 201052 events in 5 files, snapshot 203051237093733759 (task 22421 ms: download 12254 ms overlapping gunzip 1270 ms, parse 5725 ms, sort 125 ms, write 14419 ms, index 661 ms: 2 candidates, 89851 repo-hours, 1114 releases; waiting 17 ms; commit 2841 ms)
2026-11-08T02:23:21Z [load_day 2023-10-31] 2023-10-31 hour 19: 186404 events in 4 files, snapshot 7138393449702060625 (task 21710 ms: download 12551 ms overlapping gunzip 1224 ms, parse 5700 ms, sort 174 ms, write 13212 ms, index 1168 ms: 3 candidates, 78055 repo-hours, 1043 releases; waiting 22 ms; commit 3695 ms)
2026-11-08T02:23:35Z [load_day 2023-10-31] 2023-10-31 hour 20: 179747 events in 4 files, snapshot 1955552765717485642 (task 19055 ms: download 11399 ms overlapping gunzip 1150 ms, parse 5075 ms, sort 158 ms, write 11866 ms, index 592 ms: 6 candidates, 75546 repo-hours, 1083 releases; waiting 20 ms; commit 2648 ms)
2026-11-08T02:23:40Z [load_day 2023-10-31] 2023-10-31 hour 21: 165738 events in 4 files, snapshot 9149591252594824256 (task 17795 ms: download 10834 ms overlapping gunzip 1015 ms, parse 4241 ms, sort 108 ms, write 11611 ms, index 588 ms: 3 candidates, 67619 repo-hours, 1041 releases; waiting 20 ms; commit 4372 ms)
2026-11-08T02:23:53Z [load_day 2023-10-31] 2023-10-31 hour 22: 153248 events in 4 files, snapshot 480111499201518709 (task 16872 ms: download 7660 ms overlapping gunzip 1003 ms, parse 4195 ms, sort 117 ms, write 10871 ms, index 454 ms: 6 candidates, 61223 repo-hours, 881 releases; waiting 20 ms; commit 2568 ms)
2026-11-08T02:23:56Z [load_day 2023-10-31] 2023-10-31 hour 23: 141096 events in 3 files, snapshot 3956405227754094131 (task 13957 ms: download 6145 ms overlapping gunzip 1061 ms, parse 4247 ms, sort 109 ms, write 7875 ms, index 450 ms: 4 candidates, 56748 repo-hours, 875 releases; waiting 15 ms; commit 2369 ms)
2026-11-08T02:24:05Z [load_day 2023-10-31] 2023-10-31: repo_hour merged from 24 files to 1 (1978559 rows, 9522 ms)
2026-11-08T02:24:08Z [load_day 2023-10-31] 2023-10-31: index committed in 12443 ms (24 + 1 + 24 files)
2026-11-08T02:24:08Z [load_day 2023-10-31] 2023-10-31: 4456482 events in 24 hours
2026-11-08T02:24:08Z [load_day 2023-10-31] attempt 1 succeeded