orcaLog in

Runs · run 1636

load_day 2023-10-17, attempt 1 succeeded

Host
3c2b29713ce8:446625
Started
2026-11-08T01:51:41Z
Ended
2026-11-08T01:56:13Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1636/load_day/2023-10-17/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:51:41Z [load_day 2023-10-17] attempt 1 started
2026-11-08T01:51:43Z [load_day 2023-10-17] 2023-10-17: 2 hours in flight, about 1200 MB
2026-11-08T01:52:04Z [load_day 2023-10-17] 2023-10-17 hour 0: 172928 events in 4 files, snapshot 5382543032412420893 (task 17068 ms: download 7703 ms overlapping gunzip 1143 ms, parse 5008 ms, sort 188 ms, write 9270 ms, index 718 ms: 14 candidates, 92231 repo-hours, 924 releases; waiting 504 ms; commit 2778 ms)
2026-11-08T01:52:07Z [load_day 2023-10-17] 2023-10-17 hour 1: 174060 events in 4 files, snapshot 5471247243279914292 (task 16712 ms: download 6530 ms overlapping gunzip 910 ms, parse 4096 ms, sort 219 ms, write 9952 ms, index 705 ms: 4 candidates, 95929 repo-hours, 855 releases; waiting 490 ms; commit 2514 ms)
2026-11-08T01:52:20Z [load_day 2023-10-17] 2023-10-17 hour 2: 146906 events in 3 files, snapshot 6842413591895538912 (task 15198 ms: download 5919 ms overlapping gunzip 1010 ms, parse 4478 ms, sort 153 ms, write 8428 ms, index 603 ms: 9 candidates, 65799 repo-hours, 867 releases; waiting 227 ms; commit 2331 ms)
2026-11-08T01:52:24Z [load_day 2023-10-17] 2023-10-17 hour 3: 141790 events in 3 files, snapshot 5528550025476571297 (task 15430 ms: download 6672 ms overlapping gunzip 825 ms, parse 3717 ms, sort 148 ms, write 9162 ms, index 772 ms: 8 candidates, 63182 repo-hours, 951 releases; waiting 516 ms; commit 3283 ms)
2026-11-08T01:52:36Z [load_day 2023-10-17] 2023-10-17 hour 4: 141980 events in 3 files, snapshot 6624009402760980695 (task 13874 ms: download 5867 ms overlapping gunzip 806 ms, parse 3689 ms, sort 122 ms, write 7811 ms, index 577 ms: 3 candidates, 60698 repo-hours, 722 releases; waiting 475 ms; commit 3007 ms)
2026-11-08T01:52:40Z [load_day 2023-10-17] 2023-10-17 hour 5: 148610 events in 3 files, snapshot 5917348163849705721 (task 14917 ms: download 5985 ms overlapping gunzip 886 ms, parse 3688 ms, sort 136 ms, write 8819 ms, index 551 ms: 5 candidates, 64966 repo-hours, 737 releases; waiting 528 ms; commit 2766 ms)
2026-11-08T01:52:55Z [load_day 2023-10-17] 2023-10-17 hour 6: 177400 events in 4 files, snapshot 94261952861963257 (task 17712 ms: download 10199 ms overlapping gunzip 1119 ms, parse 4814 ms, sort 103 ms, write 10472 ms, index 711 ms: 12 candidates, 82200 repo-hours, 906 releases; waiting 187 ms; commit 2823 ms)
2026-11-08T01:53:03Z [load_day 2023-10-17] 2023-10-17 hour 7: 184880 events in 4 files, snapshot 5247075950977471440 (task 21339 ms: download 11882 ms overlapping gunzip 1251 ms, parse 5809 ms, sort 120 ms, write 12597 ms, index 680 ms: 6 candidates, 83437 repo-hours, 978 releases; waiting 537 ms; commit 2804 ms)
2026-11-08T01:53:18Z [load_day 2023-10-17] 2023-10-17 hour 8: 191303 events in 4 files, snapshot 4295670902565696693 (task 21769 ms: download 12785 ms overlapping gunzip 1118 ms, parse 4935 ms, sort 134 ms, write 13797 ms, index 1015 ms: 6 candidates, 88147 repo-hours, 1086 releases; waiting 519 ms; commit 2601 ms)
2026-11-08T01:53:28Z [load_day 2023-10-17] 2023-10-17 hour 9: 204106 events in 5 files, snapshot 8525250159903295070 (task 23529 ms: download 12384 ms overlapping gunzip 1296 ms, parse 6012 ms, sort 243 ms, write 14306 ms, index 634 ms: 10 candidates, 90963 repo-hours, 1230 releases; waiting 787 ms; commit 3134 ms)
2026-11-08T01:53:40Z [load_day 2023-10-17] 2023-10-17 hour 10: 195318 events in 4 files, snapshot 6297838886001659777 (task 21302 ms: download 13661 ms overlapping gunzip 1222 ms, parse 5486 ms, sort 172 ms, write 12969 ms, index 639 ms: 6 candidates, 86365 repo-hours, 1062 releases; waiting 477 ms; commit 2351 ms)
2026-11-08T01:53:46Z [load_day 2023-10-17] 2023-10-17 hour 11: 196441 events in 4 files, snapshot 9091957055765196026 (task 18272 ms: download 10604 ms overlapping gunzip 1219 ms, parse 5564 ms, sort 140 ms, write 9970 ms, index 571 ms: 9 candidates, 84208 repo-hours, 1245 releases; waiting 525 ms; commit 2303 ms)
2026-11-08T01:54:04Z [load_day 2023-10-17] 2023-10-17 hour 12: 243467 events in 5 files, snapshot 3584214956995941309 (task 22237 ms: download 12237 ms overlapping gunzip 1268 ms, parse 5664 ms, sort 211 ms, write 13504 ms, index 1121 ms: 12 candidates, 117834 repo-hours, 1156 releases; waiting 193 ms; commit 2801 ms)
2026-11-08T01:54:10Z [load_day 2023-10-17] 2023-10-17 hour 13: 221266 events in 5 files, snapshot 828060650636405460 (task 23156 ms: download 14920 ms overlapping gunzip 1328 ms, parse 6008 ms, sort 145 ms, write 13930 ms, index 867 ms: 18 candidates, 112450 repo-hours, 1155 releases; waiting 526 ms; commit 2234 ms)
2026-11-08T01:54:30Z [load_day 2023-10-17] 2023-10-17 hour 14: 210584 events in 5 files, snapshot 984174695508308467 (task 24403 ms: download 17253 ms overlapping gunzip 1386 ms, parse 6321 ms, sort 194 ms, write 15023 ms, index 763 ms: 11 candidates, 99165 repo-hours, 1290 releases; waiting 491 ms; commit 3114 ms)
2026-11-08T01:54:35Z [load_day 2023-10-17] 2023-10-17 hour 15: 189930 events in 4 files, snapshot 8794589644792487944 (task 22467 ms: download 13424 ms overlapping gunzip 1384 ms, parse 6301 ms, sort 144 ms, write 13203 ms, index 708 ms: 8 candidates, 93337 repo-hours, 1204 releases; waiting 503 ms; commit 3156 ms)
2026-11-08T01:54:55Z [load_day 2023-10-17] 2023-10-17 hour 16: 193959 events in 4 files, snapshot 857991068824890545 (task 23389 ms: download 13795 ms overlapping gunzip 1562 ms, parse 7353 ms, sort 181 ms, write 12759 ms, index 822 ms: 13 candidates, 91686 repo-hours, 969 releases; waiting 397 ms; commit 3560 ms)
2026-11-08T01:54:59Z [load_day 2023-10-17] 2023-10-17 hour 17: 197255 events in 4 files, snapshot 4572897000958951563 (task 23438 ms: download 13219 ms overlapping gunzip 1242 ms, parse 5498 ms, sort 128 ms, write 14768 ms, index 962 ms: 15 candidates, 88911 repo-hours, 919 releases; waiting 578 ms; commit 3465 ms)
2026-11-08T01:55:18Z [load_day 2023-10-17] 2023-10-17 hour 18: 193723 events in 4 files, snapshot 2044869197290514040 (task 21501 ms: download 11914 ms overlapping gunzip 1246 ms, parse 5567 ms, sort 156 ms, write 13096 ms, index 628 ms: 14 candidates, 89856 repo-hours, 1044 releases; waiting 517 ms; commit 3922 ms)
2026-11-08T01:55:22Z [load_day 2023-10-17] 2023-10-17 hour 19: 183614 events in 4 files, snapshot 4733706350624436251 (task 21983 ms: download 13016 ms overlapping gunzip 1313 ms, parse 5616 ms, sort 189 ms, write 13361 ms, index 703 ms: 10 candidates, 79544 repo-hours, 1009 releases; waiting 486 ms; commit 2846 ms)
2026-11-08T01:55:42Z [load_day 2023-10-17] 2023-10-17 hour 20: 185188 events in 4 files, snapshot 5248721684881816457 (task 22699 ms: download 13196 ms overlapping gunzip 1425 ms, parse 6258 ms, sort 154 ms, write 13492 ms, index 616 ms: 4 candidates, 78608 repo-hours, 921 releases; waiting 484 ms; commit 4146 ms)
2026-11-08T01:55:47Z [load_day 2023-10-17] 2023-10-17 hour 21: 170394 events in 4 files, snapshot 3305884196350186822 (task 20064 ms: download 12721 ms overlapping gunzip 1174 ms, parse 5119 ms, sort 165 ms, write 12011 ms, index 807 ms: 7 candidates, 70188 repo-hours, 933 releases; waiting 489 ms; commit 4686 ms)
2026-11-08T01:56:00Z [load_day 2023-10-17] 2023-10-17 hour 22: 151518 events in 4 files, snapshot 8944492822613846538 (task 18147 ms: download 9102 ms overlapping gunzip 1271 ms, parse 4971 ms, sort 125 ms, write 10444 ms, index 552 ms: 6 candidates, 62116 repo-hours, 837 releases; waiting 484 ms; commit 2525 ms)
2026-11-08T01:56:02Z [load_day 2023-10-17] 2023-10-17 hour 23: 139868 events in 3 files, snapshot 697020997628824662 (task 15755 ms: download 7598 ms overlapping gunzip 928 ms, parse 4219 ms, sort 120 ms, write 9380 ms, index 461 ms: 5 candidates, 56725 repo-hours, 758 releases; waiting 364 ms; commit 2199 ms)
2026-11-08T01:56:11Z [load_day 2023-10-17] 2023-10-17: repo_hour merged from 24 files to 1 (1998545 rows, 8919 ms)
2026-11-08T01:56:13Z [load_day 2023-10-17] 2023-10-17: index committed in 11024 ms (24 + 1 + 24 files)
2026-11-08T01:56:13Z [load_day 2023-10-17] 2023-10-17: 4356488 events in 24 hours
2026-11-08T01:56:13Z [load_day 2023-10-17] attempt 1 succeeded