orcaLog in

Runs · run 1659

load_day 2023-10-24, attempt 1 succeeded

Host
3c2b29713ce8:472351
Started
2026-11-08T02:05:19Z
Ended
2026-11-08T02:09:51Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1659/load_day/2023-10-24/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:05:19Z [load_day 2023-10-24] attempt 1 started
2026-11-08T02:05:21Z [load_day 2023-10-24] 2023-10-24: 2 hours in flight, about 1200 MB
2026-11-08T02:05:45Z [load_day 2023-10-24] 2023-10-24 hour 0: 181322 events in 4 files, snapshot 5563737671369148809 (task 19073 ms: download 12283 ms overlapping gunzip 1324 ms, parse 5550 ms, sort 128 ms, write 10644 ms, index 643 ms: 12 candidates, 97477 repo-hours, 956 releases; waiting 528 ms; commit 3124 ms)
2026-11-08T02:05:48Z [load_day 2023-10-24] 2023-10-24 hour 1: 180662 events in 4 files, snapshot 4347771456777713349 (task 19457 ms: download 11634 ms overlapping gunzip 1207 ms, parse 4959 ms, sort 107 ms, write 11501 ms, index 857 ms: 12 candidates, 99244 repo-hours, 755 releases; waiting 488 ms; commit 2364 ms)
2026-11-08T02:06:03Z [load_day 2023-10-24] 2023-10-24 hour 2: 149730 events in 3 files, snapshot 5290223765644980113 (task 16534 ms: download 7783 ms overlapping gunzip 1024 ms, parse 4197 ms, sort 113 ms, write 9660 ms, index 493 ms: 8 candidates, 66008 repo-hours, 810 releases; waiting 690 ms; commit 2754 ms)
2026-11-08T02:06:08Z [load_day 2023-10-24] 2023-10-24 hour 3: 143275 events in 3 files, snapshot 4647602711147901701 (task 13925 ms: download 5901 ms overlapping gunzip 938 ms, parse 3853 ms, sort 95 ms, write 7687 ms, index 549 ms: 12 candidates, 62192 repo-hours, 1056 releases; waiting 484 ms; commit 4989 ms)
2026-11-08T02:06:18Z [load_day 2023-10-24] 2023-10-24 hour 4: 135560 events in 3 files, snapshot 7914040981165799012 (task 14483 ms: download 6282 ms overlapping gunzip 860 ms, parse 3469 ms, sort 113 ms, write 8675 ms, index 579 ms: 5 candidates, 56923 repo-hours, 682 releases; waiting 500 ms; commit 3022 ms)
2026-11-08T02:06:23Z [load_day 2023-10-24] 2023-10-24 hour 5: 142809 events in 3 files, snapshot 525460542652669018 (task 16300 ms: download 7361 ms overlapping gunzip 1055 ms, parse 4766 ms, sort 113 ms, write 9147 ms, index 474 ms: 9 candidates, 62235 repo-hours, 693 releases; waiting 393 ms; commit 2907 ms)
2026-11-08T02:06:38Z [load_day 2023-10-24] 2023-10-24 hour 6: 174080 events in 4 files, snapshot 530381589380385318 (task 18195 ms: download 10466 ms overlapping gunzip 1073 ms, parse 4789 ms, sort 108 ms, write 10692 ms, index 676 ms: 3 candidates, 78694 repo-hours, 915 releases; waiting 506 ms; commit 3914 ms)
2026-11-08T02:06:44Z [load_day 2023-10-24] 2023-10-24 hour 7: 179272 events in 4 files, snapshot 5431818177867170068 (task 19567 ms: download 11039 ms overlapping gunzip 1304 ms, parse 5786 ms, sort 134 ms, write 10931 ms, index 559 ms: 23 candidates, 79910 repo-hours, 990 releases; waiting 505 ms; commit 2486 ms)
2026-11-08T02:07:05Z [load_day 2023-10-24] 2023-10-24 hour 8: 196655 events in 4 files, snapshot 5825176901393089973 (task 24145 ms: download 14661 ms overlapping gunzip 1720 ms, parse 7428 ms, sort 153 ms, write 13371 ms, index 671 ms: 16 candidates, 88241 repo-hours, 1155 releases; waiting 523 ms; commit 5066 ms)
2026-11-08T02:07:08Z [load_day 2023-10-24] 2023-10-24 hour 9: 197657 events in 4 files, snapshot 716742588611861244 (task 21746 ms: download 12180 ms overlapping gunzip 1172 ms, parse 5029 ms, sort 156 ms, write 13801 ms, index 924 ms: 13 candidates, 86266 repo-hours, 1304 releases; waiting 339 ms; commit 2937 ms)
2026-11-08T02:07:24Z [load_day 2023-10-24] 2023-10-24 hour 10: 172787 events in 4 files, snapshot 8771062401749953649 (task 19783 ms: download 12277 ms overlapping gunzip 1213 ms, parse 5636 ms, sort 118 ms, write 11454 ms, index 630 ms: 9 candidates, 78908 repo-hours, 1046 releases; waiting 498 ms; commit 3143 ms)
2026-11-08T02:07:31Z [load_day 2023-10-24] 2023-10-24 hour 11: 193074 events in 4 files, snapshot 2669098841556526024 (task 21409 ms: download 12180 ms overlapping gunzip 1227 ms, parse 5442 ms, sort 164 ms, write 13119 ms, index 866 ms: 11 candidates, 83224 repo-hours, 1040 releases; waiting 337 ms; commit 3167 ms)
2026-11-08T02:07:50Z [load_day 2023-10-24] 2023-10-24 hour 12: 248123 events in 5 files, snapshot 8891364737643608788 (task 25454 ms: download 16083 ms overlapping gunzip 1427 ms, parse 6190 ms, sort 162 ms, write 15496 ms, index 1401 ms: 14 candidates, 116988 repo-hours, 1285 releases; waiting 491 ms; commit 3011 ms)
2026-11-08T02:07:56Z [load_day 2023-10-24] 2023-10-24 hour 13: 230730 events in 5 files, snapshot 3701249857399358827 (task 24165 ms: download 15359 ms overlapping gunzip 1387 ms, parse 6818 ms, sort 138 ms, write 14134 ms, index 967 ms: 9 candidates, 114613 repo-hours, 1176 releases; waiting 481 ms; commit 2539 ms)
2026-11-08T02:08:16Z [load_day 2023-10-24] 2023-10-24 hour 14: 223865 events in 5 files, snapshot 5002591775373605008 (task 24654 ms: download 17083 ms overlapping gunzip 1400 ms, parse 6096 ms, sort 144 ms, write 15400 ms, index 740 ms: 8 candidates, 101502 repo-hours, 1189 releases; waiting 528 ms; commit 2385 ms)
2026-11-08T02:08:20Z [load_day 2023-10-24] 2023-10-24 hour 15: 217985 events in 5 files, snapshot 7069762574460122394 (task 22635 ms: download 16133 ms overlapping gunzip 1273 ms, parse 5648 ms, sort 147 ms, write 14149 ms, index 649 ms: 12 candidates, 99666 repo-hours, 1316 releases; waiting 485 ms; commit 2522 ms)
2026-11-08T02:08:35Z [load_day 2023-10-24] 2023-10-24 hour 16: 208262 events in 5 files, snapshot 2487452459712341219 (task 18776 ms: download 10334 ms overlapping gunzip 1143 ms, parse 4970 ms, sort 142 ms, write 10997 ms, index 644 ms: 15 candidates, 93299 repo-hours, 1124 releases; waiting 561 ms; commit 2141 ms)
2026-11-08T02:08:42Z [load_day 2023-10-24] 2023-10-24 hour 17: 203677 events in 5 files, snapshot 4524671207593909166 (task 17741 ms: download 10096 ms overlapping gunzip 1039 ms, parse 4636 ms, sort 122 ms, write 10587 ms, index 600 ms: 8 candidates, 88420 repo-hours, 1116 releases; waiting 501 ms; commit 6153 ms)
2026-11-08T02:08:57Z [load_day 2023-10-24] 2023-10-24 hour 18: 199150 events in 4 files, snapshot 6121585638764307815 (task 19693 ms: download 9837 ms overlapping gunzip 1166 ms, parse 5620 ms, sort 129 ms, write 11106 ms, index 757 ms: 6 candidates, 90822 repo-hours, 1101 releases; waiting 512 ms; commit 2872 ms)
2026-11-08T02:09:07Z [load_day 2023-10-24] 2023-10-24 hour 19: 191693 events in 4 files, snapshot 5573680251962089873 (task 20668 ms: download 11644 ms overlapping gunzip 1141 ms, parse 5153 ms, sort 182 ms, write 12778 ms, index 785 ms: 10 candidates, 82969 repo-hours, 1029 releases; waiting 83 ms; commit 4952 ms)
2026-11-08T02:09:19Z [load_day 2023-10-24] 2023-10-24 hour 20: 187326 events in 4 files, snapshot 5541131608841987210 (task 19688 ms: download 11302 ms overlapping gunzip 1250 ms, parse 5445 ms, sort 142 ms, write 11400 ms, index 700 ms: 9 candidates, 82557 repo-hours, 1022 releases; waiting 486 ms; commit 4751 ms)
2026-11-08T02:09:25Z [load_day 2023-10-24] 2023-10-24 hour 21: 172347 events in 4 files, snapshot 7699664423853194509 (task 18839 ms: download 10520 ms overlapping gunzip 1072 ms, parse 4782 ms, sort 118 ms, write 11802 ms, index 542 ms: 11 candidates, 70198 repo-hours, 974 releases; waiting 203 ms; commit 2959 ms)
2026-11-08T02:09:38Z [load_day 2023-10-24] 2023-10-24 hour 22: 156031 events in 4 files, snapshot 6846556584646351369 (task 17729 ms: download 7900 ms overlapping gunzip 1030 ms, parse 4384 ms, sort 104 ms, write 10877 ms, index 460 ms: 15 candidates, 63448 repo-hours, 932 releases; waiting 616 ms; commit 4507 ms)
2026-11-08T02:09:40Z [load_day 2023-10-24] 2023-10-24 hour 23: 142624 events in 3 files, snapshot 1288720218763576996 (task 14609 ms: download 6376 ms overlapping gunzip 927 ms, parse 3999 ms, sort 150 ms, write 8233 ms, index 491 ms: 14 candidates, 58627 repo-hours, 768 releases; waiting 518 ms; commit 2227 ms)
2026-11-08T02:09:48Z [load_day 2023-10-24] 2023-10-24: repo_hour merged from 24 files to 1 (2002431 rows, 8340 ms)
2026-11-08T02:09:51Z [load_day 2023-10-24] 2023-10-24: index committed in 10826 ms (24 + 1 + 24 files)
2026-11-08T02:09:51Z [load_day 2023-10-24] 2023-10-24: 4428696 events in 24 hours
2026-11-08T02:09:51Z [load_day 2023-10-24] attempt 1 succeeded