orcaLog in

Runs · run 1706

load_day 2023-11-06, attempt 1 succeeded

Host
3c2b29713ce8:522121
Started
2026-11-08T02:31:07Z
Ended
2026-11-08T02:35:47Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1706/load_day/2023-11-06/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:31:07Z [load_day 2023-11-06] attempt 1 started
2026-11-08T02:31:08Z [load_day 2023-11-06] 2023-11-06: 2 hours in flight, about 1200 MB
2026-11-08T02:31:30Z [load_day 2023-11-06] 2023-11-06 hour 0: 161687 events in 4 files, snapshot 5230103391810211380 (task 16719 ms: download 7335 ms overlapping gunzip 1099 ms, parse 4537 ms, sort 102 ms, write 9674 ms, index 580 ms: 14 candidates, 80809 repo-hours, 862 releases; waiting 502 ms; commit 3149 ms)
2026-11-08T02:31:35Z [load_day 2023-11-06] 2023-11-06 hour 1: 174722 events in 4 files, snapshot 2522429606338294156 (task 17710 ms: download 10624 ms overlapping gunzip 991 ms, parse 4035 ms, sort 142 ms, write 11459 ms, index 632 ms: 8 candidates, 91215 repo-hours, 782 releases; waiting 203 ms; commit 5014 ms)
2026-11-08T02:31:50Z [load_day 2023-11-06] 2023-11-06 hour 2: 147491 events in 3 files, snapshot 1500101331805291591 (task 16371 ms: download 7980 ms overlapping gunzip 1214 ms, parse 4506 ms, sort 134 ms, write 9286 ms, index 460 ms: 13 candidates, 63665 repo-hours, 761 releases; waiting 509 ms; commit 5090 ms)
2026-11-08T02:31:56Z [load_day 2023-11-06] 2023-11-06 hour 3: 145881 events in 3 files, snapshot 5910015906375926678 (task 15478 ms: download 6868 ms overlapping gunzip 1009 ms, parse 4343 ms, sort 109 ms, write 9121 ms, index 670 ms: 5 candidates, 62031 repo-hours, 810 releases; waiting 21 ms; commit 5868 ms)
2026-11-08T02:32:05Z [load_day 2023-11-06] 2023-11-06 hour 4: 143407 events in 3 files, snapshot 9084370440114387112 (task 15332 ms: download 5989 ms overlapping gunzip 944 ms, parse 4020 ms, sort 131 ms, write 9461 ms, index 523 ms: 6 candidates, 56906 repo-hours, 625 releases; waiting 25 ms; commit 3444 ms)
2026-11-08T02:32:12Z [load_day 2023-11-06] 2023-11-06 hour 5: 146768 events in 3 files, snapshot 7989803365653965369 (task 15797 ms: download 6167 ms overlapping gunzip 993 ms, parse 4080 ms, sort 201 ms, write 9745 ms, index 559 ms: 4 candidates, 61344 repo-hours, 659 releases; waiting 20 ms; commit 4931 ms)
2026-11-08T02:32:25Z [load_day 2023-11-06] 2023-11-06 hour 6: 169624 events in 4 files, snapshot 6281755244012579382 (task 17224 ms: download 10360 ms overlapping gunzip 1117 ms, parse 4651 ms, sort 142 ms, write 10469 ms, index 582 ms: 9 candidates, 75467 repo-hours, 887 releases; waiting 22 ms; commit 4931 ms)
2026-11-08T02:32:33Z [load_day 2023-11-06] 2023-11-06 hour 7: 177663 events in 4 files, snapshot 6652706915778871507 (task 19902 ms: download 11432 ms overlapping gunzip 1202 ms, parse 5079 ms, sort 156 ms, write 12666 ms, index 546 ms: 7 candidates, 75933 repo-hours, 1155 releases; waiting 22 ms; commit 4627 ms)
2026-11-08T02:32:46Z [load_day 2023-11-06] 2023-11-06 hour 8: 192811 events in 4 files, snapshot 697355648371225959 (task 20780 ms: download 11584 ms overlapping gunzip 1279 ms, parse 5251 ms, sort 128 ms, write 13211 ms, index 664 ms: 8 candidates, 84826 repo-hours, 1184 releases; waiting 10 ms; commit 4337 ms)
2026-11-08T02:32:56Z [load_day 2023-11-06] 2023-11-06 hour 9: 203767 events in 5 files, snapshot 8381124402869192345 (task 23440 ms: download 12833 ms overlapping gunzip 1506 ms, parse 6626 ms, sort 158 ms, write 14258 ms, index 667 ms: 10 candidates, 87944 repo-hours, 1394 releases; waiting 18 ms; commit 3511 ms)
2026-11-08T02:33:06Z [load_day 2023-11-06] 2023-11-06 hour 10: 203457 events in 5 files, snapshot 977208276437282335 (task 19986 ms: download 14298 ms overlapping gunzip 1317 ms, parse 5518 ms, sort 140 ms, write 12156 ms, index 618 ms: 2 candidates, 87239 repo-hours, 1137 releases; waiting 28 ms; commit 2496 ms)
2026-11-08T02:33:15Z [load_day 2023-11-06] 2023-11-06 hour 11: 206434 events in 5 files, snapshot 265413023576682011 (task 18076 ms: download 9916 ms overlapping gunzip 1211 ms, parse 4968 ms, sort 145 ms, write 10985 ms, index 551 ms: 6 candidates, 84329 repo-hours, 1244 releases; waiting 24 ms; commit 3103 ms)
2026-11-08T02:33:31Z [load_day 2023-11-06] 2023-11-06 hour 12: 249011 events in 5 files, snapshot 121768931203273159 (task 23473 ms: download 13558 ms overlapping gunzip 1412 ms, parse 6140 ms, sort 225 ms, write 14490 ms, index 997 ms: 13 candidates, 116917 repo-hours, 1103 releases; waiting 15 ms; commit 2631 ms)
2026-11-08T02:33:41Z [load_day 2023-11-06] 2023-11-06 hour 13: 235160 events in 5 files, snapshot 6509184478904734591 (task 25483 ms: download 16190 ms overlapping gunzip 1700 ms, parse 7148 ms, sort 257 ms, write 15006 ms, index 1121 ms: 11 candidates, 114074 repo-hours, 1136 releases; waiting 28 ms; commit 2756 ms)
2026-11-08T02:33:56Z [load_day 2023-11-06] 2023-11-06 hour 14: 197360 events in 4 files, snapshot 1569666706470352696 (task 21886 ms: download 11475 ms overlapping gunzip 1292 ms, parse 5412 ms, sort 178 ms, write 14106 ms, index 674 ms: 10 candidates, 91499 repo-hours, 1031 releases; waiting 18 ms; commit 5024 ms)
2026-11-08T02:34:08Z [load_day 2023-11-06] 2023-11-06 hour 15: 196648 events in 4 files, snapshot 7649765487333537609 (task 23978 ms: download 13695 ms overlapping gunzip 1485 ms, parse 6369 ms, sort 162 ms, write 14628 ms, index 596 ms: 11 candidates, 91844 repo-hours, 1256 releases; waiting 477 ms; commit 4736 ms)
2026-11-08T02:34:22Z [load_day 2023-11-06] 2023-11-06 hour 16: 220527 events in 5 files, snapshot 8551615494683901116 (task 26285 ms: download 18266 ms overlapping gunzip 1865 ms, parse 8282 ms, sort 142 ms, write 15008 ms, index 759 ms: 8 candidates, 97758 repo-hours, 1172 releases; waiting 24 ms; commit 3298 ms)
2026-11-08T02:34:32Z [load_day 2023-11-06] 2023-11-06 hour 17: 212517 events in 5 files, snapshot 3098406030740525446 (task 24217 ms: download 16049 ms overlapping gunzip 1538 ms, parse 6190 ms, sort 184 ms, write 14817 ms, index 1254 ms: 8 candidates, 90938 repo-hours, 1251 releases; waiting 22 ms; commit 3119 ms)
2026-11-08T02:34:47Z [load_day 2023-11-06] 2023-11-06 hour 18: 210857 events in 5 files, snapshot 2743562370895701881 (task 24587 ms: download 17212 ms overlapping gunzip 1511 ms, parse 6734 ms, sort 138 ms, write 14996 ms, index 983 ms: 8 candidates, 93246 repo-hours, 1237 releases; waiting 19 ms; commit 3081 ms)
2026-11-08T02:34:56Z [load_day 2023-11-06] 2023-11-06 hour 19: 194394 events in 4 files, snapshot 4693018560904368880 (task 22423 ms: download 12252 ms overlapping gunzip 1285 ms, parse 5677 ms, sort 147 ms, write 14224 ms, index 879 ms: 16 candidates, 81910 repo-hours, 1077 releases; waiting 13 ms; commit 2977 ms)
2026-11-08T02:35:11Z [load_day 2023-11-06] 2023-11-06 hour 20: 194430 events in 4 files, snapshot 3462454197063425442 (task 21875 ms: download 12544 ms overlapping gunzip 1386 ms, parse 5847 ms, sort 121 ms, write 13390 ms, index 906 ms: 13 candidates, 81252 repo-hours, 1073 releases; waiting 23 ms; commit 3008 ms)
2026-11-08T02:35:17Z [load_day 2023-11-06] 2023-11-06 hour 21: 183478 events in 4 files, snapshot 7031209845529654835 (task 20477 ms: download 11584 ms overlapping gunzip 1229 ms, parse 5290 ms, sort 125 ms, write 12923 ms, index 690 ms: 7 candidates, 74532 repo-hours, 1070 releases; waiting 26 ms; commit 2725 ms)
2026-11-08T02:35:33Z [load_day 2023-11-06] 2023-11-06 hour 22: 169560 events in 4 files, snapshot 3802292560368754607 (task 19986 ms: download 12533 ms overlapping gunzip 1172 ms, parse 5011 ms, sort 159 ms, write 12864 ms, index 552 ms: 3 candidates, 68490 repo-hours, 955 releases; waiting 20 ms; commit 3700 ms)
2026-11-08T02:35:35Z [load_day 2023-11-06] 2023-11-06 hour 23: 154459 events in 4 files, snapshot 7277758451767854951 (task 16708 ms: download 11030 ms overlapping gunzip 1107 ms, parse 4650 ms, sort 105 ms, write 9566 ms, index 457 ms: 8 candidates, 63463 repo-hours, 830 releases; waiting 517 ms; commit 2234 ms)
2026-11-08T02:35:44Z [load_day 2023-11-06] 2023-11-06: repo_hour merged from 24 files to 1 (1977631 rows, 9322 ms)
2026-11-08T02:35:47Z [load_day 2023-11-06] 2023-11-06: index committed in 11495 ms (24 + 1 + 24 files)
2026-11-08T02:35:47Z [load_day 2023-11-06] 2023-11-06: 4492113 events in 24 hours
2026-11-08T02:35:47Z [load_day 2023-11-06] attempt 1 succeeded