orcaLog in

Runs · run 1758

load_day 2023-11-20, attempt 1 succeeded

Host
3c2b29713ce8:575057
Started
2026-11-08T02:58:39Z
Ended
2026-11-08T03:03:48Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1758/load_day/2023-11-20/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:58:39Z [load_day 2023-11-20] attempt 1 started
2026-11-08T02:58:41Z [load_day 2023-11-20] 2023-11-20: 2 hours in flight, about 1200 MB
2026-11-08T02:59:00Z [load_day 2023-11-20] 2023-11-20 hour 0: 164949 events in 4 files, snapshot 4269545223271475130 (task 14690 ms: download 5908 ms overlapping gunzip 983 ms, parse 4107 ms, sort 104 ms, write 8448 ms, index 823 ms: 6 candidates, 80035 repo-hours, 905 releases; waiting 20 ms; commit 3101 ms)
2026-11-08T02:59:05Z [load_day 2023-11-20] 2023-11-20 hour 1: 182712 events in 4 files, snapshot 2515854284326891052 (task 16529 ms: download 9402 ms overlapping gunzip 1131 ms, parse 4898 ms, sort 135 ms, write 9408 ms, index 724 ms: 15 candidates, 91899 repo-hours, 845 releases; waiting 27 ms; commit 5120 ms)
2026-11-08T02:59:16Z [load_day 2023-11-20] 2023-11-20 hour 2: 158071 events in 4 files, snapshot 1299068919199853546 (task 15220 ms: download 6266 ms overlapping gunzip 1013 ms, parse 4105 ms, sort 108 ms, write 9259 ms, index 521 ms: 8 candidates, 62254 repo-hours, 740 releases; waiting 20 ms; commit 3148 ms)
2026-11-08T02:59:22Z [load_day 2023-11-20] 2023-11-20 hour 3: 157465 events in 4 files, snapshot 5793789137678490785 (task 16337 ms: download 6905 ms overlapping gunzip 1158 ms, parse 4585 ms, sort 121 ms, write 9504 ms, index 725 ms: 7 candidates, 60911 repo-hours, 936 releases; waiting 25 ms; commit 4664 ms)
2026-11-08T02:59:33Z [load_day 2023-11-20] 2023-11-20 hour 4: 156996 events in 4 files, snapshot 476366249824818718 (task 15635 ms: download 7005 ms overlapping gunzip 1111 ms, parse 4592 ms, sort 157 ms, write 9165 ms, index 391 ms: 4 candidates, 56599 repo-hours, 734 releases; waiting 22 ms; commit 2839 ms)
2026-11-08T02:59:38Z [load_day 2023-11-20] 2023-11-20 hour 5: 159656 events in 4 files, snapshot 6497252551879250480 (task 16503 ms: download 6840 ms overlapping gunzip 1145 ms, parse 4525 ms, sort 88 ms, write 9663 ms, index 842 ms: 10 candidates, 60325 repo-hours, 649 releases; waiting 23 ms; commit 2878 ms)
2026-11-08T02:59:55Z [load_day 2023-11-20] 2023-11-20 hour 6: 182500 events in 4 files, snapshot 611101767140277025 (task 17925 ms: download 10171 ms overlapping gunzip 1015 ms, parse 4160 ms, sort 154 ms, write 11804 ms, index 572 ms: 9 candidates, 74712 repo-hours, 819 releases; waiting 28 ms; commit 5156 ms)
2026-11-08T03:00:01Z [load_day 2023-11-20] 2023-11-20 hour 7: 187555 events in 4 files, snapshot 2800750605637771207 (task 19121 ms: download 10589 ms overlapping gunzip 1268 ms, parse 5453 ms, sort 167 ms, write 10918 ms, index 1083 ms: 19 candidates, 74174 repo-hours, 804 releases; waiting 21 ms; commit 4955 ms)
2026-11-08T03:00:26Z [load_day 2023-11-20] 2023-11-20 hour 8: 202568 events in 5 files, snapshot 8512967837964454866 (task 26708 ms: download 15148 ms overlapping gunzip 1656 ms, parse 7197 ms, sort 229 ms, write 16080 ms, index 710 ms: 23 candidates, 83020 repo-hours, 988 releases; waiting 517 ms; commit 8378 ms)
2026-11-08T03:00:32Z [load_day 2023-11-20] 2023-11-20 hour 9: 210389 events in 5 files, snapshot 1115320484404530723 (task 29777 ms: download 20680 ms overlapping gunzip 2183 ms, parse 9263 ms, sort 174 ms, write 16929 ms, index 1018 ms: 12 candidates, 87805 repo-hours, 1279 releases; waiting 18 ms; commit 4690 ms)
2026-11-08T03:00:51Z [load_day 2023-11-20] 2023-11-20 hour 10: 207451 events in 5 files, snapshot 8179311417871000590 (task 27481 ms: download 19116 ms overlapping gunzip 1751 ms, parse 7706 ms, sort 179 ms, write 16064 ms, index 778 ms: 17 candidates, 86754 repo-hours, 1050 releases; waiting 606 ms; commit 4664 ms)
2026-11-08T03:01:19Z [load_day 2023-11-20] 2023-11-20 hour 11: 204611 events in 5 files, snapshot 5252453887518264750 (task 28152 ms: download 14312 ms overlapping gunzip 2211 ms, parse 9334 ms, sort 247 ms, write 15414 ms, index 730 ms: 10 candidates, 82842 repo-hours, 1108 releases; waiting 17 ms; commit 21851 ms)
2026-11-08T03:01:22Z [load_day 2023-11-20] 2023-11-20 hour 12: 250035 events in 6 files, snapshot 997165310278005597 (task 30580 ms: download 19234 ms overlapping gunzip 2206 ms, parse 9533 ms, sort 213 ms, write 17528 ms, index 873 ms: 17 candidates, 114278 repo-hours, 1077 releases; waiting 25 ms; commit 2899 ms)
2026-11-08T03:01:31Z [load_day 2023-11-20] 2023-11-20 hour 13: 237173 events in 5 files, snapshot 4634463850601780285 (task 27390 ms: download 19652 ms overlapping gunzip 1827 ms, parse 8048 ms, sort 185 ms, write 16303 ms, index 784 ms: 14 candidates, 109891 repo-hours, 1186 releases; waiting 23 ms; commit 5322 ms)
2026-11-08T03:01:55Z [load_day 2023-11-20] 2023-11-20 hour 14: 235177 events in 5 files, snapshot 9085599477689188923 (task 24220 ms: download 15898 ms overlapping gunzip 1351 ms, parse 5799 ms, sort 145 ms, write 15050 ms, index 1123 ms: 14 candidates, 98267 repo-hours, 1183 releases; waiting 492 ms; commit 10233 ms)
2026-11-08T03:02:00Z [load_day 2023-11-20] 2023-11-20 hour 15: 231375 events in 5 files, snapshot 1147117935538553598 (task 26188 ms: download 15019 ms overlapping gunzip 1548 ms, parse 6712 ms, sort 203 ms, write 16280 ms, index 1216 ms: 17 candidates, 99285 repo-hours, 1292 releases; waiting 22 ms; commit 5749 ms)
2026-11-08T03:02:20Z [load_day 2023-11-20] 2023-11-20 hour 16: 220943 events in 5 files, snapshot 735599885044941980 (task 30099 ms: download 20530 ms overlapping gunzip 1921 ms, parse 8312 ms, sort 203 ms, write 18560 ms, index 857 ms: 22 candidates, 95561 repo-hours, 1110 releases; waiting 38 ms; commit 4119 ms)
2026-11-08T03:02:26Z [load_day 2023-11-20] 2023-11-20 hour 17: 208248 events in 5 files, snapshot 4978371853207206532 (task 27537 ms: download 19835 ms overlapping gunzip 1909 ms, parse 8478 ms, sort 206 ms, write 16016 ms, index 717 ms: 13 candidates, 87984 repo-hours, 1059 releases; waiting 30 ms; commit 2461 ms)
2026-11-08T03:02:47Z [load_day 2023-11-20] 2023-11-20 hour 18: 212309 events in 5 files, snapshot 7257732864783175705 (task 23563 ms: download 16541 ms overlapping gunzip 1545 ms, parse 6512 ms, sort 160 ms, write 14501 ms, index 579 ms: 11 candidates, 91679 repo-hours, 1137 releases; waiting 29 ms; commit 5868 ms)
2026-11-08T03:02:53Z [load_day 2023-11-20] 2023-11-20 hour 19: 204628 events in 5 files, snapshot 4923966642582771167 (task 23276 ms: download 13158 ms overlapping gunzip 1684 ms, parse 6982 ms, sort 176 ms, write 13563 ms, index 652 ms: 14 candidates, 83323 repo-hours, 1032 releases; waiting 27 ms; commit 4855 ms)
2026-11-08T03:03:13Z [load_day 2023-11-20] 2023-11-20 hour 20: 201415 events in 5 files, snapshot 2111443789450174096 (task 25612 ms: download 14558 ms overlapping gunzip 1879 ms, parse 7521 ms, sort 249 ms, write 15156 ms, index 574 ms: 10 candidates, 81910 repo-hours, 954 releases; waiting 17 ms; commit 5448 ms)
2026-11-08T03:03:16Z [load_day 2023-11-20] 2023-11-20 hour 21: 191643 events in 4 files, snapshot 317243054501459545 (task 23692 ms: download 13654 ms overlapping gunzip 1468 ms, parse 6559 ms, sort 163 ms, write 14474 ms, index 798 ms: 6 candidates, 74133 repo-hours, 1082 releases; waiting 12 ms; commit 2930 ms)
2026-11-08T03:03:34Z [load_day 2023-11-20] 2023-11-20 hour 22: 179776 events in 4 files, snapshot 5285529316457257530 (task 19935 ms: download 12320 ms overlapping gunzip 1253 ms, parse 5294 ms, sort 133 ms, write 12030 ms, index 481 ms: 9 candidates, 70158 repo-hours, 1037 releases; waiting 466 ms; commit 5203 ms)
2026-11-08T03:03:37Z [load_day 2023-11-20] 2023-11-20 hour 23: 169677 events in 4 files, snapshot 8822725645836199353 (task 17824 ms: download 10894 ms overlapping gunzip 1119 ms, parse 4586 ms, sort 158 ms, write 11267 ms, index 482 ms: 4 candidates, 64438 repo-hours, 790 releases; waiting 14 ms; commit 2435 ms)
2026-11-08T03:03:45Z [load_day 2023-11-20] 2023-11-20: repo_hour merged from 24 files to 1 (1972237 rows, 8596 ms)
2026-11-08T03:03:48Z [load_day 2023-11-20] 2023-11-20: index committed in 10797 ms (24 + 1 + 24 files)
2026-11-08T03:03:48Z [load_day 2023-11-20] 2023-11-20: 4717322 events in 24 hours
2026-11-08T03:03:48Z [load_day 2023-11-20] attempt 1 succeeded