orcaLog in

Runs · run 1717

load_day 2023-11-09, attempt 1 succeeded

Host
3c2b29713ce8:534842
Started
2026-11-08T02:37:35Z
Ended
2026-11-08T02:41:57Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1717/load_day/2023-11-09/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:37:35Z [load_day 2023-11-09] attempt 1 started
2026-11-08T02:37:37Z [load_day 2023-11-09] 2023-11-09: 2 hours in flight, about 1200 MB
2026-11-08T02:38:00Z [load_day 2023-11-09] 2023-11-09 hour 0: 177351 events in 4 files, snapshot 5610345185021694454 (task 18200 ms: download 10820 ms overlapping gunzip 1145 ms, parse 5615 ms, sort 180 ms, write 10354 ms, index 671 ms: 9 candidates, 95690 repo-hours, 1229 releases; waiting 23 ms; commit 2755 ms)
2026-11-08T02:38:03Z [load_day 2023-11-09] 2023-11-09 hour 1: 173246 events in 4 files, snapshot 5152978607009462998 (task 18192 ms: download 11055 ms overlapping gunzip 1137 ms, parse 5357 ms, sort 151 ms, write 10659 ms, index 664 ms: 3 candidates, 96837 repo-hours, 1029 releases; waiting 28 ms; commit 2899 ms)
2026-11-08T02:38:16Z [load_day 2023-11-09] 2023-11-09 hour 2: 148978 events in 3 files, snapshot 1271481410800266351 (task 14017 ms: download 6043 ms overlapping gunzip 1000 ms, parse 4407 ms, sort 138 ms, write 7774 ms, index 490 ms: 5 candidates, 64269 repo-hours, 887 releases; waiting 18 ms; commit 3656 ms)
2026-11-08T02:38:19Z [load_day 2023-11-09] 2023-11-09 hour 3: 142617 events in 3 files, snapshot 5795828888610608828 (task 13182 ms: download 5610 ms overlapping gunzip 762 ms, parse 3160 ms, sort 101 ms, write 8353 ms, index 579 ms: 8 candidates, 62407 repo-hours, 946 releases; waiting 19 ms; commit 3471 ms)
2026-11-08T02:38:32Z [load_day 2023-11-09] 2023-11-09 hour 4: 136083 events in 3 files, snapshot 1031902411214321700 (task 13119 ms: download 5479 ms overlapping gunzip 771 ms, parse 3371 ms, sort 126 ms, write 7609 ms, index 458 ms: 6 candidates, 57191 repo-hours, 711 releases; waiting 509 ms; commit 5436 ms)
2026-11-08T02:38:35Z [load_day 2023-11-09] 2023-11-09 hour 5: 142502 events in 3 files, snapshot 7905691133349965721 (task 14320 ms: download 5485 ms overlapping gunzip 814 ms, parse 3465 ms, sort 90 ms, write 9076 ms, index 672 ms: 6 candidates, 61729 repo-hours, 715 releases; waiting 17 ms; commit 2588 ms)
2026-11-08T02:38:47Z [load_day 2023-11-09] 2023-11-09 hour 6: 158391 events in 4 files, snapshot 5549923341645290833 (task 16519 ms: download 6442 ms overlapping gunzip 956 ms, parse 4476 ms, sort 183 ms, write 10090 ms, index 602 ms: 7 candidates, 73796 repo-hours, 723 releases; waiting 15 ms; commit 3147 ms)
2026-11-08T02:38:55Z [load_day 2023-11-09] 2023-11-09 hour 7: 169454 events in 4 files, snapshot 3687386513423571054 (task 18071 ms: download 10870 ms overlapping gunzip 1085 ms, parse 4905 ms, sort 163 ms, write 10521 ms, index 626 ms: 4 candidates, 74545 repo-hours, 853 releases; waiting 498 ms; commit 2920 ms)
2026-11-08T02:39:09Z [load_day 2023-11-09] 2023-11-09 hour 8: 185122 events in 4 files, snapshot 10444889687573301 (task 20651 ms: download 12179 ms overlapping gunzip 1286 ms, parse 5699 ms, sort 121 ms, write 12554 ms, index 762 ms: 12 candidates, 82349 repo-hours, 1012 releases; waiting 21 ms; commit 2839 ms)
2026-11-08T02:39:18Z [load_day 2023-11-09] 2023-11-09 hour 9: 197719 events in 4 files, snapshot 8206578967508241299 (task 22103 ms: download 12274 ms overlapping gunzip 1330 ms, parse 6476 ms, sort 157 ms, write 13281 ms, index 630 ms: 7 candidates, 86423 repo-hours, 1203 releases; waiting 19 ms; commit 3264 ms)
2026-11-08T02:39:32Z [load_day 2023-11-09] 2023-11-09 hour 10: 196851 events in 4 files, snapshot 7442730441458809770 (task 21709 ms: download 11940 ms overlapping gunzip 1412 ms, parse 6283 ms, sort 174 ms, write 12940 ms, index 687 ms: 9 candidates, 86141 repo-hours, 1269 releases; waiting 15 ms; commit 3006 ms)
2026-11-08T02:39:41Z [load_day 2023-11-09] 2023-11-09 hour 11: 193765 events in 4 files, snapshot 6592886759566242953 (task 21927 ms: download 12253 ms overlapping gunzip 1292 ms, parse 5755 ms, sort 135 ms, write 13265 ms, index 909 ms: 6 candidates, 82471 repo-hours, 1161 releases; waiting 17 ms; commit 2822 ms)
2026-11-08T02:39:55Z [load_day 2023-11-09] 2023-11-09 hour 12: 244754 events in 5 files, snapshot 4702309777002007196 (task 22324 ms: download 13945 ms overlapping gunzip 1319 ms, parse 6352 ms, sort 178 ms, write 13448 ms, index 798 ms: 11 candidates, 112988 repo-hours, 1165 releases; waiting 24 ms; commit 2687 ms)
2026-11-08T02:40:03Z [load_day 2023-11-09] 2023-11-09 hour 13: 222150 events in 5 files, snapshot 5582343951355719476 (task 21604 ms: download 14651 ms overlapping gunzip 1475 ms, parse 6608 ms, sort 178 ms, write 12367 ms, index 755 ms: 5 candidates, 109194 repo-hours, 1148 releases; waiting 15 ms; commit 2473 ms)
2026-11-08T02:40:15Z [load_day 2023-11-09] 2023-11-09 hour 14: 220417 events in 5 files, snapshot 2773763803512263742 (task 19485 ms: download 13381 ms overlapping gunzip 1264 ms, parse 5565 ms, sort 137 ms, write 11667 ms, index 642 ms: 6 candidates, 97111 repo-hours, 1402 releases; waiting 18 ms; commit 2390 ms)
2026-11-08T02:40:26Z [load_day 2023-11-09] 2023-11-09 hour 15: 220512 events in 5 files, snapshot 4296247732986714204 (task 21321 ms: download 12286 ms overlapping gunzip 1177 ms, parse 5189 ms, sort 121 ms, write 13589 ms, index 1027 ms: 9 candidates, 98425 repo-hours, 1476 releases; waiting 16 ms; commit 2842 ms)
2026-11-08T02:40:41Z [load_day 2023-11-09] 2023-11-09 hour 16: 211913 events in 5 files, snapshot 4118243426934748436 (task 23449 ms: download 16507 ms overlapping gunzip 1305 ms, parse 6240 ms, sort 171 ms, write 14506 ms, index 1026 ms: 8 candidates, 94807 repo-hours, 1155 releases; waiting 15 ms; commit 3176 ms)
2026-11-08T02:40:49Z [load_day 2023-11-09] 2023-11-09 hour 17: 203229 events in 5 files, snapshot 5379900367958579306 (task 21122 ms: download 11120 ms overlapping gunzip 1269 ms, parse 5637 ms, sort 141 ms, write 13220 ms, index 637 ms: 12 candidates, 86979 repo-hours, 1086 releases; waiting 15 ms; commit 3126 ms)
2026-11-08T02:41:02Z [load_day 2023-11-09] 2023-11-09 hour 18: 202431 events in 5 files, snapshot 2776166416819318148 (task 20552 ms: download 10987 ms overlapping gunzip 1258 ms, parse 5698 ms, sort 169 ms, write 12472 ms, index 741 ms: 16 candidates, 89246 repo-hours, 1247 releases; waiting 16 ms; commit 2857 ms)
2026-11-08T02:41:11Z [load_day 2023-11-09] 2023-11-09 hour 19: 191343 events in 4 files, snapshot 4781976202758885591 (task 20856 ms: download 11149 ms overlapping gunzip 1297 ms, parse 5685 ms, sort 193 ms, write 12753 ms, index 718 ms: 7 candidates, 77464 repo-hours, 1172 releases; waiting 20 ms; commit 3091 ms)
2026-11-08T02:41:24Z [load_day 2023-11-09] 2023-11-09 hour 20: 176736 events in 4 files, snapshot 779996241099852545 (task 18857 ms: download 11643 ms overlapping gunzip 976 ms, parse 4415 ms, sort 148 ms, write 11868 ms, index 614 ms: 7 candidates, 75943 repo-hours, 1013 releases; waiting 532 ms; commit 4827 ms)
2026-11-08T02:41:31Z [load_day 2023-11-09] 2023-11-09 hour 21: 169173 events in 4 files, snapshot 2848686732347856472 (task 19419 ms: download 10645 ms overlapping gunzip 1268 ms, parse 5740 ms, sort 183 ms, write 11008 ms, index 975 ms: 7 candidates, 70469 repo-hours, 1055 releases; waiting 20 ms; commit 2912 ms)
2026-11-08T02:41:41Z [load_day 2023-11-09] 2023-11-09 hour 22: 163631 events in 4 files, snapshot 5772698166765257639 (task 17729 ms: download 10904 ms overlapping gunzip 1392 ms, parse 5307 ms, sort 114 ms, write 10149 ms, index 536 ms: 9 candidates, 66900 repo-hours, 991 releases; waiting 18 ms; commit 2837 ms)
2026-11-08T02:41:46Z [load_day 2023-11-09] 2023-11-09 hour 23: 149006 events in 3 files, snapshot 319231298230558851 (task 13784 ms: download 5640 ms overlapping gunzip 780 ms, parse 3312 ms, sort 98 ms, write 8947 ms, index 437 ms: 6 candidates, 59718 repo-hours, 853 releases; waiting 22 ms; commit 2483 ms)
2026-11-08T02:41:55Z [load_day 2023-11-09] 2023-11-09: repo_hour merged from 24 files to 1 (1963092 rows, 8943 ms)
2026-11-08T02:41:57Z [load_day 2023-11-09] 2023-11-09: index committed in 10908 ms (24 + 1 + 24 files)
2026-11-08T02:41:57Z [load_day 2023-11-09] 2023-11-09: 4397374 events in 24 hours
2026-11-08T02:41:57Z [load_day 2023-11-09] attempt 1 succeeded