orcaLog in

Runs · run 1486

load_day 2023-09-11, attempt 1 succeeded

Host
3c2b29713ce8:316498
Started
2026-11-08T00:42:38Z
Ended
2026-11-08T00:47:09Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1486/load_day/2023-09-11/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:42:38Z [load_day 2023-09-11] attempt 1 started
2026-11-08T00:42:40Z [load_day 2023-09-11] 2023-09-11: 2 hours in flight, about 1200 MB
2026-11-08T00:43:01Z [load_day 2023-09-11] 2023-09-11 hour 0: 156181 events in 4 files, snapshot 3753240473428515102 (task 14983 ms: download 6552 ms overlapping gunzip 980 ms, parse 4488 ms, sort 146 ms, write 8003 ms, index 593 ms: 10 candidates, 84422 repo-hours, 804 releases; waiting 489 ms; commit 4524 ms)
2026-11-08T00:43:06Z [load_day 2023-09-11] 2023-09-11 hour 1: 167363 events in 4 files, snapshot 3199338184958483800 (task 17307 ms: download 10517 ms overlapping gunzip 1194 ms, parse 5101 ms, sort 183 ms, write 9342 ms, index 697 ms: 7 candidates, 89459 repo-hours, 754 releases; waiting 494 ms; commit 4781 ms)
2026-11-08T00:43:14Z [load_day 2023-09-11] 2023-09-11 hour 2: 141107 events in 3 files, snapshot 1336289550574577508 (task 14261 ms: download 6433 ms overlapping gunzip 1005 ms, parse 4124 ms, sort 137 ms, write 7708 ms, index 492 ms: 5 candidates, 58205 repo-hours, 769 releases; waiting 525 ms; commit 2417 ms)
2026-11-08T00:43:19Z [load_day 2023-09-11] 2023-09-11 hour 3: 140283 events in 3 files, snapshot 8108098242436207006 (task 14042 ms: download 6589 ms overlapping gunzip 943 ms, parse 4088 ms, sort 134 ms, write 7507 ms, index 557 ms: 9 candidates, 58388 repo-hours, 840 releases; waiting 521 ms; commit 2701 ms)
2026-11-08T00:43:32Z [load_day 2023-09-11] 2023-09-11 hour 4: 146047 events in 3 files, snapshot 5549535616485940268 (task 16395 ms: download 7324 ms overlapping gunzip 1225 ms, parse 4853 ms, sort 146 ms, write 8949 ms, index 500 ms: 6 candidates, 56123 repo-hours, 623 releases; waiting 496 ms; commit 2730 ms)
2026-11-08T00:43:36Z [load_day 2023-09-11] 2023-09-11 hour 5: 151557 events in 4 files, snapshot 1963782934834785350 (task 16185 ms: download 6919 ms overlapping gunzip 987 ms, parse 4471 ms, sort 196 ms, write 9262 ms, index 478 ms: 4 candidates, 62619 repo-hours, 752 releases; waiting 496 ms; commit 2437 ms)
2026-11-08T00:43:54Z [load_day 2023-09-11] 2023-09-11 hour 6: 181584 events in 4 files, snapshot 6278305237780956787 (task 19878 ms: download 11442 ms overlapping gunzip 1323 ms, parse 5831 ms, sort 121 ms, write 11221 ms, index 633 ms: 7 candidates, 80634 repo-hours, 854 releases; waiting 475 ms; commit 3281 ms)
2026-11-08T00:43:59Z [load_day 2023-09-11] 2023-09-11 hour 7: 183456 events in 4 files, snapshot 5368118842346395852 (task 22116 ms: download 13212 ms overlapping gunzip 1340 ms, parse 5728 ms, sort 222 ms, write 13513 ms, index 669 ms: 6 candidates, 79229 repo-hours, 929 releases; waiting 387 ms; commit 2640 ms)
2026-11-08T00:44:20Z [load_day 2023-09-11] 2023-09-11 hour 8: 192287 events in 4 files, snapshot 5842516777101639095 (task 24230 ms: download 13628 ms overlapping gunzip 1516 ms, parse 6389 ms, sort 297 ms, write 14605 ms, index 657 ms: 14 candidates, 83724 repo-hours, 1008 releases; waiting 492 ms; commit 4029 ms)
2026-11-08T00:44:24Z [load_day 2023-09-11] 2023-09-11 hour 9: 206194 events in 5 files, snapshot 4724259277075218424 (task 23930 ms: download 16789 ms overlapping gunzip 1450 ms, parse 5875 ms, sort 168 ms, write 14780 ms, index 789 ms: 9 candidates, 86380 repo-hours, 1303 releases; waiting 564 ms; commit 2558 ms)
2026-11-08T00:44:43Z [load_day 2023-09-11] 2023-09-11 hour 10: 197171 events in 4 files, snapshot 1887301161354571926 (task 23449 ms: download 13406 ms overlapping gunzip 1519 ms, parse 6475 ms, sort 273 ms, write 12945 ms, index 538 ms: 13 candidates, 82894 repo-hours, 1098 releases; waiting 864 ms; commit 3070 ms)
2026-11-08T00:44:46Z [load_day 2023-09-11] 2023-09-11 hour 11: 198544 events in 4 files, snapshot 5385780887959122323 (task 21179 ms: download 13060 ms overlapping gunzip 1303 ms, parse 5808 ms, sort 157 ms, write 12578 ms, index 647 ms: 17 candidates, 84412 repo-hours, 972 releases; waiting 404 ms; commit 2223 ms)
2026-11-08T00:45:05Z [load_day 2023-09-11] 2023-09-11 hour 12: 249075 events in 5 files, snapshot 4004721902119506703 (task 21104 ms: download 11956 ms overlapping gunzip 1211 ms, parse 5411 ms, sort 167 ms, write 12562 ms, index 1100 ms: 12 candidates, 114360 repo-hours, 1117 releases; waiting 342 ms; commit 3053 ms)
2026-11-08T00:45:11Z [load_day 2023-09-11] 2023-09-11 hour 13: 227891 events in 5 files, snapshot 716333608153459471 (task 22878 ms: download 14134 ms overlapping gunzip 1319 ms, parse 5753 ms, sort 147 ms, write 14144 ms, index 812 ms: 14 candidates, 108451 repo-hours, 1142 releases; waiting 485 ms; commit 2941 ms)
2026-11-08T00:45:36Z [load_day 2023-09-11] 2023-09-11 hour 14: 218883 events in 5 files, snapshot 6288214914040584293 (task 24603 ms: download 16533 ms overlapping gunzip 1313 ms, parse 5876 ms, sort 210 ms, write 15224 ms, index 814 ms: 16 candidates, 96631 repo-hours, 1197 releases; waiting 520 ms; commit 8566 ms)
2026-11-08T00:45:41Z [load_day 2023-09-11] 2023-09-11 hour 15: 200542 events in 5 files, snapshot 4713604207963068843 (task 22256 ms: download 12384 ms overlapping gunzip 1275 ms, parse 5558 ms, sort 152 ms, write 13090 ms, index 1044 ms: 9 candidates, 91078 repo-hours, 1119 releases; waiting 705 ms; commit 4624 ms)
2026-11-08T00:45:53Z [load_day 2023-09-11] 2023-09-11 hour 16: 196638 events in 4 files, snapshot 5730633962426887988 (task 21888 ms: download 12616 ms overlapping gunzip 1377 ms, parse 6037 ms, sort 146 ms, write 12835 ms, index 648 ms: 4 candidates, 87020 repo-hours, 942 releases; waiting 522 ms; commit 2226 ms)
2026-11-08T00:46:02Z [load_day 2023-09-11] 2023-09-11 hour 17: 196584 events in 4 files, snapshot 1489371737810653919 (task 22357 ms: download 12658 ms overlapping gunzip 1288 ms, parse 5841 ms, sort 242 ms, write 13386 ms, index 779 ms: 7 candidates, 82469 repo-hours, 877 releases; waiting 493 ms; commit 2375 ms)
2026-11-08T00:46:18Z [load_day 2023-09-11] 2023-09-11 hour 18: 198541 events in 4 files, snapshot 1253059585691383018 (task 21830 ms: download 11987 ms overlapping gunzip 1246 ms, parse 5285 ms, sort 214 ms, write 13483 ms, index 744 ms: 14 candidates, 85993 repo-hours, 1087 releases; waiting 509 ms; commit 4501 ms)
2026-11-08T00:46:24Z [load_day 2023-09-11] 2023-09-11 hour 19: 188361 events in 4 files, snapshot 8507074048735641716 (task 21019 ms: download 12093 ms overlapping gunzip 1220 ms, parse 5100 ms, sort 121 ms, write 12836 ms, index 822 ms: 12 candidates, 76544 repo-hours, 991 releases; waiting 620 ms; commit 2315 ms)
2026-11-08T00:46:37Z [load_day 2023-09-11] 2023-09-11 hour 20: 178924 events in 4 files, snapshot 3860376273514723283 (task 19863 ms: download 11623 ms overlapping gunzip 1199 ms, parse 4975 ms, sort 159 ms, write 12039 ms, index 559 ms: 7 candidates, 74113 repo-hours, 827 releases; waiting 500 ms; commit 2536 ms)
2026-11-08T00:46:44Z [load_day 2023-09-11] 2023-09-11 hour 21: 168453 events in 4 files, snapshot 1267245872572128358 (task 18533 ms: download 11128 ms overlapping gunzip 1037 ms, parse 4498 ms, sort 190 ms, write 11604 ms, index 465 ms: 7 candidates, 65184 repo-hours, 989 releases; waiting 505 ms; commit 2404 ms)
2026-11-08T00:46:53Z [load_day 2023-09-11] 2023-09-11 hour 22: 153398 events in 4 files, snapshot 5086692036431600639 (task 16222 ms: download 6979 ms overlapping gunzip 1001 ms, parse 4124 ms, sort 156 ms, write 10005 ms, index 530 ms: 9 candidates, 57732 repo-hours, 866 releases; waiting 195 ms; commit 2091 ms)
2026-11-08T00:46:59Z [load_day 2023-09-11] 2023-09-11 hour 23: 143916 events in 3 files, snapshot 1657232296669773114 (task 14063 ms: download 6598 ms overlapping gunzip 982 ms, parse 4023 ms, sort 115 ms, write 7727 ms, index 371 ms: 8 candidates, 55964 repo-hours, 680 releases; waiting 508 ms; commit 2043 ms)
2026-11-08T00:47:07Z [load_day 2023-09-11] 2023-09-11: repo_hour merged from 24 files to 1 (1902028 rows, 8714 ms)
2026-11-08T00:47:09Z [load_day 2023-09-11] 2023-09-11: index committed in 10283 ms (24 + 1 + 24 files)
2026-11-08T00:47:09Z [load_day 2023-09-11] 2023-09-11: 4382980 events in 24 hours
2026-11-08T00:47:09Z [load_day 2023-09-11] attempt 1 succeeded