orcaLog in

Runs · run 1626

load_day 2023-10-15, attempt 1 succeeded

Host
3c2b29713ce8:441508
Started
2026-11-08T01:48:57Z
Ended
2026-11-08T01:51:40Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1626/load_day/2023-10-15/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:48:57Z [load_day 2023-10-15] attempt 1 started
2026-11-08T01:48:59Z [load_day 2023-10-15] 2023-10-15: 2 hours in flight, about 1200 MB
2026-11-08T01:49:11Z [load_day 2023-10-15] 2023-10-15 hour 0: 122243 events in 3 files, snapshot 500420679560150169 (task 8277 ms: download 1953 ms overlapping gunzip 434 ms, parse 2137 ms, sort 76 ms, write 4301 ms, index 504 ms: 7 candidates, 68432 repo-hours, 885 releases; waiting 508 ms; commit 2233 ms)
2026-11-08T01:49:13Z [load_day 2023-10-15] 2023-10-15 hour 1: 124415 events in 3 files, snapshot 6158631340708461740 (task 7778 ms: download 1742 ms overlapping gunzip 414 ms, parse 1947 ms, sort 67 ms, write 4224 ms, index 491 ms: 5 candidates, 73751 repo-hours, 663 releases; waiting 357 ms; commit 2469 ms)
2026-11-08T01:49:20Z [load_day 2023-10-15] 2023-10-15 hour 2: 95397 events in 2 files, snapshot 8203475864047966990 (task 7889 ms: download 2631 ms overlapping gunzip 370 ms, parse 1582 ms, sort 61 ms, write 4147 ms, index 489 ms: 4 candidates, 40185 repo-hours, 624 releases; waiting 819 ms; commit 2632 ms)
2026-11-08T01:49:23Z [load_day 2023-10-15] 2023-10-15 hour 3: 95621 events in 2 files, snapshot 8398378124568331986 (task 7627 ms: download 1587 ms overlapping gunzip 444 ms, parse 2141 ms, sort 66 ms, write 3991 ms, index 273 ms: 1 candidates, 40639 repo-hours, 716 releases; waiting 519 ms; commit 2630 ms)
2026-11-08T01:49:29Z [load_day 2023-10-15] 2023-10-15 hour 4: 96322 events in 2 files, snapshot 5531740754974391317 (task 7824 ms: download 2379 ms overlapping gunzip 419 ms, parse 2087 ms, sort 63 ms, write 4178 ms, index 277 ms: 1 candidates, 40821 repo-hours, 506 releases; waiting 490 ms; commit 2396 ms)
2026-11-08T01:49:34Z [load_day 2023-10-15] 2023-10-15 hour 5: 102450 events in 3 files, snapshot 8074512960225195974 (task 8077 ms: download 1737 ms overlapping gunzip 461 ms, parse 2063 ms, sort 67 ms, write 4244 ms, index 405 ms: 0 candidates, 44286 repo-hours, 550 releases; waiting 556 ms; commit 4081 ms)
2026-11-08T01:49:40Z [load_day 2023-10-15] 2023-10-15 hour 6: 116144 events in 3 files, snapshot 8435261967007720803 (task 9571 ms: download 2413 ms overlapping gunzip 459 ms, parse 2040 ms, sort 73 ms, write 5482 ms, index 663 ms: 1 candidates, 52928 repo-hours, 733 releases; waiting 578 ms; commit 2646 ms)
2026-11-08T01:49:44Z [load_day 2023-10-15] 2023-10-15 hour 7: 114740 events in 3 files, snapshot 3514075118309716321 (task 10414 ms: download 2342 ms overlapping gunzip 518 ms, parse 2538 ms, sort 131 ms, write 5971 ms, index 432 ms: 4 candidates, 49278 repo-hours, 650 releases; waiting 497 ms; commit 2645 ms)
2026-11-08T01:49:52Z [load_day 2023-10-15] 2023-10-15 hour 8: 121548 events in 3 files, snapshot 8142323901798190864 (task 10737 ms: download 2027 ms overlapping gunzip 648 ms, parse 2847 ms, sort 70 ms, write 5857 ms, index 417 ms: 3 candidates, 52536 repo-hours, 673 releases; waiting 582 ms; commit 3375 ms)
2026-11-08T01:49:57Z [load_day 2023-10-15] 2023-10-15 hour 9: 128566 events in 3 files, snapshot 6801183371354337524 (task 12250 ms: download 3261 ms overlapping gunzip 668 ms, parse 2916 ms, sort 119 ms, write 7306 ms, index 385 ms: 5 candidates, 54995 repo-hours, 822 releases; waiting 526 ms; commit 2494 ms)
2026-11-08T01:50:05Z [load_day 2023-10-15] 2023-10-15 hour 10: 128378 events in 3 files, snapshot 417076889533507845 (task 10821 ms: download 1974 ms overlapping gunzip 634 ms, parse 2839 ms, sort 85 ms, write 6656 ms, index 394 ms: 2 candidates, 55473 repo-hours, 654 releases; waiting 16 ms; commit 4101 ms)
2026-11-08T01:50:11Z [load_day 2023-10-15] 2023-10-15 hour 11: 129172 events in 3 files, snapshot 1796746522097476780 (task 11105 ms: download 1788 ms overlapping gunzip 679 ms, parse 3119 ms, sort 109 ms, write 6568 ms, index 403 ms: 7 candidates, 54128 repo-hours, 802 releases; waiting 26 ms; commit 3773 ms)
2026-11-08T01:50:21Z [load_day 2023-10-15] 2023-10-15 hour 12: 206163 events in 5 files, snapshot 2277384228203034133 (task 15037 ms: download 4443 ms overlapping gunzip 841 ms, parse 4091 ms, sort 165 ms, write 8944 ms, index 790 ms: 4 candidates, 88335 repo-hours, 786 releases; waiting 18 ms; commit 2976 ms)
2026-11-08T01:50:27Z [load_day 2023-10-15] 2023-10-15 hour 13: 169527 events in 4 files, snapshot 1615273549206917226 (task 13603 ms: download 4598 ms overlapping gunzip 842 ms, parse 3863 ms, sort 111 ms, write 7835 ms, index 713 ms: 4 candidates, 83132 repo-hours, 800 releases; waiting 24 ms; commit 4737 ms)
2026-11-08T01:50:36Z [load_day 2023-10-15] 2023-10-15 hour 14: 164031 events in 4 files, snapshot 4164929527047732268 (task 13991 ms: download 4457 ms overlapping gunzip 755 ms, parse 3176 ms, sort 155 ms, write 9160 ms, index 505 ms: 7 candidates, 68518 repo-hours, 799 releases; waiting 39 ms; commit 2793 ms)
2026-11-08T01:50:42Z [load_day 2023-10-15] 2023-10-15 hour 15: 166224 events in 4 files, snapshot 698951149372333129 (task 15865 ms: download 6270 ms overlapping gunzip 993 ms, parse 4316 ms, sort 138 ms, write 8903 ms, index 709 ms: 4 candidates, 71043 repo-hours, 1024 releases; waiting 518 ms; commit 2722 ms)
2026-11-08T01:50:50Z [load_day 2023-10-15] 2023-10-15 hour 16: 159084 events in 4 files, snapshot 7528782983695603690 (task 12790 ms: download 5035 ms overlapping gunzip 787 ms, parse 3477 ms, sort 107 ms, write 7641 ms, index 547 ms: 8 candidates, 68968 repo-hours, 796 releases; waiting 22 ms; commit 3065 ms)
2026-11-08T01:50:57Z [load_day 2023-10-15] 2023-10-15 hour 17: 153354 events in 4 files, snapshot 9037946737588046377 (task 13100 ms: download 4862 ms overlapping gunzip 852 ms, parse 3719 ms, sort 117 ms, write 7591 ms, index 582 ms: 9 candidates, 65748 repo-hours, 759 releases; waiting 30 ms; commit 4259 ms)
2026-11-08T01:51:04Z [load_day 2023-10-15] 2023-10-15 hour 18: 155899 events in 4 files, snapshot 4817433357504869259 (task 13069 ms: download 4366 ms overlapping gunzip 791 ms, parse 3577 ms, sort 135 ms, write 7774 ms, index 571 ms: 8 candidates, 70215 repo-hours, 833 releases; waiting 25 ms; commit 2953 ms)
2026-11-08T01:51:10Z [load_day 2023-10-15] 2023-10-15 hour 19: 138802 events in 3 files, snapshot 5978948567926832824 (task 13492 ms: download 4998 ms overlapping gunzip 920 ms, parse 4120 ms, sort 136 ms, write 7563 ms, index 540 ms: 5 candidates, 58145 repo-hours, 817 releases; waiting 22 ms; commit 2443 ms)
2026-11-08T01:51:16Z [load_day 2023-10-15] 2023-10-15 hour 20: 132827 events in 3 files, snapshot 8831960756961384513 (task 11204 ms: download 1378 ms overlapping gunzip 652 ms, parse 2912 ms, sort 122 ms, write 6808 ms, index 465 ms: 4 candidates, 56567 repo-hours, 814 releases; waiting 17 ms; commit 2497 ms)
2026-11-08T01:51:22Z [load_day 2023-10-15] 2023-10-15 hour 21: 118597 events in 3 files, snapshot 5303039565604980325 (task 10698 ms: download 2155 ms overlapping gunzip 513 ms, parse 2373 ms, sort 99 ms, write 6353 ms, index 585 ms: 7 candidates, 49655 repo-hours, 814 releases; waiting 484 ms; commit 3140 ms)
2026-11-08T01:51:26Z [load_day 2023-10-15] 2023-10-15 hour 22: 106234 events in 3 files, snapshot 6558616805657426222 (task 9083 ms: download 768 ms overlapping gunzip 460 ms, parse 2064 ms, sort 71 ms, write 5888 ms, index 390 ms: 7 candidates, 44149 repo-hours, 714 releases; waiting 19 ms; commit 2209 ms)
2026-11-08T01:51:31Z [load_day 2023-10-15] 2023-10-15 hour 23: 101411 events in 3 files, snapshot 706850630433997746 (task 7680 ms: download 1158 ms overlapping gunzip 527 ms, parse 2317 ms, sort 65 ms, write 4191 ms, index 354 ms: 3 candidates, 43114 repo-hours, 635 releases; waiting 18 ms; commit 2671 ms)
2026-11-08T01:51:38Z [load_day 2023-10-15] 2023-10-15: repo_hour merged from 24 files to 1 (1395041 rows, 6905 ms)
2026-11-08T01:51:40Z [load_day 2023-10-15] 2023-10-15: index committed in 9084 ms (23 + 1 + 24 files)
2026-11-08T01:51:40Z [load_day 2023-10-15] 2023-10-15: 3147149 events in 24 hours
2026-11-08T01:51:40Z [load_day 2023-10-15] attempt 1 succeeded