orcaLog in

Runs · run 1218

load_day 2023-06-27, attempt 1 succeeded

Host
3c2b29713ce8:48136
Started
2026-11-07T22:10:45Z
Ended
2026-11-07T22:16:11Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1218/load_day/2023-06-27/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T22:10:45Z [load_day 2023-06-27] attempt 1 started
2026-11-07T22:10:47Z [load_day 2023-06-27] 2023-06-27: 1 hours in flight, about 750 MB
2026-11-07T22:11:00Z [load_day 2023-06-27] 2023-06-27 hour 0: 148404 events in 3 files, snapshot 7162649358565635203 (task 10543 ms: download 4831 ms overlapping gunzip 697 ms, parse 3203 ms, sort 97 ms, write 5225 ms, index 454 ms: 10 candidates, 71074 repo-hours, 802 releases; waiting 504 ms; commit 1791 ms)
2026-11-07T22:11:12Z [load_day 2023-06-27] 2023-06-27 hour 1: 165051 events in 4 files, snapshot 2249797403298329875 (task 10884 ms: download 4757 ms overlapping gunzip 662 ms, parse 2986 ms, sort 87 ms, write 5227 ms, index 547 ms: 5 candidates, 89511 repo-hours, 806 releases; waiting 1025 ms; commit 1670 ms)
2026-11-07T22:11:23Z [load_day 2023-06-27] 2023-06-27 hour 2: 145398 events in 3 files, snapshot 2512903192778847120 (task 10228 ms: download 4339 ms overlapping gunzip 633 ms, parse 2776 ms, sort 91 ms, write 5041 ms, index 485 ms: 4 candidates, 72682 repo-hours, 739 releases; waiting 669 ms; commit 1895 ms)
2026-11-07T22:11:33Z [load_day 2023-06-27] 2023-06-27 hour 3: 139503 events in 3 files, snapshot 8888672521662229900 (task 9245 ms: download 4148 ms overlapping gunzip 601 ms, parse 2608 ms, sort 85 ms, write 4512 ms, index 371 ms: 2 candidates, 61204 repo-hours, 839 releases; waiting 730 ms; commit 1927 ms)
2026-11-07T22:11:42Z [load_day 2023-06-27] 2023-06-27 hour 4: 127102 events in 3 files, snapshot 3375206688586552690 (task 8271 ms: download 3586 ms overlapping gunzip 520 ms, parse 2342 ms, sort 66 ms, write 4128 ms, index 343 ms: 3 candidates, 56223 repo-hours, 668 releases; waiting 596 ms; commit 1808 ms)
2026-11-07T22:11:53Z [load_day 2023-06-27] 2023-06-27 hour 5: 139729 events in 3 files, snapshot 8797701035408198890 (task 9121 ms: download 3711 ms overlapping gunzip 625 ms, parse 2797 ms, sort 93 ms, write 4702 ms, index 389 ms: 6 candidates, 62131 repo-hours, 721 releases; waiting 198 ms; commit 2012 ms)
2026-11-07T22:12:05Z [load_day 2023-06-27] 2023-06-27 hour 6: 169428 events in 4 files, snapshot 4370305213166221995 (task 11452 ms: download 6227 ms overlapping gunzip 750 ms, parse 3304 ms, sort 85 ms, write 6086 ms, index 692 ms: 8 candidates, 78433 repo-hours, 976 releases; waiting 278 ms; commit 1878 ms)
2026-11-07T22:12:19Z [load_day 2023-06-27] 2023-06-27 hour 7: 182160 events in 4 files, snapshot 5447718834665973886 (task 13306 ms: download 7712 ms overlapping gunzip 907 ms, parse 4026 ms, sort 121 ms, write 6860 ms, index 518 ms: 2 candidates, 80819 repo-hours, 1186 releases; waiting 504 ms; commit 1688 ms)
2026-11-07T22:12:34Z [load_day 2023-06-27] 2023-06-27 hour 8: 190297 events in 4 files, snapshot 4672713980011294521 (task 13783 ms: download 7709 ms overlapping gunzip 911 ms, parse 4033 ms, sort 99 ms, write 7291 ms, index 542 ms: 3 candidates, 87199 repo-hours, 1110 releases; waiting 529 ms; commit 1750 ms)
2026-11-07T22:12:49Z [load_day 2023-06-27] 2023-06-27 hour 9: 195631 events in 4 files, snapshot 8038949196111428711 (task 14186 ms: download 8143 ms overlapping gunzip 948 ms, parse 4236 ms, sort 111 ms, write 7310 ms, index 561 ms: 3 candidates, 88413 repo-hours, 1283 releases; waiting 743 ms; commit 1792 ms)
2026-11-07T22:13:03Z [load_day 2023-06-27] 2023-06-27 hour 10: 184265 events in 4 files, snapshot 4016719189586996773 (task 12945 ms: download 7399 ms overlapping gunzip 863 ms, parse 3824 ms, sort 100 ms, write 6801 ms, index 512 ms: 8 candidates, 83372 repo-hours, 1066 releases; waiting 525 ms; commit 1961 ms)
2026-11-07T22:13:17Z [load_day 2023-06-27] 2023-06-27 hour 11: 187515 events in 4 files, snapshot 7194195002351622315 (task 13783 ms: download 7987 ms overlapping gunzip 937 ms, parse 4213 ms, sort 107 ms, write 7189 ms, index 508 ms: 4 candidates, 80634 repo-hours, 1134 releases; waiting 527 ms; commit 1841 ms)
2026-11-07T22:13:33Z [load_day 2023-06-27] 2023-06-27 hour 12: 220260 events in 5 files, snapshot 6057749209293542368 (task 14862 ms: download 9743 ms overlapping gunzip 994 ms, parse 4471 ms, sort 122 ms, write 7700 ms, index 653 ms: 5 candidates, 111053 repo-hours, 1097 releases; waiting 629 ms; commit 1803 ms)
2026-11-07T22:13:49Z [load_day 2023-06-27] 2023-06-27 hour 13: 204734 events in 5 files, snapshot 1339144593769839272 (task 15303 ms: download 8469 ms overlapping gunzip 1084 ms, parse 4721 ms, sort 130 ms, write 7910 ms, index 628 ms: 11 candidates, 103906 repo-hours, 1027 releases; waiting 496 ms; commit 1659 ms)
2026-11-07T22:14:05Z [load_day 2023-06-27] 2023-06-27 hour 14: 190572 events in 4 files, snapshot 1950199308196612233 (task 14465 ms: download 8099 ms overlapping gunzip 983 ms, parse 4339 ms, sort 110 ms, write 7836 ms, index 577 ms: 1 candidates, 89477 repo-hours, 1104 releases; waiting 354 ms; commit 1736 ms)
2026-11-07T22:14:19Z [load_day 2023-06-27] 2023-06-27 hour 15: 174379 events in 4 files, snapshot 8155059134244221973 (task 13277 ms: download 7704 ms overlapping gunzip 870 ms, parse 3896 ms, sort 100 ms, write 6975 ms, index 639 ms: 6 candidates, 84133 repo-hours, 1087 releases; waiting 506 ms; commit 1696 ms)
2026-11-07T22:14:33Z [load_day 2023-06-27] 2023-06-27 hour 16: 177748 events in 4 files, snapshot 1911111042317381387 (task 13460 ms: download 7945 ms overlapping gunzip 910 ms, parse 4048 ms, sort 104 ms, write 7040 ms, index 537 ms: 9 candidates, 82724 repo-hours, 1066 releases; waiting 509 ms; commit 1765 ms)
2026-11-07T22:14:48Z [load_day 2023-06-27] 2023-06-27 hour 17: 181583 events in 4 files, snapshot 6359874955752765065 (task 13293 ms: download 7700 ms overlapping gunzip 942 ms, parse 4121 ms, sort 98 ms, write 6742 ms, index 513 ms: 6 candidates, 79304 repo-hours, 904 releases; waiting 513 ms; commit 1714 ms)
2026-11-07T22:15:01Z [load_day 2023-06-27] 2023-06-27 hour 18: 179550 events in 4 files, snapshot 9219259696938243525 (task 12699 ms: download 7440 ms overlapping gunzip 850 ms, parse 3806 ms, sort 93 ms, write 6529 ms, index 491 ms: 8 candidates, 81974 repo-hours, 1003 releases; waiting 586 ms; commit 1840 ms)
2026-11-07T22:15:15Z [load_day 2023-06-27] 2023-06-27 hour 19: 171021 events in 4 files, snapshot 4542484142130714718 (task 12581 ms: download 7355 ms overlapping gunzip 848 ms, parse 3718 ms, sort 107 ms, write 6622 ms, index 472 ms: 6 candidates, 72365 repo-hours, 1008 releases; waiting 500 ms; commit 1630 ms)
2026-11-07T22:15:28Z [load_day 2023-06-27] 2023-06-27 hour 20: 166607 events in 4 files, snapshot 7056632384797165136 (task 12122 ms: download 7057 ms overlapping gunzip 823 ms, parse 3683 ms, sort 99 ms, write 6202 ms, index 487 ms: 5 candidates, 71961 repo-hours, 944 releases; waiting 497 ms; commit 1947 ms)
2026-11-07T22:15:40Z [load_day 2023-06-27] 2023-06-27 hour 21: 150115 events in 4 files, snapshot 8033247319937032584 (task 11145 ms: download 4851 ms overlapping gunzip 735 ms, parse 3230 ms, sort 99 ms, write 5792 ms, index 416 ms: 9 candidates, 63247 repo-hours, 983 releases; waiting 516 ms; commit 2075 ms)
2026-11-07T22:15:51Z [load_day 2023-06-27] 2023-06-27 hour 22: 134138 events in 3 files, snapshot 4023634963170533158 (task 10010 ms: download 4485 ms overlapping gunzip 654 ms, parse 2907 ms, sort 80 ms, write 5211 ms, index 368 ms: 9 candidates, 55975 repo-hours, 834 releases; waiting 501 ms; commit 1936 ms)
2026-11-07T22:16:02Z [load_day 2023-06-27] 2023-06-27 hour 23: 129760 events in 3 files, snapshot 1636610909796094971 (task 10161 ms: download 4733 ms overlapping gunzip 676 ms, parse 2970 ms, sort 73 ms, write 5026 ms, index 402 ms: 2 candidates, 55686 repo-hours, 783 releases; waiting 658 ms; commit 1934 ms)
2026-11-07T22:16:10Z [load_day 2023-06-27] 2023-06-27: repo_hour merged from 24 files to 1 (1863500 rows, 7843 ms)
2026-11-07T22:16:11Z [load_day 2023-06-27] 2023-06-27: index committed in 9740 ms (24 + 1 + 24 files)
2026-11-07T22:16:11Z [load_day 2023-06-27] 2023-06-27: 4054950 events in 24 hours
2026-11-07T22:16:11Z [load_day 2023-06-27] attempt 1 succeeded