orcaLog in

Runs · run 1251

load_day 2023-06-30, attempt 1 succeeded

Host
3c2b29713ce8:70743
Started
2026-11-07T22:26:47Z
Ended
2026-11-07T22:31:40Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1251/load_day/2023-06-30/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T22:26:47Z [load_day 2023-06-30] attempt 1 started
2026-11-07T22:26:48Z [load_day 2023-06-30] 2023-06-30: 1 hours in flight, about 750 MB
2026-11-07T22:27:01Z [load_day 2023-06-30] 2023-06-30 hour 0: 136651 events in 3 files, snapshot 9191510294187694990 (task 9510 ms: download 4197 ms overlapping gunzip 588 ms, parse 2764 ms, sort 76 ms, write 4569 ms, index 479 ms: 5 candidates, 69683 repo-hours, 808 releases; waiting 753 ms; commit 1745 ms)
2026-11-07T22:27:10Z [load_day 2023-06-30] 2023-06-30 hour 1: 144009 events in 3 files, snapshot 5874667744805723556 (task 8650 ms: download 3229 ms overlapping gunzip 571 ms, parse 2587 ms, sort 75 ms, write 4406 ms, index 510 ms: 6 candidates, 82268 repo-hours, 863 releases; waiting 185 ms; commit 1741 ms)
2026-11-07T22:27:20Z [load_day 2023-06-30] 2023-06-30 hour 2: 131507 events in 3 files, snapshot 4487091519316920046 (task 8442 ms: download 3366 ms overlapping gunzip 546 ms, parse 2431 ms, sort 67 ms, write 4216 ms, index 443 ms: 6 candidates, 67074 repo-hours, 736 releases; waiting 460 ms; commit 2070 ms)
2026-11-07T22:27:28Z [load_day 2023-06-30] 2023-06-30 hour 3: 121628 events in 3 files, snapshot 3578025734634236219 (task 7810 ms: download 2562 ms overlapping gunzip 493 ms, parse 2188 ms, sort 63 ms, write 3881 ms, index 350 ms: 2 candidates, 56342 repo-hours, 884 releases; waiting 511 ms; commit 1826 ms)
2026-11-07T22:27:37Z [load_day 2023-06-30] 2023-06-30 hour 4: 113483 events in 3 files, snapshot 301418432151178724 (task 7549 ms: download 1880 ms overlapping gunzip 487 ms, parse 2156 ms, sort 57 ms, write 3792 ms, index 314 ms: 2 candidates, 50234 repo-hours, 646 releases; waiting 494 ms; commit 1851 ms)
2026-11-07T22:27:46Z [load_day 2023-06-30] 2023-06-30 hour 5: 131587 events in 3 files, snapshot 7091265634414052889 (task 8255 ms: download 3484 ms overlapping gunzip 550 ms, parse 2416 ms, sort 71 ms, write 4062 ms, index 348 ms: 1 candidates, 56343 repo-hours, 760 releases; waiting 496 ms; commit 1819 ms)
2026-11-07T22:27:57Z [load_day 2023-06-30] 2023-06-30 hour 6: 151841 events in 4 files, snapshot 5687615694778410917 (task 10060 ms: download 4147 ms overlapping gunzip 645 ms, parse 2879 ms, sort 77 ms, write 5139 ms, index 464 ms: 4 candidates, 73251 repo-hours, 869 releases; waiting 521 ms; commit 1877 ms)
2026-11-07T22:28:10Z [load_day 2023-06-30] 2023-06-30 hour 7: 162622 events in 4 files, snapshot 1913835522540921230 (task 11893 ms: download 7039 ms overlapping gunzip 801 ms, parse 3452 ms, sort 94 ms, write 6226 ms, index 498 ms: 6 candidates, 74183 repo-hours, 918 releases; waiting 515 ms; commit 2123 ms)
2026-11-07T22:28:24Z [load_day 2023-06-30] 2023-06-30 hour 8: 180196 events in 4 files, snapshot 8775145255246291362 (task 13031 ms: download 7336 ms overlapping gunzip 879 ms, parse 3906 ms, sort 112 ms, write 6831 ms, index 547 ms: 10 candidates, 81694 repo-hours, 1055 releases; waiting 505 ms; commit 1752 ms)
2026-11-07T22:28:37Z [load_day 2023-06-30] 2023-06-30 hour 9: 177946 events in 4 files, snapshot 7072485315768523216 (task 12446 ms: download 7088 ms overlapping gunzip 854 ms, parse 3748 ms, sort 103 ms, write 6611 ms, index 519 ms: 5 candidates, 80283 repo-hours, 1202 releases; waiting 265 ms; commit 1869 ms)
2026-11-07T22:28:50Z [load_day 2023-06-30] 2023-06-30 hour 10: 167053 events in 4 files, snapshot 2912174830090270340 (task 12179 ms: download 7017 ms overlapping gunzip 792 ms, parse 3530 ms, sort 97 ms, write 6082 ms, index 560 ms: 6 candidates, 75634 repo-hours, 1072 releases; waiting 642 ms; commit 1801 ms)
2026-11-07T22:29:03Z [load_day 2023-06-30] 2023-06-30 hour 11: 166003 events in 4 files, snapshot 3298589515804987781 (task 11976 ms: download 7146 ms overlapping gunzip 774 ms, parse 3512 ms, sort 90 ms, write 5985 ms, index 458 ms: 8 candidates, 71428 repo-hours, 1046 releases; waiting 864 ms; commit 2060 ms)
2026-11-07T22:29:18Z [load_day 2023-06-30] 2023-06-30 hour 12: 212005 events in 5 files, snapshot 358258955179610053 (task 13441 ms: download 7029 ms overlapping gunzip 848 ms, parse 3878 ms, sort 112 ms, write 7166 ms, index 670 ms: 5 candidates, 104790 repo-hours, 1128 releases; waiting 499 ms; commit 2201 ms)
2026-11-07T22:29:32Z [load_day 2023-06-30] 2023-06-30 hour 13: 196726 events in 4 files, snapshot 7817265782888515954 (task 13793 ms: download 7420 ms overlapping gunzip 904 ms, parse 4134 ms, sort 111 ms, write 7191 ms, index 623 ms: 5 candidates, 98804 repo-hours, 1305 releases; waiting 491 ms; commit 1831 ms)
2026-11-07T22:29:46Z [load_day 2023-06-30] 2023-06-30 hour 14: 187173 events in 4 files, snapshot 3102285786387206782 (task 13694 ms: download 7922 ms overlapping gunzip 903 ms, parse 4027 ms, sort 94 ms, write 7122 ms, index 520 ms: 11 candidates, 84298 repo-hours, 1141 releases; waiting 688 ms; commit 1700 ms)
2026-11-07T22:30:01Z [load_day 2023-06-30] 2023-06-30 hour 15: 182483 events in 4 files, snapshot 1858965893403417218 (task 13203 ms: download 7682 ms overlapping gunzip 882 ms, parse 3967 ms, sort 94 ms, write 6896 ms, index 540 ms: 5 candidates, 84000 repo-hours, 1187 releases; waiting 516 ms; commit 2000 ms)
2026-11-07T22:30:14Z [load_day 2023-06-30] 2023-06-30 hour 16: 169495 events in 4 files, snapshot 3726761950357540458 (task 12616 ms: download 7456 ms overlapping gunzip 845 ms, parse 3735 ms, sort 114 ms, write 6537 ms, index 485 ms: 2 candidates, 76110 repo-hours, 1014 releases; waiting 507 ms; commit 2125 ms)
2026-11-07T22:30:26Z [load_day 2023-06-30] 2023-06-30 hour 17: 162107 events in 4 files, snapshot 6817952369971381810 (task 11604 ms: download 4925 ms overlapping gunzip 764 ms, parse 3386 ms, sort 95 ms, write 6102 ms, index 453 ms: 8 candidates, 71249 repo-hours, 843 releases; waiting 506 ms; commit 2007 ms)
2026-11-07T22:30:39Z [load_day 2023-06-30] 2023-06-30 hour 18: 164796 events in 4 files, snapshot 4842435115105109984 (task 11697 ms: download 6831 ms overlapping gunzip 783 ms, parse 3516 ms, sort 89 ms, write 5814 ms, index 489 ms: 5 candidates, 74925 repo-hours, 964 releases; waiting 648 ms; commit 2130 ms)
2026-11-07T22:30:51Z [load_day 2023-06-30] 2023-06-30 hour 19: 148530 events in 3 files, snapshot 8838412620925287661 (task 10856 ms: download 4693 ms overlapping gunzip 713 ms, parse 3259 ms, sort 112 ms, write 5542 ms, index 436 ms: 7 candidates, 63177 repo-hours, 899 releases; waiting 506 ms; commit 2140 ms)
2026-11-07T22:31:02Z [load_day 2023-06-30] 2023-06-30 hour 20: 145245 events in 3 files, snapshot 2168686815351691533 (task 10693 ms: download 4794 ms overlapping gunzip 688 ms, parse 3080 ms, sort 78 ms, write 5284 ms, index 415 ms: 6 candidates, 63280 repo-hours, 972 releases; waiting 789 ms; commit 1886 ms)
2026-11-07T22:31:13Z [load_day 2023-06-30] 2023-06-30 hour 21: 131950 events in 3 files, snapshot 2043264126038113243 (task 9291 ms: download 4132 ms overlapping gunzip 645 ms, parse 2826 ms, sort 83 ms, write 4571 ms, index 341 ms: 3 candidates, 57342 repo-hours, 941 releases; waiting 521 ms; commit 1857 ms)
2026-11-07T22:31:23Z [load_day 2023-06-30] 2023-06-30 hour 22: 125187 events in 3 files, snapshot 3882862467748235214 (task 8962 ms: download 4274 ms overlapping gunzip 627 ms, parse 2704 ms, sort 71 ms, write 4400 ms, index 353 ms: 5 candidates, 56344 repo-hours, 757 releases; waiting 495 ms; commit 2001 ms)
2026-11-07T22:31:31Z [load_day 2023-06-30] 2023-06-30 hour 23: 106771 events in 3 files, snapshot 9212513608606689489 (task 7714 ms: download 2418 ms overlapping gunzip 516 ms, parse 2236 ms, sort 60 ms, write 3848 ms, index 307 ms: 2 candidates, 45627 repo-hours, 706 releases; waiting 516 ms; commit 1836 ms)
2026-11-07T22:31:38Z [load_day 2023-06-30] 2023-06-30: repo_hour merged from 24 files to 1 (1718363 rows, 7221 ms)
2026-11-07T22:31:40Z [load_day 2023-06-30] 2023-06-30: index committed in 8945 ms (24 + 1 + 24 files)
2026-11-07T22:31:40Z [load_day 2023-06-30] 2023-06-30: 3716994 events in 24 hours
2026-11-07T22:31:40Z [load_day 2023-06-30] attempt 1 succeeded