orcaLog in

Runs · run 1056

load_day 2023-05-04, attempt 1 succeeded

Host
caf251265124:334018
Started
2026-11-07T19:37:21Z
Ended
2026-11-07T19:40:29Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1056/load_day/2023-05-04/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T19:37:21Z [load_day 2023-05-04] attempt 1 started
2026-11-07T19:37:22Z [load_day 2023-05-04] 2023-05-04: 2 hours in flight, about 1200 MB
2026-11-07T19:37:37Z [load_day 2023-05-04] 2023-05-04 hour 0: 153603 events in 4 files, snapshot 8596064819716947532 (task 11655 ms: download 5102 ms overlapping gunzip 709 ms, parse 3158 ms, sort 116 ms, write 6327 ms, index 494 ms: 7 candidates, 77124 repo-hours, 967 releases; waiting 488 ms; commit 1676 ms)
2026-11-07T19:37:38Z [load_day 2023-05-04] 2023-05-04 hour 1: 165695 events in 4 files, snapshot 1991898853230160543 (task 12898 ms: download 5602 ms overlapping gunzip 766 ms, parse 3375 ms, sort 100 ms, write 6935 ms, index 595 ms: 8 candidates, 90534 repo-hours, 798 releases; waiting 813 ms; commit 1809 ms)
2026-11-07T19:37:49Z [load_day 2023-05-04] 2023-05-04 hour 2: 143836 events in 3 files, snapshot 8920044706966775380 (task 11793 ms: download 5828 ms overlapping gunzip 661 ms, parse 2803 ms, sort 80 ms, write 5634 ms, index 402 ms: 9 candidates, 64721 repo-hours, 685 releases; waiting 1892 ms; commit 1917 ms)
2026-11-07T19:37:51Z [load_day 2023-05-04] 2023-05-04 hour 3: 137344 events in 3 files, snapshot 1037066179499109266 (task 10496 ms: download 4846 ms overlapping gunzip 585 ms, parse 2577 ms, sort 86 ms, write 5341 ms, index 396 ms: 9 candidates, 59333 repo-hours, 897 releases; waiting 1198 ms; commit 1719 ms)
2026-11-07T19:37:59Z [load_day 2023-05-04] 2023-05-04 hour 4: 131052 events in 3 files, snapshot 5051783798727915943 (task 9237 ms: download 3859 ms overlapping gunzip 537 ms, parse 2362 ms, sort 71 ms, write 5083 ms, index 369 ms: 11 candidates, 56338 repo-hours, 647 releases; waiting 507 ms; commit 1590 ms)
2026-11-07T19:38:03Z [load_day 2023-05-04] 2023-05-04 hour 5: 143998 events in 3 files, snapshot 2296646876874867191 (task 10453 ms: download 4357 ms overlapping gunzip 629 ms, parse 2890 ms, sort 91 ms, write 5658 ms, index 433 ms: 8 candidates, 62361 repo-hours, 852 releases; waiting 492 ms; commit 1811 ms)
2026-11-07T19:38:13Z [load_day 2023-05-04] 2023-05-04 hour 6: 169971 events in 4 files, snapshot 3696693524872783029 (task 12516 ms: download 4962 ms overlapping gunzip 725 ms, parse 3159 ms, sort 120 ms, write 7165 ms, index 510 ms: 8 candidates, 79897 repo-hours, 1009 releases; waiting 520 ms; commit 2130 ms)
2026-11-07T19:38:16Z [load_day 2023-05-04] 2023-05-04 hour 7: 170475 events in 4 files, snapshot 2518697358704245635 (task 13032 ms: download 7358 ms overlapping gunzip 761 ms, parse 3401 ms, sort 95 ms, write 7402 ms, index 521 ms: 7 candidates, 81216 repo-hours, 856 releases; waiting 513 ms; commit 1645 ms)
2026-11-07T19:38:29Z [load_day 2023-05-04] 2023-05-04 hour 8: 178497 events in 4 files, snapshot 1147196756404559894 (task 14890 ms: download 8243 ms overlapping gunzip 867 ms, parse 3822 ms, sort 111 ms, write 8514 ms, index 567 ms: 9 candidates, 84320 repo-hours, 1041 releases; waiting 403 ms; commit 1898 ms)
2026-11-07T19:38:32Z [load_day 2023-05-04] 2023-05-04 hour 9: 179914 events in 4 files, snapshot 1409743145478346921 (task 14711 ms: download 8278 ms overlapping gunzip 868 ms, parse 3813 ms, sort 108 ms, write 8464 ms, index 625 ms: 12 candidates, 83679 repo-hours, 1231 releases; waiting 532 ms; commit 1658 ms)
2026-11-07T19:38:44Z [load_day 2023-05-04] 2023-05-04 hour 10: 179137 events in 4 files, snapshot 6462137649730497927 (task 14447 ms: download 8177 ms overlapping gunzip 830 ms, parse 3725 ms, sort 111 ms, write 8395 ms, index 542 ms: 6 candidates, 82539 repo-hours, 962 releases; waiting 510 ms; commit 1726 ms)
2026-11-07T19:38:47Z [load_day 2023-05-04] 2023-05-04 hour 11: 176631 events in 4 files, snapshot 117059436121474654 (task 14559 ms: download 8359 ms overlapping gunzip 840 ms, parse 3674 ms, sort 106 ms, write 8353 ms, index 506 ms: 8 candidates, 80925 repo-hours, 949 releases; waiting 751 ms; commit 1715 ms)
2026-11-07T19:38:59Z [load_day 2023-05-04] 2023-05-04 hour 12: 210321 events in 5 files, snapshot 1600212401170979471 (task 14766 ms: download 7418 ms overlapping gunzip 869 ms, parse 3878 ms, sort 105 ms, write 8753 ms, index 670 ms: 10 candidates, 105446 repo-hours, 1093 releases; waiting 219 ms; commit 1618 ms)
2026-11-07T19:39:04Z [load_day 2023-05-04] 2023-05-04 hour 13: 200372 events in 5 files, snapshot 7031753372105585681 (task 16191 ms: download 8795 ms overlapping gunzip 936 ms, parse 4251 ms, sort 116 ms, write 9324 ms, index 653 ms: 5 candidates, 102038 repo-hours, 1053 releases; waiting 611 ms; commit 2085 ms)
2026-11-07T19:39:18Z [load_day 2023-05-04] 2023-05-04 hour 14: 201116 events in 5 files, snapshot 6870036774902828877 (task 17703 ms: download 10364 ms overlapping gunzip 996 ms, parse 4439 ms, sort 112 ms, write 9548 ms, index 599 ms: 10 candidates, 93843 repo-hours, 1128 releases; waiting 1699 ms; commit 1854 ms)
2026-11-07T19:39:20Z [load_day 2023-05-04] 2023-05-04 hour 15: 150398 events in 4 files, snapshot 4794185747451842027 (task 12854 ms: download 5505 ms overlapping gunzip 762 ms, parse 3369 ms, sort 88 ms, write 7233 ms, index 513 ms: 5 candidates, 77734 repo-hours, 900 releases; waiting 521 ms; commit 1841 ms)
2026-11-07T19:39:31Z [load_day 2023-05-04] 2023-05-04 hour 16: 137084 events in 3 files, snapshot 7538164171704197201 (task 11808 ms: download 5006 ms overlapping gunzip 666 ms, parse 2949 ms, sort 81 ms, write 6852 ms, index 475 ms: 10 candidates, 71356 repo-hours, 640 releases; waiting 504 ms; commit 1746 ms)
2026-11-07T19:39:39Z [load_day 2023-05-04] 2023-05-04 hour 17: 189180 events in 4 files, snapshot 9047324872113035134 (task 17973 ms: download 11113 ms overlapping gunzip 979 ms, parse 4359 ms, sort 115 ms, write 9187 ms, index 563 ms: 9 candidates, 84310 repo-hours, 967 releases; waiting 2395 ms; commit 1712 ms)
2026-11-07T19:39:47Z [load_day 2023-05-04] 2023-05-04 hour 18: 189765 events in 4 files, snapshot 7260370659441798035 (task 15358 ms: download 8347 ms overlapping gunzip 908 ms, parse 4017 ms, sort 119 ms, write 8884 ms, index 590 ms: 5 candidates, 86795 repo-hours, 1008 releases; waiting 527 ms; commit 1568 ms)
2026-11-07T19:39:54Z [load_day 2023-05-04] 2023-05-04 hour 19: 181603 events in 4 files, snapshot 1319408937132005743 (task 14817 ms: download 8341 ms overlapping gunzip 866 ms, parse 3923 ms, sort 112 ms, write 8636 ms, index 517 ms: 7 candidates, 77445 repo-hours, 926 releases; waiting 399 ms; commit 1698 ms)
2026-11-07T19:40:02Z [load_day 2023-05-04] 2023-05-04 hour 20: 169626 events in 4 files, snapshot 7319424767177873612 (task 14124 ms: download 7989 ms overlapping gunzip 869 ms, parse 3875 ms, sort 122 ms, write 7976 ms, index 486 ms: 7 candidates, 74053 repo-hours, 839 releases; waiting 184 ms; commit 1665 ms)
2026-11-07T19:40:09Z [load_day 2023-05-04] 2023-05-04 hour 21: 164082 events in 4 files, snapshot 4936557411469656682 (task 13684 ms: download 8153 ms overlapping gunzip 807 ms, parse 3564 ms, sort 86 ms, write 7895 ms, index 512 ms: 9 candidates, 67077 repo-hours, 994 releases; waiting 530 ms; commit 1585 ms)
2026-11-07T19:40:15Z [load_day 2023-05-04] 2023-05-04 hour 22: 143599 events in 3 files, snapshot 5736640101897800343 (task 12393 ms: download 5438 ms overlapping gunzip 735 ms, parse 3189 ms, sort 104 ms, write 7023 ms, index 372 ms: 8 candidates, 58452 repo-hours, 781 releases; waiting 641 ms; commit 1609 ms)
2026-11-07T19:40:20Z [load_day 2023-05-04] 2023-05-04 hour 23: 129685 events in 3 files, snapshot 5121718104317201728 (task 10190 ms: download 4386 ms overlapping gunzip 600 ms, parse 2682 ms, sort 78 ms, write 5625 ms, index 365 ms: 6 candidates, 53822 repo-hours, 680 releases; waiting 502 ms; commit 1679 ms)
2026-11-07T19:40:27Z [load_day 2023-05-04] 2023-05-04: repo_hour merged from 24 files to 1 (1855358 rows, 7602 ms)
2026-11-07T19:40:29Z [load_day 2023-05-04] 2023-05-04: index committed in 9192 ms (24 + 1 + 24 files)
2026-11-07T19:40:29Z [load_day 2023-05-04] 2023-05-04: 3996984 events in 24 hours
2026-11-07T19:40:29Z [load_day 2023-05-04] attempt 1 succeeded