orcaLog in

Runs · run 1161

load_day 2023-06-08, attempt 1 succeeded

Host
3c2b29713ce8:3565
Started
2026-11-07T21:16:35Z
Ended
2026-11-07T21:19:34Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1161/load_day/2023-06-08/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T21:16:35Z [load_day 2023-06-08] attempt 1 started
2026-11-07T21:16:37Z [load_day 2023-06-08] 2023-06-08: 2 hours in flight, about 1200 MB
2026-11-07T21:16:51Z [load_day 2023-06-08] 2023-06-08 hour 0: 151956 events in 4 files, snapshot 1862529041032639940 (task 11214 ms: download 4764 ms overlapping gunzip 640 ms, parse 2900 ms, sort 99 ms, write 6210 ms, index 496 ms: 9 candidates, 74384 repo-hours, 932 releases; waiting 504 ms; commit 1785 ms)
2026-11-07T21:16:52Z [load_day 2023-06-08] 2023-06-08 hour 1: 159022 events in 4 files, snapshot 5712464733860017904 (task 11270 ms: download 4691 ms overlapping gunzip 618 ms, parse 2850 ms, sort 104 ms, write 6287 ms, index 516 ms: 14 candidates, 86464 repo-hours, 840 releases; waiting 598 ms; commit 1639 ms)
2026-11-07T21:17:02Z [load_day 2023-06-08] 2023-06-08 hour 2: 146426 events in 3 files, snapshot 5578960765574914715 (task 10574 ms: download 4395 ms overlapping gunzip 636 ms, parse 2703 ms, sort 94 ms, write 5789 ms, index 468 ms: 8 candidates, 72042 repo-hours, 725 releases; waiting 574 ms; commit 1760 ms)
2026-11-07T21:17:04Z [load_day 2023-06-08] 2023-06-08 hour 3: 130921 events in 3 files, snapshot 6982235296689373305 (task 9822 ms: download 4125 ms overlapping gunzip 581 ms, parse 2583 ms, sort 80 ms, write 5284 ms, index 415 ms: 4 candidates, 58907 repo-hours, 809 releases; waiting 530 ms; commit 1849 ms)
2026-11-07T21:17:12Z [load_day 2023-06-08] 2023-06-08 hour 4: 123769 events in 3 files, snapshot 8723165501527246282 (task 9434 ms: download 3296 ms overlapping gunzip 549 ms, parse 2410 ms, sort 76 ms, write 5215 ms, index 349 ms: 4 candidates, 54018 repo-hours, 636 releases; waiting 532 ms; commit 1779 ms)
2026-11-07T21:17:16Z [load_day 2023-06-08] 2023-06-08 hour 5: 142803 events in 3 files, snapshot 2973149358096791828 (task 10890 ms: download 4548 ms overlapping gunzip 656 ms, parse 2914 ms, sort 113 ms, write 6001 ms, index 417 ms: 11 candidates, 62024 repo-hours, 784 releases; waiting 517 ms; commit 1749 ms)
2026-11-07T21:17:25Z [load_day 2023-06-08] 2023-06-08 hour 6: 167409 events in 4 files, snapshot 1760927954298381923 (task 12341 ms: download 4740 ms overlapping gunzip 778 ms, parse 3349 ms, sort 113 ms, write 7134 ms, index 501 ms: 7 candidates, 80780 repo-hours, 933 releases; waiting 183 ms; commit 1709 ms)
2026-11-07T21:17:29Z [load_day 2023-06-08] 2023-06-08 hour 7: 153817 events in 4 files, snapshot 5608719401514872852 (task 12950 ms: download 5521 ms overlapping gunzip 742 ms, parse 3220 ms, sort 95 ms, write 7179 ms, index 501 ms: 12 candidates, 72548 repo-hours, 848 releases; waiting 862 ms; commit 1840 ms)
2026-11-07T21:17:40Z [load_day 2023-06-08] 2023-06-08 hour 8: 160746 events in 4 files, snapshot 1935233371034554656 (task 13340 ms: download 5484 ms overlapping gunzip 795 ms, parse 3556 ms, sort 105 ms, write 7711 ms, index 511 ms: 18 candidates, 77464 repo-hours, 953 releases; waiting 378 ms; commit 1960 ms)
2026-11-07T21:17:44Z [load_day 2023-06-08] 2023-06-08 hour 9: 169979 events in 4 files, snapshot 8371814365396403181 (task 13473 ms: download 7795 ms overlapping gunzip 801 ms, parse 3549 ms, sort 102 ms, write 7914 ms, index 549 ms: 8 candidates, 81336 repo-hours, 1022 releases; waiting 299 ms; commit 1726 ms)
2026-11-07T21:17:54Z [load_day 2023-06-08] 2023-06-08 hour 10: 170581 events in 4 files, snapshot 2653292395702473584 (task 13527 ms: download 7663 ms overlapping gunzip 817 ms, parse 3674 ms, sort 124 ms, write 7536 ms, index 533 ms: 9 candidates, 82384 repo-hours, 918 releases; waiting 560 ms; commit 1714 ms)
2026-11-07T21:17:59Z [load_day 2023-06-08] 2023-06-08 hour 11: 167790 events in 4 files, snapshot 7635619628391375214 (task 13748 ms: download 5643 ms overlapping gunzip 809 ms, parse 3488 ms, sort 109 ms, write 7635 ms, index 529 ms: 7 candidates, 78068 repo-hours, 971 releases; waiting 814 ms; commit 1855 ms)
2026-11-07T21:18:10Z [load_day 2023-06-08] 2023-06-08 hour 12: 204523 events in 5 files, snapshot 5348711894352434579 (task 14672 ms: download 7416 ms overlapping gunzip 826 ms, parse 3756 ms, sort 127 ms, write 8746 ms, index 653 ms: 9 candidates, 103272 repo-hours, 976 releases; waiting 228 ms; commit 1928 ms)
2026-11-07T21:18:15Z [load_day 2023-06-08] 2023-06-08 hour 13: 188439 events in 4 files, snapshot 1428780600472996119 (task 15340 ms: download 8180 ms overlapping gunzip 866 ms, parse 3877 ms, sort 120 ms, write 9053 ms, index 664 ms: 10 candidates, 97790 repo-hours, 994 releases; waiting 493 ms; commit 2053 ms)
2026-11-07T21:18:26Z [load_day 2023-06-08] 2023-06-08 hour 14: 177078 events in 4 files, snapshot 361804694777768735 (task 14780 ms: download 8615 ms overlapping gunzip 882 ms, parse 3899 ms, sort 110 ms, write 8800 ms, index 551 ms: 9 candidates, 84681 repo-hours, 933 releases; waiting 277 ms; commit 1860 ms)
2026-11-07T21:18:30Z [load_day 2023-06-08] 2023-06-08 hour 15: 174647 events in 4 files, snapshot 4690221585805620035 (task 14403 ms: download 8292 ms overlapping gunzip 834 ms, parse 3708 ms, sort 108 ms, write 8401 ms, index 523 ms: 4 candidates, 83136 repo-hours, 1057 releases; waiting 510 ms; commit 1820 ms)
2026-11-07T21:18:41Z [load_day 2023-06-08] 2023-06-08 hour 16: 177339 events in 4 files, snapshot 2876086217847141796 (task 14488 ms: download 8228 ms overlapping gunzip 871 ms, parse 3818 ms, sort 109 ms, write 8317 ms, index 553 ms: 6 candidates, 80523 repo-hours, 898 releases; waiting 512 ms; commit 1761 ms)
2026-11-07T21:18:45Z [load_day 2023-06-08] 2023-06-08 hour 17: 172493 events in 4 files, snapshot 6926632641172103078 (task 14078 ms: download 8330 ms overlapping gunzip 860 ms, parse 3854 ms, sort 123 ms, write 7968 ms, index 517 ms: 10 candidates, 75345 repo-hours, 795 releases; waiting 514 ms; commit 1723 ms)
2026-11-07T21:18:56Z [load_day 2023-06-08] 2023-06-08 hour 18: 177862 events in 4 files, snapshot 5013747868511618653 (task 14058 ms: download 8235 ms overlapping gunzip 859 ms, parse 3754 ms, sort 96 ms, write 8000 ms, index 505 ms: 15 candidates, 79072 repo-hours, 863 releases; waiting 532 ms; commit 1711 ms)
2026-11-07T21:19:03Z [load_day 2023-06-08] 2023-06-08 hour 19: 171574 events in 4 files, snapshot 8921726282494833520 (task 13188 ms: download 7502 ms overlapping gunzip 795 ms, parse 3543 ms, sort 108 ms, write 7352 ms, index 507 ms: 7 candidates, 72431 repo-hours, 925 releases; waiting 443 ms; commit 5678 ms)
2026-11-07T21:19:11Z [load_day 2023-06-08] 2023-06-08 hour 20: 166487 events in 4 files, snapshot 6617076246106885260 (task 13547 ms: download 8101 ms overlapping gunzip 809 ms, parse 3621 ms, sort 129 ms, write 7743 ms, index 479 ms: 11 candidates, 75377 repo-hours, 962 releases; waiting 498 ms; commit 1804 ms)
2026-11-07T21:19:12Z [load_day 2023-06-08] 2023-06-08 hour 21: 153740 events in 4 files, snapshot 5277340368181454571 (task 12468 ms: download 5385 ms overlapping gunzip 744 ms, parse 3232 ms, sort 105 ms, write 7055 ms, index 428 ms: 8 candidates, 66624 repo-hours, 930 releases; waiting 519 ms; commit 1587 ms)
2026-11-07T21:19:23Z [load_day 2023-06-08] 2023-06-08 hour 22: 139946 events in 3 files, snapshot 7663551386224389170 (task 11067 ms: download 4884 ms overlapping gunzip 670 ms, parse 2915 ms, sort 100 ms, write 5932 ms, index 363 ms: 7 candidates, 57729 repo-hours, 758 releases; waiting 652 ms; commit 1827 ms)
2026-11-07T21:19:25Z [load_day 2023-06-08] 2023-06-08 hour 23: 130986 events in 3 files, snapshot 7732849713017524744 (task 10734 ms: download 5201 ms overlapping gunzip 586 ms, parse 2590 ms, sort 79 ms, write 5252 ms, index 386 ms: 3 candidates, 57475 repo-hours, 679 releases; waiting 1522 ms; commit 1771 ms)
2026-11-07T21:19:32Z [load_day 2023-06-08] 2023-06-08: repo_hour merged from 24 files to 1 (1813874 rows, 7437 ms)
2026-11-07T21:19:34Z [load_day 2023-06-08] 2023-06-08: index committed in 9624 ms (24 + 1 + 24 files)
2026-11-07T21:19:34Z [load_day 2023-06-08] 2023-06-08: 3880333 events in 24 hours
2026-11-07T21:19:34Z [load_day 2023-06-08] attempt 1 succeeded