orcaLog in

Runs · run 1072

load_day 2023-05-09, attempt 1 succeeded

Host
caf251265124:350469
Started
2026-11-07T19:51:47Z
Ended
2026-11-07T19:55:07Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1072/load_day/2023-05-09/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T19:51:47Z [load_day 2023-05-09] attempt 1 started
2026-11-07T19:51:49Z [load_day 2023-05-09] 2023-05-09: 2 hours in flight, about 1200 MB
2026-11-07T19:52:04Z [load_day 2023-05-09] 2023-05-09 hour 0: 156767 events in 4 files, snapshot 3331599725184544553 (task 12414 ms: download 5649 ms overlapping gunzip 757 ms, parse 3409 ms, sort 99 ms, write 6879 ms, index 484 ms: 11 candidates, 78540 repo-hours, 883 releases; waiting 488 ms; commit 1624 ms)
2026-11-07T19:52:06Z [load_day 2023-05-09] 2023-05-09 hour 1: 167670 events in 4 files, snapshot 496062386960831760 (task 12702 ms: download 5506 ms overlapping gunzip 723 ms, parse 3236 ms, sort 86 ms, write 6852 ms, index 565 ms: 10 candidates, 92889 repo-hours, 844 releases; waiting 894 ms; commit 1582 ms)
2026-11-07T19:52:17Z [load_day 2023-05-09] 2023-05-09 hour 2: 149012 events in 3 files, snapshot 1363608051648526779 (task 11794 ms: download 4872 ms overlapping gunzip 690 ms, parse 2886 ms, sort 75 ms, write 6555 ms, index 477 ms: 2 candidates, 70123 repo-hours, 661 releases; waiting 486 ms; commit 1794 ms)
2026-11-07T19:52:19Z [load_day 2023-05-09] 2023-05-09 hour 3: 140017 events in 3 files, snapshot 679119382821446482 (task 11184 ms: download 4877 ms overlapping gunzip 659 ms, parse 2882 ms, sort 84 ms, write 6163 ms, index 415 ms: 11 candidates, 61154 repo-hours, 905 releases; waiting 624 ms; commit 1616 ms)
2026-11-07T19:52:28Z [load_day 2023-05-09] 2023-05-09 hour 4: 130174 events in 3 files, snapshot 8758061810713598725 (task 10060 ms: download 4377 ms overlapping gunzip 612 ms, parse 2633 ms, sort 73 ms, write 5485 ms, index 364 ms: 4 candidates, 55285 repo-hours, 677 releases; waiting 508 ms; commit 1587 ms)
2026-11-07T19:52:31Z [load_day 2023-05-09] 2023-05-09 hour 5: 142397 events in 3 files, snapshot 6863135973632111306 (task 11150 ms: download 4810 ms overlapping gunzip 678 ms, parse 2953 ms, sort 84 ms, write 5929 ms, index 397 ms: 7 candidates, 61173 repo-hours, 778 releases; waiting 764 ms; commit 1751 ms)
2026-11-07T19:52:43Z [load_day 2023-05-09] 2023-05-09 hour 6: 168321 events in 4 files, snapshot 2238542264735088505 (task 14422 ms: download 8747 ms overlapping gunzip 800 ms, parse 3423 ms, sort 102 ms, write 7707 ms, index 506 ms: 9 candidates, 80201 repo-hours, 922 releases; waiting 1521 ms; commit 1989 ms)
2026-11-07T19:52:45Z [load_day 2023-05-09] 2023-05-09 hour 7: 159234 events in 4 files, snapshot 5597402493801927444 (task 12987 ms: download 5330 ms overlapping gunzip 781 ms, parse 3415 ms, sort 100 ms, write 7660 ms, index 484 ms: 2 candidates, 76079 repo-hours, 828 releases; waiting 246 ms; commit 1865 ms)
2026-11-07T19:52:58Z [load_day 2023-05-09] 2023-05-09 hour 8: 172585 events in 4 files, snapshot 6794342308491684052 (task 14302 ms: download 8194 ms overlapping gunzip 839 ms, parse 3707 ms, sort 96 ms, write 8268 ms, index 561 ms: 6 candidates, 81990 repo-hours, 964 releases; waiting 505 ms; commit 1747 ms)
2026-11-07T19:53:01Z [load_day 2023-05-09] 2023-05-09 hour 9: 173105 events in 4 files, snapshot 218725052635322083 (task 15236 ms: download 8884 ms overlapping gunzip 914 ms, parse 3957 ms, sort 99 ms, write 8916 ms, index 554 ms: 6 candidates, 83804 repo-hours, 1162 releases; waiting 523 ms; commit 2068 ms)
2026-11-07T19:53:14Z [load_day 2023-05-09] 2023-05-09 hour 10: 176851 events in 4 files, snapshot 8951470030006258297 (task 15333 ms: download 9211 ms overlapping gunzip 907 ms, parse 3964 ms, sort 96 ms, write 8648 ms, index 549 ms: 6 candidates, 83938 repo-hours, 990 releases; waiting 882 ms; commit 1554 ms)
2026-11-07T19:53:16Z [load_day 2023-05-09] 2023-05-09 hour 11: 89937 events in 2 files, snapshot 4942856228089148077 (task 8100 ms: download 2320 ms overlapping gunzip 480 ms, parse 2007 ms, sort 45 ms, write 4656 ms, index 315 ms: 3 candidates, 46815 repo-hours, 470 releases; waiting 330 ms; commit 1915 ms)
2026-11-07T19:53:26Z [load_day 2023-05-09] 2023-05-09 hour 12: 120890 events in 3 files, snapshot 8396218115204975536 (task 11172 ms: download 5279 ms overlapping gunzip 631 ms, parse 2722 ms, sort 76 ms, write 6362 ms, index 465 ms: 6 candidates, 69287 repo-hours, 529 releases; waiting 576 ms; commit 1722 ms)
2026-11-07T19:53:34Z [load_day 2023-05-09] 2023-05-09 hour 13: 200785 events in 5 files, snapshot 5688647806301495273 (task 17003 ms: download 9247 ms overlapping gunzip 978 ms, parse 4474 ms, sort 126 ms, write 9804 ms, index 728 ms: 7 candidates, 107674 repo-hours, 1126 releases; waiting 572 ms; commit 1715 ms)
2026-11-07T19:53:44Z [load_day 2023-05-09] 2023-05-09 hour 14: 192586 events in 4 files, snapshot 7587409811502566741 (task 17024 ms: download 9634 ms overlapping gunzip 1007 ms, parse 4434 ms, sort 103 ms, write 9993 ms, index 623 ms: 8 candidates, 92366 repo-hours, 1106 releases; waiting 513 ms; commit 1647 ms)
2026-11-07T19:53:52Z [load_day 2023-05-09] 2023-05-09 hour 15: 169510 events in 4 files, snapshot 6604715574660161031 (task 17652 ms: download 11445 ms overlapping gunzip 921 ms, parse 4052 ms, sort 115 ms, write 8709 ms, index 563 ms: 6 candidates, 85257 repo-hours, 1093 releases; waiting 2957 ms; commit 1579 ms)
2026-11-07T19:53:59Z [load_day 2023-05-09] 2023-05-09 hour 16: 165942 events in 4 files, snapshot 2824868361798115329 (task 14492 ms: download 8614 ms overlapping gunzip 874 ms, parse 3850 ms, sort 108 ms, write 8337 ms, index 552 ms: 4 candidates, 82835 repo-hours, 850 releases; waiting 506 ms; commit 1627 ms)
2026-11-07T19:54:11Z [load_day 2023-05-09] 2023-05-09 hour 17: 171988 events in 4 files, snapshot 7779423466882823754 (task 18022 ms: download 11830 ms overlapping gunzip 995 ms, parse 4222 ms, sort 112 ms, write 8904 ms, index 546 ms: 8 candidates, 79900 repo-hours, 770 releases; waiting 2969 ms; commit 1691 ms)
2026-11-07T19:54:28Z [load_day 2023-05-09] 2023-05-09 hour 18: 225437 events in 5 files, snapshot 4179111430890140580 (task 28243 ms: download 21130 ms overlapping gunzip 2255 ms, parse 8296 ms, sort 135 ms, write 14681 ms, index 618 ms: 8 candidates, 95653 repo-hours, 815 releases; waiting 2004 ms; commit 1597 ms)
2026-11-07T19:54:30Z [load_day 2023-05-09] 2023-05-09 hour 19: 164213 events in 4 files, snapshot 6043882714457703708 (task 13556 ms: download 5776 ms overlapping gunzip 832 ms, parse 3607 ms, sort 112 ms, write 7716 ms, index 502 ms: 10 candidates, 75066 repo-hours, 947 releases; waiting 522 ms; commit 1678 ms)
2026-11-07T19:54:43Z [load_day 2023-05-09] 2023-05-09 hour 20: 159191 events in 4 files, snapshot 4010058929747788017 (task 13361 ms: download 5493 ms overlapping gunzip 788 ms, parse 3456 ms, sort 111 ms, write 7684 ms, index 471 ms: 14 candidates, 72905 repo-hours, 880 releases; waiting 430 ms; commit 1592 ms)
2026-11-07T19:54:44Z [load_day 2023-05-09] 2023-05-09 hour 21: 150682 events in 4 files, snapshot 716419043799482293 (task 13251 ms: download 5953 ms overlapping gunzip 791 ms, parse 3520 ms, sort 100 ms, write 7201 ms, index 430 ms: 7 candidates, 65362 repo-hours, 908 releases; waiting 932 ms; commit 1717 ms)
2026-11-07T19:54:56Z [load_day 2023-05-09] 2023-05-09 hour 22: 143003 events in 3 files, snapshot 7471724774513283224 (task 12303 ms: download 5453 ms overlapping gunzip 744 ms, parse 3189 ms, sort 93 ms, write 7054 ms, index 391 ms: 8 candidates, 61580 repo-hours, 780 releases; waiting 528 ms; commit 1837 ms)
2026-11-07T19:54:58Z [load_day 2023-05-09] 2023-05-09 hour 23: 126056 events in 3 files, snapshot 1009633220402210919 (task 10123 ms: download 4438 ms overlapping gunzip 586 ms, parse 2590 ms, sort 81 ms, write 5677 ms, index 379 ms: 6 candidates, 55330 repo-hours, 708 releases; waiting 536 ms; commit 1483 ms)
2026-11-07T19:55:05Z [load_day 2023-05-09] 2023-05-09: repo_hour merged from 24 files to 1 (1815206 rows, 7606 ms)
2026-11-07T19:55:07Z [load_day 2023-05-09] 2023-05-09: index committed in 9506 ms (24 + 1 + 24 files)
2026-11-07T19:55:07Z [load_day 2023-05-09] 2023-05-09: 3816353 events in 24 hours
2026-11-07T19:55:07Z [load_day 2023-05-09] attempt 1 succeeded