orcaLog in

Runs · run 1047

load_day 2023-05-01, attempt 1 succeeded

Host
caf251265124:322566
Started
2026-11-07T19:27:45Z
Ended
2026-11-07T19:31:09Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1047/load_day/2023-05-01/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T19:27:45Z [load_day 2023-05-01] attempt 1 started
2026-11-07T19:27:46Z [load_day 2023-05-01] 2023-05-01: 2 hours in flight, about 1200 MB
2026-11-07T19:27:59Z [load_day 2023-05-01] 2023-05-01 hour 0: 133306 events in 3 files, snapshot 9045009903740669532 (task 9327 ms: download 3931 ms overlapping gunzip 593 ms, parse 2570 ms, sort 77 ms, write 5203 ms, index 444 ms: 6 candidates, 66512 repo-hours, 872 releases; waiting 201 ms; commit 1564 ms)
2026-11-07T19:28:02Z [load_day 2023-05-01] 2023-05-01 hour 1: 153266 events in 4 files, snapshot 7522304730902925052 (task 12228 ms: download 5386 ms overlapping gunzip 794 ms, parse 3395 ms, sort 103 ms, write 6601 ms, index 516 ms: 5 candidates, 79096 repo-hours, 840 releases; waiting 504 ms; commit 1511 ms)
2026-11-07T19:28:11Z [load_day 2023-05-01] 2023-05-01 hour 2: 132347 events in 3 files, snapshot 9195127040034894260 (task 11344 ms: download 5688 ms overlapping gunzip 726 ms, parse 2954 ms, sort 76 ms, write 5961 ms, index 361 ms: 2 candidates, 51942 repo-hours, 814 releases; waiting 927 ms; commit 1499 ms)
2026-11-07T19:28:13Z [load_day 2023-05-01] 2023-05-01 hour 3: 128083 events in 3 files, snapshot 4682014200529301971 (task 10448 ms: download 4991 ms overlapping gunzip 681 ms, parse 2792 ms, sort 79 ms, write 5742 ms, index 364 ms: 4 candidates, 47252 repo-hours, 734 releases; waiting 532 ms; commit 1638 ms)
2026-11-07T19:28:22Z [load_day 2023-05-01] 2023-05-01 hour 4: 127000 events in 3 files, snapshot 4573677824590535090 (task 10949 ms: download 5432 ms overlapping gunzip 722 ms, parse 2931 ms, sort 73 ms, write 5861 ms, index 295 ms: 0 candidates, 47268 repo-hours, 599 releases; waiting 750 ms; commit 1746 ms)
2026-11-07T19:28:26Z [load_day 2023-05-01] 2023-05-01 hour 5: 139483 events in 3 files, snapshot 988678914876357222 (task 12271 ms: download 5977 ms overlapping gunzip 828 ms, parse 3330 ms, sort 77 ms, write 6760 ms, index 330 ms: 2 candidates, 51364 repo-hours, 665 releases; waiting 668 ms; commit 1714 ms)
2026-11-07T19:28:35Z [load_day 2023-05-01] 2023-05-01 hour 6: 151108 events in 4 files, snapshot 6383573543847632547 (task 12489 ms: download 5616 ms overlapping gunzip 790 ms, parse 3275 ms, sort 99 ms, write 7007 ms, index 398 ms: 1 candidates, 60111 repo-hours, 707 releases; waiting 572 ms; commit 1557 ms)
2026-11-07T19:28:41Z [load_day 2023-05-01] 2023-05-01 hour 7: 148928 events in 3 files, snapshot 3805081730279976261 (task 13706 ms: download 6700 ms overlapping gunzip 883 ms, parse 3562 ms, sort 111 ms, write 7043 ms, index 394 ms: 2 candidates, 56491 repo-hours, 789 releases; waiting 1338 ms; commit 1522 ms)
2026-11-07T19:28:50Z [load_day 2023-05-01] 2023-05-01 hour 8: 154501 events in 4 files, snapshot 1112337982352352482 (task 13377 ms: download 6445 ms overlapping gunzip 838 ms, parse 3488 ms, sort 103 ms, write 7133 ms, index 398 ms: 3 candidates, 61496 repo-hours, 759 releases; waiting 1118 ms; commit 1571 ms)
2026-11-07T19:28:54Z [load_day 2023-05-01] 2023-05-01 hour 9: 154385 events in 4 files, snapshot 5342791976527558872 (task 12230 ms: download 5428 ms overlapping gunzip 773 ms, parse 3318 ms, sort 106 ms, write 6796 ms, index 403 ms: 6 candidates, 61742 repo-hours, 962 releases; waiting 533 ms; commit 1482 ms)
2026-11-07T19:29:03Z [load_day 2023-05-01] 2023-05-01 hour 10: 159485 events in 4 files, snapshot 5826257823564933970 (task 12962 ms: download 5719 ms overlapping gunzip 797 ms, parse 3361 ms, sort 106 ms, write 7309 ms, index 429 ms: 4 candidates, 64386 repo-hours, 822 releases; waiting 643 ms; commit 1672 ms)
2026-11-07T19:29:08Z [load_day 2023-05-01] 2023-05-01 hour 11: 165335 events in 4 files, snapshot 6713948637517263956 (task 13377 ms: download 7996 ms overlapping gunzip 867 ms, parse 3562 ms, sort 105 ms, write 7581 ms, index 448 ms: 3 candidates, 64505 repo-hours, 761 releases; waiting 506 ms; commit 1487 ms)
2026-11-07T19:29:20Z [load_day 2023-05-01] 2023-05-01 hour 12: 221464 events in 5 files, snapshot 8967116021643957606 (task 15749 ms: download 8593 ms overlapping gunzip 930 ms, parse 4035 ms, sort 122 ms, write 8830 ms, index 686 ms: 8 candidates, 96299 repo-hours, 970 releases; waiting 827 ms; commit 1525 ms)
2026-11-07T19:29:25Z [load_day 2023-05-01] 2023-05-01 hour 13: 198473 events in 4 files, snapshot 7632480770534666948 (task 15398 ms: download 8391 ms overlapping gunzip 946 ms, parse 4034 ms, sort 125 ms, write 8810 ms, index 623 ms: 3 candidates, 93302 repo-hours, 882 releases; waiting 566 ms; commit 2672 ms)
2026-11-07T19:29:37Z [load_day 2023-05-01] 2023-05-01 hour 14: 193725 events in 4 files, snapshot 6667403212332471087 (task 16600 ms: download 9574 ms overlapping gunzip 1021 ms, parse 4263 ms, sort 123 ms, write 9392 ms, index 536 ms: 4 candidates, 82939 repo-hours, 981 releases; waiting 867 ms; commit 1560 ms)
2026-11-07T19:29:41Z [load_day 2023-05-01] 2023-05-01 hour 15: 171779 events in 4 files, snapshot 8719864387083217803 (task 14855 ms: download 8903 ms overlapping gunzip 920 ms, parse 3836 ms, sort 93 ms, write 8456 ms, index 588 ms: 4 candidates, 76276 repo-hours, 1006 releases; waiting 687 ms; commit 1829 ms)
2026-11-07T19:29:53Z [load_day 2023-05-01] 2023-05-01 hour 16: 176603 events in 4 files, snapshot 3188240099224991503 (task 14915 ms: download 8733 ms overlapping gunzip 949 ms, parse 3958 ms, sort 102 ms, write 8572 ms, index 532 ms: 1 candidates, 80445 repo-hours, 794 releases; waiting 491 ms; commit 1579 ms)
2026-11-07T19:29:58Z [load_day 2023-05-01] 2023-05-01 hour 17: 184451 events in 4 files, snapshot 3280913524868416847 (task 15748 ms: download 9213 ms overlapping gunzip 1000 ms, parse 4254 ms, sort 112 ms, write 8891 ms, index 597 ms: 7 candidates, 79244 repo-hours, 939 releases; waiting 622 ms; commit 1635 ms)
2026-11-07T19:30:12Z [load_day 2023-05-01] 2023-05-01 hour 18: 192353 events in 4 files, snapshot 3145615950103041172 (task 17671 ms: download 10658 ms overlapping gunzip 1043 ms, parse 4368 ms, sort 118 ms, write 9534 ms, index 561 ms: 7 candidates, 83412 repo-hours, 975 releases; waiting 1785 ms; commit 1711 ms)
2026-11-07T19:30:15Z [load_day 2023-05-01] 2023-05-01 hour 19: 187563 events in 4 files, snapshot 1790619256303209601 (task 16431 ms: download 9529 ms overlapping gunzip 1067 ms, parse 4478 ms, sort 130 ms, write 9294 ms, index 519 ms: 2 candidates, 78163 repo-hours, 1137 releases; waiting 565 ms; commit 1663 ms)
2026-11-07T19:30:43Z [load_day 2023-05-01] 2023-05-01 hour 20: 212496 events in 5 files, snapshot 8596687013866757889 (task 30346 ms: download 23706 ms overlapping gunzip 2075 ms, parse 7659 ms, sort 138 ms, write 14072 ms, index 608 ms: 9 candidates, 96987 repo-hours, 827 releases; waiting 5465 ms; commit 1824 ms)
2026-11-07T19:30:45Z [load_day 2023-05-01] 2023-05-01 hour 21: 188240 events in 4 files, snapshot 3643886249605231751 (task 19468 ms: download 12084 ms overlapping gunzip 1273 ms, parse 5439 ms, sort 122 ms, write 10992 ms, index 512 ms: 9 candidates, 77957 repo-hours, 1043 releases; waiting 824 ms; commit 1853 ms)
2026-11-07T19:30:59Z [load_day 2023-05-01] 2023-05-01 hour 22: 160288 events in 4 files, snapshot 1756193916267204607 (task 14889 ms: download 9496 ms overlapping gunzip 943 ms, parse 3878 ms, sort 114 ms, write 8669 ms, index 461 ms: 5 candidates, 64463 repo-hours, 834 releases; waiting 566 ms; commit 1711 ms)
2026-11-07T19:31:00Z [load_day 2023-05-01] 2023-05-01 hour 23: 145940 events in 3 files, snapshot 2558334325584003254 (task 13383 ms: download 6322 ms overlapping gunzip 866 ms, parse 3637 ms, sort 104 ms, write 7525 ms, index 421 ms: 2 candidates, 59329 repo-hours, 684 releases; waiting 544 ms; commit 1588 ms)
2026-11-07T19:31:07Z [load_day 2023-05-01] 2023-05-01: repo_hour merged from 24 files to 1 (1680981 rows, 7125 ms)
2026-11-07T19:31:09Z [load_day 2023-05-01] 2023-05-01: index committed in 8940 ms (23 + 1 + 24 files)
2026-11-07T19:31:09Z [load_day 2023-05-01] 2023-05-01: 3980602 events in 24 hours
2026-11-07T19:31:09Z [load_day 2023-05-01] attempt 1 succeeded