orcaLog in

Runs · run 1131

load_day 2023-05-29, attempt 1 succeeded

Host
caf251265124:412410
Started
2026-11-07T20:47:16Z
Ended
2026-11-07T20:50:22Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1131/load_day/2023-05-29/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T20:47:16Z [load_day 2023-05-29] attempt 1 started
2026-11-07T20:47:17Z [load_day 2023-05-29] 2023-05-29: 2 hours in flight, about 1200 MB
2026-11-07T20:47:30Z [load_day 2023-05-29] 2023-05-29 hour 0: 120062 events in 3 files, snapshot 4606192930341764707 (task 9440 ms: download 4262 ms overlapping gunzip 514 ms, parse 2298 ms, sort 66 ms, write 4806 ms, index 401 ms: 6 candidates, 64175 repo-hours, 827 releases; waiting 888 ms; commit 1575 ms)
2026-11-07T20:47:31Z [load_day 2023-05-29] 2023-05-29 hour 1: 142561 events in 3 files, snapshot 8737966009063721772 (task 11089 ms: download 4585 ms overlapping gunzip 678 ms, parse 2966 ms, sort 90 ms, write 6034 ms, index 515 ms: 3 candidates, 77689 repo-hours, 761 releases; waiting 494 ms; commit 1639 ms)
2026-11-07T20:47:41Z [load_day 2023-05-29] 2023-05-29 hour 2: 132058 events in 3 files, snapshot 3290970939028810542 (task 10293 ms: download 4619 ms overlapping gunzip 647 ms, parse 2703 ms, sort 93 ms, write 5511 ms, index 551 ms: 8 candidates, 63952 repo-hours, 716 releases; waiting 500 ms; commit 1731 ms)
2026-11-07T20:47:42Z [load_day 2023-05-29] 2023-05-29 hour 3: 124385 events in 3 files, snapshot 2423753929587207433 (task 9476 ms: download 2196 ms overlapping gunzip 589 ms, parse 2551 ms, sort 82 ms, write 5059 ms, index 355 ms: 6 candidates, 54175 repo-hours, 750 releases; waiting 509 ms; commit 1765 ms)
2026-11-07T20:47:51Z [load_day 2023-05-29] 2023-05-29 hour 4: 123540 events in 3 files, snapshot 7719340163949589914 (task 9381 ms: download 2258 ms overlapping gunzip 578 ms, parse 2478 ms, sort 72 ms, write 5142 ms, index 333 ms: 5 candidates, 49669 repo-hours, 632 releases; waiting 533 ms; commit 1660 ms)
2026-11-07T20:47:55Z [load_day 2023-05-29] 2023-05-29 hour 5: 137848 events in 3 files, snapshot 5232858995385728271 (task 11208 ms: download 5116 ms overlapping gunzip 728 ms, parse 3054 ms, sort 86 ms, write 6197 ms, index 384 ms: 4 candidates, 57325 repo-hours, 708 releases; waiting 504 ms; commit 1975 ms)
2026-11-07T20:48:05Z [load_day 2023-05-29] 2023-05-29 hour 6: 154790 events in 4 files, snapshot 3804666158972948855 (task 12850 ms: download 5764 ms overlapping gunzip 762 ms, parse 3179 ms, sort 99 ms, write 6703 ms, index 500 ms: 3 candidates, 71933 repo-hours, 938 releases; waiting 1333 ms; commit 1720 ms)
2026-11-07T20:48:15Z [load_day 2023-05-29] 2023-05-29 hour 7: 159791 events in 4 files, snapshot 1476258696442765380 (task 19585 ms: download 5956 ms overlapping gunzip 857 ms, parse 3677 ms, sort 98 ms, write 13512 ms, index 474 ms: 4 candidates, 71943 repo-hours, 774 releases; waiting 521 ms; commit 1833 ms)
2026-11-07T20:48:24Z [load_day 2023-05-29] 2023-05-29 hour 8: 174484 events in 4 files, snapshot 8724211427732293200 (task 18462 ms: download 12260 ms overlapping gunzip 863 ms, parse 3771 ms, sort 108 ms, write 12054 ms, index 491 ms: 8 candidates, 77372 repo-hours, 833 releases; waiting 513 ms; commit 2333 ms)
2026-11-07T20:48:35Z [load_day 2023-05-29] 2023-05-29 hour 9: 172019 events in 4 files, snapshot 5308992851562350269 (task 19344 ms: download 8225 ms overlapping gunzip 868 ms, parse 3714 ms, sort 98 ms, write 13360 ms, index 531 ms: 5 candidates, 76807 repo-hours, 965 releases; waiting 460 ms; commit 1608 ms)
2026-11-07T20:48:42Z [load_day 2023-05-29] 2023-05-29 hour 10: 175199 events in 4 files, snapshot 4242973332101044478 (task 13531 ms: download 7823 ms overlapping gunzip 865 ms, parse 3682 ms, sort 101 ms, write 7588 ms, index 497 ms: 4 candidates, 77722 repo-hours, 936 releases; waiting 518 ms; commit 4783 ms)
2026-11-07T20:48:51Z [load_day 2023-05-29] 2023-05-29 hour 11: 174929 events in 4 files, snapshot 5567446751231765847 (task 15898 ms: download 9983 ms overlapping gunzip 910 ms, parse 3965 ms, sort 106 ms, write 7837 ms, index 519 ms: 8 candidates, 77940 repo-hours, 959 releases; waiting 2256 ms; commit 1657 ms)
2026-11-07T20:49:00Z [load_day 2023-05-29] 2023-05-29 hour 12: 229397 events in 5 files, snapshot 4480636345217257218 (task 17253 ms: download 9547 ms overlapping gunzip 966 ms, parse 4314 ms, sort 118 ms, write 8966 ms, index 670 ms: 7 candidates, 108093 repo-hours, 1049 releases; waiting 1758 ms; commit 1742 ms)
2026-11-07T20:49:07Z [load_day 2023-05-29] 2023-05-29 hour 13: 199232 events in 4 files, snapshot 2731780226392934269 (task 14559 ms: download 7552 ms overlapping gunzip 849 ms, parse 3786 ms, sort 123 ms, write 8362 ms, index 649 ms: 6 candidates, 100710 repo-hours, 881 releases; waiting 523 ms; commit 1679 ms)
2026-11-07T20:49:16Z [load_day 2023-05-29] 2023-05-29 hour 14: 190603 events in 4 files, snapshot 5576237413096541236 (task 15050 ms: download 8489 ms overlapping gunzip 936 ms, parse 4026 ms, sort 122 ms, write 8539 ms, index 608 ms: 7 candidates, 87804 repo-hours, 907 releases; waiting 521 ms; commit 1766 ms)
2026-11-07T20:49:22Z [load_day 2023-05-29] 2023-05-29 hour 15: 186084 events in 4 files, snapshot 1093282844201952453 (task 14902 ms: download 8376 ms overlapping gunzip 941 ms, parse 3984 ms, sort 107 ms, write 8665 ms, index 573 ms: 5 candidates, 83529 repo-hours, 1062 releases; waiting 327 ms; commit 1749 ms)
2026-11-07T20:49:30Z [load_day 2023-05-29] 2023-05-29 hour 16: 182755 events in 4 files, snapshot 5293798538560606807 (task 13658 ms: download 7736 ms overlapping gunzip 822 ms, parse 3657 ms, sort 119 ms, write 7910 ms, index 543 ms: 6 candidates, 81530 repo-hours, 993 releases; waiting 313 ms; commit 1744 ms)
2026-11-07T20:49:38Z [load_day 2023-05-29] 2023-05-29 hour 17: 175660 events in 4 files, snapshot 5427188471036870007 (task 14336 ms: download 8561 ms overlapping gunzip 803 ms, parse 3461 ms, sort 110 ms, write 7702 ms, index 522 ms: 3 candidates, 75389 repo-hours, 839 releases; waiting 1361 ms; commit 1726 ms)
2026-11-07T20:49:44Z [load_day 2023-05-29] 2023-05-29 hour 18: 178074 events in 4 files, snapshot 4774224077172297595 (task 13352 ms: download 7613 ms overlapping gunzip 838 ms, parse 3604 ms, sort 112 ms, write 7376 ms, index 580 ms: 5 candidates, 80463 repo-hours, 1038 releases; waiting 517 ms; commit 1719 ms)
2026-11-07T20:49:51Z [load_day 2023-05-29] 2023-05-29 hour 19: 160872 events in 4 files, snapshot 8192311526197569486 (task 12236 ms: download 5206 ms overlapping gunzip 755 ms, parse 3250 ms, sort 104 ms, write 6835 ms, index 481 ms: 6 candidates, 68592 repo-hours, 936 releases; waiting 508 ms; commit 1967 ms)
2026-11-07T20:49:57Z [load_day 2023-05-29] 2023-05-29 hour 20: 155337 events in 4 files, snapshot 4592494137083534451 (task 11900 ms: download 5103 ms overlapping gunzip 713 ms, parse 3112 ms, sort 139 ms, write 6473 ms, index 461 ms: 3 candidates, 67196 repo-hours, 854 releases; waiting 612 ms; commit 1876 ms)
2026-11-07T20:50:02Z [load_day 2023-05-29] 2023-05-29 hour 21: 141633 events in 3 files, snapshot 5355745035340651751 (task 10827 ms: download 4597 ms overlapping gunzip 658 ms, parse 2885 ms, sort 95 ms, write 6027 ms, index 427 ms: 12 candidates, 60802 repo-hours, 867 releases; waiting 405 ms; commit 1762 ms)
2026-11-07T20:50:08Z [load_day 2023-05-29] 2023-05-29 hour 22: 129636 events in 3 files, snapshot 8235418455771834050 (task 9925 ms: download 4383 ms overlapping gunzip 598 ms, parse 2552 ms, sort 75 ms, write 5452 ms, index 457 ms: 5 candidates, 54057 repo-hours, 735 releases; waiting 528 ms; commit 1711 ms)
2026-11-07T20:50:13Z [load_day 2023-05-29] 2023-05-29 hour 23: 121556 events in 3 files, snapshot 3735598867855856418 (task 9594 ms: download 2597 ms overlapping gunzip 591 ms, parse 2547 ms, sort 88 ms, write 5190 ms, index 374 ms: 5 candidates, 51229 repo-hours, 604 releases; waiting 515 ms; commit 1712 ms)
2026-11-07T20:50:20Z [load_day 2023-05-29] 2023-05-29: repo_hour merged from 24 files to 1 (1740096 rows, 7129 ms)
2026-11-07T20:50:22Z [load_day 2023-05-29] 2023-05-29: index committed in 9417 ms (24 + 1 + 24 files)
2026-11-07T20:50:22Z [load_day 2023-05-29] 2023-05-29: 3842505 events in 24 hours
2026-11-07T20:50:22Z [load_day 2023-05-29] attempt 1 succeeded