orcaLog in

Runs · run 1070

load_day 2023-05-08, attempt 1 succeeded

Host
caf251265124:346380
Started
2026-11-07T19:48:26Z
Ended
2026-11-07T19:51:46Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1070/load_day/2023-05-08/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T19:48:26Z [load_day 2023-05-08] attempt 1 started
2026-11-07T19:48:27Z [load_day 2023-05-08] 2023-05-08: 2 hours in flight, about 1200 MB
2026-11-07T19:48:41Z [load_day 2023-05-08] 2023-05-08 hour 0: 138789 events in 3 files, snapshot 6315578906332179566 (task 10674 ms: download 4842 ms overlapping gunzip 696 ms, parse 3028 ms, sort 98 ms, write 5611 ms, index 457 ms: 7 candidates, 70504 repo-hours, 832 releases; waiting 499 ms; commit 1725 ms)
2026-11-07T19:48:44Z [load_day 2023-05-08] 2023-05-08 hour 1: 160760 events in 4 files, snapshot 3720544333960925163 (task 13154 ms: download 5809 ms overlapping gunzip 868 ms, parse 3533 ms, sort 106 ms, write 7250 ms, index 548 ms: 25 candidates, 84183 repo-hours, 751 releases; waiting 507 ms; commit 1652 ms)
2026-11-07T19:48:55Z [load_day 2023-05-08] 2023-05-08 hour 2: 151670 events in 4 files, snapshot 4366445699837464248 (task 12727 ms: download 5872 ms overlapping gunzip 869 ms, parse 3480 ms, sort 92 ms, write 7054 ms, index 407 ms: 11 candidates, 63384 repo-hours, 684 releases; waiting 517 ms; commit 1822 ms)
2026-11-07T19:48:57Z [load_day 2023-05-08] 2023-05-08 hour 3: 149236 events in 3 files, snapshot 2490085648057764057 (task 12592 ms: download 5869 ms overlapping gunzip 835 ms, parse 3365 ms, sort 86 ms, write 6919 ms, index 383 ms: 5 candidates, 62771 repo-hours, 829 releases; waiting 720 ms; commit 1608 ms)
2026-11-07T19:49:08Z [load_day 2023-05-08] 2023-05-08 hour 4: 145132 events in 3 files, snapshot 1238968387489108752 (task 12216 ms: download 5750 ms overlapping gunzip 849 ms, parse 3442 ms, sort 85 ms, write 6719 ms, index 363 ms: 4 candidates, 59078 repo-hours, 616 releases; waiting 518 ms; commit 1753 ms)
2026-11-07T19:49:11Z [load_day 2023-05-08] 2023-05-08 hour 5: 156577 events in 4 files, snapshot 3953559312794111813 (task 13595 ms: download 6414 ms overlapping gunzip 940 ms, parse 3777 ms, sort 82 ms, write 7539 ms, index 365 ms: 9 candidates, 60472 repo-hours, 691 releases; waiting 622 ms; commit 1670 ms)
2026-11-07T19:49:26Z [load_day 2023-05-08] 2023-05-08 hour 6: 173930 events in 4 files, snapshot 970261479755477765 (task 17228 ms: download 11199 ms overlapping gunzip 1051 ms, parse 4281 ms, sort 99 ms, write 8362 ms, index 466 ms: 7 candidates, 75307 repo-hours, 912 releases; waiting 2648 ms; commit 1550 ms)
2026-11-07T19:49:27Z [load_day 2023-05-08] 2023-05-08 hour 7: 159105 events in 4 files, snapshot 2856822340510839653 (task 13641 ms: download 6031 ms overlapping gunzip 871 ms, parse 3645 ms, sort 93 ms, write 7737 ms, index 469 ms: 6 candidates, 72256 repo-hours, 757 releases; waiting 532 ms; commit 1635 ms)
2026-11-07T19:49:41Z [load_day 2023-05-08] 2023-05-08 hour 8: 171223 events in 4 files, snapshot 6297524732203938070 (task 14153 ms: download 8423 ms overlapping gunzip 883 ms, parse 3717 ms, sort 112 ms, write 8053 ms, index 507 ms: 7 candidates, 81216 repo-hours, 927 releases; waiting 550 ms; commit 1781 ms)
2026-11-07T19:49:42Z [load_day 2023-05-08] 2023-05-08 hour 9: 172463 events in 4 files, snapshot 8143350218933185713 (task 13992 ms: download 8204 ms overlapping gunzip 870 ms, parse 3722 ms, sort 105 ms, write 8028 ms, index 528 ms: 11 candidates, 78265 repo-hours, 1052 releases; waiting 452 ms; commit 1576 ms)
2026-11-07T19:49:57Z [load_day 2023-05-08] 2023-05-08 hour 10: 177462 events in 4 files, snapshot 8393043671103153109 (task 15706 ms: download 9551 ms overlapping gunzip 1002 ms, parse 4197 ms, sort 126 ms, write 8890 ms, index 544 ms: 14 candidates, 77848 repo-hours, 908 releases; waiting 635 ms; commit 1609 ms)
2026-11-07T19:49:59Z [load_day 2023-05-08] 2023-05-08 hour 11: 165511 events in 4 files, snapshot 6207146114044945360 (task 14890 ms: download 6950 ms overlapping gunzip 810 ms, parse 3578 ms, sort 108 ms, write 7665 ms, index 531 ms: 8 candidates, 75778 repo-hours, 929 releases; waiting 1874 ms; commit 1638 ms)
2026-11-07T19:50:14Z [load_day 2023-05-08] 2023-05-08 hour 12: 214675 events in 5 files, snapshot 4161968208205098013 (task 16380 ms: download 8852 ms overlapping gunzip 956 ms, parse 4204 ms, sort 125 ms, write 9552 ms, index 657 ms: 11 candidates, 104940 repo-hours, 933 releases; waiting 580 ms; commit 1724 ms)
2026-11-07T19:50:16Z [load_day 2023-05-08] 2023-05-08 hour 13: 195001 events in 4 files, snapshot 8299149851992871249 (task 16374 ms: download 8928 ms overlapping gunzip 994 ms, parse 4186 ms, sort 118 ms, write 9673 ms, index 631 ms: 9 candidates, 99445 repo-hours, 928 releases; waiting 382 ms; commit 1739 ms)
2026-11-07T19:50:32Z [load_day 2023-05-08] 2023-05-08 hour 14: 188351 events in 4 files, snapshot 228945147987900591 (task 16286 ms: download 9706 ms overlapping gunzip 983 ms, parse 4263 ms, sort 114 ms, write 9553 ms, index 570 ms: 11 candidates, 90341 repo-hours, 929 releases; waiting 522 ms; commit 1759 ms)
2026-11-07T19:50:33Z [load_day 2023-05-08] 2023-05-08 hour 15: 175217 events in 4 files, snapshot 3067366526631442929 (task 15160 ms: download 9038 ms overlapping gunzip 933 ms, parse 3998 ms, sort 103 ms, write 8766 ms, index 536 ms: 6 candidates, 84374 repo-hours, 1004 releases; waiting 493 ms; commit 1693 ms)
2026-11-07T19:50:48Z [load_day 2023-05-08] 2023-05-08 hour 16: 167603 events in 4 files, snapshot 3553817206602814903 (task 14974 ms: download 9423 ms overlapping gunzip 893 ms, parse 3914 ms, sort 93 ms, write 8704 ms, index 537 ms: 8 candidates, 80008 repo-hours, 877 releases; waiting 516 ms; commit 1719 ms)
2026-11-07T19:50:49Z [load_day 2023-05-08] 2023-05-08 hour 17: 180832 events in 4 files, snapshot 4986667893511361451 (task 14831 ms: download 8502 ms overlapping gunzip 885 ms, parse 3974 ms, sort 104 ms, write 8627 ms, index 562 ms: 10 candidates, 84000 repo-hours, 890 releases; waiting 377 ms; commit 1565 ms)
2026-11-07T19:51:04Z [load_day 2023-05-08] 2023-05-08 hour 18: 183073 events in 4 files, snapshot 1203621734092387772 (task 15669 ms: download 9042 ms overlapping gunzip 959 ms, parse 4203 ms, sort 105 ms, write 8977 ms, index 575 ms: 7 candidates, 85668 repo-hours, 1055 releases; waiting 527 ms; commit 1630 ms)
2026-11-07T19:51:06Z [load_day 2023-05-08] 2023-05-08 hour 19: 173635 events in 4 files, snapshot 2712566080599465547 (task 16052 ms: download 9979 ms overlapping gunzip 957 ms, parse 4086 ms, sort 101 ms, write 8760 ms, index 550 ms: 9 candidates, 77521 repo-hours, 1024 releases; waiting 1312 ms; commit 1522 ms)
2026-11-07T19:51:21Z [load_day 2023-05-08] 2023-05-08 hour 20: 165283 events in 4 files, snapshot 1280821280761311239 (task 16575 ms: download 10738 ms overlapping gunzip 935 ms, parse 4006 ms, sort 92 ms, write 8040 ms, index 512 ms: 5 candidates, 72788 repo-hours, 906 releases; waiting 2675 ms; commit 1660 ms)
2026-11-07T19:51:23Z [load_day 2023-05-08] 2023-05-08 hour 21: 167895 events in 4 files, snapshot 4861142433307023697 (task 15028 ms: download 9290 ms overlapping gunzip 972 ms, parse 4138 ms, sort 106 ms, write 8533 ms, index 464 ms: 11 candidates, 67431 repo-hours, 941 releases; waiting 511 ms; commit 1822 ms)
2026-11-07T19:51:34Z [load_day 2023-05-08] 2023-05-08 hour 22: 144005 events in 3 files, snapshot 7740095149141769235 (task 11888 ms: download 5427 ms overlapping gunzip 728 ms, parse 3146 ms, sort 90 ms, write 6688 ms, index 376 ms: 9 candidates, 60991 repo-hours, 757 releases; waiting 541 ms; commit 1739 ms)
2026-11-07T19:51:36Z [load_day 2023-05-08] 2023-05-08 hour 23: 134098 events in 3 files, snapshot 507116848693198107 (task 11366 ms: download 5427 ms overlapping gunzip 756 ms, parse 3239 ms, sort 78 ms, write 6212 ms, index 362 ms: 7 candidates, 56711 repo-hours, 626 releases; waiting 500 ms; commit 1699 ms)
2026-11-07T19:51:44Z [load_day 2023-05-08] 2023-05-08: repo_hour merged from 24 files to 1 (1825280 rows, 7854 ms)
2026-11-07T19:51:46Z [load_day 2023-05-08] 2023-05-08: index committed in 10196 ms (24 + 1 + 24 files)
2026-11-07T19:51:46Z [load_day 2023-05-08] 2023-05-08: 4011526 events in 24 hours
2026-11-07T19:51:46Z [load_day 2023-05-08] attempt 1 succeeded