orcaLog in

Runs · run 1472

load_day 2023-09-07, attempt 1 succeeded

Host
3c2b29713ce8:304483
Started
2026-11-08T00:36:05Z
Ended
2026-11-08T00:40:07Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1472/load_day/2023-09-07/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:36:05Z [load_day 2023-09-07] attempt 1 started
2026-11-08T00:36:07Z [load_day 2023-09-07] 2023-09-07: 2 hours in flight, about 1200 MB
2026-11-08T00:36:26Z [load_day 2023-09-07] 2023-09-07 hour 0: 162432 events in 4 files, snapshot 4107394514169958852 (task 15081 ms: download 6687 ms overlapping gunzip 865 ms, parse 4086 ms, sort 119 ms, write 8625 ms, index 553 ms: 10 candidates, 86036 repo-hours, 960 releases; waiting 518 ms; commit 2229 ms)
2026-11-08T00:36:30Z [load_day 2023-09-07] 2023-09-07 hour 1: 153439 events in 4 files, snapshot 3011857875386303188 (task 14271 ms: download 5758 ms overlapping gunzip 762 ms, parse 3580 ms, sort 172 ms, write 8341 ms, index 570 ms: 3 candidates, 85540 repo-hours, 736 releases; waiting 509 ms; commit 4220 ms)
2026-11-08T00:36:42Z [load_day 2023-09-07] 2023-09-07 hour 2: 135950 events in 3 files, snapshot 7907210741776728736 (task 14640 ms: download 6543 ms overlapping gunzip 801 ms, parse 3313 ms, sort 98 ms, write 8955 ms, index 496 ms: 7 candidates, 58806 repo-hours, 771 releases; waiting 676 ms; commit 2244 ms)
2026-11-08T00:36:45Z [load_day 2023-09-07] 2023-09-07 hour 3: 131163 events in 3 files, snapshot 7301275530785541716 (task 13116 ms: download 5435 ms overlapping gunzip 847 ms, parse 3876 ms, sort 141 ms, write 7053 ms, index 424 ms: 3 candidates, 57228 repo-hours, 872 releases; waiting 187 ms; commit 2729 ms)
2026-11-08T00:36:57Z [load_day 2023-09-07] 2023-09-07 hour 4: 129643 events in 3 files, snapshot 5962732681963261710 (task 12345 ms: download 5190 ms overlapping gunzip 799 ms, parse 3531 ms, sort 100 ms, write 6686 ms, index 481 ms: 4 candidates, 53561 repo-hours, 793 releases; waiting 497 ms; commit 3399 ms)
2026-11-08T00:37:01Z [load_day 2023-09-07] 2023-09-07 hour 5: 144228 events in 3 files, snapshot 458529496010300535 (task 14786 ms: download 5745 ms overlapping gunzip 849 ms, parse 3879 ms, sort 111 ms, write 8719 ms, index 496 ms: 2 candidates, 60597 repo-hours, 775 releases; waiting 481 ms; commit 3189 ms)
2026-11-08T00:37:14Z [load_day 2023-09-07] 2023-09-07 hour 6: 175784 events in 4 files, snapshot 6388500408198250225 (task 16785 ms: download 9755 ms overlapping gunzip 1060 ms, parse 4780 ms, sort 110 ms, write 9482 ms, index 590 ms: 5 candidates, 81845 repo-hours, 884 releases; waiting 499 ms; commit 2338 ms)
2026-11-08T00:37:22Z [load_day 2023-09-07] 2023-09-07 hour 7: 179325 events in 4 files, snapshot 4115738342052380605 (task 18083 ms: download 10509 ms overlapping gunzip 1233 ms, parse 5434 ms, sort 137 ms, write 9965 ms, index 710 ms: 1 candidates, 77922 repo-hours, 1030 releases; waiting 302 ms; commit 4149 ms)
2026-11-08T00:37:36Z [load_day 2023-09-07] 2023-09-07 hour 8: 184009 events in 4 files, snapshot 2687704667162343347 (task 21007 ms: download 12467 ms overlapping gunzip 1179 ms, parse 5300 ms, sort 141 ms, write 12739 ms, index 539 ms: 6 candidates, 82588 repo-hours, 1065 releases; waiting 781 ms; commit 2316 ms)
2026-11-08T00:37:44Z [load_day 2023-09-07] 2023-09-07 hour 9: 192778 events in 4 files, snapshot 6922127826740364984 (task 20839 ms: download 11890 ms overlapping gunzip 1264 ms, parse 5744 ms, sort 139 ms, write 12069 ms, index 818 ms: 4 candidates, 83626 repo-hours, 1521 releases; waiting 480 ms; commit 4191 ms)
2026-11-08T00:38:00Z [load_day 2023-09-07] 2023-09-07 hour 10: 182365 events in 4 files, snapshot 5859414481883621695 (task 20264 ms: download 12003 ms overlapping gunzip 1273 ms, parse 5676 ms, sort 157 ms, write 11806 ms, index 666 ms: 3 candidates, 78368 repo-hours, 1168 releases; waiting 465 ms; commit 4419 ms)
2026-11-08T00:38:04Z [load_day 2023-09-07] 2023-09-07 hour 11: 182332 events in 4 files, snapshot 5446787006958361879 (task 19280 ms: download 11222 ms overlapping gunzip 968 ms, parse 4779 ms, sort 167 ms, write 11769 ms, index 853 ms: 4 candidates, 77250 repo-hours, 1142 releases; waiting 488 ms; commit 4174 ms)
2026-11-08T00:38:20Z [load_day 2023-09-07] 2023-09-07 hour 12: 240952 events in 5 files, snapshot 706839981027486590 (task 21459 ms: download 14351 ms overlapping gunzip 1269 ms, parse 5807 ms, sort 216 ms, write 12174 ms, index 695 ms: 3 candidates, 111239 repo-hours, 1377 releases; waiting 554 ms; commit 2222 ms)
2026-11-08T00:38:30Z [load_day 2023-09-07] 2023-09-07 hour 13: 219569 events in 5 files, snapshot 1954602662045989916 (task 20063 ms: download 13657 ms overlapping gunzip 1121 ms, parse 5172 ms, sort 201 ms, write 11949 ms, index 696 ms: 8 candidates, 105818 repo-hours, 1120 releases; waiting 531 ms; commit 8053 ms)
2026-11-08T00:38:42Z [load_day 2023-09-07] 2023-09-07 hour 14: 208680 events in 5 files, snapshot 7018242536275189016 (task 21070 ms: download 10915 ms overlapping gunzip 1254 ms, parse 5538 ms, sort 130 ms, write 11859 ms, index 661 ms: 7 candidates, 92469 repo-hours, 1222 releases; waiting 1185 ms; commit 2404 ms)
2026-11-08T00:38:44Z [load_day 2023-09-07] 2023-09-07 hour 15: 189741 events in 4 files, snapshot 7949871063988598599 (task 18791 ms: download 9919 ms overlapping gunzip 1061 ms, parse 4790 ms, sort 126 ms, write 11106 ms, index 934 ms: 7 candidates, 86140 repo-hours, 1182 releases; waiting 503 ms; commit 2598 ms)
2026-11-08T00:39:01Z [load_day 2023-09-07] 2023-09-07 hour 16: 181775 events in 4 files, snapshot 3630629349311255407 (task 18400 ms: download 10693 ms overlapping gunzip 1105 ms, parse 4770 ms, sort 120 ms, write 10904 ms, index 639 ms: 10 candidates, 81870 repo-hours, 1049 releases; waiting 541 ms; commit 1985 ms)
2026-11-08T00:39:04Z [load_day 2023-09-07] 2023-09-07 hour 17: 180141 events in 4 files, snapshot 18381099099736883 (task 17156 ms: download 10063 ms overlapping gunzip 1006 ms, parse 4412 ms, sort 177 ms, write 10274 ms, index 540 ms: 11 candidates, 77999 repo-hours, 1035 releases; waiting 489 ms; commit 2770 ms)
2026-11-08T00:39:20Z [load_day 2023-09-07] 2023-09-07 hour 18: 181498 events in 4 files, snapshot 7936932404452924778 (task 17369 ms: download 9561 ms overlapping gunzip 1045 ms, parse 4592 ms, sort 152 ms, write 10216 ms, index 600 ms: 21 candidates, 80936 repo-hours, 1040 releases; waiting 510 ms; commit 2651 ms)
2026-11-08T00:39:22Z [load_day 2023-09-07] 2023-09-07 hour 19: 170027 events in 4 files, snapshot 2116342113750560285 (task 15781 ms: download 9182 ms overlapping gunzip 937 ms, parse 4235 ms, sort 108 ms, write 9436 ms, index 548 ms: 16 candidates, 69779 repo-hours, 947 releases; waiting 192 ms; commit 2214 ms)
2026-11-08T00:39:38Z [load_day 2023-09-07] 2023-09-07 hour 20: 169106 events in 4 files, snapshot 629464986501984426 (task 17137 ms: download 9756 ms overlapping gunzip 989 ms, parse 4711 ms, sort 182 ms, write 10131 ms, index 538 ms: 9 candidates, 69369 repo-hours, 944 releases; waiting 292 ms; commit 2272 ms)
2026-11-08T00:39:41Z [load_day 2023-09-07] 2023-09-07 hour 21: 148175 events in 3 files, snapshot 4776737445210396907 (task 15500 ms: download 6985 ms overlapping gunzip 773 ms, parse 3555 ms, sort 138 ms, write 9482 ms, index 421 ms: 8 candidates, 60301 repo-hours, 961 releases; waiting 894 ms; commit 3126 ms)
2026-11-08T00:39:54Z [load_day 2023-09-07] 2023-09-07 hour 22: 135057 events in 3 files, snapshot 2617377822804688214 (task 13830 ms: download 5933 ms overlapping gunzip 879 ms, parse 4077 ms, sort 150 ms, write 7527 ms, index 412 ms: 7 candidates, 53392 repo-hours, 804 releases; waiting 490 ms; commit 3309 ms)
2026-11-08T00:39:56Z [load_day 2023-09-07] 2023-09-07 hour 23: 126440 events in 3 files, snapshot 2847012880249448007 (task 12205 ms: download 5933 ms overlapping gunzip 650 ms, parse 2981 ms, sort 76 ms, write 7389 ms, index 327 ms: 8 candidates, 50707 repo-hours, 648 releases; waiting 509 ms; commit 2162 ms)
2026-11-08T00:40:04Z [load_day 2023-09-07] 2023-09-07: repo_hour merged from 24 files to 1 (1823386 rows, 8379 ms)
2026-11-08T00:40:07Z [load_day 2023-09-07] 2023-09-07: index committed in 10817 ms (24 + 1 + 24 files)
2026-11-08T00:40:07Z [load_day 2023-09-07] 2023-09-07: 4104609 events in 24 hours
2026-11-08T00:40:07Z [load_day 2023-09-07] attempt 1 succeeded