orcaLog in

Runs · run 1463

load_day 2023-09-04, attempt 1 succeeded

Host
3c2b29713ce8:293251
Started
2026-11-08T00:29:59Z
Ended
2026-11-08T00:34:19Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1463/load_day/2023-09-04/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:29:59Z [load_day 2023-09-04] attempt 1 started
2026-11-08T00:30:01Z [load_day 2023-09-04] 2023-09-04: 2 hours in flight, about 1200 MB
2026-11-08T00:30:19Z [load_day 2023-09-04] 2023-09-04 hour 0: 141287 events in 3 files, snapshot 6033205446879720540 (task 14353 ms: download 5944 ms overlapping gunzip 996 ms, parse 4371 ms, sort 172 ms, write 7727 ms, index 606 ms: 7 candidates, 75708 repo-hours, 834 releases; waiting 190 ms; commit 2869 ms)
2026-11-08T00:30:42Z [load_day 2023-09-04] 2023-09-04 hour 1: 154771 events in 4 files, snapshot 9192625437096330885 (task 15870 ms: download 6417 ms overlapping gunzip 962 ms, parse 4121 ms, sort 194 ms, write 9133 ms, index 588 ms: 4 candidates, 84067 repo-hours, 757 releases; waiting 461 ms; commit 22259 ms)
2026-11-08T00:30:44Z [load_day 2023-09-04] 2023-09-04 hour 2: 129232 events in 3 files, snapshot 8377289831710723331 (task 13752 ms: download 6151 ms overlapping gunzip 771 ms, parse 3170 ms, sort 112 ms, write 8439 ms, index 405 ms: 4 candidates, 54162 repo-hours, 653 releases; waiting 530 ms; commit 2149 ms)
2026-11-08T00:30:46Z [load_day 2023-09-04] 2023-09-04 hour 3: 129951 events in 3 files, snapshot 8178759084640400274 (task 13685 ms: download 7058 ms overlapping gunzip 982 ms, parse 4061 ms, sort 104 ms, write 7335 ms, index 367 ms: 1 candidates, 53213 repo-hours, 733 releases; waiting 507 ms; commit 2448 ms)
2026-11-08T00:30:58Z [load_day 2023-09-04] 2023-09-04 hour 4: 126738 events in 3 files, snapshot 2331673118475498971 (task 12431 ms: download 5577 ms overlapping gunzip 694 ms, parse 2967 ms, sort 99 ms, write 7456 ms, index 405 ms: 2 candidates, 49848 repo-hours, 614 releases; waiting 522 ms; commit 2562 ms)
2026-11-08T00:31:01Z [load_day 2023-09-04] 2023-09-04 hour 5: 141061 events in 3 files, snapshot 8970815070477754431 (task 13690 ms: download 5752 ms overlapping gunzip 868 ms, parse 3665 ms, sort 88 ms, write 7729 ms, index 497 ms: 1 candidates, 57411 repo-hours, 710 releases; waiting 533 ms; commit 2566 ms)
2026-11-08T00:31:17Z [load_day 2023-09-04] 2023-09-04 hour 6: 169605 events in 4 files, snapshot 6505228056849065369 (task 17114 ms: download 10095 ms overlapping gunzip 1157 ms, parse 5212 ms, sort 124 ms, write 9689 ms, index 547 ms: 3 candidates, 74454 repo-hours, 882 releases; waiting 186 ms; commit 4018 ms)
2026-11-08T00:31:24Z [load_day 2023-09-04] 2023-09-04 hour 7: 178866 events in 4 files, snapshot 1190061100542473593 (task 21324 ms: download 12589 ms overlapping gunzip 1367 ms, parse 5842 ms, sort 148 ms, write 12589 ms, index 607 ms: 2 candidates, 76443 repo-hours, 937 releases; waiting 530 ms; commit 2654 ms)
2026-11-08T00:31:40Z [load_day 2023-09-04] 2023-09-04 hour 8: 185443 events in 4 files, snapshot 2553375739356842403 (task 20510 ms: download 11919 ms overlapping gunzip 1215 ms, parse 5347 ms, sort 140 ms, write 12392 ms, index 732 ms: 9 candidates, 81284 repo-hours, 956 releases; waiting 397 ms; commit 5150 ms)
2026-11-08T00:31:46Z [load_day 2023-09-04] 2023-09-04 hour 9: 183680 events in 4 files, snapshot 5151494454949544289 (task 20930 ms: download 11390 ms overlapping gunzip 1144 ms, parse 4944 ms, sort 190 ms, write 12840 ms, index 618 ms: 6 candidates, 79088 repo-hours, 1128 releases; waiting 488 ms; commit 2605 ms)
2026-11-08T00:31:57Z [load_day 2023-09-04] 2023-09-04 hour 10: 186364 events in 4 files, snapshot 8440633172555745419 (task 18367 ms: download 11508 ms overlapping gunzip 1148 ms, parse 4986 ms, sort 135 ms, write 10784 ms, index 548 ms: 1 candidates, 79012 repo-hours, 1016 releases; waiting 496 ms; commit 2124 ms)
2026-11-08T00:32:03Z [load_day 2023-09-04] 2023-09-04 hour 11: 189696 events in 4 files, snapshot 5081712879144296859 (task 16562 ms: download 9501 ms overlapping gunzip 1076 ms, parse 4695 ms, sort 140 ms, write 9311 ms, index 549 ms: 2 candidates, 78651 repo-hours, 1039 releases; waiting 476 ms; commit 2278 ms)
2026-11-08T00:32:19Z [load_day 2023-09-04] 2023-09-04 hour 12: 214665 events in 5 files, snapshot 2631104487778266703 (task 21271 ms: download 10328 ms overlapping gunzip 1255 ms, parse 5649 ms, sort 137 ms, write 12503 ms, index 884 ms: 5 candidates, 106210 repo-hours, 958 releases; waiting 521 ms; commit 2352 ms)
2026-11-08T00:32:29Z [load_day 2023-09-04] 2023-09-04 hour 13: 200399 events in 5 files, snapshot 5711223228964239251 (task 24017 ms: download 13962 ms overlapping gunzip 1580 ms, parse 6579 ms, sort 206 ms, write 13758 ms, index 1048 ms: 5 candidates, 101225 repo-hours, 920 releases; waiting 496 ms; commit 2842 ms)
2026-11-08T00:32:46Z [load_day 2023-09-04] 2023-09-04 hour 14: 216193 events in 5 files, snapshot 8822823548213059743 (task 25769 ms: download 17701 ms overlapping gunzip 1596 ms, parse 6825 ms, sort 212 ms, write 15619 ms, index 639 ms: 5 candidates, 95451 repo-hours, 1289 releases; waiting 565 ms; commit 2714 ms)
2026-11-08T00:32:52Z [load_day 2023-09-04] 2023-09-04 hour 15: 192802 events in 4 files, snapshot 4024973351125369248 (task 21408 ms: download 12213 ms overlapping gunzip 1283 ms, parse 5569 ms, sort 137 ms, write 13178 ms, index 719 ms: 5 candidates, 86440 repo-hours, 1071 releases; waiting 229 ms; commit 2825 ms)
2026-11-08T00:33:08Z [load_day 2023-09-04] 2023-09-04 hour 16: 191320 events in 4 files, snapshot 8219711753209858801 (task 21029 ms: download 12252 ms overlapping gunzip 1499 ms, parse 6131 ms, sort 152 ms, write 11911 ms, index 561 ms: 6 candidates, 81054 repo-hours, 959 releases; waiting 510 ms; commit 2543 ms)
2026-11-08T00:33:12Z [load_day 2023-09-04] 2023-09-04 hour 17: 187633 events in 4 files, snapshot 3616420305254978103 (task 19746 ms: download 11648 ms overlapping gunzip 1285 ms, parse 5346 ms, sort 124 ms, write 11027 ms, index 1129 ms: 3 candidates, 76847 repo-hours, 880 releases; waiting 501 ms; commit 2816 ms)
2026-11-08T00:33:29Z [load_day 2023-09-04] 2023-09-04 hour 18: 193832 events in 4 files, snapshot 3341959645831339699 (task 20132 ms: download 11064 ms overlapping gunzip 1327 ms, parse 5668 ms, sort 169 ms, write 11585 ms, index 579 ms: 5 candidates, 81132 repo-hours, 1034 releases; waiting 517 ms; commit 2552 ms)
2026-11-08T00:33:34Z [load_day 2023-09-04] 2023-09-04 hour 19: 181426 events in 4 files, snapshot 8590491427161298398 (task 19121 ms: download 11501 ms overlapping gunzip 1184 ms, parse 4819 ms, sort 198 ms, write 11485 ms, index 668 ms: 7 candidates, 71149 repo-hours, 881 releases; waiting 477 ms; commit 3901 ms)
2026-11-08T00:33:48Z [load_day 2023-09-04] 2023-09-04 hour 20: 172115 events in 4 files, snapshot 5417645590106858443 (task 18124 ms: download 10973 ms overlapping gunzip 1158 ms, parse 4971 ms, sort 172 ms, write 10339 ms, index 596 ms: 4 candidates, 68093 repo-hours, 849 releases; waiting 606 ms; commit 2315 ms)
2026-11-08T00:33:51Z [load_day 2023-09-04] 2023-09-04 hour 21: 157868 events in 4 files, snapshot 5981038545089469642 (task 16698 ms: download 7601 ms overlapping gunzip 997 ms, parse 4050 ms, sort 160 ms, write 10208 ms, index 421 ms: 6 candidates, 59570 repo-hours, 792 releases; waiting 519 ms; commit 2895 ms)
2026-11-08T00:34:07Z [load_day 2023-09-04] 2023-09-04 hour 22: 147216 events in 3 files, snapshot 9124738395984455078 (task 14798 ms: download 6877 ms overlapping gunzip 864 ms, parse 3645 ms, sort 102 ms, write 8819 ms, index 585 ms: 3 candidates, 54887 repo-hours, 718 releases; waiting 532 ms; commit 5170 ms)
2026-11-08T00:34:09Z [load_day 2023-09-04] 2023-09-04 hour 23: 136629 events in 3 files, snapshot 2951250952301268271 (task 14616 ms: download 7075 ms overlapping gunzip 996 ms, parse 4320 ms, sort 154 ms, write 7884 ms, index 359 ms: 3 candidates, 51656 repo-hours, 618 releases; waiting 521 ms; commit 2094 ms)
2026-11-08T00:34:17Z [load_day 2023-09-04] 2023-09-04: repo_hour merged from 24 files to 1 (1777055 rows, 8120 ms)
2026-11-08T00:34:19Z [load_day 2023-09-04] 2023-09-04: index committed in 10151 ms (24 + 1 + 24 files)
2026-11-08T00:34:19Z [load_day 2023-09-04] 2023-09-04: 4108792 events in 24 hours
2026-11-08T00:34:19Z [load_day 2023-09-04] attempt 1 succeeded