orcaLog in

Runs · run 1401

load_day 2023-08-17, attempt 1 succeeded

Host
3c2b29713ce8:235240
Started
2026-11-07T23:58:21Z
Ended
2026-11-08T00:02:21Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1401/load_day/2023-08-17/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T23:58:22Z [load_day 2023-08-17] attempt 1 started
2026-11-07T23:58:23Z [load_day 2023-08-17] 2023-08-17: 2 hours in flight, about 1200 MB
2026-11-07T23:58:42Z [load_day 2023-08-17] 2023-08-17 hour 0: 158135 events in 4 files, snapshot 1364976189024049488 (task 13566 ms: download 5483 ms overlapping gunzip 720 ms, parse 3293 ms, sort 144 ms, write 8118 ms, index 736 ms: 6 candidates, 85954 repo-hours, 982 releases; waiting 253 ms; commit 3725 ms)
2026-11-07T23:58:46Z [load_day 2023-08-17] 2023-08-17 hour 1: 152052 events in 4 files, snapshot 3868529581800403889 (task 13270 ms: download 5504 ms overlapping gunzip 724 ms, parse 3501 ms, sort 113 ms, write 7551 ms, index 587 ms: 5 candidates, 84063 repo-hours, 720 releases; waiting 515 ms; commit 4396 ms)
2026-11-07T23:58:56Z [load_day 2023-08-17] 2023-08-17 hour 2: 136107 events in 3 files, snapshot 781214376439677893 (task 13988 ms: download 6258 ms overlapping gunzip 766 ms, parse 3177 ms, sort 129 ms, write 8754 ms, index 453 ms: 5 candidates, 58473 repo-hours, 902 releases; waiting 497 ms; commit 2728 ms)
2026-11-07T23:58:59Z [load_day 2023-08-17] 2023-08-17 hour 3: 132411 events in 3 files, snapshot 5125745194033612495 (task 12718 ms: download 5754 ms overlapping gunzip 687 ms, parse 3214 ms, sort 110 ms, write 7348 ms, index 581 ms: 3 candidates, 56326 repo-hours, 909 releases; waiting 496 ms; commit 2598 ms)
2026-11-07T23:59:09Z [load_day 2023-08-17] 2023-08-17 hour 4: 129724 events in 3 files, snapshot 847226339812609554 (task 12323 ms: download 4897 ms overlapping gunzip 698 ms, parse 2910 ms, sort 85 ms, write 7269 ms, index 437 ms: 4 candidates, 54408 repo-hours, 667 releases; waiting 514 ms; commit 2184 ms)
2026-11-07T23:59:13Z [load_day 2023-08-17] 2023-08-17 hour 5: 139928 events in 3 files, snapshot 2427063523695497972 (task 12399 ms: download 4689 ms overlapping gunzip 722 ms, parse 3060 ms, sort 97 ms, write 7084 ms, index 621 ms: 3 candidates, 59013 repo-hours, 792 releases; waiting 531 ms; commit 3342 ms)
2026-11-07T23:59:27Z [load_day 2023-08-17] 2023-08-17 hour 6: 165268 events in 4 files, snapshot 7974191128849509658 (task 17128 ms: download 6930 ms overlapping gunzip 900 ms, parse 4206 ms, sort 130 ms, write 10315 ms, index 654 ms: 2 candidates, 76680 repo-hours, 896 releases; waiting 521 ms; commit 2524 ms)
2026-11-07T23:59:30Z [load_day 2023-08-17] 2023-08-17 hour 7: 155976 events in 4 files, snapshot 7749901714939675286 (task 16285 ms: download 6446 ms overlapping gunzip 960 ms, parse 4229 ms, sort 137 ms, write 9382 ms, index 908 ms: 4 candidates, 71246 repo-hours, 808 releases; waiting 371 ms; commit 2979 ms)
2026-11-07T23:59:45Z [load_day 2023-08-17] 2023-08-17 hour 8: 166461 events in 4 files, snapshot 5489268943184443627 (task 17339 ms: download 9927 ms overlapping gunzip 1151 ms, parse 5201 ms, sort 129 ms, write 9549 ms, index 661 ms: 6 candidates, 76413 repo-hours, 887 releases; waiting 21 ms; commit 2534 ms)
2026-11-07T23:59:51Z [load_day 2023-08-17] 2023-08-17 hour 9: 184304 events in 4 files, snapshot 7758465263765544635 (task 19920 ms: download 11113 ms overlapping gunzip 1133 ms, parse 5063 ms, sort 247 ms, write 11921 ms, index 762 ms: 4 candidates, 80807 repo-hours, 1146 releases; waiting 493 ms; commit 2240 ms)
2026-11-08T00:00:03Z [load_day 2023-08-17] 2023-08-17 hour 10: 182304 events in 4 files, snapshot 2507694334218269982 (task 17052 ms: download 10182 ms overlapping gunzip 968 ms, parse 4441 ms, sort 140 ms, write 10084 ms, index 622 ms: 7 candidates, 80172 repo-hours, 1087 releases; waiting 499 ms; commit 2383 ms)
2026-11-08T00:00:08Z [load_day 2023-08-17] 2023-08-17 hour 11: 182004 events in 4 files, snapshot 5811322322103556163 (task 15998 ms: download 8795 ms overlapping gunzip 983 ms, parse 4162 ms, sort 125 ms, write 9318 ms, index 588 ms: 16 candidates, 80533 repo-hours, 1092 releases; waiting 519 ms; commit 2256 ms)
2026-11-08T00:00:26Z [load_day 2023-08-17] 2023-08-17 hour 12: 226812 events in 5 files, snapshot 1130103199392432741 (task 20919 ms: download 9116 ms overlapping gunzip 1035 ms, parse 4761 ms, sort 161 ms, write 13264 ms, index 830 ms: 10 candidates, 106872 repo-hours, 1121 releases; waiting 528 ms; commit 2402 ms)
2026-11-08T00:00:34Z [load_day 2023-08-17] 2023-08-17 hour 13: 210345 events in 5 files, snapshot 3050403588020811361 (task 22962 ms: download 12742 ms overlapping gunzip 1394 ms, parse 6622 ms, sort 261 ms, write 12838 ms, index 767 ms: 4 candidates, 103322 repo-hours, 1114 releases; waiting 594 ms; commit 3913 ms)
2026-11-08T00:00:49Z [load_day 2023-08-17] 2023-08-17 hour 14: 198210 events in 4 files, snapshot 4747108872729553593 (task 21771 ms: download 11811 ms overlapping gunzip 1136 ms, parse 4902 ms, sort 189 ms, write 13820 ms, index 828 ms: 3 candidates, 90209 repo-hours, 1101 releases; waiting 223 ms; commit 2767 ms)
2026-11-08T00:00:53Z [load_day 2023-08-17] 2023-08-17 hour 15: 182371 events in 4 files, snapshot 4399288867622355707 (task 19860 ms: download 10984 ms overlapping gunzip 1191 ms, parse 5366 ms, sort 120 ms, write 11819 ms, index 703 ms: 5 candidates, 83737 repo-hours, 1159 releases; waiting 320 ms; commit 2225 ms)
2026-11-08T00:01:17Z [load_day 2023-08-17] 2023-08-17 hour 16: 182359 events in 4 files, snapshot 2623033722123468180 (task 20379 ms: download 11251 ms overlapping gunzip 1086 ms, parse 4910 ms, sort 160 ms, write 12742 ms, index 670 ms: 9 candidates, 80754 repo-hours, 971 releases; waiting 508 ms; commit 9732 ms)
2026-11-08T00:01:20Z [load_day 2023-08-17] 2023-08-17 hour 17: 187006 events in 4 files, snapshot 7350577336531035685 (task 20303 ms: download 11537 ms overlapping gunzip 977 ms, parse 4260 ms, sort 118 ms, write 13199 ms, index 922 ms: 1 candidates, 77658 repo-hours, 985 releases; waiting 520 ms; commit 2642 ms)
2026-11-08T00:01:30Z [load_day 2023-08-17] 2023-08-17 hour 18: 180520 events in 4 files, snapshot 3602132745163555447 (task 18835 ms: download 10786 ms overlapping gunzip 1042 ms, parse 4606 ms, sort 111 ms, write 11601 ms, index 703 ms: 8 candidates, 79622 repo-hours, 983 releases; waiting 508 ms; commit 2727 ms)
2026-11-08T00:01:39Z [load_day 2023-08-17] 2023-08-17 hour 19: 170913 events in 4 files, snapshot 3010501454306616015 (task 17458 ms: download 9676 ms overlapping gunzip 1066 ms, parse 4702 ms, sort 148 ms, write 10095 ms, index 648 ms: 4 candidates, 70914 repo-hours, 975 releases; waiting 443 ms; commit 2738 ms)
2026-11-08T00:01:49Z [load_day 2023-08-17] 2023-08-17 hour 20: 164775 events in 4 files, snapshot 1539940281161020316 (task 17938 ms: download 10981 ms overlapping gunzip 1014 ms, parse 4465 ms, sort 169 ms, write 10773 ms, index 651 ms: 3 candidates, 69003 repo-hours, 889 releases; waiting 538 ms; commit 2439 ms)
2026-11-08T00:01:56Z [load_day 2023-08-17] 2023-08-17 hour 21: 153006 events in 4 files, snapshot 691286543199180832 (task 16196 ms: download 6424 ms overlapping gunzip 901 ms, parse 4030 ms, sort 102 ms, write 9926 ms, index 440 ms: 3 candidates, 61551 repo-hours, 887 releases; waiting 511 ms; commit 2414 ms)
2026-11-08T00:02:04Z [load_day 2023-08-17] 2023-08-17 hour 22: 138578 events in 3 files, snapshot 8250622289501547896 (task 14038 ms: download 6129 ms overlapping gunzip 749 ms, parse 3320 ms, sort 98 ms, write 8631 ms, index 419 ms: 6 candidates, 56175 repo-hours, 960 releases; waiting 495 ms; commit 2194 ms)
2026-11-08T00:02:09Z [load_day 2023-08-17] 2023-08-17 hour 23: 125375 events in 3 files, snapshot 3410553393794938423 (task 11808 ms: download 5519 ms overlapping gunzip 804 ms, parse 3497 ms, sort 97 ms, write 6252 ms, index 333 ms: 0 candidates, 51642 repo-hours, 735 releases; waiting 499 ms; commit 2059 ms)
2026-11-08T00:02:19Z [load_day 2023-08-17] 2023-08-17: repo_hour merged from 24 files to 1 (1795547 rows, 9988 ms)
2026-11-08T00:02:21Z [load_day 2023-08-17] 2023-08-17: index committed in 12785 ms (23 + 1 + 24 files)
2026-11-08T00:02:21Z [load_day 2023-08-17] 2023-08-17: 4004944 events in 24 hours
2026-11-08T00:02:21Z [load_day 2023-08-17] attempt 1 succeeded