orcaLog in

Runs · run 930

load_day 2023-03-23, attempt 1 succeeded

Host
caf251265124:192010
Started
2026-11-07T17:32:15Z
Ended
2026-11-07T17:35:27Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/930/load_day/2023-03-23/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T17:32:15Z [load_day 2023-03-23] attempt 1 started
2026-11-07T17:32:16Z [load_day 2023-03-23] 2023-03-23: 2 hours in flight, about 1200 MB
2026-11-07T17:32:30Z [load_day 2023-03-23] 2023-03-23 hour 0: 152684 events in 4 files, snapshot 2818527923520586606 (task 11868 ms: download 4857 ms overlapping gunzip 649 ms, parse 2973 ms, sort 87 ms, write 6584 ms, index 524 ms: 4 candidates, 76350 repo-hours, 848 releases; waiting 389 ms; commit 1396 ms)
2026-11-07T17:32:32Z [load_day 2023-03-23] 2023-03-23 hour 1: 162777 events in 4 files, snapshot 1629229275117057492 (task 12269 ms: download 5182 ms overlapping gunzip 699 ms, parse 3116 ms, sort 91 ms, write 6761 ms, index 560 ms: 7 candidates, 87555 repo-hours, 788 releases; waiting 745 ms; commit 1532 ms)
2026-11-07T17:32:42Z [load_day 2023-03-23] 2023-03-23 hour 2: 146496 events in 3 files, snapshot 2624643266424722604 (task 10939 ms: download 4667 ms overlapping gunzip 648 ms, parse 2823 ms, sort 92 ms, write 5926 ms, index 463 ms: 2 candidates, 66635 repo-hours, 710 releases; waiting 683 ms; commit 1527 ms)
2026-11-07T17:32:44Z [load_day 2023-03-23] 2023-03-23 hour 3: 140655 events in 3 files, snapshot 2764288399712498245 (task 11313 ms: download 5173 ms overlapping gunzip 637 ms, parse 2765 ms, sort 85 ms, write 5843 ms, index 476 ms: 5 candidates, 63979 repo-hours, 810 releases; waiting 1216 ms; commit 1429 ms)
2026-11-07T17:32:52Z [load_day 2023-03-23] 2023-03-23 hour 4: 135340 events in 3 files, snapshot 3060262774831139491 (task 8976 ms: download 3345 ms overlapping gunzip 527 ms, parse 2300 ms, sort 92 ms, write 5103 ms, index 404 ms: 2 candidates, 63603 repo-hours, 567 releases; waiting 242 ms; commit 1469 ms)
2026-11-07T17:32:55Z [load_day 2023-03-23] 2023-03-23 hour 5: 136502 events in 3 files, snapshot 1770223885372255322 (task 10507 ms: download 4214 ms overlapping gunzip 618 ms, parse 2687 ms, sort 111 ms, write 5913 ms, index 422 ms: 7 candidates, 61353 repo-hours, 735 releases; waiting 390 ms; commit 1468 ms)
2026-11-07T17:33:04Z [load_day 2023-03-23] 2023-03-23 hour 6: 153836 events in 4 files, snapshot 3842675594024264945 (task 11114 ms: download 4242 ms overlapping gunzip 649 ms, parse 2967 ms, sort 79 ms, write 6354 ms, index 533 ms: 3 candidates, 71770 repo-hours, 871 releases; waiting 240 ms; commit 1442 ms)
2026-11-07T17:33:09Z [load_day 2023-03-23] 2023-03-23 hour 7: 166481 events in 4 files, snapshot 4867918846514707624 (task 12932 ms: download 5075 ms overlapping gunzip 784 ms, parse 3439 ms, sort 106 ms, write 7478 ms, index 564 ms: 5 candidates, 73413 repo-hours, 790 releases; waiting 228 ms; commit 1392 ms)
2026-11-07T17:33:17Z [load_day 2023-03-23] 2023-03-23 hour 8: 163839 events in 4 files, snapshot 3991122058201030212 (task 12638 ms: download 4740 ms overlapping gunzip 738 ms, parse 3247 ms, sort 93 ms, write 7454 ms, index 538 ms: 6 candidates, 81155 repo-hours, 937 releases; waiting 215 ms; commit 1495 ms)
2026-11-07T17:33:24Z [load_day 2023-03-23] 2023-03-23 hour 9: 168437 events in 4 files, snapshot 4696154424580391997 (task 14573 ms: download 8441 ms overlapping gunzip 871 ms, parse 3856 ms, sort 114 ms, write 8444 ms, index 590 ms: 5 candidates, 80308 repo-hours, 1248 releases; waiting 324 ms; commit 1577 ms)
2026-11-07T17:33:32Z [load_day 2023-03-23] 2023-03-23 hour 10: 172824 events in 4 files, snapshot 2687545078400407053 (task 14686 ms: download 8420 ms overlapping gunzip 859 ms, parse 3814 ms, sort 120 ms, write 8439 ms, index 584 ms: 2 candidates, 81686 repo-hours, 1165 releases; waiting 557 ms; commit 1535 ms)
2026-11-07T17:33:40Z [load_day 2023-03-23] 2023-03-23 hour 11: 170137 events in 4 files, snapshot 9105325039310761992 (task 14954 ms: download 8546 ms overlapping gunzip 848 ms, parse 3766 ms, sort 100 ms, write 8803 ms, index 545 ms: 3 candidates, 79838 repo-hours, 1344 releases; waiting 556 ms; commit 1536 ms)
2026-11-07T17:33:48Z [load_day 2023-03-23] 2023-03-23 hour 12: 199402 events in 4 files, snapshot 1442168670840702468 (task 15323 ms: download 8056 ms overlapping gunzip 798 ms, parse 3578 ms, sort 140 ms, write 9073 ms, index 730 ms: 2 candidates, 113334 repo-hours, 944 releases; waiting 534 ms; commit 1532 ms)
2026-11-07T17:33:57Z [load_day 2023-03-23] 2023-03-23 hour 13: 202534 events in 5 files, snapshot 359635420012441048 (task 16416 ms: download 8669 ms overlapping gunzip 933 ms, parse 4012 ms, sort 135 ms, write 9723 ms, index 734 ms: 9 candidates, 116451 repo-hours, 931 releases; waiting 516 ms; commit 1505 ms)
2026-11-07T17:34:05Z [load_day 2023-03-23] 2023-03-23 hour 14: 182664 events in 4 files, snapshot 1478447253331146684 (task 16070 ms: download 9237 ms overlapping gunzip 953 ms, parse 4090 ms, sort 104 ms, write 9353 ms, index 615 ms: 6 candidates, 91364 repo-hours, 1108 releases; waiting 623 ms; commit 1453 ms)
2026-11-07T17:34:15Z [load_day 2023-03-23] 2023-03-23 hour 15: 183050 events in 4 files, snapshot 6948217164953361189 (task 17076 ms: download 9699 ms overlapping gunzip 990 ms, parse 4285 ms, sort 122 ms, write 10218 ms, index 652 ms: 9 candidates, 93745 repo-hours, 1097 releases; waiting 561 ms; commit 1488 ms)
2026-11-07T17:34:22Z [load_day 2023-03-23] 2023-03-23 hour 16: 177882 events in 4 files, snapshot 2408698062557967101 (task 16289 ms: download 9464 ms overlapping gunzip 947 ms, parse 4170 ms, sort 108 ms, write 9583 ms, index 567 ms: 13 candidates, 86529 repo-hours, 1158 releases; waiting 531 ms; commit 1453 ms)
2026-11-07T17:34:31Z [load_day 2023-03-23] 2023-03-23 hour 17: 190071 events in 4 files, snapshot 2549224953191185945 (task 16320 ms: download 9024 ms overlapping gunzip 990 ms, parse 4165 ms, sort 153 ms, write 9591 ms, index 586 ms: 10 candidates, 92591 repo-hours, 893 releases; waiting 462 ms; commit 1411 ms)
2026-11-07T17:34:38Z [load_day 2023-03-23] 2023-03-23 hour 18: 185384 events in 4 files, snapshot 8373988207877391232 (task 15660 ms: download 8942 ms overlapping gunzip 907 ms, parse 3967 ms, sort 118 ms, write 8797 ms, index 622 ms: 8 candidates, 89485 repo-hours, 969 releases; waiting 949 ms; commit 1638 ms)
2026-11-07T17:34:47Z [load_day 2023-03-23] 2023-03-23 hour 19: 179059 events in 4 files, snapshot 6027562420070618586 (task 15123 ms: download 8772 ms overlapping gunzip 922 ms, parse 3863 ms, sort 107 ms, write 8876 ms, index 493 ms: 6 candidates, 77004 repo-hours, 944 releases; waiting 563 ms; commit 1465 ms)
2026-11-07T17:34:54Z [load_day 2023-03-23] 2023-03-23 hour 20: 178871 events in 4 files, snapshot 5337135882631178380 (task 14582 ms: download 8418 ms overlapping gunzip 909 ms, parse 3895 ms, sort 101 ms, write 8336 ms, index 507 ms: 11 candidates, 76691 repo-hours, 907 releases; waiting 536 ms; commit 1602 ms)
2026-11-07T17:35:04Z [load_day 2023-03-23] 2023-03-23 hour 21: 178149 events in 4 files, snapshot 866331184022386157 (task 15510 ms: download 9110 ms overlapping gunzip 985 ms, parse 4051 ms, sort 111 ms, write 9026 ms, index 497 ms: 4 candidates, 75010 repo-hours, 1055 releases; waiting 533 ms; commit 1577 ms)
2026-11-07T17:35:09Z [load_day 2023-03-23] 2023-03-23 hour 22: 161139 events in 4 files, snapshot 2890759358975671606 (task 14497 ms: download 6807 ms overlapping gunzip 847 ms, parse 3539 ms, sort 112 ms, write 7615 ms, index 470 ms: 3 candidates, 66071 repo-hours, 876 releases; waiting 1515 ms; commit 1361 ms)
2026-11-07T17:35:18Z [load_day 2023-03-23] 2023-03-23 hour 23: 149949 events in 3 files, snapshot 6346407646338736705 (task 13628 ms: download 6410 ms overlapping gunzip 808 ms, parse 3353 ms, sort 99 ms, write 7505 ms, index 422 ms: 3 candidates, 63482 repo-hours, 813 releases; waiting 1076 ms; commit 1463 ms)
2026-11-07T17:35:26Z [load_day 2023-03-23] 2023-03-23: repo_hour merged from 24 files to 1 (1929402 rows, 7947 ms)
2026-11-07T17:35:27Z [load_day 2023-03-23] 2023-03-23: index committed in 9665 ms (24 + 1 + 24 files)
2026-11-07T17:35:27Z [load_day 2023-03-23] 2023-03-23: 4038162 events in 24 hours
2026-11-07T17:35:27Z [load_day 2023-03-23] attempt 1 succeeded