orcaLog in

Runs · run 1086

load_day 2023-05-14, attempt 1 succeeded

Host
caf251265124:366744
Started
2026-11-07T20:06:24Z
Ended
2026-11-07T20:08:20Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1086/load_day/2023-05-14/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T20:06:24Z [load_day 2023-05-14] attempt 1 started
2026-11-07T20:06:26Z [load_day 2023-05-14] 2023-05-14: 2 hours in flight, about 1200 MB
2026-11-07T20:06:35Z [load_day 2023-05-14] 2023-05-14 hour 0: 118595 events in 3 files, snapshot 6181251000056229970 (task 6547 ms: download 1077 ms overlapping gunzip 365 ms, parse 1778 ms, sort 63 ms, write 3447 ms, index 427 ms: 4 candidates, 67774 repo-hours, 907 releases; waiting 212 ms; commit 1725 ms)
2026-11-07T20:06:37Z [load_day 2023-05-14] 2023-05-14 hour 1: 118600 events in 3 files, snapshot 7836267777196379150 (task 6775 ms: download 1976 ms overlapping gunzip 354 ms, parse 1772 ms, sort 65 ms, write 3297 ms, index 442 ms: 7 candidates, 67151 repo-hours, 727 releases; waiting 495 ms; commit 1620 ms)
2026-11-07T20:06:42Z [load_day 2023-05-14] 2023-05-14 hour 2: 96253 events in 2 files, snapshot 4832809044552228416 (task 5914 ms: download 1853 ms overlapping gunzip 315 ms, parse 1451 ms, sort 58 ms, write 3093 ms, index 289 ms: 5 candidates, 42136 repo-hours, 526 releases; waiting 391 ms; commit 1707 ms)
2026-11-07T20:06:43Z [load_day 2023-05-14] 2023-05-14 hour 3: 92721 events in 2 files, snapshot 2965515107640266170 (task 5265 ms: download 1265 ms overlapping gunzip 285 ms, parse 1398 ms, sort 59 ms, write 2783 ms, index 288 ms: 3 candidates, 37101 repo-hours, 665 releases; waiting 184 ms; commit 1645 ms)
2026-11-07T20:06:48Z [load_day 2023-05-14] 2023-05-14 hour 4: 95195 events in 2 files, snapshot 2464307791471807172 (task 5679 ms: download 1900 ms overlapping gunzip 307 ms, parse 1431 ms, sort 51 ms, write 2763 ms, index 265 ms: 4 candidates, 37625 repo-hours, 464 releases; waiting 512 ms; commit 1595 ms)
2026-11-07T20:06:50Z [load_day 2023-05-14] 2023-05-14 hour 5: 97871 events in 2 files, snapshot 3085829189845324699 (task 5957 ms: download 1601 ms overlapping gunzip 327 ms, parse 1549 ms, sort 65 ms, write 2907 ms, index 278 ms: 4 candidates, 39759 repo-hours, 583 releases; waiting 556 ms; commit 1627 ms)
2026-11-07T20:06:57Z [load_day 2023-05-14] 2023-05-14 hour 6: 112128 events in 3 files, snapshot 5310979495529270942 (task 7834 ms: download 2831 ms overlapping gunzip 385 ms, parse 1762 ms, sort 69 ms, write 3368 ms, index 311 ms: 10 candidates, 48253 repo-hours, 610 releases; waiting 1625 ms; commit 1504 ms)
2026-11-07T20:06:59Z [load_day 2023-05-14] 2023-05-14 hour 7: 116983 events in 3 files, snapshot 635646160871428790 (task 6973 ms: download 2148 ms overlapping gunzip 371 ms, parse 1735 ms, sort 71 ms, write 3594 ms, index 305 ms: 3 candidates, 46266 repo-hours, 510 releases; waiting 606 ms; commit 1802 ms)
2026-11-07T20:07:05Z [load_day 2023-05-14] 2023-05-14 hour 8: 122092 events in 3 files, snapshot 6811306549043192139 (task 7610 ms: download 2053 ms overlapping gunzip 419 ms, parse 1976 ms, sort 71 ms, write 4013 ms, index 340 ms: 6 candidates, 50511 repo-hours, 711 releases; waiting 513 ms; commit 1796 ms)
2026-11-07T20:07:07Z [load_day 2023-05-14] 2023-05-14 hour 9: 122656 events in 3 files, snapshot 4194438372408784526 (task 8023 ms: download 2175 ms overlapping gunzip 425 ms, parse 1942 ms, sort 69 ms, write 3974 ms, index 326 ms: 2 candidates, 50664 repo-hours, 780 releases; waiting 937 ms; commit 1760 ms)
2026-11-07T20:07:14Z [load_day 2023-05-14] 2023-05-14 hour 10: 124152 events in 3 files, snapshot 6313034274828013356 (task 8031 ms: download 2163 ms overlapping gunzip 420 ms, parse 1925 ms, sort 71 ms, write 4182 ms, index 387 ms: 2 candidates, 52984 repo-hours, 647 releases; waiting 744 ms; commit 1598 ms)
2026-11-07T20:07:16Z [load_day 2023-05-14] 2023-05-14 hour 11: 130668 events in 3 files, snapshot 5768076845705846895 (task 7685 ms: download 2397 ms overlapping gunzip 441 ms, parse 2016 ms, sort 74 ms, write 3975 ms, index 353 ms: 5 candidates, 54725 repo-hours, 720 releases; waiting 496 ms; commit 1664 ms)
2026-11-07T20:07:25Z [load_day 2023-05-14] 2023-05-14 hour 12: 201406 events in 5 files, snapshot 6997554524216274605 (task 10168 ms: download 3362 ms overlapping gunzip 547 ms, parse 2644 ms, sort 138 ms, write 5473 ms, index 550 ms: 6 candidates, 90361 repo-hours, 789 releases; waiting 508 ms; commit 1679 ms)
2026-11-07T20:07:27Z [load_day 2023-05-14] 2023-05-14 hour 13: 162016 events in 4 files, snapshot 6386884313469802832 (task 9417 ms: download 2945 ms overlapping gunzip 494 ms, parse 2327 ms, sort 98 ms, write 5066 ms, index 541 ms: 7 candidates, 84543 repo-hours, 752 releases; waiting 437 ms; commit 1955 ms)
2026-11-07T20:07:35Z [load_day 2023-05-14] 2023-05-14 hour 14: 147933 events in 3 files, snapshot 3214502562248520394 (task 9185 ms: download 3390 ms overlapping gunzip 502 ms, parse 2295 ms, sort 88 ms, write 4954 ms, index 485 ms: 8 candidates, 65246 repo-hours, 751 releases; waiting 524 ms; commit 1787 ms)
2026-11-07T20:07:37Z [load_day 2023-05-14] 2023-05-14 hour 15: 150935 events in 4 files, snapshot 5092542983760941410 (task 9410 ms: download 3526 ms overlapping gunzip 510 ms, parse 2340 ms, sort 96 ms, write 5168 ms, index 459 ms: 8 candidates, 68374 repo-hours, 932 releases; waiting 533 ms; commit 1584 ms)
2026-11-07T20:07:45Z [load_day 2023-05-14] 2023-05-14 hour 16: 150089 events in 4 files, snapshot 8729421985105881611 (task 9016 ms: download 3187 ms overlapping gunzip 498 ms, parse 2273 ms, sort 83 ms, write 4831 ms, index 455 ms: 7 candidates, 67496 repo-hours, 876 releases; waiting 547 ms; commit 1604 ms)
2026-11-07T20:07:47Z [load_day 2023-05-14] 2023-05-14 hour 17: 141541 events in 3 files, snapshot 8912577168429452331 (task 8300 ms: download 2214 ms overlapping gunzip 468 ms, parse 2147 ms, sort 86 ms, write 4403 ms, index 406 ms: 16 candidates, 58953 repo-hours, 811 releases; waiting 523 ms; commit 1719 ms)
2026-11-07T20:07:54Z [load_day 2023-05-14] 2023-05-14 hour 18: 158657 events in 4 files, snapshot 1511293543989153279 (task 8461 ms: download 2726 ms overlapping gunzip 475 ms, parse 2224 ms, sort 102 ms, write 4550 ms, index 511 ms: 11 candidates, 77573 repo-hours, 835 releases; waiting 342 ms; commit 1548 ms)
2026-11-07T20:07:56Z [load_day 2023-05-14] 2023-05-14 hour 19: 126236 events in 3 files, snapshot 7484261507384373789 (task 7411 ms: download 1580 ms overlapping gunzip 417 ms, parse 1893 ms, sort 73 ms, write 4122 ms, index 359 ms: 8 candidates, 56584 repo-hours, 709 releases; waiting 226 ms; commit 1762 ms)
2026-11-07T20:08:03Z [load_day 2023-05-14] 2023-05-14 hour 20: 136622 events in 3 files, snapshot 8053834281789438934 (task 8043 ms: download 1655 ms overlapping gunzip 461 ms, parse 2116 ms, sort 85 ms, write 4313 ms, index 410 ms: 6 candidates, 62292 repo-hours, 718 releases; waiting 382 ms; commit 1460 ms)
2026-11-07T20:08:04Z [load_day 2023-05-14] 2023-05-14 hour 21: 129601 events in 3 files, snapshot 8139693385817337982 (task 7277 ms: download 2445 ms overlapping gunzip 394 ms, parse 1825 ms, sort 67 ms, write 3725 ms, index 382 ms: 3 candidates, 62269 repo-hours, 709 releases; waiting 608 ms; commit 1722 ms)
2026-11-07T20:08:10Z [load_day 2023-05-14] 2023-05-14 hour 22: 108130 events in 3 files, snapshot 3005746618471732642 (task 6465 ms: download 1608 ms overlapping gunzip 365 ms, parse 1664 ms, sort 60 ms, write 3309 ms, index 307 ms: 4 candidates, 46079 repo-hours, 631 releases; waiting 524 ms; commit 1781 ms)
2026-11-07T20:08:12Z [load_day 2023-05-14] 2023-05-14 hour 23: 102916 events in 3 files, snapshot 8047824838092216982 (task 6704 ms: download 2501 ms overlapping gunzip 369 ms, parse 1661 ms, sort 63 ms, write 3285 ms, index 312 ms: 6 candidates, 43156 repo-hours, 561 releases; waiting 697 ms; commit 1621 ms)
2026-11-07T20:08:18Z [load_day 2023-05-14] 2023-05-14: repo_hour merged from 24 files to 1 (1377875 rows, 5724 ms)
2026-11-07T20:08:20Z [load_day 2023-05-14] 2023-05-14: index committed in 7854 ms (24 + 1 + 24 files)
2026-11-07T20:08:20Z [load_day 2023-05-14] 2023-05-14: 3063996 events in 24 hours
2026-11-07T20:08:20Z [load_day 2023-05-14] attempt 1 succeeded