orcaLog in

Runs · run 894

load_day 2023-03-11, attempt 1 succeeded

Host
caf251265124:151332
Started
2026-11-07T16:56:20Z
Ended
2026-11-07T16:58:30Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/894/load_day/2023-03-11/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T16:56:20Z [load_day 2023-03-11] attempt 1 started
2026-11-07T16:56:21Z [load_day 2023-03-11] 2023-03-11: 2 hours in flight, about 1200 MB
2026-11-07T16:56:34Z [load_day 2023-03-11] 2023-03-11 hour 0: 162160 events in 4 files, snapshot 4002053618245539724 (task 11120 ms: download 4390 ms overlapping gunzip 657 ms, parse 2925 ms, sort 107 ms, write 6082 ms, index 523 ms: 8 candidates, 87129 repo-hours, 950 releases; waiting 488 ms; commit 1204 ms)
2026-11-07T16:56:36Z [load_day 2023-03-11] 2023-03-11 hour 1: 160582 events in 4 files, snapshot 5085192940304803186 (task 9474 ms: download 3455 ms overlapping gunzip 548 ms, parse 2462 ms, sort 91 ms, write 5351 ms, index 466 ms: 6 candidates, 72500 repo-hours, 773 releases; waiting 240 ms; commit 1475 ms)
2026-11-07T16:56:43Z [load_day 2023-03-11] 2023-03-11 hour 2: 126855 events in 3 files, snapshot 9078847110665159929 (task 8111 ms: download 2258 ms overlapping gunzip 485 ms, parse 2072 ms, sort 71 ms, write 4332 ms, index 343 ms: 4 candidates, 51990 repo-hours, 662 releases; waiting 508 ms; commit 1329 ms)
2026-11-07T16:56:44Z [load_day 2023-03-11] 2023-03-11 hour 3: 117356 events in 3 files, snapshot 690714623837215374 (task 7291 ms: download 2037 ms overlapping gunzip 423 ms, parse 1852 ms, sort 56 ms, write 3838 ms, index 297 ms: 4 candidates, 45254 repo-hours, 698 releases; waiting 526 ms; commit 1294 ms)
2026-11-07T16:56:51Z [load_day 2023-03-11] 2023-03-11 hour 4: 108779 events in 3 files, snapshot 3475011145597176849 (task 7002 ms: download 2025 ms overlapping gunzip 386 ms, parse 1666 ms, sort 63 ms, write 3731 ms, index 293 ms: 2 candidates, 42689 repo-hours, 537 releases; waiting 537 ms; commit 1319 ms)
2026-11-07T16:56:52Z [load_day 2023-03-11] 2023-03-11 hour 5: 111655 events in 3 files, snapshot 4747664879437499314 (task 7001 ms: download 2069 ms overlapping gunzip 384 ms, parse 1723 ms, sort 63 ms, write 3645 ms, index 334 ms: 3 candidates, 44679 repo-hours, 687 releases; waiting 556 ms; commit 1394 ms)
2026-11-07T16:56:59Z [load_day 2023-03-11] 2023-03-11 hour 6: 120216 events in 3 files, snapshot 8088802122683367258 (task 7235 ms: download 1917 ms overlapping gunzip 404 ms, parse 1814 ms, sort 62 ms, write 3805 ms, index 328 ms: 4 candidates, 51063 repo-hours, 621 releases; waiting 216 ms; commit 1340 ms)
2026-11-07T16:57:00Z [load_day 2023-03-11] 2023-03-11 hour 7: 120360 events in 3 files, snapshot 6332341645368808271 (task 7310 ms: download 1892 ms overlapping gunzip 403 ms, parse 1844 ms, sort 63 ms, write 3814 ms, index 338 ms: 3 candidates, 48482 repo-hours, 610 releases; waiting 516 ms; commit 1307 ms)
2026-11-07T16:57:07Z [load_day 2023-03-11] 2023-03-11 hour 8: 124343 events in 3 files, snapshot 2666848348246449509 (task 8057 ms: download 2869 ms overlapping gunzip 441 ms, parse 1919 ms, sort 76 ms, write 4216 ms, index 362 ms: 4 candidates, 52329 repo-hours, 624 releases; waiting 675 ms; commit 1417 ms)
2026-11-07T16:57:10Z [load_day 2023-03-11] 2023-03-11 hour 9: 131460 events in 3 files, snapshot 3913736708573633860 (task 8281 ms: download 2638 ms overlapping gunzip 452 ms, parse 2007 ms, sort 86 ms, write 4259 ms, index 398 ms: 3 candidates, 62430 repo-hours, 752 releases; waiting 509 ms; commit 1523 ms)
2026-11-07T16:57:19Z [load_day 2023-03-11] 2023-03-11 hour 10: 127764 events in 3 files, snapshot 1639532635986441637 (task 10565 ms: download 3944 ms overlapping gunzip 454 ms, parse 2031 ms, sort 80 ms, write 5543 ms, index 808 ms: 3 candidates, 54970 repo-hours, 601 releases; waiting 1062 ms; commit 1634 ms)
2026-11-07T16:57:20Z [load_day 2023-03-11] 2023-03-11 hour 11: 129925 events in 3 files, snapshot 7458691923604247718 (task 8249 ms: download 1567 ms overlapping gunzip 477 ms, parse 2159 ms, sort 91 ms, write 4688 ms, index 351 ms: 6 candidates, 54306 repo-hours, 648 releases; waiting 193 ms; commit 1598 ms)
2026-11-07T16:57:30Z [load_day 2023-03-11] 2023-03-11 hour 12: 191212 events in 4 files, snapshot 8683465709834297922 (task 10389 ms: download 3933 ms overlapping gunzip 542 ms, parse 2561 ms, sort 101 ms, write 5284 ms, index 564 ms: 6 candidates, 94353 repo-hours, 772 releases; waiting 1002 ms; commit 1310 ms)
2026-11-07T16:57:31Z [load_day 2023-03-11] 2023-03-11 hour 13: 157115 events in 4 files, snapshot 7608517068350974823 (task 9365 ms: download 3240 ms overlapping gunzip 523 ms, parse 2457 ms, sort 90 ms, write 4936 ms, index 502 ms: 10 candidates, 78551 repo-hours, 687 releases; waiting 497 ms; commit 1463 ms)
2026-11-07T16:57:41Z [load_day 2023-03-11] 2023-03-11 hour 14: 150552 events in 4 files, snapshot 8237300197764898719 (task 10528 ms: download 4242 ms overlapping gunzip 562 ms, parse 2499 ms, sort 89 ms, write 5193 ms, index 441 ms: 5 candidates, 65502 repo-hours, 853 releases; waiting 1148 ms; commit 1403 ms)
2026-11-07T16:57:42Z [load_day 2023-03-11] 2023-03-11 hour 15: 152784 events in 4 files, snapshot 5630585710691757309 (task 9789 ms: download 3868 ms overlapping gunzip 565 ms, parse 2503 ms, sort 96 ms, write 5298 ms, index 427 ms: 4 candidates, 68341 repo-hours, 1014 releases; waiting 549 ms; commit 1208 ms)
2026-11-07T16:57:51Z [load_day 2023-03-11] 2023-03-11 hour 16: 146555 events in 3 files, snapshot 2113538398806263486 (task 9283 ms: download 3529 ms overlapping gunzip 558 ms, parse 2515 ms, sort 78 ms, write 4855 ms, index 424 ms: 7 candidates, 65253 repo-hours, 732 releases; waiting 502 ms; commit 1537 ms)
2026-11-07T16:57:52Z [load_day 2023-03-11] 2023-03-11 hour 17: 145768 events in 3 files, snapshot 9125251944825958108 (task 8902 ms: download 3343 ms overlapping gunzip 508 ms, parse 2286 ms, sort 78 ms, write 4815 ms, index 401 ms: 5 candidates, 62593 repo-hours, 731 releases; waiting 485 ms; commit 1358 ms)
2026-11-07T16:58:00Z [load_day 2023-03-11] 2023-03-11 hour 18: 149267 events in 3 files, snapshot 4111807587905583499 (task 8642 ms: download 2902 ms overlapping gunzip 499 ms, parse 2274 ms, sort 85 ms, write 4798 ms, index 419 ms: 7 candidates, 65061 repo-hours, 871 releases; waiting 250 ms; commit 1381 ms)
2026-11-07T16:58:02Z [load_day 2023-03-11] 2023-03-11 hour 19: 133906 events in 3 files, snapshot 1771281309387744509 (task 8257 ms: download 2264 ms overlapping gunzip 456 ms, parse 2059 ms, sort 73 ms, write 4381 ms, index 388 ms: 5 candidates, 54614 repo-hours, 709 releases; waiting 547 ms; commit 1596 ms)
2026-11-07T16:58:10Z [load_day 2023-03-11] 2023-03-11 hour 20: 129741 events in 3 files, snapshot 30531114153550121 (task 8601 ms: download 2694 ms overlapping gunzip 473 ms, parse 2141 ms, sort 72 ms, write 4383 ms, index 339 ms: 6 candidates, 53902 repo-hours, 660 releases; waiting 721 ms; commit 1474 ms)
2026-11-07T16:58:11Z [load_day 2023-03-11] 2023-03-11 hour 21: 131291 events in 3 files, snapshot 2664507865334950640 (task 7587 ms: download 2217 ms overlapping gunzip 419 ms, parse 1863 ms, sort 75 ms, write 4009 ms, index 376 ms: 6 candidates, 63104 repo-hours, 786 releases; waiting 525 ms; commit 1645 ms)
2026-11-07T16:58:18Z [load_day 2023-03-11] 2023-03-11 hour 22: 121972 events in 3 files, snapshot 4822286450327703403 (task 7281 ms: download 1972 ms overlapping gunzip 386 ms, parse 1831 ms, sort 66 ms, write 3842 ms, index 356 ms: 5 candidates, 57056 repo-hours, 724 releases; waiting 505 ms; commit 1484 ms)
2026-11-07T16:58:19Z [load_day 2023-03-11] 2023-03-11 hour 23: 112266 events in 3 files, snapshot 3444333657913974827 (task 7363 ms: download 3319 ms overlapping gunzip 370 ms, parse 1683 ms, sort 76 ms, write 3546 ms, index 351 ms: 2 candidates, 54508 repo-hours, 571 releases; waiting 962 ms; commit 1404 ms)
2026-11-07T16:58:25Z [load_day 2023-03-11] 2023-03-11: repo_hour merged from 24 files to 1 (1450659 rows, 6061 ms)
2026-11-07T16:58:30Z [load_day 2023-03-11] 2023-03-11: index committed in 10987 ms (24 + 1 + 24 files)
2026-11-07T16:58:30Z [load_day 2023-03-11] 2023-03-11: 3263884 events in 24 hours
2026-11-07T16:58:30Z [load_day 2023-03-11] attempt 1 succeeded