orcaLog in

Runs · run 1598

load_day 2023-10-07, attempt 1 succeeded

Host
3c2b29713ce8:411582
Started
2026-11-08T01:33:01Z
Ended
2026-11-08T01:35:49Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1598/load_day/2023-10-07/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:33:01Z [load_day 2023-10-07] attempt 1 started
2026-11-08T01:33:02Z [load_day 2023-10-07] 2023-10-07: 2 hours in flight, about 1200 MB
2026-11-08T01:33:21Z [load_day 2023-10-07] 2023-10-07 hour 0: 147556 events in 3 files, snapshot 6667326426878081813 (task 15074 ms: download 6769 ms overlapping gunzip 1024 ms, parse 4537 ms, sort 118 ms, write 8037 ms, index 504 ms: 7 candidates, 80644 repo-hours, 918 releases; waiting 540 ms; commit 2424 ms)
2026-11-08T01:33:24Z [load_day 2023-10-07] 2023-10-07 hour 1: 140776 events in 3 files, snapshot 5811135221234789341 (task 14336 ms: download 5882 ms overlapping gunzip 807 ms, parse 3684 ms, sort 120 ms, write 7962 ms, index 594 ms: 3 candidates, 80010 repo-hours, 783 releases; waiting 477 ms; commit 2568 ms)
2026-11-08T01:33:37Z [load_day 2023-10-07] 2023-10-07 hour 2: 123434 events in 3 files, snapshot 5774384964962746740 (task 12032 ms: download 5470 ms overlapping gunzip 825 ms, parse 3331 ms, sort 123 ms, write 6656 ms, index 369 ms: 9 candidates, 54884 repo-hours, 691 releases; waiting 504 ms; commit 5145 ms)
2026-11-08T01:33:40Z [load_day 2023-10-07] 2023-10-07 hour 3: 113314 events in 3 files, snapshot 3420157366689024327 (task 10873 ms: download 3399 ms overlapping gunzip 637 ms, parse 2606 ms, sort 89 ms, write 6250 ms, index 372 ms: 3 candidates, 49793 repo-hours, 810 releases; waiting 609 ms; commit 2851 ms)
2026-11-08T01:33:46Z [load_day 2023-10-07] 2023-10-07 hour 4: 108472 events in 3 files, snapshot 996373114284405642 (task 9631 ms: download 2274 ms overlapping gunzip 518 ms, parse 2365 ms, sort 113 ms, write 5462 ms, index 379 ms: 5 candidates, 44900 repo-hours, 640 releases; waiting 496 ms; commit 2695 ms)
2026-11-08T01:33:52Z [load_day 2023-10-07] 2023-10-07 hour 5: 111596 events in 3 files, snapshot 844933360781831785 (task 10338 ms: download 2949 ms overlapping gunzip 635 ms, parse 2807 ms, sort 110 ms, write 5344 ms, index 595 ms: 6 candidates, 49270 repo-hours, 566 releases; waiting 533 ms; commit 2969 ms)
2026-11-08T01:33:58Z [load_day 2023-10-07] 2023-10-07 hour 6: 130249 events in 3 files, snapshot 8658084390552004433 (task 11562 ms: download 4730 ms overlapping gunzip 671 ms, parse 2773 ms, sort 151 ms, write 6587 ms, index 524 ms: 7 candidates, 61190 repo-hours, 781 releases; waiting 520 ms; commit 2684 ms)
2026-11-08T01:34:04Z [load_day 2023-10-07] 2023-10-07 hour 7: 129554 events in 3 files, snapshot 1581486540886545397 (task 11472 ms: download 4539 ms overlapping gunzip 636 ms, parse 2752 ms, sort 85 ms, write 6810 ms, index 465 ms: 3 candidates, 57207 repo-hours, 709 releases; waiting 464 ms; commit 2787 ms)
2026-11-08T01:34:11Z [load_day 2023-10-07] 2023-10-07 hour 8: 136485 events in 3 files, snapshot 8429478462878083189 (task 11144 ms: download 4620 ms overlapping gunzip 674 ms, parse 2781 ms, sort 83 ms, write 6322 ms, index 466 ms: 4 candidates, 60256 repo-hours, 703 releases; waiting 492 ms; commit 3312 ms)
2026-11-08T01:34:18Z [load_day 2023-10-07] 2023-10-07 hour 9: 140387 events in 3 files, snapshot 2664173408206566713 (task 12384 ms: download 4539 ms overlapping gunzip 616 ms, parse 2663 ms, sort 90 ms, write 7410 ms, index 409 ms: 4 candidates, 60999 repo-hours, 873 releases; waiting 488 ms; commit 3130 ms)
2026-11-08T01:34:23Z [load_day 2023-10-07] 2023-10-07 hour 10: 138325 events in 3 files, snapshot 7056914193007361491 (task 11651 ms: download 4305 ms overlapping gunzip 758 ms, parse 3193 ms, sort 162 ms, write 6609 ms, index 400 ms: 9 candidates, 59968 repo-hours, 722 releases; waiting 227 ms; commit 2608 ms)
2026-11-08T01:34:31Z [load_day 2023-10-07] 2023-10-07 hour 11: 129877 events in 3 files, snapshot 2514206568527593840 (task 11008 ms: download 4053 ms overlapping gunzip 611 ms, parse 2775 ms, sort 86 ms, write 6306 ms, index 444 ms: 7 candidates, 56601 repo-hours, 703 releases; waiting 222 ms; commit 3509 ms)
2026-11-08T01:34:39Z [load_day 2023-10-07] 2023-10-07 hour 12: 204226 events in 5 files, snapshot 5179482433323539491 (task 14693 ms: download 4646 ms overlapping gunzip 860 ms, parse 4295 ms, sort 175 ms, write 7867 ms, index 702 ms: 8 candidates, 89649 repo-hours, 791 releases; waiting 482 ms; commit 2532 ms)
2026-11-08T01:34:43Z [load_day 2023-10-07] 2023-10-07 hour 13: 161217 events in 4 files, snapshot 4260974235482984961 (task 13128 ms: download 4434 ms overlapping gunzip 701 ms, parse 3204 ms, sort 105 ms, write 7934 ms, index 633 ms: 6 candidates, 81481 repo-hours, 767 releases; waiting 275 ms; commit 2303 ms)
2026-11-08T01:34:55Z [load_day 2023-10-07] 2023-10-07 hour 14: 147250 events in 3 files, snapshot 4696731107007863159 (task 12177 ms: download 4516 ms overlapping gunzip 592 ms, parse 2644 ms, sort 104 ms, write 7523 ms, index 518 ms: 8 candidates, 64063 repo-hours, 746 releases; waiting 500 ms; commit 5711 ms)
2026-11-08T01:34:58Z [load_day 2023-10-07] 2023-10-07 hour 15: 150212 events in 4 files, snapshot 4017999136534625635 (task 12131 ms: download 4714 ms overlapping gunzip 765 ms, parse 3265 ms, sort 100 ms, write 6451 ms, index 959 ms: 5 candidates, 67114 repo-hours, 880 releases; waiting 392 ms; commit 2598 ms)
2026-11-08T01:35:07Z [load_day 2023-10-07] 2023-10-07 hour 16: 145795 events in 3 files, snapshot 2277990567558555038 (task 12281 ms: download 5066 ms overlapping gunzip 781 ms, parse 3474 ms, sort 133 ms, write 6606 ms, index 523 ms: 5 candidates, 64154 repo-hours, 729 releases; waiting 503 ms; commit 3362 ms)
2026-11-08T01:35:09Z [load_day 2023-10-07] 2023-10-07 hour 17: 139306 events in 3 files, snapshot 1859731224573779726 (task 10215 ms: download 4428 ms overlapping gunzip 663 ms, parse 2811 ms, sort 96 ms, write 5483 ms, index 418 ms: 2 candidates, 59815 repo-hours, 678 releases; waiting 486 ms; commit 2463 ms)
2026-11-08T01:35:17Z [load_day 2023-10-07] 2023-10-07 hour 18: 144664 events in 3 files, snapshot 5477217820080492166 (task 10193 ms: download 3657 ms overlapping gunzip 572 ms, parse 2560 ms, sort 111 ms, write 5689 ms, index 469 ms: 2 candidates, 66053 repo-hours, 718 releases; waiting 501 ms; commit 2503 ms)
2026-11-08T01:35:19Z [load_day 2023-10-07] 2023-10-07 hour 19: 125739 events in 3 files, snapshot 6762465276586218095 (task 9045 ms: download 2350 ms overlapping gunzip 501 ms, parse 2203 ms, sort 95 ms, write 5029 ms, index 390 ms: 11 candidates, 52634 repo-hours, 742 releases; waiting 514 ms; commit 2314 ms)
2026-11-08T01:35:28Z [load_day 2023-10-07] 2023-10-07 hour 20: 122924 events in 3 files, snapshot 7195796613484032792 (task 9915 ms: download 2060 ms overlapping gunzip 514 ms, parse 2221 ms, sort 104 ms, write 5936 ms, index 365 ms: 3 candidates, 52546 repo-hours, 637 releases; waiting 514 ms; commit 2539 ms)
2026-11-08T01:35:31Z [load_day 2023-10-07] 2023-10-07 hour 21: 112088 events in 3 files, snapshot 6868656897133573985 (task 9306 ms: download 2473 ms overlapping gunzip 458 ms, parse 2146 ms, sort 66 ms, write 5378 ms, index 362 ms: 6 candidates, 46226 repo-hours, 755 releases; waiting 613 ms; commit 2786 ms)
2026-11-08T01:35:38Z [load_day 2023-10-07] 2023-10-07 hour 22: 103559 events in 3 files, snapshot 7885509470387309559 (task 8755 ms: download 2789 ms overlapping gunzip 505 ms, parse 2146 ms, sort 103 ms, write 4649 ms, index 393 ms: 5 candidates, 41643 repo-hours, 974 releases; waiting 629 ms; commit 2912 ms)
2026-11-08T01:35:40Z [load_day 2023-10-07] 2023-10-07 hour 23: 96004 events in 2 files, snapshot 8658539895035066843 (task 7553 ms: download 1919 ms overlapping gunzip 403 ms, parse 1753 ms, sort 127 ms, write 4094 ms, index 279 ms: 5 candidates, 40420 repo-hours, 734 releases; waiting 545 ms; commit 2190 ms)
2026-11-08T01:35:47Z [load_day 2023-10-07] 2023-10-07: repo_hour merged from 24 files to 1 (1441520 rows, 7029 ms)
2026-11-08T01:35:49Z [load_day 2023-10-07] 2023-10-07: index committed in 9182 ms (24 + 1 + 24 files)
2026-11-08T01:35:49Z [load_day 2023-10-07] 2023-10-07: 3203009 events in 24 hours
2026-11-08T01:35:49Z [load_day 2023-10-07] attempt 1 succeeded