orcaLog in

Runs · run 1703

load_day 2023-11-05, attempt 1 succeeded

Host
3c2b29713ce8:521078
Started
2026-11-08T02:30:29Z
Ended
2026-11-08T02:33:07Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1703/load_day/2023-11-05/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:30:29Z [load_day 2023-11-05] attempt 1 started
2026-11-08T02:30:30Z [load_day 2023-11-05] 2023-11-05: 2 hours in flight, about 1200 MB
2026-11-08T02:30:45Z [load_day 2023-11-05] 2023-11-05 hour 0: 128311 events in 3 files, snapshot 2099919829741096257 (task 10213 ms: download 1428 ms overlapping gunzip 563 ms, parse 2664 ms, sort 91 ms, write 6007 ms, index 650 ms: 9 candidates, 71099 repo-hours, 964 releases; waiting 27 ms; commit 2649 ms)
2026-11-08T02:30:49Z [load_day 2023-11-05] 2023-11-05 hour 1: 129483 events in 3 files, snapshot 5109811788288992499 (task 9598 ms: download 1213 ms overlapping gunzip 479 ms, parse 2141 ms, sort 67 ms, write 5812 ms, index 575 ms: 4 candidates, 77234 repo-hours, 807 releases; waiting 16 ms; commit 3645 ms)
2026-11-08T02:30:55Z [load_day 2023-11-05] 2023-11-05 hour 2: 98774 events in 2 files, snapshot 4796599974730010306 (task 7087 ms: download 885 ms overlapping gunzip 390 ms, parse 1664 ms, sort 110 ms, write 4397 ms, index 315 ms: 6 candidates, 42631 repo-hours, 594 releases; waiting 23 ms; commit 4191 ms)
2026-11-08T02:30:59Z [load_day 2023-11-05] 2023-11-05 hour 3: 98349 events in 2 files, snapshot 925524510790005757 (task 6963 ms: download 820 ms overlapping gunzip 390 ms, parse 1759 ms, sort 76 ms, write 4124 ms, index 385 ms: 4 candidates, 42644 repo-hours, 819 releases; waiting 23 ms; commit 3978 ms)
2026-11-08T02:31:02Z [load_day 2023-11-05] 2023-11-05 hour 4: 99567 events in 2 files, snapshot 7960844348358397120 (task 6368 ms: download 853 ms overlapping gunzip 398 ms, parse 1662 ms, sort 70 ms, write 3682 ms, index 346 ms: 6 candidates, 42363 repo-hours, 645 releases; waiting 18 ms; commit 2576 ms)
2026-11-08T02:31:05Z [load_day 2023-11-05] 2023-11-05 hour 5: 101342 events in 3 files, snapshot 5824760389769313888 (task 6360 ms: download 665 ms overlapping gunzip 364 ms, parse 1620 ms, sort 72 ms, write 3776 ms, index 315 ms: 5 candidates, 43721 repo-hours, 811 releases; waiting 16 ms; commit 2104 ms)
2026-11-08T02:31:09Z [load_day 2023-11-05] 2023-11-05 hour 6: 112929 events in 3 files, snapshot 2327316430490587798 (task 7079 ms: download 782 ms overlapping gunzip 453 ms, parse 2081 ms, sort 117 ms, write 3695 ms, index 515 ms: 4 candidates, 51384 repo-hours, 721 releases; waiting 25 ms; commit 2299 ms)
2026-11-08T02:31:13Z [load_day 2023-11-05] 2023-11-05 hour 7: 109118 events in 3 files, snapshot 4766550904252188164 (task 6502 ms: download 670 ms overlapping gunzip 371 ms, parse 1657 ms, sort 74 ms, write 3867 ms, index 326 ms: 9 candidates, 45880 repo-hours, 536 releases; waiting 16 ms; commit 2882 ms)
2026-11-08T02:31:20Z [load_day 2023-11-05] 2023-11-05 hour 8: 115650 events in 3 files, snapshot 7857002774183662081 (task 8583 ms: download 791 ms overlapping gunzip 506 ms, parse 2324 ms, sort 113 ms, write 4976 ms, index 449 ms: 9 candidates, 49088 repo-hours, 743 releases; waiting 21 ms; commit 3022 ms)
2026-11-08T02:31:24Z [load_day 2023-11-05] 2023-11-05 hour 9: 113503 events in 3 files, snapshot 2849705341455159401 (task 10471 ms: download 2383 ms overlapping gunzip 651 ms, parse 2818 ms, sort 94 ms, write 5790 ms, index 362 ms: 4 candidates, 47997 repo-hours, 936 releases; waiting 488 ms; commit 2943 ms)
2026-11-08T02:31:32Z [load_day 2023-11-05] 2023-11-05 hour 10: 131002 events in 3 files, snapshot 5043013300276899784 (task 10709 ms: download 1817 ms overlapping gunzip 734 ms, parse 3378 ms, sort 107 ms, write 5775 ms, index 486 ms: 3 candidates, 56301 repo-hours, 713 releases; waiting 31 ms; commit 3833 ms)
2026-11-08T02:31:37Z [load_day 2023-11-05] 2023-11-05 hour 11: 135435 events in 3 files, snapshot 5726324296766497796 (task 10344 ms: download 1661 ms overlapping gunzip 623 ms, parse 2919 ms, sort 88 ms, write 5787 ms, index 704 ms: 13 candidates, 55952 repo-hours, 688 releases; waiting 24 ms; commit 4490 ms)
2026-11-08T02:31:47Z [load_day 2023-11-05] 2023-11-05 hour 12: 210518 events in 5 files, snapshot 5785434669281854498 (task 15070 ms: download 4768 ms overlapping gunzip 798 ms, parse 3988 ms, sort 169 ms, write 9120 ms, index 748 ms: 8 candidates, 89724 repo-hours, 739 releases; waiting 16 ms; commit 3022 ms)
2026-11-08T02:31:53Z [load_day 2023-11-05] 2023-11-05 hour 13: 169394 events in 4 files, snapshot 97054193092721200 (task 13631 ms: download 4552 ms overlapping gunzip 802 ms, parse 3896 ms, sort 154 ms, write 7370 ms, index 1191 ms: 9 candidates, 84442 repo-hours, 830 releases; waiting 17 ms; commit 5093 ms)
2026-11-08T02:32:01Z [load_day 2023-11-05] 2023-11-05 hour 14: 156731 events in 4 files, snapshot 5966299527193577534 (task 13029 ms: download 4450 ms overlapping gunzip 780 ms, parse 3462 ms, sort 116 ms, write 7729 ms, index 728 ms: 4 candidates, 68453 repo-hours, 908 releases; waiting 19 ms; commit 3121 ms)
2026-11-08T02:32:08Z [load_day 2023-11-05] 2023-11-05 hour 15: 161468 events in 4 files, snapshot 4905684787766687655 (task 14551 ms: download 4809 ms overlapping gunzip 767 ms, parse 3789 ms, sort 156 ms, write 8956 ms, index 674 ms: 3 candidates, 71520 repo-hours, 1490 releases; waiting 14 ms; commit 5016 ms)
2026-11-08T02:32:18Z [load_day 2023-11-05] 2023-11-05 hour 16: 160804 events in 4 files, snapshot 8862545210698664446 (task 14931 ms: download 5835 ms overlapping gunzip 909 ms, parse 4066 ms, sort 101 ms, write 8547 ms, index 507 ms: 3 candidates, 69540 repo-hours, 1009 releases; waiting 498 ms; commit 3055 ms)
2026-11-08T02:32:22Z [load_day 2023-11-05] 2023-11-05 hour 17: 151485 events in 4 files, snapshot 5993985274366976421 (task 13598 ms: download 4673 ms overlapping gunzip 849 ms, parse 3854 ms, sort 207 ms, write 7957 ms, index 504 ms: 5 candidates, 63870 repo-hours, 888 releases; waiting 20 ms; commit 2961 ms)
2026-11-08T02:32:31Z [load_day 2023-11-05] 2023-11-05 hour 18: 158040 events in 4 files, snapshot 2198586665938005658 (task 11834 ms: download 3640 ms overlapping gunzip 701 ms, parse 3026 ms, sort 176 ms, write 7200 ms, index 492 ms: 7 candidates, 71182 repo-hours, 862 releases; waiting 24 ms; commit 3056 ms)
2026-11-08T02:32:36Z [load_day 2023-11-05] 2023-11-05 hour 19: 139493 events in 3 files, snapshot 5301845780046863073 (task 13326 ms: download 5260 ms overlapping gunzip 825 ms, parse 3978 ms, sort 179 ms, write 6962 ms, index 596 ms: 7 candidates, 58813 repo-hours, 787 releases; waiting 556 ms; commit 3015 ms)
2026-11-08T02:32:43Z [load_day 2023-11-05] 2023-11-05 hour 20: 136614 events in 3 files, snapshot 8153969237233295222 (task 11727 ms: download 4375 ms overlapping gunzip 643 ms, parse 2887 ms, sort 121 ms, write 7322 ms, index 533 ms: 5 candidates, 58457 repo-hours, 803 releases; waiting 28 ms; commit 3204 ms)
2026-11-08T02:32:49Z [load_day 2023-11-05] 2023-11-05 hour 21: 126200 events in 3 files, snapshot 6609993973378516850 (task 10507 ms: download 1444 ms overlapping gunzip 633 ms, parse 3029 ms, sort 137 ms, write 5713 ms, index 764 ms: 5 candidates, 53123 repo-hours, 876 releases; waiting 28 ms; commit 3663 ms)
2026-11-08T02:32:54Z [load_day 2023-11-05] 2023-11-05 hour 22: 121129 events in 3 files, snapshot 7852010051617840213 (task 10189 ms: download 1595 ms overlapping gunzip 540 ms, parse 2489 ms, sort 98 ms, write 6460 ms, index 389 ms: 19 candidates, 49762 repo-hours, 1095 releases; waiting 22 ms; commit 2505 ms)
2026-11-08T02:32:58Z [load_day 2023-11-05] 2023-11-05 hour 23: 115717 events in 3 files, snapshot 4982153933385727259 (task 9384 ms: download 2003 ms overlapping gunzip 541 ms, parse 2431 ms, sort 92 ms, write 5202 ms, index 437 ms: 6 candidates, 48565 repo-hours, 693 releases; waiting 461 ms; commit 2296 ms)
2026-11-08T02:33:05Z [load_day 2023-11-05] 2023-11-05: repo_hour merged from 24 files to 1 (1413745 rows, 6803 ms)
2026-11-08T02:33:07Z [load_day 2023-11-05] 2023-11-05: index committed in 9402 ms (24 + 1 + 24 files)
2026-11-08T02:33:07Z [load_day 2023-11-05] 2023-11-05: 3181056 events in 24 hours
2026-11-08T02:33:07Z [load_day 2023-11-05] attempt 1 succeeded