orcaLog in

Runs · run 1755

load_day 2023-11-19, attempt 1 succeeded

Host
3c2b29713ce8:574812
Started
2026-11-08T02:58:28Z
Ended
2026-11-08T03:01:39Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1755/load_day/2023-11-19/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:58:28Z [load_day 2023-11-19] attempt 1 started
2026-11-08T02:58:30Z [load_day 2023-11-19] 2023-11-19: 2 hours in flight, about 1200 MB
2026-11-08T02:58:43Z [load_day 2023-11-19] 2023-11-19 hour 0: 143490 events in 3 files, snapshot 3215316657714940040 (task 8484 ms: download 1614 ms overlapping gunzip 533 ms, parse 2498 ms, sort 79 ms, write 4717 ms, index 433 ms: 6 candidates, 70083 repo-hours, 919 releases; waiting 18 ms; commit 2800 ms)
2026-11-08T02:58:46Z [load_day 2023-11-19] 2023-11-19 hour 1: 149700 events in 3 files, snapshot 1165180455209622741 (task 8405 ms: download 1445 ms overlapping gunzip 479 ms, parse 2348 ms, sort 100 ms, write 4758 ms, index 512 ms: 6 candidates, 78612 repo-hours, 885 releases; waiting 18 ms; commit 3190 ms)
2026-11-08T02:58:53Z [load_day 2023-11-19] 2023-11-19 hour 2: 116631 events in 3 files, snapshot 1742364210369412182 (task 8217 ms: download 759 ms overlapping gunzip 448 ms, parse 2080 ms, sort 61 ms, write 5060 ms, index 363 ms: 2 candidates, 42957 repo-hours, 660 releases; waiting 17 ms; commit 2998 ms)
2026-11-08T02:58:56Z [load_day 2023-11-19] 2023-11-19 hour 3: 114064 events in 3 files, snapshot 1946470275521226199 (task 8177 ms: download 2186 ms overlapping gunzip 422 ms, parse 2050 ms, sort 120 ms, write 4466 ms, index 400 ms: 8 candidates, 41132 repo-hours, 651 releases; waiting 469 ms; commit 2815 ms)
2026-11-08T02:59:03Z [load_day 2023-11-19] 2023-11-19 hour 4: 115970 events in 3 files, snapshot 4439231819001423969 (task 8616 ms: download 779 ms overlapping gunzip 572 ms, parse 2570 ms, sort 85 ms, write 4768 ms, index 431 ms: 2 candidates, 41460 repo-hours, 521 releases; waiting 16 ms; commit 3221 ms)
2026-11-08T02:59:07Z [load_day 2023-11-19] 2023-11-19 hour 5: 119735 events in 3 files, snapshot 888827615906518053 (task 8974 ms: download 929 ms overlapping gunzip 587 ms, parse 2753 ms, sort 109 ms, write 4722 ms, index 580 ms: 6 candidates, 43808 repo-hours, 566 releases; waiting 21 ms; commit 3682 ms)
2026-11-08T02:59:13Z [load_day 2023-11-19] 2023-11-19 hour 6: 127892 events in 3 files, snapshot 5962678208549469002 (task 9162 ms: download 1076 ms overlapping gunzip 583 ms, parse 2643 ms, sort 105 ms, write 5124 ms, index 480 ms: 5 candidates, 50910 repo-hours, 677 releases; waiting 21 ms; commit 2986 ms)
2026-11-08T02:59:19Z [load_day 2023-11-19] 2023-11-19 hour 7: 126342 events in 3 files, snapshot 5659643930140644027 (task 9104 ms: download 2086 ms overlapping gunzip 514 ms, parse 2384 ms, sort 133 ms, write 4923 ms, index 383 ms: 3 candidates, 45732 repo-hours, 548 releases; waiting 498 ms; commit 5501 ms)
2026-11-08T02:59:25Z [load_day 2023-11-19] 2023-11-19 hour 8: 128553 events in 3 files, snapshot 2503842467712513674 (task 9754 ms: download 1362 ms overlapping gunzip 616 ms, parse 2696 ms, sort 124 ms, write 5451 ms, index 657 ms: 3 candidates, 47995 repo-hours, 695 releases; waiting 22 ms; commit 3741 ms)
2026-11-08T02:59:28Z [load_day 2023-11-19] 2023-11-19 hour 9: 132761 events in 3 files, snapshot 2271485219565688266 (task 10457 ms: download 1794 ms overlapping gunzip 724 ms, parse 3138 ms, sort 88 ms, write 5956 ms, index 336 ms: 4 candidates, 49099 repo-hours, 804 releases; waiting 18 ms; commit 3075 ms)
2026-11-08T02:59:36Z [load_day 2023-11-19] 2023-11-19 hour 10: 140099 events in 3 files, snapshot 4532751873770265972 (task 10677 ms: download 3514 ms overlapping gunzip 572 ms, parse 2628 ms, sort 140 ms, write 5819 ms, index 723 ms: 4 candidates, 52420 repo-hours, 648 releases; waiting 510 ms; commit 2968 ms)
2026-11-08T02:59:41Z [load_day 2023-11-19] 2023-11-19 hour 11: 137907 events in 3 files, snapshot 4495672851004139461 (task 10063 ms: download 1546 ms overlapping gunzip 658 ms, parse 2772 ms, sort 140 ms, write 5862 ms, index 410 ms: 4 candidates, 51686 repo-hours, 688 releases; waiting 27 ms; commit 4304 ms)
2026-11-08T02:59:52Z [load_day 2023-11-19] 2023-11-19 hour 12: 208503 events in 5 files, snapshot 1582087208095875569 (task 12778 ms: download 3965 ms overlapping gunzip 707 ms, parse 3598 ms, sort 165 ms, write 7467 ms, index 603 ms: 3 candidates, 86952 repo-hours, 788 releases; waiting 26 ms; commit 5245 ms)
2026-11-08T02:59:58Z [load_day 2023-11-19] 2023-11-19 hour 13: 169135 events in 4 files, snapshot 4222488791834686147 (task 12908 ms: download 4192 ms overlapping gunzip 684 ms, parse 3079 ms, sort 97 ms, write 8165 ms, index 655 ms: 14 candidates, 77825 repo-hours, 829 releases; waiting 18 ms; commit 6110 ms)
2026-11-08T03:00:14Z [load_day 2023-11-19] 2023-11-19 hour 14: 162460 events in 4 files, snapshot 5672857727027342825 (task 15750 ms: download 5221 ms overlapping gunzip 794 ms, parse 3923 ms, sort 264 ms, write 9312 ms, index 1241 ms: 8 candidates, 65863 repo-hours, 808 releases; waiting 23 ms; commit 8402 ms)
2026-11-08T03:00:21Z [load_day 2023-11-19] 2023-11-19 hour 15: 168358 events in 4 files, snapshot 3151444436100715921 (task 15484 ms: download 4895 ms overlapping gunzip 826 ms, parse 3801 ms, sort 116 ms, write 9455 ms, index 1054 ms: 6 candidates, 68360 repo-hours, 1048 releases; waiting 32 ms; commit 6383 ms)
2026-11-08T03:00:28Z [load_day 2023-11-19] 2023-11-19 hour 16: 163409 events in 4 files, snapshot 2689034412024197117 (task 16113 ms: download 5294 ms overlapping gunzip 848 ms, parse 3990 ms, sort 101 ms, write 9414 ms, index 907 ms: 10 candidates, 67299 repo-hours, 943 releases; waiting 491 ms; commit 4192 ms)
2026-11-08T03:00:34Z [load_day 2023-11-19] 2023-11-19 hour 17: 160253 events in 4 files, snapshot 3873233978118001390 (task 15034 ms: download 4882 ms overlapping gunzip 885 ms, parse 3975 ms, sort 107 ms, write 8855 ms, index 972 ms: 6 candidates, 64007 repo-hours, 946 releases; waiting 29 ms; commit 3172 ms)
2026-11-08T03:00:45Z [load_day 2023-11-19] 2023-11-19 hour 18: 165828 events in 4 files, snapshot 2569545125230123044 (task 14547 ms: download 5017 ms overlapping gunzip 842 ms, parse 3961 ms, sort 218 ms, write 8093 ms, index 868 ms: 11 candidates, 69640 repo-hours, 997 releases; waiting 29 ms; commit 5086 ms)
2026-11-08T03:00:56Z [load_day 2023-11-19] 2023-11-19 hour 19: 148708 events in 3 files, snapshot 4935701238756610780 (task 13255 ms: download 4989 ms overlapping gunzip 750 ms, parse 3631 ms, sort 134 ms, write 7514 ms, index 465 ms: 8 candidates, 58543 repo-hours, 782 releases; waiting 444 ms; commit 10238 ms)
2026-11-08T03:01:02Z [load_day 2023-11-19] 2023-11-19 hour 20: 152628 events in 4 files, snapshot 2312833642707404386 (task 15124 ms: download 5097 ms overlapping gunzip 874 ms, parse 3990 ms, sort 238 ms, write 9155 ms, index 652 ms: 6 candidates, 60158 repo-hours, 992 releases; waiting 17 ms; commit 4991 ms)
2026-11-08T03:01:08Z [load_day 2023-11-19] 2023-11-19 hour 21: 139477 events in 3 files, snapshot 1542365826223199647 (task 13280 ms: download 5077 ms overlapping gunzip 733 ms, parse 3395 ms, sort 171 ms, write 8064 ms, index 696 ms: 5 candidates, 54045 repo-hours, 931 releases; waiting 18 ms; commit 6280 ms)
2026-11-08T03:01:24Z [load_day 2023-11-19] 2023-11-19 hour 22: 132115 events in 3 files, snapshot 8790369787323337434 (task 13776 ms: download 1886 ms overlapping gunzip 877 ms, parse 4027 ms, sort 225 ms, write 8011 ms, index 379 ms: 9 candidates, 50219 repo-hours, 819 releases; waiting 27 ms; commit 12112 ms)
2026-11-08T03:01:28Z [load_day 2023-11-19] 2023-11-19 hour 23: 128126 events in 3 files, snapshot 7531376296387558504 (task 13040 ms: download 1473 ms overlapping gunzip 824 ms, parse 3775 ms, sort 102 ms, write 7722 ms, index 395 ms: 10 candidates, 49515 repo-hours, 704 releases; waiting 16 ms; commit 4322 ms)
2026-11-08T03:01:36Z [load_day 2023-11-19] 2023-11-19: repo_hour merged from 24 files to 1 (1388320 rows, 7446 ms)
2026-11-08T03:01:39Z [load_day 2023-11-19] 2023-11-19: index committed in 10279 ms (24 + 1 + 24 files)
2026-11-08T03:01:39Z [load_day 2023-11-19] 2023-11-19: 3452144 events in 24 hours
2026-11-08T03:01:39Z [load_day 2023-11-19] attempt 1 succeeded