orcaLog in

Runs · run 1560

load_day 2023-09-27, attempt 1 succeeded

Host
3c2b29713ce8:374645
Started
2026-11-08T01:13:20Z
Ended
2026-11-08T01:17:22Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1560/load_day/2023-09-27/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:13:20Z [load_day 2023-09-27] attempt 1 started
2026-11-08T01:13:21Z [load_day 2023-09-27] 2023-09-27: 2 hours in flight, about 1200 MB
2026-11-08T01:13:42Z [load_day 2023-09-27] 2023-09-27 hour 0: 156699 events in 4 files, snapshot 1190960272526447124 (task 15231 ms: download 6399 ms overlapping gunzip 1003 ms, parse 4675 ms, sort 122 ms, write 8216 ms, index 687 ms: 11 candidates, 88300 repo-hours, 1008 releases; waiting 20 ms; commit 3345 ms)
2026-11-08T01:13:45Z [load_day 2023-09-27] 2023-09-27 hour 1: 158061 events in 4 files, snapshot 120969651354356747 (task 14218 ms: download 5478 ms overlapping gunzip 801 ms, parse 3658 ms, sort 111 ms, write 7914 ms, index 672 ms: 7 candidates, 89899 repo-hours, 893 releases; waiting 743 ms; commit 3223 ms)
2026-11-08T01:13:57Z [load_day 2023-09-27] 2023-09-27 hour 2: 139966 events in 3 files, snapshot 829511030729728832 (task 13089 ms: download 5528 ms overlapping gunzip 767 ms, parse 3319 ms, sort 199 ms, write 7602 ms, index 433 ms: 10 candidates, 60362 repo-hours, 929 releases; waiting 527 ms; commit 4014 ms)
2026-11-08T01:14:00Z [load_day 2023-09-27] 2023-09-27 hour 3: 135177 events in 3 files, snapshot 2739808845783313194 (task 12698 ms: download 5381 ms overlapping gunzip 642 ms, parse 2987 ms, sort 87 ms, write 7487 ms, index 686 ms: 7 candidates, 59420 repo-hours, 982 releases; waiting 498 ms; commit 3070 ms)
2026-11-08T01:14:09Z [load_day 2023-09-27] 2023-09-27 hour 4: 133031 events in 3 files, snapshot 3660514545253309343 (task 12892 ms: download 4919 ms overlapping gunzip 732 ms, parse 3111 ms, sort 91 ms, write 7661 ms, index 500 ms: 3 candidates, 55684 repo-hours, 767 releases; waiting 533 ms; commit 2870 ms)
2026-11-08T01:14:16Z [load_day 2023-09-27] 2023-09-27 hour 5: 139824 events in 3 files, snapshot 8648624982703009639 (task 14904 ms: download 6107 ms overlapping gunzip 917 ms, parse 3861 ms, sort 129 ms, write 8575 ms, index 627 ms: 6 candidates, 60791 repo-hours, 779 releases; waiting 532 ms; commit 2787 ms)
2026-11-08T01:14:28Z [load_day 2023-09-27] 2023-09-27 hour 6: 169163 events in 4 files, snapshot 9081086980639932229 (task 17413 ms: download 10008 ms overlapping gunzip 1012 ms, parse 4381 ms, sort 117 ms, write 10237 ms, index 677 ms: 5 candidates, 78563 repo-hours, 996 releases; waiting 676 ms; commit 2630 ms)
2026-11-08T01:14:35Z [load_day 2023-09-27] 2023-09-27 hour 7: 174071 events in 4 files, snapshot 5957855889273604638 (task 17982 ms: download 10698 ms overlapping gunzip 1026 ms, parse 4743 ms, sort 114 ms, write 10652 ms, index 604 ms: 6 candidates, 78282 repo-hours, 885 releases; waiting 517 ms; commit 3170 ms)
2026-11-08T01:14:50Z [load_day 2023-09-27] 2023-09-27 hour 8: 188118 events in 4 files, snapshot 4346939594065484069 (task 20109 ms: download 11352 ms overlapping gunzip 1202 ms, parse 5384 ms, sort 157 ms, write 11837 ms, index 583 ms: 9 candidates, 84684 repo-hours, 1076 releases; waiting 548 ms; commit 2662 ms)
2026-11-08T01:14:55Z [load_day 2023-09-27] 2023-09-27 hour 9: 196948 events in 4 files, snapshot 8560298438698184251 (task 19811 ms: download 11511 ms overlapping gunzip 1285 ms, parse 5818 ms, sort 147 ms, write 11550 ms, index 591 ms: 9 candidates, 86521 repo-hours, 1296 releases; waiting 198 ms; commit 2476 ms)
2026-11-08T01:15:09Z [load_day 2023-09-27] 2023-09-27 hour 10: 187187 events in 4 files, snapshot 3715291238251039860 (task 17957 ms: download 9849 ms overlapping gunzip 1076 ms, parse 4686 ms, sort 151 ms, write 10562 ms, index 670 ms: 6 candidates, 84449 repo-hours, 1031 releases; waiting 514 ms; commit 2827 ms)
2026-11-08T01:15:14Z [load_day 2023-09-27] 2023-09-27 hour 11: 184659 events in 4 files, snapshot 543306500041552267 (task 18092 ms: download 10602 ms overlapping gunzip 988 ms, parse 4364 ms, sort 159 ms, write 11164 ms, index 602 ms: 7 candidates, 81268 repo-hours, 1095 releases; waiting 447 ms; commit 2278 ms)
2026-11-08T01:15:27Z [load_day 2023-09-27] 2023-09-27 hour 12: 235229 events in 5 files, snapshot 4284588877197578135 (task 17284 ms: download 10834 ms overlapping gunzip 994 ms, parse 4585 ms, sort 144 ms, write 9943 ms, index 750 ms: 12 candidates, 114772 repo-hours, 1086 releases; waiting 560 ms; commit 2810 ms)
2026-11-08T01:15:34Z [load_day 2023-09-27] 2023-09-27 hour 13: 214750 events in 5 files, snapshot 8141482881084230098 (task 18731 ms: download 11835 ms overlapping gunzip 1041 ms, parse 4660 ms, sort 134 ms, write 11335 ms, index 713 ms: 11 candidates, 107430 repo-hours, 1105 releases; waiting 452 ms; commit 2751 ms)
2026-11-08T01:15:49Z [load_day 2023-09-27] 2023-09-27 hour 14: 204724 events in 5 files, snapshot 5473592230652473722 (task 19787 ms: download 10755 ms overlapping gunzip 1144 ms, parse 5061 ms, sort 153 ms, write 11776 ms, index 872 ms: 10 candidates, 94346 repo-hours, 1104 releases; waiting 517 ms; commit 3636 ms)
2026-11-08T01:15:53Z [load_day 2023-09-27] 2023-09-27 hour 15: 180789 events in 4 files, snapshot 4591028079385163650 (task 17645 ms: download 10274 ms overlapping gunzip 917 ms, parse 4086 ms, sort 114 ms, write 10951 ms, index 643 ms: 8 candidates, 87221 repo-hours, 1073 releases; waiting 614 ms; commit 2814 ms)
2026-11-08T01:16:06Z [load_day 2023-09-27] 2023-09-27 hour 16: 188564 events in 4 files, snapshot 662360643340724143 (task 16789 ms: download 9590 ms overlapping gunzip 885 ms, parse 3982 ms, sort 121 ms, write 10055 ms, index 654 ms: 11 candidates, 86778 repo-hours, 1221 releases; waiting 829 ms; commit 2340 ms)
2026-11-08T01:16:11Z [load_day 2023-09-27] 2023-09-27 hour 17: 188734 events in 4 files, snapshot 5204024046705957782 (task 16798 ms: download 8995 ms overlapping gunzip 909 ms, parse 4074 ms, sort 131 ms, write 9828 ms, index 1040 ms: 6 candidates, 85047 repo-hours, 1117 releases; waiting 509 ms; commit 3102 ms)
2026-11-08T01:16:29Z [load_day 2023-09-27] 2023-09-27 hour 18: 190635 events in 4 files, snapshot 4009230955838716151 (task 19971 ms: download 11499 ms overlapping gunzip 1256 ms, parse 5319 ms, sort 126 ms, write 12142 ms, index 640 ms: 6 candidates, 87685 repo-hours, 1129 releases; waiting 187 ms; commit 4771 ms)
2026-11-08T01:16:32Z [load_day 2023-09-27] 2023-09-27 hour 19: 176810 events in 4 files, snapshot 7722276758488928387 (task 18211 ms: download 10750 ms overlapping gunzip 979 ms, parse 4292 ms, sort 134 ms, write 11180 ms, index 847 ms: 10 candidates, 76447 repo-hours, 978 releases; waiting 476 ms; commit 2830 ms)
2026-11-08T01:16:47Z [load_day 2023-09-27] 2023-09-27 hour 20: 172365 events in 4 files, snapshot 3153913226275435564 (task 19057 ms: download 10565 ms overlapping gunzip 1026 ms, parse 5000 ms, sort 215 ms, write 11426 ms, index 636 ms: 9 candidates, 74944 repo-hours, 924 releases; waiting 493 ms; commit 2492 ms)
2026-11-08T01:16:53Z [load_day 2023-09-27] 2023-09-27 hour 21: 160020 events in 4 files, snapshot 9114659836082122825 (task 16920 ms: download 6992 ms overlapping gunzip 1002 ms, parse 4341 ms, sort 111 ms, write 10077 ms, index 658 ms: 4 candidates, 68686 repo-hours, 838 releases; waiting 501 ms; commit 5701 ms)
2026-11-08T01:17:09Z [load_day 2023-09-27] 2023-09-27 hour 22: 140298 events in 3 files, snapshot 6166029342779362354 (task 15388 ms: download 6501 ms overlapping gunzip 779 ms, parse 3488 ms, sort 123 ms, write 9620 ms, index 556 ms: 5 candidates, 58722 repo-hours, 814 releases; waiting 550 ms; commit 7716 ms)
2026-11-08T01:17:11Z [load_day 2023-09-27] 2023-09-27 hour 23: 135775 events in 3 files, snapshot 7155430237598339727 (task 13997 ms: download 6644 ms overlapping gunzip 924 ms, parse 3964 ms, sort 136 ms, write 7732 ms, index 482 ms: 5 candidates, 57278 repo-hours, 694 releases; waiting 481 ms; commit 2355 ms)
2026-11-08T01:17:20Z [load_day 2023-09-27] 2023-09-27: repo_hour merged from 24 files to 1 (1907579 rows, 8538 ms)
2026-11-08T01:17:22Z [load_day 2023-09-27] 2023-09-27: index committed in 10991 ms (24 + 1 + 24 files)
2026-11-08T01:17:22Z [load_day 2023-09-27] 2023-09-27: 4151597 events in 24 hours
2026-11-08T01:17:22Z [load_day 2023-09-27] attempt 1 succeeded