orcaLog in

Runs · run 1747

load_day 2023-11-17, attempt 1 succeeded

Host
3c2b29713ce8:566997
Started
2026-11-08T02:54:16Z
Ended
2026-11-08T02:58:27Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1747/load_day/2023-11-17/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:54:16Z [load_day 2023-11-17] attempt 1 started
2026-11-08T02:54:18Z [load_day 2023-11-17] 2023-11-17: 2 hours in flight, about 1200 MB
2026-11-08T02:54:47Z [load_day 2023-11-17] 2023-11-17 hour 0: 182441 events in 4 files, snapshot 8654428441706420906 (task 17146 ms: download 9958 ms overlapping gunzip 1038 ms, parse 4691 ms, sort 107 ms, write 10258 ms, index 800 ms: 10 candidates, 90258 repo-hours, 1057 releases; waiting 17 ms; commit 10181 ms)
2026-11-08T02:54:50Z [load_day 2023-11-17] 2023-11-17 hour 1: 183597 events in 4 files, snapshot 686464094030200912 (task 16266 ms: download 8821 ms overlapping gunzip 944 ms, parse 4320 ms, sort 132 ms, write 9758 ms, index 868 ms: 38 candidates, 96141 repo-hours, 896 releases; waiting 28 ms; commit 3608 ms)
2026-11-08T02:54:56Z [load_day 2023-11-17] 2023-11-17 hour 2: 155937 events in 4 files, snapshot 2361598917598102847 (task 14944 ms: download 5454 ms overlapping gunzip 801 ms, parse 3532 ms, sort 147 ms, write 9736 ms, index 512 ms: 21 candidates, 64342 repo-hours, 862 releases; waiting 23 ms; commit 3152 ms)
2026-11-08T02:55:05Z [load_day 2023-11-17] 2023-11-17 hour 3: 145945 events in 3 files, snapshot 7856437951152474138 (task 13849 ms: download 5311 ms overlapping gunzip 896 ms, parse 3845 ms, sort 111 ms, write 8235 ms, index 539 ms: 26 candidates, 60396 repo-hours, 914 releases; waiting 20 ms; commit 2969 ms)
2026-11-08T02:55:10Z [load_day 2023-11-17] 2023-11-17 hour 4: 141148 events in 3 files, snapshot 3952945446589594274 (task 12980 ms: download 5283 ms overlapping gunzip 863 ms, parse 3705 ms, sort 97 ms, write 7370 ms, index 713 ms: 30 candidates, 55047 repo-hours, 615 releases; waiting 23 ms; commit 3215 ms)
2026-11-08T02:55:21Z [load_day 2023-11-17] 2023-11-17 hour 5: 149644 events in 3 files, snapshot 8693116587084583095 (task 13471 ms: download 5685 ms overlapping gunzip 955 ms, parse 4151 ms, sort 114 ms, write 7564 ms, index 472 ms: 29 candidates, 59810 repo-hours, 663 releases; waiting 28 ms; commit 4353 ms)
2026-11-08T02:55:26Z [load_day 2023-11-17] 2023-11-17 hour 6: 169041 events in 4 files, snapshot 964913932178024540 (task 14854 ms: download 5602 ms overlapping gunzip 941 ms, parse 4196 ms, sort 104 ms, write 8894 ms, index 495 ms: 45 candidates, 72596 repo-hours, 838 releases; waiting 15 ms; commit 3331 ms)
2026-11-08T02:55:38Z [load_day 2023-11-17] 2023-11-17 hour 7: 174527 events in 4 files, snapshot 3637172339286179448 (task 16311 ms: download 9514 ms overlapping gunzip 967 ms, parse 4188 ms, sort 112 ms, write 10222 ms, index 571 ms: 18 candidates, 71392 repo-hours, 804 releases; waiting 25 ms; commit 2780 ms)
2026-11-08T02:55:44Z [load_day 2023-11-17] 2023-11-17 hour 8: 195602 events in 4 files, snapshot 7732563351741331407 (task 17669 ms: download 10303 ms overlapping gunzip 1280 ms, parse 5432 ms, sort 176 ms, write 10004 ms, index 526 ms: 25 candidates, 79664 repo-hours, 980 releases; waiting 37 ms; commit 2631 ms)
2026-11-08T02:55:55Z [load_day 2023-11-17] 2023-11-17 hour 9: 203767 events in 5 files, snapshot 4137436347595258089 (task 16318 ms: download 9201 ms overlapping gunzip 1094 ms, parse 4776 ms, sort 152 ms, write 9499 ms, index 565 ms: 20 candidates, 81850 repo-hours, 1234 releases; waiting 33 ms; commit 2544 ms)
2026-11-08T02:56:02Z [load_day 2023-11-17] 2023-11-17 hour 10: 198195 events in 4 files, snapshot 5912580600957113524 (task 15816 ms: download 8178 ms overlapping gunzip 983 ms, parse 4353 ms, sort 141 ms, write 9146 ms, index 984 ms: 21 candidates, 83069 repo-hours, 1482 releases; waiting 20 ms; commit 3092 ms)
2026-11-08T02:56:17Z [load_day 2023-11-17] 2023-11-17 hour 11: 190348 events in 4 files, snapshot 5658131421160225544 (task 17864 ms: download 9722 ms overlapping gunzip 1030 ms, parse 4661 ms, sort 148 ms, write 11263 ms, index 562 ms: 17 candidates, 77232 repo-hours, 1003 releases; waiting 18 ms; commit 5547 ms)
2026-11-08T02:56:24Z [load_day 2023-11-17] 2023-11-17 hour 12: 238239 events in 5 files, snapshot 5055212928892533206 (task 20528 ms: download 12076 ms overlapping gunzip 1199 ms, parse 5471 ms, sort 145 ms, write 12198 ms, index 757 ms: 14 candidates, 107842 repo-hours, 1129 releases; waiting 482 ms; commit 3085 ms)
2026-11-08T02:56:38Z [load_day 2023-11-17] 2023-11-17 hour 13: 222201 events in 5 files, snapshot 3861986506780575712 (task 22357 ms: download 14350 ms overlapping gunzip 1364 ms, parse 6312 ms, sort 187 ms, write 13445 ms, index 835 ms: 29 candidates, 105205 repo-hours, 1183 releases; waiting 18 ms; commit 3034 ms)
2026-11-08T02:56:49Z [load_day 2023-11-17] 2023-11-17 hour 14: 214792 events in 5 files, snapshot 3151795108784569457 (task 22704 ms: download 15020 ms overlapping gunzip 1342 ms, parse 6093 ms, sort 181 ms, write 14042 ms, index 834 ms: 25 candidates, 92512 repo-hours, 1181 releases; waiting 22 ms; commit 5127 ms)
2026-11-08T02:57:05Z [load_day 2023-11-17] 2023-11-17 hour 15: 213933 events in 5 files, snapshot 5494211632092723542 (task 24276 ms: download 16809 ms overlapping gunzip 1470 ms, parse 6510 ms, sort 187 ms, write 15231 ms, index 673 ms: 17 candidates, 93266 repo-hours, 1386 releases; waiting 17 ms; commit 4893 ms)
2026-11-08T02:57:13Z [load_day 2023-11-17] 2023-11-17 hour 16: 193863 events in 4 files, snapshot 2327610603636390109 (task 22083 ms: download 12761 ms overlapping gunzip 1299 ms, parse 6016 ms, sort 143 ms, write 13661 ms, index 723 ms: 17 candidates, 86098 repo-hours, 1131 releases; waiting 28 ms; commit 5141 ms)
2026-11-08T02:57:28Z [load_day 2023-11-17] 2023-11-17 hour 17: 194660 events in 4 files, snapshot 344821971169853277 (task 19790 ms: download 11115 ms overlapping gunzip 1220 ms, parse 5301 ms, sort 109 ms, write 12211 ms, index 692 ms: 22 candidates, 82290 repo-hours, 1086 releases; waiting 40 ms; commit 6861 ms)
2026-11-08T02:57:34Z [load_day 2023-11-17] 2023-11-17 hour 18: 196068 events in 4 files, snapshot 2097696818875161890 (task 19981 ms: download 10526 ms overlapping gunzip 1337 ms, parse 6058 ms, sort 149 ms, write 11408 ms, index 782 ms: 21 candidates, 85682 repo-hours, 1262 releases; waiting 28 ms; commit 5328 ms)
2026-11-08T02:57:46Z [load_day 2023-11-17] 2023-11-17 hour 19: 181108 events in 4 files, snapshot 3356397012004266786 (task 18376 ms: download 10920 ms overlapping gunzip 1311 ms, parse 5716 ms, sort 118 ms, write 10466 ms, index 540 ms: 23 candidates, 71638 repo-hours, 1057 releases; waiting 20 ms; commit 4506 ms)
2026-11-08T02:57:50Z [load_day 2023-11-17] 2023-11-17 hour 20: 170371 events in 4 files, snapshot 1677702828638616698 (task 16159 ms: download 9135 ms overlapping gunzip 995 ms, parse 4303 ms, sort 115 ms, write 10028 ms, index 467 ms: 23 candidates, 68765 repo-hours, 891 releases; waiting 21 ms; commit 2808 ms)
2026-11-08T02:58:03Z [load_day 2023-11-17] 2023-11-17 hour 21: 162115 events in 4 files, snapshot 1761983121422173313 (task 17638 ms: download 6366 ms overlapping gunzip 1063 ms, parse 4573 ms, sort 101 ms, write 11054 ms, index 638 ms: 22 candidates, 64236 repo-hours, 1042 releases; waiting 20 ms; commit 3253 ms)
2026-11-08T02:58:07Z [load_day 2023-11-17] 2023-11-17 hour 22: 157355 events in 4 files, snapshot 7165795446832715248 (task 16626 ms: download 5991 ms overlapping gunzip 1083 ms, parse 4368 ms, sort 96 ms, write 10099 ms, index 768 ms: 10 candidates, 61653 repo-hours, 969 releases; waiting 25 ms; commit 2635 ms)
2026-11-08T02:58:17Z [load_day 2023-11-17] 2023-11-17 hour 23: 140024 events in 3 files, snapshot 8501438677166589659 (task 11584 ms: download 5037 ms overlapping gunzip 679 ms, parse 2916 ms, sort 92 ms, write 7321 ms, index 372 ms: 12 candidates, 55184 repo-hours, 903 releases; waiting 15 ms; commit 4558 ms)
2026-11-08T02:58:25Z [load_day 2023-11-17] 2023-11-17: repo_hour merged from 24 files to 1 (1866168 rows, 8089 ms)
2026-11-08T02:58:27Z [load_day 2023-11-17] 2023-11-17: index committed in 9878 ms (24 + 1 + 24 files)
2026-11-08T02:58:27Z [load_day 2023-11-17] 2023-11-17: 4374921 events in 24 hours
2026-11-08T02:58:27Z [load_day 2023-11-17] attempt 1 succeeded