orcaLog in

Runs · run 1555

load_day 2023-09-26, attempt 1 succeeded

Host
3c2b29713ce8:370569
Started
2026-11-08T01:11:17Z
Ended
2026-11-08T01:16:02Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1555/load_day/2023-09-26/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:11:17Z [load_day 2023-09-26] attempt 1 started
2026-11-08T01:11:18Z [load_day 2023-09-26] 2023-09-26: 2 hours in flight, about 1200 MB
2026-11-08T01:11:41Z [load_day 2023-09-26] 2023-09-26 hour 0: 173333 events in 4 files, snapshot 2566576492396201943 (task 17790 ms: download 7940 ms overlapping gunzip 1132 ms, parse 5171 ms, sort 187 ms, write 9998 ms, index 583 ms: 9 candidates, 89534 repo-hours, 952 releases; waiting 518 ms; commit 4021 ms)
2026-11-08T01:11:45Z [load_day 2023-09-26] 2023-09-26 hour 1: 170455 events in 4 files, snapshot 1255334196877410090 (task 16631 ms: download 6563 ms overlapping gunzip 1061 ms, parse 4776 ms, sort 202 ms, write 9374 ms, index 732 ms: 10 candidates, 93405 repo-hours, 780 releases; waiting 180 ms; commit 3108 ms)
2026-11-08T01:11:55Z [load_day 2023-09-26] 2023-09-26 hour 2: 145025 events in 3 files, snapshot 7226427707735902026 (task 14349 ms: download 5977 ms overlapping gunzip 945 ms, parse 4042 ms, sort 191 ms, write 7833 ms, index 490 ms: 7 candidates, 62521 repo-hours, 675 releases; waiting 545 ms; commit 2488 ms)
2026-11-08T01:12:01Z [load_day 2023-09-26] 2023-09-26 hour 3: 138022 events in 3 files, snapshot 2865952690552908680 (task 13839 ms: download 5050 ms overlapping gunzip 705 ms, parse 3067 ms, sort 101 ms, write 8849 ms, index 608 ms: 8 candidates, 59419 repo-hours, 826 releases; waiting 284 ms; commit 4067 ms)
2026-11-08T01:12:12Z [load_day 2023-09-26] 2023-09-26 hour 4: 134624 events in 3 files, snapshot 6494853761314899622 (task 14693 ms: download 5627 ms overlapping gunzip 862 ms, parse 3644 ms, sort 136 ms, write 8768 ms, index 512 ms: 5 candidates, 55719 repo-hours, 648 releases; waiting 247 ms; commit 2826 ms)
2026-11-08T01:12:17Z [load_day 2023-09-26] 2023-09-26 hour 5: 144912 events in 3 files, snapshot 338056163742115195 (task 15450 ms: download 6619 ms overlapping gunzip 922 ms, parse 4008 ms, sort 110 ms, write 9389 ms, index 517 ms: 10 candidates, 62164 repo-hours, 805 releases; waiting 205 ms; commit 3476 ms)
2026-11-08T01:12:31Z [load_day 2023-09-26] 2023-09-26 hour 6: 173605 events in 4 files, snapshot 6662466650275808013 (task 18043 ms: download 10569 ms overlapping gunzip 1111 ms, parse 4694 ms, sort 138 ms, write 10270 ms, index 856 ms: 13 candidates, 79647 repo-hours, 970 releases; waiting 615 ms; commit 3092 ms)
2026-11-08T01:12:39Z [load_day 2023-09-26] 2023-09-26 hour 7: 181628 events in 4 files, snapshot 1525523231511202344 (task 19163 ms: download 10921 ms overlapping gunzip 1161 ms, parse 5471 ms, sort 177 ms, write 10934 ms, index 613 ms: 7 candidates, 80779 repo-hours, 1066 releases; waiting 514 ms; commit 4310 ms)
2026-11-08T01:12:52Z [load_day 2023-09-26] 2023-09-26 hour 8: 194904 events in 4 files, snapshot 1096558612514586213 (task 19815 ms: download 11486 ms overlapping gunzip 1190 ms, parse 5192 ms, sort 138 ms, write 12000 ms, index 751 ms: 7 candidates, 88388 repo-hours, 1092 releases; waiting 318 ms; commit 2758 ms)
2026-11-08T01:12:59Z [load_day 2023-09-26] 2023-09-26 hour 9: 200365 events in 5 files, snapshot 7717916642047205948 (task 21222 ms: download 11728 ms overlapping gunzip 1257 ms, parse 5410 ms, sort 161 ms, write 12864 ms, index 673 ms: 10 candidates, 87215 repo-hours, 1154 releases; waiting 504 ms; commit 2304 ms)
2026-11-08T01:13:10Z [load_day 2023-09-26] 2023-09-26 hour 10: 190251 events in 4 files, snapshot 4844795549412628451 (task 17808 ms: download 10740 ms overlapping gunzip 1024 ms, parse 4314 ms, sort 139 ms, write 10339 ms, index 582 ms: 5 candidates, 82395 repo-hours, 919 releases; waiting 1093 ms; commit 2126 ms)
2026-11-08T01:13:17Z [load_day 2023-09-26] 2023-09-26 hour 11: 189800 events in 4 files, snapshot 2217839968597559701 (task 16079 ms: download 9067 ms overlapping gunzip 950 ms, parse 4210 ms, sort 118 ms, write 9433 ms, index 565 ms: 10 candidates, 82541 repo-hours, 1070 releases; waiting 518 ms; commit 2206 ms)
2026-11-08T01:13:31Z [load_day 2023-09-26] 2023-09-26 hour 12: 238137 events in 5 files, snapshot 8470968323702559892 (task 19086 ms: download 11351 ms overlapping gunzip 1025 ms, parse 4768 ms, sort 143 ms, write 11344 ms, index 1013 ms: 12 candidates, 115537 repo-hours, 1210 releases; waiting 522 ms; commit 2724 ms)
2026-11-08T01:13:40Z [load_day 2023-09-26] 2023-09-26 hour 13: 221315 events in 5 files, snapshot 3778197164500500249 (task 22232 ms: download 13951 ms overlapping gunzip 1217 ms, parse 5610 ms, sort 151 ms, write 13743 ms, index 755 ms: 14 candidates, 109778 repo-hours, 1251 releases; waiting 506 ms; commit 2332 ms)
2026-11-08T01:13:54Z [load_day 2023-09-26] 2023-09-26 hour 14: 212045 events in 5 files, snapshot 6531797186435184286 (task 22633 ms: download 14948 ms overlapping gunzip 1337 ms, parse 5916 ms, sort 154 ms, write 13452 ms, index 871 ms: 13 candidates, 99027 repo-hours, 1310 releases; waiting 536 ms; commit 2336 ms)
2026-11-08T01:14:03Z [load_day 2023-09-26] 2023-09-26 hour 15: 188841 events in 4 files, snapshot 1025036090523817348 (task 20690 ms: download 11643 ms overlapping gunzip 1239 ms, parse 5584 ms, sort 201 ms, write 11972 ms, index 894 ms: 12 candidates, 90422 repo-hours, 1265 releases; waiting 529 ms; commit 3419 ms)
2026-11-08T01:14:18Z [load_day 2023-09-26] 2023-09-26 hour 16: 196482 events in 4 files, snapshot 8340594674440000626 (task 22104 ms: download 12717 ms overlapping gunzip 1272 ms, parse 5536 ms, sort 149 ms, write 13766 ms, index 825 ms: 7 candidates, 89230 repo-hours, 1180 releases; waiting 237 ms; commit 2836 ms)
2026-11-08T01:14:24Z [load_day 2023-09-26] 2023-09-26 hour 17: 195131 events in 4 files, snapshot 7167654401297811104 (task 20110 ms: download 11148 ms overlapping gunzip 1217 ms, parse 5371 ms, sort 206 ms, write 11817 ms, index 751 ms: 7 candidates, 84543 repo-hours, 1031 releases; waiting 480 ms; commit 2989 ms)
2026-11-08T01:14:39Z [load_day 2023-09-26] 2023-09-26 hour 18: 197698 events in 4 files, snapshot 2026865196968885851 (task 19945 ms: download 10755 ms overlapping gunzip 1082 ms, parse 4715 ms, sort 151 ms, write 12186 ms, index 927 ms: 7 candidates, 88571 repo-hours, 1203 releases; waiting 483 ms; commit 2580 ms)
2026-11-08T01:14:44Z [load_day 2023-09-26] 2023-09-26 hour 19: 187047 events in 4 files, snapshot 8762791961216083185 (task 19745 ms: download 10764 ms overlapping gunzip 1084 ms, parse 4628 ms, sort 127 ms, write 12438 ms, index 783 ms: 23 candidates, 78330 repo-hours, 1155 releases; waiting 458 ms; commit 2508 ms)
2026-11-08T01:14:58Z [load_day 2023-09-26] 2023-09-26 hour 20: 175999 events in 4 files, snapshot 8815790002664819494 (task 16676 ms: download 9781 ms overlapping gunzip 936 ms, parse 4180 ms, sort 137 ms, write 9959 ms, index 692 ms: 6 candidates, 78377 repo-hours, 1226 releases; waiting 479 ms; commit 4305 ms)
2026-11-08T01:15:36Z [load_day 2023-09-26] 2023-09-26 hour 21: 164202 events in 4 files, snapshot 1200597750721283360 (task 49838 ms: download 48269 ms overlapping gunzip 1174 ms, parse 5391 ms, sort 207 ms, write 9701 ms, index 453 ms: 7 candidates, 65900 repo-hours, 1219 releases; waiting 32614 ms; commit 3911 ms)
2026-11-08T01:15:39Z [load_day 2023-09-26] 2023-09-26 hour 22: 146747 events in 3 files, snapshot 6806791789878909341 (task 14203 ms: download 6386 ms overlapping gunzip 792 ms, parse 3370 ms, sort 123 ms, write 8675 ms, index 556 ms: 7 candidates, 59639 repo-hours, 938 releases; waiting 359 ms; commit 2474 ms)
2026-11-08T01:15:51Z [load_day 2023-09-26] 2023-09-26 hour 23: 133642 events in 3 files, snapshot 4573860972462385818 (task 12009 ms: download 5523 ms overlapping gunzip 720 ms, parse 3325 ms, sort 95 ms, write 6722 ms, index 413 ms: 6 candidates, 55069 repo-hours, 849 releases; waiting 525 ms; commit 5574 ms)
2026-11-08T01:16:00Z [load_day 2023-09-26] 2023-09-26: repo_hour merged from 24 files to 1 (1938150 rows, 8714 ms)
2026-11-08T01:16:02Z [load_day 2023-09-26] 2023-09-26: index committed in 10902 ms (24 + 1 + 24 files)
2026-11-08T01:16:02Z [load_day 2023-09-26] 2023-09-26: 4294210 events in 24 hours
2026-11-08T01:16:02Z [load_day 2023-09-26] attempt 1 succeeded