orcaLog in

Runs · run 1336

load_day 2023-07-27, attempt 1 succeeded

Host
3c2b29713ce8:166541
Started
2026-11-07T23:20:15Z
Ended
2026-11-07T23:24:15Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1336/load_day/2023-07-27/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T23:20:15Z [load_day 2023-07-27] attempt 1 started
2026-11-07T23:20:16Z [load_day 2023-07-27] 2023-07-27: 2 hours in flight, about 1200 MB
2026-11-07T23:20:35Z [load_day 2023-07-27] 2023-07-27 hour 0: 152599 events in 4 files, snapshot 3213833514115507717 (task 15041 ms: download 7392 ms overlapping gunzip 867 ms, parse 4138 ms, sort 97 ms, write 8623 ms, index 529 ms: 6 candidates, 79927 repo-hours, 982 releases; waiting 481 ms; commit 2248 ms)
2026-11-07T23:20:44Z [load_day 2023-07-27] 2023-07-27 hour 1: 171871 events in 4 files, snapshot 1469434252257942082 (task 16985 ms: download 7360 ms overlapping gunzip 1020 ms, parse 4509 ms, sort 178 ms, write 9819 ms, index 649 ms: 8 candidates, 85501 repo-hours, 754 releases; waiting 470 ms; commit 8553 ms)
2026-11-07T23:20:52Z [load_day 2023-07-27] 2023-07-27 hour 2: 134366 events in 3 files, snapshot 7686483875133186435 (task 15226 ms: download 7001 ms overlapping gunzip 888 ms, parse 3720 ms, sort 103 ms, write 9315 ms, index 420 ms: 3 candidates, 58847 repo-hours, 765 releases; waiting 505 ms; commit 2314 ms)
2026-11-07T23:20:55Z [load_day 2023-07-27] 2023-07-27 hour 3: 131209 events in 3 files, snapshot 6857475094509669835 (task 14328 ms: download 6984 ms overlapping gunzip 930 ms, parse 4055 ms, sort 215 ms, write 7877 ms, index 443 ms: 4 candidates, 56654 repo-hours, 788 releases; waiting 456 ms; commit 2796 ms)
2026-11-07T23:21:07Z [load_day 2023-07-27] 2023-07-27 hour 4: 129353 events in 3 files, snapshot 8041798499219003426 (task 12168 ms: download 4881 ms overlapping gunzip 789 ms, parse 3317 ms, sort 105 ms, write 6660 ms, index 481 ms: 2 candidates, 54197 repo-hours, 662 releases; waiting 498 ms; commit 3679 ms)
2026-11-07T23:21:10Z [load_day 2023-07-27] 2023-07-27 hour 5: 138554 events in 3 files, snapshot 8813538795701467201 (task 13955 ms: download 5580 ms overlapping gunzip 721 ms, parse 3203 ms, sort 104 ms, write 8152 ms, index 800 ms: 4 candidates, 61264 repo-hours, 752 releases; waiting 626 ms; commit 2592 ms)
2026-11-07T23:21:23Z [load_day 2023-07-27] 2023-07-27 hour 6: 164273 events in 4 files, snapshot 797730253505809798 (task 16287 ms: download 6526 ms overlapping gunzip 810 ms, parse 3692 ms, sort 119 ms, write 10222 ms, index 579 ms: 3 candidates, 76614 repo-hours, 860 releases; waiting 509 ms; commit 2301 ms)
2026-11-07T23:21:29Z [load_day 2023-07-27] 2023-07-27 hour 7: 170744 events in 4 files, snapshot 5533134013365336522 (task 18177 ms: download 10742 ms overlapping gunzip 892 ms, parse 4058 ms, sort 154 ms, write 11680 ms, index 604 ms: 4 candidates, 76067 repo-hours, 948 releases; waiting 511 ms; commit 2620 ms)
2026-11-07T23:21:43Z [load_day 2023-07-27] 2023-07-27 hour 8: 169745 events in 4 files, snapshot 287643285255535550 (task 19019 ms: download 11127 ms overlapping gunzip 942 ms, parse 4178 ms, sort 122 ms, write 12382 ms, index 608 ms: 5 candidates, 78333 repo-hours, 898 releases; waiting 498 ms; commit 2421 ms)
2026-11-07T23:21:52Z [load_day 2023-07-27] 2023-07-27 hour 9: 188349 events in 4 files, snapshot 6739398843903772679 (task 20499 ms: download 11069 ms overlapping gunzip 1112 ms, parse 5238 ms, sort 248 ms, write 12426 ms, index 666 ms: 5 candidates, 82789 repo-hours, 1182 releases; waiting 490 ms; commit 4162 ms)
2026-11-07T23:22:06Z [load_day 2023-07-27] 2023-07-27 hour 10: 182882 events in 4 files, snapshot 3269359073345481190 (task 21861 ms: download 12959 ms overlapping gunzip 1100 ms, parse 4880 ms, sort 263 ms, write 14162 ms, index 598 ms: 7 candidates, 80172 repo-hours, 970 releases; waiting 519 ms; commit 2252 ms)
2026-11-07T23:22:14Z [load_day 2023-07-27] 2023-07-27 hour 11: 183814 events in 4 files, snapshot 7617252646167588879 (task 19289 ms: download 11029 ms overlapping gunzip 1122 ms, parse 5144 ms, sort 235 ms, write 11203 ms, index 745 ms: 3 candidates, 77984 repo-hours, 1022 releases; waiting 482 ms; commit 5072 ms)
2026-11-07T23:22:31Z [load_day 2023-07-27] 2023-07-27 hour 12: 234664 events in 5 files, snapshot 927193113807473441 (task 23329 ms: download 15212 ms overlapping gunzip 1325 ms, parse 6241 ms, sort 176 ms, write 13532 ms, index 1225 ms: 4 candidates, 111139 repo-hours, 1052 releases; waiting 494 ms; commit 3138 ms)
2026-11-07T23:22:35Z [load_day 2023-07-27] 2023-07-27 hour 13: 213491 events in 5 files, snapshot 450554099682898419 (task 23095 ms: download 14876 ms overlapping gunzip 1243 ms, parse 5516 ms, sort 166 ms, write 13924 ms, index 1269 ms: 2 candidates, 105997 repo-hours, 1095 releases; waiting 508 ms; commit 2247 ms)
2026-11-07T23:22:51Z [load_day 2023-07-27] 2023-07-27 hour 14: 192523 events in 4 files, snapshot 8360750390406266619 (task 18925 ms: download 11211 ms overlapping gunzip 1142 ms, parse 5061 ms, sort 160 ms, write 11128 ms, index 617 ms: 4 candidates, 88297 repo-hours, 1085 releases; waiting 512 ms; commit 2067 ms)
2026-11-07T23:22:53Z [load_day 2023-07-27] 2023-07-27 hour 15: 164222 events in 4 files, snapshot 6632563453665520900 (task 16386 ms: download 9939 ms overlapping gunzip 890 ms, parse 4180 ms, sort 133 ms, write 9607 ms, index 534 ms: 2 candidates, 79276 repo-hours, 1021 releases; waiting 687 ms; commit 2105 ms)
2026-11-07T23:23:11Z [load_day 2023-07-27] 2023-07-27 hour 16: 168964 events in 4 files, snapshot 9214383942297318701 (task 19111 ms: download 11620 ms overlapping gunzip 886 ms, parse 3904 ms, sort 94 ms, write 12080 ms, index 702 ms: 6 candidates, 78216 repo-hours, 913 releases; waiting 1142 ms; commit 2034 ms)
2026-11-07T23:23:13Z [load_day 2023-07-27] 2023-07-27 hour 17: 174652 events in 4 files, snapshot 6012465937839702594 (task 17974 ms: download 10265 ms overlapping gunzip 950 ms, parse 4139 ms, sort 138 ms, write 11150 ms, index 686 ms: 5 candidates, 75439 repo-hours, 933 releases; waiting 540 ms; commit 2191 ms)
2026-11-07T23:23:29Z [load_day 2023-07-27] 2023-07-27 hour 18: 181467 events in 4 files, snapshot 4121608868505633951 (task 17230 ms: download 9435 ms overlapping gunzip 1072 ms, parse 4796 ms, sort 121 ms, write 9832 ms, index 694 ms: 6 candidates, 80163 repo-hours, 1037 releases; waiting 332 ms; commit 2399 ms)
2026-11-07T23:23:34Z [load_day 2023-07-27] 2023-07-27 hour 19: 166557 events in 4 files, snapshot 8050542311412679963 (task 17154 ms: download 10371 ms overlapping gunzip 1019 ms, parse 4721 ms, sort 147 ms, write 9861 ms, index 549 ms: 2 candidates, 68969 repo-hours, 960 releases; waiting 524 ms; commit 5014 ms)
2026-11-07T23:23:48Z [load_day 2023-07-27] 2023-07-27 hour 20: 161624 events in 4 files, snapshot 463827524421396754 (task 17696 ms: download 7890 ms overlapping gunzip 927 ms, parse 4132 ms, sort 167 ms, write 11110 ms, index 576 ms: 7 candidates, 67701 repo-hours, 916 releases; waiting 504 ms; commit 2008 ms)
2026-11-07T23:23:51Z [load_day 2023-07-27] 2023-07-27 hour 21: 150205 events in 4 files, snapshot 6859050262221188618 (task 15462 ms: download 6795 ms overlapping gunzip 1034 ms, parse 4352 ms, sort 196 ms, write 8599 ms, index 452 ms: 7 candidates, 60186 repo-hours, 950 releases; waiting 508 ms; commit 2715 ms)
2026-11-07T23:24:02Z [load_day 2023-07-27] 2023-07-27 hour 22: 132519 events in 3 files, snapshot 1802855299177019965 (task 13460 ms: download 6189 ms overlapping gunzip 866 ms, parse 3550 ms, sort 91 ms, write 7533 ms, index 346 ms: 7 candidates, 52931 repo-hours, 819 releases; waiting 706 ms; commit 2029 ms)
2026-11-07T23:24:05Z [load_day 2023-07-27] 2023-07-27 hour 23: 125156 events in 3 files, snapshot 4917974939951174447 (task 12795 ms: download 5467 ms overlapping gunzip 753 ms, parse 3476 ms, sort 89 ms, write 7324 ms, index 355 ms: 2 candidates, 50210 repo-hours, 754 releases; waiting 451 ms; commit 2174 ms)
2026-11-07T23:24:13Z [load_day 2023-07-27] 2023-07-27: repo_hour merged from 24 files to 1 (1786873 rows, 8610 ms)
2026-11-07T23:24:15Z [load_day 2023-07-27] 2023-07-27: index committed in 10765 ms (24 + 1 + 24 files)
2026-11-07T23:24:15Z [load_day 2023-07-27] 2023-07-27: 3983803 events in 24 hours
2026-11-07T23:24:15Z [load_day 2023-07-27] attempt 1 succeeded