orcaLog in

Runs · run 1519

load_day 2023-09-18, attempt 1 succeeded

Host
3c2b29713ce8:341871
Started
2026-11-08T00:55:57Z
Ended
2026-11-08T01:00:30Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1519/load_day/2023-09-18/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T00:55:57Z [load_day 2023-09-18] attempt 1 started
2026-11-08T00:55:59Z [load_day 2023-09-18] 2023-09-18: 2 hours in flight, about 1200 MB
2026-11-08T00:56:19Z [load_day 2023-09-18] 2023-09-18 hour 0: 147731 events in 3 files, snapshot 5447289897876337537 (task 13481 ms: download 5847 ms overlapping gunzip 822 ms, parse 3907 ms, sort 97 ms, write 7327 ms, index 628 ms: 8 candidates, 78863 repo-hours, 906 releases; waiting 486 ms; commit 4588 ms)
2026-11-08T00:56:21Z [load_day 2023-09-18] 2023-09-18 hour 1: 160067 events in 4 files, snapshot 3883873293300666449 (task 15833 ms: download 6696 ms overlapping gunzip 1010 ms, parse 4260 ms, sort 129 ms, write 8768 ms, index 823 ms: 10 candidates, 86030 repo-hours, 784 releases; waiting 524 ms; commit 2340 ms)
2026-11-08T00:56:32Z [load_day 2023-09-18] 2023-09-18 hour 2: 136222 events in 3 files, snapshot 1221543029193338468 (task 14032 ms: download 6571 ms overlapping gunzip 883 ms, parse 3584 ms, sort 96 ms, write 8251 ms, index 432 ms: 3 candidates, 56906 repo-hours, 665 releases; waiting 541 ms; commit 3031 ms)
2026-11-08T00:56:35Z [load_day 2023-09-18] 2023-09-18 hour 3: 135561 events in 3 files, snapshot 639764492380948392 (task 13170 ms: download 5416 ms overlapping gunzip 774 ms, parse 3357 ms, sort 128 ms, write 7785 ms, index 650 ms: 7 candidates, 55803 repo-hours, 860 releases; waiting 214 ms; commit 2463 ms)
2026-11-08T00:56:48Z [load_day 2023-09-18] 2023-09-18 hour 4: 132800 events in 3 files, snapshot 6203735569176621359 (task 14449 ms: download 6810 ms overlapping gunzip 1123 ms, parse 4545 ms, sort 119 ms, write 7550 ms, index 357 ms: 7 candidates, 50877 repo-hours, 548 releases; waiting 516 ms; commit 3060 ms)
2026-11-08T00:56:53Z [load_day 2023-09-18] 2023-09-18 hour 5: 146699 events in 3 files, snapshot 5702088968980238940 (task 15880 ms: download 6745 ms overlapping gunzip 923 ms, parse 3621 ms, sort 167 ms, write 9406 ms, index 927 ms: 11 candidates, 58311 repo-hours, 784 releases; waiting 527 ms; commit 2978 ms)
2026-11-08T00:57:08Z [load_day 2023-09-18] 2023-09-18 hour 6: 175666 events in 4 files, snapshot 5679854868044226331 (task 18944 ms: download 11430 ms overlapping gunzip 1103 ms, parse 4546 ms, sort 157 ms, write 11801 ms, index 573 ms: 11 candidates, 77088 repo-hours, 910 releases; waiting 501 ms; commit 3600 ms)
2026-11-08T00:57:17Z [load_day 2023-09-18] 2023-09-18 hour 7: 184487 events in 4 files, snapshot 2545798939520866844 (task 20712 ms: download 12222 ms overlapping gunzip 1224 ms, parse 5562 ms, sort 168 ms, write 12340 ms, index 541 ms: 8 candidates, 78278 repo-hours, 1054 releases; waiting 541 ms; commit 5064 ms)
2026-11-08T00:57:30Z [load_day 2023-09-18] 2023-09-18 hour 8: 197339 events in 4 files, snapshot 5533301182669784541 (task 21766 ms: download 13277 ms overlapping gunzip 1265 ms, parse 5181 ms, sort 126 ms, write 13501 ms, index 600 ms: 18 candidates, 85286 repo-hours, 1235 releases; waiting 778 ms; commit 2131 ms)
2026-11-08T00:57:36Z [load_day 2023-09-18] 2023-09-18 hour 9: 197931 events in 4 files, snapshot 9185868499224277717 (task 20480 ms: download 12542 ms overlapping gunzip 1313 ms, parse 5781 ms, sort 130 ms, write 11906 ms, index 573 ms: 8 candidates, 84385 repo-hours, 1277 releases; waiting 536 ms; commit 2334 ms)
2026-11-08T00:57:51Z [load_day 2023-09-18] 2023-09-18 hour 10: 192287 events in 4 files, snapshot 8633956734504462586 (task 19915 ms: download 10087 ms overlapping gunzip 1132 ms, parse 4714 ms, sort 137 ms, write 12576 ms, index 665 ms: 13 candidates, 82104 repo-hours, 1045 releases; waiting 391 ms; commit 2412 ms)
2026-11-08T00:57:58Z [load_day 2023-09-18] 2023-09-18 hour 11: 195876 events in 4 files, snapshot 8661802780126669129 (task 21416 ms: download 11682 ms overlapping gunzip 1320 ms, parse 6131 ms, sort 134 ms, write 12421 ms, index 601 ms: 7 candidates, 82255 repo-hours, 979 releases; waiting 511 ms; commit 2591 ms)
2026-11-08T00:58:16Z [load_day 2023-09-18] 2023-09-18 hour 12: 245635 events in 5 files, snapshot 5173223099626158643 (task 23608 ms: download 15446 ms overlapping gunzip 1322 ms, parse 6022 ms, sort 193 ms, write 14249 ms, index 992 ms: 15 candidates, 114897 repo-hours, 1111 releases; waiting 519 ms; commit 2638 ms)
2026-11-08T00:58:25Z [load_day 2023-09-18] 2023-09-18 hour 13: 227286 events in 5 files, snapshot 2534068225039671878 (task 25582 ms: download 16404 ms overlapping gunzip 1674 ms, parse 7217 ms, sort 144 ms, write 14828 ms, index 922 ms: 16 candidates, 109182 repo-hours, 1102 releases; waiting 473 ms; commit 2844 ms)
2026-11-08T00:58:43Z [load_day 2023-09-18] 2023-09-18 hour 14: 217626 events in 5 files, snapshot 8965220854658859215 (task 26020 ms: download 18213 ms overlapping gunzip 1665 ms, parse 7018 ms, sort 171 ms, write 15797 ms, index 784 ms: 13 candidates, 96053 repo-hours, 1252 releases; waiting 327 ms; commit 2385 ms)
2026-11-08T00:58:50Z [load_day 2023-09-18] 2023-09-18 hour 15: 190233 events in 4 files, snapshot 1492192500519019612 (task 23810 ms: download 14177 ms overlapping gunzip 1316 ms, parse 5740 ms, sort 179 ms, write 15017 ms, index 718 ms: 9 candidates, 87708 repo-hours, 1141 releases; waiting 504 ms; commit 2972 ms)
2026-11-08T00:59:06Z [load_day 2023-09-18] 2023-09-18 hour 16: 183750 events in 4 files, snapshot 8080340795926491308 (task 22090 ms: download 12974 ms overlapping gunzip 1271 ms, parse 5338 ms, sort 197 ms, write 13630 ms, index 685 ms: 8 candidates, 83656 repo-hours, 1010 releases; waiting 673 ms; commit 2425 ms)
2026-11-08T00:59:14Z [load_day 2023-09-18] 2023-09-18 hour 17: 197598 events in 4 files, snapshot 8238242253172073535 (task 22809 ms: download 13521 ms overlapping gunzip 1672 ms, parse 7016 ms, sort 160 ms, write 12456 ms, index 793 ms: 8 candidates, 83182 repo-hours, 923 releases; waiting 491 ms; commit 2783 ms)
2026-11-08T00:59:33Z [load_day 2023-09-18] 2023-09-18 hour 18: 204067 events in 5 files, snapshot 6978078988954462400 (task 23155 ms: download 16297 ms overlapping gunzip 1318 ms, parse 5606 ms, sort 201 ms, write 14498 ms, index 697 ms: 13 candidates, 89211 repo-hours, 1028 releases; waiting 530 ms; commit 5276 ms)
2026-11-08T00:59:41Z [load_day 2023-09-18] 2023-09-18 hour 19: 189381 events in 4 files, snapshot 5774928901387521312 (task 24445 ms: download 14717 ms overlapping gunzip 1283 ms, parse 5887 ms, sort 177 ms, write 14409 ms, index 711 ms: 15 candidates, 78086 repo-hours, 1138 releases; waiting 1621 ms; commit 3923 ms)
2026-11-08T00:59:53Z [load_day 2023-09-18] 2023-09-18 hour 20: 186028 events in 4 files, snapshot 169761252568395768 (task 20604 ms: download 12688 ms overlapping gunzip 1352 ms, parse 5814 ms, sort 128 ms, write 12033 ms, index 580 ms: 7 candidates, 76688 repo-hours, 931 releases; waiting 486 ms; commit 3127 ms)
2026-11-08T01:00:02Z [load_day 2023-09-18] 2023-09-18 hour 21: 169563 events in 4 files, snapshot 1401253134039088848 (task 19781 ms: download 11180 ms overlapping gunzip 1251 ms, parse 5397 ms, sort 166 ms, write 11444 ms, index 734 ms: 10 candidates, 67248 repo-hours, 965 releases; waiting 453 ms; commit 3714 ms)
2026-11-08T01:00:12Z [load_day 2023-09-18] 2023-09-18 hour 22: 152468 events in 4 files, snapshot 5037360356833034226 (task 19454 ms: download 8217 ms overlapping gunzip 1151 ms, parse 5137 ms, sort 174 ms, write 11591 ms, index 423 ms: 12 candidates, 59884 repo-hours, 864 releases; waiting 548 ms; commit 2531 ms)
2026-11-08T01:00:20Z [load_day 2023-09-18] 2023-09-18 hour 23: 147405 events in 3 files, snapshot 5233944010162154368 (task 16554 ms: download 8033 ms overlapping gunzip 1130 ms, parse 5121 ms, sort 129 ms, write 8861 ms, index 488 ms: 9 candidates, 58013 repo-hours, 745 releases; waiting 498 ms; commit 4254 ms)
2026-11-08T01:00:28Z [load_day 2023-09-18] 2023-09-18: repo_hour merged from 24 files to 1 (1879994 rows, 7894 ms)
2026-11-08T01:00:30Z [load_day 2023-09-18] 2023-09-18: index committed in 9881 ms (24 + 1 + 24 files)
2026-11-08T01:00:30Z [load_day 2023-09-18] 2023-09-18: 4313706 events in 24 hours
2026-11-08T01:00:30Z [load_day 2023-09-18] attempt 1 succeeded