orcaLog in

Runs · run 1714

load_day 2023-11-08, attempt 1 succeeded

Host
3c2b29713ce8:531358
Started
2026-11-08T02:35:48Z
Ended
2026-11-08T02:40:11Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1714/load_day/2023-11-08/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:35:48Z [load_day 2023-11-08] attempt 1 started
2026-11-08T02:35:49Z [load_day 2023-11-08] 2023-11-08: 2 hours in flight, about 1200 MB
2026-11-08T02:36:12Z [load_day 2023-11-08] 2023-11-08 hour 0: 184167 events in 4 files, snapshot 864508584623072729 (task 18464 ms: download 11266 ms overlapping gunzip 1051 ms, parse 5360 ms, sort 173 ms, write 10995 ms, index 645 ms: 8 candidates, 95983 repo-hours, 938 releases; waiting 41 ms; commit 2379 ms)
2026-11-08T02:36:15Z [load_day 2023-11-08] 2023-11-08 hour 1: 184069 events in 4 files, snapshot 6028413857463234680 (task 17346 ms: download 9858 ms overlapping gunzip 1066 ms, parse 4858 ms, sort 128 ms, write 10267 ms, index 813 ms: 6 candidates, 98736 repo-hours, 900 releases; waiting 14 ms; commit 2943 ms)
2026-11-08T02:36:29Z [load_day 2023-11-08] 2023-11-08 hour 2: 154457 events in 4 files, snapshot 2323346501544617912 (task 14960 ms: download 5932 ms overlapping gunzip 789 ms, parse 3428 ms, sort 129 ms, write 9324 ms, index 547 ms: 6 candidates, 66927 repo-hours, 804 releases; waiting 506 ms; commit 3102 ms)
2026-11-08T02:36:32Z [load_day 2023-11-08] 2023-11-08 hour 3: 146506 events in 3 files, snapshot 1928550249439915411 (task 13547 ms: download 5545 ms overlapping gunzip 900 ms, parse 4190 ms, sort 102 ms, write 7611 ms, index 508 ms: 9 candidates, 62296 repo-hours, 997 releases; waiting 29 ms; commit 2828 ms)
2026-11-08T02:36:42Z [load_day 2023-11-08] 2023-11-08 hour 4: 138537 events in 3 files, snapshot 487896416317909240 (task 12739 ms: download 5083 ms overlapping gunzip 750 ms, parse 3330 ms, sort 132 ms, write 7186 ms, index 608 ms: 3 candidates, 57776 repo-hours, 748 releases; waiting 534 ms; commit 2869 ms)
2026-11-08T02:36:48Z [load_day 2023-11-08] 2023-11-08 hour 5: 146767 events in 3 files, snapshot 2183358292801489877 (task 14807 ms: download 5626 ms overlapping gunzip 813 ms, parse 3574 ms, sort 141 ms, write 9264 ms, index 782 ms: 2 candidates, 61939 repo-hours, 794 releases; waiting 24 ms; commit 3324 ms)
2026-11-08T02:36:59Z [load_day 2023-11-08] 2023-11-08 hour 6: 167937 events in 4 files, snapshot 5421867255224078484 (task 16026 ms: download 9658 ms overlapping gunzip 916 ms, parse 4248 ms, sort 170 ms, write 9843 ms, index 605 ms: 8 candidates, 75868 repo-hours, 979 releases; waiting 26 ms; commit 2780 ms)
2026-11-08T02:37:08Z [load_day 2023-11-08] 2023-11-08 hour 7: 175752 events in 4 files, snapshot 5479968817415871691 (task 18294 ms: download 10002 ms overlapping gunzip 1018 ms, parse 4634 ms, sort 133 ms, write 10843 ms, index 889 ms: 9 candidates, 75781 repo-hours, 954 releases; waiting 497 ms; commit 3443 ms)
2026-11-08T02:37:21Z [load_day 2023-11-08] 2023-11-08 hour 8: 190361 events in 4 files, snapshot 7748081258992541196 (task 20636 ms: download 12397 ms overlapping gunzip 1284 ms, parse 5894 ms, sort 149 ms, write 12470 ms, index 610 ms: 4 candidates, 83863 repo-hours, 1194 releases; waiting 19 ms; commit 2571 ms)
2026-11-08T02:37:28Z [load_day 2023-11-08] 2023-11-08 hour 9: 199509 events in 4 files, snapshot 6856687231200212510 (task 19039 ms: download 11502 ms overlapping gunzip 1295 ms, parse 5661 ms, sort 137 ms, write 11126 ms, index 610 ms: 4 candidates, 87595 repo-hours, 1288 releases; waiting 24 ms; commit 2599 ms)
2026-11-08T02:37:47Z [load_day 2023-11-08] 2023-11-08 hour 10: 199208 events in 4 files, snapshot 2321995154586197759 (task 16825 ms: download 9333 ms overlapping gunzip 1065 ms, parse 4798 ms, sort 122 ms, write 10045 ms, index 594 ms: 9 candidates, 86518 repo-hours, 1109 releases; waiting 13 ms; commit 10832 ms)
2026-11-08T02:37:50Z [load_day 2023-11-08] 2023-11-08 hour 11: 194602 events in 4 files, snapshot 5741363743899028435 (task 19620 ms: download 9039 ms overlapping gunzip 1224 ms, parse 5418 ms, sort 193 ms, write 11094 ms, index 937 ms: 4 candidates, 84109 repo-hours, 1200 releases; waiting 489 ms; commit 2804 ms)
2026-11-08T02:38:05Z [load_day 2023-11-08] 2023-11-08 hour 12: 244700 events in 5 files, snapshot 2021019872384851604 (task 22556 ms: download 14497 ms overlapping gunzip 1321 ms, parse 5986 ms, sort 165 ms, write 13741 ms, index 1124 ms: 10 candidates, 115294 repo-hours, 1115 releases; waiting 20 ms; commit 5080 ms)
2026-11-08T02:38:14Z [load_day 2023-11-08] 2023-11-08 hour 13: 227745 events in 5 files, snapshot 4886917185037394628 (task 22201 ms: download 13530 ms overlapping gunzip 1229 ms, parse 5855 ms, sort 135 ms, write 13855 ms, index 914 ms: 8 candidates, 112491 repo-hours, 1093 releases; waiting 25 ms; commit 2869 ms)
2026-11-08T02:38:30Z [load_day 2023-11-08] 2023-11-08 hour 14: 219165 events in 5 files, snapshot 5191264105286772762 (task 25007 ms: download 16662 ms overlapping gunzip 1501 ms, parse 6891 ms, sort 156 ms, write 14897 ms, index 775 ms: 8 candidates, 99628 repo-hours, 1236 releases; waiting 520 ms; commit 3649 ms)
2026-11-08T02:38:41Z [load_day 2023-11-08] 2023-11-08 hour 15: 220727 events in 5 files, snapshot 8711758419278173887 (task 25792 ms: download 16723 ms overlapping gunzip 1487 ms, parse 7380 ms, sort 279 ms, write 15591 ms, index 834 ms: 9 candidates, 98837 repo-hours, 1426 releases; waiting 26 ms; commit 3083 ms)
2026-11-08T02:38:57Z [load_day 2023-11-08] 2023-11-08 hour 16: 214055 events in 5 files, snapshot 2959335076876529327 (task 25464 ms: download 17998 ms overlapping gunzip 1807 ms, parse 7933 ms, sort 159 ms, write 13767 ms, index 709 ms: 5 candidates, 95939 repo-hours, 1368 releases; waiting 513 ms; commit 4078 ms)
2026-11-08T02:39:06Z [load_day 2023-11-08] 2023-11-08 hour 17: 202789 events in 5 files, snapshot 4204782919424477591 (task 23723 ms: download 13390 ms overlapping gunzip 1474 ms, parse 6727 ms, sort 127 ms, write 13926 ms, index 714 ms: 5 candidates, 87802 repo-hours, 1518 releases; waiting 503 ms; commit 3080 ms)
2026-11-08T02:39:21Z [load_day 2023-11-08] 2023-11-08 hour 18: 199485 events in 4 files, snapshot 1131252782910827401 (task 20726 ms: download 11252 ms overlapping gunzip 1326 ms, parse 5644 ms, sort 124 ms, write 12679 ms, index 728 ms: 10 candidates, 91503 repo-hours, 1104 releases; waiting 22 ms; commit 5769 ms)
2026-11-08T02:39:26Z [load_day 2023-11-08] 2023-11-08 hour 19: 186642 events in 4 files, snapshot 3662660521757342105 (task 18994 ms: download 10396 ms overlapping gunzip 1066 ms, parse 4600 ms, sort 129 ms, write 12340 ms, index 608 ms: 8 candidates, 78733 repo-hours, 1085 releases; waiting 20 ms; commit 3419 ms)
2026-11-08T02:39:38Z [load_day 2023-11-08] 2023-11-08 hour 20: 181593 events in 4 files, snapshot 6505495518350786733 (task 19189 ms: download 11623 ms overlapping gunzip 1020 ms, parse 4701 ms, sort 145 ms, write 12407 ms, index 694 ms: 9 candidates, 77650 repo-hours, 992 releases; waiting 14 ms; commit 2908 ms)
2026-11-08T02:39:45Z [load_day 2023-11-08] 2023-11-08 hour 21: 169315 events in 4 files, snapshot 5268238444113514319 (task 18016 ms: download 10664 ms overlapping gunzip 1182 ms, parse 5394 ms, sort 114 ms, write 10320 ms, index 781 ms: 5 candidates, 71134 repo-hours, 1351 releases; waiting 27 ms; commit 3197 ms)
2026-11-08T02:39:57Z [load_day 2023-11-08] 2023-11-08 hour 22: 158321 events in 4 files, snapshot 1126307746320890629 (task 17542 ms: download 7494 ms overlapping gunzip 1007 ms, parse 4323 ms, sort 110 ms, write 11189 ms, index 686 ms: 6 candidates, 65295 repo-hours, 1944 releases; waiting 29 ms; commit 2688 ms)
2026-11-08T02:39:59Z [load_day 2023-11-08] 2023-11-08 hour 23: 143009 events in 3 files, snapshot 3608099626013477582 (task 14200 ms: download 6292 ms overlapping gunzip 883 ms, parse 3980 ms, sort 118 ms, write 8565 ms, index 442 ms: 3 candidates, 61562 repo-hours, 836 releases; waiting 22 ms; commit 2300 ms)
2026-11-08T02:40:08Z [load_day 2023-11-08] 2023-11-08: repo_hour merged from 24 files to 1 (1993259 rows, 8963 ms)
2026-11-08T02:40:11Z [load_day 2023-11-08] 2023-11-08: index committed in 11529 ms (24 + 1 + 24 files)
2026-11-08T02:40:11Z [load_day 2023-11-08] 2023-11-08: 4449418 events in 24 hours
2026-11-08T02:40:11Z [load_day 2023-11-08] attempt 1 succeeded