orcaLog in

Runs · run 3212

load_day 2024-11-17, attempt 1 succeeded

Host
73ccef5e07c2:208556
Started
2026-11-08T08:52:48Z
Ended
2026-11-08T08:54:39Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
678 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
163.9 s, up to 2.5 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3212/load_day/2024-11-17/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T08:52:48Z [load_day 2024-11-17] attempt 1 started
2026-11-08T08:52:49Z [load_day 2024-11-17] 2024-11-17: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:52:55Z [load_day 2024-11-17] 2024-11-17 hour 0: 157420 events in 0 files, snapshot none (task 5588 ms: download 2608 ms overlapping gunzip 791 ms, parse 3353 ms, sort 0 ms, write 0 ms, index 587 ms: 5 candidates, 73206 repo-hours, 1190 releases; waiting 576 ms; commit 0 ms)
2026-11-08T08:52:55Z [load_day 2024-11-17] 2024-11-17 hour 1: 178102 events in 0 files, snapshot none (task 5511 ms: download 2189 ms overlapping gunzip 767 ms, parse 3403 ms, sort 0 ms, write 0 ms, index 740 ms: 5 candidates, 100444 repo-hours, 908 releases; waiting 228 ms; commit 0 ms)
2026-11-08T08:53:00Z [load_day 2024-11-17] 2024-11-17 hour 2: 127428 events in 0 files, snapshot none (task 4355 ms: download 1957 ms overlapping gunzip 621 ms, parse 2770 ms, sort 0 ms, write 0 ms, index 533 ms: 4 candidates, 57037 repo-hours, 779 releases; waiting 184 ms; commit 0 ms)
2026-11-08T08:53:03Z [load_day 2024-11-17] 2024-11-17 hour 3: 170009 events in 0 files, snapshot none (task 7310 ms: download 4087 ms overlapping gunzip 1087 ms, parse 4518 ms, sort 0 ms, write 0 ms, index 925 ms: 2 candidates, 62438 repo-hours, 3956 releases; waiting 482 ms; commit 0 ms)
2026-11-08T08:53:07Z [load_day 2024-11-17] 2024-11-17 hour 4: 155143 events in 0 files, snapshot none (task 6292 ms: download 3606 ms overlapping gunzip 919 ms, parse 3831 ms, sort 0 ms, write 0 ms, index 801 ms: 6 candidates, 50859 repo-hours, 621 releases; waiting 505 ms; commit 0 ms)
2026-11-08T08:53:10Z [load_day 2024-11-17] 2024-11-17 hour 5: 164342 events in 0 files, snapshot none (task 6481 ms: download 3955 ms overlapping gunzip 983 ms, parse 3908 ms, sort 0 ms, write 0 ms, index 670 ms: 6 candidates, 52696 repo-hours, 708 releases; waiting 581 ms; commit 0 ms)
2026-11-08T08:53:15Z [load_day 2024-11-17] 2024-11-17 hour 6: 184060 events in 0 files, snapshot none (task 7808 ms: download 4794 ms overlapping gunzip 1211 ms, parse 4817 ms, sort 0 ms, write 0 ms, index 656 ms: 7 candidates, 70417 repo-hours, 749 releases; waiting 756 ms; commit 0 ms)
2026-11-08T08:53:18Z [load_day 2024-11-17] 2024-11-17 hour 7: 170664 events in 0 files, snapshot none (task 6997 ms: download 4066 ms overlapping gunzip 1020 ms, parse 4082 ms, sort 0 ms, write 0 ms, index 900 ms: 3 candidates, 56502 repo-hours, 616 releases; waiting 461 ms; commit 0 ms)
2026-11-08T08:53:23Z [load_day 2024-11-17] 2024-11-17 hour 8: 185802 events in 0 files, snapshot none (task 6801 ms: download 4077 ms overlapping gunzip 1153 ms, parse 4508 ms, sort 0 ms, write 0 ms, index 513 ms: 5 candidates, 61095 repo-hours, 738 releases; waiting 239 ms; commit 0 ms)
2026-11-08T08:53:25Z [load_day 2024-11-17] 2024-11-17 hour 9: 190696 events in 0 files, snapshot none (task 6757 ms: download 4321 ms overlapping gunzip 1045 ms, parse 4255 ms, sort 0 ms, write 0 ms, index 674 ms: 4 candidates, 69323 repo-hours, 820 releases; waiting 526 ms; commit 0 ms)
2026-11-08T08:53:30Z [load_day 2024-11-17] 2024-11-17 hour 10: 188400 events in 0 files, snapshot none (task 6980 ms: download 4505 ms overlapping gunzip 1108 ms, parse 4502 ms, sort 0 ms, write 0 ms, index 572 ms: 4 candidates, 64477 repo-hours, 673 releases; waiting 518 ms; commit 0 ms)
2026-11-08T08:53:32Z [load_day 2024-11-17] 2024-11-17 hour 11: 192556 events in 0 files, snapshot none (task 6522 ms: download 4431 ms overlapping gunzip 1054 ms, parse 4412 ms, sort 0 ms, write 0 ms, index 433 ms: 4 candidates, 64081 repo-hours, 688 releases; waiting 376 ms; commit 0 ms)
2026-11-08T08:53:36Z [load_day 2024-11-17] 2024-11-17 hour 12: 232982 events in 0 files, snapshot none (task 5916 ms: download 2873 ms overlapping gunzip 752 ms, parse 3445 ms, sort 0 ms, write 0 ms, index 857 ms: 7 candidates, 117081 repo-hours, 916 releases; waiting 534 ms; commit 0 ms)
2026-11-08T08:53:39Z [load_day 2024-11-17] 2024-11-17 hour 13: 222356 events in 0 files, snapshot none (task 6396 ms: download 3970 ms overlapping gunzip 986 ms, parse 3951 ms, sort 0 ms, write 0 ms, index 643 ms: 6 candidates, 90120 repo-hours, 831 releases; waiting 499 ms; commit 0 ms)
2026-11-08T08:53:43Z [load_day 2024-11-17] 2024-11-17 hour 14: 218026 events in 0 files, snapshot none (task 6598 ms: download 4322 ms overlapping gunzip 1073 ms, parse 4135 ms, sort 0 ms, write 0 ms, index 576 ms: 6 candidates, 76011 repo-hours, 977 releases; waiting 521 ms; commit 0 ms)
2026-11-08T08:53:49Z [load_day 2024-11-17] 2024-11-17 hour 15: 228393 events in 0 files, snapshot none (task 9460 ms: download 5965 ms overlapping gunzip 1426 ms, parse 5526 ms, sort 0 ms, write 0 ms, index 909 ms: 7 candidates, 87251 repo-hours, 1082 releases; waiting 1272 ms; commit 0 ms)
2026-11-08T08:53:53Z [load_day 2024-11-17] 2024-11-17 hour 16: 218158 events in 0 files, snapshot none (task 8727 ms: download 6446 ms overlapping gunzip 1393 ms, parse 5912 ms, sort 0 ms, write 0 ms, index 618 ms: 5 candidates, 78646 repo-hours, 936 releases; waiting 544 ms; commit 0 ms)
2026-11-08T08:54:02Z [load_day 2024-11-17] 2024-11-17 hour 17: 214742 events in 0 files, snapshot none (task 12715 ms: download 9861 ms overlapping gunzip 1510 ms, parse 6416 ms, sort 0 ms, write 0 ms, index 723 ms: 5 candidates, 75102 repo-hours, 836 releases; waiting 3738 ms; commit 0 ms)
2026-11-08T08:54:02Z [load_day 2024-11-17] 2024-11-17 hour 18: 227307 events in 0 files, snapshot none (task 9068 ms: download 6244 ms overlapping gunzip 1402 ms, parse 5830 ms, sort 0 ms, write 0 ms, index 981 ms: 33 candidates, 89505 repo-hours, 1006 releases; waiting 509 ms; commit 0 ms)
2026-11-08T08:54:12Z [load_day 2024-11-17] 2024-11-17 hour 19: 214216 events in 0 files, snapshot none (task 9231 ms: download 6751 ms overlapping gunzip 1540 ms, parse 6375 ms, sort 0 ms, write 0 ms, index 528 ms: 3 candidates, 70260 repo-hours, 854 releases; waiting 481 ms; commit 0 ms)
2026-11-08T08:54:12Z [load_day 2024-11-17] 2024-11-17 hour 20: 204753 events in 0 files, snapshot none (task 9064 ms: download 6075 ms overlapping gunzip 1514 ms, parse 6125 ms, sort 0 ms, write 0 ms, index 593 ms: 7 candidates, 74114 repo-hours, 889 releases; waiting 533 ms; commit 0 ms)
2026-11-08T08:54:21Z [load_day 2024-11-17] 2024-11-17 hour 21: 195865 events in 0 files, snapshot none (task 8917 ms: download 5922 ms overlapping gunzip 1497 ms, parse 5947 ms, sort 0 ms, write 0 ms, index 646 ms: 10 candidates, 69165 repo-hours, 1029 releases; waiting 536 ms; commit 0 ms)
2026-11-08T08:54:22Z [load_day 2024-11-17] 2024-11-17 hour 22: 184578 events in 0 files, snapshot none (task 9234 ms: download 6498 ms overlapping gunzip 1513 ms, parse 6050 ms, sort 0 ms, write 0 ms, index 474 ms: 6 candidates, 59435 repo-hours, 1136 releases; waiting 502 ms; commit 0 ms)
2026-11-08T08:54:28Z [load_day 2024-11-17] 2024-11-17 hour 23: 177242 events in 0 files, snapshot none (task 6537 ms: download 4170 ms overlapping gunzip 1039 ms, parse 4223 ms, sort 0 ms, write 0 ms, index 466 ms: 5 candidates, 57047 repo-hours, 722 releases; waiting 535 ms; commit 0 ms)
2026-11-08T08:54:36Z [load_day 2024-11-17] 2024-11-17: repo_hour merged from 24 files to 1 (1726312 rows, 7645 ms)
2026-11-08T08:54:39Z [load_day 2024-11-17] 2024-11-17: index committed in 10634 ms (24 + 1 + 24 files)
2026-11-08T08:54:39Z [load_day 2024-11-17] 2024-11-17: 4603240 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:54:39Z [load_day 2024-11-17] attempt 1 succeeded