orcaLog in

Runs · run 3127

load_day 2024-10-25, attempt 1 succeeded

Host
73ccef5e07c2:197848
Started
2026-11-08T08:30:30Z
Ended
2026-11-08T08:32:11Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
781 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
156.6 s, up to 2.9 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3127/load_day/2024-10-25/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T08:30:30Z [load_day 2024-10-25] attempt 1 started
2026-11-08T08:30:31Z [load_day 2024-10-25] 2024-10-25: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:30:38Z [load_day 2024-10-25] 2024-10-25 hour 0: 186816 events in 0 files, snapshot none (task 6794 ms: download 3382 ms overlapping gunzip 972 ms, parse 3927 ms, sort 0 ms, write 0 ms, index 639 ms: 9 candidates, 88179 repo-hours, 1175 releases; waiting 628 ms; commit 0 ms)
2026-11-08T08:30:38Z [load_day 2024-10-25] 2024-10-25 hour 1: 203420 events in 0 files, snapshot none (task 6595 ms: download 3091 ms overlapping gunzip 1012 ms, parse 4013 ms, sort 0 ms, write 0 ms, index 824 ms: 8 candidates, 107054 repo-hours, 902 releases; waiting 496 ms; commit 0 ms)
2026-11-08T08:30:45Z [load_day 2024-10-25] 2024-10-25 hour 2: 175410 events in 0 files, snapshot none (task 5813 ms: download 2554 ms overlapping gunzip 857 ms, parse 3373 ms, sort 0 ms, write 0 ms, index 788 ms: 4 candidates, 78424 repo-hours, 1179 releases; waiting 535 ms; commit 0 ms)
2026-11-08T08:30:45Z [load_day 2024-10-25] 2024-10-25 hour 3: 189528 events in 0 files, snapshot none (task 6138 ms: download 2719 ms overlapping gunzip 913 ms, parse 3646 ms, sort 0 ms, write 0 ms, index 628 ms: 2 candidates, 76878 repo-hours, 4174 releases; waiting 515 ms; commit 0 ms)
2026-11-08T08:30:50Z [load_day 2024-10-25] 2024-10-25 hour 4: 170344 events in 0 files, snapshot none (task 4519 ms: download 2877 ms overlapping gunzip 671 ms, parse 2477 ms, sort 0 ms, write 0 ms, index 453 ms: 8 candidates, 65171 repo-hours, 610 releases; waiting 592 ms; commit 0 ms)
2026-11-08T08:30:50Z [load_day 2024-10-25] 2024-10-25 hour 5: 185962 events in 0 files, snapshot none (task 4946 ms: download 2881 ms overlapping gunzip 758 ms, parse 2769 ms, sort 0 ms, write 0 ms, index 526 ms: 5 candidates, 70808 repo-hours, 833 releases; waiting 595 ms; commit 0 ms)
2026-11-08T08:30:56Z [load_day 2024-10-25] 2024-10-25 hour 6: 219812 events in 0 files, snapshot none (task 5856 ms: download 3517 ms overlapping gunzip 910 ms, parse 3545 ms, sort 0 ms, write 0 ms, index 635 ms: 12 candidates, 96469 repo-hours, 1022 releases; waiting 502 ms; commit 0 ms)
2026-11-08T08:30:56Z [load_day 2024-10-25] 2024-10-25 hour 7: 225344 events in 0 files, snapshot none (task 5761 ms: download 3579 ms overlapping gunzip 997 ms, parse 3560 ms, sort 0 ms, write 0 ms, index 609 ms: 4 candidates, 87549 repo-hours, 1038 releases; waiting 342 ms; commit 0 ms)
2026-11-08T08:31:03Z [load_day 2024-10-25] 2024-10-25 hour 8: 238303 events in 0 files, snapshot none (task 6578 ms: download 4304 ms overlapping gunzip 1070 ms, parse 4033 ms, sort 0 ms, write 0 ms, index 654 ms: 11 candidates, 96489 repo-hours, 1174 releases; waiting 537 ms; commit 0 ms)
2026-11-08T08:31:04Z [load_day 2024-10-25] 2024-10-25 hour 9: 246492 events in 0 files, snapshot none (task 6838 ms: download 4584 ms overlapping gunzip 1150 ms, parse 4213 ms, sort 0 ms, write 0 ms, index 716 ms: 14 candidates, 99760 repo-hours, 1463 releases; waiting 538 ms; commit 0 ms)
2026-11-08T08:31:09Z [load_day 2024-10-25] 2024-10-25 hour 10: 225481 events in 0 files, snapshot none (task 5733 ms: download 3494 ms overlapping gunzip 970 ms, parse 3556 ms, sort 0 ms, write 0 ms, index 633 ms: 5 candidates, 89681 repo-hours, 1085 releases; waiting 361 ms; commit 0 ms)
2026-11-08T08:31:10Z [load_day 2024-10-25] 2024-10-25 hour 11: 239686 events in 0 files, snapshot none (task 6012 ms: download 3968 ms overlapping gunzip 1015 ms, parse 3676 ms, sort 0 ms, write 0 ms, index 644 ms: 15 candidates, 89141 repo-hours, 1208 releases; waiting 419 ms; commit 0 ms)
2026-11-08T08:31:18Z [load_day 2024-10-25] 2024-10-25 hour 12: 275306 events in 0 files, snapshot none (task 8236 ms: download 5128 ms overlapping gunzip 1251 ms, parse 5071 ms, sort 0 ms, write 0 ms, index 1257 ms: 16 candidates, 135882 repo-hours, 1310 releases; waiting 376 ms; commit 0 ms)
2026-11-08T08:31:20Z [load_day 2024-10-25] 2024-10-25 hour 13: 269976 events in 0 files, snapshot none (task 8930 ms: download 5784 ms overlapping gunzip 1480 ms, parse 5588 ms, sort 0 ms, write 0 ms, index 1090 ms: 14 candidates, 119994 repo-hours, 1351 releases; waiting 472 ms; commit 0 ms)
2026-11-08T08:31:28Z [load_day 2024-10-25] 2024-10-25 hour 14: 264308 events in 0 files, snapshot none (task 9142 ms: download 6574 ms overlapping gunzip 1618 ms, parse 5844 ms, sort 0 ms, write 0 ms, index 773 ms: 10 candidates, 103731 repo-hours, 1487 releases; waiting 594 ms; commit 0 ms)
2026-11-08T08:31:29Z [load_day 2024-10-25] 2024-10-25 hour 15: 260548 events in 0 files, snapshot none (task 8707 ms: download 6093 ms overlapping gunzip 1427 ms, parse 5561 ms, sort 0 ms, write 0 ms, index 968 ms: 9 candidates, 105841 repo-hours, 1392 releases; waiting 512 ms; commit 0 ms)
2026-11-08T08:31:36Z [load_day 2024-10-25] 2024-10-25 hour 16: 243151 events in 0 files, snapshot none (task 8034 ms: download 5276 ms overlapping gunzip 1223 ms, parse 4698 ms, sort 0 ms, write 0 ms, index 962 ms: 5 candidates, 95121 repo-hours, 1086 releases; waiting 869 ms; commit 0 ms)
2026-11-08T08:31:37Z [load_day 2024-10-25] 2024-10-25 hour 17: 229619 events in 0 files, snapshot none (task 6925 ms: download 3738 ms overlapping gunzip 1137 ms, parse 4194 ms, sort 0 ms, write 0 ms, index 1179 ms: 4 candidates, 86511 repo-hours, 1057 releases; waiting 155 ms; commit 0 ms)
2026-11-08T08:31:46Z [load_day 2024-10-25] 2024-10-25 hour 18: 238919 events in 0 files, snapshot none (task 9726 ms: download 7280 ms overlapping gunzip 1095 ms, parse 4536 ms, sort 0 ms, write 0 ms, index 823 ms: 9 candidates, 100425 repo-hours, 1170 releases; waiting 3041 ms; commit 0 ms)
2026-11-08T08:31:46Z [load_day 2024-10-25] 2024-10-25 hour 19: 219266 events in 0 files, snapshot none (task 6986 ms: download 4260 ms overlapping gunzip 1111 ms, parse 4349 ms, sort 0 ms, write 0 ms, index 714 ms: 7 candidates, 76655 repo-hours, 1121 releases; waiting 511 ms; commit 0 ms)
2026-11-08T08:31:54Z [load_day 2024-10-25] 2024-10-25 hour 20: 203769 events in 0 files, snapshot none (task 6922 ms: download 4018 ms overlapping gunzip 1117 ms, parse 4209 ms, sort 0 ms, write 0 ms, index 753 ms: 11 candidates, 74803 repo-hours, 931 releases; waiting 522 ms; commit 0 ms)
2026-11-08T08:31:54Z [load_day 2024-10-25] 2024-10-25 hour 21: 196049 events in 0 files, snapshot none (task 6123 ms: download 3594 ms overlapping gunzip 944 ms, parse 3576 ms, sort 0 ms, write 0 ms, index 877 ms: 10 candidates, 72304 repo-hours, 1013 releases; waiting 486 ms; commit 0 ms)
2026-11-08T08:32:00Z [load_day 2024-10-25] 2024-10-25 hour 22: 176409 events in 0 files, snapshot none (task 5250 ms: download 3379 ms overlapping gunzip 771 ms, parse 2903 ms, sort 0 ms, write 0 ms, index 755 ms: 16 candidates, 60653 repo-hours, 1169 releases; waiting 516 ms; commit 0 ms)
2026-11-08T08:32:00Z [load_day 2024-10-25] 2024-10-25 hour 23: 165844 events in 0 files, snapshot none (task 4632 ms: download 2262 ms overlapping gunzip 707 ms, parse 2870 ms, sort 0 ms, write 0 ms, index 630 ms: 6 candidates, 59837 repo-hours, 968 releases; waiting 213 ms; commit 0 ms)
2026-11-08T08:32:09Z [load_day 2024-10-25] 2024-10-25: repo_hour merged from 24 files to 1 (2137360 rows, 9008 ms)
2026-11-08T08:32:11Z [load_day 2024-10-25] 2024-10-25: index committed in 11451 ms (24 + 1 + 24 files)
2026-11-08T08:32:11Z [load_day 2024-10-25] 2024-10-25: 5249762 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:32:11Z [load_day 2024-10-25] attempt 1 succeeded