orcaLog in

Runs · run 3253

load_day 2024-11-28, attempt 1 succeeded

Host
73ccef5e07c2:213893
Started
2026-11-08T09:05:28Z
Ended
2026-11-08T09:07:48Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
715 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
222.3 s, up to 2.7 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3253/load_day/2024-11-28/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T09:05:28Z [load_day 2024-11-28] attempt 1 started
2026-11-08T09:05:29Z [load_day 2024-11-28] 2024-11-28: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T09:05:38Z [load_day 2024-11-28] 2024-11-28 hour 0: 190577 events in 0 files, snapshot none (task 8463 ms: download 5420 ms overlapping gunzip 1292 ms, parse 5697 ms, sort 0 ms, write 0 ms, index 689 ms: 4 candidates, 88764 repo-hours, 1148 releases; waiting 485 ms; commit 0 ms)
2026-11-08T09:05:38Z [load_day 2024-11-28] 2024-11-28 hour 1: 213009 events in 0 files, snapshot none (task 8375 ms: download 5513 ms overlapping gunzip 1346 ms, parse 5416 ms, sort 0 ms, write 0 ms, index 841 ms: 6 candidates, 114962 repo-hours, 823 releases; waiting 486 ms; commit 0 ms)
2026-11-08T09:05:45Z [load_day 2024-11-28] 2024-11-28 hour 2: 179914 events in 0 files, snapshot none (task 6666 ms: download 3464 ms overlapping gunzip 1014 ms, parse 4227 ms, sort 0 ms, write 0 ms, index 899 ms: 2 candidates, 77993 repo-hours, 820 releases; waiting 243 ms; commit 0 ms)
2026-11-08T09:05:49Z [load_day 2024-11-28] 2024-11-28 hour 3: 214165 events in 0 files, snapshot none (task 9807 ms: download 6631 ms overlapping gunzip 1612 ms, parse 6257 ms, sort 0 ms, write 0 ms, index 1013 ms: 7 candidates, 78553 repo-hours, 3922 releases; waiting 567 ms; commit 0 ms)
2026-11-08T09:05:55Z [load_day 2024-11-28] 2024-11-28 hour 4: 195831 events in 0 files, snapshot none (task 8712 ms: download 5927 ms overlapping gunzip 1454 ms, parse 5819 ms, sort 0 ms, write 0 ms, index 637 ms: 5 candidates, 65332 repo-hours, 603 releases; waiting 470 ms; commit 0 ms)
2026-11-08T09:05:58Z [load_day 2024-11-28] 2024-11-28 hour 5: 203057 events in 0 files, snapshot none (task 8657 ms: download 5288 ms overlapping gunzip 1441 ms, parse 5315 ms, sort 0 ms, write 0 ms, index 1090 ms: 3 candidates, 70299 repo-hours, 790 releases; waiting 473 ms; commit 0 ms)
2026-11-08T09:06:05Z [load_day 2024-11-28] 2024-11-28 hour 6: 227777 events in 0 files, snapshot none (task 10109 ms: download 7712 ms overlapping gunzip 1690 ms, parse 6894 ms, sort 0 ms, write 0 ms, index 719 ms: 7 candidates, 93518 repo-hours, 944 releases; waiting 509 ms; commit 0 ms)
2026-11-08T09:06:08Z [load_day 2024-11-28] 2024-11-28 hour 7: 228008 events in 0 files, snapshot none (task 10094 ms: download 6847 ms overlapping gunzip 1630 ms, parse 6255 ms, sort 0 ms, write 0 ms, index 1412 ms: 4 candidates, 83691 repo-hours, 985 releases; waiting 496 ms; commit 0 ms)
2026-11-08T09:06:17Z [load_day 2024-11-28] 2024-11-28 hour 8: 242508 events in 0 files, snapshot none (task 11474 ms: download 8847 ms overlapping gunzip 2027 ms, parse 7791 ms, sort 0 ms, write 0 ms, index 859 ms: 7 candidates, 92747 repo-hours, 1018 releases; waiting 490 ms; commit 0 ms)
2026-11-08T09:06:21Z [load_day 2024-11-28] 2024-11-28 hour 9: 258498 events in 0 files, snapshot none (task 11799 ms: download 8232 ms overlapping gunzip 2020 ms, parse 7709 ms, sort 0 ms, write 0 ms, index 1228 ms: 9 candidates, 100946 repo-hours, 1313 releases; waiting 516 ms; commit 0 ms)
2026-11-08T09:06:28Z [load_day 2024-11-28] 2024-11-28 hour 10: 248577 events in 0 files, snapshot none (task 9900 ms: download 7516 ms overlapping gunzip 1727 ms, parse 6950 ms, sort 0 ms, write 0 ms, index 725 ms: 7 candidates, 93027 repo-hours, 1141 releases; waiting 210 ms; commit 0 ms)
2026-11-08T09:06:31Z [load_day 2024-11-28] 2024-11-28 hour 11: 245408 events in 0 files, snapshot none (task 9667 ms: download 7059 ms overlapping gunzip 1679 ms, parse 6124 ms, sort 0 ms, write 0 ms, index 754 ms: 4 candidates, 89363 repo-hours, 1191 releases; waiting 783 ms; commit 0 ms)
2026-11-08T09:06:37Z [load_day 2024-11-28] 2024-11-28 hour 12: 272432 events in 0 files, snapshot none (task 8325 ms: download 5151 ms overlapping gunzip 1221 ms, parse 5277 ms, sort 0 ms, write 0 ms, index 1001 ms: 11 candidates, 135601 repo-hours, 2007 releases; waiting 514 ms; commit 0 ms)
2026-11-08T09:06:41Z [load_day 2024-11-28] 2024-11-28 hour 13: 277186 events in 0 files, snapshot none (task 9261 ms: download 6395 ms overlapping gunzip 1585 ms, parse 6121 ms, sort 0 ms, write 0 ms, index 958 ms: 7 candidates, 115844 repo-hours, 1226 releases; waiting 209 ms; commit 0 ms)
2026-11-08T09:06:48Z [load_day 2024-11-28] 2024-11-28 hour 14: 265588 events in 0 files, snapshot none (task 10710 ms: download 7893 ms overlapping gunzip 1809 ms, parse 7318 ms, sort 0 ms, write 0 ms, index 817 ms: 4 candidates, 101662 repo-hours, 1305 releases; waiting 504 ms; commit 0 ms)
2026-11-08T09:06:54Z [load_day 2024-11-28] 2024-11-28 hour 15: 267791 events in 0 files, snapshot none (task 12527 ms: download 9059 ms overlapping gunzip 2092 ms, parse 8301 ms, sort 0 ms, write 0 ms, index 965 ms: 5 candidates, 105221 repo-hours, 1364 releases; waiting 746 ms; commit 0 ms)
2026-11-08T09:07:00Z [load_day 2024-11-28] 2024-11-28 hour 16: 253028 events in 0 files, snapshot none (task 11500 ms: download 7975 ms overlapping gunzip 1877 ms, parse 7522 ms, sort 0 ms, write 0 ms, index 1275 ms: 3 candidates, 95722 repo-hours, 970 releases; waiting 456 ms; commit 0 ms)
2026-11-08T09:07:06Z [load_day 2024-11-28] 2024-11-28 hour 17: 237531 events in 0 files, snapshot none (task 11201 ms: download 7270 ms overlapping gunzip 1838 ms, parse 7242 ms, sort 0 ms, write 0 ms, index 1331 ms: 6 candidates, 85604 repo-hours, 894 releases; waiting 473 ms; commit 0 ms)
2026-11-08T09:07:11Z [load_day 2024-11-28] 2024-11-28 hour 18: 240267 events in 0 files, snapshot none (task 10408 ms: download 6577 ms overlapping gunzip 1677 ms, parse 6976 ms, sort 0 ms, write 0 ms, index 1272 ms: 7 candidates, 95091 repo-hours, 1017 releases; waiting 231 ms; commit 0 ms)
2026-11-08T09:07:17Z [load_day 2024-11-28] 2024-11-28 hour 19: 213844 events in 0 files, snapshot none (task 10063 ms: download 6616 ms overlapping gunzip 1801 ms, parse 6811 ms, sort 0 ms, write 0 ms, index 913 ms: 6 candidates, 72277 repo-hours, 949 releases; waiting 240 ms; commit 0 ms)
2026-11-08T09:07:21Z [load_day 2024-11-28] 2024-11-28 hour 20: 206379 events in 0 files, snapshot none (task 9892 ms: download 6629 ms overlapping gunzip 1750 ms, parse 6129 ms, sort 0 ms, write 0 ms, index 1259 ms: 12 candidates, 70505 repo-hours, 827 releases; waiting 509 ms; commit 0 ms)
2026-11-08T09:07:28Z [load_day 2024-11-28] 2024-11-28 hour 21: 203105 events in 0 files, snapshot none (task 10018 ms: download 6487 ms overlapping gunzip 1596 ms, parse 6739 ms, sort 0 ms, write 0 ms, index 814 ms: 6 candidates, 69581 repo-hours, 1037 releases; waiting 531 ms; commit 0 ms)
2026-11-08T09:07:30Z [load_day 2024-11-28] 2024-11-28 hour 22: 185215 events in 0 files, snapshot none (task 8681 ms: download 6165 ms overlapping gunzip 1515 ms, parse 5826 ms, sort 0 ms, write 0 ms, index 458 ms: 5 candidates, 59704 repo-hours, 986 releases; waiting 490 ms; commit 0 ms)
2026-11-08T09:07:36Z [load_day 2024-11-28] 2024-11-28 hour 23: 181708 events in 0 files, snapshot none (task 7032 ms: download 4480 ms overlapping gunzip 1140 ms, parse 4505 ms, sort 0 ms, write 0 ms, index 497 ms: 20 candidates, 57183 repo-hours, 709 releases; waiting 515 ms; commit 0 ms)
2026-11-08T09:07:45Z [load_day 2024-11-28] 2024-11-28: repo_hour merged from 24 files to 1 (2113190 rows, 9406 ms)
2026-11-08T09:07:48Z [load_day 2024-11-28] 2024-11-28: index committed in 11936 ms (24 + 1 + 24 files)
2026-11-08T09:07:48Z [load_day 2024-11-28] 2024-11-28: 5451403 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T09:07:48Z [load_day 2024-11-28] attempt 1 succeeded