orcaLog in

Runs · run 3163

load_day 2024-11-04, attempt 1 succeeded

Host
73ccef5e07c2:202280
Started
2026-11-08T08:38:41Z
Ended
2026-11-08T08:41:05Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
822 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
226.4 s, up to 2.7 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/3163/load_day/2024-11-04/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T08:38:41Z [load_day 2024-11-04] attempt 1 started
2026-11-08T08:38:41Z [load_day 2024-11-04] 2024-11-04: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T08:38:49Z [load_day 2024-11-04] 2024-11-04 hour 0: 189175 events in 0 files, snapshot none (task 6975 ms: download 4463 ms overlapping gunzip 1138 ms, parse 4636 ms, sort 0 ms, write 0 ms, index 634 ms: 6 candidates, 87320 repo-hours, 1237 releases; waiting 292 ms; commit 0 ms)
2026-11-08T08:38:49Z [load_day 2024-11-04] 2024-11-04 hour 1: 208458 events in 0 files, snapshot none (task 7803 ms: download 5116 ms overlapping gunzip 1229 ms, parse 4971 ms, sort 0 ms, write 0 ms, index 839 ms: 5 candidates, 108200 repo-hours, 918 releases; waiting 432 ms; commit 0 ms)
2026-11-08T08:38:57Z [load_day 2024-11-04] 2024-11-04 hour 2: 184229 events in 0 files, snapshot none (task 7579 ms: download 4465 ms overlapping gunzip 1131 ms, parse 4295 ms, sort 0 ms, write 0 ms, index 1019 ms: 5 candidates, 79116 repo-hours, 888 releases; waiting 530 ms; commit 0 ms)
2026-11-08T08:38:57Z [load_day 2024-11-04] 2024-11-04 hour 3: 207057 events in 0 files, snapshot none (task 7390 ms: download 4700 ms overlapping gunzip 1283 ms, parse 4919 ms, sort 0 ms, write 0 ms, index 642 ms: 7 candidates, 78595 repo-hours, 4044 releases; waiting 276 ms; commit 0 ms)
2026-11-08T08:39:04Z [load_day 2024-11-04] 2024-11-04 hour 4: 187874 events in 0 files, snapshot none (task 6922 ms: download 3741 ms overlapping gunzip 1130 ms, parse 4140 ms, sort 0 ms, write 0 ms, index 1146 ms: 10 candidates, 64796 repo-hours, 733 releases; waiting 224 ms; commit 0 ms)
2026-11-08T08:39:06Z [load_day 2024-11-04] 2024-11-04 hour 5: 186432 events in 0 files, snapshot none (task 8214 ms: download 5778 ms overlapping gunzip 1344 ms, parse 4877 ms, sort 0 ms, write 0 ms, index 577 ms: 9 candidates, 68129 repo-hours, 897 releases; waiting 930 ms; commit 0 ms)
2026-11-08T08:39:14Z [load_day 2024-11-04] 2024-11-04 hour 6: 211204 events in 0 files, snapshot none (task 8752 ms: download 5695 ms overlapping gunzip 1477 ms, parse 5575 ms, sort 0 ms, write 0 ms, index 836 ms: 7 candidates, 92091 repo-hours, 1044 releases; waiting 539 ms; commit 0 ms)
2026-11-08T08:39:15Z [load_day 2024-11-04] 2024-11-04 hour 7: 214491 events in 0 files, snapshot none (task 8903 ms: download 6563 ms overlapping gunzip 1526 ms, parse 5873 ms, sort 0 ms, write 0 ms, index 653 ms: 9 candidates, 81414 repo-hours, 876 releases; waiting 513 ms; commit 0 ms)
2026-11-08T08:39:25Z [load_day 2024-11-04] 2024-11-04 hour 8: 235191 events in 0 files, snapshot none (task 10418 ms: download 7894 ms overlapping gunzip 1593 ms, parse 5832 ms, sort 0 ms, write 0 ms, index 679 ms: 8 candidates, 91665 repo-hours, 1270 releases; waiting 1968 ms; commit 0 ms)
2026-11-08T08:39:25Z [load_day 2024-11-04] 2024-11-04 hour 9: 250816 events in 0 files, snapshot none (task 9304 ms: download 6809 ms overlapping gunzip 1627 ms, parse 6113 ms, sort 0 ms, write 0 ms, index 724 ms: 8 candidates, 102634 repo-hours, 1589 releases; waiting 547 ms; commit 0 ms)
2026-11-08T08:39:37Z [load_day 2024-11-04] 2024-11-04 hour 10: 244793 events in 0 files, snapshot none (task 11051 ms: download 8521 ms overlapping gunzip 1877 ms, parse 7793 ms, sort 0 ms, write 0 ms, index 777 ms: 11 candidates, 97950 repo-hours, 1290 releases; waiting 344 ms; commit 0 ms)
2026-11-08T08:39:37Z [load_day 2024-11-04] 2024-11-04 hour 11: 238760 events in 0 files, snapshot none (task 11133 ms: download 8492 ms overlapping gunzip 1970 ms, parse 7656 ms, sort 0 ms, write 0 ms, index 689 ms: 11 candidates, 92220 repo-hours, 1253 releases; waiting 503 ms; commit 0 ms)
2026-11-08T08:39:48Z [load_day 2024-11-04] 2024-11-04 hour 12: 276740 events in 0 files, snapshot none (task 11088 ms: download 7902 ms overlapping gunzip 1781 ms, parse 7453 ms, sort 0 ms, write 0 ms, index 1186 ms: 10 candidates, 137261 repo-hours, 1495 releases; waiting 371 ms; commit 0 ms)
2026-11-08T08:39:49Z [load_day 2024-11-04] 2024-11-04 hour 13: 273808 events in 0 files, snapshot none (task 12092 ms: download 9196 ms overlapping gunzip 1946 ms, parse 8185 ms, sort 0 ms, write 0 ms, index 1173 ms: 8 candidates, 122226 repo-hours, 1401 releases; waiting 556 ms; commit 0 ms)
2026-11-08T08:40:03Z [load_day 2024-11-04] 2024-11-04 hour 14: 267442 events in 0 files, snapshot none (task 12126 ms: download 8882 ms overlapping gunzip 2092 ms, parse 8502 ms, sort 0 ms, write 0 ms, index 948 ms: 11 candidates, 109017 repo-hours, 1350 releases; waiting 293 ms; commit 0 ms)
2026-11-08T08:40:05Z [load_day 2024-11-04] 2024-11-04 hour 15: 271583 events in 0 files, snapshot none (task 12446 ms: download 9597 ms overlapping gunzip 2201 ms, parse 8832 ms, sort 0 ms, write 0 ms, index 841 ms: 7 candidates, 113676 repo-hours, 1397 releases; waiting 278 ms; commit 0 ms)
2026-11-08T08:40:16Z [load_day 2024-11-04] 2024-11-04 hour 16: 259416 events in 0 files, snapshot none (task 12594 ms: download 9677 ms overlapping gunzip 2177 ms, parse 8697 ms, sort 0 ms, write 0 ms, index 810 ms: 13 candidates, 106054 repo-hours, 1361 releases; waiting 581 ms; commit 0 ms)
2026-11-08T08:40:17Z [load_day 2024-11-04] 2024-11-04 hour 17: 250018 events in 0 files, snapshot none (task 11632 ms: download 9163 ms overlapping gunzip 2018 ms, parse 7981 ms, sort 0 ms, write 0 ms, index 714 ms: 11 candidates, 97680 repo-hours, 1281 releases; waiting 564 ms; commit 0 ms)
2026-11-08T08:40:29Z [load_day 2024-11-04] 2024-11-04 hour 18: 252346 events in 0 files, snapshot none (task 12698 ms: download 9445 ms overlapping gunzip 2078 ms, parse 8209 ms, sort 0 ms, write 0 ms, index 1604 ms: 13 candidates, 109469 repo-hours, 1218 releases; waiting 519 ms; commit 0 ms)
2026-11-08T08:40:29Z [load_day 2024-11-04] 2024-11-04 hour 19: 238130 events in 0 files, snapshot none (task 10778 ms: download 8404 ms overlapping gunzip 1855 ms, parse 7409 ms, sort 0 ms, write 0 ms, index 703 ms: 10 candidates, 89625 repo-hours, 1125 releases; waiting 492 ms; commit 0 ms)
2026-11-08T08:40:41Z [load_day 2024-11-04] 2024-11-04 hour 20: 229121 events in 0 files, snapshot none (task 10588 ms: download 7781 ms overlapping gunzip 1779 ms, parse 6985 ms, sort 0 ms, write 0 ms, index 963 ms: 12 candidates, 87171 repo-hours, 1111 releases; waiting 517 ms; commit 0 ms)
2026-11-08T08:40:41Z [load_day 2024-11-04] 2024-11-04 hour 21: 224064 events in 0 files, snapshot none (task 10504 ms: download 7487 ms overlapping gunzip 1771 ms, parse 7083 ms, sort 0 ms, write 0 ms, index 852 ms: 9 candidates, 85684 repo-hours, 1544 releases; waiting 521 ms; commit 0 ms)
2026-11-08T08:40:52Z [load_day 2024-11-04] 2024-11-04 hour 22: 205761 events in 0 files, snapshot none (task 10604 ms: download 7887 ms overlapping gunzip 1853 ms, parse 7218 ms, sort 0 ms, write 0 ms, index 815 ms: 11 candidates, 75526 repo-hours, 1507 releases; waiting 496 ms; commit 0 ms)
2026-11-08T08:40:52Z [load_day 2024-11-04] 2024-11-04 hour 23: 197572 events in 0 files, snapshot none (task 9660 ms: download 7044 ms overlapping gunzip 1688 ms, parse 6827 ms, sort 0 ms, write 0 ms, index 667 ms: 12 candidates, 70876 repo-hours, 954 releases; waiting 202 ms; commit 0 ms)
2026-11-08T08:41:02Z [load_day 2024-11-04] 2024-11-04: repo_hour merged from 24 files to 1 (2248395 rows, 10225 ms)
2026-11-08T08:41:05Z [load_day 2024-11-04] 2024-11-04: index committed in 13147 ms (24 + 1 + 24 files)
2026-11-08T08:41:05Z [load_day 2024-11-04] 2024-11-04: 5504481 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T08:41:05Z [load_day 2024-11-04] attempt 1 succeeded