orcaLog in

Runs · run 2401

load_day 2024-05-02, attempt 1 succeeded

Host
73ccef5e07c2:117085
Started
2026-11-08T05:53:53Z
Ended
2026-11-08T05:56:01Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
700 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
202.2 s, up to 2.8 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/2401/load_day/2024-05-02/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T05:53:53Z [load_day 2024-05-02] attempt 1 started
2026-11-08T05:53:54Z [load_day 2024-05-02] 2024-05-02: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T05:54:04Z [load_day 2024-05-02] 2024-05-02 hour 0: 200414 events in 0 files, snapshot none (task 9386 ms: download 6320 ms overlapping gunzip 1419 ms, parse 6429 ms, sort 0 ms, write 0 ms, index 725 ms: 10 candidates, 94076 repo-hours, 1087 releases; waiting 532 ms; commit 0 ms)
2026-11-08T05:54:04Z [load_day 2024-05-02] 2024-05-02 hour 1: 196888 events in 0 files, snapshot none (task 8132 ms: download 4797 ms overlapping gunzip 1221 ms, parse 5294 ms, sort 0 ms, write 0 ms, index 845 ms: 6 candidates, 95915 repo-hours, 825 releases; waiting 516 ms; commit 0 ms)
2026-11-08T05:54:13Z [load_day 2024-05-02] 2024-05-02 hour 2: 171301 events in 0 files, snapshot none (task 9001 ms: download 5542 ms overlapping gunzip 1518 ms, parse 6178 ms, sort 0 ms, write 0 ms, index 519 ms: 6 candidates, 59645 repo-hours, 695 releases; waiting 521 ms; commit 0 ms)
2026-11-08T05:54:13Z [load_day 2024-05-02] 2024-05-02 hour 3: 177244 events in 0 files, snapshot none (task 9006 ms: download 5192 ms overlapping gunzip 1444 ms, parse 6253 ms, sort 0 ms, write 0 ms, index 535 ms: 10 candidates, 64955 repo-hours, 3003 releases; waiting 512 ms; commit 0 ms)
2026-11-08T05:54:22Z [load_day 2024-05-02] 2024-05-02 hour 4: 166980 events in 0 files, snapshot none (task 7701 ms: download 4795 ms overlapping gunzip 1111 ms, parse 5260 ms, sort 0 ms, write 0 ms, index 544 ms: 8 candidates, 60447 repo-hours, 852 releases; waiting 492 ms; commit 0 ms)
2026-11-08T05:54:22Z [load_day 2024-05-02] 2024-05-02 hour 5: 182506 events in 0 files, snapshot none (task 8275 ms: download 4926 ms overlapping gunzip 1317 ms, parse 5621 ms, sort 0 ms, write 0 ms, index 577 ms: 10 candidates, 66020 repo-hours, 850 releases; waiting 513 ms; commit 0 ms)
2026-11-08T05:54:30Z [load_day 2024-05-02] 2024-05-02 hour 6: 209547 events in 0 files, snapshot none (task 7889 ms: download 5252 ms overlapping gunzip 1126 ms, parse 5259 ms, sort 0 ms, write 0 ms, index 751 ms: 25 candidates, 82770 repo-hours, 926 releases; waiting 493 ms; commit 0 ms)
2026-11-08T05:54:31Z [load_day 2024-05-02] 2024-05-02 hour 7: 221949 events in 0 files, snapshot none (task 8811 ms: download 5786 ms overlapping gunzip 1337 ms, parse 5888 ms, sort 0 ms, write 0 ms, index 788 ms: 6 candidates, 76942 repo-hours, 1003 releases; waiting 513 ms; commit 0 ms)
2026-11-08T05:54:39Z [load_day 2024-05-02] 2024-05-02 hour 8: 237036 events in 0 files, snapshot none (task 8955 ms: download 6351 ms overlapping gunzip 1406 ms, parse 6399 ms, sort 0 ms, write 0 ms, index 608 ms: 8 candidates, 86316 repo-hours, 1128 releases; waiting 228 ms; commit 0 ms)
2026-11-08T05:54:42Z [load_day 2024-05-02] 2024-05-02 hour 9: 246930 events in 0 files, snapshot none (task 10505 ms: download 6890 ms overlapping gunzip 1522 ms, parse 6797 ms, sort 0 ms, write 0 ms, index 1617 ms: 11 candidates, 89704 repo-hours, 1251 releases; waiting 271 ms; commit 0 ms)
2026-11-08T05:54:49Z [load_day 2024-05-02] 2024-05-02 hour 10: 238176 events in 0 files, snapshot none (task 9728 ms: download 7021 ms overlapping gunzip 1447 ms, parse 6780 ms, sort 0 ms, write 0 ms, index 666 ms: 7 candidates, 88899 repo-hours, 1259 releases; waiting 600 ms; commit 0 ms)
2026-11-08T05:54:52Z [load_day 2024-05-02] 2024-05-02 hour 11: 245547 events in 0 files, snapshot none (task 9606 ms: download 6366 ms overlapping gunzip 1336 ms, parse 6386 ms, sort 0 ms, write 0 ms, index 979 ms: 4 candidates, 83844 repo-hours, 1079 releases; waiting 464 ms; commit 0 ms)
2026-11-08T05:55:00Z [load_day 2024-05-02] 2024-05-02 hour 12: 294915 events in 0 files, snapshot none (task 10586 ms: download 7566 ms overlapping gunzip 1515 ms, parse 7288 ms, sort 0 ms, write 0 ms, index 1017 ms: 17 candidates, 124797 repo-hours, 1074 releases; waiting 480 ms; commit 0 ms)
2026-11-08T05:55:04Z [load_day 2024-05-02] 2024-05-02 hour 13: 274844 events in 0 files, snapshot none (task 10991 ms: download 7519 ms overlapping gunzip 1668 ms, parse 7357 ms, sort 0 ms, write 0 ms, index 1202 ms: 8 candidates, 113536 repo-hours, 1363 releases; waiting 472 ms; commit 0 ms)
2026-11-08T05:55:10Z [load_day 2024-05-02] 2024-05-02 hour 14: 263265 events in 0 files, snapshot none (task 9145 ms: download 6608 ms overlapping gunzip 1478 ms, parse 6235 ms, sort 0 ms, write 0 ms, index 705 ms: 14 candidates, 101975 repo-hours, 1330 releases; waiting 491 ms; commit 0 ms)
2026-11-08T05:55:13Z [load_day 2024-05-02] 2024-05-02 hour 15: 259999 events in 0 files, snapshot none (task 8731 ms: download 5984 ms overlapping gunzip 1348 ms, parse 5873 ms, sort 0 ms, write 0 ms, index 866 ms: 7 candidates, 104452 repo-hours, 1326 releases; waiting 352 ms; commit 0 ms)
2026-11-08T05:55:19Z [load_day 2024-05-02] 2024-05-02 hour 16: 247964 events in 0 files, snapshot none (task 8010 ms: download 5459 ms overlapping gunzip 1251 ms, parse 5242 ms, sort 0 ms, write 0 ms, index 758 ms: 11 candidates, 95729 repo-hours, 1251 releases; waiting 507 ms; commit 0 ms)
2026-11-08T05:55:25Z [load_day 2024-05-02] 2024-05-02 hour 17: 258639 events in 0 files, snapshot none (task 11852 ms: download 8220 ms overlapping gunzip 1501 ms, parse 6442 ms, sort 0 ms, write 0 ms, index 915 ms: 8 candidates, 89151 repo-hours, 1147 releases; waiting 2695 ms; commit 0 ms)
2026-11-08T05:55:29Z [load_day 2024-05-02] 2024-05-02 hour 18: 255511 events in 0 files, snapshot none (task 9850 ms: download 7000 ms overlapping gunzip 1479 ms, parse 6411 ms, sort 0 ms, write 0 ms, index 855 ms: 9 candidates, 95915 repo-hours, 1162 releases; waiting 765 ms; commit 0 ms)
2026-11-08T05:55:35Z [load_day 2024-05-02] 2024-05-02 hour 19: 222098 events in 0 files, snapshot none (task 9158 ms: download 6279 ms overlapping gunzip 1429 ms, parse 6313 ms, sort 0 ms, write 0 ms, index 611 ms: 13 candidates, 81040 repo-hours, 1379 releases; waiting 505 ms; commit 0 ms)
2026-11-08T05:55:40Z [load_day 2024-05-02] 2024-05-02 hour 20: 211424 events in 0 files, snapshot none (task 9962 ms: download 5733 ms overlapping gunzip 1520 ms, parse 6748 ms, sort 0 ms, write 0 ms, index 941 ms: 10 candidates, 79029 repo-hours, 1174 releases; waiting 492 ms; commit 0 ms)
2026-11-08T05:55:43Z [load_day 2024-05-02] 2024-05-02 hour 21: 200983 events in 0 files, snapshot none (task 7814 ms: download 5056 ms overlapping gunzip 1229 ms, parse 5085 ms, sort 0 ms, write 0 ms, index 810 ms: 17 candidates, 73308 repo-hours, 1102 releases; waiting 347 ms; commit 0 ms)
2026-11-08T05:55:47Z [load_day 2024-05-02] 2024-05-02 hour 22: 181894 events in 0 files, snapshot none (task 7250 ms: download 4050 ms overlapping gunzip 1100 ms, parse 4677 ms, sort 0 ms, write 0 ms, index 700 ms: 8 candidates, 59866 repo-hours, 939 releases; waiting 528 ms; commit 0 ms)
2026-11-08T05:55:50Z [load_day 2024-05-02] 2024-05-02 hour 23: 186395 events in 0 files, snapshot none (task 5944 ms: download 3615 ms overlapping gunzip 844 ms, parse 3922 ms, sort 0 ms, write 0 ms, index 409 ms: 17 candidates, 56592 repo-hours, 744 releases; waiting 446 ms; commit 0 ms)
2026-11-08T05:55:58Z [load_day 2024-05-02] 2024-05-02: repo_hour merged from 24 files to 1 (2024923 rows, 8486 ms)
2026-11-08T05:56:01Z [load_day 2024-05-02] 2024-05-02: index committed in 11085 ms (24 + 1 + 24 files)
2026-11-08T05:56:01Z [load_day 2024-05-02] 2024-05-02: 5352449 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T05:56:01Z [load_day 2024-05-02] attempt 1 succeeded