orcaLog in

Runs · run 867

load_day 2023-03-02, attempt 1 succeeded

Host
caf251265124:113934
Started
2026-11-07T16:24:01Z
Ended
2026-11-07T16:27:47Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/867/load_day/2023-03-02/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T16:24:01Z [load_day 2023-03-02] attempt 1 started
2026-11-07T16:24:02Z [load_day 2023-03-02] 2023-03-02: 2 hours in flight, about 1200 MB
2026-11-07T16:24:18Z [load_day 2023-03-02] 2023-03-02 hour 0: 169347 events in 4 files, snapshot 1174057131568094519 (task 14018 ms: download 8756 ms overlapping gunzip 901 ms, parse 3844 ms, sort 106 ms, write 7708 ms, index 518 ms: 7 candidates, 76695 repo-hours, 1009 releases; waiting 582 ms; commit 1342 ms)
2026-11-07T16:24:20Z [load_day 2023-03-02] 2023-03-02 hour 1: 167750 events in 4 files, snapshot 9182278940007620181 (task 14418 ms: download 6892 ms overlapping gunzip 841 ms, parse 3548 ms, sort 99 ms, write 7242 ms, index 563 ms: 2 candidates, 84864 repo-hours, 1023 releases; waiting 1754 ms; commit 1305 ms)
2026-11-07T16:24:31Z [load_day 2023-03-02] 2023-03-02 hour 2: 149984 events in 3 files, snapshot 8600128649372872707 (task 12013 ms: download 5266 ms overlapping gunzip 723 ms, parse 3002 ms, sort 99 ms, write 6738 ms, index 491 ms: 3 candidates, 71176 repo-hours, 904 releases; waiting 604 ms; commit 1499 ms)
2026-11-07T16:24:33Z [load_day 2023-03-02] 2023-03-02 hour 3: 142960 events in 3 files, snapshot 2786360706491705243 (task 10863 ms: download 4670 ms overlapping gunzip 688 ms, parse 2899 ms, sort 88 ms, write 6264 ms, index 414 ms: 4 candidates, 61012 repo-hours, 856 releases; waiting 239 ms; commit 1337 ms)
2026-11-07T16:24:43Z [load_day 2023-03-02] 2023-03-02 hour 4: 133285 events in 3 files, snapshot 2708211908001932735 (task 10961 ms: download 5156 ms overlapping gunzip 689 ms, parse 2828 ms, sort 90 ms, write 5674 ms, index 352 ms: 0 candidates, 56415 repo-hours, 623 releases; waiting 988 ms; commit 1351 ms)
2026-11-07T16:24:46Z [load_day 2023-03-02] 2023-03-02 hour 5: 145840 events in 3 files, snapshot 4468391665726494260 (task 12258 ms: download 5879 ms overlapping gunzip 766 ms, parse 3236 ms, sort 85 ms, write 6371 ms, index 397 ms: 5 candidates, 61569 repo-hours, 764 releases; waiting 1099 ms; commit 1362 ms)
2026-11-07T16:24:58Z [load_day 2023-03-02] 2023-03-02 hour 6: 162530 events in 4 files, snapshot 914127379480965360 (task 14689 ms: download 7254 ms overlapping gunzip 797 ms, parse 3344 ms, sort 103 ms, write 7358 ms, index 498 ms: 5 candidates, 74068 repo-hours, 875 releases; waiting 2091 ms; commit 1334 ms)
2026-11-07T16:25:01Z [load_day 2023-03-02] 2023-03-02 hour 7: 170731 events in 4 files, snapshot 952857406714212794 (task 14375 ms: download 8523 ms overlapping gunzip 899 ms, parse 3784 ms, sort 106 ms, write 8219 ms, index 473 ms: 4 candidates, 75514 repo-hours, 847 releases; waiting 515 ms; commit 1485 ms)
2026-11-07T16:25:35Z [load_day 2023-03-02] 2023-03-02 hour 8: 191037 events in 4 files, snapshot 6972362335213789633 (task 20005 ms: download 13177 ms overlapping gunzip 1060 ms, parse 4460 ms, sort 125 ms, write 9101 ms, index 527 ms: 6 candidates, 84715 repo-hours, 1049 releases; waiting 4408 ms; commit 16729 ms)
2026-11-07T16:25:36Z [load_day 2023-03-02] 2023-03-02 hour 9: 199473 events in 4 files, snapshot 1216249407917830244 (task 18090 ms: download 10697 ms overlapping gunzip 1093 ms, parse 4712 ms, sort 132 ms, write 10084 ms, index 567 ms: 7 candidates, 87679 repo-hours, 1117 releases; waiting 1114 ms; commit 1294 ms)
2026-11-07T16:25:45Z [load_day 2023-03-02] 2023-03-02 hour 10: 194311 events in 4 files, snapshot 4218908835572110996 (task 16441 ms: download 9366 ms overlapping gunzip 1022 ms, parse 4460 ms, sort 121 ms, write 9506 ms, index 530 ms: 7 candidates, 86610 repo-hours, 1128 releases; waiting 507 ms; commit 1386 ms)
2026-11-07T16:25:55Z [load_day 2023-03-02] 2023-03-02 hour 11: 192116 events in 4 files, snapshot 8145989865141367505 (task 18153 ms: download 11011 ms overlapping gunzip 1023 ms, parse 4392 ms, sort 118 ms, write 9299 ms, index 547 ms: 4 candidates, 83566 repo-hours, 1051 releases; waiting 2376 ms; commit 1590 ms)
2026-11-07T16:26:04Z [load_day 2023-03-02] 2023-03-02 hour 12: 226323 events in 5 files, snapshot 5190620590439116095 (task 17870 ms: download 11801 ms overlapping gunzip 1061 ms, parse 4611 ms, sort 141 ms, write 10335 ms, index 660 ms: 8 candidates, 112382 repo-hours, 1091 releases; waiting 729 ms; commit 1420 ms)
2026-11-07T16:26:13Z [load_day 2023-03-02] 2023-03-02 hour 13: 222433 events in 5 files, snapshot 6155868584254827276 (task 17464 ms: download 11278 ms overlapping gunzip 1085 ms, parse 4739 ms, sort 138 ms, write 10357 ms, index 660 ms: 8 candidates, 108054 repo-hours, 1046 releases; waiting 193 ms; commit 1322 ms)
2026-11-07T16:26:24Z [load_day 2023-03-02] 2023-03-02 hour 14: 218210 events in 5 files, snapshot 8131494274765610639 (task 19539 ms: download 13424 ms overlapping gunzip 1148 ms, parse 4932 ms, sort 124 ms, write 10632 ms, index 632 ms: 8 candidates, 98070 repo-hours, 1264 releases; waiting 1719 ms; commit 1380 ms)
2026-11-07T16:26:33Z [load_day 2023-03-02] 2023-03-02 hour 15: 218099 events in 5 files, snapshot 3060391272617282065 (task 19149 ms: download 13204 ms overlapping gunzip 1182 ms, parse 5013 ms, sort 143 ms, write 10703 ms, index 676 ms: 6 candidates, 98159 repo-hours, 1341 releases; waiting 1082 ms; commit 1463 ms)
2026-11-07T16:26:43Z [load_day 2023-03-02] 2023-03-02 hour 16: 207722 events in 5 files, snapshot 8179745693292649704 (task 18229 ms: download 12653 ms overlapping gunzip 1101 ms, parse 4806 ms, sort 130 ms, write 10398 ms, index 620 ms: 9 candidates, 93956 repo-hours, 1049 releases; waiting 863 ms; commit 1276 ms)
2026-11-07T16:26:50Z [load_day 2023-03-02] 2023-03-02 hour 17: 199835 events in 4 files, snapshot 7443940470646806582 (task 16663 ms: download 9336 ms overlapping gunzip 1033 ms, parse 4447 ms, sort 124 ms, write 9621 ms, index 610 ms: 5 candidates, 87097 repo-hours, 967 releases; waiting 518 ms; commit 1435 ms)
2026-11-07T16:26:59Z [load_day 2023-03-02] 2023-03-02 hour 18: 189207 events in 4 files, snapshot 7342284316613432392 (task 15593 ms: download 8725 ms overlapping gunzip 925 ms, parse 4099 ms, sort 118 ms, write 8940 ms, index 594 ms: 14 candidates, 85820 repo-hours, 1007 releases; waiting 596 ms; commit 1268 ms)
2026-11-07T16:27:06Z [load_day 2023-03-02] 2023-03-02 hour 19: 188446 events in 4 files, snapshot 4115276000775267403 (task 15303 ms: download 8755 ms overlapping gunzip 928 ms, parse 4022 ms, sort 129 ms, write 8819 ms, index 525 ms: 4 candidates, 79562 repo-hours, 1063 releases; waiting 600 ms; commit 1379 ms)
2026-11-07T16:27:16Z [load_day 2023-03-02] 2023-03-02 hour 20: 185336 events in 4 files, snapshot 7445729864685235786 (task 16346 ms: download 10083 ms overlapping gunzip 955 ms, parse 4057 ms, sort 110 ms, write 8271 ms, index 551 ms: 10 candidates, 81034 repo-hours, 957 releases; waiting 2106 ms; commit 1324 ms)
2026-11-07T16:27:23Z [load_day 2023-03-02] 2023-03-02 hour 21: 190725 events in 4 files, snapshot 1291584934711817427 (task 15518 ms: download 8918 ms overlapping gunzip 983 ms, parse 4229 ms, sort 115 ms, write 8827 ms, index 525 ms: 4 candidates, 84391 repo-hours, 1005 releases; waiting 533 ms; commit 1313 ms)
2026-11-07T16:27:32Z [load_day 2023-03-02] 2023-03-02 hour 22: 177149 events in 4 files, snapshot 7193362019997023588 (task 14997 ms: download 8803 ms overlapping gunzip 958 ms, parse 4048 ms, sort 121 ms, write 8464 ms, index 495 ms: 3 candidates, 75657 repo-hours, 806 releases; waiting 525 ms; commit 1325 ms)
2026-11-07T16:27:37Z [load_day 2023-03-02] 2023-03-02 hour 23: 147237 events in 3 files, snapshot 2817144340926056662 (task 13601 ms: download 6654 ms overlapping gunzip 893 ms, parse 3602 ms, sort 87 ms, write 6980 ms, index 403 ms: 5 candidates, 64919 repo-hours, 541 releases; waiting 1302 ms; commit 1528 ms)
2026-11-07T16:27:45Z [load_day 2023-03-02] 2023-03-02: repo_hour merged from 24 files to 1 (1972984 rows, 8098 ms)
2026-11-07T16:27:47Z [load_day 2023-03-02] 2023-03-02: index committed in 9781 ms (23 + 1 + 24 files)
2026-11-07T16:27:47Z [load_day 2023-03-02] 2023-03-02: 4390086 events in 24 hours
2026-11-07T16:27:47Z [load_day 2023-03-02] attempt 1 succeeded