orcaLog in

Runs · run 798

load_day 2023-02-07, attempt 1 succeeded

Host
caf251265124:27464
Started
2026-11-07T15:10:47Z
Ended
2026-11-07T15:14:12Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/798/load_day/2023-02-07/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T15:10:47Z [load_day 2023-02-07] attempt 1 started
2026-11-07T15:10:48Z [load_day 2023-02-07] 2023-02-07: 2 hours in flight, about 1200 MB
2026-11-07T15:11:04Z [load_day 2023-02-07] 2023-02-07 hour 0: 156812 events in 4 files, snapshot 4209671712748463026 (task 13271 ms: download 6006 ms overlapping gunzip 805 ms, parse 3507 ms, sort 90 ms, write 7467 ms, index 518 ms: 3 candidates, 76119 repo-hours, 980 releases; waiting 527 ms; commit 1427 ms)
2026-11-07T15:11:05Z [load_day 2023-02-07] 2023-02-07 hour 1: 153408 events in 4 files, snapshot 1614021023047268380 (task 12706 ms: download 5810 ms overlapping gunzip 714 ms, parse 3137 ms, sort 84 ms, write 6806 ms, index 510 ms: 5 candidates, 79733 repo-hours, 859 releases; waiting 1129 ms; commit 1399 ms)
2026-11-07T15:11:17Z [load_day 2023-02-07] 2023-02-07 hour 2: 144841 events in 3 files, snapshot 8906223481829351450 (task 11674 ms: download 5104 ms overlapping gunzip 699 ms, parse 2961 ms, sort 99 ms, write 6342 ms, index 572 ms: 9 candidates, 69348 repo-hours, 736 releases; waiting 703 ms; commit 1630 ms)
2026-11-07T15:11:18Z [load_day 2023-02-07] 2023-02-07 hour 3: 138754 events in 3 files, snapshot 2464609284710378887 (task 11689 ms: download 5427 ms overlapping gunzip 679 ms, parse 2892 ms, sort 82 ms, write 5886 ms, index 394 ms: 3 candidates, 59493 repo-hours, 817 releases; waiting 1162 ms; commit 1305 ms)
2026-11-07T15:11:28Z [load_day 2023-02-07] 2023-02-07 hour 4: 134521 events in 3 files, snapshot 2299792575925528616 (task 11056 ms: download 5403 ms overlapping gunzip 689 ms, parse 2899 ms, sort 81 ms, write 6007 ms, index 355 ms: 7 candidates, 55861 repo-hours, 611 releases; waiting 706 ms; commit 1262 ms)
2026-11-07T15:11:30Z [load_day 2023-02-07] 2023-02-07 hour 5: 140316 events in 3 files, snapshot 6065748661725738354 (task 10797 ms: download 4793 ms overlapping gunzip 659 ms, parse 2776 ms, sort 90 ms, write 6046 ms, index 391 ms: 7 candidates, 59834 repo-hours, 884 releases; waiting 513 ms; commit 1467 ms)
2026-11-07T15:11:41Z [load_day 2023-02-07] 2023-02-07 hour 6: 155948 events in 4 files, snapshot 5797613289472799605 (task 11956 ms: download 5178 ms overlapping gunzip 739 ms, parse 3127 ms, sort 100 ms, write 6581 ms, index 480 ms: 6 candidates, 71572 repo-hours, 1056 releases; waiting 569 ms; commit 1310 ms)
2026-11-07T15:11:44Z [load_day 2023-02-07] 2023-02-07 hour 7: 163765 events in 4 files, snapshot 9036794843426724145 (task 13008 ms: download 5464 ms overlapping gunzip 798 ms, parse 3394 ms, sort 97 ms, write 7337 ms, index 471 ms: 4 candidates, 72668 repo-hours, 949 releases; waiting 502 ms; commit 1405 ms)
2026-11-07T15:11:57Z [load_day 2023-02-07] 2023-02-07 hour 8: 183160 events in 4 files, snapshot 2888507322140734892 (task 15263 ms: download 8967 ms overlapping gunzip 970 ms, parse 4129 ms, sort 113 ms, write 8687 ms, index 509 ms: 6 candidates, 81458 repo-hours, 876 releases; waiting 526 ms; commit 1317 ms)
2026-11-07T15:12:01Z [load_day 2023-02-07] 2023-02-07 hour 9: 196714 events in 4 files, snapshot 4850837838849504674 (task 16473 ms: download 9382 ms overlapping gunzip 1022 ms, parse 4475 ms, sort 122 ms, write 9395 ms, index 630 ms: 6 candidates, 86790 repo-hours, 1148 releases; waiting 556 ms; commit 1239 ms)
2026-11-07T15:12:15Z [load_day 2023-02-07] 2023-02-07 hour 10: 195797 events in 4 files, snapshot 7891012430785770691 (task 16722 ms: download 9777 ms overlapping gunzip 1049 ms, parse 4495 ms, sort 123 ms, write 9687 ms, index 539 ms: 6 candidates, 84936 repo-hours, 1102 releases; waiting 515 ms; commit 1413 ms)
2026-11-07T15:12:18Z [load_day 2023-02-07] 2023-02-07 hour 11: 193999 events in 4 files, snapshot 7071011575487392428 (task 16018 ms: download 9342 ms overlapping gunzip 1024 ms, parse 4419 ms, sort 126 ms, write 9053 ms, index 548 ms: 1 candidates, 83469 repo-hours, 1106 releases; waiting 549 ms; commit 1199 ms)
2026-11-07T15:12:33Z [load_day 2023-02-07] 2023-02-07 hour 12: 222464 events in 5 files, snapshot 1881553288710323187 (task 17198 ms: download 9392 ms overlapping gunzip 1018 ms, parse 4472 ms, sort 125 ms, write 9585 ms, index 697 ms: 5 candidates, 108091 repo-hours, 1063 releases; waiting 938 ms; commit 1306 ms)
2026-11-07T15:12:37Z [load_day 2023-02-07] 2023-02-07 hour 13: 223149 events in 5 files, snapshot 2687981523219899935 (task 18511 ms: download 12552 ms overlapping gunzip 1176 ms, parse 4923 ms, sort 136 ms, write 10609 ms, index 705 ms: 6 candidates, 106405 repo-hours, 1031 releases; waiting 632 ms; commit 1389 ms)
2026-11-07T15:12:53Z [load_day 2023-02-07] 2023-02-07 hour 14: 223236 events in 5 files, snapshot 60195979336778846 (task 19040 ms: download 12672 ms overlapping gunzip 1158 ms, parse 5010 ms, sort 148 ms, write 10825 ms, index 689 ms: 4 candidates, 105831 repo-hours, 1102 releases; waiting 825 ms; commit 1513 ms)
2026-11-07T15:12:56Z [load_day 2023-02-07] 2023-02-07 hour 15: 221566 events in 5 files, snapshot 321100020474738338 (task 18524 ms: download 12531 ms overlapping gunzip 1175 ms, parse 5069 ms, sort 143 ms, write 10777 ms, index 620 ms: 5 candidates, 102717 repo-hours, 1313 releases; waiting 404 ms; commit 1317 ms)
2026-11-07T15:13:11Z [load_day 2023-02-07] 2023-02-07 hour 16: 206983 events in 5 files, snapshot 660612030732611441 (task 18018 ms: download 10325 ms overlapping gunzip 1111 ms, parse 4860 ms, sort 134 ms, write 10368 ms, index 686 ms: 6 candidates, 96163 repo-hours, 1085 releases; waiting 545 ms; commit 1272 ms)
2026-11-07T15:13:15Z [load_day 2023-02-07] 2023-02-07 hour 17: 212657 events in 5 files, snapshot 6220505072214817360 (task 17832 ms: download 9784 ms overlapping gunzip 1057 ms, parse 4513 ms, sort 137 ms, write 10409 ms, index 705 ms: 32 candidates, 98318 repo-hours, 1074 releases; waiting 683 ms; commit 1391 ms)
2026-11-07T15:13:28Z [load_day 2023-02-07] 2023-02-07 hour 18: 204092 events in 5 files, snapshot 5052498590097818129 (task 15979 ms: download 8545 ms overlapping gunzip 923 ms, parse 4090 ms, sort 127 ms, write 9299 ms, index 633 ms: 5 candidates, 96870 repo-hours, 1080 releases; waiting 566 ms; commit 1278 ms)
2026-11-07T15:13:32Z [load_day 2023-02-07] 2023-02-07 hour 19: 191884 events in 4 files, snapshot 8388350622033925059 (task 16036 ms: download 8997 ms overlapping gunzip 939 ms, parse 4062 ms, sort 130 ms, write 9341 ms, index 642 ms: 2 candidates, 84715 repo-hours, 1040 releases; waiting 581 ms; commit 1482 ms)
2026-11-07T15:13:44Z [load_day 2023-02-07] 2023-02-07 hour 20: 191488 events in 4 files, snapshot 4205774152950080508 (task 15368 ms: download 8786 ms overlapping gunzip 982 ms, parse 4304 ms, sort 119 ms, write 8694 ms, index 549 ms: 4 candidates, 80385 repo-hours, 967 releases; waiting 390 ms; commit 1361 ms)
2026-11-07T15:13:49Z [load_day 2023-02-07] 2023-02-07 hour 21: 184787 events in 4 files, snapshot 882675529824978758 (task 15829 ms: download 9023 ms overlapping gunzip 995 ms, parse 4208 ms, sort 108 ms, write 8918 ms, index 519 ms: 8 candidates, 74590 repo-hours, 1081 releases; waiting 624 ms; commit 1227 ms)
2026-11-07T15:13:59Z [load_day 2023-02-07] 2023-02-07 hour 22: 174405 events in 4 files, snapshot 6851050135275057455 (task 14450 ms: download 8689 ms overlapping gunzip 926 ms, parse 3871 ms, sort 107 ms, write 8392 ms, index 467 ms: 10 candidates, 69636 repo-hours, 949 releases; waiting 392 ms; commit 1198 ms)
2026-11-07T15:14:03Z [load_day 2023-02-07] 2023-02-07 hour 23: 155735 events in 4 files, snapshot 6594879000537369342 (task 13251 ms: download 5917 ms overlapping gunzip 836 ms, parse 3516 ms, sort 100 ms, write 7506 ms, index 439 ms: 2 candidates, 63040 repo-hours, 782 releases; waiting 544 ms; commit 1268 ms)
2026-11-07T15:14:11Z [load_day 2023-02-07] 2023-02-07: repo_hour merged from 24 files to 1 (1968042 rows, 7976 ms)
2026-11-07T15:14:12Z [load_day 2023-02-07] 2023-02-07: index committed in 9571 ms (24 + 1 + 24 files)
2026-11-07T15:14:12Z [load_day 2023-02-07] 2023-02-07: 4370481 events in 24 hours
2026-11-07T15:14:12Z [load_day 2023-02-07] attempt 1 succeeded