orcaLog in

Runs · run 783

load_day 2023-02-02, attempt 1 succeeded

Host
caf251265124:9652
Started
2026-11-07T14:55:51Z
Ended
2026-11-07T14:59:33Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/783/load_day/2023-02-02/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T14:55:51Z [load_day 2023-02-02] attempt 1 started
2026-11-07T14:55:52Z [load_day 2023-02-02] 2023-02-02: 2 hours in flight, about 1200 MB
2026-11-07T14:56:09Z [load_day 2023-02-02] 2023-02-02 hour 0: 165572 events in 4 files, snapshot 2447724556276733121 (task 14496 ms: download 6656 ms overlapping gunzip 960 ms, parse 4031 ms, sort 119 ms, write 8028 ms, index 506 ms: 7 candidates, 78258 repo-hours, 906 releases; waiting 512 ms; commit 1193 ms)
2026-11-07T14:56:10Z [load_day 2023-02-02] 2023-02-02 hour 1: 157666 events in 4 files, snapshot 5743831548606233933 (task 12649 ms: download 5530 ms overlapping gunzip 768 ms, parse 3416 ms, sort 95 ms, write 6981 ms, index 517 ms: 8 candidates, 79741 repo-hours, 869 releases; waiting 495 ms; commit 1298 ms)
2026-11-07T14:56:21Z [load_day 2023-02-02] 2023-02-02 hour 2: 135274 events in 3 files, snapshot 2723657537424088803 (task 11412 ms: download 5343 ms overlapping gunzip 718 ms, parse 2881 ms, sort 87 ms, write 6386 ms, index 441 ms: 5 candidates, 65314 repo-hours, 660 releases; waiting 574 ms; commit 1503 ms)
2026-11-07T14:56:23Z [load_day 2023-02-02] 2023-02-02 hour 3: 138980 events in 3 files, snapshot 7667387240817921936 (task 11026 ms: download 4987 ms overlapping gunzip 688 ms, parse 2940 ms, sort 87 ms, write 6078 ms, index 396 ms: 4 candidates, 59040 repo-hours, 799 releases; waiting 539 ms; commit 1345 ms)
2026-11-07T14:56:34Z [load_day 2023-02-02] 2023-02-02 hour 4: 134415 events in 3 files, snapshot 7135208668190238626 (task 11665 ms: download 5476 ms overlapping gunzip 749 ms, parse 3046 ms, sort 81 ms, write 6413 ms, index 393 ms: 2 candidates, 55962 repo-hours, 630 releases; waiting 621 ms; commit 1469 ms)
2026-11-07T14:56:36Z [load_day 2023-02-02] 2023-02-02 hour 5: 150443 events in 4 files, snapshot 6604478586241828171 (task 13052 ms: download 5883 ms overlapping gunzip 877 ms, parse 3631 ms, sort 103 ms, write 7118 ms, index 466 ms: 4 candidates, 62349 repo-hours, 843 releases; waiting 518 ms; commit 1418 ms)
2026-11-07T14:56:53Z [load_day 2023-02-02] 2023-02-02 hour 6: 181725 events in 4 files, snapshot 4501071557930771086 (task 18825 ms: download 12228 ms overlapping gunzip 1198 ms, parse 4828 ms, sort 97 ms, write 9229 ms, index 594 ms: 3 candidates, 84282 repo-hours, 758 releases; waiting 2583 ms; commit 1410 ms)
2026-11-07T14:56:55Z [load_day 2023-02-02] 2023-02-02 hour 7: 183591 events in 4 files, snapshot 8456395815540426811 (task 18056 ms: download 11030 ms overlapping gunzip 1232 ms, parse 4844 ms, sort 124 ms, write 9893 ms, index 565 ms: 2 candidates, 79288 repo-hours, 752 releases; waiting 983 ms; commit 1427 ms)
2026-11-07T14:57:12Z [load_day 2023-02-02] 2023-02-02 hour 8: 193873 events in 4 files, snapshot 3770830695901168322 (task 18230 ms: download 10991 ms overlapping gunzip 1287 ms, parse 5168 ms, sort 125 ms, write 10232 ms, index 584 ms: 2 candidates, 87877 repo-hours, 987 releases; waiting 524 ms; commit 1496 ms)
2026-11-07T14:57:16Z [load_day 2023-02-02] 2023-02-02 hour 9: 211173 events in 5 files, snapshot 1359241083406060060 (task 20107 ms: download 13663 ms overlapping gunzip 1321 ms, parse 5508 ms, sort 136 ms, write 11241 ms, index 887 ms: 4 candidates, 95233 repo-hours, 1237 releases; waiting 524 ms; commit 1339 ms)
2026-11-07T14:57:32Z [load_day 2023-02-02] 2023-02-02 hour 10: 206185 events in 5 files, snapshot 1235779307315989174 (task 18629 ms: download 10792 ms overlapping gunzip 1263 ms, parse 5250 ms, sort 127 ms, write 10614 ms, index 554 ms: 4 candidates, 86865 repo-hours, 1066 releases; waiting 500 ms; commit 1339 ms)
2026-11-07T14:57:34Z [load_day 2023-02-02] 2023-02-02 hour 11: 194761 events in 4 files, snapshot 4432790410582672321 (task 17439 ms: download 10325 ms overlapping gunzip 1131 ms, parse 4855 ms, sort 115 ms, write 9909 ms, index 590 ms: 8 candidates, 85420 repo-hours, 1114 releases; waiting 542 ms; commit 1359 ms)
2026-11-07T14:57:53Z [load_day 2023-02-02] 2023-02-02 hour 12: 220431 events in 5 files, snapshot 754841692596396886 (task 20652 ms: download 12806 ms overlapping gunzip 1134 ms, parse 4865 ms, sort 120 ms, write 9903 ms, index 677 ms: 3 candidates, 109158 repo-hours, 1203 releases; waiting 3633 ms; commit 1418 ms)
2026-11-07T14:57:54Z [load_day 2023-02-02] 2023-02-02 hour 13: 216848 events in 5 files, snapshot 4463208148052136287 (task 18109 ms: download 12387 ms overlapping gunzip 1146 ms, parse 4772 ms, sort 133 ms, write 10430 ms, index 667 ms: 10 candidates, 105207 repo-hours, 1066 releases; waiting 651 ms; commit 1334 ms)
2026-11-07T14:58:12Z [load_day 2023-02-02] 2023-02-02 hour 14: 211933 events in 5 files, snapshot 4029534124424480327 (task 17910 ms: download 12268 ms overlapping gunzip 1101 ms, parse 4695 ms, sort 135 ms, write 10765 ms, index 633 ms: 10 candidates, 94380 repo-hours, 1334 releases; waiting 304 ms; commit 1527 ms)
2026-11-07T14:58:18Z [load_day 2023-02-02] 2023-02-02 hour 15: 217852 events in 5 files, snapshot 7099797062191083261 (task 22533 ms: download 16299 ms overlapping gunzip 1202 ms, parse 5269 ms, sort 140 ms, write 11082 ms, index 682 ms: 6 candidates, 99187 repo-hours, 1507 releases; waiting 3712 ms; commit 1461 ms)
2026-11-07T14:58:30Z [load_day 2023-02-02] 2023-02-02 hour 16: 207914 events in 5 files, snapshot 1311503191469197938 (task 17270 ms: download 9346 ms overlapping gunzip 1041 ms, parse 4524 ms, sort 147 ms, write 10289 ms, index 635 ms: 8 candidates, 100758 repo-hours, 1080 releases; waiting 283 ms; commit 1530 ms)
2026-11-07T14:58:35Z [load_day 2023-02-02] 2023-02-02 hour 17: 201320 events in 5 files, snapshot 7242215753339184538 (task 17026 ms: download 9674 ms overlapping gunzip 1078 ms, parse 4589 ms, sort 129 ms, write 10040 ms, index 595 ms: 9 candidates, 89047 repo-hours, 959 releases; waiting 283 ms; commit 1468 ms)
2026-11-07T14:58:47Z [load_day 2023-02-02] 2023-02-02 hour 18: 196239 events in 4 files, snapshot 8675860733606678241 (task 16666 ms: download 9271 ms overlapping gunzip 1016 ms, parse 4335 ms, sort 131 ms, write 9728 ms, index 564 ms: 6 candidates, 89334 repo-hours, 1033 releases; waiting 436 ms; commit 1420 ms)
2026-11-07T14:58:52Z [load_day 2023-02-02] 2023-02-02 hour 19: 187730 events in 4 files, snapshot 3259369243884732161 (task 16144 ms: download 9260 ms overlapping gunzip 1001 ms, parse 4242 ms, sort 115 ms, write 9437 ms, index 589 ms: 5 candidates, 79940 repo-hours, 968 releases; waiting 535 ms; commit 1286 ms)
2026-11-07T14:59:04Z [load_day 2023-02-02] 2023-02-02 hour 20: 188165 events in 4 files, snapshot 1876921273085464152 (task 16317 ms: download 9546 ms overlapping gunzip 984 ms, parse 4134 ms, sort 127 ms, write 9546 ms, index 596 ms: 4 candidates, 84503 repo-hours, 1087 releases; waiting 659 ms; commit 1616 ms)
2026-11-07T14:59:09Z [load_day 2023-02-02] 2023-02-02 hour 21: 182050 events in 4 files, snapshot 7024035096538661839 (task 15363 ms: download 8812 ms overlapping gunzip 1016 ms, parse 4099 ms, sort 129 ms, write 8992 ms, index 536 ms: 8 candidates, 80322 repo-hours, 984 releases; waiting 238 ms; commit 1451 ms)
2026-11-07T14:59:19Z [load_day 2023-02-02] 2023-02-02 hour 22: 162284 events in 4 files, snapshot 8542780358755454445 (task 13998 ms: download 8750 ms overlapping gunzip 890 ms, parse 3719 ms, sort 98 ms, write 8053 ms, index 435 ms: 3 candidates, 66535 repo-hours, 849 releases; waiting 477 ms; commit 1493 ms)
2026-11-07T14:59:22Z [load_day 2023-02-02] 2023-02-02 hour 23: 146363 events in 3 files, snapshot 3996697497213741057 (task 12796 ms: download 5743 ms overlapping gunzip 788 ms, parse 3296 ms, sort 95 ms, write 7331 ms, index 396 ms: 4 candidates, 61436 repo-hours, 796 releases; waiting 502 ms; commit 1417 ms)
2026-11-07T14:59:30Z [load_day 2023-02-02] 2023-02-02: repo_hour merged from 24 files to 1 (1979436 rows, 8376 ms)
2026-11-07T14:59:33Z [load_day 2023-02-02] 2023-02-02: index committed in 10946 ms (24 + 1 + 24 files)
2026-11-07T14:59:33Z [load_day 2023-02-02] 2023-02-02: 4392787 events in 24 hours
2026-11-07T14:59:33Z [load_day 2023-02-02] attempt 1 succeeded