orcaLog in

Runs · run 804

load_day 2023-02-09, attempt 1 succeeded

Host
caf251265124:36078
Started
2026-11-07T15:17:46Z
Ended
2026-11-07T15:21:11Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/804/load_day/2023-02-09/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T15:17:46Z [load_day 2023-02-09] attempt 1 started
2026-11-07T15:17:47Z [load_day 2023-02-09] 2023-02-09: 2 hours in flight, about 1200 MB
2026-11-07T15:18:03Z [load_day 2023-02-09] 2023-02-09 hour 0: 175602 events in 4 files, snapshot 7688493830193138241 (task 13632 ms: download 8137 ms overlapping gunzip 856 ms, parse 3678 ms, sort 98 ms, write 7589 ms, index 540 ms: 6 candidates, 80078 repo-hours, 851 releases; waiting 519 ms; commit 1452 ms)
2026-11-07T15:18:05Z [load_day 2023-02-09] 2023-02-09 hour 1: 169915 events in 4 files, snapshot 5315353417623873100 (task 12451 ms: download 5229 ms overlapping gunzip 737 ms, parse 3284 ms, sort 90 ms, write 7015 ms, index 530 ms: 10 candidates, 82535 repo-hours, 834 releases; waiting 509 ms; commit 1614 ms)
2026-11-07T15:18:16Z [load_day 2023-02-09] 2023-02-09 hour 2: 152479 events in 4 files, snapshot 3724415205764454087 (task 12127 ms: download 5476 ms overlapping gunzip 707 ms, parse 3045 ms, sort 104 ms, write 6533 ms, index 455 ms: 4 candidates, 71564 repo-hours, 771 releases; waiting 905 ms; commit 1452 ms)
2026-11-07T15:18:18Z [load_day 2023-02-09] 2023-02-09 hour 3: 141760 events in 3 files, snapshot 2710000188060741498 (task 10665 ms: download 4714 ms overlapping gunzip 622 ms, parse 2722 ms, sort 83 ms, write 5957 ms, index 394 ms: 2 candidates, 61596 repo-hours, 782 releases; waiting 520 ms; commit 1476 ms)
2026-11-07T15:18:31Z [load_day 2023-02-09] 2023-02-09 hour 4: 136026 events in 3 files, snapshot 8114035338610373869 (task 11005 ms: download 4941 ms overlapping gunzip 676 ms, parse 2844 ms, sort 79 ms, write 5991 ms, index 382 ms: 2 candidates, 56988 repo-hours, 572 releases; waiting 656 ms; commit 4390 ms)
2026-11-07T15:18:32Z [load_day 2023-02-09] 2023-02-09 hour 5: 141707 events in 3 files, snapshot 5602406387331105901 (task 10851 ms: download 4613 ms overlapping gunzip 621 ms, parse 2742 ms, sort 97 ms, write 6064 ms, index 434 ms: 5 candidates, 60185 repo-hours, 798 releases; waiting 581 ms; commit 1462 ms)
2026-11-07T15:18:41Z [load_day 2023-02-09] 2023-02-09 hour 6: 157919 events in 4 files, snapshot 1578004438039877686 (task 11897 ms: download 4913 ms overlapping gunzip 695 ms, parse 3063 ms, sort 98 ms, write 6703 ms, index 483 ms: 5 candidates, 71449 repo-hours, 974 releases; waiting 517 ms; commit 1447 ms)
2026-11-07T15:18:46Z [load_day 2023-02-09] 2023-02-09 hour 7: 165207 events in 4 files, snapshot 6236860903532185264 (task 12879 ms: download 5268 ms overlapping gunzip 760 ms, parse 3366 ms, sort 106 ms, write 7330 ms, index 509 ms: 5 candidates, 72550 repo-hours, 776 releases; waiting 510 ms; commit 1469 ms)
2026-11-07T15:18:56Z [load_day 2023-02-09] 2023-02-09 hour 8: 181132 events in 4 files, snapshot 2325412501663948671 (task 14807 ms: download 8481 ms overlapping gunzip 909 ms, parse 3899 ms, sort 118 ms, write 8437 ms, index 543 ms: 6 candidates, 82286 repo-hours, 976 releases; waiting 577 ms; commit 1348 ms)
2026-11-07T15:19:04Z [load_day 2023-02-09] 2023-02-09 hour 9: 193738 events in 4 files, snapshot 1109037247055050648 (task 16870 ms: download 9681 ms overlapping gunzip 1038 ms, parse 4509 ms, sort 138 ms, write 9533 ms, index 557 ms: 7 candidates, 84843 repo-hours, 1172 releases; waiting 712 ms; commit 1391 ms)
2026-11-07T15:19:13Z [load_day 2023-02-09] 2023-02-09 hour 10: 189538 events in 4 files, snapshot 9076940204354772939 (task 16126 ms: download 9116 ms overlapping gunzip 988 ms, parse 4344 ms, sort 120 ms, write 9234 ms, index 554 ms: 5 candidates, 83094 repo-hours, 1041 releases; waiting 523 ms; commit 1316 ms)
2026-11-07T15:19:20Z [load_day 2023-02-09] 2023-02-09 hour 11: 189185 events in 4 files, snapshot 7349742355258719332 (task 15774 ms: download 8990 ms overlapping gunzip 930 ms, parse 4093 ms, sort 112 ms, write 9122 ms, index 546 ms: 4 candidates, 80714 repo-hours, 1049 releases; waiting 643 ms; commit 1563 ms)
2026-11-07T15:19:30Z [load_day 2023-02-09] 2023-02-09 hour 12: 217051 events in 5 files, snapshot 7568496521226747633 (task 16054 ms: download 8298 ms overlapping gunzip 936 ms, parse 4162 ms, sort 124 ms, write 9206 ms, index 689 ms: 4 candidates, 105851 repo-hours, 957 releases; waiting 529 ms; commit 1564 ms)
2026-11-07T15:19:38Z [load_day 2023-02-09] 2023-02-09 hour 13: 220059 events in 5 files, snapshot 7923158056301732261 (task 16687 ms: download 10707 ms overlapping gunzip 962 ms, parse 4320 ms, sort 128 ms, write 9743 ms, index 707 ms: 5 candidates, 109843 repo-hours, 1039 releases; waiting 510 ms; commit 1363 ms)
2026-11-07T15:19:48Z [load_day 2023-02-09] 2023-02-09 hour 14: 213050 events in 5 files, snapshot 7625064573771989503 (task 17464 ms: download 9270 ms overlapping gunzip 1017 ms, parse 4554 ms, sort 120 ms, write 10039 ms, index 634 ms: 4 candidates, 94658 repo-hours, 1130 releases; waiting 706 ms; commit 1533 ms)
2026-11-07T15:19:56Z [load_day 2023-02-09] 2023-02-09 hour 15: 216033 events in 5 files, snapshot 4044194733367062510 (task 17559 ms: download 11728 ms overlapping gunzip 1046 ms, parse 4689 ms, sort 128 ms, write 10275 ms, index 617 ms: 2 candidates, 96874 repo-hours, 1260 releases; waiting 504 ms; commit 1248 ms)
2026-11-07T15:20:06Z [load_day 2023-02-09] 2023-02-09 hour 16: 206635 events in 5 files, snapshot 6180015249627276848 (task 17067 ms: download 9193 ms overlapping gunzip 1001 ms, parse 4338 ms, sort 121 ms, write 9852 ms, index 729 ms: 8 candidates, 94954 repo-hours, 991 releases; waiting 755 ms; commit 1431 ms)
2026-11-07T15:20:14Z [load_day 2023-02-09] 2023-02-09 hour 17: 204177 events in 5 files, snapshot 1303040604286327611 (task 17720 ms: download 9893 ms overlapping gunzip 1082 ms, parse 4757 ms, sort 139 ms, write 10305 ms, index 590 ms: 2 candidates, 86227 repo-hours, 1005 releases; waiting 490 ms; commit 1349 ms)
2026-11-07T15:20:23Z [load_day 2023-02-09] 2023-02-09 hour 18: 204943 events in 5 files, snapshot 6980667036860955493 (task 16881 ms: download 9554 ms overlapping gunzip 1019 ms, parse 4399 ms, sort 127 ms, write 9627 ms, index 600 ms: 2 candidates, 89268 repo-hours, 1084 releases; waiting 774 ms; commit 1376 ms)
2026-11-07T15:20:31Z [load_day 2023-02-09] 2023-02-09 hour 19: 194877 events in 4 files, snapshot 3935377387286530929 (task 15761 ms: download 8744 ms overlapping gunzip 974 ms, parse 4192 ms, sort 133 ms, write 9087 ms, index 537 ms: 4 candidates, 80245 repo-hours, 1001 releases; waiting 530 ms; commit 1376 ms)
2026-11-07T15:20:40Z [load_day 2023-02-09] 2023-02-09 hour 20: 194146 events in 4 files, snapshot 3230079290987356756 (task 15946 ms: download 8872 ms overlapping gunzip 1045 ms, parse 4470 ms, sort 123 ms, write 8876 ms, index 536 ms: 5 candidates, 79518 repo-hours, 859 releases; waiting 552 ms; commit 1371 ms)
2026-11-07T15:20:49Z [load_day 2023-02-09] 2023-02-09 hour 21: 189373 events in 4 files, snapshot 3487349048086302258 (task 17213 ms: download 10754 ms overlapping gunzip 948 ms, parse 4015 ms, sort 114 ms, write 8584 ms, index 518 ms: 3 candidates, 78180 repo-hours, 1688 releases; waiting 2714 ms; commit 1471 ms)
2026-11-07T15:20:55Z [load_day 2023-02-09] 2023-02-09 hour 22: 171306 events in 4 files, snapshot 2757630652226238663 (task 14578 ms: download 8542 ms overlapping gunzip 936 ms, parse 3892 ms, sort 103 ms, write 8092 ms, index 479 ms: 4 candidates, 67061 repo-hours, 887 releases; waiting 548 ms; commit 1413 ms)
2026-11-07T15:21:02Z [load_day 2023-02-09] 2023-02-09 hour 23: 153003 events in 4 files, snapshot 1970857666762401558 (task 11975 ms: download 5272 ms overlapping gunzip 718 ms, parse 3164 ms, sort 100 ms, write 6766 ms, index 412 ms: 0 candidates, 61670 repo-hours, 748 releases; waiting 519 ms; commit 1426 ms)
2026-11-07T15:21:10Z [load_day 2023-02-09] 2023-02-09: repo_hour merged from 24 files to 1 (1932231 rows, 7975 ms)
2026-11-07T15:21:11Z [load_day 2023-02-09] 2023-02-09: index committed in 9523 ms (23 + 1 + 24 files)
2026-11-07T15:21:11Z [load_day 2023-02-09] 2023-02-09: 4378861 events in 24 hours
2026-11-07T15:21:11Z [load_day 2023-02-09] attempt 1 succeeded