orcaLog in

Runs · run 780

load_day 2023-02-01, attempt 1 succeeded

Host
caf251265124:4574
Started
2026-11-07T14:51:54Z
Ended
2026-11-07T14:55:50Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/780/load_day/2023-02-01/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T14:51:54Z [load_day 2023-02-01] attempt 1 started
2026-11-07T14:51:56Z [load_day 2023-02-01] 2023-02-01: 2 hours in flight, about 1200 MB
2026-11-07T14:52:15Z [load_day 2023-02-01] 2023-02-01 hour 0: 171709 events in 4 files, snapshot 5678397747101694207 (task 16764 ms: download 11358 ms overlapping gunzip 1048 ms, parse 4292 ms, sort 116 ms, write 9722 ms, index 568 ms: 8 candidates, 74587 repo-hours, 955 releases; waiting 670 ms; commit 1474 ms)
2026-11-07T14:52:16Z [load_day 2023-02-01] 2023-02-01 hour 1: 176221 events in 4 files, snapshot 6427641449525407606 (task 14542 ms: download 8764 ms overlapping gunzip 913 ms, parse 3843 ms, sort 112 ms, write 8227 ms, index 560 ms: 7 candidates, 85403 repo-hours, 1033 releases; waiting 524 ms; commit 1281 ms)
2026-11-07T14:52:29Z [load_day 2023-02-01] 2023-02-01 hour 2: 146399 events in 3 files, snapshot 1996815243672261707 (task 13026 ms: download 6423 ms overlapping gunzip 852 ms, parse 3368 ms, sort 104 ms, write 7392 ms, index 452 ms: 6 candidates, 66615 repo-hours, 824 releases; waiting 529 ms; commit 1535 ms)
2026-11-07T14:52:31Z [load_day 2023-02-01] 2023-02-01 hour 3: 149482 events in 3 files, snapshot 9067647050779208239 (task 13727 ms: download 7201 ms overlapping gunzip 876 ms, parse 3546 ms, sort 99 ms, write 7390 ms, index 424 ms: 8 candidates, 61141 repo-hours, 783 releases; waiting 972 ms; commit 1356 ms)
2026-11-07T14:52:43Z [load_day 2023-02-01] 2023-02-01 hour 4: 148273 events in 3 files, snapshot 1151425239012279385 (task 13675 ms: download 7188 ms overlapping gunzip 890 ms, parse 3515 ms, sort 113 ms, write 7504 ms, index 381 ms: 3 candidates, 57470 repo-hours, 587 releases; waiting 974 ms; commit 1402 ms)
2026-11-07T14:52:45Z [load_day 2023-02-01] 2023-02-01 hour 5: 152192 events in 4 files, snapshot 6415146687213615708 (task 12410 ms: download 5771 ms overlapping gunzip 809 ms, parse 3288 ms, sort 119 ms, write 7004 ms, index 382 ms: 1 candidates, 62615 repo-hours, 674 releases; waiting 510 ms; commit 1366 ms)
2026-11-07T14:52:58Z [load_day 2023-02-01] 2023-02-01 hour 6: 170243 events in 4 files, snapshot 5140261666498192287 (task 13981 ms: download 8619 ms overlapping gunzip 936 ms, parse 3660 ms, sort 121 ms, write 8172 ms, index 511 ms: 4 candidates, 71937 repo-hours, 801 releases; waiting 235 ms; commit 1391 ms)
2026-11-07T14:53:00Z [load_day 2023-02-01] 2023-02-01 hour 7: 176115 events in 4 files, snapshot 4539827904771505749 (task 14940 ms: download 8940 ms overlapping gunzip 988 ms, parse 4056 ms, sort 118 ms, write 8644 ms, index 537 ms: 6 candidates, 71823 repo-hours, 839 releases; waiting 252 ms; commit 1240 ms)
2026-11-07T14:53:15Z [load_day 2023-02-01] 2023-02-01 hour 8: 189495 events in 4 files, snapshot 4501976403784562357 (task 16844 ms: download 10104 ms overlapping gunzip 1071 ms, parse 4393 ms, sort 124 ms, write 10126 ms, index 541 ms: 4 candidates, 81660 repo-hours, 991 releases; waiting 311 ms; commit 1255 ms)
2026-11-07T14:53:22Z [load_day 2023-02-01] 2023-02-01 hour 9: 202438 events in 5 files, snapshot 4569864393204333584 (task 20916 ms: download 13095 ms overlapping gunzip 1241 ms, parse 5134 ms, sort 153 ms, write 10765 ms, index 586 ms: 10 candidates, 85585 repo-hours, 1136 releases; waiting 2678 ms; commit 1139 ms)
2026-11-07T14:53:35Z [load_day 2023-02-01] 2023-02-01 hour 10: 200443 events in 5 files, snapshot 7744833617447217340 (task 18239 ms: download 10894 ms overlapping gunzip 1143 ms, parse 4699 ms, sort 137 ms, write 10659 ms, index 594 ms: 5 candidates, 84447 repo-hours, 1106 releases; waiting 667 ms; commit 1543 ms)
2026-11-07T14:53:40Z [load_day 2023-02-01] 2023-02-01 hour 11: 201633 events in 5 files, snapshot 727249366098456683 (task 17399 ms: download 10041 ms overlapping gunzip 1130 ms, parse 4749 ms, sort 144 ms, write 10360 ms, index 555 ms: 2 candidates, 87130 repo-hours, 1037 releases; waiting 233 ms; commit 1288 ms)
2026-11-07T14:53:54Z [load_day 2023-02-01] 2023-02-01 hour 12: 234761 events in 5 files, snapshot 6051266159038184340 (task 18344 ms: download 11869 ms overlapping gunzip 1158 ms, parse 4759 ms, sort 138 ms, write 10807 ms, index 730 ms: 6 candidates, 108728 repo-hours, 1040 releases; waiting 401 ms; commit 1576 ms)
2026-11-07T14:54:00Z [load_day 2023-02-01] 2023-02-01 hour 13: 228163 events in 5 files, snapshot 6605369075408191383 (task 19385 ms: download 12811 ms overlapping gunzip 1184 ms, parse 4985 ms, sort 140 ms, write 11398 ms, index 686 ms: 11 candidates, 107320 repo-hours, 1055 releases; waiting 624 ms; commit 1380 ms)
2026-11-07T14:54:16Z [load_day 2023-02-01] 2023-02-01 hour 14: 223090 events in 5 files, snapshot 2668755642654561603 (task 21761 ms: download 15582 ms overlapping gunzip 1321 ms, parse 5498 ms, sort 146 ms, write 11473 ms, index 661 ms: 7 candidates, 96539 repo-hours, 1173 releases; waiting 2437 ms; commit 1330 ms)
2026-11-07T14:54:24Z [load_day 2023-02-01] 2023-02-01 hour 15: 221081 events in 5 files, snapshot 3265812798614222852 (task 22859 ms: download 16487 ms overlapping gunzip 1331 ms, parse 5382 ms, sort 132 ms, write 11736 ms, index 689 ms: 8 candidates, 100840 repo-hours, 1296 releases; waiting 3020 ms; commit 1372 ms)
2026-11-07T14:54:36Z [load_day 2023-02-01] 2023-02-01 hour 16: 213859 events in 5 files, snapshot 4505949245772678834 (task 19127 ms: download 13453 ms overlapping gunzip 1255 ms, parse 5319 ms, sort 153 ms, write 11239 ms, index 672 ms: 10 candidates, 96211 repo-hours, 1139 releases; waiting 219 ms; commit 1462 ms)
2026-11-07T14:54:43Z [load_day 2023-02-01] 2023-02-01 hour 17: 210415 events in 5 files, snapshot 7530193697468508424 (task 18570 ms: download 13155 ms overlapping gunzip 1195 ms, parse 4979 ms, sort 135 ms, write 10830 ms, index 614 ms: 6 candidates, 89336 repo-hours, 1022 releases; waiting 508 ms; commit 1345 ms)
2026-11-07T14:54:56Z [load_day 2023-02-01] 2023-02-01 hour 18: 208918 events in 5 files, snapshot 2827790410544576196 (task 18788 ms: download 12957 ms overlapping gunzip 1203 ms, parse 4980 ms, sort 147 ms, write 11045 ms, index 633 ms: 8 candidates, 95945 repo-hours, 968 releases; waiting 473 ms; commit 1335 ms)
2026-11-07T14:55:06Z [load_day 2023-02-01] 2023-02-01 hour 19: 207960 events in 5 files, snapshot 1778563346552797062 (task 21854 ms: download 15837 ms overlapping gunzip 1509 ms, parse 5960 ms, sort 124 ms, write 12383 ms, index 602 ms: 6 candidates, 87500 repo-hours, 958 releases; waiting 870 ms; commit 1370 ms)
2026-11-07T14:55:20Z [load_day 2023-02-01] 2023-02-01 hour 20: 202608 events in 5 files, snapshot 3455025292570084283 (task 23639 ms: download 17933 ms overlapping gunzip 1470 ms, parse 5839 ms, sort 135 ms, write 11968 ms, index 658 ms: 9 candidates, 87170 repo-hours, 921 releases; waiting 3243 ms; commit 1387 ms)
2026-11-07T14:55:25Z [load_day 2023-02-01] 2023-02-01 hour 21: 188274 events in 4 files, snapshot 5082004705077823492 (task 18750 ms: download 11586 ms overlapping gunzip 1270 ms, parse 5043 ms, sort 134 ms, write 10383 ms, index 516 ms: 12 candidates, 76641 repo-hours, 942 releases; waiting 1044 ms; commit 1246 ms)
2026-11-07T14:55:36Z [load_day 2023-02-01] 2023-02-01 hour 22: 166729 events in 4 files, snapshot 2322675121908566123 (task 15955 ms: download 10155 ms overlapping gunzip 1087 ms, parse 4414 ms, sort 111 ms, write 9033 ms, index 478 ms: 7 candidates, 69297 repo-hours, 875 releases; waiting 537 ms; commit 1198 ms)
2026-11-07T14:55:40Z [load_day 2023-02-01] 2023-02-01 hour 23: 161027 events in 4 files, snapshot 3307560722879003964 (task 14578 ms: download 9362 ms overlapping gunzip 965 ms, parse 3882 ms, sort 107 ms, write 8431 ms, index 448 ms: 4 candidates, 66960 repo-hours, 864 releases; waiting 453 ms; commit 1427 ms)
2026-11-07T14:55:48Z [load_day 2023-02-01] 2023-02-01: repo_hour merged from 24 files to 1 (1972900 rows, 8046 ms)
2026-11-07T14:55:50Z [load_day 2023-02-01] 2023-02-01: index committed in 9713 ms (24 + 1 + 24 files)
2026-11-07T14:55:50Z [load_day 2023-02-01] 2023-02-01: 4551528 events in 24 hours
2026-11-07T14:55:50Z [load_day 2023-02-01] attempt 1 succeeded