orcaLog in

Runs · run 1694

load_day 2023-11-02, attempt 1 succeeded

Host
3c2b29713ce8:509068
Started
2026-11-08T02:24:08Z
Ended
2026-11-08T02:28:25Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1694/load_day/2023-11-02/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:24:08Z [load_day 2023-11-02] attempt 1 started
2026-11-08T02:24:10Z [load_day 2023-11-02] 2023-11-02: 2 hours in flight, about 1200 MB
2026-11-08T02:24:31Z [load_day 2023-11-02] 2023-11-02 hour 0: 166467 events in 4 files, snapshot 6340529711249889683 (task 15945 ms: download 6725 ms overlapping gunzip 1034 ms, parse 4801 ms, sort 139 ms, write 9180 ms, index 574 ms: 9 candidates, 88000 repo-hours, 995 releases; waiting 24 ms; commit 2635 ms)
2026-11-08T02:24:33Z [load_day 2023-11-02] 2023-11-02 hour 1: 160957 events in 4 files, snapshot 1806089564636053876 (task 15264 ms: download 5443 ms overlapping gunzip 810 ms, parse 3586 ms, sort 104 ms, write 9786 ms, index 763 ms: 5 candidates, 89790 repo-hours, 910 releases; waiting 28 ms; commit 2757 ms)
2026-11-08T02:24:46Z [load_day 2023-11-02] 2023-11-02 hour 2: 145544 events in 3 files, snapshot 8742473601735942798 (task 14410 ms: download 6740 ms overlapping gunzip 814 ms, parse 3510 ms, sort 110 ms, write 8748 ms, index 458 ms: 11 candidates, 63707 repo-hours, 872 releases; waiting 531 ms; commit 2769 ms)
2026-11-08T02:24:49Z [load_day 2023-11-02] 2023-11-02 hour 3: 138958 events in 3 files, snapshot 7616497110828476227 (task 12776 ms: download 5403 ms overlapping gunzip 809 ms, parse 3562 ms, sort 118 ms, write 7473 ms, index 582 ms: 3 candidates, 60934 repo-hours, 821 releases; waiting 17 ms; commit 2518 ms)
2026-11-08T02:25:00Z [load_day 2023-11-02] 2023-11-02 hour 4: 133745 events in 3 files, snapshot 4805666436311923672 (task 12814 ms: download 4985 ms overlapping gunzip 892 ms, parse 3772 ms, sort 99 ms, write 7352 ms, index 479 ms: 10 candidates, 56583 repo-hours, 651 releases; waiting 19 ms; commit 2785 ms)
2026-11-08T02:25:04Z [load_day 2023-11-02] 2023-11-02 hour 5: 140168 events in 3 files, snapshot 2462446180041063914 (task 14011 ms: download 5858 ms overlapping gunzip 804 ms, parse 3455 ms, sort 92 ms, write 8757 ms, index 693 ms: 6 candidates, 59965 repo-hours, 646 releases; waiting 23 ms; commit 3133 ms)
2026-11-08T02:25:18Z [load_day 2023-11-02] 2023-11-02 hour 6: 160440 events in 4 files, snapshot 5594286966424637831 (task 16057 ms: download 6481 ms overlapping gunzip 1008 ms, parse 4497 ms, sort 139 ms, write 8514 ms, index 1102 ms: 8 candidates, 73778 repo-hours, 807 releases; waiting 532 ms; commit 3033 ms)
2026-11-08T02:25:23Z [load_day 2023-11-02] 2023-11-02 hour 7: 167399 events in 4 files, snapshot 1179135300153786975 (task 16391 ms: download 8956 ms overlapping gunzip 993 ms, parse 4213 ms, sort 162 ms, write 10191 ms, index 618 ms: 6 candidates, 74318 repo-hours, 844 releases; waiting 19 ms; commit 3594 ms)
2026-11-08T02:25:39Z [load_day 2023-11-02] 2023-11-02 hour 8: 187139 events in 4 files, snapshot 5024687001990763663 (task 18758 ms: download 10884 ms overlapping gunzip 1308 ms, parse 6020 ms, sort 114 ms, write 10432 ms, index 681 ms: 7 candidates, 83581 repo-hours, 1013 releases; waiting 18 ms; commit 4485 ms)
2026-11-08T02:25:44Z [load_day 2023-11-02] 2023-11-02 hour 9: 193371 events in 4 files, snapshot 6527377873168020879 (task 20636 ms: download 10709 ms overlapping gunzip 1286 ms, parse 5927 ms, sort 126 ms, write 12416 ms, index 664 ms: 12 candidates, 86843 repo-hours, 1177 releases; waiting 17 ms; commit 3070 ms)
2026-11-08T02:26:02Z [load_day 2023-11-02] 2023-11-02 hour 10: 192793 events in 4 files, snapshot 5088136967631560023 (task 20432 ms: download 12059 ms overlapping gunzip 1272 ms, parse 5869 ms, sort 169 ms, write 12199 ms, index 686 ms: 11 candidates, 84915 repo-hours, 1222 releases; waiting 27 ms; commit 5236 ms)
2026-11-08T02:26:05Z [load_day 2023-11-02] 2023-11-02 hour 11: 188687 events in 4 files, snapshot 6048571852696997889 (task 19727 ms: download 10344 ms overlapping gunzip 1178 ms, parse 5222 ms, sort 199 ms, write 11908 ms, index 995 ms: 11 candidates, 81743 repo-hours, 1191 releases; waiting 17 ms; commit 2782 ms)
2026-11-08T02:26:21Z [load_day 2023-11-02] 2023-11-02 hour 12: 244762 events in 5 files, snapshot 5923752063503074779 (task 20952 ms: download 13926 ms overlapping gunzip 1292 ms, parse 6200 ms, sort 144 ms, write 12291 ms, index 792 ms: 7 candidates, 113500 repo-hours, 1173 releases; waiting 24 ms; commit 2642 ms)
2026-11-08T02:26:27Z [load_day 2023-11-02] 2023-11-02 hour 13: 228266 events in 5 files, snapshot 2240211486020069572 (task 20637 ms: download 13378 ms overlapping gunzip 1268 ms, parse 5890 ms, sort 144 ms, write 12452 ms, index 675 ms: 6 candidates, 110463 repo-hours, 1241 releases; waiting 20 ms; commit 2924 ms)
2026-11-08T02:26:45Z [load_day 2023-11-02] 2023-11-02 hour 14: 217992 events in 5 files, snapshot 100142214651608751 (task 22847 ms: download 14431 ms overlapping gunzip 1319 ms, parse 5694 ms, sort 151 ms, write 14587 ms, index 875 ms: 9 candidates, 99046 repo-hours, 1344 releases; waiting 25 ms; commit 2915 ms)
2026-11-08T02:26:52Z [load_day 2023-11-02] 2023-11-02 hour 15: 222096 events in 5 files, snapshot 1813194184028663552 (task 22730 ms: download 15062 ms overlapping gunzip 1291 ms, parse 6048 ms, sort 189 ms, write 14300 ms, index 706 ms: 15 candidates, 101583 repo-hours, 1408 releases; waiting 15 ms; commit 4077 ms)
2026-11-08T02:27:11Z [load_day 2023-11-02] 2023-11-02 hour 16: 211746 events in 5 files, snapshot 8741657993122140796 (task 23923 ms: download 16528 ms overlapping gunzip 1287 ms, parse 5892 ms, sort 184 ms, write 15329 ms, index 1020 ms: 8 candidates, 94938 repo-hours, 1219 releases; waiting 20 ms; commit 3034 ms)
2026-11-08T02:27:15Z [load_day 2023-11-02] 2023-11-02 hour 17: 200092 events in 5 files, snapshot 3336740273408278674 (task 22162 ms: download 12766 ms overlapping gunzip 1420 ms, parse 6501 ms, sort 191 ms, write 12928 ms, index 871 ms: 8 candidates, 87929 repo-hours, 1153 releases; waiting 17 ms; commit 3098 ms)
2026-11-08T02:27:32Z [load_day 2023-11-02] 2023-11-02 hour 18: 200298 events in 5 files, snapshot 473444538929753750 (task 20688 ms: download 11785 ms overlapping gunzip 1238 ms, parse 5708 ms, sort 198 ms, write 12218 ms, index 572 ms: 11 candidates, 89440 repo-hours, 1042 releases; waiting 494 ms; commit 2556 ms)
2026-11-08T02:27:39Z [load_day 2023-11-02] 2023-11-02 hour 19: 182250 events in 4 files, snapshot 723396412438963521 (task 19545 ms: download 11350 ms overlapping gunzip 989 ms, parse 4388 ms, sort 178 ms, write 13011 ms, index 734 ms: 10 candidates, 77141 repo-hours, 1081 releases; waiting 27 ms; commit 6290 ms)
2026-11-08T02:27:56Z [load_day 2023-11-02] 2023-11-02 hour 20: 178388 events in 4 files, snapshot 6517911750207283065 (task 20304 ms: download 12637 ms overlapping gunzip 1142 ms, parse 5712 ms, sort 229 ms, write 11790 ms, index 621 ms: 8 candidates, 76007 repo-hours, 1008 releases; waiting 516 ms; commit 4524 ms)
2026-11-08T02:27:59Z [load_day 2023-11-02] 2023-11-02 hour 21: 177601 events in 4 files, snapshot 2925755448738539253 (task 18920 ms: download 11717 ms overlapping gunzip 1345 ms, parse 5765 ms, sort 155 ms, write 10837 ms, index 582 ms: 6 candidates, 72077 repo-hours, 1116 releases; waiting 32 ms; commit 3120 ms)
2026-11-08T02:28:11Z [load_day 2023-11-02] 2023-11-02 hour 22: 155542 events in 4 files, snapshot 3673072420884045562 (task 16697 ms: download 6941 ms overlapping gunzip 1099 ms, parse 4956 ms, sort 149 ms, write 9707 ms, index 570 ms: 6 candidates, 63961 repo-hours, 979 releases; waiting 18 ms; commit 2369 ms)
2026-11-08T02:28:14Z [load_day 2023-11-02] 2023-11-02 hour 23: 136344 events in 3 files, snapshot 1789364741744222507 (task 13689 ms: download 6326 ms overlapping gunzip 803 ms, parse 3568 ms, sort 114 ms, write 8541 ms, index 450 ms: 8 candidates, 57083 repo-hours, 815 releases; waiting 28 ms; commit 2260 ms)
2026-11-08T02:28:23Z [load_day 2023-11-02] 2023-11-02: repo_hour merged from 24 files to 1 (1947325 rows, 9174 ms)
2026-11-08T02:28:25Z [load_day 2023-11-02] 2023-11-02: index committed in 11446 ms (24 + 1 + 24 files)
2026-11-08T02:28:25Z [load_day 2023-11-02] 2023-11-02: 4331045 events in 24 hours
2026-11-08T02:28:25Z [load_day 2023-11-02] attempt 1 succeeded