orcaLog in

Runs · run 1767

load_day 2023-11-22, attempt 1 succeeded

Host
3c2b29713ce8:584394
Started
2026-11-08T03:03:49Z
Ended
2026-11-08T03:08:06Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1767/load_day/2023-11-22/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:03:49Z [load_day 2023-11-22] attempt 1 started
2026-11-08T03:03:51Z [load_day 2023-11-22] 2023-11-22: 2 hours in flight, about 1200 MB
2026-11-08T03:04:11Z [load_day 2023-11-22] 2023-11-22 hour 0: 176166 events in 4 files, snapshot 792206303080870453 (task 15736 ms: download 6861 ms overlapping gunzip 984 ms, parse 4629 ms, sort 170 ms, write 9119 ms, index 613 ms: 11 candidates, 83779 repo-hours, 957 releases; waiting 20 ms; commit 3519 ms)
2026-11-08T03:04:14Z [load_day 2023-11-22] 2023-11-22 hour 1: 184728 events in 4 files, snapshot 5988814704680350014 (task 16385 ms: download 9282 ms overlapping gunzip 950 ms, parse 4337 ms, sort 154 ms, write 9973 ms, index 761 ms: 8 candidates, 96014 repo-hours, 1015 releases; waiting 19 ms; commit 2888 ms)
2026-11-08T03:04:27Z [load_day 2023-11-22] 2023-11-22 hour 2: 157108 events in 4 files, snapshot 5979194277428319989 (task 14146 ms: download 5512 ms overlapping gunzip 935 ms, parse 4185 ms, sort 117 ms, write 8235 ms, index 470 ms: 8 candidates, 65796 repo-hours, 786 releases; waiting 17 ms; commit 3024 ms)
2026-11-08T03:04:31Z [load_day 2023-11-22] 2023-11-22 hour 3: 155274 events in 4 files, snapshot 1134424622941376982 (task 14826 ms: download 6182 ms overlapping gunzip 922 ms, parse 4046 ms, sort 103 ms, write 8911 ms, index 620 ms: 5 candidates, 63987 repo-hours, 895 releases; waiting 28 ms; commit 3406 ms)
2026-11-08T03:04:43Z [load_day 2023-11-22] 2023-11-22 hour 4: 146666 events in 3 files, snapshot 6171332086246290166 (task 13701 ms: download 5257 ms overlapping gunzip 858 ms, parse 3740 ms, sort 122 ms, write 7717 ms, index 438 ms: 7 candidates, 60213 repo-hours, 644 releases; waiting 568 ms; commit 4667 ms)
2026-11-08T03:04:47Z [load_day 2023-11-22] 2023-11-22 hour 5: 153275 events in 4 files, snapshot 1530751917938880879 (task 15271 ms: download 6159 ms overlapping gunzip 967 ms, parse 3940 ms, sort 110 ms, write 8978 ms, index 492 ms: 6 candidates, 64103 repo-hours, 770 releases; waiting 515 ms; commit 2986 ms)
2026-11-08T03:04:59Z [load_day 2023-11-22] 2023-11-22 hour 6: 171319 events in 4 files, snapshot 5628516924822308963 (task 16459 ms: download 9709 ms overlapping gunzip 972 ms, parse 4570 ms, sort 118 ms, write 9989 ms, index 593 ms: 5 candidates, 76168 repo-hours, 957 releases; waiting 26 ms; commit 3008 ms)
2026-11-08T03:05:08Z [load_day 2023-11-22] 2023-11-22 hour 7: 181653 events in 4 files, snapshot 6775103914693013173 (task 18225 ms: download 9676 ms overlapping gunzip 1059 ms, parse 4914 ms, sort 113 ms, write 11214 ms, index 699 ms: 6 candidates, 75029 repo-hours, 980 releases; waiting 22 ms; commit 4425 ms)
2026-11-08T03:05:21Z [load_day 2023-11-22] 2023-11-22 hour 8: 193968 events in 4 files, snapshot 8092472739623639385 (task 21490 ms: download 12125 ms overlapping gunzip 1166 ms, parse 5406 ms, sort 135 ms, write 13821 ms, index 711 ms: 7 candidates, 84935 repo-hours, 1132 releases; waiting 31 ms; commit 2715 ms)
2026-11-08T03:05:31Z [load_day 2023-11-22] 2023-11-22 hour 9: 205223 events in 5 files, snapshot 6678627778545846987 (task 23240 ms: download 12325 ms overlapping gunzip 1513 ms, parse 6543 ms, sort 197 ms, write 13606 ms, index 1164 ms: 5 candidates, 90728 repo-hours, 1612 releases; waiting 14 ms; commit 2892 ms)
2026-11-08T03:05:46Z [load_day 2023-11-22] 2023-11-22 hour 10: 202103 events in 5 files, snapshot 459214577410364163 (task 21506 ms: download 11926 ms overlapping gunzip 1221 ms, parse 5458 ms, sort 152 ms, write 13709 ms, index 750 ms: 9 candidates, 87127 repo-hours, 1260 releases; waiting 18 ms; commit 4435 ms)
2026-11-08T03:05:53Z [load_day 2023-11-22] 2023-11-22 hour 11: 198882 events in 4 files, snapshot 1909941470227764200 (task 21484 ms: download 11611 ms overlapping gunzip 1231 ms, parse 5566 ms, sort 150 ms, write 13654 ms, index 670 ms: 5 candidates, 82300 repo-hours, 983 releases; waiting 20 ms; commit 2769 ms)
2026-11-08T03:06:07Z [load_day 2023-11-22] 2023-11-22 hour 12: 250735 events in 6 files, snapshot 2002681671025356853 (task 21515 ms: download 14171 ms overlapping gunzip 1365 ms, parse 6419 ms, sort 180 ms, write 12610 ms, index 710 ms: 9 candidates, 113896 repo-hours, 1198 releases; waiting 25 ms; commit 2966 ms)
2026-11-08T03:06:14Z [load_day 2023-11-22] 2023-11-22 hour 13: 231084 events in 5 files, snapshot 2115047206846061794 (task 19615 ms: download 12505 ms overlapping gunzip 1127 ms, parse 5111 ms, sort 149 ms, write 12255 ms, index 756 ms: 10 candidates, 107976 repo-hours, 1072 releases; waiting 20 ms; commit 2579 ms)
2026-11-08T03:06:27Z [load_day 2023-11-22] 2023-11-22 hour 14: 221810 events in 5 files, snapshot 8345746515802407553 (task 19150 ms: download 11734 ms overlapping gunzip 1223 ms, parse 5412 ms, sort 141 ms, write 11272 ms, index 887 ms: 5 candidates, 96381 repo-hours, 1234 releases; waiting 17 ms; commit 2704 ms)
2026-11-08T03:06:36Z [load_day 2023-11-22] 2023-11-22 hour 15: 225725 events in 5 files, snapshot 520773557258937038 (task 21381 ms: download 13123 ms overlapping gunzip 1361 ms, parse 6070 ms, sort 159 ms, write 12768 ms, index 807 ms: 10 candidates, 98046 repo-hours, 1420 releases; waiting 18 ms; commit 2761 ms)
2026-11-08T03:06:50Z [load_day 2023-11-22] 2023-11-22 hour 16: 217460 events in 5 files, snapshot 3709391925759056829 (task 21700 ms: download 14309 ms overlapping gunzip 1233 ms, parse 5469 ms, sort 153 ms, write 13869 ms, index 749 ms: 7 candidates, 93984 repo-hours, 1319 releases; waiting 19 ms; commit 2864 ms)
2026-11-08T03:07:00Z [load_day 2023-11-22] 2023-11-22 hour 17: 207147 events in 5 files, snapshot 7745970190043954399 (task 21726 ms: download 11612 ms overlapping gunzip 1345 ms, parse 6149 ms, sort 259 ms, write 12368 ms, index 851 ms: 11 candidates, 86212 repo-hours, 1080 releases; waiting 493 ms; commit 3364 ms)
2026-11-08T03:07:14Z [load_day 2023-11-22] 2023-11-22 hour 18: 202519 events in 5 files, snapshot 6696265714937921789 (task 20194 ms: download 10170 ms overlapping gunzip 1038 ms, parse 4700 ms, sort 150 ms, write 13459 ms, index 623 ms: 13 candidates, 85546 repo-hours, 1076 releases; waiting 23 ms; commit 5280 ms)
2026-11-08T03:07:21Z [load_day 2023-11-22] 2023-11-22 hour 19: 185819 events in 4 files, snapshot 3093694939239095871 (task 20203 ms: download 11096 ms overlapping gunzip 1202 ms, parse 5535 ms, sort 214 ms, write 11852 ms, index 703 ms: 13 candidates, 75044 repo-hours, 1045 releases; waiting 460 ms; commit 3005 ms)
2026-11-08T03:07:32Z [load_day 2023-11-22] 2023-11-22 hour 20: 182397 events in 4 files, snapshot 3143779155938918882 (task 19063 ms: download 10433 ms overlapping gunzip 1297 ms, parse 5751 ms, sort 153 ms, write 10888 ms, index 737 ms: 8 candidates, 74028 repo-hours, 1068 releases; waiting 26 ms; commit 3553 ms)
2026-11-08T03:07:39Z [load_day 2023-11-22] 2023-11-22 hour 21: 176256 events in 4 files, snapshot 7277282100522053549 (task 16662 ms: download 9727 ms overlapping gunzip 998 ms, parse 4173 ms, sort 110 ms, write 10660 ms, index 513 ms: 9 candidates, 67635 repo-hours, 1062 releases; waiting 22 ms; commit 3085 ms)
2026-11-08T03:07:48Z [load_day 2023-11-22] 2023-11-22 hour 22: 158755 events in 4 files, snapshot 808398636422045426 (task 15053 ms: download 5658 ms overlapping gunzip 846 ms, parse 3600 ms, sort 120 ms, write 9835 ms, index 406 ms: 8 candidates, 60190 repo-hours, 947 releases; waiting 20 ms; commit 2470 ms)
2026-11-08T03:07:53Z [load_day 2023-11-22] 2023-11-22 hour 23: 148394 events in 3 files, snapshot 4592445475490387384 (task 12514 ms: download 5463 ms overlapping gunzip 917 ms, parse 4054 ms, sort 116 ms, write 6777 ms, index 428 ms: 10 candidates, 55632 repo-hours, 731 releases; waiting 24 ms; commit 3198 ms)
2026-11-08T03:08:01Z [load_day 2023-11-22] 2023-11-22: repo_hour merged from 24 files to 1 (1944749 rows, 8393 ms)
2026-11-08T03:08:06Z [load_day 2023-11-22] 2023-11-22: index committed in 13521 ms (24 + 1 + 24 files)
2026-11-08T03:08:06Z [load_day 2023-11-22] 2023-11-22: 4534466 events in 24 hours
2026-11-08T03:08:06Z [load_day 2023-11-22] attempt 1 succeeded