orcaLog in

Runs · run 1781

load_day 2023-11-26, attempt 1 succeeded

Host
3c2b29713ce8:599675
Started
2026-11-08T03:11:44Z
Ended
2026-11-08T03:14:38Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1781/load_day/2023-11-26/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:11:44Z [load_day 2023-11-26] attempt 1 started
2026-11-08T03:11:45Z [load_day 2023-11-26] 2023-11-26: 2 hours in flight, about 1200 MB
2026-11-08T03:12:00Z [load_day 2023-11-26] 2023-11-26 hour 0: 165720 events in 4 files, snapshot 7569066550133007909 (task 10833 ms: download 1331 ms overlapping gunzip 587 ms, parse 2974 ms, sort 141 ms, write 6403 ms, index 524 ms: 4 candidates, 67447 repo-hours, 922 releases; waiting 22 ms; commit 2769 ms)
2026-11-08T03:12:06Z [load_day 2023-11-26] 2023-11-26 hour 1: 181179 events in 4 files, snapshot 3666496377635432036 (task 11307 ms: download 4006 ms overlapping gunzip 612 ms, parse 3093 ms, sort 184 ms, write 6001 ms, index 699 ms: 12 candidates, 74566 repo-hours, 713 releases; waiting 496 ms; commit 5859 ms)
2026-11-08T03:12:12Z [load_day 2023-11-26] 2023-11-26 hour 2: 185556 events in 4 files, snapshot 5839918662373310386 (task 10042 ms: download 2479 ms overlapping gunzip 668 ms, parse 2988 ms, sort 122 ms, write 5630 ms, index 419 ms: 12 candidates, 41353 repo-hours, 638 releases; waiting 17 ms; commit 3018 ms)
2026-11-08T03:12:16Z [load_day 2023-11-26] 2023-11-26 hour 3: 207257 events in 5 files, snapshot 3993398340705716228 (task 11361 ms: download 4177 ms overlapping gunzip 750 ms, parse 3686 ms, sort 271 ms, write 6074 ms, index 327 ms: 7 candidates, 41946 repo-hours, 697 releases; waiting 29 ms; commit 3056 ms)
2026-11-08T03:12:23Z [load_day 2023-11-26] 2023-11-26 hour 4: 201490 events in 5 files, snapshot 8966310144306749450 (task 10136 ms: download 2937 ms overlapping gunzip 704 ms, parse 3432 ms, sort 218 ms, write 5109 ms, index 447 ms: 3 candidates, 42432 repo-hours, 549 releases; waiting 24 ms; commit 3017 ms)
2026-11-08T03:12:26Z [load_day 2023-11-26] 2023-11-26 hour 5: 135638 events in 3 files, snapshot 8677696334403158961 (task 7904 ms: download 1100 ms overlapping gunzip 511 ms, parse 2323 ms, sort 83 ms, write 4462 ms, index 312 ms: 2 candidates, 44431 repo-hours, 541 releases; waiting 18 ms; commit 2859 ms)
2026-11-08T03:12:35Z [load_day 2023-11-26] 2023-11-26 hour 6: 137593 events in 3 files, snapshot 5012361078528037195 (task 7829 ms: download 895 ms overlapping gunzip 434 ms, parse 1993 ms, sort 127 ms, write 4597 ms, index 478 ms: 6 candidates, 51391 repo-hours, 618 releases; waiting 15 ms; commit 5176 ms)
2026-11-08T03:12:39Z [load_day 2023-11-26] 2023-11-26 hour 7: 122643 events in 3 files, snapshot 3620535835570071247 (task 8699 ms: download 1184 ms overlapping gunzip 604 ms, parse 2364 ms, sort 87 ms, write 4991 ms, index 439 ms: 5 candidates, 45793 repo-hours, 591 releases; waiting 17 ms; commit 4719 ms)
2026-11-08T03:12:44Z [load_day 2023-11-26] 2023-11-26 hour 8: 133668 events in 3 files, snapshot 3449766888359483021 (task 8194 ms: download 1410 ms overlapping gunzip 644 ms, parse 2777 ms, sort 82 ms, write 4124 ms, index 318 ms: 7 candidates, 49936 repo-hours, 637 releases; waiting 25 ms; commit 4373 ms)
2026-11-08T03:12:46Z [load_day 2023-11-26] 2023-11-26 hour 9: 155149 events in 4 files, snapshot 3606779084629637209 (task 7829 ms: download 1033 ms overlapping gunzip 468 ms, parse 2277 ms, sort 96 ms, write 4445 ms, index 325 ms: 8 candidates, 51477 repo-hours, 689 releases; waiting 17 ms; commit 2555 ms)
2026-11-08T03:12:52Z [load_day 2023-11-26] 2023-11-26 hour 10: 175640 events in 4 files, snapshot 1946324773246769525 (task 8556 ms: download 2443 ms overlapping gunzip 510 ms, parse 2393 ms, sort 90 ms, write 4981 ms, index 371 ms: 5 candidates, 54594 repo-hours, 635 releases; waiting 17 ms; commit 2648 ms)
2026-11-08T03:12:57Z [load_day 2023-11-26] 2023-11-26 hour 11: 190427 events in 4 files, snapshot 4467299011344502942 (task 8859 ms: download 2758 ms overlapping gunzip 554 ms, parse 2590 ms, sort 177 ms, write 4936 ms, index 384 ms: 7 candidates, 54234 repo-hours, 725 releases; waiting 21 ms; commit 3338 ms)
2026-11-08T03:13:08Z [load_day 2023-11-26] 2023-11-26 hour 12: 242314 events in 5 files, snapshot 777417409024707803 (task 14953 ms: download 4775 ms overlapping gunzip 828 ms, parse 4112 ms, sort 249 ms, write 8741 ms, index 801 ms: 6 candidates, 86887 repo-hours, 722 releases; waiting 15 ms; commit 2920 ms)
2026-11-08T03:13:14Z [load_day 2023-11-26] 2023-11-26 hour 13: 250678 events in 6 files, snapshot 3237634391541267544 (task 15524 ms: download 7595 ms overlapping gunzip 974 ms, parse 4883 ms, sort 238 ms, write 8458 ms, index 743 ms: 10 candidates, 77127 repo-hours, 686 releases; waiting 25 ms; commit 3163 ms)
2026-11-08T03:13:23Z [load_day 2023-11-26] 2023-11-26 hour 14: 273643 events in 6 files, snapshot 9029061807340187438 (task 14359 ms: download 5505 ms overlapping gunzip 778 ms, parse 3864 ms, sort 223 ms, write 7907 ms, index 734 ms: 5 candidates, 59520 repo-hours, 613 releases; waiting 570 ms; commit 3028 ms)
2026-11-08T03:13:29Z [load_day 2023-11-26] 2023-11-26 hour 15: 246292 events in 5 files, snapshot 1423666560019139071 (task 14339 ms: download 6436 ms overlapping gunzip 875 ms, parse 3951 ms, sort 204 ms, write 8558 ms, index 533 ms: 5 candidates, 68102 repo-hours, 952 releases; waiting 29 ms; commit 2965 ms)
2026-11-08T03:13:39Z [load_day 2023-11-26] 2023-11-26 hour 16: 255796 events in 6 files, snapshot 8932046019702839190 (task 14652 ms: download 6633 ms overlapping gunzip 831 ms, parse 3951 ms, sort 206 ms, write 8328 ms, index 566 ms: 6 candidates, 65651 repo-hours, 832 releases; waiting 513 ms; commit 2948 ms)
2026-11-08T03:13:46Z [load_day 2023-11-26] 2023-11-26 hour 17: 248983 events in 5 files, snapshot 1226837023815240767 (task 14626 ms: download 6179 ms overlapping gunzip 930 ms, parse 4783 ms, sort 235 ms, write 7910 ms, index 549 ms: 8 candidates, 61960 repo-hours, 764 releases; waiting 25 ms; commit 3254 ms)
2026-11-08T03:13:54Z [load_day 2023-11-26] 2023-11-26 hour 18: 242581 events in 5 files, snapshot 669856340245327839 (task 13202 ms: download 6003 ms overlapping gunzip 835 ms, parse 4105 ms, sort 153 ms, write 7346 ms, index 551 ms: 2 candidates, 64894 repo-hours, 827 releases; waiting 19 ms; commit 3402 ms)
2026-11-08T03:14:02Z [load_day 2023-11-26] 2023-11-26 hour 19: 278292 events in 6 files, snapshot 2497959349972998106 (task 15827 ms: download 8324 ms overlapping gunzip 947 ms, parse 4517 ms, sort 314 ms, write 8824 ms, index 390 ms: 9 candidates, 54895 repo-hours, 747 releases; waiting 530 ms; commit 2818 ms)
2026-11-08T03:14:11Z [load_day 2023-11-26] 2023-11-26 hour 20: 291971 events in 6 files, snapshot 7741573861917940554 (task 14626 ms: download 7496 ms overlapping gunzip 865 ms, parse 4189 ms, sort 262 ms, write 8595 ms, index 496 ms: 8 candidates, 52800 repo-hours, 638 releases; waiting 20 ms; commit 4215 ms)
2026-11-08T03:14:16Z [load_day 2023-11-26] 2023-11-26 hour 21: 209164 events in 5 files, snapshot 3707951097403966237 (task 12315 ms: download 5299 ms overlapping gunzip 762 ms, parse 3700 ms, sort 195 ms, write 7043 ms, index 400 ms: 13 candidates, 53403 repo-hours, 813 releases; waiting 28 ms; commit 3355 ms)
2026-11-08T03:14:23Z [load_day 2023-11-26] 2023-11-26 hour 22: 255457 events in 6 files, snapshot 7278729859191990070 (task 13213 ms: download 5576 ms overlapping gunzip 817 ms, parse 3972 ms, sort 275 ms, write 7545 ms, index 384 ms: 5 candidates, 46806 repo-hours, 726 releases; waiting 24 ms; commit 2535 ms)
2026-11-08T03:14:28Z [load_day 2023-11-26] 2023-11-26 hour 23: 200557 events in 5 files, snapshot 2053677473769127288 (task 11160 ms: download 4409 ms overlapping gunzip 728 ms, parse 3567 ms, sort 211 ms, write 6056 ms, index 373 ms: 3 candidates, 49653 repo-hours, 670 releases; waiting 19 ms; commit 2663 ms)
2026-11-08T03:14:35Z [load_day 2023-11-26] 2023-11-26: repo_hour merged from 24 files to 1 (1361298 rows, 6920 ms)
2026-11-08T03:14:38Z [load_day 2023-11-26] 2023-11-26: index committed in 9590 ms (24 + 1 + 24 files)
2026-11-08T03:14:38Z [load_day 2023-11-26] 2023-11-26: 4987688 events in 24 hours
2026-11-08T03:14:38Z [load_day 2023-11-26] attempt 1 succeeded