orcaLog in

Runs · run 1011

load_day 2023-04-19, attempt 1 succeeded

Host
caf251265124:283920
Started
2026-11-07T18:53:06Z
Ended
2026-11-07T18:56:21Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1011/load_day/2023-04-19/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T18:53:06Z [load_day 2023-04-19] attempt 1 started
2026-11-07T18:53:10Z [load_day 2023-04-19] 2023-04-19: 2 hours in flight, about 1200 MB
2026-11-07T18:53:23Z [load_day 2023-04-19] 2023-04-19 hour 0: 158076 events in 4 files, snapshot 8275986447497575655 (task 11069 ms: download 4297 ms overlapping gunzip 632 ms, parse 2913 ms, sort 101 ms, write 6373 ms, index 558 ms: 4 candidates, 79072 repo-hours, 895 releases; waiting 227 ms; commit 1652 ms)
2026-11-07T18:53:25Z [load_day 2023-04-19] 2023-04-19 hour 1: 178801 events in 4 files, snapshot 433186001744932501 (task 12826 ms: download 7245 ms overlapping gunzip 769 ms, parse 3348 ms, sort 134 ms, write 7163 ms, index 606 ms: 4 candidates, 90425 repo-hours, 800 releases; waiting 515 ms; commit 1520 ms)
2026-11-07T18:53:35Z [load_day 2023-04-19] 2023-04-19 hour 2: 145794 events in 3 files, snapshot 380987815502000208 (task 10256 ms: download 4008 ms overlapping gunzip 616 ms, parse 2645 ms, sort 93 ms, write 5880 ms, index 469 ms: 6 candidates, 68198 repo-hours, 699 releases; waiting 206 ms; commit 1937 ms)
2026-11-07T18:53:36Z [load_day 2023-04-19] 2023-04-19 hour 3: 138389 events in 3 files, snapshot 9073158330635840286 (task 10003 ms: download 3994 ms overlapping gunzip 572 ms, parse 2495 ms, sort 85 ms, write 5553 ms, index 441 ms: 1 candidates, 61908 repo-hours, 784 releases; waiting 555 ms; commit 1622 ms)
2026-11-07T18:53:45Z [load_day 2023-04-19] 2023-04-19 hour 4: 129689 events in 3 files, snapshot 4811247059970041020 (task 9365 ms: download 3907 ms overlapping gunzip 553 ms, parse 2444 ms, sort 79 ms, write 5122 ms, index 390 ms: 4 candidates, 57434 repo-hours, 542 releases; waiting 514 ms; commit 2172 ms)
2026-11-07T18:53:48Z [load_day 2023-04-19] 2023-04-19 hour 5: 141753 events in 3 files, snapshot 8664801461846710355 (task 11071 ms: download 4727 ms overlapping gunzip 645 ms, parse 2682 ms, sort 91 ms, write 6205 ms, index 410 ms: 8 candidates, 62179 repo-hours, 690 releases; waiting 722 ms; commit 1584 ms)
2026-11-07T18:53:58Z [load_day 2023-04-19] 2023-04-19 hour 6: 171823 events in 4 files, snapshot 4860284493582927540 (task 12831 ms: download 7240 ms overlapping gunzip 734 ms, parse 3187 ms, sort 97 ms, write 7453 ms, index 554 ms: 9 candidates, 83300 repo-hours, 954 releases; waiting 522 ms; commit 1558 ms)
2026-11-07T18:54:03Z [load_day 2023-04-19] 2023-04-19 hour 7: 167483 events in 4 files, snapshot 3013683881308126280 (task 13749 ms: download 7870 ms overlapping gunzip 799 ms, parse 3419 ms, sort 113 ms, write 7957 ms, index 566 ms: 2 candidates, 78732 repo-hours, 872 releases; waiting 547 ms; commit 1631 ms)
2026-11-07T18:54:13Z [load_day 2023-04-19] 2023-04-19 hour 8: 170756 events in 4 files, snapshot 2031734652236959476 (task 14344 ms: download 8246 ms overlapping gunzip 827 ms, parse 3650 ms, sort 112 ms, write 8236 ms, index 595 ms: 4 candidates, 83894 repo-hours, 903 releases; waiting 605 ms; commit 1788 ms)
2026-11-07T18:54:20Z [load_day 2023-04-19] 2023-04-19 hour 9: 190530 events in 4 files, snapshot 1254993650451406469 (task 16412 ms: download 9337 ms overlapping gunzip 935 ms, parse 4119 ms, sort 122 ms, write 9217 ms, index 670 ms: 4 candidates, 90425 repo-hours, 1039 releases; waiting 1095 ms; commit 1552 ms)
2026-11-07T18:54:29Z [load_day 2023-04-19] 2023-04-19 hour 10: 187538 events in 4 files, snapshot 697221409683461306 (task 15700 ms: download 9062 ms overlapping gunzip 924 ms, parse 4008 ms, sort 131 ms, write 8864 ms, index 580 ms: 3 candidates, 86974 repo-hours, 935 releases; waiting 915 ms; commit 1443 ms)
2026-11-07T18:54:36Z [load_day 2023-04-19] 2023-04-19 hour 11: 187463 events in 4 files, snapshot 6044200607413269687 (task 15502 ms: download 8655 ms overlapping gunzip 887 ms, parse 3912 ms, sort 112 ms, write 9023 ms, index 589 ms: 6 candidates, 83813 repo-hours, 1114 releases; waiting 640 ms; commit 1550 ms)
2026-11-07T18:54:46Z [load_day 2023-04-19] 2023-04-19 hour 12: 216547 events in 5 files, snapshot 7120077213494338491 (task 15936 ms: download 8044 ms overlapping gunzip 949 ms, parse 4247 ms, sort 135 ms, write 9304 ms, index 698 ms: 3 candidates, 107255 repo-hours, 1036 releases; waiting 253 ms; commit 1621 ms)
2026-11-07T18:54:53Z [load_day 2023-04-19] 2023-04-19 hour 13: 206511 events in 5 files, snapshot 3790966279879504648 (task 16507 ms: download 8387 ms overlapping gunzip 959 ms, parse 4282 ms, sort 131 ms, write 9584 ms, index 718 ms: 8 candidates, 107477 repo-hours, 1013 releases; waiting 509 ms; commit 1597 ms)
2026-11-07T18:55:04Z [load_day 2023-04-19] 2023-04-19 hour 14: 198079 events in 4 files, snapshot 7213116373437032696 (task 16873 ms: download 9274 ms overlapping gunzip 988 ms, parse 4345 ms, sort 132 ms, write 9899 ms, index 656 ms: 5 candidates, 93847 repo-hours, 1096 releases; waiting 521 ms; commit 1472 ms)
2026-11-07T18:55:09Z [load_day 2023-04-19] 2023-04-19 hour 15: 178668 events in 4 files, snapshot 8956870239652521547 (task 15351 ms: download 8658 ms overlapping gunzip 931 ms, parse 4103 ms, sort 122 ms, write 8916 ms, index 640 ms: 6 candidates, 89081 repo-hours, 1013 releases; waiting 370 ms; commit 1547 ms)
2026-11-07T18:55:21Z [load_day 2023-04-19] 2023-04-19 hour 16: 185506 events in 4 files, snapshot 1610758496188654476 (task 16323 ms: download 9294 ms overlapping gunzip 929 ms, parse 4101 ms, sort 132 ms, write 9450 ms, index 591 ms: 8 candidates, 87278 repo-hours, 911 releases; waiting 721 ms; commit 1526 ms)
2026-11-07T18:55:26Z [load_day 2023-04-19] 2023-04-19 hour 17: 183283 events in 4 files, snapshot 2378575440729702254 (task 15371 ms: download 8630 ms overlapping gunzip 883 ms, parse 3876 ms, sort 127 ms, write 8984 ms, index 555 ms: 3 candidates, 81701 repo-hours, 899 releases; waiting 629 ms; commit 1681 ms)
2026-11-07T18:55:38Z [load_day 2023-04-19] 2023-04-19 hour 18: 197496 events in 4 files, snapshot 5631671247778005717 (task 16664 ms: download 9314 ms overlapping gunzip 1019 ms, parse 4333 ms, sort 146 ms, write 9678 ms, index 586 ms: 7 candidates, 88608 repo-hours, 1056 releases; waiting 604 ms; commit 1655 ms)
2026-11-07T18:55:44Z [load_day 2023-04-19] 2023-04-19 hour 19: 191495 events in 4 files, snapshot 2449402839287826924 (task 17882 ms: download 10747 ms overlapping gunzip 1148 ms, parse 4955 ms, sort 119 ms, write 10065 ms, index 546 ms: 6 candidates, 82264 repo-hours, 897 releases; waiting 773 ms; commit 1579 ms)
2026-11-07T18:55:54Z [load_day 2023-04-19] 2023-04-19 hour 20: 181178 events in 4 files, snapshot 1272516370057233812 (task 15277 ms: download 8925 ms overlapping gunzip 917 ms, parse 4008 ms, sort 111 ms, write 8896 ms, index 484 ms: 3 candidates, 77824 repo-hours, 883 releases; waiting 531 ms; commit 1666 ms)
2026-11-07T18:56:00Z [load_day 2023-04-19] 2023-04-19 hour 21: 167944 events in 4 files, snapshot 1327625735886097200 (task 14729 ms: download 8965 ms overlapping gunzip 870 ms, parse 3770 ms, sort 106 ms, write 8341 ms, index 580 ms: 4 candidates, 70677 repo-hours, 941 releases; waiting 833 ms; commit 1448 ms)
2026-11-07T18:56:07Z [load_day 2023-04-19] 2023-04-19 hour 22: 146725 events in 3 files, snapshot 1108524882771379638 (task 11686 ms: download 5045 ms overlapping gunzip 667 ms, parse 2916 ms, sort 108 ms, write 6787 ms, index 410 ms: 8 candidates, 61069 repo-hours, 799 releases; waiting 518 ms; commit 1665 ms)
2026-11-07T18:56:11Z [load_day 2023-04-19] 2023-04-19 hour 23: 139976 events in 3 files, snapshot 2151822990375465891 (task 10550 ms: download 4468 ms overlapping gunzip 624 ms, parse 2742 ms, sort 92 ms, write 5888 ms, index 403 ms: 3 candidates, 58328 repo-hours, 601 releases; waiting 528 ms; commit 1664 ms)
2026-11-07T18:56:19Z [load_day 2023-04-19] 2023-04-19: repo_hour merged from 24 files to 1 (1931763 rows, 8025 ms)
2026-11-07T18:56:21Z [load_day 2023-04-19] 2023-04-19: index committed in 9967 ms (24 + 1 + 24 files)
2026-11-07T18:56:21Z [load_day 2023-04-19] 2023-04-19: 4161503 events in 24 hours
2026-11-07T18:56:21Z [load_day 2023-04-19] attempt 1 succeeded