orcaLog in

Runs · run 1005

load_day 2023-04-17, attempt 1 succeeded

Host
caf251265124:275994
Started
2026-11-07T18:46:18Z
Ended
2026-11-07T18:49:49Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1005/load_day/2023-04-17/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T18:46:18Z [load_day 2023-04-17] attempt 1 started
2026-11-07T18:46:19Z [load_day 2023-04-17] 2023-04-17: 2 hours in flight, about 1200 MB
2026-11-07T18:46:31Z [load_day 2023-04-17] 2023-04-17 hour 0: 140835 events in 3 files, snapshot 6248835315114942350 (task 9378 ms: download 2576 ms overlapping gunzip 523 ms, parse 2431 ms, sort 91 ms, write 4894 ms, index 488 ms: 3 candidates, 76350 repo-hours, 863 releases; waiting 541 ms; commit 1631 ms)
2026-11-07T18:46:34Z [load_day 2023-04-17] 2023-04-17 hour 1: 158499 events in 4 files, snapshot 8894811415891059877 (task 11597 ms: download 4743 ms overlapping gunzip 672 ms, parse 2891 ms, sort 93 ms, write 6475 ms, index 611 ms: 5 candidates, 88016 repo-hours, 720 releases; waiting 541 ms; commit 1786 ms)
2026-11-07T18:46:43Z [load_day 2023-04-17] 2023-04-17 hour 2: 140865 events in 3 files, snapshot 657573208064574755 (task 10798 ms: download 4690 ms overlapping gunzip 663 ms, parse 2805 ms, sort 87 ms, write 5992 ms, index 417 ms: 4 candidates, 64583 repo-hours, 727 releases; waiting 528 ms; commit 1744 ms)
2026-11-07T18:46:45Z [load_day 2023-04-17] 2023-04-17 hour 3: 139062 events in 3 files, snapshot 1397895218389622063 (task 10459 ms: download 4623 ms overlapping gunzip 642 ms, parse 2730 ms, sort 92 ms, write 5808 ms, index 423 ms: 3 candidates, 60095 repo-hours, 781 releases; waiting 484 ms; commit 1486 ms)
2026-11-07T18:46:54Z [load_day 2023-04-17] 2023-04-17 hour 4: 133242 events in 3 files, snapshot 7798214663091848172 (task 10076 ms: download 4524 ms overlapping gunzip 619 ms, parse 2640 ms, sort 88 ms, write 5568 ms, index 368 ms: 3 candidates, 55828 repo-hours, 583 releases; waiting 524 ms; commit 1753 ms)
2026-11-07T18:46:58Z [load_day 2023-04-17] 2023-04-17 hour 5: 149750 events in 3 files, snapshot 8817228148588106587 (task 11656 ms: download 5100 ms overlapping gunzip 752 ms, parse 3103 ms, sort 96 ms, write 6617 ms, index 440 ms: 3 candidates, 63351 repo-hours, 818 releases; waiting 344 ms; commit 1797 ms)
2026-11-07T18:47:09Z [load_day 2023-04-17] 2023-04-17 hour 6: 169407 events in 4 files, snapshot 2572003555219548876 (task 14149 ms: download 8509 ms overlapping gunzip 849 ms, parse 3579 ms, sort 103 ms, write 7918 ms, index 490 ms: 6 candidates, 78072 repo-hours, 944 releases; waiting 765 ms; commit 1724 ms)
2026-11-07T18:47:18Z [load_day 2023-04-17] 2023-04-17 hour 7: 163931 events in 4 files, snapshot 1910766140384626780 (task 14195 ms: download 8547 ms overlapping gunzip 867 ms, parse 3710 ms, sort 95 ms, write 8122 ms, index 548 ms: 4 candidates, 75080 repo-hours, 888 releases; waiting 535 ms; commit 7258 ms)
2026-11-07T18:47:25Z [load_day 2023-04-17] 2023-04-17 hour 8: 168999 events in 4 files, snapshot 3989320385776332513 (task 14872 ms: download 8995 ms overlapping gunzip 908 ms, parse 3811 ms, sort 101 ms, write 8618 ms, index 545 ms: 1 candidates, 79562 repo-hours, 882 releases; waiting 597 ms; commit 1515 ms)
2026-11-07T18:47:35Z [load_day 2023-04-17] 2023-04-17 hour 9: 188938 events in 4 files, snapshot 5035501289096665485 (task 16436 ms: download 9271 ms overlapping gunzip 990 ms, parse 4282 ms, sort 105 ms, write 9517 ms, index 616 ms: 3 candidates, 86973 repo-hours, 1020 releases; waiting 564 ms; commit 1714 ms)
2026-11-07T18:47:42Z [load_day 2023-04-17] 2023-04-17 hour 10: 188879 events in 4 files, snapshot 7359018302164953658 (task 16600 ms: download 9617 ms overlapping gunzip 994 ms, parse 4210 ms, sort 121 ms, write 9554 ms, index 600 ms: 9 candidates, 82825 repo-hours, 982 releases; waiting 707 ms; commit 1708 ms)
2026-11-07T18:47:55Z [load_day 2023-04-17] 2023-04-17 hour 11: 190783 events in 4 files, snapshot 4116007142313405795 (task 19828 ms: download 12314 ms overlapping gunzip 1043 ms, parse 4413 ms, sort 123 ms, write 9526 ms, index 590 ms: 4 candidates, 83835 repo-hours, 990 releases; waiting 3339 ms; commit 1608 ms)
2026-11-07T18:48:01Z [load_day 2023-04-17] 2023-04-17 hour 12: 229499 events in 5 files, snapshot 3937803656439924663 (task 17651 ms: download 11564 ms overlapping gunzip 1026 ms, parse 4454 ms, sort 126 ms, write 10271 ms, index 728 ms: 7 candidates, 116750 repo-hours, 1063 releases; waiting 689 ms; commit 1634 ms)
2026-11-07T18:48:14Z [load_day 2023-04-17] 2023-04-17 hour 13: 210572 events in 5 files, snapshot 8745181369418613223 (task 17744 ms: download 9529 ms overlapping gunzip 1050 ms, parse 4528 ms, sort 129 ms, write 10445 ms, index 705 ms: 10 candidates, 106726 repo-hours, 1057 releases; waiting 563 ms; commit 1712 ms)
2026-11-07T18:48:20Z [load_day 2023-04-17] 2023-04-17 hour 14: 208780 events in 5 files, snapshot 920197605817946100 (task 18588 ms: download 10304 ms overlapping gunzip 1109 ms, parse 4736 ms, sort 116 ms, write 10896 ms, index 660 ms: 4 candidates, 97150 repo-hours, 1087 releases; waiting 785 ms; commit 1672 ms)
2026-11-07T18:48:32Z [load_day 2023-04-17] 2023-04-17 hour 15: 187696 events in 4 files, snapshot 9124563929260093874 (task 17414 ms: download 10165 ms overlapping gunzip 1014 ms, parse 4416 ms, sort 141 ms, write 10124 ms, index 593 ms: 2 candidates, 89160 repo-hours, 1071 releases; waiting 798 ms; commit 1403 ms)
2026-11-07T18:48:37Z [load_day 2023-04-17] 2023-04-17 hour 16: 180732 events in 4 files, snapshot 4815442550501066706 (task 16013 ms: download 9335 ms overlapping gunzip 974 ms, parse 4108 ms, sort 106 ms, write 9435 ms, index 536 ms: 6 candidates, 83898 repo-hours, 854 releases; waiting 534 ms; commit 1596 ms)
2026-11-07T18:48:51Z [load_day 2023-04-17] 2023-04-17 hour 17: 201642 events in 5 files, snapshot 1598109486345292629 (task 18096 ms: download 10093 ms overlapping gunzip 1141 ms, parse 4737 ms, sort 134 ms, write 10651 ms, index 624 ms: 6 candidates, 86931 repo-hours, 1056 releases; waiting 517 ms; commit 1717 ms)
2026-11-07T18:48:54Z [load_day 2023-04-17] 2023-04-17 hour 18: 189274 events in 4 files, snapshot 6764619165041056162 (task 16391 ms: download 9206 ms overlapping gunzip 988 ms, parse 4288 ms, sort 117 ms, write 9443 ms, index 590 ms: 4 candidates, 87510 repo-hours, 1051 releases; waiting 634 ms; commit 1616 ms)
2026-11-07T18:49:09Z [load_day 2023-04-17] 2023-04-17 hour 19: 191863 events in 4 files, snapshot 5393302441143026296 (task 17615 ms: download 10164 ms overlapping gunzip 1107 ms, parse 4734 ms, sort 136 ms, write 10240 ms, index 573 ms: 8 candidates, 80624 repo-hours, 941 releases; waiting 504 ms; commit 1397 ms)
2026-11-07T18:49:11Z [load_day 2023-04-17] 2023-04-17 hour 20: 186375 events in 4 files, snapshot 2842969023685668815 (task 16496 ms: download 9659 ms overlapping gunzip 1018 ms, parse 4326 ms, sort 137 ms, write 9574 ms, index 546 ms: 5 candidates, 79812 repo-hours, 952 releases; waiting 616 ms; commit 1757 ms)
2026-11-07T18:49:25Z [load_day 2023-04-17] 2023-04-17 hour 21: 180012 events in 4 files, snapshot 3607927566040996726 (task 15642 ms: download 9459 ms overlapping gunzip 962 ms, parse 4088 ms, sort 129 ms, write 8798 ms, index 485 ms: 7 candidates, 73058 repo-hours, 988 releases; waiting 870 ms; commit 1477 ms)
2026-11-07T18:49:27Z [load_day 2023-04-17] 2023-04-17 hour 22: 159555 events in 4 files, snapshot 1617866384210632814 (task 13603 ms: download 6008 ms overlapping gunzip 831 ms, parse 3531 ms, sort 103 ms, write 7661 ms, index 418 ms: 6 candidates, 63394 repo-hours, 802 releases; waiting 658 ms; commit 1442 ms)
2026-11-07T18:49:38Z [load_day 2023-04-17] 2023-04-17 hour 23: 144759 events in 3 files, snapshot 2185978076676062531 (task 12001 ms: download 5362 ms overlapping gunzip 740 ms, parse 3173 ms, sort 105 ms, write 6748 ms, index 404 ms: 6 candidates, 59635 repo-hours, 669 releases; waiting 505 ms; commit 1379 ms)
2026-11-07T18:49:46Z [load_day 2023-04-17] 2023-04-17: repo_hour merged from 24 files to 1 (1919218 rows, 8225 ms)
2026-11-07T18:49:49Z [load_day 2023-04-17] 2023-04-17: index committed in 10427 ms (24 + 1 + 24 files)
2026-11-07T18:49:49Z [load_day 2023-04-17] 2023-04-17: 4203949 events in 24 hours
2026-11-07T18:49:49Z [load_day 2023-04-17] attempt 1 succeeded