orcaLog in

Runs · run 1700

load_day 2023-11-04, attempt 1 succeeded

Host
3c2b29713ce8:517460
Started
2026-11-08T02:28:26Z
Ended
2026-11-08T02:31:07Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1700/load_day/2023-11-04/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:28:26Z [load_day 2023-11-04] attempt 1 started
2026-11-08T02:28:28Z [load_day 2023-11-04] 2023-11-04: 2 hours in flight, about 1200 MB
2026-11-08T02:28:46Z [load_day 2023-11-04] 2023-11-04 hour 0: 149231 events in 3 files, snapshot 2658114268773700923 (task 13909 ms: download 6282 ms overlapping gunzip 938 ms, parse 4050 ms, sort 132 ms, write 8019 ms, index 551 ms: 10 candidates, 81231 repo-hours, 992 releases; waiting 18 ms; commit 2673 ms)
2026-11-08T02:28:49Z [load_day 2023-11-04] 2023-11-04 hour 1: 137946 events in 3 files, snapshot 4578366414248944297 (task 13145 ms: download 5059 ms overlapping gunzip 892 ms, parse 3897 ms, sort 88 ms, write 7247 ms, index 801 ms: 4 candidates, 79391 repo-hours, 744 releases; waiting 18 ms; commit 2765 ms)
2026-11-08T02:28:56Z [load_day 2023-11-04] 2023-11-04 hour 2: 108253 events in 3 files, snapshot 829204014651103373 (task 9093 ms: download 1146 ms overlapping gunzip 684 ms, parse 2675 ms, sort 80 ms, write 5114 ms, index 327 ms: 3 candidates, 46595 repo-hours, 610 releases; waiting 25 ms; commit 2998 ms)
2026-11-08T02:29:01Z [load_day 2023-11-04] 2023-11-04 hour 3: 110057 events in 3 files, snapshot 4895431913407326460 (task 10145 ms: download 1898 ms overlapping gunzip 577 ms, parse 2477 ms, sort 73 ms, write 5492 ms, index 662 ms: 2 candidates, 46084 repo-hours, 677 releases; waiting 525 ms; commit 3827 ms)
2026-11-08T02:29:08Z [load_day 2023-11-04] 2023-11-04 hour 4: 109789 events in 3 files, snapshot 5037628907228791863 (task 10679 ms: download 2504 ms overlapping gunzip 643 ms, parse 2669 ms, sort 105 ms, write 5946 ms, index 369 ms: 6 candidates, 45578 repo-hours, 538 releases; waiting 572 ms; commit 2715 ms)
2026-11-08T02:29:11Z [load_day 2023-11-04] 2023-11-04 hour 5: 108188 events in 3 files, snapshot 3553078058069052135 (task 8892 ms: download 1246 ms overlapping gunzip 586 ms, parse 2456 ms, sort 61 ms, write 5033 ms, index 483 ms: 4 candidates, 46379 repo-hours, 553 releases; waiting 38 ms; commit 2836 ms)
2026-11-08T02:29:18Z [load_day 2023-11-04] 2023-11-04 hour 6: 117572 events in 3 files, snapshot 1342451722000572586 (task 8677 ms: download 1362 ms overlapping gunzip 519 ms, parse 2280 ms, sort 65 ms, write 5190 ms, index 408 ms: 4 candidates, 53016 repo-hours, 558 releases; waiting 25 ms; commit 2542 ms)
2026-11-08T02:29:22Z [load_day 2023-11-04] 2023-11-04 hour 7: 111418 events in 3 files, snapshot 4927566407352536128 (task 8382 ms: download 1350 ms overlapping gunzip 439 ms, parse 2109 ms, sort 95 ms, write 5108 ms, index 409 ms: 14 candidates, 47782 repo-hours, 517 releases; waiting 24 ms; commit 4436 ms)
2026-11-08T02:29:30Z [load_day 2023-11-04] 2023-11-04 hour 8: 119423 events in 3 files, snapshot 6928989256448774088 (task 9471 ms: download 891 ms overlapping gunzip 573 ms, parse 2456 ms, sort 73 ms, write 5749 ms, index 423 ms: 7 candidates, 52171 repo-hours, 590 releases; waiting 16 ms; commit 4142 ms)
2026-11-08T02:29:32Z [load_day 2023-11-04] 2023-11-04 hour 9: 123062 events in 3 files, snapshot 1718612136986763801 (task 10133 ms: download 1683 ms overlapping gunzip 577 ms, parse 2783 ms, sort 74 ms, write 5975 ms, index 488 ms: 7 candidates, 53080 repo-hours, 731 releases; waiting 30 ms; commit 2642 ms)
2026-11-08T02:29:39Z [load_day 2023-11-04] 2023-11-04 hour 10: 129137 events in 3 files, snapshot 7447812809555589523 (task 9766 ms: download 1469 ms overlapping gunzip 604 ms, parse 2736 ms, sort 96 ms, write 5637 ms, index 474 ms: 5 candidates, 54722 repo-hours, 689 releases; waiting 20 ms; commit 2986 ms)
2026-11-08T02:29:44Z [load_day 2023-11-04] 2023-11-04 hour 11: 127585 events in 3 files, snapshot 1845557485279711690 (task 10471 ms: download 1492 ms overlapping gunzip 674 ms, parse 2973 ms, sort 86 ms, write 6120 ms, index 398 ms: 5 candidates, 54183 repo-hours, 673 releases; waiting 24 ms; commit 2890 ms)
2026-11-08T02:30:01Z [load_day 2023-11-04] 2023-11-04 hour 12: 201990 events in 5 files, snapshot 9185837807231934956 (task 13864 ms: download 4132 ms overlapping gunzip 753 ms, parse 3879 ms, sort 148 ms, write 8255 ms, index 609 ms: 7 candidates, 87006 repo-hours, 783 releases; waiting 22 ms; commit 9748 ms)
2026-11-08T02:30:06Z [load_day 2023-11-04] 2023-11-04 hour 13: 160287 events in 4 files, snapshot 8979210323124832719 (task 13699 ms: download 4501 ms overlapping gunzip 843 ms, parse 3884 ms, sort 96 ms, write 7439 ms, index 1208 ms: 5 candidates, 80543 repo-hours, 711 releases; waiting 34 ms; commit 5361 ms)
2026-11-08T02:30:11Z [load_day 2023-11-04] 2023-11-04 hour 14: 151275 events in 4 files, snapshot 5977834663725890674 (task 12618 ms: download 4572 ms overlapping gunzip 598 ms, parse 2717 ms, sort 162 ms, write 8357 ms, index 554 ms: 4 candidates, 66038 repo-hours, 904 releases; waiting 25 ms; commit 4902 ms)
2026-11-08T02:30:19Z [load_day 2023-11-04] 2023-11-04 hour 15: 153854 events in 4 files, snapshot 2480192575234402386 (task 12609 ms: download 4816 ms overlapping gunzip 730 ms, parse 3408 ms, sort 104 ms, write 7579 ms, index 527 ms: 7 candidates, 68513 repo-hours, 1101 releases; waiting 21 ms; commit 3520 ms)
2026-11-08T02:30:21Z [load_day 2023-11-04] 2023-11-04 hour 16: 153147 events in 4 files, snapshot 7930111033841593726 (task 11292 ms: download 4396 ms overlapping gunzip 679 ms, parse 3228 ms, sort 113 ms, write 6558 ms, index 509 ms: 9 candidates, 66792 repo-hours, 817 releases; waiting 18 ms; commit 2639 ms)
2026-11-08T02:30:29Z [load_day 2023-11-04] 2023-11-04 hour 17: 144262 events in 3 files, snapshot 8511530728232453349 (task 9806 ms: download 3480 ms overlapping gunzip 539 ms, parse 2584 ms, sort 111 ms, write 5535 ms, index 422 ms: 5 candidates, 61868 repo-hours, 887 releases; waiting 361 ms; commit 2895 ms)
2026-11-08T02:30:32Z [load_day 2023-11-04] 2023-11-04 hour 18: 149800 events in 3 files, snapshot 2831996809017906930 (task 9075 ms: download 3346 ms overlapping gunzip 540 ms, parse 2440 ms, sort 106 ms, write 5235 ms, index 551 ms: 1 candidates, 66857 repo-hours, 945 releases; waiting 15 ms; commit 2613 ms)
2026-11-08T02:30:40Z [load_day 2023-11-04] 2023-11-04 hour 19: 127612 events in 3 files, snapshot 4933546221703003721 (task 9712 ms: download 2050 ms overlapping gunzip 517 ms, parse 2394 ms, sort 72 ms, write 5432 ms, index 446 ms: 2 candidates, 53044 repo-hours, 872 releases; waiting 570 ms; commit 3167 ms)
2026-11-08T02:30:43Z [load_day 2023-11-04] 2023-11-04 hour 20: 123933 events in 3 files, snapshot 3748106010076865816 (task 9345 ms: download 791 ms overlapping gunzip 551 ms, parse 2546 ms, sort 106 ms, write 5428 ms, index 503 ms: 3 candidates, 52487 repo-hours, 716 releases; waiting 17 ms; commit 2823 ms)
2026-11-08T02:30:51Z [load_day 2023-11-04] 2023-11-04 hour 21: 115822 events in 3 files, snapshot 1280025378484841112 (task 8943 ms: download 1269 ms overlapping gunzip 501 ms, parse 2353 ms, sort 74 ms, write 5340 ms, index 442 ms: 2 candidates, 47724 repo-hours, 775 releases; waiting 30 ms; commit 3802 ms)
2026-11-08T02:30:53Z [load_day 2023-11-04] 2023-11-04 hour 22: 105035 events in 3 files, snapshot 2595701142810717646 (task 8740 ms: download 1875 ms overlapping gunzip 432 ms, parse 2051 ms, sort 124 ms, write 4790 ms, index 522 ms: 3 candidates, 43788 repo-hours, 840 releases; waiting 519 ms; commit 2361 ms)
2026-11-08T02:30:58Z [load_day 2023-11-04] 2023-11-04 hour 23: 101050 events in 3 files, snapshot 8562761925494380377 (task 6962 ms: download 1231 ms overlapping gunzip 465 ms, parse 2170 ms, sort 56 ms, write 3758 ms, index 294 ms: 5 candidates, 43275 repo-hours, 680 releases; waiting 24 ms; commit 2429 ms)
2026-11-08T02:31:04Z [load_day 2023-11-04] 2023-11-04: repo_hour merged from 24 files to 1 (1398147 rows, 6862 ms)
2026-11-08T02:31:07Z [load_day 2023-11-04] 2023-11-04: index committed in 9111 ms (24 + 1 + 24 files)
2026-11-08T02:31:07Z [load_day 2023-11-04] 2023-11-04: 3139728 events in 24 hours
2026-11-08T02:31:07Z [load_day 2023-11-04] attempt 1 succeeded