orcaLog in

Runs · run 1681

load_day 2023-10-30, attempt 1 succeeded

Host
3c2b29713ce8:495587
Started
2026-11-08T02:17:21Z
Ended
2026-11-08T02:22:03Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1681/load_day/2023-10-30/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:17:21Z [load_day 2023-10-30] attempt 1 started
2026-11-08T02:17:22Z [load_day 2023-10-30] 2023-10-30: 2 hours in flight, about 1200 MB
2026-11-08T02:17:42Z [load_day 2023-10-30] 2023-10-30 hour 0: 149709 events in 3 files, snapshot 2680433332603681876 (task 13132 ms: download 5570 ms overlapping gunzip 897 ms, parse 3873 ms, sort 120 ms, write 7437 ms, index 604 ms: 9 candidates, 80777 repo-hours, 831 releases; waiting 13 ms; commit 4798 ms)
2026-11-08T02:17:47Z [load_day 2023-10-30] 2023-10-30 hour 1: 165736 events in 4 files, snapshot 2248521153928515825 (task 16663 ms: download 6692 ms overlapping gunzip 1048 ms, parse 4508 ms, sort 114 ms, write 9718 ms, index 1078 ms: 7 candidates, 91832 repo-hours, 675 releases; waiting 18 ms; commit 4900 ms)
2026-11-08T02:17:59Z [load_day 2023-10-30] 2023-10-30 hour 2: 139785 events in 3 files, snapshot 2638394446386464574 (task 14905 ms: download 6482 ms overlapping gunzip 1047 ms, parse 4425 ms, sort 164 ms, write 8604 ms, index 435 ms: 6 candidates, 60564 repo-hours, 744 releases; waiting 19 ms; commit 5043 ms)
2026-11-08T02:18:02Z [load_day 2023-10-30] 2023-10-30 hour 3: 144626 events in 3 files, snapshot 1989120081317546542 (task 14722 ms: download 6344 ms overlapping gunzip 1047 ms, parse 4364 ms, sort 93 ms, write 8458 ms, index 544 ms: 8 candidates, 59828 repo-hours, 784 releases; waiting 18 ms; commit 3389 ms)
2026-11-08T02:18:12Z [load_day 2023-10-30] 2023-10-30 hour 4: 135920 events in 3 files, snapshot 5764446544766425575 (task 14211 ms: download 5175 ms overlapping gunzip 860 ms, parse 3793 ms, sort 147 ms, write 8740 ms, index 430 ms: 7 candidates, 55096 repo-hours, 588 releases; waiting 28 ms; commit 2852 ms)
2026-11-08T02:18:18Z [load_day 2023-10-30] 2023-10-30 hour 5: 141398 events in 3 files, snapshot 3030619342702648216 (task 14893 ms: download 6592 ms overlapping gunzip 1286 ms, parse 4937 ms, sort 111 ms, write 7832 ms, index 517 ms: 4 candidates, 59186 repo-hours, 616 releases; waiting 19 ms; commit 2664 ms)
2026-11-08T02:18:33Z [load_day 2023-10-30] 2023-10-30 hour 6: 172173 events in 4 files, snapshot 4517061465711327299 (task 17702 ms: download 10069 ms overlapping gunzip 1079 ms, parse 4366 ms, sort 128 ms, write 10994 ms, index 913 ms: 13 candidates, 74808 repo-hours, 1010 releases; waiting 16 ms; commit 5254 ms)
2026-11-08T02:18:41Z [load_day 2023-10-30] 2023-10-30 hour 7: 180961 events in 4 files, snapshot 1415480258058288149 (task 21913 ms: download 11451 ms overlapping gunzip 1558 ms, parse 6366 ms, sort 177 ms, write 12657 ms, index 942 ms: 5 candidates, 73930 repo-hours, 1133 releases; waiting 17 ms; commit 2747 ms)
2026-11-08T02:18:55Z [load_day 2023-10-30] 2023-10-30 hour 8: 200034 events in 5 files, snapshot 2687493945355975595 (task 22572 ms: download 12670 ms overlapping gunzip 1597 ms, parse 6383 ms, sort 210 ms, write 13326 ms, index 819 ms: 10 candidates, 84008 repo-hours, 976 releases; waiting 19 ms; commit 2983 ms)
2026-11-08T02:19:08Z [load_day 2023-10-30] 2023-10-30 hour 9: 206211 events in 5 files, snapshot 5444776458790449854 (task 23043 ms: download 16072 ms overlapping gunzip 1465 ms, parse 6470 ms, sort 154 ms, write 13895 ms, index 842 ms: 11 candidates, 87215 repo-hours, 1179 releases; waiting 11 ms; commit 5553 ms)
2026-11-08T02:19:22Z [load_day 2023-10-30] 2023-10-30 hour 10: 202883 events in 5 files, snapshot 3534966543760650685 (task 23037 ms: download 12678 ms overlapping gunzip 1461 ms, parse 6357 ms, sort 136 ms, write 13905 ms, index 689 ms: 9 candidates, 87612 repo-hours, 1040 releases; waiting 22 ms; commit 5497 ms)
2026-11-08T02:19:30Z [load_day 2023-10-30] 2023-10-30 hour 11: 198597 events in 4 files, snapshot 7213063911009920351 (task 23229 ms: download 13088 ms overlapping gunzip 1493 ms, parse 6546 ms, sort 143 ms, write 14197 ms, index 632 ms: 6 candidates, 84506 repo-hours, 1150 releases; waiting 18 ms; commit 2858 ms)
2026-11-08T02:19:42Z [load_day 2023-10-30] 2023-10-30 hour 12: 251907 events in 6 files, snapshot 2874978298854576971 (task 22012 ms: download 14584 ms overlapping gunzip 1509 ms, parse 6657 ms, sort 171 ms, write 12675 ms, index 767 ms: 10 candidates, 115530 repo-hours, 1426 releases; waiting 17 ms; commit 2154 ms)
2026-11-08T02:19:52Z [load_day 2023-10-30] 2023-10-30 hour 13: 238907 events in 5 files, snapshot 6787839375916162689 (task 21327 ms: download 13022 ms overlapping gunzip 1370 ms, parse 5742 ms, sort 150 ms, write 12718 ms, index 1116 ms: 13 candidates, 113033 repo-hours, 1211 releases; waiting 17 ms; commit 2767 ms)
2026-11-08T02:20:10Z [load_day 2023-10-30] 2023-10-30 hour 14: 234213 events in 5 files, snapshot 7892684466086072457 (task 25655 ms: download 16787 ms overlapping gunzip 1507 ms, parse 6523 ms, sort 196 ms, write 16401 ms, index 816 ms: 11 candidates, 101410 repo-hours, 1237 releases; waiting 19 ms; commit 3258 ms)
2026-11-08T02:20:25Z [load_day 2023-10-30] 2023-10-30 hour 15: 239074 events in 5 files, snapshot 4268612717361566707 (task 28937 ms: download 19080 ms overlapping gunzip 1825 ms, parse 7765 ms, sort 266 ms, write 17580 ms, index 694 ms: 7 candidates, 102881 repo-hours, 1306 releases; waiting 524 ms; commit 5469 ms)
2026-11-08T02:20:40Z [load_day 2023-10-30] 2023-10-30 hour 16: 225098 events in 5 files, snapshot 7083836027367453623 (task 26773 ms: download 18626 ms overlapping gunzip 1666 ms, parse 7139 ms, sort 166 ms, write 16862 ms, index 702 ms: 7 candidates, 98047 repo-hours, 1248 releases; waiting 23 ms; commit 5471 ms)
2026-11-08T02:20:49Z [load_day 2023-10-30] 2023-10-30 hour 17: 213788 events in 5 files, snapshot 5852673150260207055 (task 24894 ms: download 17216 ms overlapping gunzip 1494 ms, parse 6731 ms, sort 151 ms, write 15690 ms, index 614 ms: 18 candidates, 91461 repo-hours, 1239 releases; waiting 18 ms; commit 2917 ms)
2026-11-08T02:21:03Z [load_day 2023-10-30] 2023-10-30 hour 18: 214084 events in 5 files, snapshot 8135785708445614252 (task 25115 ms: download 17133 ms overlapping gunzip 1836 ms, parse 7768 ms, sort 250 ms, write 13978 ms, index 1072 ms: 10 candidates, 95166 repo-hours, 1137 releases; waiting 24 ms; commit 2793 ms)
2026-11-08T02:21:12Z [load_day 2023-10-30] 2023-10-30 hour 19: 199608 events in 4 files, snapshot 7676360098422514328 (task 22492 ms: download 12280 ms overlapping gunzip 1419 ms, parse 6206 ms, sort 154 ms, write 13842 ms, index 659 ms: 9 candidates, 83672 repo-hours, 982 releases; waiting 23 ms; commit 2686 ms)
2026-11-08T02:21:29Z [load_day 2023-10-30] 2023-10-30 hour 20: 198403 events in 4 files, snapshot 4226605557386290396 (task 23781 ms: download 12559 ms overlapping gunzip 1442 ms, parse 6105 ms, sort 262 ms, write 15001 ms, index 746 ms: 8 candidates, 81338 repo-hours, 1140 releases; waiting 28 ms; commit 3283 ms)
2026-11-08T02:21:35Z [load_day 2023-10-30] 2023-10-30 hour 21: 179031 events in 4 files, snapshot 1551522470108318890 (task 21344 ms: download 13063 ms overlapping gunzip 1325 ms, parse 5632 ms, sort 119 ms, write 13502 ms, index 552 ms: 10 candidates, 72146 repo-hours, 1242 releases; waiting 14 ms; commit 2869 ms)
2026-11-08T02:21:49Z [load_day 2023-10-30] 2023-10-30 hour 22: 167470 events in 4 files, snapshot 7820926722642443816 (task 18459 ms: download 11252 ms overlapping gunzip 1140 ms, parse 4863 ms, sort 129 ms, write 11385 ms, index 536 ms: 13 candidates, 66577 repo-hours, 1058 releases; waiting 195 ms; commit 3360 ms)
2026-11-08T02:21:52Z [load_day 2023-10-30] 2023-10-30 hour 23: 154251 events in 4 files, snapshot 6765531938665530619 (task 16347 ms: download 7271 ms overlapping gunzip 988 ms, parse 4506 ms, sort 172 ms, write 10052 ms, index 421 ms: 10 candidates, 62733 repo-hours, 794 releases; waiting 21 ms; commit 2178 ms)
2026-11-08T02:22:01Z [load_day 2023-10-30] 2023-10-30: repo_hour merged from 24 files to 1 (1983356 rows, 9157 ms)
2026-11-08T02:22:03Z [load_day 2023-10-30] 2023-10-30: index committed in 11067 ms (24 + 1 + 24 files)
2026-11-08T02:22:03Z [load_day 2023-10-30] 2023-10-30: 4553867 events in 24 hours
2026-11-08T02:22:03Z [load_day 2023-10-30] attempt 1 succeeded