orcaLog in

Runs · run 1565

load_day 2023-09-28, attempt 1 succeeded

Host
3c2b29713ce8:379547
Started
2026-11-08T01:16:03Z
Ended
2026-11-08T01:20:03Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1565/load_day/2023-09-28/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:16:03Z [load_day 2023-09-28] attempt 1 started
2026-11-08T01:16:05Z [load_day 2023-09-28] 2023-09-28: 2 hours in flight, about 1200 MB
2026-11-08T01:16:24Z [load_day 2023-09-28] 2023-09-28 hour 0: 164464 events in 4 files, snapshot 965364383659378540 (task 15654 ms: download 7418 ms overlapping gunzip 952 ms, parse 4600 ms, sort 141 ms, write 8490 ms, index 674 ms: 6 candidates, 91379 repo-hours, 937 releases; waiting 486 ms; commit 2314 ms)
2026-11-08T01:16:27Z [load_day 2023-09-28] 2023-09-28 hour 1: 154437 events in 4 files, snapshot 3391241724397862317 (task 14589 ms: download 5523 ms overlapping gunzip 839 ms, parse 3929 ms, sort 161 ms, write 8247 ms, index 633 ms: 6 candidates, 88813 repo-hours, 918 releases; waiting 466 ms; commit 2615 ms)
2026-11-08T01:16:37Z [load_day 2023-09-28] 2023-09-28 hour 2: 129994 events in 3 files, snapshot 1859432806255156303 (task 11856 ms: download 4855 ms overlapping gunzip 679 ms, parse 3033 ms, sort 140 ms, write 6867 ms, index 441 ms: 6 candidates, 58397 repo-hours, 711 releases; waiting 362 ms; commit 2538 ms)
2026-11-08T01:16:41Z [load_day 2023-09-28] 2023-09-28 hour 3: 127396 events in 3 files, snapshot 1122373909596802032 (task 11875 ms: download 2568 ms overlapping gunzip 657 ms, parse 2770 ms, sort 80 ms, write 7029 ms, index 671 ms: 10 candidates, 57683 repo-hours, 823 releases; waiting 393 ms; commit 3243 ms)
2026-11-08T01:16:50Z [load_day 2023-09-28] 2023-09-28 hour 4: 124613 events in 3 files, snapshot 2599206663985904806 (task 11032 ms: download 2556 ms overlapping gunzip 608 ms, parse 2678 ms, sort 91 ms, write 6550 ms, index 368 ms: 5 candidates, 52578 repo-hours, 582 releases; waiting 494 ms; commit 2780 ms)
2026-11-08T01:16:56Z [load_day 2023-09-28] 2023-09-28 hour 5: 133159 events in 3 files, snapshot 3289793648720490387 (task 12214 ms: download 4560 ms overlapping gunzip 746 ms, parse 3216 ms, sort 104 ms, write 6685 ms, index 612 ms: 8 candidates, 58534 repo-hours, 738 releases; waiting 520 ms; commit 4932 ms)
2026-11-08T01:17:13Z [load_day 2023-09-28] 2023-09-28 hour 6: 161382 events in 4 files, snapshot 5220305911617601371 (task 15546 ms: download 7151 ms overlapping gunzip 1063 ms, parse 4627 ms, sort 112 ms, write 8763 ms, index 498 ms: 6 candidates, 74591 repo-hours, 997 releases; waiting 217 ms; commit 9083 ms)
2026-11-08T01:17:16Z [load_day 2023-09-28] 2023-09-28 hour 7: 172356 events in 4 files, snapshot 237432186874119399 (task 17149 ms: download 9979 ms overlapping gunzip 926 ms, parse 4162 ms, sort 196 ms, write 10267 ms, index 619 ms: 9 candidates, 76491 repo-hours, 1022 releases; waiting 514 ms; commit 2534 ms)
2026-11-08T01:17:24Z [load_day 2023-09-28] 2023-09-28 hour 8: 183698 events in 4 files, snapshot 4381720978494793195 (task 16987 ms: download 9406 ms overlapping gunzip 928 ms, parse 4071 ms, sort 118 ms, write 10470 ms, index 541 ms: 7 candidates, 81184 repo-hours, 1070 releases; waiting 531 ms; commit 2361 ms)
2026-11-08T01:17:35Z [load_day 2023-09-28] 2023-09-28 hour 9: 184571 events in 4 files, snapshot 827223225722508357 (task 17984 ms: download 8960 ms overlapping gunzip 947 ms, parse 4255 ms, sort 140 ms, write 11211 ms, index 707 ms: 7 candidates, 81413 repo-hours, 1221 releases; waiting 481 ms; commit 2395 ms)
2026-11-08T01:17:45Z [load_day 2023-09-28] 2023-09-28 hour 10: 179581 events in 4 files, snapshot 8375189709145463124 (task 17857 ms: download 10234 ms overlapping gunzip 1101 ms, parse 4898 ms, sort 179 ms, write 10300 ms, index 586 ms: 6 candidates, 78412 repo-hours, 1100 releases; waiting 507 ms; commit 4483 ms)
2026-11-08T01:17:59Z [load_day 2023-09-28] 2023-09-28 hour 11: 180573 events in 4 files, snapshot 2297172718398028192 (task 19001 ms: download 10470 ms overlapping gunzip 1221 ms, parse 5457 ms, sort 201 ms, write 10322 ms, index 715 ms: 8 candidates, 76510 repo-hours, 1038 releases; waiting 795 ms; commit 6593 ms)
2026-11-08T01:18:06Z [load_day 2023-09-28] 2023-09-28 hour 12: 235712 events in 5 files, snapshot 5527255570200130381 (task 21747 ms: download 13300 ms overlapping gunzip 1204 ms, parse 5658 ms, sort 247 ms, write 13107 ms, index 727 ms: 10 candidates, 111643 repo-hours, 1254 releases; waiting 542 ms; commit 2905 ms)
2026-11-08T01:18:19Z [load_day 2023-09-28] 2023-09-28 hour 13: 218057 events in 5 files, snapshot 5147593933799489572 (task 23580 ms: download 15702 ms overlapping gunzip 1320 ms, parse 5687 ms, sort 133 ms, write 14685 ms, index 966 ms: 5 candidates, 107725 repo-hours, 1207 releases; waiting 509 ms; commit 2390 ms)
2026-11-08T01:18:32Z [load_day 2023-09-28] 2023-09-28 hour 14: 209007 events in 5 files, snapshot 4086858279584487604 (task 21727 ms: download 11650 ms overlapping gunzip 1168 ms, parse 5091 ms, sort 133 ms, write 14061 ms, index 702 ms: 8 candidates, 94045 repo-hours, 1089 releases; waiting 317 ms; commit 5042 ms)
2026-11-08T01:18:41Z [load_day 2023-09-28] 2023-09-28 hour 15: 189122 events in 4 files, snapshot 5289172092615296869 (task 20219 ms: download 10560 ms overlapping gunzip 1195 ms, parse 5549 ms, sort 180 ms, write 11832 ms, index 695 ms: 11 candidates, 87681 repo-hours, 1171 releases; waiting 500 ms; commit 2825 ms)
2026-11-08T01:18:52Z [load_day 2023-09-28] 2023-09-28 hour 16: 189464 events in 4 files, snapshot 3152247033915214403 (task 20498 ms: download 11574 ms overlapping gunzip 1269 ms, parse 5558 ms, sort 178 ms, write 12148 ms, index 765 ms: 9 candidates, 85537 repo-hours, 1017 releases; waiting 359 ms; commit 3906 ms)
2026-11-08T01:19:03Z [load_day 2023-09-28] 2023-09-28 hour 17: 190688 events in 4 files, snapshot 1347200031559331898 (task 19307 ms: download 10200 ms overlapping gunzip 1069 ms, parse 4937 ms, sort 128 ms, write 11722 ms, index 613 ms: 5 candidates, 79679 repo-hours, 1010 releases; waiting 547 ms; commit 4164 ms)
2026-11-08T01:19:11Z [load_day 2023-09-28] 2023-09-28 hour 18: 190697 events in 4 files, snapshot 7328411074433901906 (task 19831 ms: download 10523 ms overlapping gunzip 1256 ms, parse 5594 ms, sort 177 ms, write 11148 ms, index 845 ms: 9 candidates, 86508 repo-hours, 990 releases; waiting 475 ms; commit 2499 ms)
2026-11-08T01:19:21Z [load_day 2023-09-28] 2023-09-28 hour 19: 178225 events in 4 files, snapshot 7141830229201774448 (task 18869 ms: download 10751 ms overlapping gunzip 1173 ms, parse 5196 ms, sort 184 ms, write 10908 ms, index 548 ms: 13 candidates, 75301 repo-hours, 970 releases; waiting 508 ms; commit 2220 ms)
2026-11-08T01:19:30Z [load_day 2023-09-28] 2023-09-28 hour 20: 168196 events in 4 files, snapshot 4314160528501127471 (task 17021 ms: download 9677 ms overlapping gunzip 918 ms, parse 4237 ms, sort 175 ms, write 10132 ms, index 744 ms: 8 candidates, 72163 repo-hours, 908 releases; waiting 503 ms; commit 2932 ms)
2026-11-08T01:19:40Z [load_day 2023-09-28] 2023-09-28 hour 21: 158054 events in 4 files, snapshot 2373951154049159968 (task 17939 ms: download 7703 ms overlapping gunzip 1104 ms, parse 4863 ms, sort 188 ms, write 10033 ms, index 645 ms: 7 candidates, 64082 repo-hours, 997 releases; waiting 748 ms; commit 2434 ms)
2026-11-08T01:19:46Z [load_day 2023-09-28] 2023-09-28 hour 22: 140010 events in 3 files, snapshot 4525602449522137628 (task 15296 ms: download 6632 ms overlapping gunzip 931 ms, parse 3875 ms, sort 130 ms, write 8816 ms, index 476 ms: 11 candidates, 58291 repo-hours, 869 releases; waiting 481 ms; commit 2717 ms)
2026-11-08T01:19:52Z [load_day 2023-09-28] 2023-09-28 hour 23: 128979 events in 3 files, snapshot 4938957296037166725 (task 11081 ms: download 4709 ms overlapping gunzip 607 ms, parse 2734 ms, sort 100 ms, write 6765 ms, index 350 ms: 5 candidates, 53765 repo-hours, 700 releases; waiting 218 ms; commit 2866 ms)
2026-11-08T01:20:00Z [load_day 2023-09-28] 2023-09-28: repo_hour merged from 24 files to 1 (1852405 rows, 8046 ms)
2026-11-08T01:20:03Z [load_day 2023-09-28] 2023-09-28: index committed in 10414 ms (24 + 1 + 24 files)
2026-11-08T01:20:03Z [load_day 2023-09-28] 2023-09-28: 4092435 events in 24 hours
2026-11-08T01:20:03Z [load_day 2023-09-28] attempt 1 succeeded