orcaLog in

Runs · run 1568

load_day 2023-09-29, attempt 1 succeeded

Host
3c2b29713ce8:381909
Started
2026-11-08T01:17:23Z
Ended
2026-11-08T01:21:05Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1568/load_day/2023-09-29/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T01:17:23Z [load_day 2023-09-29] attempt 1 started
2026-11-08T01:17:24Z [load_day 2023-09-29] 2023-09-29: 2 hours in flight, about 1200 MB
2026-11-08T01:17:43Z [load_day 2023-09-29] 2023-09-29 hour 0: 157656 events in 4 files, snapshot 3857445909545882174 (task 14153 ms: download 6181 ms overlapping gunzip 768 ms, parse 3697 ms, sort 160 ms, write 8201 ms, index 523 ms: 9 candidates, 84196 repo-hours, 1046 releases; waiting 537 ms; commit 2255 ms)
2026-11-08T01:17:48Z [load_day 2023-09-29] 2023-09-29 hour 1: 146070 events in 3 files, snapshot 7270931962074967831 (task 13088 ms: download 5334 ms overlapping gunzip 646 ms, parse 3082 ms, sort 109 ms, write 7854 ms, index 579 ms: 5 candidates, 82015 repo-hours, 704 releases; waiting 528 ms; commit 5386 ms)
2026-11-08T01:17:56Z [load_day 2023-09-29] 2023-09-29 hour 2: 118575 events in 3 files, snapshot 3055558166332108596 (task 11000 ms: download 2318 ms overlapping gunzip 649 ms, parse 2853 ms, sort 80 ms, write 6230 ms, index 326 ms: 4 candidates, 49118 repo-hours, 655 releases; waiting 544 ms; commit 3608 ms)
2026-11-08T01:18:01Z [load_day 2023-09-29] 2023-09-29 hour 3: 114773 events in 3 files, snapshot 7236471857551743176 (task 10689 ms: download 2312 ms overlapping gunzip 608 ms, parse 2889 ms, sort 103 ms, write 5646 ms, index 726 ms: 6 candidates, 47636 repo-hours, 957 releases; waiting 489 ms; commit 4947 ms)
2026-11-08T01:18:10Z [load_day 2023-09-29] 2023-09-29 hour 4: 118943 events in 3 files, snapshot 3971489655224050146 (task 11784 ms: download 2168 ms overlapping gunzip 651 ms, parse 2864 ms, sort 81 ms, write 6908 ms, index 482 ms: 3 candidates, 49448 repo-hours, 670 releases; waiting 496 ms; commit 4222 ms)
2026-11-08T01:18:12Z [load_day 2023-09-29] 2023-09-29 hour 5: 121046 events in 3 files, snapshot 2321317722499410172 (task 12251 ms: download 5093 ms overlapping gunzip 618 ms, parse 2801 ms, sort 159 ms, write 7165 ms, index 650 ms: 9 candidates, 51862 repo-hours, 723 releases; waiting 520 ms; commit 2747 ms)
2026-11-08T01:18:23Z [load_day 2023-09-29] 2023-09-29 hour 6: 142542 events in 3 files, snapshot 2882702842054637643 (task 13291 ms: download 5254 ms overlapping gunzip 799 ms, parse 3782 ms, sort 103 ms, write 7211 ms, index 624 ms: 4 candidates, 64639 repo-hours, 804 releases; waiting 494 ms; commit 2501 ms)
2026-11-08T01:18:29Z [load_day 2023-09-29] 2023-09-29 hour 7: 154079 events in 4 files, snapshot 562928807718329280 (task 15634 ms: download 6553 ms overlapping gunzip 870 ms, parse 4251 ms, sort 156 ms, write 9133 ms, index 459 ms: 5 candidates, 65467 repo-hours, 955 releases; waiting 482 ms; commit 2676 ms)
2026-11-08T01:18:44Z [load_day 2023-09-29] 2023-09-29 hour 8: 171559 events in 4 files, snapshot 4876312781045774914 (task 18149 ms: download 10950 ms overlapping gunzip 965 ms, parse 4554 ms, sort 173 ms, write 10883 ms, index 779 ms: 11 candidates, 73084 repo-hours, 1068 releases; waiting 564 ms; commit 4098 ms)
2026-11-08T01:18:50Z [load_day 2023-09-29] 2023-09-29 hour 9: 175741 events in 4 files, snapshot 4139027736936038213 (task 19475 ms: download 11514 ms overlapping gunzip 1225 ms, parse 5387 ms, sort 136 ms, write 11363 ms, index 559 ms: 8 candidates, 75304 repo-hours, 1170 releases; waiting 524 ms; commit 2528 ms)
2026-11-08T01:19:00Z [load_day 2023-09-29] 2023-09-29 hour 10: 169175 events in 4 files, snapshot 7137923250976843641 (task 16915 ms: download 9352 ms overlapping gunzip 1080 ms, parse 4853 ms, sort 204 ms, write 9490 ms, index 501 ms: 8 candidates, 72745 repo-hours, 968 releases; waiting 517 ms; commit 2299 ms)
2026-11-08T01:19:08Z [load_day 2023-09-29] 2023-09-29 hour 11: 170724 events in 4 files, snapshot 3962196040452275781 (task 17289 ms: download 9937 ms overlapping gunzip 869 ms, parse 3910 ms, sort 116 ms, write 11064 ms, index 554 ms: 7 candidates, 72750 repo-hours, 1024 releases; waiting 489 ms; commit 2681 ms)
2026-11-08T01:19:23Z [load_day 2023-09-29] 2023-09-29 hour 12: 229837 events in 5 files, snapshot 5482072601829003618 (task 20266 ms: download 10472 ms overlapping gunzip 1134 ms, parse 5525 ms, sort 231 ms, write 11745 ms, index 973 ms: 9 candidates, 106201 repo-hours, 1091 releases; waiting 380 ms; commit 4218 ms)
2026-11-08T01:19:32Z [load_day 2023-09-29] 2023-09-29 hour 13: 214019 events in 5 files, snapshot 7622957094090697729 (task 21322 ms: download 10692 ms overlapping gunzip 1253 ms, parse 5935 ms, sort 128 ms, write 12336 ms, index 841 ms: 11 candidates, 101374 repo-hours, 1132 releases; waiting 568 ms; commit 4059 ms)
2026-11-08T01:19:44Z [load_day 2023-09-29] 2023-09-29 hour 14: 204638 events in 5 files, snapshot 5058865717764126524 (task 21179 ms: download 11575 ms overlapping gunzip 1263 ms, parse 5462 ms, sort 175 ms, write 12737 ms, index 741 ms: 5 candidates, 89256 repo-hours, 1280 releases; waiting 492 ms; commit 2496 ms)
2026-11-08T01:19:50Z [load_day 2023-09-29] 2023-09-29 hour 15: 189089 events in 4 files, snapshot 8335143601208011019 (task 19135 ms: download 11149 ms overlapping gunzip 1102 ms, parse 5031 ms, sort 132 ms, write 11471 ms, index 602 ms: 13 candidates, 84730 repo-hours, 1146 releases; waiting 507 ms; commit 2333 ms)
2026-11-08T01:20:01Z [load_day 2023-09-29] 2023-09-29 hour 16: 187542 events in 4 files, snapshot 2873613896249369921 (task 15930 ms: download 9157 ms overlapping gunzip 949 ms, parse 4070 ms, sort 132 ms, write 9403 ms, index 628 ms: 12 candidates, 83212 repo-hours, 977 releases; waiting 534 ms; commit 2616 ms)
2026-11-08T01:20:06Z [load_day 2023-09-29] 2023-09-29 hour 17: 180158 events in 4 files, snapshot 3351629341143543625 (task 14983 ms: download 8614 ms overlapping gunzip 886 ms, parse 3829 ms, sort 100 ms, write 8851 ms, index 547 ms: 12 candidates, 77709 repo-hours, 902 releases; waiting 522 ms; commit 2648 ms)
2026-11-08T01:20:18Z [load_day 2023-09-29] 2023-09-29 hour 18: 181313 events in 4 files, snapshot 4738732925773864453 (task 16297 ms: download 8169 ms overlapping gunzip 902 ms, parse 4060 ms, sort 136 ms, write 9716 ms, index 695 ms: 6 candidates, 81520 repo-hours, 1046 releases; waiting 521 ms; commit 2580 ms)
2026-11-08T01:20:23Z [load_day 2023-09-29] 2023-09-29 hour 19: 168642 events in 4 files, snapshot 1873080090459932821 (task 15566 ms: download 6408 ms overlapping gunzip 866 ms, parse 3798 ms, sort 113 ms, write 9545 ms, index 519 ms: 9 candidates, 70794 repo-hours, 985 releases; waiting 398 ms; commit 2619 ms)
2026-11-08T01:20:36Z [load_day 2023-09-29] 2023-09-29 hour 20: 160635 events in 4 files, snapshot 1122628509261918420 (task 16337 ms: download 6250 ms overlapping gunzip 871 ms, parse 3925 ms, sort 205 ms, write 9135 ms, index 890 ms: 4 candidates, 66857 repo-hours, 917 releases; waiting 944 ms; commit 2496 ms)
2026-11-08T01:20:39Z [load_day 2023-09-29] 2023-09-29 hour 21: 148334 events in 3 files, snapshot 919694486396653152 (task 14813 ms: download 6855 ms overlapping gunzip 1000 ms, parse 4315 ms, sort 174 ms, write 7939 ms, index 603 ms: 6 candidates, 58872 repo-hours, 931 releases; waiting 506 ms; commit 2852 ms)
2026-11-08T01:20:49Z [load_day 2023-09-29] 2023-09-29 hour 22: 128179 events in 3 files, snapshot 8140498336508554396 (task 11650 ms: download 5138 ms overlapping gunzip 642 ms, parse 2963 ms, sort 80 ms, write 6391 ms, index 329 ms: 7 candidates, 51506 repo-hours, 846 releases; waiting 515 ms; commit 3029 ms)
2026-11-08T01:20:53Z [load_day 2023-09-29] 2023-09-29 hour 23: 110945 events in 3 files, snapshot 5933431906929216585 (task 9334 ms: download 2410 ms overlapping gunzip 498 ms, parse 2232 ms, sort 93 ms, write 5240 ms, index 336 ms: 5 candidates, 45789 repo-hours, 650 releases; waiting 551 ms; commit 3940 ms)
2026-11-08T01:21:01Z [load_day 2023-09-29] 2023-09-29: repo_hour merged from 24 files to 1 (1706084 rows, 7760 ms)
2026-11-08T01:21:05Z [load_day 2023-09-29] 2023-09-29: index committed in 11819 ms (24 + 1 + 24 files)
2026-11-08T01:21:05Z [load_day 2023-09-29] 2023-09-29: 3864214 events in 24 hours
2026-11-08T01:21:05Z [load_day 2023-09-29] attempt 1 succeeded