orcaLog in

Runs · run 1653

load_day 2023-10-22, attempt 1 succeeded

Host
3c2b29713ce8:467740
Started
2026-11-08T02:02:39Z
Ended
2026-11-08T02:05:19Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1653/load_day/2023-10-22/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:02:39Z [load_day 2023-10-22] attempt 1 started
2026-11-08T02:02:41Z [load_day 2023-10-22] 2023-10-22: 2 hours in flight, about 1200 MB
2026-11-08T02:02:54Z [load_day 2023-10-22] 2023-10-22 hour 0: 120738 events in 3 files, snapshot 5056736161087726439 (task 9969 ms: download 2167 ms overlapping gunzip 606 ms, parse 2893 ms, sort 116 ms, write 4996 ms, index 537 ms: 10 candidates, 68579 repo-hours, 918 releases; waiting 502 ms; commit 2277 ms)
2026-11-08T02:03:02Z [load_day 2023-10-22] 2023-10-22 hour 1: 126960 events in 3 files, snapshot 1056184117360736967 (task 10182 ms: download 1751 ms overlapping gunzip 568 ms, parse 2633 ms, sort 88 ms, write 5564 ms, index 582 ms: 5 candidates, 74572 repo-hours, 712 releases; waiting 499 ms; commit 7420 ms)
2026-11-08T02:03:06Z [load_day 2023-10-22] 2023-10-22 hour 2: 95585 events in 2 files, snapshot 1289882797491777396 (task 7624 ms: download 1784 ms overlapping gunzip 484 ms, parse 2129 ms, sort 54 ms, write 3830 ms, index 267 ms: 6 candidates, 40556 repo-hours, 580 releases; waiting 527 ms; commit 4603 ms)
2026-11-08T02:03:11Z [load_day 2023-10-22] 2023-10-22 hour 3: 97675 events in 2 files, snapshot 4859219927910253562 (task 7915 ms: download 2047 ms overlapping gunzip 400 ms, parse 1986 ms, sort 80 ms, write 4228 ms, index 342 ms: 5 candidates, 40905 repo-hours, 688 releases; waiting 533 ms; commit 4809 ms)
2026-11-08T02:03:14Z [load_day 2023-10-22] 2023-10-22 hour 4: 98847 events in 2 files, snapshot 8975695000758836779 (task 7595 ms: download 1843 ms overlapping gunzip 370 ms, parse 1790 ms, sort 66 ms, write 4273 ms, index 357 ms: 6 candidates, 41114 repo-hours, 506 releases; waiting 493 ms; commit 3089 ms)
2026-11-08T02:03:19Z [load_day 2023-10-22] 2023-10-22 hour 5: 100514 events in 3 files, snapshot 4386764868202285742 (task 9141 ms: download 2101 ms overlapping gunzip 509 ms, parse 2326 ms, sort 65 ms, write 4531 ms, index 1003 ms: 11 candidates, 43045 repo-hours, 903 releases; waiting 478 ms; commit 2670 ms)
2026-11-08T02:03:25Z [load_day 2023-10-22] 2023-10-22 hour 6: 114873 events in 3 files, snapshot 8705164868373998023 (task 8706 ms: download 2208 ms overlapping gunzip 462 ms, parse 2164 ms, sort 106 ms, write 4568 ms, index 594 ms: 8 candidates, 51892 repo-hours, 701 releases; waiting 513 ms; commit 3800 ms)
2026-11-08T02:03:30Z [load_day 2023-10-22] 2023-10-22 hour 7: 110398 events in 3 files, snapshot 7175802411201080007 (task 9657 ms: download 1833 ms overlapping gunzip 516 ms, parse 2472 ms, sort 61 ms, write 5361 ms, index 438 ms: 9 candidates, 47540 repo-hours, 504 releases; waiting 494 ms; commit 2385 ms)
2026-11-08T02:03:34Z [load_day 2023-10-22] 2023-10-22 hour 8: 120372 events in 3 files, snapshot 2320372906580799126 (task 9746 ms: download 2215 ms overlapping gunzip 507 ms, parse 2253 ms, sort 101 ms, write 5377 ms, index 383 ms: 9 candidates, 52210 repo-hours, 566 releases; waiting 492 ms; commit 2211 ms)
2026-11-08T02:03:40Z [load_day 2023-10-22] 2023-10-22 hour 9: 124379 events in 3 files, snapshot 3789447518093902417 (task 8623 ms: download 2244 ms overlapping gunzip 540 ms, parse 2374 ms, sort 57 ms, write 4428 ms, index 348 ms: 6 candidates, 52010 repo-hours, 735 releases; waiting 519 ms; commit 2393 ms)
2026-11-08T02:03:44Z [load_day 2023-10-22] 2023-10-22 hour 10: 127437 events in 3 files, snapshot 1093238565357644560 (task 8463 ms: download 2037 ms overlapping gunzip 474 ms, parse 2169 ms, sort 75 ms, write 4546 ms, index 414 ms: 11 candidates, 55525 repo-hours, 671 releases; waiting 511 ms; commit 2377 ms)
2026-11-08T02:03:50Z [load_day 2023-10-22] 2023-10-22 hour 11: 126526 events in 3 files, snapshot 2730238650499605205 (task 8201 ms: download 2098 ms overlapping gunzip 487 ms, parse 2161 ms, sort 75 ms, write 4518 ms, index 414 ms: 5 candidates, 53929 repo-hours, 704 releases; waiting 201 ms; commit 2662 ms)
2026-11-08T02:03:57Z [load_day 2023-10-22] 2023-10-22 hour 12: 201826 events in 5 files, snapshot 3690308658621164883 (task 11365 ms: download 3580 ms overlapping gunzip 584 ms, parse 2880 ms, sort 104 ms, write 6031 ms, index 937 ms: 10 candidates, 87616 repo-hours, 774 releases; waiting 518 ms; commit 3257 ms)
2026-11-08T02:04:04Z [load_day 2023-10-22] 2023-10-22 hour 13: 163013 events in 4 files, snapshot 7507073647041388533 (task 12634 ms: download 3315 ms overlapping gunzip 607 ms, parse 2901 ms, sort 135 ms, write 7579 ms, index 754 ms: 8 candidates, 81714 repo-hours, 805 releases; waiting 375 ms; commit 3229 ms)
2026-11-08T02:04:12Z [load_day 2023-10-22] 2023-10-22 hour 14: 152648 events in 4 files, snapshot 7796283907271409238 (task 12962 ms: download 4969 ms overlapping gunzip 837 ms, parse 3883 ms, sort 110 ms, write 6863 ms, index 552 ms: 11 candidates, 67680 repo-hours, 803 releases; waiting 486 ms; commit 3839 ms)
2026-11-08T02:04:18Z [load_day 2023-10-22] 2023-10-22 hour 15: 158135 events in 4 files, snapshot 7571727341168521070 (task 13412 ms: download 4412 ms overlapping gunzip 853 ms, parse 3683 ms, sort 90 ms, write 7644 ms, index 632 ms: 10 candidates, 70149 repo-hours, 935 releases; waiting 191 ms; commit 2904 ms)
2026-11-08T02:04:25Z [load_day 2023-10-22] 2023-10-22 hour 16: 155123 events in 4 files, snapshot 8146802689947114202 (task 13108 ms: download 5201 ms overlapping gunzip 703 ms, parse 3144 ms, sort 133 ms, write 8144 ms, index 525 ms: 10 candidates, 66960 repo-hours, 829 releases; waiting 207 ms; commit 3218 ms)
2026-11-08T02:04:32Z [load_day 2023-10-22] 2023-10-22 hour 17: 151950 events in 4 files, snapshot 7376625951572388236 (task 12083 ms: download 4622 ms overlapping gunzip 669 ms, parse 3042 ms, sort 137 ms, write 7092 ms, index 389 ms: 8 candidates, 63862 repo-hours, 761 releases; waiting 491 ms; commit 3698 ms)
2026-11-08T02:04:46Z [load_day 2023-10-22] 2023-10-22 hour 18: 158866 events in 4 files, snapshot 5844886291409412367 (task 13074 ms: download 4704 ms overlapping gunzip 775 ms, parse 3565 ms, sort 89 ms, write 7548 ms, index 608 ms: 2 candidates, 70184 repo-hours, 838 releases; waiting 197 ms; commit 10145 ms)
2026-11-08T02:04:49Z [load_day 2023-10-22] 2023-10-22 hour 19: 136398 events in 3 files, snapshot 482930167284864819 (task 13400 ms: download 4851 ms overlapping gunzip 847 ms, parse 4029 ms, sort 122 ms, write 6942 ms, index 672 ms: 3 candidates, 57674 repo-hours, 796 releases; waiting 531 ms; commit 2675 ms)
2026-11-08T02:04:52Z [load_day 2023-10-22] 2023-10-22 hour 20: 133647 events in 3 files, snapshot 5398360953047745326 (task 11641 ms: download 3041 ms overlapping gunzip 675 ms, parse 2989 ms, sort 129 ms, write 6575 ms, index 488 ms: 8 candidates, 56776 repo-hours, 784 releases; waiting 497 ms; commit 3093 ms)
2026-11-08T02:05:02Z [load_day 2023-10-22] 2023-10-22 hour 21: 122980 events in 3 files, snapshot 5798190430851127239 (task 10028 ms: download 2363 ms overlapping gunzip 548 ms, parse 2451 ms, sort 64 ms, write 5216 ms, index 879 ms: 7 candidates, 50760 repo-hours, 783 releases; waiting 570 ms; commit 4407 ms)
2026-11-08T02:05:04Z [load_day 2023-10-22] 2023-10-22 hour 22: 110555 events in 3 files, snapshot 5695107695142144151 (task 8586 ms: download 1862 ms overlapping gunzip 431 ms, parse 1952 ms, sort 53 ms, write 4952 ms, index 386 ms: 12 candidates, 45644 repo-hours, 720 releases; waiting 536 ms; commit 2276 ms)
2026-11-08T02:05:10Z [load_day 2023-10-22] 2023-10-22 hour 23: 104133 events in 3 files, snapshot 5091742968447775810 (task 8642 ms: download 2429 ms overlapping gunzip 529 ms, parse 2317 ms, sort 79 ms, write 4600 ms, index 324 ms: 7 candidates, 44155 repo-hours, 647 releases; waiting 489 ms; commit 2343 ms)
2026-11-08T02:05:17Z [load_day 2023-10-22] 2023-10-22: repo_hour merged from 24 files to 1 (1385051 rows, 6525 ms)
2026-11-08T02:05:19Z [load_day 2023-10-22] 2023-10-22: index committed in 8840 ms (24 + 1 + 24 files)
2026-11-08T02:05:19Z [load_day 2023-10-22] 2023-10-22: 3113578 events in 24 hours
2026-11-08T02:05:19Z [load_day 2023-10-22] attempt 1 succeeded