orcaLog in

Runs · run 1727

load_day 2023-11-11, attempt 1 succeeded

Host
3c2b29713ce8:543724
Started
2026-11-08T02:41:58Z
Ended
2026-11-08T02:44:57Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1727/load_day/2023-11-11/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T02:41:58Z [load_day 2023-11-11] attempt 1 started
2026-11-08T02:41:59Z [load_day 2023-11-11] 2023-11-11: 2 hours in flight, about 1200 MB
2026-11-08T02:42:17Z [load_day 2023-11-11] 2023-11-11 hour 0: 155335 events in 4 files, snapshot 6923129792574793877 (task 13163 ms: download 5848 ms overlapping gunzip 736 ms, parse 3259 ms, sort 82 ms, write 8343 ms, index 534 ms: 8 candidates, 84921 repo-hours, 979 releases; waiting 21 ms; commit 2929 ms)
2026-11-08T02:42:25Z [load_day 2023-11-11] 2023-11-11 hour 1: 81224 events in 2 files, snapshot 6229479270614640772 (task 7572 ms: download 853 ms overlapping gunzip 405 ms, parse 1958 ms, sort 44 ms, write 4388 ms, index 542 ms: 4 candidates, 48605 repo-hours, 484 releases; waiting 24 ms; commit 8283 ms)
2026-11-08T02:42:31Z [load_day 2023-11-11] 2023-11-11 hour 2: 137498 events in 3 files, snapshot 174962975219952102 (task 13636 ms: download 3978 ms overlapping gunzip 767 ms, parse 3087 ms, sort 134 ms, write 8791 ms, index 641 ms: 2 candidates, 70648 repo-hours, 754 releases; waiting 22 ms; commit 2623 ms)
2026-11-08T02:42:34Z [load_day 2023-11-11] 2023-11-11 hour 3: 111726 events in 3 files, snapshot 5884016884772536441 (task 12698 ms: download 2873 ms overlapping gunzip 844 ms, parse 3834 ms, sort 70 ms, write 6231 ms, index 761 ms: 10 candidates, 47188 repo-hours, 724 releases; waiting 467 ms; commit 2775 ms)
2026-11-08T02:42:41Z [load_day 2023-11-11] 2023-11-11 hour 4: 104348 events in 3 files, snapshot 8283887286318480530 (task 8901 ms: download 1984 ms overlapping gunzip 525 ms, parse 2296 ms, sort 92 ms, write 5316 ms, index 403 ms: 3 candidates, 43604 repo-hours, 572 releases; waiting 59 ms; commit 2524 ms)
2026-11-08T02:43:01Z [load_day 2023-11-11] 2023-11-11 hour 5: 109005 events in 3 files, snapshot 2609971988825193151 (task 9238 ms: download 1893 ms overlapping gunzip 591 ms, parse 2400 ms, sort 64 ms, write 5108 ms, index 363 ms: 5 candidates, 45665 repo-hours, 590 releases; waiting 478 ms; commit 19404 ms)
2026-11-08T02:43:06Z [load_day 2023-11-11] 2023-11-11 hour 6: 117634 events in 3 files, snapshot 7131081546368030570 (task 9830 ms: download 1051 ms overlapping gunzip 579 ms, parse 2436 ms, sort 82 ms, write 5479 ms, index 1038 ms: 4 candidates, 52689 repo-hours, 634 releases; waiting 21 ms; commit 4514 ms)
2026-11-08T02:43:11Z [load_day 2023-11-11] 2023-11-11 hour 7: 112251 events in 3 files, snapshot 8838333260875631893 (task 11226 ms: download 1618 ms overlapping gunzip 619 ms, parse 2420 ms, sort 69 ms, write 7549 ms, index 349 ms: 2 candidates, 46826 repo-hours, 548 releases; waiting 25 ms; commit 5119 ms)
2026-11-08T02:43:16Z [load_day 2023-11-11] 2023-11-11 hour 8: 118752 events in 3 files, snapshot 1537384987580228299 (task 11169 ms: download 1541 ms overlapping gunzip 769 ms, parse 3293 ms, sort 88 ms, write 6379 ms, index 425 ms: 9 candidates, 50193 repo-hours, 630 releases; waiting 17 ms; commit 2881 ms)
2026-11-08T02:43:20Z [load_day 2023-11-11] 2023-11-11 hour 9: 125827 events in 3 files, snapshot 889799552060320663 (task 10696 ms: download 1600 ms overlapping gunzip 667 ms, parse 2935 ms, sort 78 ms, write 6431 ms, index 370 ms: 7 candidates, 51175 repo-hours, 755 releases; waiting 25 ms; commit 2619 ms)
2026-11-08T02:43:29Z [load_day 2023-11-11] 2023-11-11 hour 10: 129387 events in 3 files, snapshot 6215596040556993246 (task 10582 ms: download 3516 ms overlapping gunzip 641 ms, parse 2671 ms, sort 119 ms, write 6176 ms, index 764 ms: 6 candidates, 53675 repo-hours, 606 releases; waiting 19 ms; commit 3963 ms)
2026-11-08T02:43:32Z [load_day 2023-11-11] 2023-11-11 hour 11: 123509 events in 3 files, snapshot 1993511128358431389 (task 10966 ms: download 1430 ms overlapping gunzip 711 ms, parse 2978 ms, sort 95 ms, write 6452 ms, index 496 ms: 5 candidates, 51179 repo-hours, 654 releases; waiting 24 ms; commit 2805 ms)
2026-11-08T02:43:45Z [load_day 2023-11-11] 2023-11-11 hour 12: 197421 events in 4 files, snapshot 5992300943007054270 (task 13147 ms: download 4507 ms overlapping gunzip 701 ms, parse 3643 ms, sort 184 ms, write 7852 ms, index 553 ms: 7 candidates, 85715 repo-hours, 681 releases; waiting 23 ms; commit 5107 ms)
2026-11-08T02:43:53Z [load_day 2023-11-11] 2023-11-11 hour 13: 159582 events in 4 files, snapshot 4285441177849483654 (task 12357 ms: download 4368 ms overlapping gunzip 699 ms, parse 3149 ms, sort 90 ms, write 7133 ms, index 1043 ms: 6 candidates, 80240 repo-hours, 718 releases; waiting 25 ms; commit 8028 ms)
2026-11-08T02:43:58Z [load_day 2023-11-11] 2023-11-11 hour 14: 151820 events in 4 files, snapshot 6470820883229310657 (task 14424 ms: download 6034 ms overlapping gunzip 901 ms, parse 3711 ms, sort 121 ms, write 8462 ms, index 434 ms: 5 candidates, 65359 repo-hours, 800 releases; waiting 513 ms; commit 2790 ms)
2026-11-08T02:44:02Z [load_day 2023-11-11] 2023-11-11 hour 15: 153800 events in 4 files, snapshot 2270013942692840245 (task 13252 ms: download 5348 ms overlapping gunzip 853 ms, parse 3563 ms, sort 176 ms, write 7829 ms, index 605 ms: 4 candidates, 68254 repo-hours, 935 releases; waiting 22 ms; commit 2841 ms)
2026-11-08T02:44:10Z [load_day 2023-11-11] 2023-11-11 hour 16: 149200 events in 3 files, snapshot 4322309547227241098 (task 11290 ms: download 4333 ms overlapping gunzip 669 ms, parse 2896 ms, sort 134 ms, write 6921 ms, index 451 ms: 13 candidates, 66480 repo-hours, 742 releases; waiting 18 ms; commit 2419 ms)
2026-11-08T02:44:15Z [load_day 2023-11-11] 2023-11-11 hour 17: 137610 events in 3 files, snapshot 6507324671178871285 (task 8990 ms: download 3467 ms overlapping gunzip 572 ms, parse 2410 ms, sort 86 ms, write 5281 ms, index 440 ms: 14 candidates, 59448 repo-hours, 691 releases; waiting 19 ms; commit 4616 ms)
2026-11-08T02:44:21Z [load_day 2023-11-11] 2023-11-11 hour 18: 145917 events in 3 files, snapshot 4246630714938016439 (task 9452 ms: download 3347 ms overlapping gunzip 556 ms, parse 2518 ms, sort 89 ms, write 5608 ms, index 444 ms: 4 candidates, 65842 repo-hours, 887 releases; waiting 23 ms; commit 2287 ms)
2026-11-08T02:44:23Z [load_day 2023-11-11] 2023-11-11 hour 19: 130724 events in 3 files, snapshot 3973441930892400521 (task 8387 ms: download 995 ms overlapping gunzip 493 ms, parse 2189 ms, sort 63 ms, write 5092 ms, index 348 ms: 5 candidates, 54166 repo-hours, 780 releases; waiting 17 ms; commit 2280 ms)
2026-11-08T02:44:32Z [load_day 2023-11-11] 2023-11-11 hour 20: 129345 events in 3 files, snapshot 7081848086605108613 (task 8107 ms: download 941 ms overlapping gunzip 470 ms, parse 2194 ms, sort 78 ms, write 4770 ms, index 389 ms: 7 candidates, 53976 repo-hours, 746 releases; waiting 16 ms; commit 3886 ms)
2026-11-08T02:44:34Z [load_day 2023-11-11] 2023-11-11 hour 21: 119474 events in 3 files, snapshot 7051462706003677765 (task 8886 ms: download 857 ms overlapping gunzip 541 ms, parse 2341 ms, sort 90 ms, write 5126 ms, index 584 ms: 4 candidates, 48992 repo-hours, 797 releases; waiting 18 ms; commit 2812 ms)
2026-11-08T02:44:45Z [load_day 2023-11-11] 2023-11-11 hour 22: 111633 events in 3 files, snapshot 3468429475433663494 (task 9361 ms: download 1319 ms overlapping gunzip 690 ms, parse 2807 ms, sort 73 ms, write 5211 ms, index 357 ms: 8 candidates, 45509 repo-hours, 757 releases; waiting 24 ms; commit 7001 ms)
2026-11-08T02:44:48Z [load_day 2023-11-11] 2023-11-11 hour 23: 102264 events in 3 files, snapshot 3108883477598623351 (task 7960 ms: download 1083 ms overlapping gunzip 539 ms, parse 2305 ms, sort 63 ms, write 4465 ms, index 364 ms: 9 candidates, 43369 repo-hours, 641 releases; waiting 17 ms; commit 2820 ms)
2026-11-08T02:44:55Z [load_day 2023-11-11] 2023-11-11: repo_hour merged from 24 files to 1 (1383718 rows, 6495 ms)
2026-11-08T02:44:57Z [load_day 2023-11-11] 2023-11-11: index committed in 8645 ms (24 + 1 + 24 files)
2026-11-08T02:44:57Z [load_day 2023-11-11] 2023-11-11: 3115286 events in 24 hours
2026-11-08T02:44:57Z [load_day 2023-11-11] attempt 1 succeeded