orcaLog in

Runs · run 1209

load_day 2023-06-24, attempt 1 succeeded

Host
3c2b29713ce8:32718
Started
2026-11-07T21:58:38Z
Ended
2026-11-07T22:01:52Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1209/load_day/2023-06-24/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T21:58:38Z [load_day 2023-06-24] attempt 1 started
2026-11-07T21:58:40Z [load_day 2023-06-24] 2023-06-24: 1 hours in flight, about 750 MB
2026-11-07T21:58:51Z [load_day 2023-06-24] 2023-06-24 hour 0: 118107 events in 3 files, snapshot 2185491574511128003 (task 7909 ms: download 2057 ms overlapping gunzip 526 ms, parse 2368 ms, sort 69 ms, write 3782 ms, index 349 ms: 7 candidates, 57101 repo-hours, 692 releases; waiting 517 ms; commit 1833 ms)
2026-11-07T21:59:01Z [load_day 2023-06-24] 2023-06-24 hour 1: 139609 events in 3 files, snapshot 220254511596900387 (task 9128 ms: download 3969 ms overlapping gunzip 595 ms, parse 2616 ms, sort 107 ms, write 4367 ms, index 451 ms: 1 candidates, 69002 repo-hours, 726 releases; waiting 648 ms; commit 2144 ms)
2026-11-07T21:59:08Z [load_day 2023-06-24] 2023-06-24 hour 2: 103970 events in 3 files, snapshot 7365017677426620579 (task 6397 ms: download 1966 ms overlapping gunzip 398 ms, parse 1712 ms, sort 56 ms, write 3079 ms, index 312 ms: 2 candidates, 50098 repo-hours, 571 releases; waiting 505 ms; commit 1804 ms)
2026-11-07T21:59:14Z [load_day 2023-06-24] 2023-06-24 hour 3: 96857 events in 2 files, snapshot 1515306418516322823 (task 5833 ms: download 1825 ms overlapping gunzip 370 ms, parse 1595 ms, sort 52 ms, write 2786 ms, index 283 ms: 1 candidates, 41020 repo-hours, 684 releases; waiting 454 ms; commit 1652 ms)
2026-11-07T21:59:21Z [load_day 2023-06-24] 2023-06-24 hour 4: 100155 events in 3 files, snapshot 1283539176366977212 (task 6282 ms: download 1943 ms overlapping gunzip 393 ms, parse 1666 ms, sort 67 ms, write 2963 ms, index 316 ms: 3 candidates, 41487 repo-hours, 535 releases; waiting 519 ms; commit 1721 ms)
2026-11-07T21:59:28Z [load_day 2023-06-24] 2023-06-24 hour 5: 103993 events in 3 files, snapshot 4953380665885862696 (task 6254 ms: download 1878 ms overlapping gunzip 420 ms, parse 1834 ms, sort 75 ms, write 2838 ms, index 297 ms: 3 candidates, 42645 repo-hours, 607 releases; waiting 502 ms; commit 1640 ms)
2026-11-07T21:59:36Z [load_day 2023-06-24] 2023-06-24 hour 6: 111107 events in 3 files, snapshot 208848774305345404 (task 6270 ms: download 2418 ms overlapping gunzip 379 ms, parse 1693 ms, sort 47 ms, write 2928 ms, index 314 ms: 5 candidates, 50483 repo-hours, 638 releases; waiting 646 ms; commit 1825 ms)
2026-11-07T21:59:43Z [load_day 2023-06-24] 2023-06-24 hour 7: 108834 events in 3 files, snapshot 5722516074919919677 (task 6786 ms: download 2587 ms overlapping gunzip 399 ms, parse 1734 ms, sort 71 ms, write 3016 ms, index 306 ms: 3 candidates, 46793 repo-hours, 626 releases; waiting 828 ms; commit 1930 ms)
2026-11-07T21:59:51Z [load_day 2023-06-24] 2023-06-24 hour 8: 113493 events in 3 files, snapshot 1966503828671750694 (task 6568 ms: download 1788 ms overlapping gunzip 415 ms, parse 1834 ms, sort 72 ms, write 3105 ms, index 309 ms: 2 candidates, 50003 repo-hours, 545 releases; waiting 506 ms; commit 1943 ms)
2026-11-07T21:59:59Z [load_day 2023-06-24] 2023-06-24 hour 9: 119146 events in 3 files, snapshot 6131325544225580423 (task 6854 ms: download 2163 ms overlapping gunzip 431 ms, parse 1915 ms, sort 64 ms, write 3306 ms, index 318 ms: 2 candidates, 51092 repo-hours, 699 releases; waiting 511 ms; commit 2169 ms)
2026-11-07T22:00:06Z [load_day 2023-06-24] 2023-06-24 hour 10: 117231 events in 3 files, snapshot 3643870456487237328 (task 7030 ms: download 2002 ms overlapping gunzip 445 ms, parse 1947 ms, sort 71 ms, write 3441 ms, index 351 ms: 5 candidates, 52094 repo-hours, 597 releases; waiting 498 ms; commit 1927 ms)
2026-11-07T22:00:14Z [load_day 2023-06-24] 2023-06-24 hour 11: 116947 events in 3 files, snapshot 8621592488654036610 (task 6866 ms: download 2122 ms overlapping gunzip 425 ms, parse 1848 ms, sort 77 ms, write 3196 ms, index 322 ms: 7 candidates, 48354 repo-hours, 627 releases; waiting 646 ms; commit 1836 ms)
2026-11-07T22:00:24Z [load_day 2023-06-24] 2023-06-24 hour 12: 169636 events in 4 files, snapshot 3970765999632935663 (task 8664 ms: download 3123 ms overlapping gunzip 527 ms, parse 2451 ms, sort 95 ms, write 4249 ms, index 535 ms: 7 candidates, 82286 repo-hours, 744 releases; waiting 517 ms; commit 1784 ms)
2026-11-07T22:00:33Z [load_day 2023-06-24] 2023-06-24 hour 13: 148424 events in 3 files, snapshot 1110050752526317405 (task 7968 ms: download 2920 ms overlapping gunzip 482 ms, parse 2202 ms, sort 92 ms, write 3912 ms, index 475 ms: 5 candidates, 76111 repo-hours, 634 releases; waiting 502 ms; commit 1999 ms)
2026-11-07T22:00:41Z [load_day 2023-06-24] 2023-06-24 hour 14: 134158 events in 3 files, snapshot 7136038690627945381 (task 7327 ms: download 2118 ms overlapping gunzip 456 ms, parse 2052 ms, sort 84 ms, write 3521 ms, index 374 ms: 6 candidates, 58871 repo-hours, 706 releases; waiting 524 ms; commit 1940 ms)
2026-11-07T22:00:49Z [load_day 2023-06-24] 2023-06-24 hour 15: 134418 events in 3 files, snapshot 8360205330090983175 (task 7597 ms: download 2640 ms overlapping gunzip 469 ms, parse 2088 ms, sort 76 ms, write 3669 ms, index 407 ms: 3 candidates, 61028 repo-hours, 785 releases; waiting 508 ms; commit 2001 ms)
2026-11-07T22:00:57Z [load_day 2023-06-24] 2023-06-24 hour 16: 131755 events in 3 files, snapshot 9110287589954894467 (task 7228 ms: download 1949 ms overlapping gunzip 446 ms, parse 2048 ms, sort 77 ms, write 3468 ms, index 402 ms: 8 candidates, 57306 repo-hours, 637 releases; waiting 512 ms; commit 1816 ms)
2026-11-07T22:01:05Z [load_day 2023-06-24] 2023-06-24 hour 17: 121617 events in 3 files, snapshot 48765177413565140 (task 6975 ms: download 1857 ms overlapping gunzip 447 ms, parse 2049 ms, sort 76 ms, write 3261 ms, index 330 ms: 5 candidates, 52436 repo-hours, 634 releases; waiting 514 ms; commit 1967 ms)
2026-11-07T22:01:13Z [load_day 2023-06-24] 2023-06-24 hour 18: 127090 events in 3 files, snapshot 1792509413062679831 (task 6798 ms: download 1898 ms overlapping gunzip 413 ms, parse 1933 ms, sort 81 ms, write 3211 ms, index 401 ms: 6 candidates, 57710 repo-hours, 730 releases; waiting 502 ms; commit 2059 ms)
2026-11-07T22:01:20Z [load_day 2023-06-24] 2023-06-24 hour 19: 109457 events in 3 files, snapshot 4582336360570359263 (task 6351 ms: download 1673 ms overlapping gunzip 415 ms, parse 1887 ms, sort 69 ms, write 2831 ms, index 307 ms: 8 candidates, 45413 repo-hours, 636 releases; waiting 513 ms; commit 2077 ms)
2026-11-07T22:01:27Z [load_day 2023-06-24] 2023-06-24 hour 20: 105693 events in 3 files, snapshot 361924075118231668 (task 5993 ms: download 2116 ms overlapping gunzip 345 ms, parse 1604 ms, sort 67 ms, write 2675 ms, index 310 ms: 6 candidates, 45565 repo-hours, 555 releases; waiting 674 ms; commit 1793 ms)
2026-11-07T22:01:33Z [load_day 2023-06-24] 2023-06-24 hour 21: 97375 events in 2 files, snapshot 4665993951960943722 (task 5462 ms: download 1657 ms overlapping gunzip 325 ms, parse 1513 ms, sort 63 ms, write 2509 ms, index 251 ms: 6 candidates, 40155 repo-hours, 681 releases; waiting 499 ms; commit 1554 ms)
2026-11-07T22:01:39Z [load_day 2023-06-24] 2023-06-24 hour 22: 88748 events in 2 files, snapshot 6850393489054308990 (task 5305 ms: download 1697 ms overlapping gunzip 346 ms, parse 1496 ms, sort 50 ms, write 2346 ms, index 257 ms: 6 candidates, 37015 repo-hours, 603 releases; waiting 500 ms; commit 1875 ms)
2026-11-07T22:01:45Z [load_day 2023-06-24] 2023-06-24 hour 23: 84458 events in 2 files, snapshot 7759796069744685437 (task 4951 ms: download 1430 ms overlapping gunzip 313 ms, parse 1417 ms, sort 40 ms, write 2138 ms, index 240 ms: 3 candidates, 35838 repo-hours, 532 releases; waiting 506 ms; commit 1811 ms)
2026-11-07T22:01:50Z [load_day 2023-06-24] 2023-06-24: repo_hour merged from 24 files to 1 (1249906 rows, 5493 ms)
2026-11-07T22:01:52Z [load_day 2023-06-24] 2023-06-24: index committed in 7516 ms (24 + 1 + 24 files)
2026-11-07T22:01:52Z [load_day 2023-06-24] 2023-06-24: 2802278 events in 24 hours
2026-11-07T22:01:52Z [load_day 2023-06-24] attempt 1 succeeded