orcaLog in

Runs · run 1116

load_day 2023-05-24, attempt 1 succeeded

Host
caf251265124:397996
Started
2026-11-07T20:34:01Z
Ended
2026-11-07T20:37:07Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1116/load_day/2023-05-24/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T20:34:01Z [load_day 2023-05-24] attempt 1 started
2026-11-07T20:34:03Z [load_day 2023-05-24] 2023-05-24: 2 hours in flight, about 1200 MB
2026-11-07T20:34:17Z [load_day 2023-05-24] 2023-05-24 hour 0: 145412 events in 3 files, snapshot 2540522607844942672 (task 11356 ms: download 5050 ms overlapping gunzip 682 ms, parse 3070 ms, sort 83 ms, write 6237 ms, index 517 ms: 6 candidates, 77388 repo-hours, 797 releases; waiting 519 ms; commit 1494 ms)
2026-11-07T20:34:19Z [load_day 2023-05-24] 2023-05-24 hour 1: 160198 events in 4 files, snapshot 4162882251700415738 (task 12117 ms: download 4884 ms overlapping gunzip 717 ms, parse 3261 ms, sort 82 ms, write 7068 ms, index 557 ms: 6 candidates, 88502 repo-hours, 817 releases; waiting 207 ms; commit 1655 ms)
2026-11-07T20:34:29Z [load_day 2023-05-24] 2023-05-24 hour 2: 141856 events in 3 files, snapshot 2659928698545465065 (task 11203 ms: download 4751 ms overlapping gunzip 655 ms, parse 2831 ms, sort 91 ms, write 6300 ms, index 479 ms: 5 candidates, 68857 repo-hours, 827 releases; waiting 517 ms; commit 1548 ms)
2026-11-07T20:34:31Z [load_day 2023-05-24] 2023-05-24 hour 3: 130576 events in 3 files, snapshot 7650886826080414612 (task 9705 ms: download 4099 ms overlapping gunzip 594 ms, parse 2486 ms, sort 80 ms, write 5367 ms, index 398 ms: 4 candidates, 59251 repo-hours, 845 releases; waiting 509 ms; commit 1724 ms)
2026-11-07T20:34:40Z [load_day 2023-05-24] 2023-05-24 hour 4: 127065 events in 3 files, snapshot 4288651074378504453 (task 9381 ms: download 3944 ms overlapping gunzip 546 ms, parse 2431 ms, sort 73 ms, write 5154 ms, index 380 ms: 3 candidates, 54773 repo-hours, 628 releases; waiting 435 ms; commit 1698 ms)
2026-11-07T20:34:43Z [load_day 2023-05-24] 2023-05-24 hour 5: 140569 events in 3 files, snapshot 8866405887875803722 (task 10837 ms: download 4516 ms overlapping gunzip 634 ms, parse 2834 ms, sort 101 ms, write 5862 ms, index 412 ms: 5 candidates, 63533 repo-hours, 811 releases; waiting 651 ms; commit 1814 ms)
2026-11-07T20:34:53Z [load_day 2023-05-24] 2023-05-24 hour 6: 164546 events in 4 files, snapshot 5826956865612105796 (task 12942 ms: download 5006 ms overlapping gunzip 759 ms, parse 3280 ms, sort 95 ms, write 7412 ms, index 537 ms: 11 candidates, 77801 repo-hours, 901 releases; waiting 529 ms; commit 1569 ms)
2026-11-07T20:34:58Z [load_day 2023-05-24] 2023-05-24 hour 7: 162359 events in 4 files, snapshot 7624540008113831025 (task 14175 ms: download 8473 ms overlapping gunzip 797 ms, parse 3590 ms, sort 100 ms, write 8185 ms, index 533 ms: 7 candidates, 76748 repo-hours, 892 releases; waiting 612 ms; commit 1754 ms)
2026-11-07T20:35:09Z [load_day 2023-05-24] 2023-05-24 hour 8: 171351 events in 4 files, snapshot 6062739730071413951 (task 14997 ms: download 8770 ms overlapping gunzip 896 ms, parse 3989 ms, sort 115 ms, write 8603 ms, index 562 ms: 8 candidates, 82147 repo-hours, 998 releases; waiting 516 ms; commit 2007 ms)
2026-11-07T20:35:14Z [load_day 2023-05-24] 2023-05-24 hour 9: 177392 events in 4 files, snapshot 3915379814092306034 (task 15448 ms: download 8937 ms overlapping gunzip 887 ms, parse 3919 ms, sort 105 ms, write 8980 ms, index 567 ms: 11 candidates, 82620 repo-hours, 1346 releases; waiting 695 ms; commit 1736 ms)
2026-11-07T20:35:24Z [load_day 2023-05-24] 2023-05-24 hour 10: 170953 events in 4 files, snapshot 3702327018682553647 (task 14309 ms: download 8487 ms overlapping gunzip 865 ms, parse 3915 ms, sort 116 ms, write 8071 ms, index 528 ms: 4 candidates, 79383 repo-hours, 992 releases; waiting 498 ms; commit 1926 ms)
2026-11-07T20:35:29Z [load_day 2023-05-24] 2023-05-24 hour 11: 173485 events in 4 files, snapshot 8195864878538151953 (task 13895 ms: download 7871 ms overlapping gunzip 816 ms, parse 3576 ms, sort 90 ms, write 7990 ms, index 572 ms: 7 candidates, 82660 repo-hours, 904 releases; waiting 524 ms; commit 1989 ms)
2026-11-07T20:35:41Z [load_day 2023-05-24] 2023-05-24 hour 12: 207949 events in 5 files, snapshot 8330735960318656819 (task 15671 ms: download 7948 ms overlapping gunzip 872 ms, parse 3946 ms, sort 121 ms, write 9182 ms, index 737 ms: 7 candidates, 108277 repo-hours, 1007 releases; waiting 514 ms; commit 1788 ms)
2026-11-07T20:35:47Z [load_day 2023-05-24] 2023-05-24 hour 13: 194200 events in 4 files, snapshot 3523667806242085742 (task 15817 ms: download 8353 ms overlapping gunzip 923 ms, parse 4145 ms, sort 131 ms, write 9282 ms, index 678 ms: 9 candidates, 101460 repo-hours, 1044 releases; waiting 376 ms; commit 3512 ms)
2026-11-07T20:35:59Z [load_day 2023-05-24] 2023-05-24 hour 14: 183111 events in 4 files, snapshot 2465112376876675847 (task 17379 ms: download 10494 ms overlapping gunzip 946 ms, parse 4170 ms, sort 120 ms, write 10790 ms, index 603 ms: 4 candidates, 90946 repo-hours, 1018 releases; waiting 419 ms; commit 1808 ms)
2026-11-07T20:36:03Z [load_day 2023-05-24] 2023-05-24 hour 15: 169955 events in 4 files, snapshot 2939090793658707183 (task 17218 ms: download 10977 ms overlapping gunzip 912 ms, parse 4042 ms, sort 112 ms, write 8911 ms, index 593 ms: 6 candidates, 84497 repo-hours, 1161 releases; waiting 2341 ms; commit 1601 ms)
2026-11-07T20:36:14Z [load_day 2023-05-24] 2023-05-24 hour 16: 166469 events in 4 files, snapshot 7637482388067776421 (task 14323 ms: download 8483 ms overlapping gunzip 833 ms, parse 3682 ms, sort 101 ms, write 8328 ms, index 534 ms: 4 candidates, 80390 repo-hours, 1069 releases; waiting 520 ms; commit 1767 ms)
2026-11-07T20:36:20Z [load_day 2023-05-24] 2023-05-24 hour 17: 181665 events in 4 files, snapshot 7095731561391052088 (task 15617 ms: download 8874 ms overlapping gunzip 942 ms, parse 4170 ms, sort 110 ms, write 8934 ms, index 566 ms: 6 candidates, 82134 repo-hours, 1167 releases; waiting 529 ms; commit 1754 ms)
2026-11-07T20:36:30Z [load_day 2023-05-24] 2023-05-24 hour 18: 180187 events in 4 files, snapshot 2470394648968945732 (task 14956 ms: download 8544 ms overlapping gunzip 873 ms, parse 3915 ms, sort 108 ms, write 8698 ms, index 548 ms: 9 candidates, 84177 repo-hours, 942 releases; waiting 530 ms; commit 1755 ms)
2026-11-07T20:36:34Z [load_day 2023-05-24] 2023-05-24 hour 19: 159077 events in 4 files, snapshot 7135239137253621302 (task 13669 ms: download 8162 ms overlapping gunzip 833 ms, parse 3673 ms, sort 101 ms, write 7759 ms, index 477 ms: 5 candidates, 71558 repo-hours, 952 releases; waiting 499 ms; commit 1793 ms)
2026-11-07T20:36:44Z [load_day 2023-05-24] 2023-05-24 hour 20: 163832 events in 4 files, snapshot 1331025919082997891 (task 13544 ms: download 7976 ms overlapping gunzip 804 ms, parse 3578 ms, sort 101 ms, write 7607 ms, index 487 ms: 7 candidates, 72744 repo-hours, 889 releases; waiting 500 ms; commit 1758 ms)
2026-11-07T20:36:47Z [load_day 2023-05-24] 2023-05-24 hour 21: 150562 events in 4 files, snapshot 259167884267711534 (task 12409 ms: download 5496 ms overlapping gunzip 726 ms, parse 3244 ms, sort 101 ms, write 7011 ms, index 451 ms: 7 candidates, 64046 repo-hours, 1004 releases; waiting 598 ms; commit 1968 ms)
2026-11-07T20:36:56Z [load_day 2023-05-24] 2023-05-24 hour 22: 136347 events in 3 files, snapshot 7729305994235397306 (task 10920 ms: download 4857 ms overlapping gunzip 674 ms, parse 2884 ms, sort 76 ms, write 6061 ms, index 365 ms: 4 candidates, 57193 repo-hours, 833 releases; waiting 517 ms; commit 1719 ms)
2026-11-07T20:36:58Z [load_day 2023-05-24] 2023-05-24 hour 23: 123783 events in 3 files, snapshot 8234889958203060416 (task 10126 ms: download 4523 ms overlapping gunzip 624 ms, parse 2767 ms, sort 69 ms, write 5582 ms, index 337 ms: 9 candidates, 53182 repo-hours, 712 releases; waiting 503 ms; commit 1654 ms)
2026-11-07T20:37:05Z [load_day 2023-05-24] 2023-05-24: repo_hour merged from 24 files to 1 (1844267 rows, 7465 ms)
2026-11-07T20:37:07Z [load_day 2023-05-24] 2023-05-24: index committed in 9483 ms (24 + 1 + 24 files)
2026-11-07T20:37:07Z [load_day 2023-05-24] 2023-05-24: 3882899 events in 24 hours
2026-11-07T20:37:07Z [load_day 2023-05-24] attempt 1 succeeded