orcaLog in

Runs · run 1017

load_day 2023-04-21, attempt 1 succeeded

Host
caf251265124:291648
Started
2026-11-07T18:59:43Z
Ended
2026-11-07T19:03:06Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1017/load_day/2023-04-21/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T18:59:43Z [load_day 2023-04-21] attempt 1 started
2026-11-07T18:59:44Z [load_day 2023-04-21] 2023-04-21: 2 hours in flight, about 1200 MB
2026-11-07T18:59:58Z [load_day 2023-04-21] 2023-04-21 hour 0: 151961 events in 4 files, snapshot 9026346017580446524 (task 11586 ms: download 4760 ms overlapping gunzip 614 ms, parse 2816 ms, sort 104 ms, write 6594 ms, index 545 ms: 4 candidates, 77349 repo-hours, 994 releases; waiting 521 ms; commit 1522 ms)
2026-11-07T19:00:02Z [load_day 2023-04-21] 2023-04-21 hour 1: 187234 events in 4 files, snapshot 1559238526910843920 (task 15999 ms: download 9336 ms overlapping gunzip 1092 ms, parse 4290 ms, sort 138 ms, write 8987 ms, index 611 ms: 5 candidates, 87341 repo-hours, 772 releases; waiting 504 ms; commit 1556 ms)
2026-11-07T19:00:14Z [load_day 2023-04-21] 2023-04-21 hour 2: 167967 events in 4 files, snapshot 4363469733606805828 (task 14937 ms: download 9171 ms overlapping gunzip 1080 ms, parse 4093 ms, sort 118 ms, write 8683 ms, index 423 ms: 4 candidates, 65186 repo-hours, 810 releases; waiting 194 ms; commit 1731 ms)
2026-11-07T19:00:20Z [load_day 2023-04-21] 2023-04-21 hour 3: 160624 events in 4 files, snapshot 9211197167697179067 (task 17260 ms: download 11895 ms overlapping gunzip 1085 ms, parse 4322 ms, sort 97 ms, write 7866 ms, index 413 ms: 2 candidates, 60262 repo-hours, 800 releases; waiting 3119 ms; commit 1411 ms)
2026-11-07T19:00:30Z [load_day 2023-04-21] 2023-04-21 hour 4: 148537 events in 3 files, snapshot 2659666440603958003 (task 15681 ms: download 10955 ms overlapping gunzip 993 ms, parse 3925 ms, sort 96 ms, write 7378 ms, index 389 ms: 1 candidates, 54309 repo-hours, 614 releases; waiting 2618 ms; commit 1620 ms)
2026-11-07T19:00:36Z [load_day 2023-04-21] 2023-04-21 hour 5: 159920 events in 4 files, snapshot 3168269622813739618 (task 14964 ms: download 9401 ms overlapping gunzip 1048 ms, parse 4206 ms, sort 108 ms, write 8108 ms, index 427 ms: 1 candidates, 60838 repo-hours, 828 releases; waiting 634 ms; commit 1527 ms)
2026-11-07T19:00:49Z [load_day 2023-04-21] 2023-04-21 hour 6: 186603 events in 4 files, snapshot 5178594029498416991 (task 17843 ms: download 10971 ms overlapping gunzip 1133 ms, parse 4682 ms, sort 123 ms, write 9225 ms, index 509 ms: 2 candidates, 76869 repo-hours, 895 releases; waiting 1903 ms; commit 1629 ms)
2026-11-07T19:00:55Z [load_day 2023-04-21] 2023-04-21 hour 7: 188870 events in 4 files, snapshot 151012689054153133 (task 17324 ms: download 10038 ms overlapping gunzip 1154 ms, parse 4743 ms, sort 112 ms, write 9744 ms, index 518 ms: 5 candidates, 78084 repo-hours, 962 releases; waiting 731 ms; commit 1796 ms)
2026-11-07T19:01:06Z [load_day 2023-04-21] 2023-04-21 hour 8: 178333 events in 4 files, snapshot 7721410528194026983 (task 15914 ms: download 9411 ms overlapping gunzip 948 ms, parse 3969 ms, sort 105 ms, write 9206 ms, index 536 ms: 2 candidates, 80790 repo-hours, 1078 releases; waiting 827 ms; commit 1558 ms)
2026-11-07T19:01:11Z [load_day 2023-04-21] 2023-04-21 hour 9: 182824 events in 4 files, snapshot 8768397682815152121 (task 15781 ms: download 8733 ms overlapping gunzip 903 ms, parse 3954 ms, sort 114 ms, write 9206 ms, index 600 ms: 3 candidates, 81805 repo-hours, 1236 releases; waiting 690 ms; commit 1462 ms)
2026-11-07T19:01:21Z [load_day 2023-04-21] 2023-04-21 hour 10: 175236 events in 4 files, snapshot 6904028395659893215 (task 14176 ms: download 8124 ms overlapping gunzip 808 ms, parse 3555 ms, sort 104 ms, write 8276 ms, index 507 ms: 3 candidates, 79188 repo-hours, 994 releases; waiting 545 ms; commit 1750 ms)
2026-11-07T19:01:26Z [load_day 2023-04-21] 2023-04-21 hour 11: 172329 events in 4 files, snapshot 8229241631349988746 (task 13818 ms: download 7739 ms overlapping gunzip 839 ms, parse 3769 ms, sort 118 ms, write 7944 ms, index 532 ms: 2 candidates, 76244 repo-hours, 1040 releases; waiting 285 ms; commit 1771 ms)
2026-11-07T19:01:37Z [load_day 2023-04-21] 2023-04-21 hour 12: 222248 events in 5 files, snapshot 5321666992722216419 (task 15867 ms: download 8085 ms overlapping gunzip 901 ms, parse 4125 ms, sort 136 ms, write 9279 ms, index 635 ms: 5 candidates, 105518 repo-hours, 1075 releases; waiting 498 ms; commit 1534 ms)
2026-11-07T19:01:43Z [load_day 2023-04-21] 2023-04-21 hour 13: 201341 events in 5 files, snapshot 352453722468263639 (task 16528 ms: download 8627 ms overlapping gunzip 939 ms, parse 4194 ms, sort 136 ms, write 9590 ms, index 664 ms: 6 candidates, 99474 repo-hours, 1175 releases; waiting 553 ms; commit 1510 ms)
2026-11-07T19:01:54Z [load_day 2023-04-21] 2023-04-21 hour 14: 185913 events in 4 files, snapshot 4341781420623607314 (task 15665 ms: download 8765 ms overlapping gunzip 916 ms, parse 3976 ms, sort 115 ms, write 9171 ms, index 633 ms: 0 candidates, 85989 repo-hours, 1024 releases; waiting 510 ms; commit 1702 ms)
2026-11-07T19:01:59Z [load_day 2023-04-21] 2023-04-21 hour 15: 168286 events in 4 files, snapshot 6856058371057147658 (task 15326 ms: download 9311 ms overlapping gunzip 896 ms, parse 3842 ms, sort 104 ms, write 8700 ms, index 565 ms: 1 candidates, 79239 repo-hours, 1049 releases; waiting 916 ms; commit 1468 ms)
2026-11-07T19:02:09Z [load_day 2023-04-21] 2023-04-21 hour 16: 178938 events in 4 files, snapshot 459561902645974072 (task 14916 ms: download 8528 ms overlapping gunzip 904 ms, parse 3884 ms, sort 103 ms, write 8668 ms, index 544 ms: 4 candidates, 78833 repo-hours, 1071 releases; waiting 506 ms; commit 1422 ms)
2026-11-07T19:02:15Z [load_day 2023-04-21] 2023-04-21 hour 17: 176845 events in 4 files, snapshot 5608772619205756912 (task 14722 ms: download 8391 ms overlapping gunzip 880 ms, parse 3848 ms, sort 112 ms, write 8320 ms, index 548 ms: 2 candidates, 76238 repo-hours, 887 releases; waiting 677 ms; commit 1524 ms)
2026-11-07T19:02:24Z [load_day 2023-04-21] 2023-04-21 hour 18: 177112 events in 4 files, snapshot 563582671262162738 (task 13940 ms: download 8002 ms overlapping gunzip 834 ms, parse 3666 ms, sort 114 ms, write 7948 ms, index 561 ms: 3 candidates, 78589 repo-hours, 969 releases; waiting 530 ms; commit 1538 ms)
2026-11-07T19:02:29Z [load_day 2023-04-21] 2023-04-21 hour 19: 156720 events in 4 files, snapshot 8943111973846467555 (task 12876 ms: download 5362 ms overlapping gunzip 775 ms, parse 3473 ms, sort 125 ms, write 7088 ms, index 471 ms: 4 candidates, 65153 repo-hours, 990 releases; waiting 667 ms; commit 1722 ms)
2026-11-07T19:02:43Z [load_day 2023-04-21] 2023-04-21 hour 20: 176057 events in 4 files, snapshot 5408217223024470263 (task 17753 ms: download 9372 ms overlapping gunzip 1028 ms, parse 4389 ms, sort 97 ms, write 10784 ms, index 606 ms: 3 candidates, 77565 repo-hours, 876 releases; waiting 585 ms; commit 1714 ms)
2026-11-07T19:02:44Z [load_day 2023-04-21] 2023-04-21 hour 21: 150072 events in 4 files, snapshot 6419728312446894847 (task 13965 ms: download 5566 ms overlapping gunzip 794 ms, parse 3337 ms, sort 97 ms, write 8458 ms, index 455 ms: 7 candidates, 60455 repo-hours, 1040 releases; waiting 525 ms; commit 1542 ms)
2026-11-07T19:02:55Z [load_day 2023-04-21] 2023-04-21 hour 22: 135009 events in 3 files, snapshot 4644240138406221962 (task 11624 ms: download 5526 ms overlapping gunzip 734 ms, parse 3005 ms, sort 74 ms, write 6365 ms, index 349 ms: 2 candidates, 55206 repo-hours, 745 releases; waiting 745 ms; commit 1779 ms)
2026-11-07T19:02:57Z [load_day 2023-04-21] 2023-04-21 hour 23: 116362 events in 3 files, snapshot 5112731175388304699 (task 9001 ms: download 2480 ms overlapping gunzip 548 ms, parse 2325 ms, sort 66 ms, write 4933 ms, index 314 ms: 4 candidates, 48102 repo-hours, 817 releases; waiting 524 ms; commit 1572 ms)
2026-11-07T19:03:05Z [load_day 2023-04-21] 2023-04-21: repo_hour merged from 24 files to 1 (1788626 rows, 7699 ms)
2026-11-07T19:03:06Z [load_day 2023-04-21] 2023-04-21: index committed in 9365 ms (23 + 1 + 24 files)
2026-11-07T19:03:06Z [load_day 2023-04-21] 2023-04-21: 4105341 events in 24 hours
2026-11-07T19:03:06Z [load_day 2023-04-21] attempt 1 succeeded