orcaLog in

Runs · run 975

load_day 2023-04-07, attempt 1 succeeded

Host
caf251265124:243783
Started
2026-11-07T18:18:00Z
Ended
2026-11-07T18:20:54Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/975/load_day/2023-04-07/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T18:18:00Z [load_day 2023-04-07] attempt 1 started
2026-11-07T18:18:04Z [load_day 2023-04-07] 2023-04-07: 2 hours in flight, about 1200 MB
2026-11-07T18:18:17Z [load_day 2023-04-07] 2023-04-07 hour 0: 151486 events in 4 files, snapshot 8899091559366724007 (task 11261 ms: download 4804 ms overlapping gunzip 610 ms, parse 2811 ms, sort 101 ms, write 6400 ms, index 551 ms: 5 candidates, 79155 repo-hours, 1035 releases; waiting 519 ms; commit 1548 ms)
2026-11-07T18:18:19Z [load_day 2023-04-07] 2023-04-07 hour 1: 165639 events in 4 files, snapshot 2355156618030766869 (task 12619 ms: download 5078 ms overlapping gunzip 752 ms, parse 3238 ms, sort 115 ms, write 6923 ms, index 594 ms: 6 candidates, 85772 repo-hours, 697 releases; waiting 569 ms; commit 1854 ms)
2026-11-07T18:18:28Z [load_day 2023-04-07] 2023-04-07 hour 2: 136077 events in 3 files, snapshot 6743441799787465425 (task 10026 ms: download 4343 ms overlapping gunzip 619 ms, parse 2672 ms, sort 80 ms, write 5412 ms, index 389 ms: 5 candidates, 61733 repo-hours, 867 releases; waiting 529 ms; commit 1399 ms)
2026-11-07T18:18:30Z [load_day 2023-04-07] 2023-04-07 hour 3: 129127 events in 3 files, snapshot 5216266756313991262 (task 10109 ms: download 4676 ms overlapping gunzip 546 ms, parse 2403 ms, sort 76 ms, write 5000 ms, index 388 ms: 6 candidates, 57211 repo-hours, 805 releases; waiting 1327 ms; commit 1434 ms)
2026-11-07T18:18:38Z [load_day 2023-04-07] 2023-04-07 hour 4: 121029 events in 3 files, snapshot 7547312391031438353 (task 8687 ms: download 2591 ms overlapping gunzip 513 ms, parse 2242 ms, sort 73 ms, write 4604 ms, index 332 ms: 2 candidates, 51877 repo-hours, 615 releases; waiting 594 ms; commit 1487 ms)
2026-11-07T18:18:41Z [load_day 2023-04-07] 2023-04-07 hour 5: 131137 events in 3 files, snapshot 4142120359488653789 (task 9869 ms: download 4501 ms overlapping gunzip 572 ms, parse 2439 ms, sort 80 ms, write 5163 ms, index 425 ms: 4 candidates, 57072 repo-hours, 757 releases; waiting 891 ms; commit 1711 ms)
2026-11-07T18:18:49Z [load_day 2023-04-07] 2023-04-07 hour 6: 153325 events in 4 files, snapshot 9012905012204860872 (task 10672 ms: download 4229 ms overlapping gunzip 626 ms, parse 2741 ms, sort 96 ms, write 5957 ms, index 452 ms: 4 candidates, 72419 repo-hours, 784 releases; waiting 515 ms; commit 1460 ms)
2026-11-07T18:18:53Z [load_day 2023-04-07] 2023-04-07 hour 7: 160020 events in 4 files, snapshot 4768046942116177860 (task 11846 ms: download 4798 ms overlapping gunzip 688 ms, parse 2984 ms, sort 91 ms, write 6754 ms, index 537 ms: 5 candidates, 70921 repo-hours, 748 releases; waiting 512 ms; commit 1595 ms)
2026-11-07T18:19:04Z [load_day 2023-04-07] 2023-04-07 hour 8: 167830 events in 4 files, snapshot 3950049177954338642 (task 13010 ms: download 5124 ms overlapping gunzip 755 ms, parse 3331 ms, sort 110 ms, write 7391 ms, index 575 ms: 5 candidates, 76376 repo-hours, 873 releases; waiting 458 ms; commit 2220 ms)
2026-11-07T18:19:07Z [load_day 2023-04-07] 2023-04-07 hour 9: 171605 events in 4 files, snapshot 3891197411382696372 (task 13041 ms: download 7369 ms overlapping gunzip 764 ms, parse 3312 ms, sort 105 ms, write 7510 ms, index 532 ms: 11 candidates, 76730 repo-hours, 1022 releases; waiting 511 ms; commit 1493 ms)
2026-11-07T18:19:16Z [load_day 2023-04-07] 2023-04-07 hour 10: 163098 events in 4 files, snapshot 1342248747612286863 (task 11913 ms: download 4836 ms overlapping gunzip 693 ms, parse 3086 ms, sort 102 ms, write 6695 ms, index 485 ms: 5 candidates, 72948 repo-hours, 887 releases; waiting 512 ms; commit 1504 ms)
2026-11-07T18:19:20Z [load_day 2023-04-07] 2023-04-07 hour 11: 153715 events in 4 files, snapshot 2609276180478497589 (task 11750 ms: download 4984 ms overlapping gunzip 669 ms, parse 3002 ms, sort 103 ms, write 6430 ms, index 466 ms: 9 candidates, 66857 repo-hours, 799 releases; waiting 815 ms; commit 1713 ms)
2026-11-07T18:19:33Z [load_day 2023-04-07] 2023-04-07 hour 12: 211945 events in 5 files, snapshot 6405777333667827011 (task 16084 ms: download 9319 ms overlapping gunzip 794 ms, parse 3596 ms, sort 108 ms, write 7929 ms, index 626 ms: 8 candidates, 98305 repo-hours, 878 releases; waiting 2724 ms; commit 1530 ms)
2026-11-07T18:19:35Z [load_day 2023-04-07] 2023-04-07 hour 13: 190552 events in 4 files, snapshot 1054431331673481280 (task 13587 ms: download 7178 ms overlapping gunzip 793 ms, parse 3486 ms, sort 123 ms, write 7704 ms, index 596 ms: 11 candidates, 94725 repo-hours, 962 releases; waiting 547 ms; commit 1589 ms)
2026-11-07T18:19:49Z [load_day 2023-04-07] 2023-04-07 hour 14: 184008 events in 4 files, snapshot 4636273003829025567 (task 14193 ms: download 7909 ms overlapping gunzip 814 ms, parse 3537 ms, sort 113 ms, write 8297 ms, index 544 ms: 7 candidates, 82385 repo-hours, 1030 releases; waiting 540 ms; commit 1837 ms)
2026-11-07T18:19:50Z [load_day 2023-04-07] 2023-04-07 hour 15: 182246 events in 4 files, snapshot 3098775941513706757 (task 13962 ms: download 7819 ms overlapping gunzip 836 ms, parse 3649 ms, sort 111 ms, write 8016 ms, index 546 ms: 10 candidates, 81069 repo-hours, 1096 releases; waiting 530 ms; commit 1572 ms)
2026-11-07T18:20:03Z [load_day 2023-04-07] 2023-04-07 hour 16: 170037 events in 4 files, snapshot 3502418756018798922 (task 13311 ms: download 7845 ms overlapping gunzip 822 ms, parse 3491 ms, sort 111 ms, write 7553 ms, index 554 ms: 5 candidates, 75781 repo-hours, 887 releases; waiting 524 ms; commit 1782 ms)
2026-11-07T18:20:04Z [load_day 2023-04-07] 2023-04-07 hour 17: 170127 events in 4 files, snapshot 7528688301855863751 (task 12837 ms: download 7457 ms overlapping gunzip 780 ms, parse 3374 ms, sort 100 ms, write 7365 ms, index 483 ms: 13 candidates, 72135 repo-hours, 831 releases; waiting 475 ms; commit 1583 ms)
2026-11-07T18:20:16Z [load_day 2023-04-07] 2023-04-07 hour 18: 171315 events in 4 files, snapshot 8153455761003602222 (task 13224 ms: download 7609 ms overlapping gunzip 772 ms, parse 3422 ms, sort 99 ms, write 7516 ms, index 536 ms: 6 candidates, 77202 repo-hours, 1019 releases; waiting 550 ms; commit 1569 ms)
2026-11-07T18:20:18Z [load_day 2023-04-07] 2023-04-07 hour 19: 161636 events in 4 files, snapshot 3337979380505813725 (task 13608 ms: download 8265 ms overlapping gunzip 789 ms, parse 3459 ms, sort 102 ms, write 7463 ms, index 519 ms: 6 candidates, 66429 repo-hours, 934 releases; waiting 995 ms; commit 1505 ms)
2026-11-07T18:20:30Z [load_day 2023-04-07] 2023-04-07 hour 20: 157611 events in 4 files, snapshot 106709170432886253 (task 12686 ms: download 7620 ms overlapping gunzip 767 ms, parse 3364 ms, sort 86 ms, write 7194 ms, index 457 ms: 6 candidates, 67223 repo-hours, 876 releases; waiting 525 ms; commit 1463 ms)
2026-11-07T18:20:31Z [load_day 2023-04-07] 2023-04-07 hour 21: 146967 events in 3 files, snapshot 6810089096339226436 (task 10470 ms: download 4345 ms overlapping gunzip 654 ms, parse 2858 ms, sort 84 ms, write 6016 ms, index 382 ms: 6 candidates, 60182 repo-hours, 929 releases; waiting 210 ms; commit 1443 ms)
2026-11-07T18:20:43Z [load_day 2023-04-07] 2023-04-07 hour 22: 132199 events in 3 files, snapshot 4011662842020829266 (task 9885 ms: download 4209 ms overlapping gunzip 595 ms, parse 2525 ms, sort 83 ms, write 5480 ms, index 383 ms: 16 candidates, 56736 repo-hours, 732 releases; waiting 513 ms; commit 4282 ms)
2026-11-07T18:20:45Z [load_day 2023-04-07] 2023-04-07 hour 23: 116081 events in 3 files, snapshot 1025295577993621790 (task 8943 ms: download 2328 ms overlapping gunzip 534 ms, parse 2337 ms, sort 74 ms, write 4885 ms, index 350 ms: 3 candidates, 49096 repo-hours, 677 releases; waiting 388 ms; commit 1486 ms)
2026-11-07T18:20:52Z [load_day 2023-04-07] 2023-04-07: repo_hour merged from 24 files to 1 (1710339 rows, 6821 ms)
2026-11-07T18:20:54Z [load_day 2023-04-07] 2023-04-07: index committed in 8799 ms (24 + 1 + 24 files)
2026-11-07T18:20:54Z [load_day 2023-04-07] 2023-04-07: 3798812 events in 24 hours
2026-11-07T18:20:54Z [load_day 2023-04-07] attempt 1 succeeded