orcaLog in

Runs · run 1107

load_day 2023-05-21, attempt 1 succeeded

Host
caf251265124:388204
Started
2026-11-07T20:25:36Z
Ended
2026-11-07T20:27:35Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1107/load_day/2023-05-21/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T20:25:36Z [load_day 2023-05-21] attempt 1 started
2026-11-07T20:25:37Z [load_day 2023-05-21] 2023-05-21: 2 hours in flight, about 1200 MB
2026-11-07T20:25:47Z [load_day 2023-05-21] 2023-05-21 hour 0: 106405 events in 3 files, snapshot 4481505494529084498 (task 7192 ms: download 2048 ms overlapping gunzip 374 ms, parse 1840 ms, sort 61 ms, write 3763 ms, index 373 ms: 8 candidates, 59444 repo-hours, 848 releases; waiting 511 ms; commit 1766 ms)
2026-11-07T20:25:49Z [load_day 2023-05-21] 2023-05-21 hour 1: 109447 events in 3 files, snapshot 5323518871720904270 (task 6867 ms: download 2208 ms overlapping gunzip 377 ms, parse 1910 ms, sort 70 ms, write 3533 ms, index 429 ms: 4 candidates, 65657 repo-hours, 822 releases; waiting 265 ms; commit 1659 ms)
2026-11-07T20:25:54Z [load_day 2023-05-21] 2023-05-21 hour 2: 90628 events in 2 files, snapshot 435332806078919492 (task 5895 ms: download 1974 ms overlapping gunzip 314 ms, parse 1409 ms, sort 44 ms, write 2911 ms, index 291 ms: 3 candidates, 45400 repo-hours, 585 releases; waiting 641 ms; commit 1973 ms)
2026-11-07T20:25:56Z [load_day 2023-05-21] 2023-05-21 hour 3: 87179 events in 2 files, snapshot 3316543614424269727 (task 5603 ms: download 2121 ms overlapping gunzip 285 ms, parse 1379 ms, sort 39 ms, write 2660 ms, index 280 ms: 6 candidates, 37400 repo-hours, 687 releases; waiting 631 ms; commit 1895 ms)
2026-11-07T20:26:00Z [load_day 2023-05-21] 2023-05-21 hour 4: 90849 events in 2 files, snapshot 7780641550909225308 (task 5500 ms: download 1596 ms overlapping gunzip 295 ms, parse 1366 ms, sort 54 ms, write 2753 ms, index 267 ms: 3 candidates, 38688 repo-hours, 516 releases; waiting 491 ms; commit 1733 ms)
2026-11-07T20:26:03Z [load_day 2023-05-21] 2023-05-21 hour 5: 93904 events in 2 files, snapshot 3531502395096757824 (task 5939 ms: download 1813 ms overlapping gunzip 317 ms, parse 1525 ms, sort 55 ms, write 2979 ms, index 303 ms: 6 candidates, 41251 repo-hours, 665 releases; waiting 499 ms; commit 1796 ms)
2026-11-07T20:26:08Z [load_day 2023-05-21] 2023-05-21 hour 6: 106868 events in 3 files, snapshot 5596576763189756202 (task 6732 ms: download 1622 ms overlapping gunzip 351 ms, parse 1599 ms, sort 62 ms, write 3538 ms, index 354 ms: 6 candidates, 49358 repo-hours, 689 releases; waiting 498 ms; commit 1670 ms)
2026-11-07T20:26:11Z [load_day 2023-05-21] 2023-05-21 hour 7: 113218 events in 3 files, snapshot 1439249103131318474 (task 7171 ms: download 2313 ms overlapping gunzip 375 ms, parse 1702 ms, sort 66 ms, write 3777 ms, index 319 ms: 4 candidates, 46725 repo-hours, 648 releases; waiting 600 ms; commit 1829 ms)
2026-11-07T20:26:17Z [load_day 2023-05-21] 2023-05-21 hour 8: 118510 events in 3 files, snapshot 1258613387208105766 (task 7752 ms: download 2182 ms overlapping gunzip 420 ms, parse 1923 ms, sort 73 ms, write 4070 ms, index 361 ms: 3 candidates, 51067 repo-hours, 664 releases; waiting 519 ms; commit 1997 ms)
2026-11-07T20:26:20Z [load_day 2023-05-21] 2023-05-21 hour 9: 119686 events in 3 files, snapshot 2998514416704852122 (task 8369 ms: download 3042 ms overlapping gunzip 425 ms, parse 1934 ms, sort 66 ms, write 4366 ms, index 357 ms: 6 candidates, 52142 repo-hours, 791 releases; waiting 861 ms; commit 1736 ms)
2026-11-07T20:26:25Z [load_day 2023-05-21] 2023-05-21 hour 10: 124079 events in 3 files, snapshot 1221714535257862575 (task 7740 ms: download 2097 ms overlapping gunzip 426 ms, parse 1947 ms, sort 69 ms, write 4205 ms, index 403 ms: 5 candidates, 56958 repo-hours, 730 releases; waiting 350 ms; commit 1722 ms)
2026-11-07T20:26:29Z [load_day 2023-05-21] 2023-05-21 hour 11: 121310 events in 3 files, snapshot 7212807495617678035 (task 7988 ms: download 2321 ms overlapping gunzip 425 ms, parse 1964 ms, sort 70 ms, write 4290 ms, index 373 ms: 6 candidates, 53408 repo-hours, 672 releases; waiting 499 ms; commit 1699 ms)
2026-11-07T20:26:36Z [load_day 2023-05-21] 2023-05-21 hour 12: 195707 events in 4 files, snapshot 5897704751497907914 (task 10012 ms: download 3397 ms overlapping gunzip 536 ms, parse 2613 ms, sort 96 ms, write 5478 ms, index 523 ms: 5 candidates, 86631 repo-hours, 871 releases; waiting 502 ms; commit 1669 ms)
2026-11-07T20:26:39Z [load_day 2023-05-21] 2023-05-21 hour 13: 157600 events in 4 files, snapshot 7013018697909531304 (task 9344 ms: download 3103 ms overlapping gunzip 500 ms, parse 2366 ms, sort 88 ms, write 5093 ms, index 499 ms: 5 candidates, 81059 repo-hours, 808 releases; waiting 517 ms; commit 1715 ms)
2026-11-07T20:26:47Z [load_day 2023-05-21] 2023-05-21 hour 14: 144950 events in 3 files, snapshot 5872605738614399340 (task 10868 ms: download 5028 ms overlapping gunzip 559 ms, parse 2519 ms, sort 104 ms, write 5146 ms, index 453 ms: 4 candidates, 65891 repo-hours, 832 releases; waiting 1788 ms; commit 1663 ms)
2026-11-07T20:26:50Z [load_day 2023-05-21] 2023-05-21 hour 15: 150414 events in 4 files, snapshot 2301717348731642761 (task 9723 ms: download 3675 ms overlapping gunzip 529 ms, parse 2412 ms, sort 88 ms, write 5283 ms, index 477 ms: 5 candidates, 69725 repo-hours, 890 releases; waiting 564 ms; commit 1891 ms)
2026-11-07T20:26:59Z [load_day 2023-05-21] 2023-05-21 hour 16: 139846 events in 3 files, snapshot 9011471867138654994 (task 9548 ms: download 2299 ms overlapping gunzip 507 ms, parse 2284 ms, sort 86 ms, write 4898 ms, index 986 ms: 6 candidates, 63376 repo-hours, 775 releases; waiting 520 ms; commit 2990 ms)
2026-11-07T20:27:01Z [load_day 2023-05-21] 2023-05-21 hour 17: 135871 events in 3 files, snapshot 3679250406784331852 (task 8918 ms: download 2700 ms overlapping gunzip 471 ms, parse 2155 ms, sort 69 ms, write 4618 ms, index 430 ms: 4 candidates, 58633 repo-hours, 701 releases; waiting 863 ms; commit 1763 ms)
2026-11-07T20:27:09Z [load_day 2023-05-21] 2023-05-21 hour 18: 142166 events in 3 files, snapshot 6510373141527157919 (task 8557 ms: download 1531 ms overlapping gunzip 474 ms, parse 2227 ms, sort 80 ms, write 4804 ms, index 454 ms: 4 candidates, 66753 repo-hours, 938 releases; waiting 209 ms; commit 1748 ms)
2026-11-07T20:27:11Z [load_day 2023-05-21] 2023-05-21 hour 19: 126988 events in 3 files, snapshot 522575922786881786 (task 8254 ms: download 1998 ms overlapping gunzip 447 ms, parse 2140 ms, sort 77 ms, write 4413 ms, index 357 ms: 3 candidates, 56176 repo-hours, 777 releases; waiting 509 ms; commit 1638 ms)
2026-11-07T20:27:18Z [load_day 2023-05-21] 2023-05-21 hour 20: 125672 events in 3 files, snapshot 1617747263858286340 (task 8433 ms: download 1839 ms overlapping gunzip 466 ms, parse 2212 ms, sort 96 ms, write 4516 ms, index 375 ms: 5 candidates, 55073 repo-hours, 748 releases; waiting 496 ms; commit 1610 ms)
2026-11-07T20:27:20Z [load_day 2023-05-21] 2023-05-21 hour 21: 113651 events in 3 files, snapshot 1064078579654782311 (task 7752 ms: download 2115 ms overlapping gunzip 438 ms, parse 1988 ms, sort 70 ms, write 4115 ms, index 346 ms: 2 candidates, 49375 repo-hours, 828 releases; waiting 503 ms; commit 1623 ms)
2026-11-07T20:27:26Z [load_day 2023-05-21] 2023-05-21 hour 22: 101547 events in 3 files, snapshot 8849493856415350778 (task 7561 ms: download 2084 ms overlapping gunzip 439 ms, parse 1866 ms, sort 66 ms, write 3978 ms, index 357 ms: 2 candidates, 43031 repo-hours, 689 releases; waiting 546 ms; commit 1649 ms)
2026-11-07T20:27:28Z [load_day 2023-05-21] 2023-05-21 hour 23: 99824 events in 2 files, snapshot 2612085698585348563 (task 6566 ms: download 1568 ms overlapping gunzip 361 ms, parse 1653 ms, sort 69 ms, write 3372 ms, index 308 ms: 5 candidates, 44071 repo-hours, 602 releases; waiting 514 ms; commit 1649 ms)
2026-11-07T20:27:34Z [load_day 2023-05-21] 2023-05-21: repo_hour merged from 24 files to 1 (1337292 rows, 5816 ms)
2026-11-07T20:27:35Z [load_day 2023-05-21] 2023-05-21: index committed in 7484 ms (24 + 1 + 24 files)
2026-11-07T20:27:35Z [load_day 2023-05-21] 2023-05-21: 2916319 events in 24 hours
2026-11-07T20:27:35Z [load_day 2023-05-21] attempt 1 succeeded