orcaLog in

Runs · run 1104

load_day 2023-05-20, attempt 1 succeeded

Host
caf251265124:386286
Started
2026-11-07T20:23:34Z
Ended
2026-11-07T20:25:35Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1104/load_day/2023-05-20/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T20:23:34Z [load_day 2023-05-20] attempt 1 started
2026-11-07T20:23:35Z [load_day 2023-05-20] 2023-05-20: 2 hours in flight, about 1200 MB
2026-11-07T20:23:47Z [load_day 2023-05-20] 2023-05-20 hour 0: 128711 events in 3 files, snapshot 3533356684422489986 (task 9148 ms: download 2686 ms overlapping gunzip 504 ms, parse 2380 ms, sort 65 ms, write 4869 ms, index 474 ms: 8 candidates, 70648 repo-hours, 876 releases; waiting 537 ms; commit 1662 ms)
2026-11-07T20:23:49Z [load_day 2023-05-20] 2023-05-20 hour 1: 127351 events in 3 files, snapshot 2842671911642141962 (task 8408 ms: download 2274 ms overlapping gunzip 500 ms, parse 2240 ms, sort 67 ms, write 4361 ms, index 470 ms: 4 candidates, 74704 repo-hours, 689 releases; waiting 510 ms; commit 1435 ms)
2026-11-07T20:23:55Z [load_day 2023-05-20] 2023-05-20 hour 2: 100602 events in 3 files, snapshot 5441907367150593816 (task 7103 ms: download 1858 ms overlapping gunzip 400 ms, parse 1762 ms, sort 63 ms, write 3694 ms, index 372 ms: 2 candidates, 44742 repo-hours, 584 releases; waiting 516 ms; commit 1648 ms)
2026-11-07T20:23:57Z [load_day 2023-05-20] 2023-05-20 hour 3: 97284 events in 2 files, snapshot 7427347616827483763 (task 6982 ms: download 2238 ms overlapping gunzip 370 ms, parse 1714 ms, sort 56 ms, write 3472 ms, index 283 ms: 5 candidates, 41859 repo-hours, 687 releases; waiting 762 ms; commit 1709 ms)
2026-11-07T20:24:02Z [load_day 2023-05-20] 2023-05-20 hour 4: 93996 events in 2 files, snapshot 8158611106078584489 (task 6366 ms: download 1911 ms overlapping gunzip 374 ms, parse 1669 ms, sort 60 ms, write 3108 ms, index 282 ms: 4 candidates, 41725 repo-hours, 479 releases; waiting 532 ms; commit 1737 ms)
2026-11-07T20:24:04Z [load_day 2023-05-20] 2023-05-20 hour 5: 102057 events in 3 files, snapshot 1284182571526682514 (task 6628 ms: download 2376 ms overlapping gunzip 376 ms, parse 1712 ms, sort 62 ms, write 3352 ms, index 284 ms: 3 candidates, 44446 repo-hours, 663 releases; waiting 496 ms; commit 1801 ms)
2026-11-07T20:24:12Z [load_day 2023-05-20] 2023-05-20 hour 6: 113538 events in 3 files, snapshot 8461829574989151437 (task 8475 ms: download 4451 ms overlapping gunzip 397 ms, parse 1814 ms, sort 64 ms, write 3777 ms, index 359 ms: 6 candidates, 52226 repo-hours, 696 releases; waiting 1608 ms; commit 1764 ms)
2026-11-07T20:24:13Z [load_day 2023-05-20] 2023-05-20 hour 7: 116432 events in 3 files, snapshot 4050686165288412177 (task 7442 ms: download 1940 ms overlapping gunzip 399 ms, parse 1831 ms, sort 68 ms, write 4002 ms, index 403 ms: 1 candidates, 53184 repo-hours, 613 releases; waiting 504 ms; commit 1530 ms)
2026-11-07T20:24:20Z [load_day 2023-05-20] 2023-05-20 hour 8: 117850 events in 3 files, snapshot 2324130705388184994 (task 7516 ms: download 2220 ms overlapping gunzip 409 ms, parse 1823 ms, sort 66 ms, write 4030 ms, index 340 ms: 9 candidates, 53304 repo-hours, 667 releases; waiting 517 ms; commit 1597 ms)
2026-11-07T20:24:22Z [load_day 2023-05-20] 2023-05-20 hour 9: 119560 events in 3 files, snapshot 400045595377881680 (task 7785 ms: download 1795 ms overlapping gunzip 411 ms, parse 1920 ms, sort 66 ms, write 4205 ms, index 349 ms: 6 candidates, 54580 repo-hours, 774 releases; waiting 536 ms; commit 1676 ms)
2026-11-07T20:24:29Z [load_day 2023-05-20] 2023-05-20 hour 10: 121016 events in 3 files, snapshot 7959903861503994391 (task 7841 ms: download 2092 ms overlapping gunzip 420 ms, parse 1988 ms, sort 70 ms, write 4077 ms, index 398 ms: 3 candidates, 56296 repo-hours, 606 releases; waiting 505 ms; commit 1615 ms)
2026-11-07T20:24:30Z [load_day 2023-05-20] 2023-05-20 hour 11: 116742 events in 3 files, snapshot 7154308417989096928 (task 7521 ms: download 1805 ms overlapping gunzip 408 ms, parse 1848 ms, sort 66 ms, write 4045 ms, index 332 ms: 5 candidates, 52578 repo-hours, 651 releases; waiting 516 ms; commit 1510 ms)
2026-11-07T20:24:41Z [load_day 2023-05-20] 2023-05-20 hour 12: 190070 events in 4 files, snapshot 4780453488752550348 (task 10465 ms: download 3169 ms overlapping gunzip 527 ms, parse 2597 ms, sort 105 ms, write 5461 ms, index 1197 ms: 4 candidates, 84622 repo-hours, 739 releases; waiting 275 ms; commit 2457 ms)
2026-11-07T20:24:42Z [load_day 2023-05-20] 2023-05-20 hour 13: 149949 events in 3 files, snapshot 161668507345929877 (task 9340 ms: download 3469 ms overlapping gunzip 486 ms, parse 2273 ms, sort 91 ms, write 4806 ms, index 497 ms: 6 candidates, 77966 repo-hours, 763 releases; waiting 910 ms; commit 1813 ms)
2026-11-07T20:24:50Z [load_day 2023-05-20] 2023-05-20 hour 14: 137872 events in 3 files, snapshot 61358097985137201 (task 8893 ms: download 3436 ms overlapping gunzip 510 ms, parse 2289 ms, sort 85 ms, write 4903 ms, index 406 ms: 7 candidates, 62618 repo-hours, 714 releases; waiting 502 ms; commit 1836 ms)
2026-11-07T20:24:52Z [load_day 2023-05-20] 2023-05-20 hour 15: 140332 events in 3 files, snapshot 5745980644573678075 (task 9056 ms: download 3417 ms overlapping gunzip 502 ms, parse 2290 ms, sort 76 ms, write 4862 ms, index 412 ms: 13 candidates, 64350 repo-hours, 853 releases; waiting 519 ms; commit 1736 ms)
2026-11-07T20:24:59Z [load_day 2023-05-20] 2023-05-20 hour 16: 136643 events in 3 files, snapshot 6040652221368146882 (task 9069 ms: download 2169 ms overlapping gunzip 509 ms, parse 2319 ms, sort 96 ms, write 4738 ms, index 445 ms: 5 candidates, 63170 repo-hours, 764 releases; waiting 711 ms; commit 1619 ms)
2026-11-07T20:25:01Z [load_day 2023-05-20] 2023-05-20 hour 17: 135429 events in 3 files, snapshot 8653886750923160824 (task 8181 ms: download 1688 ms overlapping gunzip 456 ms, parse 2119 ms, sort 87 ms, write 4600 ms, index 416 ms: 12 candidates, 61241 repo-hours, 808 releases; waiting 220 ms; commit 2044 ms)
2026-11-07T20:25:09Z [load_day 2023-05-20] 2023-05-20 hour 18: 132206 events in 3 files, snapshot 352569110880856869 (task 9286 ms: download 3455 ms overlapping gunzip 452 ms, parse 2153 ms, sort 97 ms, write 4687 ms, index 419 ms: 5 candidates, 61958 repo-hours, 823 releases; waiting 1131 ms; commit 1561 ms)
2026-11-07T20:25:11Z [load_day 2023-05-20] 2023-05-20 hour 19: 125054 events in 3 files, snapshot 5473565684579158530 (task 8108 ms: download 2299 ms overlapping gunzip 422 ms, parse 2018 ms, sort 98 ms, write 4374 ms, index 347 ms: 8 candidates, 51173 repo-hours, 627 releases; waiting 517 ms; commit 1839 ms)
2026-11-07T20:25:18Z [load_day 2023-05-20] 2023-05-20 hour 20: 116502 events in 3 files, snapshot 5219787590024430531 (task 7612 ms: download 2146 ms overlapping gunzip 416 ms, parse 1958 ms, sort 69 ms, write 3984 ms, index 344 ms: 6 candidates, 50814 repo-hours, 693 releases; waiting 532 ms; commit 1721 ms)
2026-11-07T20:25:20Z [load_day 2023-05-20] 2023-05-20 hour 21: 103444 events in 3 files, snapshot 3015102194247213667 (task 6580 ms: download 1838 ms overlapping gunzip 368 ms, parse 1719 ms, sort 81 ms, write 3335 ms, index 298 ms: 3 candidates, 44201 repo-hours, 737 releases; waiting 486 ms; commit 1725 ms)
2026-11-07T20:25:25Z [load_day 2023-05-20] 2023-05-20 hour 22: 93179 events in 2 files, snapshot 4451284982082130950 (task 6243 ms: download 2096 ms overlapping gunzip 348 ms, parse 1518 ms, sort 55 ms, write 3064 ms, index 272 ms: 5 candidates, 39570 repo-hours, 677 releases; waiting 668 ms; commit 1767 ms)
2026-11-07T20:25:27Z [load_day 2023-05-20] 2023-05-20 hour 23: 89986 events in 2 files, snapshot 7486695299010432868 (task 5973 ms: download 2016 ms overlapping gunzip 309 ms, parse 1444 ms, sort 53 ms, write 2907 ms, index 262 ms: 4 candidates, 38237 repo-hours, 557 releases; waiting 617 ms; commit 1690 ms)
2026-11-07T20:25:33Z [load_day 2023-05-20] 2023-05-20: repo_hour merged from 24 files to 1 (1340212 rows, 5880 ms)
2026-11-07T20:25:35Z [load_day 2023-05-20] 2023-05-20: index committed in 7746 ms (24 + 1 + 24 files)
2026-11-07T20:25:35Z [load_day 2023-05-20] 2023-05-20: 2905805 events in 24 hours
2026-11-07T20:25:35Z [load_day 2023-05-20] attempt 1 succeeded