orcaLog in

Runs · run 1150

load_day 2023-06-04, attempt 1 succeeded

Host
caf251265124:432656
Started
2026-11-07T21:04:56Z
Ended
2026-11-07T21:06:55Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1150/load_day/2023-06-04/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-07T21:04:56Z [load_day 2023-06-04] attempt 1 started
2026-11-07T21:04:57Z [load_day 2023-06-04] 2023-06-04: 2 hours in flight, about 1200 MB
2026-11-07T21:05:07Z [load_day 2023-06-04] 2023-06-04 hour 0: 102671 events in 3 files, snapshot 7796578936730848712 (task 6686 ms: download 1609 ms overlapping gunzip 415 ms, parse 1907 ms, sort 67 ms, write 3351 ms, index 334 ms: 4 candidates, 54699 repo-hours, 763 releases; waiting 326 ms; commit 1770 ms)
2026-11-07T21:05:09Z [load_day 2023-06-04] 2023-06-04 hour 1: 108590 events in 3 files, snapshot 683351635429335606 (task 7275 ms: download 1696 ms overlapping gunzip 389 ms, parse 1869 ms, sort 65 ms, write 3828 ms, index 397 ms: 7 candidates, 62175 repo-hours, 753 releases; waiting 468 ms; commit 1753 ms)
2026-11-07T21:05:14Z [load_day 2023-06-04] 2023-06-04 hour 2: 106356 events in 3 files, snapshot 8554842235294167024 (task 6536 ms: download 1960 ms overlapping gunzip 353 ms, parse 1555 ms, sort 69 ms, write 3253 ms, index 374 ms: 7 candidates, 60443 repo-hours, 602 releases; waiting 570 ms; commit 1757 ms)
2026-11-07T21:05:16Z [load_day 2023-06-04] 2023-06-04 hour 3: 88752 events in 2 files, snapshot 4945230270834400003 (task 5990 ms: download 1788 ms overlapping gunzip 312 ms, parse 1489 ms, sort 52 ms, write 2966 ms, index 279 ms: 4 candidates, 39722 repo-hours, 752 releases; waiting 580 ms; commit 1864 ms)
2026-11-07T21:05:21Z [load_day 2023-06-04] 2023-06-04 hour 4: 89616 events in 2 files, snapshot 2380663607455062656 (task 5666 ms: download 1657 ms overlapping gunzip 301 ms, parse 1394 ms, sort 61 ms, write 2826 ms, index 261 ms: 3 candidates, 38725 repo-hours, 558 releases; waiting 513 ms; commit 1666 ms)
2026-11-07T21:05:23Z [load_day 2023-06-04] 2023-06-04 hour 5: 91499 events in 2 files, snapshot 3430079869407764465 (task 6153 ms: download 1998 ms overlapping gunzip 359 ms, parse 1696 ms, sort 65 ms, write 2885 ms, index 314 ms: 7 candidates, 41208 repo-hours, 693 releases; waiting 511 ms; commit 1671 ms)
2026-11-07T21:05:28Z [load_day 2023-06-04] 2023-06-04 hour 6: 106360 events in 3 files, snapshot 1873148725460798895 (task 6551 ms: download 1602 ms overlapping gunzip 349 ms, parse 1629 ms, sort 62 ms, write 3367 ms, index 326 ms: 6 candidates, 49914 repo-hours, 759 releases; waiting 520 ms; commit 1793 ms)
2026-11-07T21:05:31Z [load_day 2023-06-04] 2023-06-04 hour 7: 105509 events in 3 files, snapshot 6845839713537049706 (task 6877 ms: download 1883 ms overlapping gunzip 360 ms, parse 1692 ms, sort 70 ms, write 3598 ms, index 319 ms: 4 candidates, 47396 repo-hours, 618 releases; waiting 507 ms; commit 1921 ms)
2026-11-07T21:05:36Z [load_day 2023-06-04] 2023-06-04 hour 8: 119093 events in 3 files, snapshot 709169114917778549 (task 7723 ms: download 2088 ms overlapping gunzip 402 ms, parse 1837 ms, sort 77 ms, write 4263 ms, index 342 ms: 6 candidates, 56611 repo-hours, 670 releases; waiting 508 ms; commit 1651 ms)
2026-11-07T21:05:40Z [load_day 2023-06-04] 2023-06-04 hour 9: 117006 events in 3 files, snapshot 7980338552110339211 (task 7823 ms: download 2117 ms overlapping gunzip 413 ms, parse 1919 ms, sort 67 ms, write 4239 ms, index 363 ms: 5 candidates, 52634 repo-hours, 1152 releases; waiting 518 ms; commit 1899 ms)
2026-11-07T21:05:45Z [load_day 2023-06-04] 2023-06-04 hour 10: 131983 events in 3 files, snapshot 8619564956320113860 (task 8106 ms: download 2123 ms overlapping gunzip 438 ms, parse 1990 ms, sort 80 ms, write 4476 ms, index 404 ms: 6 candidates, 65692 repo-hours, 669 releases; waiting 392 ms; commit 1570 ms)
2026-11-07T21:05:48Z [load_day 2023-06-04] 2023-06-04 hour 11: 123482 events in 3 files, snapshot 991466564089380901 (task 7857 ms: download 2273 ms overlapping gunzip 449 ms, parse 2022 ms, sort 63 ms, write 4263 ms, index 363 ms: 2 candidates, 53875 repo-hours, 642 releases; waiting 322 ms; commit 1995 ms)
2026-11-07T21:05:56Z [load_day 2023-06-04] 2023-06-04 hour 12: 198757 events in 4 files, snapshot 6639033970147139600 (task 9914 ms: download 3312 ms overlapping gunzip 546 ms, parse 2659 ms, sort 101 ms, write 5230 ms, index 526 ms: 7 candidates, 85988 repo-hours, 841 releases; waiting 507 ms; commit 1794 ms)
2026-11-07T21:05:58Z [load_day 2023-06-04] 2023-06-04 hour 13: 166516 events in 4 files, snapshot 2564487378088877224 (task 9119 ms: download 2856 ms overlapping gunzip 503 ms, parse 2400 ms, sort 100 ms, write 5011 ms, index 550 ms: 4 candidates, 87374 repo-hours, 777 releases; waiting 245 ms; commit 1744 ms)
2026-11-07T21:06:06Z [load_day 2023-06-04] 2023-06-04 hour 14: 160361 events in 4 files, snapshot 7439746840618718991 (task 9133 ms: download 2946 ms overlapping gunzip 515 ms, parse 2364 ms, sort 109 ms, write 5152 ms, index 484 ms: 3 candidates, 82889 repo-hours, 810 releases; waiting 196 ms; commit 1663 ms)
2026-11-07T21:06:08Z [load_day 2023-06-04] 2023-06-04 hour 15: 151027 events in 4 files, snapshot 3237452697033121491 (task 9280 ms: download 3119 ms overlapping gunzip 513 ms, parse 2346 ms, sort 92 ms, write 5245 ms, index 475 ms: 5 candidates, 73436 repo-hours, 889 releases; waiting 313 ms; commit 1744 ms)
2026-11-07T21:06:19Z [load_day 2023-06-04] 2023-06-04 hour 16: 146825 events in 3 files, snapshot 7573019285999176254 (task 8957 ms: download 3280 ms overlapping gunzip 497 ms, parse 2307 ms, sort 94 ms, write 4799 ms, index 429 ms: 5 candidates, 69080 repo-hours, 754 releases; waiting 505 ms; commit 5445 ms)
2026-11-07T21:06:21Z [load_day 2023-06-04] 2023-06-04 hour 17: 143831 events in 3 files, snapshot 8369389050469247620 (task 8701 ms: download 3333 ms overlapping gunzip 492 ms, parse 2256 ms, sort 88 ms, write 4662 ms, index 435 ms: 6 candidates, 66588 repo-hours, 705 releases; waiting 493 ms; commit 1705 ms)
2026-11-07T21:06:26Z [load_day 2023-06-04] 2023-06-04 hour 18: 143622 events in 3 files, snapshot 2711303707362892754 (task 9068 ms: download 3274 ms overlapping gunzip 507 ms, parse 2324 ms, sort 83 ms, write 4944 ms, index 439 ms: 8 candidates, 66847 repo-hours, 818 releases; waiting 511 ms; commit 1918 ms)
2026-11-07T21:06:31Z [load_day 2023-06-04] 2023-06-04 hour 19: 128352 events in 3 files, snapshot 271277901872404636 (task 9148 ms: download 3811 ms overlapping gunzip 520 ms, parse 2325 ms, sort 62 ms, write 4800 ms, index 359 ms: 3 candidates, 56568 repo-hours, 786 releases; waiting 778 ms; commit 1825 ms)
2026-11-07T21:06:34Z [load_day 2023-06-04] 2023-06-04 hour 20: 120539 events in 3 files, snapshot 20918426500412433 (task 7883 ms: download 1891 ms overlapping gunzip 446 ms, parse 2010 ms, sort 67 ms, write 4213 ms, index 436 ms: 7 candidates, 54005 repo-hours, 795 releases; waiting 394 ms; commit 1691 ms)
2026-11-07T21:06:39Z [load_day 2023-06-04] 2023-06-04 hour 21: 112021 events in 3 files, snapshot 1338950725488675854 (task 7669 ms: download 2040 ms overlapping gunzip 401 ms, parse 1859 ms, sort 69 ms, write 4025 ms, index 332 ms: 7 candidates, 48788 repo-hours, 873 releases; waiting 696 ms; commit 1740 ms)
2026-11-07T21:06:42Z [load_day 2023-06-04] 2023-06-04 hour 22: 99355 events in 2 files, snapshot 5415269455789365935 (task 7200 ms: download 2736 ms overlapping gunzip 394 ms, parse 1700 ms, sort 64 ms, write 3433 ms, index 298 ms: 5 candidates, 42291 repo-hours, 741 releases; waiting 1064 ms; commit 1708 ms)
2026-11-07T21:06:47Z [load_day 2023-06-04] 2023-06-04 hour 23: 97062 events in 2 files, snapshot 7541282740385936462 (task 6918 ms: download 1853 ms overlapping gunzip 383 ms, parse 1786 ms, sort 62 ms, write 3441 ms, index 287 ms: 2 candidates, 41532 repo-hours, 681 releases; waiting 684 ms; commit 1679 ms)
2026-11-07T21:06:53Z [load_day 2023-06-04] 2023-06-04: repo_hour merged from 24 files to 1 (1398480 rows, 5698 ms)
2026-11-07T21:06:55Z [load_day 2023-06-04] 2023-06-04: index committed in 7353 ms (24 + 1 + 24 files)
2026-11-07T21:06:55Z [load_day 2023-06-04] 2023-06-04: 2959185 events in 24 hours
2026-11-07T21:06:55Z [load_day 2023-06-04] attempt 1 succeeded