orcaLog in

Runs · run 1805

load_day 2023-12-02, attempt 1 succeeded

Host
3c2b29713ce8:622729
Started
2026-11-08T03:24:08Z
Ended
2026-11-08T03:27:41Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1805/load_day/2023-12-02/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:24:08Z [load_day 2023-12-02] attempt 1 started
2026-11-08T03:24:09Z [load_day 2023-12-02] 2023-12-02: 2 hours in flight, about 1200 MB
2026-11-08T03:24:31Z [load_day 2023-12-02] 2023-12-02 hour 0: 281996 events in 6 files, snapshot 3412845661670432996 (task 16851 ms: download 9717 ms overlapping gunzip 1030 ms, parse 4827 ms, sort 260 ms, write 9673 ms, index 570 ms: 9 candidates, 74700 repo-hours, 878 releases; waiting 248 ms; commit 2389 ms)
2026-11-08T03:24:34Z [load_day 2023-12-02] 2023-12-02 hour 1: 275374 events in 6 files, snapshot 8228972842096265617 (task 15555 ms: download 8659 ms overlapping gunzip 885 ms, parse 4495 ms, sort 300 ms, write 8847 ms, index 560 ms: 5 candidates, 69841 repo-hours, 610 releases; waiting 208 ms; commit 3456 ms)
2026-11-08T03:24:46Z [load_day 2023-12-02] 2023-12-02 hour 2: 258301 events in 6 files, snapshot 4398135138714594695 (task 13827 ms: download 6679 ms overlapping gunzip 796 ms, parse 3825 ms, sort 242 ms, write 7830 ms, index 365 ms: 3 candidates, 45086 repo-hours, 597 releases; waiting 511 ms; commit 2872 ms)
2026-11-08T03:24:50Z [load_day 2023-12-02] 2023-12-02 hour 3: 261974 events in 6 files, snapshot 9152787791706423782 (task 14673 ms: download 7015 ms overlapping gunzip 915 ms, parse 4497 ms, sort 310 ms, write 7780 ms, index 384 ms: 2 candidates, 44871 repo-hours, 642 releases; waiting 556 ms; commit 3168 ms)
2026-11-08T03:25:00Z [load_day 2023-12-02] 2023-12-02 hour 4: 259641 events in 6 files, snapshot 2029009359483370730 (task 12630 ms: download 5952 ms overlapping gunzip 765 ms, parse 3615 ms, sort 214 ms, write 6960 ms, index 360 ms: 5 candidates, 44154 repo-hours, 499 releases; waiting 476 ms; commit 2847 ms)
2026-11-08T03:25:04Z [load_day 2023-12-02] 2023-12-02 hour 5: 262942 events in 6 files, snapshot 388796366949030942 (task 13286 ms: download 5940 ms overlapping gunzip 820 ms, parse 3905 ms, sort 245 ms, write 7111 ms, index 466 ms: 4 candidates, 46765 repo-hours, 506 releases; waiting 459 ms; commit 2784 ms)
2026-11-08T03:25:16Z [load_day 2023-12-02] 2023-12-02 hour 6: 267313 events in 6 files, snapshot 1990193141890640365 (task 14849 ms: download 6094 ms overlapping gunzip 985 ms, parse 4876 ms, sort 255 ms, write 7564 ms, index 409 ms: 1 candidates, 51998 repo-hours, 843 releases; waiting 505 ms; commit 2999 ms)
2026-11-08T03:25:20Z [load_day 2023-12-02] 2023-12-02 hour 7: 263474 events in 6 files, snapshot 2522607688050885328 (task 14001 ms: download 6171 ms overlapping gunzip 798 ms, parse 4074 ms, sort 216 ms, write 7473 ms, index 662 ms: 1 candidates, 47759 repo-hours, 728 releases; waiting 493 ms; commit 3641 ms)
2026-11-08T03:25:36Z [load_day 2023-12-02] 2023-12-02 hour 8: 264312 events in 6 files, snapshot 2382538362935983133 (task 13798 ms: download 6756 ms overlapping gunzip 815 ms, parse 4280 ms, sort 309 ms, write 7248 ms, index 398 ms: 5 candidates, 50754 repo-hours, 794 releases; waiting 490 ms; commit 7536 ms)
2026-11-08T03:25:40Z [load_day 2023-12-02] 2023-12-02 hour 9: 285457 events in 6 files, snapshot 6751776463267664777 (task 14366 ms: download 6012 ms overlapping gunzip 889 ms, parse 4660 ms, sort 291 ms, write 7358 ms, index 449 ms: 1 candidates, 49635 repo-hours, 785 releases; waiting 448 ms; commit 4492 ms)
2026-11-08T03:25:47Z [load_day 2023-12-02] 2023-12-02 hour 10: 293026 events in 6 files, snapshot 5206607751055330064 (task 15219 ms: download 8259 ms overlapping gunzip 893 ms, parse 4476 ms, sort 318 ms, write 8593 ms, index 452 ms: 1 candidates, 52182 repo-hours, 528 releases; waiting 204 ms; commit 2914 ms)
2026-11-08T03:26:06Z [load_day 2023-12-02] 2023-12-02 hour 11: 292741 events in 6 files, snapshot 5485667965433399068 (task 14464 ms: download 7065 ms overlapping gunzip 783 ms, parse 3985 ms, sort 236 ms, write 8144 ms, index 551 ms: 9 candidates, 51593 repo-hours, 546 releases; waiting 504 ms; commit 12767 ms)
2026-11-08T03:26:08Z [load_day 2023-12-02] 2023-12-02 hour 12: 314289 events in 7 files, snapshot 9123557657802872786 (task 18290 ms: download 7957 ms overlapping gunzip 1063 ms, parse 5624 ms, sort 270 ms, write 9613 ms, index 880 ms: 8 candidates, 79063 repo-hours, 619 releases; waiting 498 ms; commit 2403 ms)
2026-11-08T03:26:16Z [load_day 2023-12-02] 2023-12-02 hour 13: 300431 events in 7 files, snapshot 8899693530669798793 (task 16661 ms: download 9538 ms overlapping gunzip 1019 ms, parse 5111 ms, sort 270 ms, write 8987 ms, index 596 ms: 5 candidates, 70996 repo-hours, 620 releases; waiting 409 ms; commit 4173 ms)
2026-11-08T03:26:26Z [load_day 2023-12-02] 2023-12-02 hour 14: 297031 events in 6 files, snapshot 4530812275203354643 (task 15800 ms: download 8375 ms overlapping gunzip 876 ms, parse 4398 ms, sort 283 ms, write 8850 ms, index 469 ms: 8 candidates, 61693 repo-hours, 644 releases; waiting 511 ms; commit 2736 ms)
2026-11-08T03:26:31Z [load_day 2023-12-02] 2023-12-02 hour 15: 290961 events in 6 files, snapshot 310386249507381968 (task 15831 ms: download 8700 ms overlapping gunzip 966 ms, parse 4796 ms, sort 247 ms, write 8461 ms, index 573 ms: 7 candidates, 61078 repo-hours, 769 releases; waiting 476 ms; commit 2959 ms)
2026-11-08T03:26:43Z [load_day 2023-12-02] 2023-12-02 hour 16: 285395 events in 6 files, snapshot 7452587086904057928 (task 15763 ms: download 8445 ms overlapping gunzip 859 ms, parse 4381 ms, sort 307 ms, write 8991 ms, index 481 ms: 6 candidates, 60219 repo-hours, 816 releases; waiting 494 ms; commit 3086 ms)
2026-11-08T03:26:47Z [load_day 2023-12-02] 2023-12-02 hour 17: 285978 events in 6 files, snapshot 7722527735060317536 (task 15227 ms: download 8219 ms overlapping gunzip 887 ms, parse 4432 ms, sort 285 ms, write 8475 ms, index 425 ms: 4 candidates, 57408 repo-hours, 586 releases; waiting 507 ms; commit 2427 ms)
2026-11-08T03:26:59Z [load_day 2023-12-02] 2023-12-02 hour 18: 297863 events in 6 files, snapshot 5236311856351317410 (task 14192 ms: download 7805 ms overlapping gunzip 775 ms, parse 3913 ms, sort 245 ms, write 7976 ms, index 406 ms: 4 candidates, 60936 repo-hours, 727 releases; waiting 499 ms; commit 3625 ms)
2026-11-08T03:27:02Z [load_day 2023-12-02] 2023-12-02 hour 19: 292923 events in 6 files, snapshot 7294022812016745280 (task 13387 ms: download 7153 ms overlapping gunzip 824 ms, parse 4155 ms, sort 276 ms, write 6632 ms, index 352 ms: 5 candidates, 51004 repo-hours, 588 releases; waiting 524 ms; commit 2420 ms)
2026-11-08T03:27:10Z [load_day 2023-12-02] 2023-12-02 hour 20: 302924 events in 7 files, snapshot 2807551195534232972 (task 11881 ms: download 6179 ms overlapping gunzip 677 ms, parse 3395 ms, sort 248 ms, write 6375 ms, index 345 ms: 2 candidates, 50214 repo-hours, 910 releases; waiting 498 ms; commit 2498 ms)
2026-11-08T03:27:16Z [load_day 2023-12-02] 2023-12-02 hour 21: 304397 events in 7 files, snapshot 6248123862776559255 (task 12187 ms: download 6014 ms overlapping gunzip 741 ms, parse 3664 ms, sort 246 ms, write 6277 ms, index 503 ms: 2 candidates, 45740 repo-hours, 653 releases; waiting 485 ms; commit 3414 ms)
2026-11-08T03:27:26Z [load_day 2023-12-02] 2023-12-02 hour 22: 302850 events in 7 files, snapshot 5645645212572313640 (task 14168 ms: download 7195 ms overlapping gunzip 920 ms, parse 4251 ms, sort 262 ms, write 7562 ms, index 377 ms: 5 candidates, 42630 repo-hours, 628 releases; waiting 492 ms; commit 2830 ms)
2026-11-08T03:27:32Z [load_day 2023-12-02] 2023-12-02 hour 23: 302914 events in 7 files, snapshot 5692782929988787264 (task 12944 ms: download 7380 ms overlapping gunzip 720 ms, parse 4042 ms, sort 336 ms, write 6705 ms, index 421 ms: 10 candidates, 39432 repo-hours, 493 releases; waiting 472 ms; commit 4712 ms)
2026-11-08T03:27:38Z [load_day 2023-12-02] 2023-12-02: repo_hour merged from 24 files to 1 (1309751 rows, 6449 ms)
2026-11-08T03:27:41Z [load_day 2023-12-02] 2023-12-02: index committed in 8890 ms (24 + 1 + 24 files)
2026-11-08T03:27:41Z [load_day 2023-12-02] 2023-12-02: 6844507 events in 24 hours
2026-11-08T03:27:41Z [load_day 2023-12-02] attempt 1 succeeded