orcaLog in

Runs · run 1810

load_day 2023-12-03, attempt 1 succeeded

Host
3c2b29713ce8:627935
Started
2026-11-08T03:27:08Z
Ended
2026-11-08T03:30:32Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
not measured
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
not measured
Log file
s3://census-mlake-dev/logs/gharchive/1810/load_day/2023-12-03/1.log
Code
pipelines/gharchive.mojo:116

Log

2026-11-08T03:27:08Z [load_day 2023-12-03] attempt 1 started
2026-11-08T03:27:09Z [load_day 2023-12-03] 2023-12-03: 2 hours in flight, about 1200 MB
2026-11-08T03:27:29Z [load_day 2023-12-03] 2023-12-03 hour 0: 310482 events in 7 files, snapshot 8860389171466411666 (task 14400 ms: download 7969 ms overlapping gunzip 879 ms, parse 4687 ms, sort 290 ms, write 7386 ms, index 364 ms: 9 candidates, 57120 repo-hours, 658 releases; waiting 471 ms; commit 3414 ms)
2026-11-08T03:27:34Z [load_day 2023-12-03] 2023-12-03 hour 1: 320864 events in 7 files, snapshot 7717813009742516697 (task 14229 ms: download 7501 ms overlapping gunzip 832 ms, parse 4468 ms, sort 397 ms, write 7404 ms, index 417 ms: 3 candidates, 53885 repo-hours, 499 releases; waiting 466 ms; commit 4517 ms)
2026-11-08T03:27:42Z [load_day 2023-12-03] 2023-12-03 hour 2: 325649 events in 7 files, snapshot 4265088307864957407 (task 10761 ms: download 4352 ms overlapping gunzip 599 ms, parse 3401 ms, sort 269 ms, write 5545 ms, index 217 ms: 2 candidates, 30367 repo-hours, 321 releases; waiting 468 ms; commit 2296 ms)
2026-11-08T03:27:44Z [load_day 2023-12-03] 2023-12-03 hour 3: 325063 events in 7 files, snapshot 6628823523710232536 (task 10503 ms: download 5248 ms overlapping gunzip 576 ms, parse 3139 ms, sort 317 ms, write 5538 ms, index 187 ms: 4 candidates, 28715 repo-hours, 389 releases; waiting 524 ms; commit 2469 ms)
2026-11-08T03:27:56Z [load_day 2023-12-03] 2023-12-03 hour 4: 315679 events in 7 files, snapshot 3100465971739813387 (task 12021 ms: download 5430 ms overlapping gunzip 676 ms, parse 3631 ms, sort 245 ms, write 6392 ms, index 308 ms: 6 candidates, 37046 repo-hours, 401 releases; waiting 537 ms; commit 3193 ms)
2026-11-08T03:27:59Z [load_day 2023-12-03] 2023-12-03 hour 5: 315897 events in 7 files, snapshot 8536172166063599680 (task 13028 ms: download 5894 ms overlapping gunzip 817 ms, parse 3989 ms, sort 272 ms, write 6670 ms, index 537 ms: 6 candidates, 40942 repo-hours, 413 releases; waiting 488 ms; commit 2749 ms)
2026-11-08T03:28:09Z [load_day 2023-12-03] 2023-12-03 hour 6: 326558 events in 7 files, snapshot 7424195403233065759 (task 12240 ms: download 5110 ms overlapping gunzip 699 ms, parse 3765 ms, sort 333 ms, write 6352 ms, index 269 ms: 2 candidates, 38036 repo-hours, 346 releases; waiting 510 ms; commit 3014 ms)
2026-11-08T03:28:14Z [load_day 2023-12-03] 2023-12-03 hour 7: 297531 events in 6 files, snapshot 8403987401377273233 (task 12517 ms: download 5182 ms overlapping gunzip 784 ms, parse 3839 ms, sort 275 ms, write 6274 ms, index 592 ms: 7 candidates, 41998 repo-hours, 469 releases; waiting 504 ms; commit 4827 ms)
2026-11-08T03:28:24Z [load_day 2023-12-03] 2023-12-03 hour 8: 321442 events in 7 files, snapshot 8868882449355149082 (task 14412 ms: download 7920 ms overlapping gunzip 838 ms, parse 4503 ms, sort 356 ms, write 7581 ms, index 352 ms: 5 candidates, 44449 repo-hours, 406 releases; waiting 496 ms; commit 2623 ms)
2026-11-08T03:28:30Z [load_day 2023-12-03] 2023-12-03 hour 9: 302791 events in 7 files, snapshot 6955874159507173584 (task 14443 ms: download 7674 ms overlapping gunzip 817 ms, parse 4303 ms, sort 330 ms, write 7667 ms, index 540 ms: 7 candidates, 46749 repo-hours, 612 releases; waiting 454 ms; commit 4358 ms)
2026-11-08T03:28:42Z [load_day 2023-12-03] 2023-12-03 hour 10: 311944 events in 7 files, snapshot 5184707614096448653 (task 15690 ms: download 8323 ms overlapping gunzip 863 ms, parse 4244 ms, sort 365 ms, write 8769 ms, index 646 ms: 6 candidates, 50771 repo-hours, 496 releases; waiting 517 ms; commit 3197 ms)
2026-11-08T03:28:46Z [load_day 2023-12-03] 2023-12-03 hour 11: 318886 events in 7 files, snapshot 2717838695404594026 (task 15417 ms: download 8006 ms overlapping gunzip 948 ms, parse 4873 ms, sort 358 ms, write 7809 ms, index 649 ms: 4 candidates, 43784 repo-hours, 393 releases; waiting 526 ms; commit 4335 ms)
2026-11-08T03:28:59Z [load_day 2023-12-03] 2023-12-03 hour 12: 326296 events in 7 files, snapshot 1712251128386577994 (task 15478 ms: download 7643 ms overlapping gunzip 864 ms, parse 4941 ms, sort 414 ms, write 7900 ms, index 538 ms: 3 candidates, 63614 repo-hours, 376 releases; waiting 549 ms; commit 3058 ms)
2026-11-08T03:29:01Z [load_day 2023-12-03] 2023-12-03 hour 13: 323434 events in 7 files, snapshot 8345215433000077050 (task 15544 ms: download 8212 ms overlapping gunzip 946 ms, parse 4584 ms, sort 382 ms, write 8286 ms, index 577 ms: 3 candidates, 56935 repo-hours, 416 releases; waiting 510 ms; commit 2788 ms)
2026-11-08T03:29:15Z [load_day 2023-12-03] 2023-12-03 hour 14: 318823 events in 7 files, snapshot 1897971508922804417 (task 15191 ms: download 8080 ms overlapping gunzip 831 ms, parse 4256 ms, sort 338 ms, write 8691 ms, index 382 ms: 4 candidates, 50794 repo-hours, 442 releases; waiting 476 ms; commit 3282 ms)
2026-11-08T03:29:18Z [load_day 2023-12-03] 2023-12-03 hour 15: 323923 events in 7 files, snapshot 9158634688115048600 (task 15019 ms: download 7644 ms overlapping gunzip 802 ms, parse 4225 ms, sort 333 ms, write 8486 ms, index 483 ms: 8 candidates, 50836 repo-hours, 561 releases; waiting 485 ms; commit 3294 ms)
2026-11-08T03:29:31Z [load_day 2023-12-03] 2023-12-03 hour 16: 321074 events in 7 files, snapshot 7271661145933964139 (task 14553 ms: download 7488 ms overlapping gunzip 780 ms, parse 4051 ms, sort 263 ms, write 8303 ms, index 396 ms: 3 candidates, 49882 repo-hours, 560 releases; waiting 497 ms; commit 2847 ms)
2026-11-08T03:29:35Z [load_day 2023-12-03] 2023-12-03 hour 17: 321533 events in 7 files, snapshot 227073512526450036 (task 13904 ms: download 7202 ms overlapping gunzip 838 ms, parse 4691 ms, sort 361 ms, write 6955 ms, index 379 ms: 5 candidates, 46663 repo-hours, 411 releases; waiting 447 ms; commit 3316 ms)
2026-11-08T03:29:48Z [load_day 2023-12-03] 2023-12-03 hour 18: 327057 events in 7 files, snapshot 937625327378853608 (task 15575 ms: download 8316 ms overlapping gunzip 956 ms, parse 4796 ms, sort 363 ms, write 8281 ms, index 363 ms: 3 candidates, 49708 repo-hours, 495 releases; waiting 507 ms; commit 3285 ms)
2026-11-08T03:29:52Z [load_day 2023-12-03] 2023-12-03 hour 19: 324198 events in 7 files, snapshot 8839228709150536086 (task 15726 ms: download 8684 ms overlapping gunzip 947 ms, parse 4848 ms, sort 347 ms, write 8265 ms, index 582 ms: 2 candidates, 44176 repo-hours, 485 releases; waiting 446 ms; commit 3110 ms)
2026-11-08T03:30:04Z [load_day 2023-12-03] 2023-12-03 hour 20: 325000 events in 7 files, snapshot 5051595643453969649 (task 14317 ms: download 7251 ms overlapping gunzip 814 ms, parse 4121 ms, sort 314 ms, write 7919 ms, index 377 ms: 5 candidates, 45647 repo-hours, 575 releases; waiting 496 ms; commit 2971 ms)
2026-11-08T03:30:07Z [load_day 2023-12-03] 2023-12-03 hour 21: 322041 events in 7 files, snapshot 1233526833260436444 (task 14384 ms: download 6885 ms overlapping gunzip 896 ms, parse 4532 ms, sort 322 ms, write 7277 ms, index 556 ms: 6 candidates, 41332 repo-hours, 496 releases; waiting 478 ms; commit 3474 ms)
2026-11-08T03:30:21Z [load_day 2023-12-03] 2023-12-03 hour 22: 325359 events in 7 files, snapshot 2697133030479297489 (task 15471 ms: download 7977 ms overlapping gunzip 862 ms, parse 4592 ms, sort 345 ms, write 8547 ms, index 367 ms: 5 candidates, 38824 repo-hours, 924 releases; waiting 491 ms; commit 3660 ms)
2026-11-08T03:30:24Z [load_day 2023-12-03] 2023-12-03 hour 23: 327965 events in 7 files, snapshot 6169393362679749496 (task 15758 ms: download 8632 ms overlapping gunzip 898 ms, parse 4905 ms, sort 360 ms, write 8535 ms, index 326 ms: 4 candidates, 37015 repo-hours, 389 releases; waiting 450 ms; commit 2696 ms)
2026-11-08T03:30:30Z [load_day 2023-12-03] 2023-12-03: repo_hour merged from 24 files to 1 (1089288 rows, 5626 ms)
2026-11-08T03:30:32Z [load_day 2023-12-03] 2023-12-03: index committed in 8272 ms (24 + 1 + 24 files)
2026-11-08T03:30:32Z [load_day 2023-12-03] 2023-12-03: 7679489 events in 24 hours
2026-11-08T03:30:32Z [load_day 2023-12-03] attempt 1 succeeded