orcaLog in

Runs · run 1830

load_day 2023-12-07, attempt 1 succeeded

Host
73ccef5e07c2:61
Started
2026-11-08T03:37:07Z
Ended
2026-11-08T03:41:49Z
Peak memoryThe most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗
1096 MB
CPU usedThe CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗
477.1 s, up to 2.7 cores at once
Log file
s3://census-mlake-dev/logs/gharchive/1830/load_day/2023-12-07/1.log
Code
pipelines/gharchive.mojo:135

Log

2026-11-08T03:37:07Z [load_day 2023-12-07] attempt 1 started
2026-11-08T03:37:08Z [load_day 2023-12-07] 2023-12-07: 2 hours in flight, about 1200 MB
2026-11-08T03:37:35Z [load_day 2023-12-07] 2023-12-07 hour 0: 299572 events in 6 files, snapshot 7152323893060079782 (task 20479 ms: download 12373 ms overlapping gunzip 1185 ms, parse 5957 ms, sort 450 ms, write 11636 ms, index 558 ms: 8 candidates, 73500 repo-hours, 724 releases; waiting 464 ms; commit 4047 ms)
2026-11-08T03:37:41Z [load_day 2023-12-07] 2023-12-07 hour 1: 298110 events in 6 files, snapshot 6209698931826726544 (task 20027 ms: download 11695 ms overlapping gunzip 1118 ms, parse 5497 ms, sort 391 ms, write 11617 ms, index 634 ms: 5 candidates, 76514 repo-hours, 630 releases; waiting 453 ms; commit 5758 ms)
2026-11-08T03:37:53Z [load_day 2023-12-07] 2023-12-07 hour 2: 292398 events in 6 files, snapshot 4048800483215374681 (task 17081 ms: download 9859 ms overlapping gunzip 1041 ms, parse 4783 ms, sort 266 ms, write 9858 ms, index 394 ms: 3 candidates, 55723 repo-hours, 568 releases; waiting 485 ms; commit 4341 ms)
2026-11-08T03:37:59Z [load_day 2023-12-07] 2023-12-07 hour 3: 294178 events in 6 files, snapshot 7230102267326834342 (task 18074 ms: download 9855 ms overlapping gunzip 1073 ms, parse 5029 ms, sort 391 ms, write 10115 ms, index 611 ms: 7 candidates, 54241 repo-hours, 644 releases; waiting 526 ms; commit 4330 ms)
2026-11-08T03:38:11Z [load_day 2023-12-07] 2023-12-07 hour 4: 301605 events in 7 files, snapshot 7183620948319052811 (task 17523 ms: download 9344 ms overlapping gunzip 982 ms, parse 4899 ms, sort 281 ms, write 10054 ms, index 628 ms: 5 candidates, 49562 repo-hours, 486 releases; waiting 451 ms; commit 3201 ms)
2026-11-08T03:38:16Z [load_day 2023-12-07] 2023-12-07 hour 5: 296074 events in 6 files, snapshot 3031543996048490035 (task 17369 ms: download 9487 ms overlapping gunzip 1062 ms, parse 5075 ms, sort 430 ms, write 9468 ms, index 541 ms: 7 candidates, 53660 repo-hours, 678 releases; waiting 473 ms; commit 3247 ms)
2026-11-08T03:38:32Z [load_day 2023-12-07] 2023-12-07 hour 6: 301341 events in 7 files, snapshot 4674948720674520499 (task 18793 ms: download 10312 ms overlapping gunzip 1335 ms, parse 6170 ms, sort 409 ms, write 9481 ms, index 930 ms: 7 candidates, 63611 repo-hours, 615 releases; waiting 251 ms; commit 3424 ms)
2026-11-08T03:38:38Z [load_day 2023-12-07] 2023-12-07 hour 7: 303025 events in 7 files, snapshot 6549612073304646013 (task 20014 ms: download 10497 ms overlapping gunzip 1130 ms, parse 5692 ms, sort 328 ms, write 11507 ms, index 536 ms: 6 candidates, 63643 repo-hours, 604 releases; waiting 561 ms; commit 3303 ms)
2026-11-08T03:38:54Z [load_day 2023-12-07] 2023-12-07 hour 8: 307321 events in 7 files, snapshot 1941105644991478728 (task 20998 ms: download 13649 ms overlapping gunzip 1131 ms, parse 5033 ms, sort 299 ms, write 13211 ms, index 540 ms: 10 candidates, 67740 repo-hours, 672 releases; waiting 511 ms; commit 3298 ms)
2026-11-08T03:39:01Z [load_day 2023-12-07] 2023-12-07 hour 9: 303718 events in 7 files, snapshot 7946840555613832320 (task 22240 ms: download 14313 ms overlapping gunzip 1282 ms, parse 5880 ms, sort 302 ms, write 13375 ms, index 648 ms: 5 candidates, 70528 repo-hours, 777 releases; waiting 501 ms; commit 2879 ms)
2026-11-08T03:39:19Z [load_day 2023-12-07] 2023-12-07 hour 10: 304376 events in 7 files, snapshot 6060081339672696630 (task 22962 ms: download 15596 ms overlapping gunzip 1266 ms, parse 6039 ms, sort 481 ms, write 13757 ms, index 572 ms: 7 candidates, 70131 repo-hours, 765 releases; waiting 514 ms; commit 4294 ms)
2026-11-08T03:39:25Z [load_day 2023-12-07] 2023-12-07 hour 11: 305032 events in 7 files, snapshot 8565064479773262565 (task 22117 ms: download 14258 ms overlapping gunzip 1263 ms, parse 5887 ms, sort 384 ms, write 13121 ms, index 726 ms: 5 candidates, 67748 repo-hours, 739 releases; waiting 457 ms; commit 3569 ms)
2026-11-08T03:39:40Z [load_day 2023-12-07] 2023-12-07 hour 12: 313650 events in 7 files, snapshot 9061002031872212434 (task 20220 ms: download 12374 ms overlapping gunzip 1068 ms, parse 4980 ms, sort 335 ms, write 12146 ms, index 899 ms: 7 candidates, 90195 repo-hours, 766 releases; waiting 507 ms; commit 3439 ms)
2026-11-08T03:39:49Z [load_day 2023-12-07] 2023-12-07 hour 13: 302227 events in 7 files, snapshot 4243930040022343455 (task 23672 ms: download 13841 ms overlapping gunzip 1382 ms, parse 6803 ms, sort 340 ms, write 13665 ms, index 685 ms: 11 candidates, 85640 repo-hours, 874 releases; waiting 509 ms; commit 2803 ms)
2026-11-08T03:40:08Z [load_day 2023-12-07] 2023-12-07 hour 14: 286314 events in 6 files, snapshot 4064153126888755819 (task 27101 ms: download 18506 ms overlapping gunzip 1498 ms, parse 6931 ms, sort 342 ms, write 16719 ms, index 805 ms: 11 candidates, 85186 repo-hours, 957 releases; waiting 526 ms; commit 3735 ms)
2026-11-08T03:40:15Z [load_day 2023-12-07] 2023-12-07 hour 15: 290913 events in 6 files, snapshot 4585522534147698615 (task 24741 ms: download 16474 ms overlapping gunzip 1255 ms, parse 5887 ms, sort 396 ms, write 15593 ms, index 786 ms: 10 candidates, 84149 repo-hours, 977 releases; waiting 529 ms; commit 3079 ms)
2026-11-08T03:40:31Z [load_day 2023-12-07] 2023-12-07 hour 16: 290231 events in 6 files, snapshot 1811529132277224646 (task 22398 ms: download 15381 ms overlapping gunzip 1219 ms, parse 5498 ms, sort 258 ms, write 13841 ms, index 726 ms: 5 candidates, 79878 repo-hours, 1002 releases; waiting 550 ms; commit 2977 ms)
2026-11-08T03:40:40Z [load_day 2023-12-07] 2023-12-07 hour 17: 284609 events in 6 files, snapshot 8489813094541795807 (task 23261 ms: download 13517 ms overlapping gunzip 1364 ms, parse 6433 ms, sort 341 ms, write 13525 ms, index 846 ms: 7 candidates, 74791 repo-hours, 824 releases; waiting 466 ms; commit 3100 ms)
2026-11-08T03:40:54Z [load_day 2023-12-07] 2023-12-07 hour 18: 285473 events in 6 files, snapshot 2331495074022083155 (task 21173 ms: download 12336 ms overlapping gunzip 1257 ms, parse 5817 ms, sort 232 ms, write 12477 ms, index 596 ms: 9 candidates, 77796 repo-hours, 882 releases; waiting 486 ms; commit 3167 ms)
2026-11-08T03:41:02Z [load_day 2023-12-07] 2023-12-07 hour 19: 272898 events in 6 files, snapshot 4891456641957385510 (task 21232 ms: download 12757 ms overlapping gunzip 1135 ms, parse 5267 ms, sort 327 ms, write 12921 ms, index 699 ms: 6 candidates, 68670 repo-hours, 948 releases; waiting 512 ms; commit 3121 ms)
2026-11-08T03:41:14Z [load_day 2023-12-07] 2023-12-07 hour 20: 268258 events in 6 files, snapshot 7729832171783396271 (task 19886 ms: download 11187 ms overlapping gunzip 1290 ms, parse 6194 ms, sort 292 ms, write 10737 ms, index 638 ms: 7 candidates, 63744 repo-hours, 709 releases; waiting 497 ms; commit 2933 ms)
2026-11-08T03:41:23Z [load_day 2023-12-07] 2023-12-07 hour 21: 275253 events in 6 files, snapshot 6897049670659672621 (task 20358 ms: download 11883 ms overlapping gunzip 1185 ms, parse 5137 ms, sort 284 ms, write 12527 ms, index 414 ms: 5 candidates, 60835 repo-hours, 782 releases; waiting 460 ms; commit 2471 ms)
2026-11-08T03:41:32Z [load_day 2023-12-07] 2023-12-07 hour 22: 275380 events in 6 files, snapshot 6403298280511025567 (task 16574 ms: download 9770 ms overlapping gunzip 1048 ms, parse 4787 ms, sort 287 ms, write 9362 ms, index 402 ms: 8 candidates, 58065 repo-hours, 784 releases; waiting 390 ms; commit 2498 ms)
2026-11-08T03:41:39Z [load_day 2023-12-07] 2023-12-07 hour 23: 270429 events in 6 files, snapshot 4583954366107742806 (task 14068 ms: download 8233 ms overlapping gunzip 839 ms, parse 4062 ms, sort 235 ms, write 7585 ms, index 580 ms: 5 candidates, 53416 repo-hours, 637 releases; waiting 523 ms; commit 2611 ms)
2026-11-08T03:41:47Z [load_day 2023-12-07] 2023-12-07: repo_hour merged from 24 files to 1 (1648966 rows, 7732 ms)
2026-11-08T03:41:49Z [load_day 2023-12-07] 2023-12-07: index committed in 10511 ms (24 + 1 + 24 files)
2026-11-08T03:41:49Z [load_day 2023-12-07] 2023-12-07: 7022385 events in 24 hours
2026-11-08T03:41:49Z [load_day 2023-12-07] attempt 1 succeeded