load_day 2024-04-12, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:108846

Started

2026-11-08T05:34:20Z

Ended

2026-11-08T05:36:16Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

690 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

176.2 s, up to 3.1 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2315/load_day/2024-04-12/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T05:34:20Z [load_day 2024-04-12] attempt 1 started
2026-11-08T05:34:21Z [load_day 2024-04-12] 2024-04-12: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T05:34:30Z [load_day 2024-04-12] 2024-04-12 hour 0: 212493 events in 0 files, snapshot none (task 8990 ms: download 6036 ms overlapping gunzip 1390 ms, parse 6107 ms, sort 0 ms, write 0 ms, index 709 ms: 8 candidates, 98285 repo-hours, 1110 releases; waiting 509 ms; commit 0 ms)
2026-11-08T05:34:30Z [load_day 2024-04-12] 2024-04-12 hour 1: 214149 events in 0 files, snapshot none (task 9061 ms: download 6160 ms overlapping gunzip 1410 ms, parse 5977 ms, sort 0 ms, write 0 ms, index 888 ms: 10 candidates, 103142 repo-hours, 790 releases; waiting 480 ms; commit 0 ms)
2026-11-08T05:34:38Z [load_day 2024-04-12] 2024-04-12 hour 2: 189936 events in 0 files, snapshot none (task 7458 ms: download 4672 ms overlapping gunzip 1159 ms, parse 4784 ms, sort 0 ms, write 0 ms, index 703 ms: 8 candidates, 68225 repo-hours, 679 releases; waiting 506 ms; commit 0 ms)
2026-11-08T05:34:38Z [load_day 2024-04-12] 2024-04-12 hour 3: 185242 events in 0 files, snapshot none (task 7407 ms: download 4745 ms overlapping gunzip 1151 ms, parse 4738 ms, sort 0 ms, write 0 ms, index 693 ms: 3 candidates, 65013 repo-hours, 1003 releases; waiting 504 ms; commit 0 ms)
2026-11-08T05:34:45Z [load_day 2024-04-12] 2024-04-12 hour 4: 178789 events in 0 files, snapshot none (task 6604 ms: download 3995 ms overlapping gunzip 1047 ms, parse 4085 ms, sort 0 ms, write 0 ms, index 671 ms: 21 candidates, 63900 repo-hours, 696 releases; waiting 516 ms; commit 0 ms)
2026-11-08T05:34:45Z [load_day 2024-04-12] 2024-04-12 hour 5: 181901 events in 0 files, snapshot none (task 6330 ms: download 3756 ms overlapping gunzip 961 ms, parse 3831 ms, sort 0 ms, write 0 ms, index 708 ms: 4 candidates, 65286 repo-hours, 823 releases; waiting 545 ms; commit 0 ms)
2026-11-08T05:34:53Z [load_day 2024-04-12] 2024-04-12 hour 6: 215338 events in 0 files, snapshot none (task 8005 ms: download 5479 ms overlapping gunzip 1187 ms, parse 5395 ms, sort 0 ms, write 0 ms, index 601 ms: 4 candidates, 82089 repo-hours, 1017 releases; waiting 497 ms; commit 0 ms)
2026-11-08T05:34:54Z [load_day 2024-04-12] 2024-04-12 hour 7: 219945 events in 0 files, snapshot none (task 8048 ms: download 5237 ms overlapping gunzip 1366 ms, parse 5313 ms, sort 0 ms, write 0 ms, index 888 ms: 8 candidates, 82522 repo-hours, 1060 releases; waiting 208 ms; commit 0 ms)
2026-11-08T05:35:05Z [load_day 2024-04-12] 2024-04-12 hour 8: 230705 events in 0 files, snapshot none (task 8743 ms: download 5795 ms overlapping gunzip 1484 ms, parse 6013 ms, sort 0 ms, write 0 ms, index 751 ms: 5 candidates, 88583 repo-hours, 1321 releases; waiting 179 ms; commit 0 ms)
2026-11-08T05:35:05Z [load_day 2024-04-12] 2024-04-12 hour 9: 225569 events in 0 files, snapshot none (task 9103 ms: download 6516 ms overlapping gunzip 1497 ms, parse 6191 ms, sort 0 ms, write 0 ms, index 647 ms: 6 candidates, 87625 repo-hours, 1371 releases; waiting 489 ms; commit 0 ms)
2026-11-08T05:35:14Z [load_day 2024-04-12] 2024-04-12 hour 10: 217643 events in 0 files, snapshot none (task 8485 ms: download 5641 ms overlapping gunzip 1416 ms, parse 5601 ms, sort 0 ms, write 0 ms, index 662 ms: 4 candidates, 82615 repo-hours, 1100 releases; waiting 499 ms; commit 0 ms)
2026-11-08T05:35:15Z [load_day 2024-04-12] 2024-04-12 hour 11: 224101 events in 0 files, snapshot none (task 8780 ms: download 6159 ms overlapping gunzip 1411 ms, parse 5966 ms, sort 0 ms, write 0 ms, index 607 ms: 9 candidates, 81027 repo-hours, 1070 releases; waiting 504 ms; commit 0 ms)
2026-11-08T05:35:25Z [load_day 2024-04-12] 2024-04-12 hour 12: 283571 events in 0 files, snapshot none (task 9988 ms: download 6904 ms overlapping gunzip 1576 ms, parse 6838 ms, sort 0 ms, write 0 ms, index 803 ms: 10 candidates, 116611 repo-hours, 1262 releases; waiting 497 ms; commit 0 ms)
2026-11-08T05:35:25Z [load_day 2024-04-12] 2024-04-12 hour 13: 256535 events in 0 files, snapshot none (task 8895 ms: download 6331 ms overlapping gunzip 1418 ms, parse 6022 ms, sort 0 ms, write 0 ms, index 746 ms: 13 candidates, 109692 repo-hours, 1277 releases; waiting 492 ms; commit 0 ms)
2026-11-08T05:35:35Z [load_day 2024-04-12] 2024-04-12 hour 14: 255519 events in 0 files, snapshot none (task 9834 ms: download 7127 ms overlapping gunzip 1540 ms, parse 6348 ms, sort 0 ms, write 0 ms, index 733 ms: 17 candidates, 100999 repo-hours, 1305 releases; waiting 781 ms; commit 0 ms)
2026-11-08T05:35:35Z [load_day 2024-04-12] 2024-04-12 hour 15: 244766 events in 0 files, snapshot none (task 9081 ms: download 6540 ms overlapping gunzip 1499 ms, parse 5867 ms, sort 0 ms, write 0 ms, index 692 ms: 18 candidates, 94909 repo-hours, 1366 releases; waiting 689 ms; commit 0 ms)
2026-11-08T05:35:43Z [load_day 2024-04-12] 2024-04-12 hour 16: 237801 events in 0 files, snapshot none (task 7483 ms: download 4948 ms overlapping gunzip 1117 ms, parse 4633 ms, sort 0 ms, write 0 ms, index 645 ms: 13 candidates, 88862 repo-hours, 1066 releases; waiting 767 ms; commit 0 ms)
2026-11-08T05:35:43Z [load_day 2024-04-12] 2024-04-12 hour 17: 225024 events in 0 files, snapshot none (task 6687 ms: download 4243 ms overlapping gunzip 1016 ms, parse 4208 ms, sort 0 ms, write 0 ms, index 604 ms: 10 candidates, 85225 repo-hours, 997 releases; waiting 565 ms; commit 0 ms)
2026-11-08T05:35:51Z [load_day 2024-04-12] 2024-04-12 hour 18: 222299 events in 0 files, snapshot none (task 7236 ms: download 3999 ms overlapping gunzip 1046 ms, parse 4329 ms, sort 0 ms, write 0 ms, index 915 ms: 11 candidates, 86219 repo-hours, 1098 releases; waiting 599 ms; commit 0 ms)
2026-11-08T05:35:51Z [load_day 2024-04-12] 2024-04-12 hour 19: 205275 events in 0 files, snapshot none (task 6911 ms: download 3885 ms overlapping gunzip 1060 ms, parse 4219 ms, sort 0 ms, write 0 ms, index 749 ms: 12 candidates, 73374 repo-hours, 1019 releases; waiting 506 ms; commit 0 ms)
2026-11-08T05:35:58Z [load_day 2024-04-12] 2024-04-12 hour 20: 201660 events in 0 files, snapshot none (task 7004 ms: download 4095 ms overlapping gunzip 1079 ms, parse 4452 ms, sort 0 ms, write 0 ms, index 643 ms: 12 candidates, 72183 repo-hours, 916 releases; waiting 527 ms; commit 0 ms)
2026-11-08T05:35:58Z [load_day 2024-04-12] 2024-04-12 hour 21: 191672 events in 0 files, snapshot none (task 7003 ms: download 4424 ms overlapping gunzip 1128 ms, parse 4565 ms, sort 0 ms, write 0 ms, index 511 ms: 12 candidates, 65743 repo-hours, 953 releases; waiting 576 ms; commit 0 ms)
2026-11-08T05:36:05Z [load_day 2024-04-12] 2024-04-12 hour 22: 174531 events in 0 files, snapshot none (task 6645 ms: download 4305 ms overlapping gunzip 1114 ms, parse 4279 ms, sort 0 ms, write 0 ms, index 445 ms: 6 candidates, 58339 repo-hours, 860 releases; waiting 545 ms; commit 0 ms)
2026-11-08T05:36:05Z [load_day 2024-04-12] 2024-04-12 hour 23: 161317 events in 0 files, snapshot none (task 5951 ms: download 3323 ms overlapping gunzip 938 ms, parse 3854 ms, sort 0 ms, write 0 ms, index 387 ms: 3 candidates, 53314 repo-hours, 677 releases; waiting 501 ms; commit 0 ms)
2026-11-08T05:36:14Z [load_day 2024-04-12] 2024-04-12: repo_hour merged from 24 files to 1 (1973782 rows, 8504 ms)
2026-11-08T05:36:16Z [load_day 2024-04-12] 2024-04-12: index committed in 11064 ms (24 + 1 + 24 files)
2026-11-08T05:36:16Z [load_day 2024-04-12] 2024-04-12: 5155781 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T05:36:16Z [load_day 2024-04-12] attempt 1 succeeded