load_day 2024-01-10, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:17508

Started

2026-11-08T04:05:20Z

Ended

2026-11-08T04:07:13Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

694 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

175.2 s, up to 3.3 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/1954/load_day/2024-01-10/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:05:20Z [load_day 2024-01-10] attempt 1 started
2026-11-08T04:05:22Z [load_day 2024-01-10] 2024-01-10: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:05:29Z [load_day 2024-01-10] 2024-01-10 hour 0: 205987 events in 0 files, snapshot none (task 6729 ms: download 4292 ms overlapping gunzip 1019 ms, parse 4398 ms, sort 0 ms, write 0 ms, index 588 ms: 6 candidates, 83575 repo-hours, 1229 releases; waiting 502 ms; commit 0 ms)
2026-11-08T04:05:29Z [load_day 2024-01-10] 2024-01-10 hour 1: 209568 events in 0 files, snapshot none (task 6956 ms: download 4310 ms overlapping gunzip 1008 ms, parse 4464 ms, sort 0 ms, write 0 ms, index 670 ms: 6 candidates, 91998 repo-hours, 847 releases; waiting 510 ms; commit 0 ms)
2026-11-08T04:05:36Z [load_day 2024-01-10] 2024-01-10 hour 2: 191376 events in 0 files, snapshot none (task 6833 ms: download 4387 ms overlapping gunzip 1055 ms, parse 4536 ms, sort 0 ms, write 0 ms, index 467 ms: 5 candidates, 64543 repo-hours, 840 releases; waiting 503 ms; commit 0 ms)
2026-11-08T04:05:36Z [load_day 2024-01-10] 2024-01-10 hour 3: 169584 events in 0 files, snapshot none (task 6417 ms: download 3788 ms overlapping gunzip 973 ms, parse 4191 ms, sort 0 ms, write 0 ms, index 420 ms: 5 candidates, 60239 repo-hours, 786 releases; waiting 474 ms; commit 0 ms)
2026-11-08T04:05:43Z [load_day 2024-01-10] 2024-01-10 hour 4: 139101 events in 0 files, snapshot none (task 5803 ms: download 3035 ms overlapping gunzip 908 ms, parse 3702 ms, sort 0 ms, write 0 ms, index 454 ms: 3 candidates, 58199 repo-hours, 645 releases; waiting 525 ms; commit 0 ms)
2026-11-08T04:05:43Z [load_day 2024-01-10] 2024-01-10 hour 5: 151613 events in 0 files, snapshot none (task 6364 ms: download 4020 ms overlapping gunzip 936 ms, parse 4141 ms, sort 0 ms, write 0 ms, index 460 ms: 6 candidates, 64179 repo-hours, 685 releases; waiting 525 ms; commit 0 ms)
2026-11-08T04:05:49Z [load_day 2024-01-10] 2024-01-10 hour 6: 209671 events in 0 files, snapshot none (task 5552 ms: download 2949 ms overlapping gunzip 775 ms, parse 3305 ms, sort 0 ms, write 0 ms, index 602 ms: 5 candidates, 75832 repo-hours, 805 releases; waiting 489 ms; commit 0 ms)
2026-11-08T04:05:50Z [load_day 2024-01-10] 2024-01-10 hour 7: 213739 events in 0 files, snapshot none (task 5659 ms: download 3291 ms overlapping gunzip 837 ms, parse 3465 ms, sort 0 ms, write 0 ms, index 596 ms: 2 candidates, 74327 repo-hours, 909 releases; waiting 512 ms; commit 0 ms)
2026-11-08T04:05:56Z [load_day 2024-01-10] 2024-01-10 hour 8: 228653 events in 0 files, snapshot none (task 6539 ms: download 4184 ms overlapping gunzip 1012 ms, parse 4093 ms, sort 0 ms, write 0 ms, index 580 ms: 2 candidates, 82420 repo-hours, 1002 releases; waiting 606 ms; commit 0 ms)
2026-11-08T04:05:57Z [load_day 2024-01-10] 2024-01-10 hour 9: 240018 events in 0 files, snapshot none (task 6912 ms: download 4461 ms overlapping gunzip 1066 ms, parse 4451 ms, sort 0 ms, write 0 ms, index 592 ms: 3 candidates, 87154 repo-hours, 1259 releases; waiting 481 ms; commit 0 ms)
2026-11-08T04:06:06Z [load_day 2024-01-10] 2024-01-10 hour 10: 233595 events in 0 files, snapshot none (task 9324 ms: download 6735 ms overlapping gunzip 1409 ms, parse 6522 ms, sort 0 ms, write 0 ms, index 665 ms: 6 candidates, 85378 repo-hours, 1035 releases; waiting 528 ms; commit 0 ms)
2026-11-08T04:06:06Z [load_day 2024-01-10] 2024-01-10 hour 11: 231798 events in 0 files, snapshot none (task 8447 ms: download 5873 ms overlapping gunzip 1276 ms, parse 5735 ms, sort 0 ms, write 0 ms, index 676 ms: 4 candidates, 81550 repo-hours, 985 releases; waiting 499 ms; commit 0 ms)
2026-11-08T04:06:15Z [load_day 2024-01-10] 2024-01-10 hour 12: 275250 events in 0 files, snapshot none (task 8976 ms: download 5245 ms overlapping gunzip 1247 ms, parse 5602 ms, sort 0 ms, write 0 ms, index 1340 ms: 7 candidates, 114129 repo-hours, 1135 releases; waiting 523 ms; commit 0 ms)
2026-11-08T04:06:16Z [load_day 2024-01-10] 2024-01-10 hour 13: 262249 events in 0 files, snapshot none (task 9345 ms: download 6348 ms overlapping gunzip 1478 ms, parse 6200 ms, sort 0 ms, write 0 ms, index 885 ms: 8 candidates, 105815 repo-hours, 1005 releases; waiting 465 ms; commit 0 ms)
2026-11-08T04:06:25Z [load_day 2024-01-10] 2024-01-10 hour 14: 252978 events in 0 files, snapshot none (task 9761 ms: download 6696 ms overlapping gunzip 1476 ms, parse 6848 ms, sort 0 ms, write 0 ms, index 676 ms: 7 candidates, 94729 repo-hours, 1090 releases; waiting 470 ms; commit 0 ms)
2026-11-08T04:06:26Z [load_day 2024-01-10] 2024-01-10 hour 15: 256473 events in 0 files, snapshot none (task 10086 ms: download 7491 ms overlapping gunzip 1545 ms, parse 7067 ms, sort 0 ms, write 0 ms, index 687 ms: 4 candidates, 98166 repo-hours, 1278 releases; waiting 519 ms; commit 0 ms)
2026-11-08T04:06:35Z [load_day 2024-01-10] 2024-01-10 hour 16: 230177 events in 0 files, snapshot none (task 8875 ms: download 6232 ms overlapping gunzip 1449 ms, parse 5999 ms, sort 0 ms, write 0 ms, index 673 ms: 7 candidates, 96255 repo-hours, 1146 releases; waiting 482 ms; commit 0 ms)
2026-11-08T04:06:36Z [load_day 2024-01-10] 2024-01-10 hour 17: 220400 events in 0 files, snapshot none (task 9124 ms: download 6549 ms overlapping gunzip 1450 ms, parse 6042 ms, sort 0 ms, write 0 ms, index 702 ms: 10 candidates, 88614 repo-hours, 1029 releases; waiting 586 ms; commit 0 ms)
2026-11-08T04:06:44Z [load_day 2024-01-10] 2024-01-10 hour 18: 213364 events in 0 files, snapshot none (task 8290 ms: download 5681 ms overlapping gunzip 1268 ms, parse 5536 ms, sort 0 ms, write 0 ms, index 629 ms: 6 candidates, 89797 repo-hours, 1255 releases; waiting 547 ms; commit 0 ms)
2026-11-08T04:06:44Z [load_day 2024-01-10] 2024-01-10 hour 19: 190937 events in 0 files, snapshot none (task 7377 ms: download 4927 ms overlapping gunzip 1107 ms, parse 4840 ms, sort 0 ms, write 0 ms, index 563 ms: 6 candidates, 76863 repo-hours, 985 releases; waiting 528 ms; commit 0 ms)
2026-11-08T04:06:53Z [load_day 2024-01-10] 2024-01-10 hour 20: 184125 events in 0 files, snapshot none (task 8312 ms: download 5543 ms overlapping gunzip 1347 ms, parse 5569 ms, sort 0 ms, write 0 ms, index 576 ms: 12 candidates, 76653 repo-hours, 1039 releases; waiting 525 ms; commit 0 ms)
2026-11-08T04:06:53Z [load_day 2024-01-10] 2024-01-10 hour 21: 182593 events in 0 files, snapshot none (task 8275 ms: download 5812 ms overlapping gunzip 1382 ms, parse 5522 ms, sort 0 ms, write 0 ms, index 537 ms: 6 candidates, 69275 repo-hours, 1079 releases; waiting 530 ms; commit 0 ms)
2026-11-08T04:07:01Z [load_day 2024-01-10] 2024-01-10 hour 22: 184250 events in 0 files, snapshot none (task 8127 ms: download 5518 ms overlapping gunzip 1281 ms, parse 5074 ms, sort 0 ms, write 0 ms, index 449 ms: 5 candidates, 64643 repo-hours, 1110 releases; waiting 905 ms; commit 0 ms)
2026-11-08T04:07:01Z [load_day 2024-01-10] 2024-01-10 hour 23: 170677 events in 0 files, snapshot none (task 6905 ms: download 4373 ms overlapping gunzip 1130 ms, parse 4541 ms, sort 0 ms, write 0 ms, index 469 ms: 7 candidates, 59807 repo-hours, 746 releases; waiting 477 ms; commit 0 ms)
2026-11-08T04:07:10Z [load_day 2024-01-10] 2024-01-10: repo_hour merged from 24 files to 1 (1944140 rows, 9287 ms)
2026-11-08T04:07:13Z [load_day 2024-01-10] 2024-01-10: index committed in 11895 ms (24 + 1 + 24 files)
2026-11-08T04:07:13Z [load_day 2024-01-10] 2024-01-10: 5048176 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:07:13Z [load_day 2024-01-10] attempt 1 succeeded