load_day 2024-01-29, attempt 1 succeeded

Rerun from here

Log in to rerun from here

Host

73ccef5e07c2:75116

Started

2026-11-08T04:22:35Z

Ended

2026-11-08T04:24:53Z

Peak memory?The most memory its processes held at once: the run's process and what it started (dista's local workers too), sampled every second by the scheduler or worker that started it, and at the end the largest single process. Not measured on EC2 or Kubernetes. Docs ↗

688 MB

CPU used?The CPU time its processes used, user and system, and the most cores they kept busy over one sample's second. macOS gives only the time, and only for a process that made one attempt. Docs ↗

199.8 s, up to 2.6 cores at once

Log file

s3://census-mlake-dev/logs/gharchive/2019/load_day/2024-01-29/1.log

Code

pipelines/gharchive.mojo:135↗

2026-11-08T04:22:35Z [load_day 2024-01-29] attempt 1 started
2026-11-08T04:22:36Z [load_day 2024-01-29] 2024-01-29: 2 hours in flight, about 700 MB (GHARCHIVE_EVENTS=off: the index only)
2026-11-08T04:22:42Z [load_day 2024-01-29] 2024-01-29 hour 0: 188931 events in 0 files, snapshot none (task 6199 ms: download 3128 ms overlapping gunzip 927 ms, parse 3840 ms, sort 0 ms, write 0 ms, index 800 ms: 5 candidates, 83575 repo-hours, 1195 releases; waiting 335 ms; commit 0 ms)
2026-11-08T04:22:43Z [load_day 2024-01-29] 2024-01-29 hour 1: 200642 events in 0 files, snapshot none (task 7061 ms: download 3925 ms overlapping gunzip 1151 ms, parse 4440 ms, sort 0 ms, write 0 ms, index 705 ms: 8 candidates, 90879 repo-hours, 797 releases; waiting 499 ms; commit 0 ms)
2026-11-08T04:22:49Z [load_day 2024-01-29] 2024-01-29 hour 2: 174350 events in 0 files, snapshot none (task 6361 ms: download 3941 ms overlapping gunzip 1057 ms, parse 4091 ms, sort 0 ms, write 0 ms, index 489 ms: 4 candidates, 60450 repo-hours, 725 releases; waiting 506 ms; commit 0 ms)
2026-11-08T04:22:50Z [load_day 2024-01-29] 2024-01-29 hour 3: 175406 events in 0 files, snapshot none (task 6627 ms: download 4248 ms overlapping gunzip 1085 ms, parse 4200 ms, sort 0 ms, write 0 ms, index 523 ms: 3 candidates, 59596 repo-hours, 900 releases; waiting 513 ms; commit 0 ms)
2026-11-08T04:22:56Z [load_day 2024-01-29] 2024-01-29 hour 4: 176744 events in 0 files, snapshot none (task 6011 ms: download 3629 ms overlapping gunzip 898 ms, parse 3854 ms, sort 0 ms, write 0 ms, index 471 ms: 2 candidates, 57782 repo-hours, 740 releases; waiting 513 ms; commit 0 ms)
2026-11-08T04:22:57Z [load_day 2024-01-29] 2024-01-29 hour 5: 184696 events in 0 files, snapshot none (task 6503 ms: download 3792 ms overlapping gunzip 1057 ms, parse 4107 ms, sort 0 ms, write 0 ms, index 448 ms: 4 candidates, 61831 repo-hours, 741 releases; waiting 460 ms; commit 0 ms)
2026-11-08T04:23:04Z [load_day 2024-01-29] 2024-01-29 hour 6: 217249 events in 0 files, snapshot none (task 7699 ms: download 4449 ms overlapping gunzip 1269 ms, parse 5026 ms, sort 0 ms, write 0 ms, index 596 ms: 3 candidates, 75244 repo-hours, 927 releases; waiting 495 ms; commit 0 ms)
2026-11-08T04:23:06Z [load_day 2024-01-29] 2024-01-29 hour 7: 214611 events in 0 files, snapshot none (task 8111 ms: download 5512 ms overlapping gunzip 1321 ms, parse 5139 ms, sort 0 ms, write 0 ms, index 650 ms: 7 candidates, 74454 repo-hours, 843 releases; waiting 559 ms; commit 0 ms)
2026-11-08T04:23:15Z [load_day 2024-01-29] 2024-01-29 hour 8: 226008 events in 0 files, snapshot none (task 10254 ms: download 7003 ms overlapping gunzip 1663 ms, parse 6770 ms, sort 0 ms, write 0 ms, index 851 ms: 7 candidates, 82016 repo-hours, 957 releases; waiting 599 ms; commit 0 ms)
2026-11-08T04:23:18Z [load_day 2024-01-29] 2024-01-29 hour 9: 247825 events in 0 files, snapshot none (task 10901 ms: download 7288 ms overlapping gunzip 1745 ms, parse 7073 ms, sort 0 ms, write 0 ms, index 1301 ms: 5 candidates, 89483 repo-hours, 1283 releases; waiting 447 ms; commit 0 ms)
2026-11-08T04:23:27Z [load_day 2024-01-29] 2024-01-29 hour 10: 246452 events in 0 files, snapshot none (task 11156 ms: download 7185 ms overlapping gunzip 1895 ms, parse 7635 ms, sort 0 ms, write 0 ms, index 815 ms: 5 candidates, 88879 repo-hours, 1275 releases; waiting 502 ms; commit 0 ms)
2026-11-08T04:23:28Z [load_day 2024-01-29] 2024-01-29 hour 11: 241764 events in 0 files, snapshot none (task 10015 ms: download 7368 ms overlapping gunzip 1648 ms, parse 6595 ms, sort 0 ms, write 0 ms, index 656 ms: 7 candidates, 83096 repo-hours, 1152 releases; waiting 780 ms; commit 0 ms)
2026-11-08T04:23:39Z [load_day 2024-01-29] 2024-01-29 hour 12: 295646 events in 0 files, snapshot none (task 11639 ms: download 7934 ms overlapping gunzip 1682 ms, parse 7200 ms, sort 0 ms, write 0 ms, index 1598 ms: 7 candidates, 115217 repo-hours, 1233 releases; waiting 773 ms; commit 0 ms)
2026-11-08T04:23:40Z [load_day 2024-01-29] 2024-01-29 hour 13: 269041 events in 0 files, snapshot none (task 11116 ms: download 7932 ms overlapping gunzip 1757 ms, parse 7344 ms, sort 0 ms, write 0 ms, index 1222 ms: 9 candidates, 108763 repo-hours, 1143 releases; waiting 540 ms; commit 0 ms)
2026-11-08T04:23:51Z [load_day 2024-01-29] 2024-01-29 hour 14: 271229 events in 0 files, snapshot none (task 11636 ms: download 8730 ms overlapping gunzip 2008 ms, parse 7946 ms, sort 0 ms, write 0 ms, index 892 ms: 8 candidates, 101793 repo-hours, 1210 releases; waiting 483 ms; commit 0 ms)
2026-11-08T04:23:51Z [load_day 2024-01-29] 2024-01-29 hour 15: 277102 events in 0 files, snapshot none (task 11383 ms: download 8607 ms overlapping gunzip 1920 ms, parse 7889 ms, sort 0 ms, write 0 ms, index 861 ms: 6 candidates, 102423 repo-hours, 1470 releases; waiting 494 ms; commit 0 ms)
2026-11-08T04:24:02Z [load_day 2024-01-29] 2024-01-29 hour 16: 270679 events in 0 files, snapshot none (task 9962 ms: download 7345 ms overlapping gunzip 1693 ms, parse 6662 ms, sort 0 ms, write 0 ms, index 773 ms: 6 candidates, 98787 repo-hours, 1162 releases; waiting 530 ms; commit 0 ms)
2026-11-08T04:24:02Z [load_day 2024-01-29] 2024-01-29 hour 17: 270690 events in 0 files, snapshot none (task 10103 ms: download 7447 ms overlapping gunzip 1713 ms, parse 6837 ms, sort 0 ms, write 0 ms, index 714 ms: 7 candidates, 93847 repo-hours, 1066 releases; waiting 482 ms; commit 0 ms)
2026-11-08T04:24:15Z [load_day 2024-01-29] 2024-01-29 hour 18: 276920 events in 0 files, snapshot none (task 12992 ms: download 8498 ms overlapping gunzip 1963 ms, parse 7996 ms, sort 0 ms, write 0 ms, index 2119 ms: 9 candidates, 98410 repo-hours, 1188 releases; waiting 536 ms; commit 0 ms)
2026-11-08T04:24:16Z [load_day 2024-01-29] 2024-01-29 hour 19: 257197 events in 0 files, snapshot none (task 13401 ms: download 9172 ms overlapping gunzip 2044 ms, parse 8245 ms, sort 0 ms, write 0 ms, index 2291 ms: 11 candidates, 83843 repo-hours, 1107 releases; waiting 504 ms; commit 0 ms)
2026-11-08T04:24:26Z [load_day 2024-01-29] 2024-01-29 hour 20: 252252 events in 0 files, snapshot none (task 10352 ms: download 6730 ms overlapping gunzip 1723 ms, parse 6962 ms, sort 0 ms, write 0 ms, index 773 ms: 7 candidates, 82229 repo-hours, 966 releases; waiting 552 ms; commit 0 ms)
2026-11-08T04:24:28Z [load_day 2024-01-29] 2024-01-29 hour 21: 238575 events in 0 files, snapshot none (task 11095 ms: download 8215 ms overlapping gunzip 1827 ms, parse 7591 ms, sort 0 ms, write 0 ms, index 760 ms: 12 candidates, 75742 repo-hours, 1147 releases; waiting 521 ms; commit 0 ms)
2026-11-08T04:24:36Z [load_day 2024-01-29] 2024-01-29 hour 22: 224360 events in 0 files, snapshot none (task 9452 ms: download 5975 ms overlapping gunzip 1579 ms, parse 6154 ms, sort 0 ms, write 0 ms, index 814 ms: 11 candidates, 69147 repo-hours, 982 releases; waiting 606 ms; commit 0 ms)
2026-11-08T04:24:38Z [load_day 2024-01-29] 2024-01-29 hour 23: 216441 events in 0 files, snapshot none (task 9999 ms: download 7318 ms overlapping gunzip 1738 ms, parse 6903 ms, sort 0 ms, write 0 ms, index 511 ms: 6 candidates, 63937 repo-hours, 719 releases; waiting 493 ms; commit 0 ms)
2026-11-08T04:24:50Z [load_day 2024-01-29] 2024-01-29: repo_hour merged from 24 files to 1 (2001423 rows, 11611 ms)
2026-11-08T04:24:52Z [load_day 2024-01-29] 2024-01-29: index committed in 14245 ms (24 + 1 + 24 files)
2026-11-08T04:24:52Z [load_day 2024-01-29] 2024-01-29: 5614810 events in 24 hours (not kept: GHARCHIVE_EVENTS=off)
2026-11-08T04:24:52Z [load_day 2024-01-29] attempt 1 succeeded