OSS catalog — 47 projects from one source

Situation

An open-source project catalog assembled by hand ages faster than it can be updated. Stars, releases and languages change on GitHub while the site keeps numbers from a month ago. Worse, the same card exists in two places — a data table and the markup — and disagrees with itself.

Task

I needed a catalog where the data comes from one machine-owned source, pages are built from it, and a repeated sync does not overwrite authored edits.

Actions

Data source. A Python script calls the GitHub API and writes one TOML file: 47 records with 15 fields each — name, description, stars, forks, releases, licence, language, contributors, last update and others. The API response cache lives outside the repository.

Ownership split. The generator rewrites only the frontmatter block under an «updated automatically» marker; the rest of the page stays authored. An exclude: field prunes noisy machine values, and explicitly set languages survive a repeated sync.

Build. Astro 7 with content collections: a glob loader reads project and note pages, a Zod schema checks types, and the project page renders metadata and links. The site is static with no client framework.

Validation. A separate validator checks notes and project pages: required fields, key parity with the source, no empty blocks. In strict mode a warning about a record without metadata becomes an error.

Tests. 26 tests cover the three record-matching branches — owner/name, short name, directory name — render idempotency, preservation of editorial fields, and the real page tree.

CI. Steps are ordered by cost: the validator and Python tests run before Astro dependencies are installed, then astro check, build, Pagefind and deploy. The cheap gate fails before the expensive one runs.

Result

• 47 project pages and 47 repository records from one TOML file.
• 15 metadata fields per record, including stars, forks, releases, licence and last update.
• 26 tests, including byte-level idempotency of a repeated render.
• 8 topic pages, 76 tag pages and 5 language pages generated from the same source.
• The build never goes to the network: the site reads only committed frontmatter, so the deploy is reproducible.
• The validator passes on every page, and astro check catches schema drift before the build.

Limitations

Validation is mechanical: description drift, placeholders and alt text remain a human or separate editorial pass.

The page-count test asserts a floor, not the exact number, so drift away from 47 stays green.

website, readme_image, latest_release and first_commit are fetched or declared in the schema but never rendered: the field exists, the consumer does not.

Pagefind is installed and runs in the build, but there is no search UI — search remains an unused dependency.

The catalog is a showcase of an engineering pipeline, not a product with metrics: there is no user analytics and no A/B testing here.

Documentation

• README.md — the data source, quick start and the boundaries of the claim.
• scripts/update-github-repos.py — the GitHub API sync into TOML.
• scripts/update-project-frontmatter.py — the metadata block generator with the ownership marker.
• Checks: bun run validate → uv run tests/test_scripts.py → bun run check → bun run build.