Show HN: Offline Tiny-LMs 25MB for personal goals, career and pitch deck advice (github.com)

2 points by andyvane •

I wanted tiny models that could quickly match what I was writing with advice from books, so while I typed I could see ideas and concepts related to what I was doing, 28,000+ advice rules.

WEB DEMO: the web app and models run in-browser, nothing is uploaded (demo link in github)

CLI: a package manager for models and source data (pipx etiny)

TECH: 3 tiny fine-tuned embedding models, trained on 5GB of synth data using 4x NVIDIA H100s (EuroHPC credit)

Ideas for other models would be much appreciated! a friend suggested a 'preppers model'? not sure... tks, Shane

I have been toying with combining offline tools, tiny models and anti-hallucination concepts (RAG, lookup, etc.) I wanted something that would loosely keyword match my documents and help me think through things, whether it was for my next year goals or business ideas, like an autocomplete but matching sources of information and ideas quickly, and allowing me to click for details and quickly go back to my source to update etc. etc. (no doubt there are others tools like this). It helps you explore different advice from different authors.

Model/corpus (web app + downloadable model packages) - Personal Goals: 23k advice rules / 29 books / 25MB bundle / fine-tune corpus of 1.2M documents (2.3GB, 320 seed documents) - Startup Pitch Deck: 75 advice rules / 5 guides / fine-tune corpus of 2K documents (2.2GB, 349 seed documents) - Career Advice: 5K advice rules / 5 books / fine-tune corpus of 17K documents (91MB, 278 seed documents)

TECHNOLOGY: Each of the 3 models is its own nano embedding model, fine-tuned on NVIDIA 4x H100s (EuroHPC credit). Nothing is uploaded in the web app, everything runs locally fronteend in-browser via the nano embedding models and WASM/WEBGPU. At scan time the app matches your text against the rule set in two ways: exact pattern matches and meaning matches. The advice rules themselves are machine-derived from the books, 28K across the three verticals, and the regex + embed model helps map to user text. Nothing is AI generated at scan time, pure matching, so the same text gives the same re-ranked matchings (I can explain this better if needed).

Source material - I have used public domain material for 98% - I have used table of contents from copyrighted books (only accessible in web app) (fyi: I have emailed penguin publishing re: 'Atomic Habits'... doubt I will get a response) - full texts are NOT included in downloads (source excerpts for public docs are)

Roadmap/wish-list - get feedback on concept - fine-tune more nano-models or improve existing - new model x10: prepping guides, outdoor hobbies, musicical genealogy trees - npm package and cli tools (py scripts) + feature parity with web app - build a true generative Nano-LLMs (ie. not embedding models) - feat: state management (allow for dismissing/removing sources) - feat: allow for indexing your own personal PDF/books - ...Obsidian plugin? proper domain? Claude Skill?

Thanks for reading, would love to know what you think, and I hope you enjoy ETINY Shane

Hacker News