Skip to main content

AI

About​

👉🏼 I've added a demo @ /ai

Semantic search is powerful and it's part of the reason I build and publish this Today I Learned section on my website. Semantic search is even more powerful when you add natural language processing trained on a large language model. I'm making this page for notes on building a personal AI Assistant.

I written several essays in the last couple years on using tools to organize my workflow and thinking 1,2. In the last six months I've written two on Artificial Intelligence 3,4. I now use several different AI assistants within my text editor for coding. The same way I've applied my code documentation to the rest of my life here, I'm starting to realize how helpful AI assistants can be. I think I'll need one to interact with my public documentation and a second for me to interact with my private information like my calendar, chats, and emails.

I've recently started learning how to create embeddings using vector databases which can be chained to existing LLMs. I've started some documentation using LangChain and OpenAi 5. Because the field is rapidly developing, I need a place to keep notes on it, so here we are.

Log​

  • 23-07-14 - finally closed a set of tabs this morning I noted here. I'd been stuck working on for a while because I've had to focus on other work. Will come back to this in the coming months.

ToDo​

  • fix all relative links so that result links work
  • try alternative LLMs like Llama
  • control the data training
  • self host supabase or postgres for data
  • sync the data with Github actions.
  • combine my posts into a monorepo using markdown
    • return to exporting my post into markdown or migrate
  • iterative testing of various prompts

Docs​

Stack​

  • React
  • Next.js
  • Supabase / PostGres PgVector

Context Injection​

  • Tokenization
  • Stop-word removal
  • Stemming / Lemmatization
  • Vectorization

Preprocessing​

Embeddings​

Data Storage​

Prompt Engineering​

Repos​

Resources​

Commercial​

References​

  1. Slow Thinking - https://davidawindham.com/slow-thinking/
  2. A Second Brain - https://davidawindham.com/a-second-brain/
  3. Artificial Intelligence - https://davidawindham.com/artificial-intelligence/
  4. Artificial Intelligence (Part 2) - https://davidawindham.com/artificial-intelligence-2/
  5. TIL SasS OpenAI - /docs/saas/OpenAI
  6. MarkPrompt - https://markprompt.com/docs#react