This course focuses on efficient machine learning and systems. This is a crucial area as deep neural networks demand extraordinary levels of computation, hindering its deployment on everyday devices and burdening the cloud infrastructure. This course introduces efficient AI computing techniques that enable powerful deep learning applications on resource-constrained devices. Topics include model compression, pruning, quantization, neural architecture search, distributed training, data/model parallelism, gradient compression, and on-device fine-tuning. It also introduces application-specific acceleration techniques for large language models and diffusion models. Students will get hands-on experience implementing model compression techniques and deploying large language models (Llama2-7B) on a laptop.

using a thunderbolt 5 bridge and https://github.com/mit-han-lab/TinyChatEngine

Andrej Karpathy - The Tokenizer is a necessary and pervasive component of Large Language Models (LLMs), where it translates between strings and tokens (text chunks).

A complete GPT2 implementation as a single SQL query in PostgreSQL.

Mozilla’s innovation group and Justine Tunney just released llamafile, and I think it’s now the single best way to get started running Large Language Models,

Harvard Business Publishing Education

Digging into the philosophical roots of the battle between Essentialists and Pragmatists

on Github CoPilot

Large language models (LLMs) have only just emerged into mainstream thought, and already they’ve shown themselves to be a powerful tool for interacting with data. While some might classify them as merely a really cool new form of UI, others think that this may be the start of artificial general intelligence.

Much has been written about the Reddit boycott recently, but I have kind of a wild take. what if we thought of Reddit as, functionally, subservient to OpenAI?

What would a copilot for writing and thinking look like? To try answering this question, I built a prototype: Obsidian-Copilot

In a now-taken-down blog post summarizing an event with Sam Altman, Altman revealed that he doesn’t believe that ChatGPT plugins have product-market fit (outside of the browsing plugin) and won’t be coming to the API soon. Why? A few hypotheses (not mutually exclusive). "Chat is not the right UX for plugins. If you know what you want to do, it’s often easier to just do a few clicks on the website. If you don’t, just a chat interface makes it hard to steer the model toward your goal."

Clinical predictive models can help physicians and administrators make decisions by forecasting clinical and operational events.

Amelia Wattenberger makes a convincing argument for why chatbots are a terrible interface for LLMs.

Introducing MPT-7B, the latest entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens of text and code. It is open source, available for commercial use, and matches the quality of LLaMA-7B. MPT-7B was trained on the MosaicML platform in 9.5 days with zero human intervention at a cost of ~$200k. Starting today, you can train, finetune, and deploy your own private MPT models, either starting from one of our checkpoints or training from scratch. For inspiration, we are also releasing three finetuned models in addition to the base MPT-7B: MPT-7B-Instruct, MPT-7B-Chat, and MPT-7B-StoryWriter-65k+, the last of which uses a context length of 65k tokens!

With the MosaicBERT architecture + training recipe, you can now pretrain a competitive BERT-Base model from scratch on the MosaicML platform for $20. We’ve released the pretraining and finetuning code, as well as the pretrained weights.

Good basic explainer on vector databases.

Sam Altman is the CEO of OpenAI, the company behind GPT-4, ChatGPT, DALL-E, Codex, and many other state-of-the-art AI technologies.

Use the new GPT-4 api to build a chatGPT chatbot for multiple Large PDF files.

This time, we spent the whole episode talking about large language models: ChatGPT, GPT-4, Bing, Bard, Claude, LLaMA and more.

Long before the invention of the general-purpose computer, bureaucrats and researchers had begun gathering and cross-tabulating sets of numbers about populations—heights, weights, ages, sexes, races, political parties, incomes—using punch cards and tabulating machines. Sergey Brin said "Auto insurance companies analyse accident data and set insurance rates of individuals according to age, gender, vehicle type,” he pointed out. “If they were allowed to by law, they would also use race, religion, handicap, and any other attributes they find are related to accident rate.”

Talk with an LLaMA AI in your terminal / Port of OpenAI's Whisper model in C/C++. Contribute to ggerganov/whisper.cpp development by creating an account on GitHub.

Twitter thread from Jackson Fall on using gpt to create a website for making money.

We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers.

Dataset Repository of Awesome ChatGPT Prompts from https://github.com/f/awesome-chatgpt-prompts