What is Generative AI (”GenAI”)?

• GenAI Model: The latest generation of machine-learning models that can understand human instructions and generate new content. They can be multimodal (input and/or output can be text, code, image, audio, video).
• Video: Generative AI in a Nutshell
• Overview
  • https://www.deeplearning.ai/courses/generative-ai-for-everyone/
  • https://github.com/microsoft/generative-ai-for-beginners
  • Where AI Fits in Engineering Organizations | Matt Rickard
• Advanced
  • Takeaways from Mastering LLMs Course
  • https://a16z.com/ai-canon/

Courses

• https://www.deeplearning.ai/courses/
• https://maven.com/courses/ai
• https://www.youtube.com/@LangChain
• https://github.com/mlabonne/llm-course

What is LLMOps / GenAIOps

• LLMOps could mean either the (1) training & serving of GenAI models/LLMs, or (2) lifecycle of building a GenAI app. My notes here cover mostly the latter in the context of a product company adopting GenAI (not building a GenAI infrastructure company).

https://github.com/a16z-infra/llm-app-stack

Mindset Change

For a software engineer to become an AI engineer, mental approach has to be different:

1. Data: Shift from code mindset to data mindset.
2. Evals: Just like TDD, do eval-driven development. (”offline evaluations”)
3. Data Science: Look at logs daily (”online evaluations”) and refine the system to improve coverage (success % of responses) gradually.
   1. Daily: 20 annotations a day, keeps low-quality data away
   2. Weekly: Do a team data bash, where team looks at defects ~1hr weekly (via)
4. Iterate: It’s a game of experimentation, iterations and growing a system. It's not a one time build (unlike traditional software engineering).

https://docs.aws.amazon.com/wellarchitected/latest/machine-learning-lens/well-architected-machine-learning-lifecycle.html

Caution

• Still early technology
  • "While GenAI is fun, I think its economic value is grossly over estimated, because it’s unreliable, risky and expensive to make and serve. It’s fine for creative tasks, but not (yet) autonomous agents" — via
• Autonomous LLM Agents Are At Least 10 Years Out | Matt Rickard

https://marketoonist.com/2023/08/generative-ai-adoption.html

Glossary

• https://a16z.com/ai-glossary/