loader from loading.io

MLG 033 Transformers

Machine Learning Guide

Release Date: 02/09/2025

MLA 022 Code AI Tools show art MLA 022 Code AI Tools

Machine Learning Guide

Try a walking desk while studying ML or working on your projects! Show notes: Tools discussed: Windsurf: Copilot: Cursor: Cline: Roo Code: Aider: Other: Leaderboards: Video of speed-demon: Reddit: Boost programming productivity by acting as a pair programming partner. Groups these tools into three categories: • Hands-Off Tools: These include solutions that work on fixed monthly fees and require minimal user intervention. GitHub Copilot started with simple tab completions and now offers an agent mode similar to Cursor, which stands out for its advanced codebase indexing and...

info_outline
MLG 033 Transformers show art MLG 033 Transformers

Machine Learning Guide

Try a walking desk while studying ML or working on your projects! Show notes: 3Blue1Brown videos: Background & Motivation: RNN Limitations: Sequential processing prevents full parallelization—even with attention tweaks—making them inefficient on modern hardware. Breakthrough: “Attention Is All You Need” replaced recurrence with self-attention, unlocking massive parallelism and scalability. Core Architecture: Layer Stack: Consists of alternating self-attention and feed-forward (MLP) layers, each wrapped in residual connections and layer normalization. Positional...

info_outline
MLA 021 Databricks show art MLA 021 Databricks

Machine Learning Guide

Try a walking desk while studying ML or working on your projects! Discussing Databricks with Ming Chang from  (part of )

info_outline
MLA 020 Kubeflow show art MLA 020 Kubeflow

Machine Learning Guide

Try a walking desk while studying ML or working on your projects! Conversation with Dirk-Jan Kubeflow (vs cloud native solutions like SageMaker)  - Data Scientist at Dept Agency . (From the website:) The Machine Learning Toolkit for Kubernetes. The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable. Our goal is not to recreate other services, but to provide a straightforward way to deploy best-of-breed open-source systems for ML to diverse infrastructures. Anywhere you are running Kubernetes, you should be...

info_outline
MLA 019 DevOps show art MLA 019 DevOps

Machine Learning Guide

Try a walking desk while studying ML or working on your projects! Chatting with co-workers about the role of DevOps in a machine learning engineer's life Expert coworkers at Dept  - Principal Software Developer  - DevOps Lead  (where Matt features often) Devops tools Pictures (funny and serious)

info_outline
MLA 018 Descript show art MLA 018 Descript

Machine Learning Guide

Try a walking desk while studying ML or working on your projects! (Optional episode) just showcasing a cool application using machine learning Dept uses Descript for some of their podcasting. I'm using it like a maniac, I think they're surprised at how into it I am. Check out the transcript & see how it performed.  How to ship software, from the front lines. We talk with software developers about their craft, developer tools, developer productivity and what makes software development awesome. Hosted by your friends at Rocket Insights. AKA shipit.io  An agency podcast with...

info_outline
MLA 017 AWS Local Development show art MLA 017 AWS Local Development

Machine Learning Guide

while studying ML or working on your projects! Show notes:  Developing on AWS first (SageMaker or other) Consider developing against AWS as your local development environment, rather than only your cloud deployment environment. Solutions: Stick to AWS Cloud IDEs (, ,  Connect to deployed infrastructure via  Infrastructure as Code

info_outline
MLA 016 SageMaker 2 show art MLA 016 SageMaker 2

Machine Learning Guide

while studying ML or working on your projects! Part 2 of deploying your ML models to the cloud with SageMaker (MLOps) MLOps is deploying your ML models to the cloud. See  for an overview of tooling (also generally a great ML educational run-down.)

info_outline
MLA 015 SageMaker 1 show art MLA 015 SageMaker 1

Machine Learning Guide

while studying ML or working on your projects! Part 1 of deploying your ML models to the cloud with SageMaker (MLOps) MLOps is deploying your ML models to the cloud. See  for an overview of tooling (also generally a great ML educational run-down.) And I forgot to mention , I'll mention next time.

info_outline
MLA 014 Machine Learning Server show art MLA 014 Machine Learning Server

Machine Learning Guide

while studying ML or working on your projects! Server-side ML. Training & hosting for inference, with a goal towards serverless. AWS SageMaker, Batch, Lambda, EFS, Cortex.dev

info_outline
 
More Episodes

Try a walking desk while studying ML or working on your projects! https://ocdevel.com/walk

Show notes: https://ocdevel.com/mlg/33

3Blue1Brown videos: https://3blue1brown.com/


  • Background & Motivation:

    • RNN Limitations: Sequential processing prevents full parallelization—even with attention tweaks—making them inefficient on modern hardware.
    • Breakthrough: “Attention Is All You Need” replaced recurrence with self-attention, unlocking massive parallelism and scalability.
  • Core Architecture:

    • Layer Stack: Consists of alternating self-attention and feed-forward (MLP) layers, each wrapped in residual connections and layer normalization.
    • Positional Encodings: Since self-attention is permutation invariant, add sinusoidal or learned positional embeddings to inject sequence order.
  • Self-Attention Mechanism:

    • Q, K, V Explained:
      • Query (Q): The representation of the token seeking contextual info.
      • Key (K): The representation of tokens being compared against.
      • Value (V): The information to be aggregated based on the attention scores.
    • Multi-Head Attention: Splits Q, K, V into multiple “heads” to capture diverse relationships and nuances across different subspaces.
    • Dot-Product & Scaling: Computes similarity between Q and K (scaled to avoid large gradients), then applies softmax to weigh V accordingly.
  • Masking:

    • Causal Masking: In autoregressive models, prevents a token from “seeing” future tokens, ensuring proper generation.
    • Padding Masks: Ignore padded (non-informative) parts of sequences to maintain meaningful attention distributions.
  • Feed-Forward Networks (MLPs):

    • Transformation & Storage: Post-attention MLPs apply non-linear transformations; many argue they’re where the “facts” or learned knowledge really get stored.
    • Depth & Expressivity: Their layered nature deepens the model’s capacity to represent complex patterns.
  • Residual Connections & Normalization:

    • Residual Links: Crucial for gradient flow in deep architectures, preventing vanishing/exploding gradients.
    • Layer Normalization: Stabilizes training by normalizing across features, enhancing convergence.
  • Scalability & Efficiency Considerations:

    • Parallelization Advantage: Entire architecture is designed to exploit modern parallel hardware, a huge win over RNNs.
    • Complexity Trade-offs: Self-attention’s quadratic complexity with sequence length remains a challenge; spurred innovations like sparse or linearized attention.
  • Training Paradigms & Emergent Properties:

    • Pretraining & Fine-Tuning: Massive self-supervised pretraining on diverse data, followed by task-specific fine-tuning, is the norm.
    • Emergent Behavior: With scale comes abilities like in-context learning and few-shot adaptation, aspects that are still being unpacked.
  • Interpretability & Knowledge Distribution:

    • Distributed Representation: “Facts” aren’t stored in a single layer but are embedded throughout both attention heads and MLP layers.
    • Debate on Attention: While some see attention weights as interpretable, a growing view is that real “knowledge” is diffused across the network’s parameters.