Skip to content
English

Switch theme

Discover▾

Not sure where to start? This week’s hottest software, models and stacks in three minutes.

AI Apps▾

AI software people use: from assistants like ChatGPT to tools for coding, video and workflows, picked by purpose.

For Agents▾

Software for AI: tools that let your agent (Claude Code, Codex, Cursor…) browse, read docs, run code and remember.

Models▾

Choose models, estimate cost, self-host: prices, rankings, benchmarks, free tiers and compute.

Solutions▾

A goal or an error message? Ready-made stacks, steps and fixes by scenario.

Learn▾

A systematic course from zero to the frontier, with an algorithm library, must-read papers and tutorials.

Frontier▾

Daily AI news, weekly highlights, the latest papers and how AI is changing every discipline.

// Stage 7 · Multimodal & Generative AI

Vision Transformers & Self-Supervised Vision

ViT, SimCLR, MAE, JEPA and SAM.

What you will learn ↓ Code & exercises ↓

about 8 hours12 algorithms4 paperslesson text in progress

01Goals

Do these first: Transformers & Attention · CNNs, RNNs and the Birth of Attention

02Algorithms to master

Each card has the original paper, prerequisites and resources.

Vision Transformer (ViT) Treats 16×16 image patches as words for a standard Transformer. 2020core SimCLR Learns visual features without labels by matching two augmentations. 2020ext Masked Autoencoders (MAE) Masks 75% of image patches and reconstructs them to learn representations. 2021core Joint-Embedding Predictive Architecture (JEPA) Predicts masked content in representation space rather than pixels. 2023core Segment Anything (SAM) A promptable foundation model that segments any object from a click or box. 2023core Contrastive Learning (InfoNCE) Learns embeddings by pulling positives together and pushing negatives apart. 2018core U-Net A U-shaped conv net: downsample, upsample, and skip-connect matching levels. 2015core Faster R-CNN (Object Detection) Two-stage detector: a network proposes boxes, then classifies and refines them. 2015core YOLO (Real-Time Object Detection) Predicts all boxes and classes in a single forward pass – very fast. 2015core DETR (Detection Transformer) Casts detection as set prediction with a Transformer – no anchors, no NMS. 2020core Swin Transformer Hierarchical vision Transformer with shifted local-window attention. 2021ext DINOv2 (Self-Supervised Visual Features) Label-free self-distillation on curated images yields general-purpose visual features. 2023core

03Must-read papers

04Courses & videos

05Hands-on: project, code & exercises

Free, official or author-published pages. Most run in the browser or on a free Colab GPU.

PROJECT

Do few-shot image classification with a pre-trained ViT (linear probe vs fine-tuning) and auto-label a batch of images with SAM.

  1. 1
    huggingface.co ↗Courses

    Do the hands-on units of the Hugging Face computer-vision course.

  2. 2
    huggingface.co ↗Official docs

    Load ViT, Swin, DINOv2 backbones with timm and fine-tune on a small dataset.

  3. 3
    docs.ultralytics.com ↗Official docs

    Train a YOLO detector on your own photos.

06Self-test

Answer in your head, then open to check.

How does ViT turn an image into a sequence?

It cuts it into fixed-size patches and projects each linearly.

Why does MAE mask as much as 75%?

Images are highly redundant; low masking is too easy.

What can SAM’s prompts be?

Points, boxes, masks and so on.

On these routes: Domain Scientists

Data updated: AI algorithms & curriculum(UTC+8)