--- title: "Computer Vision" aliases: ["CV", "Image Recognition", "Visual Computing"] tags: - computer-vision - deep-learning - image-processing --- # Computer Vision ## Descriptors and correspondence - **SIFT** - Extracts local descriptors invariant to scale and rotation for matching objects or scenes across images; robustness to viewpoint and illumination changes is empirical, not unlimited invariance. [Lowe 2004]( "Distinctive Image Features from Scale-Invariant Keypoints; IJCV"). - **Histograms of oriented gradients** - Pools local gradient orientations with overlapping block normalization. The human-detection study isolates how spatial binning, orientation resolution, and contrast normalization affect recognition. [Dalal and Triggs 2005]( "Histograms of Oriented Gradients for Human Detection; CVPR"). - **SuperPoint** - Jointly predicts interest points and descriptors from full images; Homographic Adaptation supplies self-supervision for repeatable matching across transformed views. [DeTone et al. 2018]( "SuperPoint: Self-Supervised Interest Point Detection and Description; CVPR DL4VSLAM Workshop"). - **SuperGlue** - Uses attention and a graph neural network to predict matching costs, then differentiable optimal transport to assign local features and reject unmatched points. [Sarlin et al. 2020]( "SuperGlue: Learning Feature Matching with Graph Neural Networks; CVPR"). - **LoFTR** - Matches images without a separate keypoint detector: cross-image attention establishes coarse correspondences, then refines selected matches at higher resolution. [Sun et al. 2021]( "LoFTR: Detector-Free Local Feature Matching with Transformers; CVPR"). ## Classification backbones - **Cresceptron** - Grows a hierarchy of neurons and connections as new concepts appear. Uses hierarchical tolerance for recognition and traces response paths for segmentation. [Weng et al. 1992]( "Cresceptron: A Self-Organizing Neural Network Which Grows Adaptively; IJCNN"). - **AlexNet** - Trains a deep convolutional classifier on ImageNet using GPU computation, non-saturating neurons, and regularization; a reference for large-scale supervised visual learning. [Krizhevsky et al. 2012]( "ImageNet Classification with Deep Convolutional Neural Networks; NeurIPS"). - **VGG** - Studies depth using stacks of small 3×3 convolution filters; provides classification networks whose learned representations transfer to other image datasets. [Simonyan and Zisserman 2015]( "Very Deep Convolutional Networks for Large-Scale Image Recognition; ICLR"). - **ResNet** - Learns residual functions relative to layer inputs, easing optimization of deep networks. The paper evaluates recognition and transfers the backbone to detection. [He et al. 2016]( "Deep Residual Learning for Image Recognition; CVPR"). - **DenseNet** - Feeds each layer the feature maps of preceding layers within densely connected blocks, promoting feature reuse and short paths for information and gradients. [Huang et al. 2017]( "Densely Connected Convolutional Networks; CVPR"). - **EfficientNet** - Scales network depth, width, and input resolution together using a compound coefficient; combines this scaling rule with a searched convolutional baseline. [Tan and Le 2019]( "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks; ICML"). - **Vision Transformer** - Applies a Transformer to image-patch sequences. The study emphasizes large-scale pretraining and transfer rather than assuming attention needs less data than convolutions. [Dosovitskiy et al. 2021]( "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale; ICLR"). - **DeiT** - Trains image Transformers on ImageNet without external training data; an attention-based distillation token transfers knowledge from a teacher, including a convolutional teacher. [Touvron et al. 2021]( "Training data-efficient image transformers & distillation through attention; ICML"). - **Swin Transformer** - Builds hierarchical image features using local attention windows shifted between layers. Cross-window connections retain spatial interaction while limiting attention cost. [Liu et al. 2021]( "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows; ICCV"). - **ConvNeXt** - Revisits convolutional design by progressively modifying a ResNet toward choices used in visual Transformers; supplies convolutional backbones for classification and dense prediction. [Liu et al. 2022]( "A ConvNet for the 2020s; CVPR"). ## Learning reusable visual features - **Self-supervised representations** - See [Feature Extraction](https://mlpapers.org/feature-extraction/#learned-representations) for contrastive learning, BYOL, DINO and masked autoencoders, with their direct analyses and follow-ups. - **Vision-language models** - CLIP supports language-defined recognition; Flamingo and LLaVA generate text from visual context. See [Multi-View Learning](https://mlpapers.org/multiview-learning/) for alignment and generation methods. ## Object detection - **R-CNN** - Applies convolutional features to bottom-up region proposals; transfers supervised pretraining to detection through domain-specific fine-tuning. [Girshick et al. 2014]( "Rich feature hierarchies for accurate object detection and semantic segmentation; CVPR"). - **Fast R-CNN** - Builds on region-based convolutional detection to classify object proposals more efficiently; studies training and inference improvements over R-CNN and SPPnet. [Girshick 2015]( "Fast R-CNN; ICCV"). - **Faster R-CNN** - Learns region proposals with a network that shares image features with the detector, replacing a separate proposal-generation algorithm. [Ren et al. 2015]( "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks; NeurIPS"). - **YOLO** - Predicts bounding boxes and class probabilities directly from a full image in one network evaluation, framing detection as joint regression rather than proposal classification. [Redmon et al. 2016]( "You Only Look Once: Unified, Real-Time Object Detection; CVPR"). - **SSD** - Predicts class scores and offsets for default boxes at multiple feature-map resolutions, handling different object scales without a separate proposal stage. [Liu et al. 2016]( "SSD: Single Shot MultiBox Detector; LNCS"). - **Feature pyramid networks** - Combines a top-down pathway with lateral connections to construct semantically rich features at multiple resolutions for detecting objects at different scales. [Lin et al. 2017]( "Feature Pyramid Networks for Object Detection; CVPR"). - **RetinaNet and focal loss** - Downweights well-classified examples in cross-entropy training so abundant easy background examples do not dominate a dense detector's loss. [Lin et al. 2017]( "Focal Loss for Dense Object Detection; ICCV"). - **DETR** - Treats detection as set prediction: learned object queries and bipartite matching produce unique predictions without anchor generation or non-maximum suppression. [Carion et al. 2020]( "End-to-End Object Detection with Transformers; ECCV"). - **Deformable DETR** - Restricts attention to a small set of sampling points around each reference point, addressing DETR's training convergence and high-resolution feature limitations. [Zhu et al. 2021]( "Deformable DETR: Deformable Transformers for End-to-End Object Detection; ICLR"). - **Grounding DINO** - Combines a DINO object detector with language-guided queries and cross-modal fusion to detect objects specified by category names or referring expressions. [Liu et al. 2024]( "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection; ECCV"). ## Semantic, instance, and prompted segmentation - **Fully convolutional networks** - Adapts classification networks to pixelwise prediction; combines deep coarse semantics with shallower spatial detail to produce segmentation maps for variable-sized images. [Long et al. 2015]( "Fully Convolutional Networks for Semantic Segmentation; CVPR"). - **U-Net** - Combines a contracting context path with a symmetric expanding localization path. The biomedical study uses strong augmentation to train from limited annotated images. [Ronneberger et al. 2015]( "U-Net: Convolutional Networks for Biomedical Image Segmentation; MICCAI"). - **Mask R-CNN** - Adds a parallel mask-prediction branch to Faster R-CNN, producing a separate segmentation mask for each detected object instance. [He et al. 2017]( "Mask R-CNN; ICCV"). - **DeepLab** - Uses atrous convolution and spatial pyramid pooling for multiscale semantic segmentation; a fully connected conditional random field refines object boundaries. [Chen et al. 2018]( "DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs; IEEE Transactions on Pattern Analysis and Machine Intelligence"). - **Mask2Former** - Constrains cross-attention to predicted mask regions. A shared architecture supports semantic, instance, and panoptic segmentation through their different grouping targets. [Cheng et al. 2022]( "Masked-attention Mask Transformer for Universal Image Segmentation; CVPR"). - **Segment Anything** - Trains a promptable segmentation model alongside a model-assisted mask collection process; evaluates transfer to new distributions and tasks without task-specific training. [Kirillov et al. 2023]( "Segment Anything; ICCV"). - **SAM 2** - Extends promptable segmentation to images and videos with a Transformer using streaming memory; couples model development with interactive video-mask collection. [Ravi et al. 2024]( "SAM 2: Segment Anything in Images and Videos; arXiv"). ## Video and motion - **Two-stream networks** - Combines spatial appearance and temporal motion networks for action recognition; the temporal stream learns from stacked dense optical-flow fields. [Simonyan and Zisserman 2014]( "Two-Stream Convolutional Networks for Action Recognition in Videos; NeurIPS"). - **I3D** - Inflates 2D convolution and pooling kernels into 3D to learn video features while reusing image-model designs and parameters; introduces the Kinetics action dataset. [Carreira and Zisserman 2017]( "Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset; CVPR"). - **SlowFast** - Pairs a low-frame-rate pathway for spatial semantics with a lightweight high-frame-rate pathway for motion, separating temporal resolution from channel capacity. [Feichtenhofer et al. 2019]( "SlowFast Networks for Video Recognition; ICCV"). - **RAFT** - Builds multiscale all-pairs pixel correlation volumes and recurrently updates an optical-flow field by querying those correlations. [Teed and Deng 2020]( "RAFT: Recurrent All-Pairs Field Transforms for Optical Flow; ECCV"). - **CoTracker** - Jointly tracks multiple 2D points with a transformer that models dependencies between tracks, using windowed inference to extend trajectories through video and occlusions. [Karaev et al. 2024]( "CoTracker: It is Better to Track Together; ECCV"). - **AdaCodec** - Encodes video as keyframes plus motion/residual tokens. At the same 2 FPS, retains benchmark accuracy with about one-seventh of RGB visual tokens on Qwen3-VL-8B. Equal-token comparisons instead allocate the savings to denser video coverage. [Hou et al. 2026]( "AdaCodec: A Predictive Visual Code for Video MLLMs; NeurIPS"). ## Geometry and scene representations - **Structure-from-Motion revisited** - Develops an incremental reconstruction pipeline for unordered image collections, addressing robustness, accuracy, completeness, and scalability; releases the pipeline as open-source software. [Schonberger and Frahm 2016]( "Structure-From-Motion Revisited; CVPR"). - **PointNet** - Processes point clouds directly while respecting invariance to input-point ordering; supports object classification, part segmentation, and scene labeling without voxelizing the input. [Qi et al. 2017]( "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation; CVPR"). - **NeRF** - Fits a continuous field of volume density and view-dependent radiance to posed images; differentiable volume rendering synthesizes new views of the fitted scene. [Mildenhall et al. 2020]( "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis; ECCV"). - **Instant neural graphics primitives** - Augments a small neural network with trainable multiresolution hash-table features; optimized GPU kernels support efficient fitting and evaluation of neural graphics representations. [Müller et al. 2022]( "Instant Neural Graphics Primitives with a Multiresolution Hash Encoding; ACM Trans. Graph."). - **3D Gaussian splatting** - Represents a scene with optimized anisotropic Gaussians, interleaves parameter fitting with density control, and renders through visibility-aware splatting. [Kerbl et al. 2023]( "3D Gaussian Splatting for Real-Time Radiance Field Rendering; ACM TOG"). - **2D Gaussian splatting** - Represents surfaces with oriented planar Gaussian disks; ray-splat intersections and depth/normal regularization address geometry inconsistency in volumetric Gaussian representations. [Huang et al. 2024]( "2D Gaussian Splatting for Geometrically Accurate Radiance Fields; SIGGRAPH"). - **DUSt3R** - Predicts paired pointmaps without supplied camera calibration or poses; aligns these maps into a common frame for reconstruction from multiple images. [Wang et al. 2024]( "DUSt3R: Geometric 3D Vision Made Easy; CVPR"). - **Depth Anything V2** - Trains a depth teacher on synthetic labels, then students on pseudo-labeled real images. Produces relative monocular depth; metric-depth models require separate fine-tuning. [Yang et al. 2024]( "Depth Anything V2; NeurIPS"). - **VGGT** - Uses a feed-forward network to infer camera parameters, depth, pointmaps, and point tracks from scene views, combining several geometric prediction tasks in one model. [Wang et al. 2025]( "VGGT: Visual Geometry Grounded Transformer; CVPR"). - **DéjàView** - Loops a shared Transformer block for multi-view reconstruction. Controlled ablations improve over separate blocks at each step; time and state gates add gains. The learned recurrence fails beyond its trained step range. [Burzio et al. 2026]( "Déjà View: Looping Transformers for Multi-View 3D Reconstruction; NeurIPS"). ## Evaluation and dataset shift - **COCO** - Introduces everyday scenes with object-instance segmentation annotations and detection baselines, moving recognition evaluation toward objects in context. [Lin et al. 2014]( "Microsoft COCO: Common Objects in Context; ECCV"). - **Perceptual similarity and LPIPS** - Evaluates deep-feature distances against human judgments of image similarity. Results concern the studied perceptual comparisons; classification accuracy alone is not the evaluation target. [Zhang et al. 2018]( "The Unreasonable Effectiveness of Deep Features as a Perceptual Metric; CVPR"). - **LVIS** - Evaluates large-vocabulary instance segmentation with a long tail of rare categories. Category frequency exposes failures hidden by aggregate accuracy; the original paper motivates evaluation where many categories have few labeled instances. [Gupta et al. 2019]( "LVIS: A Dataset for Large Vocabulary Instance Segmentation; CVPR"). - **ImageNet-C and ImageNet-P** - Benchmark image classifiers under common corruptions and perturbations. These tests measure a different robustness setting from worst-case adversarial attacks. [Hendrycks and Dietterich 2019]( "Benchmarking Neural Network Robustness to Common Corruptions and Perturbations; ICLR"). - **ImageNet replication** - Recreates CIFAR-10 and ImageNet test sets and observes accuracy drops despite similar collection procedures; probes generalization beyond repeatedly reused benchmark images. [Recht et al. 2019]( "Do ImageNet Classifiers Generalize to ImageNet?; ICML"). - **TIDE detection diagnostics** - Separates six detection/segmentation error types and estimates their effects on average precision. Classification, localization and missed detections can yield the same headline score; the decomposition uses counterfactual error corrections. [Bolya et al. 2020]( "TIDE: A General Toolbox for Identifying Object Detection Errors; ECCV"). - **Dynamic view synthesis evaluation** - Shows how monocular-video protocols can contain effective multiview signals; proposes co-visibility-aware image metrics and correspondence evaluation to expose this dependence. [Gao et al. 2022]( "Monocular Dynamic View Synthesis: A Reality Check; NeurIPS"). - **ImageNet reannotation** - Revises class definitions and adds multiple object labels. GPT-5.4 gains 5.10 accuracy points, versus 0.05 for EVA-02. The comparison excludes images with no valid class; annotation choices affect model families differently. [Volkov et al. 2026]( "Doomed to Re-Annotate, Forever: The ImageNet Story; NeurIPS"). - **Detection annotation errors** - Reannotation of KITTI, COCO, Cityscapes and VOC finds missing boxes dominate label errors. Scores shift while model rankings remain largely stable. Soft labels use at least 11 annotators per object; evaluated subsets and classes differ by dataset. [Penquitt et al. 2026]( "Object Detection Benchmarks are Incomplete: The Role of Label Errors and Annotation Uncertainty; NeurIPS"). - **DD-Ranking** - Separates image content from teacher soft labels and augmentation. On ImageNet-1k, random subsets with the same labeler beat most tested distilled sets. Composite scores measure selected robustness criteria, not universal dataset quality. [Li et al. 2026]( "DD-Ranking: Rethinking the Evaluation of Dataset Distillation; NeurIPS"). ## Related topics - [Neural Networks](https://mlpapers.org/neural-nets/) - [Generative Models](https://mlpapers.org/generative-models/) - [NLP](https://mlpapers.org/nlp/) - [Multi-View Learning](https://mlpapers.org/multiview-learning/)