Highlight

Learning Transferable Visual Models From Natural Language Supervision

Learns visual representations by predicting image-caption pairings from 400 million web image-text pairs, enabling zero-shot transfer.

Standard vision systems train on fixed predetermined categories, limiting generality without more labeled data. This paper learns directly from raw image text via a simple pre-training task predicting which caption matches which image, scaled to 400 million web image-text pairs. After pre-training, natural language enables zero-shot transfer, benchmarked across 30+ vision datasets spanning OCR, action recognition, and fine-grained classification, often matching supervised baselines, e.g. matching ResNet-50's zero-shot ImageNet accuracy without its training data.

Based on: Learning Transferable Visual Models From Natural Language Supervision · International Conference on Machine Learning

Curated by Aramai Editorial

Read summary →
Highlight

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Details the design principles behind PyTorch, a deep learning library combining Pythonic imperative usability with GPU-accelerated speed.

Deep learning frameworks have typically traded off usability against speed. PyTorch is presented as a library showing these goals are compatible, designed to support an imperative, Pythonic programming style where code acts as the model, debugging is easy, and it stays consistent with other scientific computing libraries, while remaining efficient on hardware accelerators like GPUs. The paper details the principles behind PyTorch's implementation and architecture, and demonstrates the efficiency of its subsystems and overall speed on common benchmarks.

Based on: PyTorch: An Imperative Style, High-Performance Deep Learning Library · Neural Information Processing Systems

Curated by Aramai Editorial

Read summary →
Highlight

Microsoft COCO: Common Objects in Context

Presents the COCO dataset of everyday scenes with 2.5 million segmented object instances to advance scene-level object recognition.

This paper presents a dataset advancing object recognition by situating it within scene understanding, using images of complex everyday scenes with common objects in natural context. Objects across 91 categories are labeled with per-instance segmentations for precise localization, totaling 2.5 million instances across 328,000 images gathered via crowd-worker annotation interfaces. The paper compares statistics to PASCAL, ImageNet, and SUN, and gives baseline detection/segmentation results using a Deformable Parts Model.

Based on: Microsoft COCO: Common Objects in Context · European Conference on Computer Vision

Curated by Aramai Editorial

Read summary →
Highlight

Particle swarm optimization

Introduces particle swarm optimization, a population-based method for optimizing nonlinear functions inspired by swarm behavior.

This paper introduces a concept for optimizing nonlinear functions using particle swarm methodology, outlining the evolution of several paradigm variants and discussing the implementation of one of them. The chosen paradigm is subjected to benchmark testing, with proposed applications including nonlinear function optimization and neural network training. The paper also describes how particle swarm optimization relates to both artificial life and genetic algorithms.

Based on: Particle swarm optimization · International Conference on Neural Networks

Curated by Aramai Editorial

Read summary →
Highlight

Language Models are Few-Shot Learners

Trains the 175-billion-parameter GPT-3 and shows strong few-shot task performance from text prompts alone, without fine-tuning.

Prior NLP gains came from pretraining plus fine-tuning needing many labeled examples, unlike humans who learn from a few examples. This paper trains GPT-3, a 175-billion-parameter autoregressive language model 10x larger than prior non-sparse models, evaluated via pure in-context few-shot prompting with no gradient updates. GPT-3 performs strongly on translation, QA, cloze, and reasoning tasks, rivaling some fine-tuned methods, though it struggles on some datasets and can write news articles hard to distinguish from human-written ones.

Based on: Language Models are Few-Shot Learners · Neural Information Processing Systems

Curated by Aramai Editorial

Read summary →
Highlight

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Applies a pure Transformer directly to sequences of image patches, matching or beating CNNs on classification with less training compute.

Transformers are standard in NLP but have had limited use in vision, where attention is usually combined with convolutional networks. This paper shows a pure Transformer applied directly to sequences of image patches, without convolutions, performs well on image classification. Pre-trained on large data and transferred to ImageNet, CIFAR-100, and VTAB, the resulting Vision Transformer (ViT) matches or beats state-of-the-art CNNs while needing substantially less compute to train.

Based on: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale · International Conference on Learning Representations

Curated by Aramai Editorial

Read summary →
Highlight

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Introduces a Region Proposal Network sharing convolutional features with the detector for near cost-free, real-time object detection.

Object detection pipelines relied on separate region proposal algorithms, which became the speed bottleneck once Fast R-CNN sped up detection itself. This paper introduces a Region Proposal Network (RPN) that shares image features with the detector to predict object bounds and objectness scores at near-zero extra cost. Merged with Fast R-CNN into one network, the system runs at 5fps on VGG-16 with state-of-the-art accuracy on PASCAL VOC and MS COCO using only 300 proposals per image, and underpinned several ILSVRC/COCO 2015 winning entries.

Based on: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks · IEEE Transactions on Pattern Analysis and Machine Intelligence

Curated by Aramai Editorial

Read summary →
Highlight

Very Deep Convolutional Networks for Large-Scale Image Recognition

Shows that stacking small 3x3 convolution filters into 16-19 layer networks greatly improves large-scale image recognition.

This work studies how convolutional network depth affects accuracy in large-scale image recognition, using architectures built from very small 3x3 convolution filters. Pushing depth to 16-19 weight layers gives a significant accuracy gain over prior configurations, underlying a top ImageNet Challenge 2014 submission that placed first in localisation and second in classification. The representations generalise well to other datasets, and the two best models were released publicly.

Based on: Very Deep Convolutional Networks for Large-Scale Image Recognition · International Conference on Learning Representations

Curated by Aramai Editorial

Read summary →
Highlight

Squeeze-and-Excitation Networks

Introduces Squeeze-and-Excitation blocks that adaptively recalibrate channel-wise CNN features to boost representational power.

The paper focuses on channel relationships in CNNs, proposing the Squeeze-and-Excitation (SE) block, an architectural unit that adaptively recalibrates channel-wise feature responses by explicitly modeling interdependencies between channels. Stacking SE blocks yields SENet architectures that generalize well across challenging datasets, giving performance gains for state-of-the-art architectures at minimal extra cost. SENets underpinned the ILSVRC 2017 winning classification submission, cutting top-5 error to 2.251%, about 25% better than the 2016 winner.

Based on: Squeeze-and-Excitation Networks · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

SSD: Single Shot MultiBox Detector

Presents SSD, a single deep network for object detection that discretizes output space into default boxes across feature maps.

SSD detects objects using a single deep network that discretizes bounding box output into default boxes across aspect ratios and scales per feature-map location, producing category scores and box adjustments at prediction time. Combining predictions from multiple-resolution feature maps handles objects of varying sizes and eliminates proposal generation and resampling stages, simplifying training and inference. On VOC2007, SSD reaches 72.1% mAP at 58 FPS (300x300 input) and 75.1% mAP (500x500), matching or exceeding Faster R-CNN while being much faster.

Based on: SSD: Single Shot MultiBox Detector · European Conference on Computer Vision

Curated by Aramai Editorial

Read summary →
Highlight

Semi-Supervised Classification with Graph Convolutional Networks

Proposes graph convolutional networks, a scalable spectral-approximation CNN variant for semi-supervised learning on graphs.

The paper presents a scalable semi-supervised learning approach for graph-structured data using an efficient convolutional neural network variant operating directly on graphs, motivated by a localized first-order approximation of spectral graph convolutions. The model scales linearly with the number of graph edges and learns hidden-layer representations encoding both local graph structure and node features. Experiments on citation networks and a knowledge graph dataset show the approach outperforms related methods by a significant margin.

Based on: Semi-Supervised Classification with Graph Convolutional Networks · International Conference on Learning Representations

Curated by Aramai Editorial

Read summary →
Highlight

A Unified Approach to Interpreting Model Predictions

Introduces SHAP, a unified additive feature-attribution framework unifying six prior methods for interpreting model predictions.

Complex models like ensembles and deep networks often achieve top accuracy but are hard to interpret, creating tension between accuracy and interpretability. The paper presents SHAP (SHapley Additive exPlanations), a unified framework assigning each feature an importance value for a prediction, identifying a class of additive feature importance measures and proving a unique solution with desirable properties. This unifies six existing interpretation methods and motivates new methods with better computational performance and human-intuition alignment.

Based on: A Unified Approach to Interpreting Model Predictions · Neural Information Processing Systems

Curated by Aramai Editorial

Read summary →