Highlight

GhostNet: More Features From Cheap Operations

Proposes the Ghost module, generating extra feature maps via cheap linear operations, to build the efficient GhostNet for embedded devices.

GhostNet targets the difficulty of deploying CNNs on embedded devices with limited memory and compute. Observing that feature-map redundancy is characteristic of successful CNNs, the authors propose a Ghost module that generates more feature maps from cheap operations: from a set of intrinsic feature maps, inexpensive linear transformations produce additional ghost features. Stacking these modules into Ghost bottlenecks forms the lightweight GhostNet, which reaches 75.7% top-1 accuracy on ImageNet, exceeding MobileNetV3 at similar cost.

Based on: GhostNet: More Features From Cheap Operations · Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

Shows how to train ImageNet ResNet-50 in one hour using large 8192 minibatches with a linear LR scaling rule and warmup scheme.

This paper tackles the long training times of deep networks by scaling distributed synchronous SGD across many parallel workers, which requires large minibatches. The authors show that large minibatches cause optimization difficulties on ImageNet, but once these are addressed, networks generalize with no loss of accuracy up to a minibatch size of 8192. Key techniques are a hyper-parameter-free linear scaling rule for learning rates and a new warmup scheme. Their Caffe2 system trains ResNet-50 in one hour on 256 GPUs with about 90% scaling efficiency.

Based on: Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour · arXiv.org

Curated by Aramai Editorial

Read summary →
Highlight

Learning to Compare: Relation Network for Few-Shot Learning

Introduces the Relation Network, an end-to-end few-shot classifier that learns a distance metric to compare examples, and extends to zero-shot learning.

The authors present a flexible, general framework for few-shot learning, where a classifier must recognize new classes from only a few examples, and train their Relation Network (RN) end-to-end from scratch. During meta-learning it learns a deep distance metric to compare images within episodes that simulate the few-shot setting. Once trained, it classifies new-class images via relation scores against the few examples, with no further updates. The framework also extends to zero-shot learning, and on five benchmarks it provides a unified, effective approach to both tasks.

Based on: Learning to Compare: Relation Network for Few-Shot Learning · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering

Combines bottom-up and top-down attention over image regions, setting new state-of-the-art on MSCOCO captioning and winning the 2017 VQA Challenge.

Top-down visual attention is common in image captioning and visual question answering (VQA) for fine-grained analysis and reasoning. This work proposes a combined bottom-up and top-down mechanism operating at the level of objects and salient image regions. A bottom-up module based on Faster R-CNN proposes regions with feature vectors, while the top-down module sets feature weightings. On the MSCOCO test server it achieves a new state-of-the-art (CIDEr 117.9, SPICE 21.5, BLEU-4 36.9), and the same method won first place in the 2017 VQA Challenge.

Based on: Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

CornerNet: Detecting Objects as Paired Keypoints

Introduces CornerNet, an anchor-free detector that finds objects as pairs of top-left and bottom-right corner keypoints with corner pooling.

CornerNet reframes object detection by detecting each bounding box as a pair of keypoints, the top-left and bottom-right corners, using a single convolutional neural network. This formulation removes the need for the anchor boxes common in prior single-stage detectors. The authors also introduce corner pooling, a new pooling layer that helps the network localize corners more accurately. CornerNet reaches 42.2% AP on MS COCO, outperforming all existing one-stage detectors.

Based on: CornerNet: Detecting Objects as Paired Keypoints · International Journal of Computer Vision

Curated by Aramai Editorial

Read summary →
Highlight

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

Proposes weighing multi-task losses by each task's homoscedastic uncertainty, enabling joint depth, semantic, and instance segmentation.

This paper observes that multi-task deep learning performance depends heavily on the relative weighting of each task's loss, which is difficult and costly to tune by hand. It proposes a principled method that weighs multiple loss functions using the homoscedastic uncertainty of each task, enabling simultaneous learning of quantities with different units and scales across classification and regression. Applied to per-pixel depth regression, semantic and instance segmentation from a single image, the model learns its own weightings and outperforms separately trained models.

Based on: Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition

Describes the CoNLL-2003 shared task on language-independent named entity recognition, its English and German datasets, evaluation, and systems.

This paper describes the CoNLL-2003 shared task on language-independent named entity recognition. It provides background on the two data sets used, for English and German, and explains the evaluation method applied to the task. The authors give a general overview of the systems that participated and discuss their performance.

Based on: Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition · Conference on Computational Natural Language Learning

Curated by Aramai Editorial

Read summary →
Highlight

Self-Attention Generative Adversarial Networks

Introduces SAGAN, adding self-attention and spectral normalization to GANs for long-range dependency modeling in image generation.

The Self-Attention Generative Adversarial Network (SAGAN) introduces attention-driven, long-range dependency modeling for image generation. Unlike convolutional GANs that build details from spatially local points, SAGAN generates details using cues from all feature locations, and its discriminator checks consistency across distant image regions. Applying spectral normalization to the generator further improves training. SAGAN raises the Inception score from 36.8 to 52.52 and lowers FID from 27.62 to 18.65 on ImageNet.

Based on: Self-Attention Generative Adversarial Networks · International Conference on Machine Learning

Curated by Aramai Editorial

Read summary →
Highlight

A large annotated corpus for learning natural language inference

Introduces SNLI, a 570K-pair human-annotated corpus for natural language inference, two orders of magnitude larger than prior resources.

Understanding entailment and contradiction is fundamental to language understanding and a testbed for semantic representations, but progress was limited by small datasets. The authors introduce the Stanford Natural Language Inference (SNLI) corpus, a freely available collection of human-labeled sentence pairs from a grounded task based on image captioning. At 570K pairs it is two orders of magnitude larger than prior resources, letting lexicalized classifiers beat some sophisticated entailment models and letting a neural network perform competitively for the first time.

Based on: A large annotated corpus for learning natural language inference · Conference on Empirical Methods in Natural Language Processing

Curated by Aramai Editorial

Read summary →
Highlight

Stochastic Processes

Introduces stochastic processes as probabilistic models of data streams and their mathematical basis for science and engineering.

This work presents stochastic processes as probabilistic models of data streams, including speech, audio and video signals, stock market prices, and measurements of physical phenomena captured by digital sensors like medical instruments, GPS receivers, or seismographs. It stresses that understanding the mathematical basis of these models is essential for interpreting phenomena and processing information across many fields of science and engineering, such as physics, communications, signal processing, automation, and structural dynamics.

Based on: Stochastic Processes · Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics

Curated by Aramai Editorial

Read summary →
Highlight

Instance Normalization: The Missing Ingredient for Fast Stylization

Shows that swapping batch normalization for instance normalization at train and test time markedly improves fast neural image stylization.

This paper revisits a fast image stylization method and shows that a small architectural change yields a significant qualitative improvement in generated images. The change is simply swapping batch normalization for instance normalization, applied at both training and testing time. This modification enables training of high-performance architectures for real-time image generation, and the code is released publicly.

Based on: Instance Normalization: The Missing Ingredient for Fast Stylization · arXiv.org

Curated by Aramai Editorial

Read summary →
Highlight

Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization

Produces a reliable, publicly available intrusion detection dataset with benign and seven common attack flows, and evaluates features and ML algorithms.

Growing networks raise the potential damage from attacks, making intrusion detection and prevention systems vital defenses. Anomaly-based approaches suffer from a lack of adequate datasets: reviewing eleven datasets since 1998, the authors find many outdated or unreliable, lacking traffic diversity, attack variety, or full feature sets. This paper produces a reliable, publicly available dataset with benign traffic and seven common attack flows, then evaluates traffic features and machine learning algorithms to identify the best features for detecting each attack category.

Based on: Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization · International Conference on Information Systems Security and Privacy

Curated by Aramai Editorial

Read summary →