Highlight

Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression

Introduces Generalized IoU (GIoU), a metric and loss for bounding box regression that handles non-overlapping boxes and improves object detectors.

Intersection over Union (IoU) is the standard object-detection metric, yet common distance-based losses for regressing bounding boxes do not directly optimize it. IoU can act as a regression loss for axis-aligned 2D boxes, but its plateau makes optimization infeasible for non-overlapping boxes. The authors introduce Generalized IoU (GIoU) as both a new metric and a new loss that fixes this weakness. Adding GIoU as a loss to state-of-the-art detectors yields consistent improvements on PASCAL VOC and MS COCO under both IoU- and GIoU-based measures.

Based on: Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression · Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

The Power of Scale for Parameter-Efficient Prompt Tuning

Introduces prompt tuning, learning soft prompts via backpropagation to adapt frozen language models, matching full model tuning as scale grows.

The paper explores prompt tuning, a mechanism for learning soft prompts that condition frozen language models for downstream tasks. Unlike GPT-3's discrete text prompts, soft prompts are learned via backpropagation from labeled examples and outperform GPT-3's few-shot learning by a large margin. Ablations with T5 show the method grows more competitive with scale, matching full model tuning once models exceed billions of parameters. Soft prompts also improve robustness to domain transfer and enable efficient prompt ensembling.

Based on: The Power of Scale for Parameter-Efficient Prompt Tuning · Conference on Empirical Methods in Natural Language Processing

Curated by Aramai Editorial

Read summary →
Highlight

RoFormer: Enhanced Transformer with Rotary Position Embedding

Introduces Rotary Position Embedding (RoPE), encoding absolute position via a rotation matrix and adding relative-position dependency in self-attention.

The paper investigates how to integrate positional information into transformer-based language models and proposes Rotary Position Embedding (RoPE). RoPE encodes absolute position with a rotation matrix while incorporating explicit relative-position dependency into self-attention. It offers flexibility in sequence length, decaying inter-token dependency with distance, and compatibility with linear self-attention. Evaluated as RoFormer on long-text classification benchmarks, it consistently outperforms alternatives and is supported by theoretical analysis.

Based on: RoFormer: Enhanced Transformer with Rotary Position Embedding · Neurocomputing

Curated by Aramai Editorial

Read summary →
Highlight

Book Reviews: Foundations of Statistical Natural Language Processing

The first comprehensive introduction to statistical natural language processing, covering the theory and algorithms needed to build NLP tools.

Statistical approaches to natural language text had become dominant, and this foundational text offers the first comprehensive introduction to statistical natural language processing (NLP). It provides the theory and algorithms needed to build NLP tools, with broad but rigorous coverage of mathematical and linguistic foundations and detailed discussion of statistical methods, letting students and researchers build their own implementations. Covered topics include collocation finding, word sense disambiguation, probabilistic parsing, and information retrieval.

Based on: Book Reviews: Foundations of Statistical Natural Language Processing · International Conference on Computational Logic

Curated by Aramai Editorial

Read summary →
Highlight

OrthoMCL: identification of ortholog groups for eukaryotic genomes.

OrthoMCL is a scalable Markov Cluster algorithm-based method for identifying ortholog and paralog groups across multiple eukaryotic genomes.

Identifying orthologous gene groups aids genome annotation, evolutionary studies, and comparative genomics, but prokaryote-oriented methods struggle with eukaryotes, which have many paralogs and incomplete sequences. OrthoMCL scalably constructs orthologous groups across multiple eukaryotic taxa using a Markov Cluster algorithm to group orthologs and paralogs. It matches INPARANOID on two genomes but extends to many species, and agreement with EC annotations suggests high reliability for automated annotation. It was applied to seven genomes and offers a web interface.

Based on: OrthoMCL: identification of ortholog groups for eukaryotic genomes. · Genome Research

Curated by Aramai Editorial

Read summary →
Highlight

REVIGO Summarizes and Visualizes Long Lists of Gene Ontology Terms

REVIGO is a web server that summarizes long, redundant Gene Ontology term lists into a representative subset using semantic-similarity clustering.

High-throughput biology is often interpreted by testing for enriched Gene Ontology (GO) categories, but the resulting GO term lists are frequently long and redundant, making interpretation difficult. REVIGO is a web server that summarizes such lists by finding a representative subset of terms using a simple clustering algorithm based on semantic similarity measures. It visualizes this non-redundant set several ways, including multidimensional scaling and graph-based layouts that render semantic relationships, plus treemaps and tag clouds. REVIGO is freely available online.

Based on: REVIGO Summarizes and Visualizes Long Lists of Gene Ontology Terms · PLoS ONE

Curated by Aramai Editorial

Read summary →
Highlight

Instant neural graphics primitives with a multiresolution hash encoding

Introduces a multiresolution hash encoding that lets small networks train and render neural graphics primitives orders of magnitude faster.

Neural graphics primitives built from fully connected networks are costly to train and evaluate. This paper reduces that cost with an input encoding that lets a smaller network keep quality while cutting compute and memory operations. A small network is augmented by a multiresolution hash table of trainable feature vectors optimized by SGD, and the multiresolution structure disambiguates hash collisions. Implemented as fully-fused CUDA kernels, it achieves several orders of magnitude speedup: training in seconds and rendering 1920x1080 in tens of milliseconds.

Based on: Instant neural graphics primitives with a multiresolution hash encoding · ACM Transactions on Graphics

Curated by Aramai Editorial

Read summary →
Highlight

VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text

Presents VADER, a parsimonious rule-based sentiment analysis model tuned for social media text using a validated lexicon and grammatical intensity rules.

VADER is a simple, rule-based model for general sentiment analysis designed for social media text. The authors build and empirically validate a gold-standard lexicon with sentiment-intensity measures attuned to microblog-like contexts, then combine it with five rules capturing grammatical and syntactic conventions for expressing sentiment. Compared against eleven benchmarks including LIWC, ANEW, SentiWordNet, and ML methods (Naive Bayes, Maximum Entropy, SVM), VADER outperforms individual human raters on tweets (F1 of 0.96 vs 0.84) and generalizes better across contexts.

Based on: VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text · International Conference on Web and Social Media

Curated by Aramai Editorial

Read summary →
Highlight

Learning Transferable Features with Deep Adaptation Networks

Proposes Deep Adaptation Networks that match domain distributions in a reproducing kernel Hilbert space for transferable features in domain adaptation.

Deep networks learn transferable features, but transferability drops in higher layers as features grow specific and domain discrepancy rises. The authors propose the Deep Adaptation Network (DAN), generalizing deep CNNs to domain adaptation to reduce dataset bias in task-specific layers. DAN embeds task-specific layers' representations in a reproducing kernel Hilbert space and matches domains' mean embeddings, via optimal multikernel selection. It yields transferable features with statistical guarantees, scales linearly, and sets state of the art on standard benchmarks.

Based on: Learning Transferable Features with Deep Adaptation Networks · International Conference on Machine Learning

Curated by Aramai Editorial

Read summary →
Highlight

Context Encoders: Feature Learning by Inpainting

Introduces Context Encoders, a CNN that learns visual features unsupervised by inpainting missing image regions from their surrounding context.

Context Encoders is an unsupervised feature-learning method based on context-driven pixel prediction. A CNN is trained to generate an arbitrary missing image region conditioned on its surroundings, forcing it to understand the whole image and hypothesize plausible content. A combined reconstruction and adversarial loss yields sharper results than reconstruction alone by handling multiple output modes. The learned features capture appearance and semantics, improve CNN pre-training for classification, detection, and segmentation, plus enable semantic inpainting.

Based on: Context Encoders: Feature Learning by Inpainting · Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

On the difficulty of training recurrent neural networks

Analyzes the vanishing and exploding gradient problems in RNNs and proposes gradient norm clipping and a soft constraint as remedies.

Training recurrent neural networks is hampered by two well-known problems, the vanishing and exploding gradients described by Bengio et al. (1994). This paper deepens understanding of these issues by analyzing them from analytical, geometric, and dynamical-systems perspectives. That analysis motivates a simple yet effective solution: a gradient norm clipping strategy for exploding gradients and a soft constraint for vanishing gradients. The authors validate their hypotheses and proposed solutions empirically.

Based on: On the difficulty of training recurrent neural networks · International Conference on Machine Learning

Curated by Aramai Editorial

Read summary →
Highlight

Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories

Presents an incremental Bayesian method that learns generative object-category models from only a few training images, tested across 101 object categories.

Existing methods for learning object categories need thousands of images, are slow, cannot learn incrementally, and ignore prior knowledge. The authors learn categories from just a few images by reusing priors from previously learned, unrelated categories. A generative probabilistic model captures the shape and appearance of a feature constellation, learned incrementally in a Bayesian way. On 101 diverse categories, the incremental method matches a batch Bayesian version on small training sets while being much faster, and both Bayesian methods beat maximum likelihood.

Based on: Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories · 2004 Conference on Computer Vision and Pattern Recognition Workshop

Curated by Aramai Editorial

Read summary →