Highlight

XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

Proposes Binary-Weight-Networks and XNOR-Networks that binarize CNNs for large memory savings and faster convolutions on ImageNet.

This work proposes two efficient approximations to standard CNNs. In Binary-Weight-Networks, filters are approximated with binary values, yielding 32x memory savings. In XNOR-Networks, both filters and convolutional inputs are binary, so convolutions use primarily binary operations, giving about 58x faster convolution and enabling real-time inference on CPUs. On ImageNet, a Binary-Weight-Network version of AlexNet matches full-precision AlexNet accuracy, and the method outperforms prior binarization methods BinaryConnect and BinaryNet by over 16% in top-1 accuracy.

Based on: XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks · European Conference on Computer Vision

Curated by Aramai Editorial

Read summary →
Highlight

HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

Proposes HuBERT, a self-supervised speech representation model using masked prediction of offline-clustered hidden units.

Self-supervised speech learning faces three challenges: multiple sound units per utterance, no lexicon during pre-training, and variable-length units without explicit segmentation. HuBERT (Hidden-Unit BERT) uses offline clustering to provide aligned target labels for a BERT-like prediction loss applied only over masked regions, forcing a combined acoustic and language model over continuous inputs. Relying on clustering consistency rather than label quality, HuBERT matches or beats wav2vec 2.0 on LibriSpeech and Libri-light, with up to 19% relative WER reduction at 1B parameters.

Based on: HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units · IEEE/ACM Transactions on Audio Speech and Language Processing

Curated by Aramai Editorial

Read summary →
Highlight

A random forest guided tour

Reviews recent theoretical and methodological developments of random forests, emphasizing the math behind the algorithm for non-experts.

The random forest algorithm, proposed by Breiman in 2001, is a highly successful general-purpose classification and regression method combining several randomized decision trees and averaging their predictions. It performs well when variables far outnumber observations, scales to large problems, adapts to varied tasks, and returns variable importance. This article reviews recent theoretical and methodological developments, emphasizing the math driving the algorithm, with attention to parameter selection, resampling, and variable importance for non-experts.

Based on: A random forest guided tour · Test (Madrid)

Curated by Aramai Editorial

Read summary →
Highlight

Fundamentals of Power Electronics

A comprehensive textbook on power electronics covering converter analysis, dynamics and control, magnetics, rectifiers, and resonant converters.

This textbook covers the fundamentals of power electronics in five parts. Part one treats converters in equilibrium: steady-state analysis, equivalent circuit modeling, losses and efficiency, switch realization, the discontinuous conduction mode, and converter circuits. Part two covers converter dynamics and control, including AC equivalent circuit modeling, transfer functions, controller and input filter design, and current programmed control. Remaining parts address magnetics, modern rectifiers and power system harmonics, and resonant conversion with soft switching.

Based on: Fundamentals of Power Electronics · Semantic Scholar

Curated by Aramai Editorial

Read summary →
Highlight

Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

Recasts semantic segmentation as sequence-to-sequence prediction using a pure transformer encoder (SETR) instead of an FCN encoder.

Most recent semantic segmentation uses fully-convolutional encoder-decoder networks that reduce resolution while enlarging receptive fields, with recent work adding dilated convolutions or attention. This paper instead treats segmentation as a sequence-to-sequence task, using a pure transformer without convolution or resolution reduction to encode an image as patches. With global context in every layer, the encoder pairs with a simple decoder to form SETR. It sets new SOTA on ADE20K (50.28% mIoU) and Pascal Context (55.83% mIoU), with competitive Cityscapes results.

Based on: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers · Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

OPT: Open Pre-trained Transformer Language Models

Releases OPT, a suite of open decoder-only pretrained transformers from 125M to 175B parameters, matching GPT-3.

Large language models trained for hundreds of thousands of compute days show strong zero- and few-shot ability, but their cost makes them hard to replicate, and available ones expose no full weights for study. OPT (Open Pre-trained Transformers) is a suite of decoder-only pretrained transformers from 125M to 175B parameters, shared fully and responsibly with researchers. OPT-175B is comparable to GPT-3 while requiring only 1/7th the carbon footprint to develop. The authors also release a logbook of infrastructure challenges and code for experimenting with the models.

Based on: OPT: Open Pre-trained Transformer Language Models · arXiv.org

Curated by Aramai Editorial

Read summary →
Highlight

Artificial intelligence in healthcare: past, present and future

Surveys AI applications in healthcare across data types and techniques, with a detailed focus on stroke, plus deployment hurdles.

This survey reviews the status and future of AI in healthcare, driven by growing data and advancing analytics. AI applies to structured and unstructured data, using machine learning such as support vector machines, neural networks, and deep learning for structured data, and NLP for unstructured data. Major disease areas include cancer, neurology, and cardiology. The paper reviews AI in stroke across detection and diagnosis, treatment, and outcome prediction, and discusses systems like IBM Watson and hurdles for real-life deployment.

Based on: Artificial intelligence in healthcare: past, present and future · Stroke and vascular neurology

Curated by Aramai Editorial

Read summary →
Highlight

Deep High-Resolution Representation Learning for Visual Recognition

Introduces HRNet, a backbone that maintains high-resolution representations throughout for pose estimation, segmentation, and detection.

High-resolution representations are crucial for position-sensitive vision tasks like human pose estimation, semantic segmentation, and object detection. Unlike frameworks that encode an image into a low-resolution representation and then recover it, the High-Resolution Network (HRNet) maintains high resolution throughout. It connects high-to-low resolution convolution streams in parallel and repeatedly exchanges information across resolutions, yielding semantically richer and spatially more precise representations. HRNet is a stronger backbone for these vision problems.

Based on: Deep High-Resolution Representation Learning for Visual Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence

Curated by Aramai Editorial

Read summary →
Highlight

SoilGrids250m: Global gridded soil information based on machine learning

Describes SoilGrids at 250m resolution, using machine-learning ensembles and remote-sensing covariates to map global soil properties.

The paper describes and assesses the 250m-resolution SoilGrids system, giving global predictions of standard soil properties (organic carbon, bulk density, CEC, pH, texture, coarse fragments) at seven depths, plus depth to bedrock and soil classes, about 280 raster layers total. Predictions used ~150,000 soil profiles and 158 remote-sensing covariates fitted with an ensemble of random forest, gradient boosting, and multinomial logistic regression. Ten-fold cross-validation explains 56-83% of variation (average 61%), a 60-230% accuracy gain over the prior 1km version.

Based on: SoilGrids250m: Global gridded soil information based on machine learning · PLoS ONE

Curated by Aramai Editorial

Read summary →
Highlight

SwinIR: Image Restoration Using Swin Transformer

Proposes SwinIR, a Swin Transformer baseline for image restoration covering super-resolution, denoising, and JPEG artifact reduction.

Image restoration recovers high-quality images from degraded ones, and while CNNs dominate, Transformers remain underexplored for it. SwinIR is a strong baseline built on the Swin Transformer, with shallow feature extraction, deep feature extraction, and high-quality reconstruction. Its deep module stacks residual Swin Transformer blocks (RSTB) of Swin Transformer layers with residual connections. On super-resolution, denoising, and JPEG artifact reduction, SwinIR beats state-of-the-art methods by up to 0.14-0.45 dB while using up to 67% fewer parameters.

Based on: SwinIR: Image Restoration Using Swin Transformer · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

Curated by Aramai Editorial

Read summary →
Highlight

HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering

Introduces HotpotQA, a 113k-pair Wikipedia dataset for explainable multi-hop question answering with sentence-level supporting facts.

Existing QA datasets do not train systems for complex reasoning or explanation. HotpotQA provides 113k Wikipedia-based question-answer pairs whose questions require reasoning over multiple supporting documents. Questions are diverse and not tied to any knowledge base or schema, and each includes sentence-level supporting facts enabling strong supervision and explainable predictions. It adds factoid comparison questions to test fact extraction and comparison. The dataset is challenging for leading systems, and supporting facts help models improve and explain answers.

Based on: HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering · Conference on Empirical Methods in Natural Language Processing

Curated by Aramai Editorial

Read summary →
Highlight

Generative Adversarial Imitation Learning

Proposes a framework that directly learns a policy from expert behavior, drawing an analogy between imitation learning and GANs.

The paper addresses learning a policy from expert behavior without interacting with the expert or accessing a reward signal. A common indirect approach recovers the expert's cost via inverse RL then extracts a policy with RL, which is slow. The authors propose a framework to directly extract a policy from data, as if produced by RL after inverse RL. One instantiation draws an analogy between imitation learning and GANs, giving a model-free algorithm that significantly outperforms existing model-free methods on complex, high-dimensional imitation tasks.

Based on: Generative Adversarial Imitation Learning · Neural Information Processing Systems

Curated by Aramai Editorial

Read summary →