Highlight

LAMMPS - A flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales

Definitive reference for LAMMPS, the open-source molecular dynamics simulator for particle-based materials modeling from atomic to continuum scales.

Since its 2004 open-source release, LAMMPS has become a widely used tool for particle-based materials modeling from atomic to mesoscale to continuum length scales, growing from fifty thousand lines of code to a million. The paper describes several of its fundamental algorithms and the design strategies that keep it flexible for users and developers. It also highlights recently added capabilities, including dynamic load balancing, on-the-fly visualization, magnetic spin dynamics models, and quantum-accuracy machine learning interatomic potentials.

Based on: LAMMPS - A flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales · Computer Physics Communications

Curated by Aramai Editorial

Read summary →
Highlight

Prototypical Networks for Few-shot Learning

Proposes Prototypical Networks for few-shot classification, learning a metric space where classes are predicted by distance to class prototypes.

Prototypical Networks address few-shot classification, where a classifier must generalize to classes unseen in training given only a few examples of each. They learn a metric space in which classification is performed by computing distances to prototype representations of each class, a simpler inductive bias than recent approaches that benefits the limited-data regime. Analysis shows simple design decisions yield substantial gains over approaches with complicated architectures and meta-learning, and a zero-shot extension achieves state-of-the-art results on CU-Birds.

Based on: Prototypical Networks for Few-shot Learning · Neural Information Processing Systems

Curated by Aramai Editorial

Read summary →
Highlight

Visual Instruction Tuning

Introduces LLaVA, a large multimodal model instruction-tuned on GPT-4-generated language-image data, connecting a vision encoder with an LLM.

Instruction tuning LLMs on machine-generated data improves zero-shot capabilities but is less explored in the multimodal field. This paper makes the first attempt to use language-only GPT-4 to generate multimodal language-image instruction-following data, and instruction-tunes LLaVA, an end-to-end large multimodal model connecting a vision encoder and an LLM. LLaVA shows strong multimodal chat ability, scoring 85.1% relative to GPT-4 on a synthetic multimodal benchmark, and with GPT-4 achieves state-of-the-art 92.53% on Science QA; data, model, and code are public.

Based on: Visual Instruction Tuning · Neural Information Processing Systems

Curated by Aramai Editorial

Read summary →
Highlight

Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding

Introduces deep compression, a pruning, trained quantization, and Huffman coding pipeline cutting neural network storage 35x-49x with no accuracy loss.

Neural networks are compute- and memory-intensive, making deployment on resource-limited embedded systems difficult. Deep compression is a three-stage pipeline of pruning, trained quantization, and Huffman coding, with retraining to fine-tune remaining connections and quantized centroids; it cuts storage 35x to 49x without accuracy loss. AlexNet shrinks from 240MB to 6.9MB and VGG-16 from 552MB to 11.3MB, letting models fit in on-chip SRAM rather than off-chip DRAM, with 3x-4x layerwise speedup and 3x-7x better energy efficiency.

Based on: Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding · International Conference on Learning Representations

Curated by Aramai Editorial

Read summary →
Highlight

Non-local Neural Networks

Presents non-local operations as generic building blocks that capture long-range dependencies by computing responses as weighted sums over all positions.

Convolutional and recurrent operations process one local neighborhood at a time. Inspired by the classical non-local means method, this paper presents non-local operations, a generic family of building blocks that compute the response at a position as a weighted sum of the features at all positions and can be plugged into many computer vision architectures. Without bells and whistles, non-local models compete with or outperform competition winners on Kinetics and Charades video classification, and improve object detection, segmentation, and pose estimation on COCO.

Based on: Non-local Neural Networks · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

Overcoming catastrophic forgetting in neural networks

Proposes training neural networks sequentially by selectively slowing learning on weights important to previous tasks, overcoming catastrophic forgetting

Deep neural networks, unlike humans, struggle to learn multiple tasks sequentially, and catastrophic forgetting was widely thought to be an inevitable feature of connectionist models. Inspired by synaptic consolidation in neuroscience, this work trains networks sequentially by selectively slowing learning on the weights important for previous tasks. The approach proves scalable and effective on hand-written digit classification tasks and sequentially learned Atari 2600 games, maintaining expertise on tasks not experienced for a long time.

Based on: Overcoming catastrophic forgetting in neural networks · Proceedings of the National Academy of Sciences of the United States of America

Curated by Aramai Editorial

Read summary →
Highlight

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Develops diffusion-based generative models where a forward process slowly destroys data structure and a learned reverse process restores it.

The work targets a central machine learning problem: modeling complex data with distributions that stay flexible yet tractable for learning, sampling, inference, and evaluation. Inspired by non-equilibrium statistical physics, an iterative forward diffusion process slowly destroys structure in the data, and a learned reverse diffusion restores it, yielding a flexible, tractable generative model. The method scales to thousands of layers or time steps, supports conditional and posterior probabilities, and has an open-source reference implementation.

Based on: Deep Unsupervised Learning using Nonequilibrium Thermodynamics · International Conference on Machine Learning

Curated by Aramai Editorial

Read summary →
Highlight

Evaluating Large Language Models Trained on Code

Introduces Codex, a GPT model fine-tuned on GitHub code, and HumanEval, a benchmark for functional correctness of programs synthesized from docstrings.

The paper introduces Codex, a GPT language model fine-tuned on public GitHub code, and studies its Python code-writing abilities; a distinct production version powers GitHub Copilot. On HumanEval, a newly released benchmark measuring functional correctness of programs synthesized from docstrings, Codex solves 28.8% of problems versus 0% for GPT-3 and 11.4% for GPT-J, and repeated sampling solves 70.2% with 100 samples per problem. The authors also examine limitations, such as long chains of operations and variable binding, and discuss safety, security, and economic impacts.

Based on: Evaluating Large Language Models Trained on Code · arXiv.org

Curated by Aramai Editorial

Read summary →
Highlight

Improved Techniques for Training GANs

Presents new GAN training techniques that achieve state-of-the-art semi-supervised classification and generate images humans find visually realistic.

The authors present new architectural features and training procedures for the GAN framework, focused on semi-supervised learning and generating images humans find visually realistic, rather than maximizing test-data likelihood. The techniques achieve state-of-the-art semi-supervised classification on MNIST, CIFAR-10, and SVHN. A visual Turing test confirms image quality: generated MNIST samples are indistinguishable from real data, CIFAR-10 samples yield a 21.3% human error rate, and ImageNet samples at unprecedented resolution show recognizable class features.

Based on: Improved Techniques for Training GANs · Neural Information Processing Systems

Curated by Aramai Editorial

Read summary →
Highlight

YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors

Introduces YOLOv7, a real-time object detector combining trainable bag-of-freebies training tools with a new architecture and compound scaling method.

Real-time object detection is a central computer vision problem, and the authors identify two research topics that emerged from recent architecture and training optimization methods. They address these with a trainable bag-of-freebies solution, combining flexible, efficient training tools with a proposed architecture and compound scaling method. YOLOv7 surpasses all known object detectors in speed and accuracy across 5-120 FPS, reaching 56.8% AP, the highest among real-time detectors at 30 FPS or higher on a V100 GPU; source code is released.

Based on: YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors · Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →
Highlight

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Introduces Batch Normalization, normalizing layer inputs per mini-batch to speed up and stabilize deep neural network training.

Training deep networks is complicated by internal covariate shift, where each layer's input distribution shifts as prior layers update, forcing low learning rates and careful initialization. Batch Normalization addresses this by normalizing layer inputs per mini-batch as part of the architecture, enabling higher learning rates and less careful initialization, sometimes removing the need for Dropout. Applied to a state-of-the-art classifier, it matches accuracy with 14x fewer steps, and a batch-normalized ensemble reaches 4.82% top-5 ImageNet error, beating human raters.

Based on: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift · International Conference on Machine Learning

Curated by Aramai Editorial

Read summary →
Highlight

Going deeper with convolutions

Proposes the Inception architecture (GoogLeNet), a 22-layer deep network improving compute efficiency for ImageNet classification and detection.

This paper proposes the Inception deep convolutional architecture, achieving new state-of-the-art results in ILSVRC14 classification and detection. Its key feature is improved computing resource utilization: depth and width increase while the computational budget stays constant, guided by the Hebbian principle and multi-scale processing intuition. The ILSVRC14 submission, a 22-layer network called GoogLeNet, is evaluated on both classification and detection quality.

Based on: Going deeper with convolutions · Computer Vision and Pattern Recognition

Curated by Aramai Editorial

Read summary →