Keep Current

Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

Data can be represented as a vector in a continuous space, but often it is concentrated in a narrow area. Can it be improved?

DeepNightLearners

Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration

Augmenting images training data to add variability doesn't always end well since the augmentation changes the semantic content of the image. Here's how to solve it.

DeepNightLearners

Highly accurate protein structure prediction with AlphaFold

AlphaFold2 is the strongest tool today to predict protein 3D structure. It will be massively used to bring the world protein research forward. Since it is very complex and contains many new architectures, in this post, we gradually decompose the system into its base components.

DeepNightLearners

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

This paper suggests a new computationally efficient method for constructing low-dimensional representation of unlabeled data.

Computer Vision

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

A review and clear explanation of the NeRF method, which can be used to synthesize 3D scenes out of an input image. This method is the base for many other research methods that followed.

Computer Vision

Perceiver: General Perception with Iterative Attention

To overcome Transformers' squared complexity (w.r.t input length), the Perceiver article here offers a novel method to learn the QKV matrices. Check it out!

DeepNightLearners

Discriminator Rejection Sampling

A summary of a method to improve GAN generated images by exploiting accumulated "information" from the Discriminator during the GAN training process.

DeepNightLearners

Sharpness-Aware Minimization for Efficiently Improving Generalization

This very relevant article suggests an unusual method to improve neural network regularization abilities. I believe that this method has a great potential to enter the standard neural network training toolkit. I was also impressed by the detailed comparison to other methods.

DeepNightLearners

Representation learning via invariant causal mechanisms

This article suggests a method to construct augmentation resilient and styling invariant image representation in lower dimension (embeddings).

DeepNightLearners

Rethinking Attention with Performers

The article suggests a method to lower the Transformer's complexity to a linear order and proves all the arguments also in a rigorous form. The article is not easy to read, but luckily, to understand the main idea, the first 5-6 pages are more than enough.

DeepNightLearners

A causal view of compositional zero-shot recognition

One of the main challenges in zero-shot learning is enabling compositional generalization to the mode. This review is part of a series of reviews in Machine & Deep Learning that are originally published in Hebrew, aiming to make it accessible in a plain language under the name #DeepNightLearners.

DeepNightLearners

Supermasks in Superposition

The article suggests a brilliant training method, called SupSup, of a large neural network, which allows it to perform several different tasks.

Subscribe to Keep Current