Keep Current

Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

Data can be represented as a vector in a continuous space, but often it is concentrated in a narrow area. Can it be improved?

DeepNightLearners

Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration

Augmenting images training data to add variability doesn't always end well since the augmentation changes the semantic content of the image. Here's how to solve it.

DeepNightLearners

Highly accurate protein structure prediction with AlphaFold

AlphaFold2 is the strongest tool today to predict protein 3D structure. It will be massively used to bring the world protein research forward. Since it is very complex and contains many new architectures, in this post, we gradually decompose the system into its base components.

DeepNightLearners

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

This paper suggests a new computationally efficient method for constructing low-dimensional representation of unlabeled data.

Computer Vision

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

A review and clear explanation of the NeRF method, which can be used to synthesize 3D scenes out of an input image. This method is the base for many other research methods that followed.

Monthly Updates

Keep-Current - April 2021 Summary

Self-Supervised Learning keynote talk by Yann LeCunn; Perceiver - computation efficient transformer algorithm; Data Visualization also for summaries and more news in this monthly update.

Computer Vision

Convolutional Networks on Tabular data

Yam Peleg examines a kaggle solution using convolutional neural networks which can process tabular data while being columns order agnostic.

Computer Vision

Perceiver: General Perception with Iterative Attention

To overcome Transformers' squared complexity (w.r.t input length), the Perceiver article here offers a novel method to learn the QKV matrices. Check it out!

DeepNightLearners

Discriminator Rejection Sampling

A summary of a method to improve GAN generated images by exploiting accumulated "information" from the Discriminator during the GAN training process.

DeepNightLearners

Sharpness-Aware Minimization for Efficiently Improving Generalization

This very relevant article suggests an unusual method to improve neural network regularization abilities. I believe that this method has a great potential to enter the standard neural network training toolkit. I was also impressed by the detailed comparison to other methods.

DeepNightLearners

Representation learning via invariant causal mechanisms

This article suggests a method to construct augmentation resilient and styling invariant image representation in lower dimension (embeddings).

Monthly Updates

Keep-Current - March 2021 Edition

Every month, we deliver a community update with a list of the latest articles and technologies, in various topics, which worth knowing about. This is March 2021 edition.

DeepNightLearners

Rethinking Attention with Performers

The article suggests a method to lower the Transformer's complexity to a linear order and proves all the arguments also in a rigorous form. The article is not easy to read, but luckily, to understand the main idea, the first 5-6 pages are more than enough.

DeepNightLearners

A causal view of compositional zero-shot recognition

One of the main challenges in zero-shot learning is enabling compositional generalization to the mode. This review is part of a series of reviews in Machine & Deep Learning that are originally published in Hebrew, aiming to make it accessible in a plain language under the name #DeepNightLearners.

DeepNightLearners

Supermasks in Superposition

The article suggests a brilliant training method, called SupSup, of a large neural network, which allows it to perform several different tasks.

Machine Learning

Various Machine-Learning and AI related articles: computer vision, natural language processing, deep learning model tuning and more.

Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration

Highly accurate protein structure prediction with AlphaFold

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

Keep-Current - April 2021 Summary

Convolutional Networks on Tabular data

Perceiver: General Perception with Iterative Attention

Discriminator Rejection Sampling

Sharpness-Aware Minimization for Efficiently Improving Generalization

Representation learning via invariant causal mechanisms

Keep-Current - March 2021 Edition

Rethinking Attention with Performers

A causal view of compositional zero-shot recognition

Supermasks in Superposition

Subscribe to Keep Current