Keep Current

Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

Data can be represented as a vector in a continuous space, but often it is concentrated in a narrow area. Can it be improved?

DeepNightLearners

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

This paper suggests a new computationally efficient method for constructing low-dimensional representation of unlabeled data.

Monthly Updates

Keep-Current - April 2021 Summary

Self-Supervised Learning keynote talk by Yann LeCunn; Perceiver - computation efficient transformer algorithm; Data Visualization also for summaries and more news in this monthly update.

Computer Vision

Perceiver: General Perception with Iterative Attention

To overcome Transformers' squared complexity (w.r.t input length), the Perceiver article here offers a novel method to learn the QKV matrices. Check it out!

DeepNightLearners

Rethinking Attention with Performers

The article suggests a method to lower the Transformer's complexity to a linear order and proves all the arguments also in a rigorous form. The article is not easy to read, but luckily, to understand the main idea, the first 5-6 pages are more than enough.

Natural Language Processing (NLP)

Natural Language Processing (NLP) has significantly advanced in the last few years. It became hard to keep track with all the changes and technologies in the field. These selected articles will help you to keep-current with NLP.

Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

Keep-Current - April 2021 Summary

Perceiver: General Perception with Iterative Attention

Rethinking Attention with Performers

Subscribe to Keep Current