EE5438 Applied Deep Learning

Useful Links

The best way to learn is to learn from the best.

AI Blogs

The following leading company's blogs are great resources to stay updated on the latest AI advancements from these leading companies.

Language Model Papers

22 research papers you should read to master Language modeling.

  1. A Mathematical Theory of Communication
    Foundational paper in information theory by Claude Shannon.
  2. A Neural Probabilistic Language Model
    Early neural language model by Bengio et al.
  3. NLP (Almost) from scratch
    Collobert et al.'s unified neural network approach to NLP.
  4. Phoneme Recognition using Time Delay Neural Networks
    Early TDNN application to speech recognition.
  5. Efficient Estimation of words in vector space (Word2Vec)
    Mikolov et al.'s Word2Vec: CBOW and Skip-gram models.
  6. GloVe: Global Vectors for Word Representations
    Global vector approach to word embeddings.
  7. Enriching word vectors with subword information (FastText)
    FastText paper on subword information for word representations.
  8. A Convolution Neural Network for modeling sentences
    CNN-based approach to sentence modeling.
  9. Learning Internal Representations by error propagation
    Classic PDP chapter on backpropagation.
  10. Sequence Modeling (from the deep learning book)
    Deep learning book chapter on sequence modeling.
  11. Long Short Term Memory (LSTM)
    Original LSTM paper by Hochreiter and Schmidhuber.
  12. Colah's blog to understanding LSTM
    Excellent visual explanation of LSTM networks.
  13. Training Recurrent Neural Networks (PhD Thesis)
    Sutskever's comprehensive PhD thesis on RNN training.
  14. Deep contextualized word representations (ELMo)
    ELMo paper on contextualized word representations.
  15. Attention is all you need (Transformer)
    Foundational Transformer paper.
  16. Bidirectional Encoder Representations from Transformers (BERT)
    BERT paper introducing bidirectional transformer representations.
  17. Improving language understanding by generative pretraining (GPT-1)
    Original GPT paper on generative pretraining.
  18. Language Models are multi task learner (GPT-2)
    GPT-2 paper on unsupervised multitask learning.
  19. Language models are few shot learners (GPT-3)
    GPT-3 paper on few-shot learning capabilities.
  20. Sentence BERT
    Sentence embeddings using Siamese BERT networks.
  21. ChatGPT Blog
    OpenAI's blog post on ChatGPT.
  22. LlaMA-2
    Meta's LLaMA-2 technical paper.

Python 3 & Development Tools

PyTorch

Mathematics

Department of Electrical Engineering, City University of Hong Kong

EE Department Logo