EE4016

AI Blogs

• Google: Google is known for its constant innovation, and their AI Blog is a great source to stay updated on their latest advancements. They explain their ideas in a professional manner. You can also find academic papers focused on AI in the Google Research Publication Database.
• DeepMind: Although a part of Google, DeepMind operates somewhat independently and has made significant breakthroughs in AI, such as AlphaGo and AlphaFold. Their Research Blog and publications are excellent sources to stay informed about the latest developments in AI.
• Meta: Meta also publishes many AI-related papers, which can be found on their publications page. Their blog is more similar to reading Wired articles compared to Google's AI Blog.
• Microsoft Research: Microsoft has made strides in various areas, including AI research and capabilities. They have a significant number of high-level AI publications. They also have two blogs, the AI Blog and Research Blog, which have a similar feel to reading Wired articles.
• SAIL (Stanford AI Lab): Stanford University is a renowned academic institution, and their SAIL blog is a valuable resource. With notable figures like Andrew Ng and Yoav Shoham associated with it, the blog is worth checking frequently.
• MIT: Massachusetts Institute of Technology (MIT) is another well-known academic institute. They have two AI-related resources: MIT News on Artificial Intelligence and MIT Technology Review on Artificial Intelligence. These publications provide a different reading experience, somewhat resembling an AI-focused branch of TechCrunch. They are recommended for a contrasting perspective compared to the other blogs mentioned above.

Language Model Papers

1. A Mathematical Theory of Communication
2. A Neural Probabilistic Language Model
3. NLP (Almost) from scratch
4. Phoneme Recognition using Time Delay Neural Networks
5. Efficient Estimation of words in vector space (Word2Vec: CBOW and Skip-gram)
6. GloVe: Global Vectors for Word Representations
7. Enriching word vectors with subword information (FastText)
8. A Convolution Neural Network for modeling sentences
9. Learning Internal Representations by error propagation
10. Sequence Modeling (from the deep learning book)
11. Long Short Term Memory (LSTM)
12. Colah's blog to understanding LSTM
13. Training Recurrent Neural Networks (PhD Thesis)
14. Deep contextualized word representations (ELMo)
15. Attention is all you need (Transformer)
16. Bidirectional Encoder Representations from Tranformers (BERT)
17. Improving language understanding by generative pretraining (GPT-1)
18. Language Models are multi task learner (GPT-2)
19. Language models are few shot learners (GPT-3)
20. Sentence BERT
21. ChatGPT Blog
22. LlaMA-2

Python 3