Lecture 2 Short story of DL, OpenAI, and nVidia

2012 

Alexnet, big bang of AI

Toronto University

ImageNet with 14 m images

Error 25 down to 15 %

G4 GTX nvidia

CNN

Alex krizhevsky

Llya sutskever

Geoffrey Hinton

2013 

Alex, Geoffrey found DNN

Sold to Google

Used to power social media feed recommendations

YouTube

Instagram

Duopoly by Google and FB

2015 

openai

Elon musk

Llya (left Google)

Andrej karpathy …

Found OpenAI

Image supervised

Language embedded structure (word learned by joint attention while language by self teaching)

2017 

the transformer paper by Google

Attention is all you need.

Large amount of data needed

Computationally expensive n squared complexity

GPU came in

Parallel nature

OpenAI turned for profit

2018 

ChatGPT 1

Predicting the next word in a sentence

2019 

ChatGPT 2

$1 B from Microsoft

2020 ChatGPT 3

2021

Copilot

$2 B from Microsoft

2022 (FTX fell, triggering mass layoffs of big techs)

ChatGPT 3.5 (nov 30)

$10 B from Microsoft

2023 ChatGPT 4

Outlook

Data center, cloud computing

X86 Intel based CPU

80% of streaming video

GPU comes in, on chip RAM

Parallel computing, graphics, video

Data center as one computer

Questions

1. LLM = AI?

2. Do exams well, enough?

3. Infants, animals, nonverbal

Cognitive test