Naive Bayes

Models the probability of observing data $x$ and that the class of our target is $c$. Generative model that aims to maximize the joint likelihood $\sum _n\log p(x_n,y_n)$.

$P(X=x,Y=c)={\pi }_c\prod _k{\theta }_{ck}^{z_k},X\in {\mathbb{R}}^D,Y\in [C]$

• Joint probability of getting your data

$({\pi }_c^{\ast },{\theta }_{ck}^{\ast })=\text{argmax}\sum _n\log {\pi }_{y_n}+\sum _{n,k}{z}_{nk}\log {\theta }_{y_{n}k}$

${\pi }_C^{\ast }=\frac{\text{\# of data points labeled C}}{N}$

${\theta }_{ck}^{\ast }=\frac{\text{\# of times word k shows up in data points labeled c}}{\text{\# words in data points labeled c}}$