Correct Answer:

$\frac{a^{2}-1}{a^{2}-b^{2}}$

Solution : Given: $\frac{\cos\alpha}{\cos\beta}=a$ and $\frac{\sin\alpha}{\sin\beta}=b$
⇒ $\cos\alpha=a\cos\beta$------------------------(1)
⇒ $\sin\alpha=b\sin\beta$-------------------------(2)
Squaring and adding the equations (1) and (2), we have,
$\cos^{2}\alpha+\sin^{2}\alpha=a^{2}\cos^{2}\beta+b^{2}\sin^{2}\beta$
⇒ $1=a^{2}\cos^{2}\beta+b^{2}\sin^{2}\beta$
⇒ $1=a^{2}(1-\sin^{2}\beta)+b^{2}\sin^{2}\beta$
⇒ $1=a^{2}-a^{2}\sin^{2}\beta+b^{2}\sin^{2}\beta$
⇒ $a^{2}-1=\sin^{2}\beta(a^{2}-b^{2})$
$\therefore\sin^{2}\beta=\frac{a^{2}-1}{a^{2}-b^{2}}$
Hence, the correct answer is $\frac{a^{2}-1}{a^{2}-b^{2}}$.