Yes, you can apply in GATE after a M.Sc. in Mathematics , but perhaps you will have to prepare yourself for the GATE Exam in Respective Subject. As far as GATE Mathematics is concerned, most students go for a Doctoral Education in Pure Mathematics, Mathematics and Computing or Operations Research to higher technical institutes like Top IITs and IISc Bangalore. GATE Score is also accepted by certain Universities abroad like NUS and NTU in Singapore, TU Munich and RWTH Aachen which provide equally exciting and perhaps much better research and career prospects.

Valuing the current trend and Technical Universities like IITs, NITs, BITS Pilani as well as IIITs liberalising their admission policies, many students after a M.Sc in Mathematics also pursue a M.Tech or PhD in Computer Science, Artificial Intelligence, Machine Learning and other Novel research based subjects.

You must check the admission and eligibility brochure of the institutes you are targetting in detail to know about the PG and Doctoral courses, application procedure and eligibility requirements. Because many of the Institutes require a GATE Score in Computer Science as an essential requirement, while many CSE Departments have liberalised their admission criteria and select students on basis of their Past research work, projects, Research interest as well as in your case GATE Mathematics Score.

For more information about GATE Exam do follow the link below

https://engineering.careers360.com/articles/gate/amp