If you entirely jump the portions of physics then you might get lower ranks as compared to the others which may be dangerous for your dream to study in IISC. So I would recommend you if you have problems with Physics you should try to resolve it. Hypothetically, if you get a chance in IISC Bangalore without attempting Physics then also you will need the basic foundations of Physics in order to sustain your course work at IISC. So you should study physics and try to attempt as much as you can.

I hope you get my point. All the very best for your future endeavors!