How can it not impact your job, seriously?
Being an MP is arguably the most important role in the country. You're representing tens of thousands of constituents and providing leadership and governance for the entire country.
If you've got time for a second job then you're not taking it seriously enough.
There are also absolutely obvious questions about undue influence either directly from your employer or the general area that provides you with another income (e.g. finance, energy, media).
Politely, I think we disagree on this one mate.