Moving to France/company in Canada?

My family is planning to move to France from Canada in the near future and start a business. We all hold British passports as well as canadian citizenship. We own an incorporated company in Canada which provides telemarketing services to a number of clients exclusively in Canada. We operate two offices in Canada which are managed by two on-site managers and the staff they hire. My wife is the sole shareholder and I oversee the operations and client relations as well as numerous administrative tasks.

Apart from client meetings, and visits to our locations, pretty much all the work I do is done from home using the internet and phone. My wife and I pay ourselves a management fee for the work we do and of course my wife can take a dividend from the company. Once we move to France, I can pretty much do what I currently do over the internet and phone, plus periodic visits to Canada.

I was under the impression that once moved to France I could pretty much continue to do what I do remotely over the internet and phone plus periodic visits to Canada, understanding that we would be liable for tax in France on the management fees and dividends. However, I am not sure if this is true or not. Some people have suggested that I would need to set up some kind of entity in France in order to do this legally even though none of the company income is generated in France because the actual money making part of it is conducted exclusively in Canada. Could you please advise?