The answer is yes... and no!
Contrary to what you might think, or what some coursebooks might say, you don’t need to sound like a native speaker to speak well. In fact, if you believe so, you're going to put extra pressure on yourself to speak perfectly, when all you really need to do, is communicate as clearly as possible.
I make a point not to focus on accents to begin with – and only where absolutely necessary – so that my students can focus instead on how to put sentences together. Pronunciation can be very distracting, so much so that it often leads students to forgetting entirely what they meant to say in the first place!
Besides, as I always remind those who feel self-conscious, accents are generally considered as cute by locals, so the pressure is definitely off!
What you need to do, however, is to pronounce key sounds correctly so you’re not misunderstood. For example, if a person learning English doesn't pronounce the consonant 'h':
'heat' becomes 'eat' and 'hour' becomes 'our'
And if they shorten sounds as in:
'paper' becomes 'pepper' and 'later' becomes 'letter'
They might still be understood, because of the context, but if they are, the process will neither be comfortable nor straightforward.
If an English speaker systematically pronounces consonants (such as B, C, D, F, G, etc.), which are normally mute in French (in brackets below):
tem(ps), beaucou(p), t(h)éâtre, (h)ôpital, com(p)ter, quan(d), etc.
native speakers will struggle to understand them, or might not understand them at all, even if they're using the right word in the right context.
Paying attention to accents means giving yourself as many opportunities as possible to be understood, so that you can grow into a confident speaker and make progress naturally and easily. It's a means to an end, rather than an end in itself.
Don't hesitate to send me your questions.