Is it a bad idea to let your phone keep charging all the time?

For the first part of your question, see Ideal charging / discharging percentage for maximum battery life?. Answers there explain in detail, so yes, it is true

Second part, after how long... depends on how often you charge it the wrong way, your usage (how many cycles of charge) etc and one cannot put a figure to it. Bear in mind that batteries even without being used lose a significant amount of life due to aging. So, it's a bit complicated to separate cause and effect. I don't think it will show up soon but maybe about an year or so (no backing to substantiate)

See answer here for more details Battery capacity wear-down, and its relationship to charging practices