This is one of my least favorite weaknesses in the game.
Not that Accursed Follower is particularly tough. On the contrary; he has basically zero immediate impact, which is a very good thing in a weakness. It will take a good amount of time for the Curses to have any noticeable impact on the scenario; if you draw him later on there's every chance that the scenario will end before the Curses swing a single test. Occasionally he can have vicious interactions with Mysterious Chanting or something like that, but that's not too common. Plus there are a wide variety player cards that benefit from his Curses.
No, what I dislike about this card are the severe memory issues it causes. This guy spawns as far away as possible on the map and never moves or really does anything consequential. It's like he's designed for you to forget about him. And so almost unfailingly, at some point in the scenario someone in my group will end up saying, "wait, we forgot to put a Curse in the bag last turn. Hmm, how many turns have we forgotten about him?" Honestly, I can barely remember Sister Mary's investigator ability when I play her--and the designers expect me to remember this guy adding a Curse at the end of the enemy phase on the other side of map? Not happening.