Sweeping Changes oleh Mara Lynn Johnstone

When the old street sweeper's memory deserts him, this smacks of magic, not a senior moment -- he's left with his broom and a reflection that talks back in a snarky voice. But when he sets out to learn more, he finds himself up against foreign invaders who take a dim view of his interference. One officer decides he’s in the way (and making her look bad), so she sends assassins after him directly. His fighting skills with the trusty broom won’t be enough to save him. But his memories are starting to trickle back, thanks in part to clues from that talking reflection. It turns out he was someone important. Someone with magic of his own. Someone who might just be able to stop the invasion, if he can keep himself alive and regain the rest of those memories in time.