If your stylus is digging grot out, you did not get it clean.

When you clean it with the edger, are you applying any pressure? multiple passes or just once around and rinse?

if your microfiber isnt clean and exclusively for this purpose, you might just be putting stuff back on

EDIT: Also, get something like this