Use lens cleaning solution and Q-Tips, but do not "scrub" with the Q-Tip. Use a "rolling action" with Q-Tip to avoid grinding any dirt into the surface. That's the technique we use to clean front-surface mirrors and coated lens in the optical paths of laser engravers/cutters. The rolling action will lift the crud off the filter, but you may have use two or more Q-Tips depending on what you have on the filter.
Andy.