In a recent case of a family member, it was the physician who broke the bad news that driving was no longer an option. If your FIL really is too feeble to be driving, his doctor will back you up. Schedule an appointment, and talk to the doctor in advance about your concerns. Let the doctor know the purpose of the appointment: to evaluate whether your FIL has the skills still needed to drive: reflexes, response times, flexibility, strength in his legs (i.e., to slam on the brakes), eyesight, etc. The doctor can do this kind of assessment without making it obvious, if you think that is necessary.
Sometimes hearing it from a non-family member, and from a professional, carries more impact. Your FIL still won't be happy about it, but at least he can blame the doctor for being the bad guy.
Good luck! I think this is one of hardest issues when dealing with an aging parent.
Grand Goddess of GPS of America - BCBW Chapter