I have seen more than one guy get FEB'd for non-flying related actions. It sounds like whatever you did to receive an LOR is bad enough they don't want you flying, and that's their prerogative. Really do some introspection here and determine if this is something worth spending a ton of money and stress on. It's very possible you really fucked up something in life/officership, and if that's the case, I'm not saying incriminate yourself further or not lawyer up, but also I wouldn't burn yourself out and drain your bank account to fight a losing battle. If this hypothetical is hitting close to home, then learn from the past, do what you need to improve, and move forward on a new path in life...flying is not everything, despite what we all think in our 20s.