An incomplete list of recommendations.
Do a Noise Audit .
Get multiple, independent (in a vacuum) predictors.
Actively seek information that would disprove your thinking.
Decide on how to decide. If there will be repeated decisions, make sure the judgement process is standardized.
Keep the dimensions separately rated and as orthogonal as possible.