Flexible Fair Metric Learning