If a judge has a conflict of interest on a submission or is unable to complete their scoring, you can remove them from it.
Go into the submission(s) the judge is assigned to, find their name on the judge name list and click the Remove hyperlink next to their name. When you return to the submission dashboard list, you'll see that they will no longer appear underneath it.
You can also assign a new judge by clicking the Assign button within the submission and selecting another name from the list.