The default error stream always points to be displayed out to the console. You can create a new stream to be written out to for the error stream, but you can't use the redirection method in conjunction with this method (unless they are to be redirected to the same file).
// sets the error stream to redirect all error to a file called "errors.txt". Auto-flushing is enabled
System.setErr(new PrintStream(new FileOutputStream("errors.txt"),true);
System.err.println("Start of error logging in errors.txt");