Fix up FileProcessor.file_tokens property

We opted to not copy the file_tokens attribute each time it's accessed
in the merge request discussion but it was never reflected in the code.
Further, the attribute had no documentation or docstring, so we've added
that. Finally, we address a personal style nit that I otherwise wouldn't
have picked at.