Fasten the excluded file check (#604)

Before we build a potentially very big list of all the excluded file matching
every exclude pattern, which could be a lot and then do a containment check on
the resulting list.

A first step to speed up the process is to returns a set instead of a list so
containment checks will not be dependent to the number of matching files
anymore.

If the set still ends up being too large, introducing an intermediary solution
that check if a file name match a glob pattern instead of expanding the glob
pattern might be a good solution. Right now, it seems overkill in comparison
of this patch.

With this patch, building my project that exclude `nodes_modules/**/*` went
from 4 minutes to 7 seconds.

Fix #603