Simplify extras handling (#6372)

Another random bit of code-tidying:

- remove some dead code that constructs an unused dictionary
- simplify (considerably) the code that walks the dependency tree looking for packages introduced by extras