A Guide to Listing Files and Directories with (Py)Spark, or How To Summon the Beast

Different methods for traversing file-systems with PySpark. From the mundane to the occult.
A collection of 2 posts
Different methods for traversing file-systems with PySpark. From the mundane to the occult.
It can be difficult to plan your next development step, or know who to hire. How do you build and recognize the good from the mediocre experience? Consider polyglots: People that can program in multiple languages.