Schema Madness, handling (incompatible) schema changes in incoming json files with Databricks

Databricks has a schema evolution feature which can automatically handle schema changes. This seems amazing in theory, but there are quite a bit of practical gotcha's I've run into. This is a notes-from-the-field demo rich session where I show you what problems I ran into and how to fix them.