Introducing U-SQL: Big Data Processing made easy

Making Big Data processing easy requires great developer support that hides the complexity of managing scale, provides easy integration of custom code to handle the complex processing requirements ranging from data cleanup to advanced processing of unstructured data, and provides great tool support for the developer to help in the iterative development process. Thus when we at Microsoft introduced the Azure Data Lake, we decided to also include a new language called U-SQL to make Big data processing easy. It unifies the declarative power of SQL and the extensibility of C# to make writing custom processing of big data easy. It also unifies processing over all data – structured, semistructured and unstructured data – and queries over both local data and remote SQL data sources. This presentation will give you an overview on U-SQL, why we decided to build a new language, what its core philosophical underpinnings are as well as show the language in its natural habitat – the development tooling – showing the language capabilities as well as the tool support from starting your first script to analyzing its performance.