Key FeaturesA quick way to get started with Spark – and reap the rewardsFrom analytics to engineering your big data architecture, weve got it coveredBring your Scala and Java knowledge – and put it to work on new and exciting problemsBook DescriptionWhen people want a way to process big data at speed, Spark is invariably the solution. With its ease of development (in comparison to the relative complexity of Hadoop), its unsurprising that its becoming popular with data analysts and engineers everywhere.Beginning with the fundamentals, well show you how to get set up with Spark with minimum fuss. Youll then get to grips with some simple APIs before investigating machine learning and graph processing – throughout well make sure you know exactly how to apply your knowledge.You will also learn how to use the Spark shell, how to load data before finding out how to build and run your own Spark applications. Discover how to manipulate your RDD and get stuck into a range of DataFrame APIs. As if thats not enough, youll also learn some useful Machine Learning algorithms with the help of Spark MLlib and integrating Spark with R. Well also make sure youre confident and prepared for graph processing, as you learn more about the GraphX API.What you will learnInstall and set up Spark in your clusterPrototype distributed applications with Sparks interactive shellPerform data wrangling using the new DataFrame APIsGet to know the different ways to interact with Sparks distributed representation of data (RDDs)Query Spark with a SQL-like query syntaxSee how Spark works with big dataImplement machine learning systems with highly scalable algorithmsUse R, the popular statistical language, to work with SparkApply interesting graph algorithms and graph processing with GraphXAbout the AuthorKrishna Sankar is a Senior Specialist—AI Data Scientist with Volvo Cars focusing on Autonomous Vehicles. His earlier stints include Chief Data Scientist at http://cadenttech.tv/, Principal Architect/Data Scientist at Tata America Intl. Corp., Director of Data Science at a bioinformatics startup, and as a Distinguished Engineer at Cisco. He has been speaking at various conferences including ML tutorials at Strata SJC and London 2016, Spark Summit [goo.gl/ab30lD], Strata-Spark Camp, OSCON, PyCon, and PyData, writes about Robots Rules of Order [goo.gl/5yyRv6], Big Data Analytics—Best of the Worst [goo.gl/ImWCaz], predicting NFL, Spark [http://goo.gl/E4kqMD], Data Science [http://goo.gl/9pyJMH], Machine Learning [http://goo.gl/SXF53n], Social Media Analysis [http://goo.gl/D9YpVQ] as well as has been a guest lecturer at the Naval Postgraduate School. His occasional blogs can be found at https://doubleclix.wordpress.com/. His other passion is flying drones (working towards Drone Pilot License (FAA UAS Pilot) and Lego Robotics—you will find him at the St.Louis FLL World Competition as Robots Design Judge.Table of ContentsInstalling Spark and Setting Up Your ClusterUsing the Spark ShellBuilding and Running a Spark ApplicationCreating a SparkSession ObjectLoading and Saving Data in SparkManipulating Your RDDSpark 2.0 ConceptsSpark SQLFoundations of Datasets/DataFrames – The Proverbial Workhorse for DataScientistsSpark with Big DataMachine Learning with Spark ML PipelinesGraphX