Hosted in Toronto Flag of Canada

Apache Spark Quick Start Guide: Quickly learn the art of writing efficient big data applications with Apache Spark

My highlights and comments

'//Scala
val sales_df = spark.read.option("sep", "\t").option("header", "true").csv("file:///opt/data/sales/sample_10000.txt")
' ('Data Sources' section)

I loaded up data into a Spark dataset, reading several CSV files that I had in a folder:

val sourceDS = spark.read
.option("header", "true")
.option("inferSchema", "true")
.csv("data/*.csv")
.as[DataSource]


Created: 2020-10-05 13:17:11 Last updated: 2020-10-05 13:17:11

Top