In older version(before 1+) of Spark there was different contexts that was entrypoints to the different api (sparkcontext for the core api, sql context for the spark-sql api, streaming context for the Dstream api etc...) this was source of confusion for the developer and was a point of optimization for the spark team, so in the most recent version of spark there is only one entrypoint (the spark session) and from this you can get the various other entrypoint (the spark context , the streaming context , etc ....)
Another difference with sparksession is, now different users can submit same applications with different configurations. even though its not advisable to run more than 1 session at a time, its one of the differences to be metioned.
No comments:
Post a Comment