/>
X

Alpine Data Spark Auto Tuning

Alpine Data's approach to automating Spark

|
georgios-anadiotis-author.jpg
|
Topic: Big Data
sparkconfig.png
1 of 6 Alpine Data

Spark Configuration

Configuring a Spark cluster is not the easiest thing in the world, even if you are a data scientist

biintegration.png
2 of 6 Alpine Data

Look ma, no data scientist

Alpine Labs wants to support scenarios in which there is no data scientist involved

automl.png
3 of 6 Alpine Data

​Machine Learning to the rescue

Alpine Labs uses ML under the hood.

sparkoverhead.png
4 of 6 Alpine Data

Spark Overheads

Spark has its nuances, and these need to be taken into account when deploying jobs and tuning clusters

sparkarchitecture.png
5 of 6 Alpine Data

Spark Architecture

But can data scientists really know everything about Spark?

sparkchallenges.png
6 of 6 Alpine Data

Spark cluster tuning challenges

Finding a good tuning solution for Spark clusters is not trivial.

Related Galleries

Azure Synapse Analytics data lake features: up close
azure-ml-integration-via-notebook.png

Related Galleries

Azure Synapse Analytics data lake features: up close

Pitfalls to Avoid when Interpreting Machine Learning Models
1-bad-model-generalization.jpg

Related Galleries

Pitfalls to Avoid when Interpreting Machine Learning Models

When chatbots are a very bad idea
When Chatbots are a very bad idea ZDNet

Related Galleries

When chatbots are a very bad idea

How ubiquitous AI will permeate everything we do without our knowledge.
How ubiquitous AI will permeate everything we do without our knowledge ZDNet

Related Galleries

How ubiquitous AI will permeate everything we do without our knowledge.

Streaming becomes mainstream
streaming1.png

Related Galleries

Streaming becomes mainstream

Photos: How FC Barcelona uses football player data to win games
barca4.jpg

Related Galleries

Photos: How FC Barcelona uses football player data to win games

Heart and sleep apps that work with the Apple Watch
heart-graph.jpg

Related Galleries

Heart and sleep apps that work with the Apple Watch