/>
X

Pepperdata Code Analyzer for Apache Spark

What can Pepperdata Code Analyzer for Apache Spark do and where does it sit in Pepperdata's product line?

|
georgios-anadiotis-author.jpg
|
Topic: Big Data
sparkapptoolong.png
1 of 4 Pepperdata

Breaking down application execution in 5 phases (0-4) can give insights in spotting run time bottlenecks.


In this case, looking at the default Spark UI (bottom) it looks as if phase 3 is the bottleneck, using up lots of CPU.

Looking at Pepperdata's custom UI however (top), it becomes clear that phases 2 and 3 run in parallel, and while phase 3 completes phase 2 continues running and using up CPU.

sparkruntimes1.png
2 of 4 Pepperdata

Inconsistent runtimes

In 2 runs of the same application, run times are not consistent.

sparkruntimes2.png
3 of 4 Pepperdata

Cluster weather

This has to do with cluster weather: in the 2nd run there is less memory available to the available, as there are other applications running at the same time.

pepperdataproducst.png
4 of 4 Pepperdata

Pepperdata product line

Pepperdata Code Analyzer for Apache Spark is aimed at engineers and complements Pepperdata existing line of products

Related Galleries

Azure Synapse Analytics data lake features: up close
azure-ml-integration-via-notebook.png

Related Galleries

Azure Synapse Analytics data lake features: up close

19 Photos
Pitfalls to Avoid when Interpreting Machine Learning Models
1-bad-model-generalization.jpg

Related Galleries

Pitfalls to Avoid when Interpreting Machine Learning Models

8 Photos
When chatbots are a very bad idea
When Chatbots are a very bad idea ZDNet

Related Galleries

When chatbots are a very bad idea

6 Photos
How ubiquitous AI will permeate everything we do without our knowledge.
How ubiquitous AI will permeate everything we do without our knowledge ZDNet

Related Galleries

How ubiquitous AI will permeate everything we do without our knowledge.

6 Photos
Streaming becomes mainstream
streaming1.png

Related Galleries

Streaming becomes mainstream

3 Photos
Photos: How FC Barcelona uses football player data to win games
barca4.jpg

Related Galleries

Photos: How FC Barcelona uses football player data to win games

8 Photos
Heart and sleep apps that work with the Apple Watch
heart-graph.jpg

Related Galleries

Heart and sleep apps that work with the Apple Watch

7 Photos