• sparkapptoolong.png

    Breaking down application execution in 5 phases (0-4) can give insights in spotting run time bottlenecks.


    In this case, looking at the default Spark UI (bottom) it looks as if phase 3 is the bottleneck, using up lots of CPU.

    Looking at Pepperdata's custom UI however (top), it becomes clear that phases 2 and 3 run in parallel, and while phase 3 completes phase 2 continues running and using up CPU.

    Photo by: Pepperdata

    Caption by: George Anadiotis

  • Inconsistent runtimes

    Inconsistent runtimes

    In 2 runs of the same application, run times are not consistent.

    Photo by: Pepperdata

    Caption by: George Anadiotis

  • Cluster weather

    Cluster weather

    This has to do with cluster weather: in the 2nd run there is less memory available to the available, as there are other applications running at the same time.

    Photo by: Pepperdata

    Caption by: George Anadiotis

  • Pepperdata product line

    Pepperdata product line

    Pepperdata Code Analyzer for Apache Spark is aimed at engineers and complements Pepperdata existing line of products

    Photo by: Pepperdata

    Caption by: George Anadiotis

1 of 4

Pepperdata Code Analyzer for Apache Spark

What can Pepperdata Code Analyzer for Apache Spark do and where does it sit in Pepperdata's product line?

Read More Read Less

Breaking down application execution in 5 phases (0-4) can give insights in spotting run time bottlenecks.


In this case, looking at the default Spark UI (bottom) it looks as if phase 3 is the bottleneck, using up lots of CPU.

Looking at Pepperdata's custom UI however (top), it becomes clear that phases 2 and 3 run in parallel, and while phase 3 completes phase 2 continues running and using up CPU.

Caption by: George Anadiotis

Related Topics:

Big Data Analytics Innovation Digital Transformation Robotics Internet of Things Enterprise Software
Add Your Comment
Add Your Comment

Related Galleries