Search
  • Videos
  • 5G
  • Windows 10
  • Cloud
  • Innovation
  • Security
  • Tech Pro
  • more
    • Apple
    • ZDNet Academy
    • Microsoft
    • Mobility
    • Hardware
    • Executive Guides
    • Best VPN Services
    • See All Topics
    • White Papers
    • Downloads
    • Reviews
    • Galleries
    • Videos
    • TechRepublic Forums
  • Newsletters
  • All Writers
    • Log In to ZDNET
    • Join ZDNet
    • About ZDNet
    • Preferences
    • Community
    • Newsletters
    • Log Out
  • Menu
    • Videos
    • 5G
    • Windows 10
    • Cloud
    • Innovation
    • Security
    • Tech Pro
    • Apple
    • ZDNet Academy
    • Microsoft
    • Mobility
    • Hardware
    • Executive Guides
    • Best VPN Services
    • See All Topics
    • White Papers
    • Downloads
    • Reviews
    • Galleries
    • Videos
    • TechRepublic Forums
      • Log In to ZDNET
      • Join ZDNet
      • About ZDNet
      • Preferences
      • Community
      • Newsletters
      • Log Out
  • us
    • Asia
    • Australia
    • Europe
    • India
    • United Kingdom
    • United States
    • ZDNet around the globe:
    • ZDNet China
    • ZDNet France
    • ZDNet Germany
    • ZDNet Korea
    • ZDNet Japan

Pepperdata Code Analyzer for Apache Spark

1 of 4 NEXT PREV
  • sparkapptoolong.png

    Breaking down application execution in 5 phases (0-4) can give insights in spotting run time bottlenecks.


    In this case, looking at the default Spark UI (bottom) it looks as if phase 3 is the bottleneck, using up lots of CPU.

    Looking at Pepperdata's custom UI however (top), it becomes clear that phases 2 and 3 run in parallel, and while phase 3 completes phase 2 continues running and using up CPU.

    Published: June 1, 2017 -- 12:45 GMT (05:45 PDT)

    Photo by: Pepperdata

    Caption by: George Anadiotis

  • Inconsistent runtimes

    Inconsistent runtimes

    In 2 runs of the same application, run times are not consistent.

    Published: June 1, 2017 -- 12:45 GMT (05:45 PDT)

    Photo by: Pepperdata

    Caption by: George Anadiotis

  • Cluster weather

    Cluster weather

    This has to do with cluster weather: in the 2nd run there is less memory available to the available, as there are other applications running at the same time.

    Published: June 1, 2017 -- 12:45 GMT (05:45 PDT)

    Photo by: Pepperdata

    Caption by: George Anadiotis

  • Pepperdata product line

    Pepperdata product line

    Pepperdata Code Analyzer for Apache Spark is aimed at engineers and complements Pepperdata existing line of products

    Published: June 1, 2017 -- 12:45 GMT (05:45 PDT)

    Photo by: Pepperdata

    Caption by: George Anadiotis

1 of 4 NEXT PREV
  • sparkapptoolong.png
  • Inconsistent runtimes
  • Cluster weather
  • Pepperdata product line

What can Pepperdata Code Analyzer for Apache Spark do and where does it sit in Pepperdata's product line?

Read More Read Less

Breaking down application execution in 5 phases (0-4) can give insights in spotting run time bottlenecks.


In this case, looking at the default Spark UI (bottom) it looks as if phase 3 is the bottleneck, using up lots of CPU.

Looking at Pepperdata's custom UI however (top), it becomes clear that phases 2 and 3 run in parallel, and while phase 3 completes phase 2 continues running and using up CPU.

Published: June 1, 2017 -- 12:45 GMT (05:45 PDT)

Caption by: George Anadiotis

1 of 4 NEXT PREV

Related Topics:

Big Data Analytics Innovation Digital Transformation Robotics Internet of Things Enterprise Software
LOG IN TO COMMENT
  • My Profile
  • Log Out
| Community Guidelines

Join Discussion

Add Your Comment
Add Your Comment

Related Galleries

  • 1 of 3
  • When chatbots are a very bad idea

    Not every business problem can be solved by using chatbots. Here are some inappropriate uses for the AI tool.

  • How ubiquitous AI will permeate everything we do without our knowledge.

    Most of us do not know that we are using chatbots to talk to service agents, so how will we know that AI will be seamlessly interacting in with our future lives? ...

  • Streaming becomes mainstream

    The endless streams of data generated by applications lends its name to this paradigm, but also brings some hard to deal with requirements to the table: How do you deal with querying ...

  • Photos: How FC Barcelona uses football player data to win games

    FC Barcelona is focusing on data analysis to give it an edge on the soccer field and at the bank.

  • Heart and sleep apps that work with the Apple Watch

    If you want to track sleep and heart health, these apps will get you going.

  • Azure HDInsight click-by-click guide: Get cloud-based Hadoop up and running today

    Click by click, we'll show you how to get Microsoft's Apache Hadoop-based big bata service up and running.

  • Hands-on with Azure Data Lake: How to get productive fast

    Microsoft's Azure Data Lake is now generally available, but what does it do, and how does it work? Here's a tour around the service's tooling and capabilities, to help you understand ...

ZDNet
Connect with us

© 2019 CBS Interactive. All rights reserved. Privacy Policy | Cookies | Ad Choice | Advertise | Terms of Use | Mobile User Agreement

  • Topics
  • All Authors
  • Galleries
  • Videos
  • Sponsored Narratives
  • About ZDNet
  • Meet The Team
  • Site Map
  • RSS Feeds
  • Reprint Policy
  • Manage | Log Out
  • Join | Log In | Membership
  • Newsletters
  • Site Assistance
  • ZDNet Academy
  • TechRepublic Forums