This web site uses cookies to improve your experience. By viewing our content, you are accepting the use of cookies. To find out more and change your cookie settings, please view our cookie policy.

Search
  • Videos
  • Smart Cities
  • Windows 10
  • Cloud
  • Innovation
  • Security
  • Tech Pro
  • more
    • ZDNet Academy
    • Microsoft
    • Mobility
    • IoT
    • Hardware
    • Executive Guides
    • Best VPN Services
    • See All Topics
    • White Papers
    • Downloads
    • Reviews
    • Galleries
    • Videos
  • Newsletters
  • All Writers
    • Log In to ZDNET
    • Join ZDNet
    • About ZDNet
    • Preferences
    • Community
    • Newsletters
    • Log Out
  • Menu
    • Videos
    • Smart Cities
    • Windows 10
    • Cloud
    • Innovation
    • Security
    • Tech Pro
    • ZDNet Academy
    • Microsoft
    • Mobility
    • IoT
    • Hardware
    • Executive Guides
    • Best VPN Services
    • See All Topics
    • White Papers
    • Downloads
    • Reviews
    • Galleries
    • Videos
      • Log In to ZDNET
      • Join ZDNet
      • About ZDNet
      • Preferences
      • Community
      • Newsletters
      • Log Out
  • us
    • Asia
    • Australia
    • Europe
    • India
    • United Kingdom
    • United States
    • ZDNet around the globe:
    • ZDNet China
    • ZDNet France
    • ZDNet Germany
    • ZDNet Korea
    • ZDNet Japan

Pepperdata Code Analyzer for Apache Spark

1 of 4 NEXT PREV
  • sparkapptoolong.png

    Breaking down application execution in 5 phases (0-4) can give insights in spotting run time bottlenecks.


    In this case, looking at the default Spark UI (bottom) it looks as if phase 3 is the bottleneck, using up lots of CPU.

    Looking at Pepperdata's custom UI however (top), it becomes clear that phases 2 and 3 run in parallel, and while phase 3 completes phase 2 continues running and using up CPU.

    Published: June 1, 2017 -- 12:45 GMT (05:45 PDT)

    Photo by: Pepperdata

    Caption by: George Anadiotis

  • Inconsistent runtimes

    Inconsistent runtimes

    In 2 runs of the same application, run times are not consistent.

    Published: June 1, 2017 -- 12:45 GMT (05:45 PDT)

    Photo by: Pepperdata

    Caption by: George Anadiotis

  • Cluster weather

    Cluster weather

    This has to do with cluster weather: in the 2nd run there is less memory available to the available, as there are other applications running at the same time.

    Published: June 1, 2017 -- 12:45 GMT (05:45 PDT)

    Photo by: Pepperdata

    Caption by: George Anadiotis

  • Pepperdata product line

    Pepperdata product line

    Pepperdata Code Analyzer for Apache Spark is aimed at engineers and complements Pepperdata existing line of products

    Published: June 1, 2017 -- 12:45 GMT (05:45 PDT)

    Photo by: Pepperdata

    Caption by: George Anadiotis

1 of 4 NEXT PREV
  • 0
  • sparkapptoolong.png
  • Inconsistent runtimes
  • Cluster weather
  • Pepperdata product line

What can Pepperdata Code Analyzer for Apache Spark do and where does it sit in Pepperdata's product line?

Read More Read Less

Breaking down application execution in 5 phases (0-4) can give insights in spotting run time bottlenecks.


In this case, looking at the default Spark UI (bottom) it looks as if phase 3 is the bottleneck, using up lots of CPU.

Looking at Pepperdata's custom UI however (top), it becomes clear that phases 2 and 3 run in parallel, and while phase 3 completes phase 2 continues running and using up CPU.

Published: June 1, 2017 -- 12:45 GMT (05:45 PDT)

Caption by: George Anadiotis

Related Topics:

Big Data Analytics Innovation Digital Transformation Robotics Internet of Things Enterprise Software
  • 0
LOG IN TO COMMENT
  • My Profile
  • Log Out
| Community Guidelines

Join Discussion

Add Your Comment
Add Your Comment

Related Galleries

  • How ubiquitous AI will permeate everything we do without our knowledge.

    Artificial Intelligence

    How ubiquitous AI will permeate everything we do without our knowledge.

  • Streaming becomes mainstream

    Data Management

    Streaming becomes mainstream

  • Photos: How FC Barcelona uses football player data to win games

    Big Data Analytics

    Photos: How FC Barcelona uses football player data to win games

  • Heart and sleep apps that work with the Apple Watch

    Innovation

    Heart and sleep apps that work with the Apple Watch

ZDNet
Connect with us

© 2018 CBS Interactive. All rights reserved. Privacy Policy | Cookies | Ad Choice | Advertise | Terms of Use | Mobile User Agreement

  • Topics
  • All Authors
  • Galleries
  • Videos
  • Sponsored Narratives
  • About ZDNet
  • Meet The Team
  • Site Map
  • RSS Feeds
  • Reprint Policy
  • Manage | Log Out
  • Log In to ZDNET | Join ZDNet
  • Membership
  • Newsletters
  • Site Assistance
  • ZDNet Academy