X

Flink 1.2 new features

Flink is moving forward in release 1.2, with an array of new features.
By George Anadiotis, Contributor
dynamic-scaling.png
1 of 7 George Anadiotis/ZDNET

Dynamic scaling

Changing the number of machines allocated to a job while running requires being able to split or merge state to redistribute it to a smaller or larger number of machines. Much of the work on dynamic scaling focused on the internal representation of state. It's now possible to adapt the scale of running jobs while guaranteeing state consistency. Image: Data Artisans

queryable-state.png
2 of 7 George Anadiotis/ZDNET

Queryable state

"Turning the stream processor into a database."It means that key-lookup requests are sent directly to Flink (rather than a key-value store or database), and Flink answers directly from its internal operator state. This is significant because 1) it means the results from a Flink computation are immediately accessible without first being sent to a storage layer and 2) the storage layer becomes unnecessary for certain use cases. Image: Data Artisans

asynchronous-io.png
3 of 7 George Anadiotis/ZDNET

Asynchronous I/O

Improvements in the way Flink interacts with external data stores, for instance, when enriching stream events with data stored in a database. Image: Apache Flink

flink-metrics.png
4 of 7 George Anadiotis/ZDNET

Metrics

In Flink 1.1, the community released Flink's first metrics system for monitoring and debugging. Metrics are now exposed in Flink's web interface. Image: Apache Flink

security-kerberos.jpg
5 of 7 George Anadiotis/ZDNET

Security

Kerberos authentication for YARN, HDFS, Zookeeper, and Kafka, and state backend security. Image: Kerberos Project/MIT

table-api-example.png
6 of 7 George Anadiotis/ZDNET

Table API & StreamSQL

The Table API is a declarative, embedded API to define relational queries on dynamic and static tables. Table API & SQL major improvements: 1) streaming group windows on the Table API, 2) support for user-defined scalar and table-generating functions, 3) much improved coverage of built-in functions, data types, and relational operators. Image: Data Artisans

cluster-management.png
7 of 7 George Anadiotis/ZDNET

Cluster management

Flink can be deployed in a variety of environments, from YARN to Mesos to Kubernetes. Flink 1.2 includes resource manager improvements and makes it straightforward to add support for new tools in the future. Image: Apache Flink/Mesos/Hadoop

Related Galleries

Holiday wallpaper for your phone: Christmas, Hanukkah, New Year's, and winter scenes
Holiday lights in Central Park background

Related Galleries

Holiday wallpaper for your phone: Christmas, Hanukkah, New Year's, and winter scenes

21 Photos
Winter backgrounds for your next virtual meeting
Wooden lodge in pine forest with heavy snow reflection on Lake O'hara at Yoho national park

Related Galleries

Winter backgrounds for your next virtual meeting

21 Photos
Holiday backgrounds for Zoom: Christmas cheer, New Year's Eve, Hanukkah and winter scenes
3D Rendering Christmas interior

Related Galleries

Holiday backgrounds for Zoom: Christmas cheer, New Year's Eve, Hanukkah and winter scenes

21 Photos
Hyundai Ioniq 5 and Kia EV6: Electric vehicle extravaganza
img-8825

Related Galleries

Hyundai Ioniq 5 and Kia EV6: Electric vehicle extravaganza

26 Photos
A weekend with Google's Chrome OS Flex
img-9792-2

Related Galleries

A weekend with Google's Chrome OS Flex

22 Photos
Cybersecurity flaws, customer experiences, smartphone losses, and more: ZDNet's research roundup
shutterstock-1024665187.jpg

Related Galleries

Cybersecurity flaws, customer experiences, smartphone losses, and more: ZDNet's research roundup

8 Photos
Inside a fake $20 '16TB external M.2 SSD'
Full of promises!

Related Galleries

Inside a fake $20 '16TB external M.2 SSD'

8 Photos