Since Samsung debuted the first Galaxy Note in 2011, the company had a working group, now 10 years old, solely dedicated to improving the S Pen stylus experience, said Sally Jeong, vice president of Samsung's Android Framework R&D, who leads the development team for the S Pen, multi-tasking, Edge SDK, and Android platforms.
"Every year, we ask, how can we improve the S Pen experience?" she said in an interview with ZDNet.
"S Pen users are clearly divided to those who don't use it at all, and those who use it a lot. And those who use it a lot always give us a ton of feedback on what they were satisfied with, what they want improved, and what they want us to add."
Samsung changed its head of smartphone business to Tae Moon Roh earlier this year in its annual reshuffle. Since taking the mantle, Roh has stressed the importance of going "back to the basics" and focusing on fundamentals, said Jeong.
"At the end of the day, customer experience is the most important. So instead of doing everything ourselves, we wanted to focus on what we do well and integrate what our partners such as Microsoft do well," she said.
"Our approach towards the S Pen and Samsung Notes has been consistency, convenience, and intuitiveness. So this time around, for our part, we really wanted to up our game in the fundamental stylus and note-taking experience with the Galaxy Note 20 Ultra and Galaxy Note 20, and I think we succeeded in offering the most complete S pen experience to date."
"Increasing the speed of the S Pen was one of our most aggressive goals for the new Note," Jeong said. "On the platform-level, we drastically reduced the latency overhead in Android for the new Samsung Note. This allows the coordinate of the S Pen to be recorded on the frame level, allowing the app to be used immediately."
The team also analysed a billion coordinates through machine learning to allow the screen to predict where the stylus will go next. Speed was further enhanced thanks to the 120Hz refresh rate being put into the Galaxy Note 20 Ultra.
"The combination of these three upgrades makes the latest S Pen the most seamless we have offered so far," the VP said.
Samsung also wanted to provide what the customer needed right away, hence the addition of PDF Annotation.
"Note users no longer need to install a separate note-taking app. With the native PDF Annotation, they can also sync to the cloud right away," Jeong said.
See also: Galaxy Note 20 hands-on: Samsung at its strongest
The new Anywhere Actions -- where users can make certain gestures with the S Pen while holding it in the air for certain commands -- was also added to make using the device faster and more intuitive. Five gestures are provided by default and users can customise them to their liking.
"Since we introduced Bluetooth on the S Pen with the Note 9, we have received a lot of positive feedback from fans that gesture control was very convenient to use and that they wanted us to add more.
"Gesture control options for the camera on the Note 10 was especially popular. So we always think about what gestures can be used for certain apps like the camera and what gestures can be used more generally. Continuing along this line, we added Anywhere Actions on the Note 20 so that customers have more options to cater for how they use their device further."
Since being showcased at Unpacked, the new Auto Straighten feature has been one of the most well-received by fans, the VP added. Most Note users write down their notes slightly tilted, and the feature allows them to straighten the text horizontally afterwards.
"We sampled 4 million handwriting samples for the feature," she said.
Jeong also touted the new Audio Bookmark feature where users can record audio and take notes simultaneously. Users can playback the audio when they return to a particular text that they scribbled at the same time of recording.
"It supports up to six hours of audio recording, so there will be no limitation in use," she said.
Samsung talked up its collaboration with Microsoft during its virtual Unpacked event earlier this month, as well as its new tablets: The Galaxy Tab S7 and Tab S7 Plus. Note apps will be soon added to Microsoft's Your Phone, allowing users further control over their smartphones, tablets and PCs on one screen.
"We really want to actively use our partnership with Microsoft that started last year to boost the productivity of the Note series," Jeong said.
"Notes taken on Samsung Notes can be 'exported' to PowerPoint right now. Within the year, notes taken on Samsung Notes will be synced to OneNote and Outlook so that texts written as notes on smartphones can be pasted onto the main texts of emails directly. Users will no longer need to retype what they wrote on their Note app on the PC."
Samsung has also seen its tablet sales increase since the start of COVID-19 pandemic, Jeong said, adding that the S Pen can be a great productivity booster when paired with tablets.
When it comes to the stylus, and productivity in general, the Samsung VP feels the company is the best in the Android camp due to its long and successful experience with the Note series.
"There was a lot of opposition to launching a large-screened device paired with a stylus when we launched the first Galaxy Note in 2011 from both our competitors and internally within the company," Jeong said.
"But this view changed in earnest with the launch of the Galaxy Note 2. We were the first to introduce multi-window features on Android devices, even before the Android operating system was ready to support such multi-tasking features. I would say we have taken a leading position in productivity in the Android space since then."
This focus on productivity is continuing in Samsung's development of foldable smartphones, the VP said, where the company is closely collaborating with Google.
"Years prior, we mostly tried to develop and differentiate our own UI and UX. Today, I would say there is a mutual trust between us and Google. We recognise our respective expertise," Jeong said.
On the absence of the S Pen on the Galaxy Z Fold 2, which launches next month, the VP said that Samsung is aware of the strong consumer demand to add a stylus to the foldable phone, but wanted to get the foldable experience down first.
This month, Samsung has opened the beta testing for its upcoming One UI 3.0 software based on Android 11. The official update will roll out by the end of the year, Jeong said, and more Galaxy smartphones will have access to the latest features introduced at Unpacked.
On the future of the Galaxy Note series, the VP stressed that the category offers a unique combination of portability and productivity over other devices.
"Consumers can expect us to continue offering the smartphone and stylus combo down the road. We feel the Note series will always have a role to play, especially when paired with Windows PCs."