Microsoft Patent app describes multimedia note-taking for Windows Mobile devices
In a newly published Microsoft Patent application entitled User experience for mobile multimedia notetaking, a way is described that would enable users who are watching video on a Windows Mobile-enabled handset to take notes about that video at the same time.
The core concept of this application is briefly addressed in this Patent application's Abstract. The Abstract describes:
A software-based mechanism for taking multimedia notes while using a mobile computing device is provided. The interface is stream-based so as to enable complete control with a text or numeric keypad, a directional pad, and command button.
A note document containing textual data and objects representing other data types may be closed and revisited after it has been created.
It may be either read or edited in a single session. Changes may be manually or automatically saved to the note document for ease of user experience.
I've read through this entire Patent Application. I think Figure 4, and its accompanying documentation, best describes what is being depicted here.
Now referring to FIG. 4, a block diagram illustrating a software environment according to one embodiment is shown. Several scenarios may be described to illustrate the advantages of an application that can provide a unified environment for different data types in a mobile computing device.
For example, a user may capture images and/or audio recordings during a meeting and combine those into a single document with textual annotations. According to another scenario, a task list may be generated using images combined with ink entries that were made earlier with annotations for each entry.
In both scenarios, the user may desire to modify the unified document repeatedly, for example updating the task list as tasks are being accomplished.
These scenarios are not intended to be limiting; rather, they are intended to illustrate the flexibility of a multimedia note taking application in handling different data types and information obtained from the software environment of the mobile computing device.
According to embodiments, application program 302 is configured to generate a document (also called "note" herein) that includes textual data along with objects that are aligned with the textual data.
The textual data may be rich text, allowing formatting of the text, creation of bulleted or numbered lists, insertion of hyperlinks, and the like. Aligning the objects with the text allows users to handle the note even on a mobile computing device that does not include touch screen capability.
The objects are placeholders for different types of data captured or received by the mobile computing device. According to one embodiment, following data types may be combined in a document in a unified manner:
Images (from either the device's on-board camera or from an image file)
- Audio (recorded from the device's microphone or from an audio file)
- Video (from either the device's on-board camera or from a video file)
- Textual annotations
- Lists
- Tables
- Ink entries
Application program 402 can communicate with operating system 464 through an application program interface (API) 406. Application program 402 can make calls to methods of API 406 to request OS 464 to activate applications specific to each data type.
For example, an audio player program may be activated by the OS 464 when called by application program 402. Furthermore, OS 464 may communicate with application program 402 to provide data from other applications such as video stream, ink entry, and the like. In alternative embodiments, the application program 402 communicates directly with OS 464.
Application program 402 also communicates with a user through OS 464, input/output control module 410 and input/output devices 412 and 414. Input devices 412 can include an on-board camera, a microphone, an inking canvas, and the like, such as described above.
In this embodiment, application program 402 receives input signals to generate respective objects and insert them into the note providing the unified environment.
The data associated with each object, as well as the note itself, may be stored by application program 402 in memory system 462 through OS 464 and through a memory control module 406.