Data Processing and Visualization
Do you want to process larger amounts of data and extract meaningful and valuable information from it?
Turn data into meaningful information
Data is omnipresent. The amount of available or collectable data is almost infinite. However, simply collecting and presenting data does not add value per se. Therefore, it is important to ask what constitutes meaningful and valuable information. Meaningful information allows a target user to make informed decisions or it triggers specific actions. Therefore, only data that is necessary to provide relevant information for the user should be collected, processed and visualized.
Prepare for customization and individualization
Customization and individualization capabilities are important on both sides of your data processing chain—the sourcing and the presentation side. The sourcing side of the chain must be able to handle semi structured, unstructured, sparse or incomplete data with non-uniform or evolving data formats and models. The presentation side of the chain must support varying information needs. This applies for APIs and User Interfaces. APIs should support individual queries and query parameters. User interfaces should be tailored to the specific needs of each user group and should display only what is relevant for a user.
Decouple data sourcing from data presentation
The pace and amount of events that either push or pull data varies usually a lot on the two sides of a data processing chain. Therefore at least two decoupled systems—so called micro services—are needed. One that covers data sourcing and one that covers data presentation. For sourcing, data can be generated at a constant frequency, in automatic or manual batches or on events triggered by user activities for example. On the presentation side, information can be displayed constantly, on request, or only be pushed to its recipient when it matters—for example a push notification or alert when something goes wrong. The amount of data sources might also be different from the amount of data consumers. A few sources can supply hundreds of thousands of consumers, or vice versa.
For their next product, a wearable system that measures and interprets brain signals from the ear canal, IDUN Technologies aims to implement a scalable, secure and versatile cloud data processing pipeline. EMBRIO.tech quickly and efficiently develops the minimum necessary functionalities, and also ensures future expandability of the solution for secure operations at scale. Within a few weeks, an event-based data processing pipeline based on state-of-the-art technology and cloud services is realized, which allows for the first customer applications to be implemented.
Continue reading