{"id":11105,"date":"2021-06-15T08:00:30","date_gmt":"2021-06-15T15:00:30","guid":{"rendered":"https:\/\/softwareengineeringdaily.com\/?p=11105"},"modified":"2021-06-15T14:11:46","modified_gmt":"2021-06-15T21:11:46","slug":"emerging-architectures-for-real-time-analytics-in-applications","status":"publish","type":"post","link":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/","title":{"rendered":"Emerging Architectures for Real-Time Analytics in Applications"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">There\u2019s been a proliferation of applications that are all about using data to derive actionable insights. Companies like<\/span><a href=\"https:\/\/rockset.com\/blog\/real-time-analytics-construction-logistics-command-alkon\/\"><span style=\"font-weight: 400;\"> Command Alkon<\/span><\/a> <span style=\"font-weight: 400;\">digitize construction logistics by providing a platform for suppliers, transportation providers and contractors to monitor and coordinate the shipment of construction materials. And, fintech startup<\/span><a href=\"https:\/\/rockset.com\/blog\/matter-uses-rockset-to-bring-ai-powered-sustainable-insights-to-investors\/\"> <span style=\"font-weight: 400;\">Matter<\/span><\/a><span style=\"font-weight: 400;\"> uses natural language processing to evaluate the sustainability of investments. In the last five years, the<\/span> <span style=\"font-weight: 400;\">number of these data applications has grown by 73%, representing more than 20% of SaaS applications (data from<\/span><a href=\"https:\/\/www.crunchbase.com\/\"> <span style=\"font-weight: 400;\">Crunchbase<\/span><\/a><span style=\"font-weight: 400;\">).\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">This wave of data applications incorporates predictive features, to automate decision-making, as well as drill-down features so users can reach their own conclusions. For example, security analytics applications notify users to potential threats but security teams still want to dig into the data to determine the risk to the organization. In many scenarios, including in security analytics applications, there\u2019s a symbiotic relationship between predictive capabilities and embedded search and analytics.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The challenge in delivering real-time analytics in applications is different than surfacing analytics to a team of analysts. For one, users don\u2019t want to wait seconds for in-app analytics to load. They want the same snappy, responsive experience across the application. User engagement wanes when users have to wait for queries to load. They run fewer queries, log in less frequently and don\u2019t take the desired actions in the product. So, slow analytics ends up reducing the value of the application to the organization.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The other big change is that users want to take immediate action on the freshest data. Industries like security, logistics, and advertising need to access the latest data to make operational decisions; data delays can result in security breaches, suboptimal routes and inefficient bidding. These aren\u2019t just periodic, strategic questions that are being asked. Many applications are using analytics for operational decision-making and that requires the latest data.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Getting data applications to meet user expectations requires a new type of data architecture: a data architecture designed for sub-second search and analytics across 1000s of concurrent users at scale. In this article, two emerging architectures are shared that support this new wave of real-time analytics in applications. One architecture that accelerates data warehouse performance to meet the sub-second query latency requirement of applications. And, the other, that indexes streaming data from event streaming platforms or database change data capture to support time-critical use cases.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Accelerate Data Warehouse Performance<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">Interactive analytics enables users to slice and dice the data to derive strategic insights. One goal of accelerating data warehouse performance is to shorten the query latency or the time from when a request is made and the result is returned to the application. To keep users engaged, you want that latency to be under a second.<\/span><\/p>\n<p><img fetchpriority=\"high\" decoding=\"async\" data-attachment-id=\"11106\" data-permalink=\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/rockset_loading\/\" data-orig-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?fit=1040%2C1336&amp;ssl=1\" data-orig-size=\"1040,1336\" data-comments-opened=\"0\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Rockset_loading\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?fit=234%2C300&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?fit=797%2C1024&amp;ssl=1\" class=\"aligncenter wp-image-11106\" src=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?resize=400%2C514\" alt=\"\" width=\"400\" height=\"514\" srcset=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?resize=234%2C300&amp;ssl=1 234w, https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?resize=797%2C1024&amp;ssl=1 797w, https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?resize=768%2C987&amp;ssl=1 768w, https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?w=1040&amp;ssl=1 1040w\" sizes=\"(max-width: 400px) 100vw, 400px\" data-recalc-dims=\"1\" \/><\/p>\n<p style=\"text-align: center;\"><i><span style=\"font-weight: 400;\">Waiting for analytics in applications to load is a less than desirable user experience.<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">For example, a music analytics application is used by record labels to uncover trends in the industry and monitor the success of artists and songs across channels. When users hop into the application, they want to dive into the data to quickly uncover insights. Waiting seconds for each query to load adds time to the decision-making process and isn\u2019t a desirable user experience.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Many engineering teams will look to build data applications on data warehouses as they excel at analytics. While data warehouses are excellent solutions for BI and data science, they were built for a world where analytical queries were executed manually by teams of analysts. Not for a new world where hundreds to thousands of users are executing queries from within an application.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">That\u2019s why it&#8217;s a best practice to<\/span><a href=\"https:\/\/rockset.com\/why-you-shouldnt-build-apps-on-data-warehouses\"> <span style=\"font-weight: 400;\">pair a data warehouse with Rockset<\/span><\/a><span style=\"font-weight: 400;\"> to accelerate the performance of analytics for applications. Rockset\u2019s indexing approach optimizes for millisecond-latency, high-concurrency queries in a cost-effective way.<\/span><\/p>\n<p><img decoding=\"async\" data-attachment-id=\"11107\" data-permalink=\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/rockset_dw\/\" data-orig-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DW.png?fit=1044%2C393&amp;ssl=1\" data-orig-size=\"1044,393\" data-comments-opened=\"0\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Rockset_DW\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DW.png?fit=300%2C113&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DW.png?fit=1024%2C385&amp;ssl=1\" class=\"aligncenter wp-image-11107\" src=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DW.png?resize=500%2C188\" alt=\"\" width=\"500\" height=\"188\" srcset=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DW.png?resize=300%2C113&amp;ssl=1 300w, https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DW.png?resize=1024%2C385&amp;ssl=1 1024w, https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DW.png?resize=768%2C289&amp;ssl=1 768w, https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DW.png?w=1044&amp;ssl=1 1044w\" sizes=\"(max-width: 500px) 100vw, 500px\" data-recalc-dims=\"1\" \/><\/p>\n<p style=\"text-align: center;\"><i><span style=\"font-weight: 400;\">In this architecture, Rockset is accelerating the performance of data warehouses for interactive, sub-second analytics.<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">There are four main components of this data architecture:<\/span><\/p>\n<p><b>Data Warehouse:<\/b><span style=\"font-weight: 400;\"> Data warehouses such as Amazon Redshift, Google BigQuery and Snowflake serve as a central repository for data in the enterprise for business intelligence, reporting and data science. Data warehouses are optimized for processing large-scale data but have not been designed to serve applications.<\/span><\/p>\n<p><b>dbt:<\/b><span style=\"font-weight: 400;\"> dbt allows teams to transform data in their warehouse using SQL. In dbt, SQL queries are models that can be represented as materialized views. These views can be ingested into Rockset and served to applications.<\/span><\/p>\n<p><b>Rockset: <\/b><span style=\"font-weight: 400;\">Rockset is a real-time indexing layer on Snowflake that accelerates query response times using an indexing approach. Rockset uses a<\/span><a href=\"https:\/\/rockset.com\/blog\/how-rocksets-converged-index-powers-real-time-analytics\/\"> <span style=\"font-weight: 400;\">Converged Index<\/span><span style=\"font-weight: 400;\">TM<\/span><\/a><span style=\"font-weight: 400;\">\u00a0 to index any data- structured, semi-structured, geo or time series data- in a columnar store, search index and row store for fast search and analytics. Rockset is designed for sub-second, high-concurrency analytics.\u00a0<\/span><\/p>\n<p><b>API<\/b><span style=\"font-weight: 400;\">: Creating a data API with Rockset is as simple as saving your SQL query as a REST endpoint and triggering it from your application.<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Real-Time Analytics for Data Streams and Operational Databases<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">In time-critical scenarios,<\/span><a href=\"https:\/\/rockset.com\/real-time-analytics-explained\/\"> <span style=\"font-weight: 400;\">real-time analytics<\/span><\/a> <span style=\"font-weight: 400;\">uses newly generated data to make predictions, ask questions, and automate decision-making in the application. Real-time analytics optimizes for both low query latency and low data latency, or the time from when data is generated to when it is queryable.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">For example,<\/span><a href=\"https:\/\/rockset.com\/blog\/case-study-rumbles-real-time-leaderboards-empower-users-to-lead-healthier\/\"><span style=\"font-weight: 400;\"> Rumble<\/span><\/a><span style=\"font-weight: 400;\"> is an application that enables users to earn money and redeem rewards by adopting and maintaining healthy habits. To nudge individuals towards desired behavior, such as exercising on a regular basis, Rumble counts steps and creates leaderboards so users can track activity relative to their peers. Users of the application want to see in real time how their steps help move them up or down the leaderboard. In this scenario, data freshness is key.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Real-time analytics relies on real-time data. Real-time data can either be streamed from a transactional database using change data capture or from applications or devices using an event streaming platform. Rockset ingests and indexes the data to serve real-time analytics in applications.\u00a0<\/span><\/p>\n<p><img decoding=\"async\" data-attachment-id=\"11108\" data-permalink=\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/rockset_ds\/\" data-orig-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DS.png?fit=1276%2C589&amp;ssl=1\" data-orig-size=\"1276,589\" data-comments-opened=\"0\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Rockset_DS\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DS.png?fit=300%2C138&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DS.png?fit=1024%2C473&amp;ssl=1\" class=\"aligncenter wp-image-11108\" src=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DS.png?resize=500%2C231\" alt=\"\" width=\"500\" height=\"231\" srcset=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DS.png?resize=300%2C138&amp;ssl=1 300w, https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DS.png?resize=1024%2C473&amp;ssl=1 1024w, https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DS.png?resize=768%2C355&amp;ssl=1 768w, https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_DS.png?w=1276&amp;ssl=1 1276w\" sizes=\"(max-width: 500px) 100vw, 500px\" data-recalc-dims=\"1\" \/><\/p>\n<p style=\"text-align: center;\"><i><span style=\"font-weight: 400;\">In this architecture, Rockset is indexing real-time data from data streams and operational databases to serve real-time analytics.<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">There are four main components of this data architecture:<\/span><\/p>\n<p><b>OLTP Databases:<\/b><span style=\"font-weight: 400;\"> Operational databases such as<\/span><a href=\"https:\/\/rockset.com\/solutions\/mongodb\/\"> <span style=\"font-weight: 400;\">MongoDB<\/span><\/a><span style=\"font-weight: 400;\">,<\/span><a href=\"https:\/\/rockset.com\/sql-on-dynamodb\/\"> <span style=\"font-weight: 400;\">Amazon DynamoDB<\/span><\/a><span style=\"font-weight: 400;\">,<\/span><a href=\"https:\/\/rockset.com\/solutions\/postgresql\"> <span style=\"font-weight: 400;\">PostgreSQL<\/span><\/a><span style=\"font-weight: 400;\"> and<\/span><a href=\"https:\/\/rockset.com\/solutions\/mysql\"> <span style=\"font-weight: 400;\">MySQL<\/span><\/a><span style=\"font-weight: 400;\"> are excellent at processing transactions but have not been designed for complex analytical workloads at scale. Engineering teams will often choose to move analytics to a separate serving layer for complex search, aggregations and joins at scale.<\/span><\/p>\n<p><b>Data Streams<\/b><span style=\"font-weight: 400;\">: Application and device data is streamed through event streaming platforms such as Amazon Kinesis and<\/span><a href=\"https:\/\/rockset.com\/sql-on-kafka\/\"> <span style=\"font-weight: 400;\">Kafka<\/span><\/a><span style=\"font-weight: 400;\"> to data sinks, like Rockset, to power downstream applications.<\/span><\/p>\n<p><b>Rockset<\/b><span style=\"font-weight: 400;\">: Rockset continuously syncs data from OLTP databases and streams so new data is queryable within seconds of being generated. Rocket indexes the data in real time to support millisecond-latency SQL queries.<\/span><\/p>\n<p><b>API<\/b><span style=\"font-weight: 400;\">: Create an API and query Rockset directly from your application using<\/span><a href=\"https:\/\/docs.rockset.com\/query-lambdas\/#gatsby-focus-wrapper\"> <span style=\"font-weight: 400;\">Query Lambdas<\/span><\/a><span style=\"font-weight: 400;\">,\u00a0 which are named parameterized SQL queries stored in Rockset and executed from a REST endpoint.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Integrating Real-Time Analytics into Applications<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">Many SaaS applications look at active users as one of the golden measures of product health. If a user goes into an application and takes an action on a recurrent basis, the product is sticky. More and more, data is the key ingredient helping users take desired actions in products.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Keeping users engaged in products also requires a seamless, snappy experience. That means analytics loads incredibly fast, recommendations are made on the latest data and that the application experience does not change as usage grows.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The two emerging architectures for real-time analytics shared in this article have been designed to achieve the low query and data latency demanded of applications in a scalable way.\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/srv.buysellads.com\/ads\/long\/x\/T6LZ2LZTTTTTTTDE5IFNTTTTTTT5WCO5KTTTTTTTYTDKLYTTTTTTTJDDQJZN62DD5E76C2PEK2LNKBP2V2DNWWDGQMNE\"><em>Learn more about Rockset here.<\/em><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>There\u2019s been a proliferation of applications that are all about using data to derive actionable insights. Companies like Command Alkon digitize construction logistics by providing a platform for suppliers, transportation providers and contractors to monitor and coordinate the shipment of construction materials. And, fintech startup Matter uses natural language processing to evaluate the sustainability of<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"Emerging Architectures for Real-Time Analytics in Applications @RocksetCloud","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[1363,83,2143],"tags":[425,4774,1957,1888,2617],"class_list":["post-11105","post","type-post","status-publish","format-standard","hentry","category-all-episodes","category-articles","category-exclusive-content","tag-apis","tag-data-architecture","tag-data-streams","tag-data-warehouses","tag-rockset"],"jetpack_publicize_connections":[],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Emerging Architectures for Real-Time Analytics in Applications - Software Engineering Daily<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Emerging Architectures for Real-Time Analytics in Applications - Software Engineering Daily\" \/>\n<meta property=\"og:description\" content=\"There\u2019s been a proliferation of applications that are all about using data to derive actionable insights. Companies like Command Alkon digitize construction logistics by providing a platform for suppliers, transportation providers and contractors to monitor and coordinate the shipment of construction materials. And, fintech startup Matter uses natural language processing to evaluate the sustainability of\" \/>\n<meta property=\"og:url\" content=\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/\" \/>\n<meta property=\"og:site_name\" content=\"Software Engineering Daily\" \/>\n<meta property=\"article:published_time\" content=\"2021-06-15T15:00:30+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-06-15T21:11:46+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading-234x300.png\" \/>\n<meta name=\"author\" content=\"SE Daily\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@software_daily\" \/>\n<meta name=\"twitter:site\" content=\"@software_daily\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"SE Daily\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/\"},\"author\":{\"name\":\"SE Daily\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/822f06fe7d6f895baba29a9c0a3aa6c8\"},\"headline\":\"Emerging Architectures for Real-Time Analytics in Applications\",\"datePublished\":\"2021-06-15T15:00:30+00:00\",\"dateModified\":\"2021-06-15T21:11:46+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/\"},\"wordCount\":1281,\"publisher\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading-234x300.png\",\"keywords\":[\"APIs\",\"data architecture\",\"data streams\",\"data warehouses\",\"Rockset\"],\"articleSection\":[\"All Content\",\"Exclusive Articles\",\"Exclusive Content\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/\",\"url\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/\",\"name\":\"Emerging Architectures for Real-Time Analytics in Applications - Software Engineering Daily\",\"isPartOf\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading-234x300.png\",\"datePublished\":\"2021-06-15T15:00:30+00:00\",\"dateModified\":\"2021-06-15T21:11:46+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#primaryimage\",\"url\":\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?fit=1040%2C1336&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?fit=1040%2C1336&ssl=1\",\"width\":1040,\"height\":1336},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/softwareengineeringdaily.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Emerging Architectures for Real-Time Analytics in Applications\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#website\",\"url\":\"https:\/\/softwareengineeringdaily.com\/\",\"name\":\"Software Engineering Daily\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/softwareengineeringdaily.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#organization\",\"name\":\"Software Engineering Daily\",\"url\":\"https:\/\/softwareengineeringdaily.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2024\/01\/cropped-sed_website_banner.png?fit=549%2C169&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2024\/01\/cropped-sed_website_banner.png?fit=549%2C169&ssl=1\",\"width\":549,\"height\":169,\"caption\":\"Software Engineering Daily\"},\"image\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/software_daily\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/822f06fe7d6f895baba29a9c0a3aa6c8\",\"name\":\"SE Daily\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/87cddb92997841dafa04aa675bbdaca4?s=96&d=retro&r=pg\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/87cddb92997841dafa04aa675bbdaca4?s=96&d=retro&r=pg\",\"caption\":\"SE Daily\"},\"description\":\"The SE Daily podcast.\",\"sameAs\":[\"https:\/\/softwareengineeringdaily.com\"],\"url\":\"https:\/\/softwareengineeringdaily.com\/author\/erikawho\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Emerging Architectures for Real-Time Analytics in Applications - Software Engineering Daily","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/","og_locale":"en_US","og_type":"article","og_title":"Emerging Architectures for Real-Time Analytics in Applications - Software Engineering Daily","og_description":"There\u2019s been a proliferation of applications that are all about using data to derive actionable insights. Companies like Command Alkon digitize construction logistics by providing a platform for suppliers, transportation providers and contractors to monitor and coordinate the shipment of construction materials. And, fintech startup Matter uses natural language processing to evaluate the sustainability of","og_url":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/","og_site_name":"Software Engineering Daily","article_published_time":"2021-06-15T15:00:30+00:00","article_modified_time":"2021-06-15T21:11:46+00:00","og_image":[{"url":"http:\/\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading-234x300.png"}],"author":"SE Daily","twitter_card":"summary_large_image","twitter_creator":"@software_daily","twitter_site":"@software_daily","twitter_misc":{"Written by":"SE Daily","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#article","isPartOf":{"@id":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/"},"author":{"name":"SE Daily","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/822f06fe7d6f895baba29a9c0a3aa6c8"},"headline":"Emerging Architectures for Real-Time Analytics in Applications","datePublished":"2021-06-15T15:00:30+00:00","dateModified":"2021-06-15T21:11:46+00:00","mainEntityOfPage":{"@id":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/"},"wordCount":1281,"publisher":{"@id":"https:\/\/softwareengineeringdaily.com\/#organization"},"image":{"@id":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#primaryimage"},"thumbnailUrl":"http:\/\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading-234x300.png","keywords":["APIs","data architecture","data streams","data warehouses","Rockset"],"articleSection":["All Content","Exclusive Articles","Exclusive Content"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/","url":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/","name":"Emerging Architectures for Real-Time Analytics in Applications - Software Engineering Daily","isPartOf":{"@id":"https:\/\/softwareengineeringdaily.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#primaryimage"},"image":{"@id":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#primaryimage"},"thumbnailUrl":"http:\/\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading-234x300.png","datePublished":"2021-06-15T15:00:30+00:00","dateModified":"2021-06-15T21:11:46+00:00","breadcrumb":{"@id":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#primaryimage","url":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?fit=1040%2C1336&ssl=1","contentUrl":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/06\/Rockset_loading.png?fit=1040%2C1336&ssl=1","width":1040,"height":1336},{"@type":"BreadcrumbList","@id":"https:\/\/softwareengineeringdaily.com\/2021\/06\/15\/emerging-architectures-for-real-time-analytics-in-applications\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/softwareengineeringdaily.com\/"},{"@type":"ListItem","position":2,"name":"Emerging Architectures for Real-Time Analytics in Applications"}]},{"@type":"WebSite","@id":"https:\/\/softwareengineeringdaily.com\/#website","url":"https:\/\/softwareengineeringdaily.com\/","name":"Software Engineering Daily","description":"","publisher":{"@id":"https:\/\/softwareengineeringdaily.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/softwareengineeringdaily.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/softwareengineeringdaily.com\/#organization","name":"Software Engineering Daily","url":"https:\/\/softwareengineeringdaily.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/","url":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2024\/01\/cropped-sed_website_banner.png?fit=549%2C169&ssl=1","contentUrl":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2024\/01\/cropped-sed_website_banner.png?fit=549%2C169&ssl=1","width":549,"height":169,"caption":"Software Engineering Daily"},"image":{"@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/software_daily"]},{"@type":"Person","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/822f06fe7d6f895baba29a9c0a3aa6c8","name":"SE Daily","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/87cddb92997841dafa04aa675bbdaca4?s=96&d=retro&r=pg","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/87cddb92997841dafa04aa675bbdaca4?s=96&d=retro&r=pg","caption":"SE Daily"},"description":"The SE Daily podcast.","sameAs":["https:\/\/softwareengineeringdaily.com"],"url":"https:\/\/softwareengineeringdaily.com\/author\/erikawho\/"}]}},"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p7GuoD-2T7","_links":{"self":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/posts\/11105"}],"collection":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/comments?post=11105"}],"version-history":[{"count":0,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/posts\/11105\/revisions"}],"wp:attachment":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/media?parent=11105"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/categories?post=11105"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/tags?post=11105"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}