Posts

From OLTP & OLAP to HTAP and LTAP: The Evolution Toward a Lakehouse-Native Data Architecture

-
Image
  I am still second-reading Designing Data-Intensive Applications, and recently came across HTAP. It immediately made me think about hashtag #Databricks announcement of hashtag #LTAP and how it fits into the evolution from databases to the Lakehouse. The story starts with two very different workloads: - OLTP, Online Transaction Processing, runs operational applications. Handling workloads require low latency, high concurrency and reliable transactions (recording payments...). - OLAP, Online Analytical Processing, helps organizations understand what is happening across large volumes of data (reporting, ML...). Traditionally, these workloads were separated. Operational data lived in transactional databases and was later "copied" into analytical systems. 🔴 hashtag #HTAP , Hybrid Transactional/Analytical Processing, then emerged to bring these two worlds closer together. HTAP is not one specific implementation. The common objective is to support transactional and analytical pr...
Post a Comment
See more »

Databricks Game Changer

-
Image
  Data is growing. AI expectations are accelerating. Yet many organizations still manage data engineering, analytics, governance and AI through disconnected tools. This is where Databricks changes the game. 🚀 Built on lakehouse architecture, Databricks brings data engineering, analytics, business intelligence, machine learning and GenAI into one unified platform. What makes it stand out? 🔹 Unified workspace for data and AI teams 🔹 Open and reliable data foundation with Delta Lake 🔹 Centralized data and AI governance through Unity Catalog 🔹 Scalable analytics with Databricks SQL 🔹 End-to-end ML and GenAI capabilities 🔹 Consistent experience across AWS, Azure and GCP For data leaders, the real value is simplification: fewer silos, stronger governance, better collaboration and a faster path from raw data to business intelligence. Databricks is not simply about processing more data—it is about turning trusted data into meaningful outcomes at scale. Which Databricks capability cr...
Post a Comment
See more »

Cloudera Data Hub: Where Agility Meets Control

-
Image
Cloudera’s new Data Hub cloud service, powered by  Cloudera Data Platform , enables users to seamlessly migrate on-premises data management and analytics workloads to the cloud as well as implement new cloud workloads in pursuit of your cloud-first data management strategy. On August 22 nd ,  Cloudera demonstrated its Data Hub  service during a webinar highlighting key business benefits, use cases, and product capabilities. Below is a brief overview of the topics covered and some of the most frequently asked questions from attendees. What is Cloudera Data Hub? Cloudera Data Hub  is a powerful cloud service on Cloudera Data Platform (CDP) that makes it easier, safer, and faster to build modern, mission-critical, data-driven applications with enterprise security, governance, scale, and control. The cloud-native service is powered by a suite of integrated open source technologies that delivers the widest range of analytical workloads such as data marts and data...
Post a Comment
See more »

HBase Performance CDH5 (HBase1) vs CDH6 (HBase2)

-
Image
Thanks to Cloudera Blog HBase Customers upgrading to CDH 6 from CDH 5, will also get an HBase upgrade moving from HBase1 to HBase2. Performance is an important aspect customers consider.   We measured performance of CDH 5 HBase1 vs CDH 6 HBase2 using YCSB workloads to understand the performance implications of the upgrade on customers doing in-place upgrades (no changes to hardware).  About YCSB For our testing we used the  Yahoo! Cloud Serving Benchmark  (YCSB). YCSB is an open-source specification and program suite for evaluating retrieval and maintenance capabilities of computer programs. It is often used to compare relative performance of  NoSQL  database management systems. The original benchmark was developed by workers in the research division of  Yahoo!  who released it in 2010.  More info on YCSB at  https://github.com/brianfrankcooper/YCSB In our test environment YCSB @1TB data scale was used, and run workloads in...
Post a Comment
See more »

How-to: Use Parquet with Impala, Hive, Pig, and MapReduce

-
Source: Cloudera Blog The CDH software stack lets you use your tool of choice with the Parquet file format – – offering the benefits of columnar storage at each phase of data processing.  An open source project co-founded by Twitter and Cloudera,  Parquet  was designed from the ground up as a state-of-the-art, general-purpose, columnar file format for the Apache Hadoop ecosystem. In particular, Parquet has several features that make it highly suited to use with  Cloudera Impala  for data warehouse-style operations: Columnar storage layout:  A query can examine and perform calculations on all values for a column while reading only a small fraction of the data from a data file or table. Flexible compression options:  The data can be compressed with any of several codecs. Different data files can be compressed differently. The compression is transparent to applications that read the data files. Innovative encoding schemes:  Sequences of ide...
Post a Comment
See more »

Big Data Trendz

Loading...