Tabular Foundation Models (TFMs) logopeterindia.net
Knowledge Hub

Data Lakes

Data lakes: cloud and self-hosted object storage, file formats, query engines, governance and reference reading.

42 topics•7 categories•Updated October 2026

What Is a Data Lake?

7 entries

A data lake stores large volumes of raw data in its native format, usually on low-cost object storage, so it can be analyzed later with different engines. Lakes hold structured, semi-structured and unstructured data, which makes governance, cataloguing and open file and table formats important to keep them usable. Descriptions below follow each vendor's or project's own pages and are not rankings; release facts come from GitHub.

01Data Lakehouse PlatformsLakehouse platforms, on this site. 02Data Lakehouse TechnologiesOpen table formats, catalogs and engines, on this site. 03Cloud Data WarehousesCloud warehouse platforms, on this site. 04Object Storage SolutionsObject storage products, on this site. 05ETL and ELT ToolsLoading and transformation tools, on this site. 06Change Data Capture (CDC) PlatformsCDC into lakes, on this site. 07Data EngineeringData engineering hub, on this site.

Cloud Data Lake Storage and Services

9 entries

Managed storage and services commonly used as the base of a data lake.

08Amazon S3AWS describes cloud object storage with scalability, availability, security and performance suited to data lakes, mobile apps and backups. 09AWS Lake FormationAWS describes making it easier to centrally govern, secure and share data for analytics and machine learning. 10Amazon AthenaAWS describes a serverless, interactive analytics service for analyzing data where it lives. 11Azure Data Lake StorageMicrosoft documents Azure Data Lake Storage capabilities for big data analytics on Blob Storage. 12Google Cloud StorageGoogle describes storing data with multiple redundancy options. 13Google Knowledge Catalog (formerly Dataplex)Google describes an always-on context engine that unifies structured, unstructured and SaaS data; the old Dataplex address redirects here. 14Microsoft PurviewMicrosoft describes integrated solutions to govern, manage and secure data. 15Cloudflare R2Cloudflare describes S3-compatible object storage with zero egress fees and R2 Data Catalog for analytics; the pricing claim is the vendor's own. 16Backblaze B2Backblaze describes capacity object storage for AI and data-intensive workloads with predictable costs.

Self-Hosted Object Stores and File Systems

7 entries

Open-source storage layers for on-premises and private lakes.

17MinIOS3-compatible object store whose community repository was archived on 25 April 2026 and says it is no longer maintained; MinIO points to AIStor Free and AIStor Enterprise as its current editions. 18CephOpen-source distributed storage platform providing object, block and file storage; the ceph.io site returned a server error to automated checks, so the card links to the documentation. 19SeaweedFSDistributed storage system for object storage (S3), file systems and Iceberg tables; GitHub lists 4.51 (25 October 2026) as the latest release. 20GarageLightweight S3-compatible object store that its project says can run outside data centers. 21Apache OzoneScalable, reliable distributed storage system optimized for analytics and object store workloads; GitHub lists ozone-2.2.2 (30 October 2026) as the latest release. 22JuiceFSDistributed file system built on object storage, separating data and metadata; GitHub lists v1.4.2 (22 October 2026) as the latest release. 23Apache Hadoop (HDFS)The original data lake platform and distributed file system, which object storage replaced in many stacks.

File Formats

4 entries

Open columnar and row formats used for data in lakes.

24Apache ParquetOpen source, column-oriented data file format for efficient storage and retrieval; GitHub lists apache-parquet-1.18.1 (13 September 2026) for the Java implementation. 25Apache ORCColumnar file format for Hadoop workloads; GitHub lists v2.3.1 (16 July 2026) as the latest release. 26Apache AvroRow-oriented data serialization system with rich schemas; GitHub lists release-1.12.2 (23 August 2026) as the latest release. 27Apache ArrowUniversal columnar format and multi-language toolbox for fast data interchange and in-memory analytics.

Query Engines and Table Formats

6 entries

Engines that query lake data and the table formats that add ACID semantics.

28TrinoDistributed SQL query engine for big data; GitHub lists 484 (16 October 2026) as the latest release. 29Apache SparkMulti-language engine for data engineering, data science and machine learning; the Spark news page lists 4.3.0 as released on 30 October 2026. 30DuckDBSQL OLAP database that can query Parquet in object storage; PyPI lists duckdb 2.0.0 (21 October 2026). 31StarburstStarburst describes querying across sources with data federation, Trino and Apache Iceberg while limiting data copies. 32DremioOpen lakehouse platform on Apache Iceberg; SAP completed its acquisition of Dremio on 6 July 2026, and the site returned 403 to automated checks. 33Apache Iceberg, Delta Lake and Apache HudiOpen table formats that turn data lakes into lakehouses; see the Data Lakehouse Technologies page for versions and comparisons.

Governance and Data Management

3 entries

Catalogs, versioning and platforms for managing lakes.

34Unity Catalog (Databricks)Databricks describes unified governance for data, models, agents and apps across any cloud. 35lakeFSGit-like version control for data lakes; GitHub lists v1.88.0 (1 October 2026) as the latest release. 36Cloudera Data PlatformCloudera describes a data and AI platform for managing data wherever it lives.

Guides and Reference Reading

6 entries

Explainers on data lakes and architecture.

37Data Lake (Martin Fowler)Martin Fowler's note on the data lake idea and its risks. 38What is a data lake? (IBM)IBM describes a repository for large volumes of raw data stored as-is, typically on low-cost cloud object storage. 39What is a data lake? (AWS)AWS explainer on how and why businesses use data lakes. 40What is a data lake? (Google Cloud)Google Cloud explainer on scalable repositories for raw, full-fidelity data. 41Data lakes (Azure Architecture Center)Microsoft guide to the advantages of data lake storage repositories. 42Big Data ComputingHadoop, Spark, Flink, Kafka and big data platforms.

Frequently Asked Questions

What is a data lake?

A repository that stores large volumes of raw data as-is, usually on low-cost object storage such as Amazon S3 or Azure Data Lake Storage, so it can be processed later by different engines.

What is the difference between a data lake, a data warehouse and a lakehouse?

A lake stores raw data cheaply but lacks transactions and strong governance, a warehouse stores modeled data for fast SQL analytics, and a lakehouse adds table formats such as Iceberg or Delta Lake on top of lake storage to give warehouse-like features. See the Data Lakehouse and Cloud Data Warehouses pages.

What is a data swamp?

A data lake that has become unusable because data is undocumented, duplicated or ungoverned. Catalogs, metadata management, quality checks and clear ownership help avoid it.

Which file format should I use in a data lake?

Parquet is the common default for analytics because it is columnar and widely supported. ORC is common in Hadoop environments, and Avro suits row-based and streaming data. Table formats such as Iceberg or Delta Lake sit on top of Parquet.

What happened to MinIO?

The MinIO community repository was archived on 25 April 2026 and says it is no longer maintained. MinIO points to AIStor Free and AIStor Enterprise as its editions, and alternatives such as Ceph, SeaweedFS, Garage and Apache Ozone are listed on this page.

How current is this page?

Links and release facts were checked on 31 October 2026 from GitHub, PyPI and the Apache Spark news page. The Dremio site returned 403 and the ceph.io site returned a server error to automated checks. Vendor descriptions follow their own pages and are not rankings.