Skip to main content

What Is a Data Lake? A Plain-Language Guide

21 min read

Cover showing many data sources feeding cheap storage that query tools read, the parts of a data lake

Key takeaways

A data lake is one central place where you dump data in its original form, whether it's a table, a log file, an image or a PDF. You decide how to structure it later, when someone needs to read it. That's what makes a lake cheap and flexible.

It won't clean or organise anything for you, so without rules and upkeep it turns into a "data swamp".

  • A data lake keeps data in its original form. AWS, IBM, Google Cloud and Microsoft all define it as a central repository for structured and unstructured data that isn't reshaped before it's stored.
  • The big difference from a warehouse is timing. A lake applies structure when you read the data. A warehouse checks structure before it stores the data, according to Microsoft.
  • Cheap doesn't mean tidy. IBM warns that without a management layer a lake can decay into a data swamp, where good data is hard to find or trust.
  • File format changes the bill. In AWS's own Athena pricing example, the same question costs $15 on plain text and $1.25 on compressed Parquet (checked Oct 2026).
  • Filling a lake from a database needs an ingestion tool. OLake Go, our open-source tool (Apache 2.0), copies PostgreSQL, MySQL and MariaDB, MongoDB and SQL Server into Apache Iceberg or Parquet.
  • Lakes haven't gone anywhere. Microsoft says it is common for large organisations to run both a lake and a warehouse. IBM says the low-cost flexibility of lakes continues to prove advantageous. IBM sells lake and lakehouse software, so read that as its view.

Picture a company with a sales database, a pile of website logs, thousands of product photos and a folder of PDF contracts. A data lake is a single store where all of that can sit side by side, as is. Analysts, data scientists and machine learning (ML) programs then pick out what they need.

Every fact below comes from a named page, read on October 1, 2026, with the link next to the claim. Where a source gives no number, we say so. We read vendor documents and a research paper, and we did not build or test a data lake or the other products named here.

What is a data lake?​

A data lake is a central store that holds large amounts of data in its original, raw form. You don't have to decide how the data will be used before you save it.

Amazon Web Services (AWS) defines it as "a centralized repository that allows you to store all your structured and unstructured data at any scale" (AWS).

International Business Machines (IBM) calls it a repository for large volumes of raw data, typically on low-cost cloud object storage (IBM). Object storage is a kind of cloud storage that keeps each item as a file with a name, instead of as a row in a table.

The big cloud vendors use almost the same words. The table shows where they agree, plus the small differences.

Source (read 2026-10-01)How it describes a data lakeWhat it adds
AWSCentral repository for structured and unstructured data, stored as isWarns of "data swamps" without governance and cataloguing
IBMRepository for large volumes of raw data, usually on cheap cloud object storageSays lakes do not enforce strict schemas
Microsoft AzureCentral repository that stores data in its original formSays data is often organised in stages: raw, cleansed and curated
Google CloudScalable repository for structured, semi-structured and unstructured data in native formatSays ingestion does not need a schema defined first
DatabricksCentral location holding a large amount of data in its native, raw formatLists problems of lakes, then describes its own lakehouse product

Notice what's missing. None of these pages gives a size at which a pile of files becomes a "lake". The word describes a way of storing data, not a size.

Where does the term "data lake" come from?​

James Dixon, a technology chief at the software firm Pentaho, used the image of a lake in a blog post dated October 14, 2010 (Dixon's post). He compared a data mart, a small tidy store built for one team, to "a store of bottled water, cleansed and packaged and structured for easy consumption". A data lake, he wrote, is "a large body of water in a more natural state".

IBM dates the coining of the term to 2011, which is a year later than that post (IBM). We couldn't find a source that settles the gap, so we use the 2010 post as the earliest one we found.

Early lakes ran on Hadoop, an open-source system for spreading data across many cheap computers. Its storage half is the Hadoop Distributed File System (HDFS), which the Apache project describes as built to "reliably store very large files across machines in a large cluster" on commodity hardware (Apache Hadoop).

A 2021 research paper by Databricks authors says that from 2015 cloud lakes on Amazon S3, Azure Data Lake Storage (ADLS) and Google Cloud Storage (GCS) started replacing HDFS (CIDR 2021 paper). It cites better durability, copies in several places, and very low cost. The authors sell a lakehouse product, so take the reasons as their view.

What kinds of data can a data lake hold?​

A lake can hold three broad kinds of data.

  • Structured data fits neatly into rows and columns, like a sales table in a business database.
  • Semi-structured data has some labels but no fixed table shape, such as logs, web pages or JSON files.
  • Unstructured data has no built-in shape at all, such as photos, audio, video or PDFs.

AWS says a lake can hold relational data from business apps and non-relational data from mobile apps, internet-connected devices and social media (AWS). Microsoft Learn adds that it is usually kept as files or blobs, which means binary large objects (Microsoft Learn, Azure Data Lake Storage introduction).

What do people use a data lake for?​

Four uses come up most often at IBM: all-purpose storage, backups and archiving, advanced analytics and AI, and data integration. Google Cloud adds examples such as recommendation systems built on raw user interaction data in media, and machine learning models on market data in finance.

Some organisations also keep "just-in-case" data with no defined use yet, because object storage is relatively inexpensive and scalable, says IBM. Data integration here just means pulling data from many sources into one place, so teams stop working from separate silos.

AWS cites a 451 Research survey in which more than half of surveyed enterprises had a data lake and another 22% planned one within 36 months (AWS). That's a vendor-cited survey, and AWS gives no date for it on the page, so treat it as a rough sign of adoption, nothing more.

Teams like lakes, IBM says, for three reasons: flexible ingestion of any format, lower cost from skipping upfront cleaning, and storage and compute that scale separately (IBM). Those are IBM's claims, and IBM sells lake and lakehouse software, so keep that in mind.

How does a data lake work?​

No single product is "the data lake". It's a set of parts, and you choose each one.

  1. Storage. Usually cloud object storage. Amazon S3 stores data as objects in buckets, where an object is a file plus its description, and a bucket is a container (S3 user guide).
  2. Ingestion. Something has to copy data in from databases, apps and files. That something is a job you pick. AWS Lake Formation, for example, says it can import data in bulk or step by step from MySQL, PostgreSQL, SQL Server, MariaDB and Oracle databases on Amazon RDS or EC2 (Lake Formation docs).
  3. A catalog. Think of it as a searchable directory of what's in the lake. The same AWS docs say crawlers can read files in S3, pull out table layouts and save them in a catalog.
  4. Permissions. The same docs say Lake Formation can restrict access at database, table, column, row and cell level.
  5. Query tools. These run SQL (the standard language for asking questions of tables) or ML code straight on the files. Amazon Athena is one example, and it runs SQL on files in S3 and bills by the amount of data scanned.

Data lake parts: ingestion and object storage bring data in; catalog, permissions and query tools get it out

Some lakes add a sixth part, a table layer. The Apache Iceberg site says Iceberg lets engines such as Spark, Trino, Flink, Presto, Hive and Impala work with the same tables safely at the same time (Apache Iceberg). A lake with such a layer is often called a lakehouse (defined below).

Why a table layer is sometimes needed​

Plain object storage has a limit you'll want to know about. The S3 guide says updates are per key, so there's no way to make atomic updates across several files unless your own code handles it (S3 user guide). Atomic just means all or nothing.

The CIDR paper says table layers such as Delta Lake and Apache Iceberg provide transactional views of a data lake. We didn't test this, so read it as the paper's claim.

What does "schema-on-read" mean?​

Schema sounds scary, but it's just the plan for how data is laid out: which columns exist and what type each one holds. Schema-on-read means the lake applies that plan when you read the data, not when you save it.

Microsoft says a lake uses schema-on-read and a warehouse uses schema-on-write (Azure). The picture shows the order of steps.

Lake stores data first and applies structure on read; warehouse checks structure before storing data

That's the whole trick behind the lake's flexibility. It's also the source of its biggest risk. The CIDR paper says the Hadoop-era lake made it cheap to store any data, but pushed data quality and governance problems downstream.

A worked example: why file format changes the bill​

How you keep files in a lake changes what it costs to ask a question. This example comes from a vendor price page.

AWS's Athena page works through one case. Take a table with four equal columns, stored as a 3 TB plain text file on S3. One terabyte (TB) is about a thousand gigabytes.

Athena bills $5 for each TB it scans (Athena pricing) (checked Oct 2026).

To read one column of the text file, Athena scans all of it, so one query costs $15 (checked Oct 2026). If you compress the file with GZIP (a common way to shrink files), AWS says you might see 3:1 gains. The file becomes 1 TB and the same query costs $5.

Convert it to Parquet, a columnar file format that stores each column together, and AWS says Athena reads only the one column the query needs. That scans 0.25 TB and costs $1.25 (checked Oct 2026). AWS calls this 3x savings from compression and 4x from reading one column.

Athena example: 3 TB text costs 15 dollars per query, compressed 1 TB costs 5, Parquet one column costs 1.25

It's AWS's own illustration for its own service, it assumes a query that touches one of four columns, and rates differ by region. It is not a benchmark, and your savings will depend on your data and questions.

Storing files "as is" is allowed, but the format you pick is a cost decision.

How is a data lake different from a data warehouse?​

Warehouses are databases built for reports and analysis on cleaned, organised data. Google Cloud says a warehouse is optimised for repeatable business reporting and structured SQL analysis, while a lake suits the diverse raw data used for machine learning (Google Cloud).

The table gives each its strengths and its weak spot, with the source.

Data lakeData warehouse
Data it holdsAny type, stored as is (AWS)Organised, mostly structured data (Google Cloud)
When structure is appliedOn read (Azure)On write (Azure)
Described as suited toRaw data for ML and data science (Google Cloud)Repeatable reports and SQL analysis (Google Cloud)
Where it fitsVideo, audio, text and sensor files; keeping raw data for laterTrusted numbers for dashboards that many people use
Where it falls shortQuality and governance are on you; swamp risk (IBM, AWS)Early warehouses could not store or query video, audio and text documents (CIDR paper)

That "falls short" cell for warehouses refers to first-generation designs in the 2021 paper. Modern warehouses may handle more, and we didn't test any.

Microsoft says it's common for a large organisation to have both. So the real question is often "which data goes where?", not "lake or warehouse?".

What is the difference between a data lake and a data lakehouse?​

A lakehouse is a lake with a table layer on top that adds safe updates and past versions of a table. It doesn't replace the lake. Our guides to lake vs lakehouse and warehouse vs lakehouse go further, and what is a data lakehouse covers the definition.

What goes wrong with data lakes?​

The sources name specific problems.

  • The data swamp. IBM warns that without a management layer, lakes can deteriorate into swamps, where good data is inaccessible. AWS says lakes need governance, cataloguing and security mechanisms so users can find data and trust it.
  • Nobody checks quality on the way in. Azure says a poorly managed lake can suffer from quality-control problems, data corruption and improper partitioning.
  • No built-in updates and transactions. Databricks lists no support for transactions, and says lakes are hard to secure and govern because data is hard to delete or update. Databricks sells a lakehouse product, so weigh that as a vendor view.
  • Slower queries as data grows. Databricks lists slower performance as volume grows. IBM says query and analytics performance can suffer as data volume grows, especially if data is not optimized for retrieval.
  • Security. IBM says it can be hard to make sure the large volumes of mixed data in a lake are not accessed without authorisation and meet data privacy rules.

None of these sources gives a failure rate. You'll see "most data lakes fail" repeated around the web, but the pages we read don't back it, so we won't repeat it.

How do you keep a lake from becoming a swamp?​

Per IBM, swamps are avoidable with strong data governance, data quality and data security practices from day one. It names three habits: defined data standards, metadata management and documentation, and access controls (IBM). IBM adds that unstructured data needs the same governance as structured data if the lake feeds generative AI.

A starting checklist follows from this. Write down the standards for incoming data, and keep a catalog so people can find it. Then decide who may read what before the first file lands.

What do practitioners say about the parts of a lake?​

The vendor pages above describe how lakes should work. Here's what some people who ran the parts say, with the site and date. These are single comments, not surveys, and we don't name the PeerSpot reviewers. They are individual opinions and prices; one person's view, with its date and link.

  • Amazon S3 (storage). A Hacker News commenter in the thread "AWS in 2025" (August 2025) said "S3 is actually quite expensive compared to the competition for both storage costs and egress costs". It added "For small scale stuff, S3s storage and egress charges are unlikely to be impactful." Egress means fees for moving data out.
  • Apache Hadoop (the early lake system). A reviewer on PeerSpot's Apache Hadoop reviews (reviews dated 2024 to 2025, so not new) wrote "Apache Hadoop was reliable. However, you can't stop managing it."
  • AWS Lake Formation (catalog and permissions). A reviewer on PeerSpot's AWS Lake Formation reviews (September 2025) wrote that it "is a complex service, so I recommend studying and gaining substantial knowledge before implementation". The same review said it improved the structure of their data mesh.

Our read: the pattern is cost and upkeep, which fits the swamp warning. Independent talk about the lake idea itself is thin, so we won't claim a wider trend.

When should you use a data lake?​

The sources describe reasons to pick a lake, but none gives a size or a budget cut-off. The table matches your situation to what the sources say, and says so where they're silent.

Your situationWhat the sources sayFit
You keep video, audio, text or sensor filesLakes hold any data type as is (AWS, IBM)A lake is a likely fit
You train ML models on large raw datasetsGoogle Cloud says a lake suits the diverse raw data used for MLA lake is a likely fit
You mainly run the same reports on clean tablesWarehouses are optimised for repeatable reporting (Google Cloud)A warehouse may be enough
You need trusted numbers, with no one to run governanceLakes need governance and cataloguing to avoid swamps (AWS, IBM)Be careful: start with a warehouse or a managed platform
You want both raw files and strict reportsCommon for large organisations to run both (Azure)A lake plus a warehouse, or a lakehouse
You need updates and deletes on lake dataDatabricks says lakes are hard to govern without the ability to delete or update dataLook at a table layer such as Iceberg

That "be careful" row is our reading of the sources, not a rule any of them states. Before you commit, ask the person who'd own the data whether they actually have time to run it.

How does data get into a data lake?​

A brand new lake is just an empty bucket. Data has to be copied in from business databases, event streams and files, and that job is called ingestion.

One common way to copy a database is change data capture (CDC). It reads the database's own change log and copies each insert, update and delete. OLake Go, our open-source tool, does this for PostgreSQL, MySQL and MariaDB, MongoDB and SQL Server, and writes Apache Iceberg or Parquet into your lake (see our docs). For the wider picture, read what is data ingestion and the best data lake tools by layer.

Common myths about data lakes​

  • "A data lake is a product you buy." No. AWS lists data lakes as one use of S3, and Azure Data Lake Storage is a set of capabilities on Blob Storage (Microsoft Learn). A lake is a way of using storage, not a thing in a box.
  • "A lake needs no planning because there is no schema." No. The schema is applied later, and the cleaning work doesn't go away. It just waits for you.
  • "Lakes are the same as warehouses." No. Microsoft and Google Cloud both describe them as different designs with different strengths.
  • "Lakehouses made lakes obsolete." Not shown by these sources. IBM says the low-cost flexibility of lakes continues to prove advantageous.
  • "A lake is only for big companies." None of the pages we read sets a minimum size, so no.

Questions to ask before you build one​

  1. What data do you have? Mostly clean tables, or files, logs and images too?
  2. Who will own governance? Someone has to run the catalog, access rules and quality checks.
  3. What will read the data? Reports, SQL, ML code? Check each tool reads your file format.
  4. How will you keep files efficient? The Athena example shows format and compression change the cost of a query.
  5. How will data get in, and how fresh must it be? This sets which ingestion tool you need.

Quick check.

Test your data lake basics

Question 1 of 5

Picture a company with a sales database, website logs, product photos and PDF contracts. How would a data lake treat all of that?

FAQs​

Q1. What is a data lake in simple terms?
It's a central store that keeps data in its original form, such as tables, logs, images and documents. You decide how to structure the data later, when you read it. AWS, IBM, Google Cloud and Microsoft all describe it this way.
Q2. What is the difference between a data lake and a data warehouse?
A lake stores data as is and applies structure when you read it. A warehouse checks structure before it stores anything, and it's built for repeatable reports, per Microsoft and Google Cloud. Large organisations often run both.
Q3. What is a data swamp?
It's a data lake that has turned messy and hard to use. IBM says that without a management layer a lake can decay into a swamp, where good data is inaccessible. AWS says governance, cataloguing and security help prevent it. IBM says that means standards, metadata, documentation and access controls from day one.
Q8. What is a data lake used for?
IBM lists all-purpose storage, backups and archiving, advanced analytics and AI, and data integration. Google Cloud gives examples such as recommendation systems and finance models built on raw data. It's a poor fit for quick, trusted dashboards unless someone runs governance.
Q4. Is Amazon S3 a data lake?
Not by itself. S3 is object storage, and AWS lists data lakes as one use for it. A lake also needs ingestion, a catalog, permissions and query tools around the storage.
Q5. What is the difference between a data lake and a data lakehouse?
A lakehouse is a lake with a management layer that adds warehouse features, such as safe updates and table versions. Microsoft says it aims to address the quality problems of traditional lakes. It sits on top of a lake, not instead of one.
Q6. Who coined the term data lake?
James Dixon of Pentaho used the idea in a blog post dated October 14, 2010, while IBM dates the term to 2011. He compared a data mart to bottled water and a data lake to a large body of water in a more natural state.
Q7. Are data lakes still relevant in 2026?
Yes, according to IBM, which says the low cost and flexibility of lakes continue to prove advantageous despite newer designs. Microsoft says it is common for large organisations to run both a lake and a warehouse. Check your own needs, though, because no source sets a size threshold.

Conclusion​

A data lake is a central store that keeps data in its original form and applies structure when you read it. You get low cost and flexibility, and you also get the job of quality, cataloguing and security. AWS's own Athena example shows that even file format can swing the cost of one query from $15 to $1.25 (checked Oct 2026).

Before you decide, jot down what data you hold, who'll own governance, and what tools must read it. Then run through the decision table above, and check the vendor page for each tool you shortlist. To fill the lake from PostgreSQL, MySQL, MongoDB or SQL Server, OLake Go is an open-source option.

Next steps

Related posts