Data Quality & Testing Frameworks logopeterindia.net
Knowledge Hub

Data Quality & Testing Frameworks

Bad data breaks dashboards, and bad data breaks AI agents even faster. A directory of the frameworks and platforms teams use to validate, monitor and certify data before it reaches production — from open-source unit tests to AI-native observability.

37 topics•6 categories•Updated October 2026

Open-Source Validation Frameworks

8 entries

Code-first libraries for defining unit tests for data and running them inside pipelines.

01Great Expectations (GX Core)Widely used open-source data quality framework to define, test and document expectations about your data. The project's GitHub repository now sits under the fivetran organization. 02dbt Data TestsBuilt-in tests in dbt that help data teams ship trusted data. Fivetran and dbt Labs completed their merger in June 2026, according to their press release. 03dbtThe open standard for AI-ready data transformation, with built-in tests. Fivetran and dbt Labs completed their merger in June 2026, according to their press release. 04DeequAmazon's Spark-based library for defining unit tests for data and measuring quality on large datasets. 05Soda CoreOpen-source data contracts engine for data quality checks. 06PanderaLightweight Python library for validating dataframes. 07Frictionless DataOpen-source standards and tools for describing and validating data. 08Apache GriffinData quality solution for big data (project mirror on GitHub). The GitHub repository was archived by its owner in November 2025.

AI-Native Data Observability Platforms

11 entries

Platforms that monitor pipelines, detect anomalies and extend observability to AI inputs and outputs.

09SodaAI-native automated data quality platform that finds, explains and helps fix issues from table to record level. 10Monte CarloData and AI observability platform for agent trust in production. 11Elementarydbt-native control plane for observability, quality, governance and discovery. 12DatafoldData engineering automation with data diffing, CI/CD testing and monitoring. 13BigeyeData observability and AI trust platform. 14AnomaloAutomated data quality monitoring using machine learning. 15ValidioAgentic data quality and observability platform. 16AcceldataAutonomous data and AI observability for hybrid clouds. 17LightupEnterprise data observability platform. 18Metaplane (Datadog)Data observability for modern data teams, now part of Datadog. 19IBM Data ObservabilityIBM's data observability offering, which grew from Databand.

Enterprise Data Quality & Governance

4 entries

Suites combining quality, catalog, lineage and governance.

20Ataccama ONEAgentic data trust software unifying quality, observability, catalog and lineage. 21Qlik Talend Data QualityUnified data quality and governance capabilities from Qlik Talend. 22Informatica Data QualityData quality and observability solutions from Informatica. Informatica is now part of Salesforce, which completed its acquisition in November 2025. 23Collibra Data Quality & ObservabilityData quality and observability within the Collibra platform.

Cloud-Native Data Quality

3 entries

Data quality features built into cloud data platforms.

24AWS Glue Data QualityRule-based data quality checks in AWS Glue. 25Google Knowledge Catalog – Auto Data QualityAutomatic data quality scans in Google Cloud's Knowledge Catalog (formerly Dataplex). 26Databricks Data Quality MonitoringData profiling and monitoring in Databricks Unity Catalog.

Data & Model Validation for ML

4 entries

Monitoring data and model quality for machine learning workloads.

27whylogsOpen-source data logging library for ML and data pipelines. The GitHub repository's last commit was in January 2025, so check its activity before adopting it. 28WhyLabsAI observability platform behind whylogs. 29Evidently AIOpen-source AI evaluation and observability, including data drift. 30DeepchecksEvaluation and testing for ML and LLM applications.

Standards, Lineage & Related Pages

7 entries

Reference bodies and related directories.

31OpenLineageOpen standard for collecting data lineage metadata. 32DAMA InternationalGlobal data management association behind the DMBOK. 33Data Quality SolutionsEnd-to-end data quality solutions, on this site. 34Data Observability ToolsTools for monitoring data pipeline health, on this site. 35Data Governance PlatformsGovernance platforms, on this site. 36DataOps PlatformsDataOps platforms and tools, on this site. 37Data EngineeringHub of data engineering directories.

Frequently Asked Questions

What is a data quality framework?

A data quality framework lets teams define, run and report checks on data, such as schema, completeness, uniqueness, freshness and value ranges, ideally inside the pipeline so bad data is caught before it reaches users.

What is the difference between data quality and data observability?

Data quality checks assert known rules, while data observability monitors pipelines continuously to detect unexpected anomalies, schema drift and freshness problems, often using machine learning.

Which open-source tools should I start with?

Great Expectations, dbt tests and Soda Core are common starting points, Deequ suits Spark, and Pandera suits Python dataframes.

How does data quality matter for AI?

Models and agents inherit the quality of their data, so validating inputs, monitoring drift and tracking lineage reduces wrong or unsafe outputs.

Where do data contracts fit in?

Data contracts define the expected schema and quality of data between producers and consumers, and engines such as Soda Core enforce them automatically.