CCC ETL
Service provides capabilities for extracting, transforming, and loading (ETL) data across diverse sources and sinks. It supports batch and real-time streaming architectures, managed data orchestration (DAGs), and serverless execution engines to process large-scale datasets with built-in fault tolerance.
Release Details
Version:
DEV
Assurance Level:
Release Manager:
DB
Development Build
Contributors
DT
Development Team
Change Log
- Development build - no formal changelog available
Capabilities
| ID | Title | Description | Threat Mappings |
|---|---|---|---|
| CCC.ETL.CP01 | Batch Processing | Supports the processing of bounded (batch) data sources using a consistent programming model or engine. | 0 |
| CCC.ETL.CP02 | Stream Processing | Supports the processing of unbounded (streaming) data sources using a consistent programming model or engine. | 0 |
| CCC.ETL.CP03 | Schema Evolution | Automatically detects source data structures and manages changes in schema (e.g., column additions) over time without pipeline failure. | 0 |
| CCC.ETL.CP04 | Distributed Data Shuffling | Provides an internal service to re-partition and group data across distributed workers for complex operations like joins and aggregations. | 0 |
| CCC.ETL.CP05 | Windowing and Event-Time Processing | Enables grouping of data based on time attributes, supporting tumbling, hopping, and session windows with late-data handling (watermarking). | 0 |
| CCC.ETL.CP06 | Change Data Capture (CDC) Integration | Supports incremental data ingestion by tracking changes in source transaction logs rather than full table scans. | 0 |
| CCC.ETL.CP07 | Connectivity and Connector Library | Provides pre-built, managed connectors for a variety of sources and sinks (e.g., Object Storage, RDBMS, NoSQL, Pub/Sub). | 0 |
| CCC.ETL.CP08 | Job Bookmarks | Persists the state of a processing job (e.g., checkpointing or bookmarks) to ensure exactly-once processing and fault tolerance. | 0 |
| CCC.ETL.CP09 | Pushdown Optimization | The ability to translate transformation logic into the native language of the source or sink (e.g., SQL) to minimize data movement. | 0 |
| CCC.ETL.CP10 | Visual Orchestration | Provides a graphical interface to define dependencies between extraction, transformation, and loading tasks. | 0 |
| CCC.ETL.CP11 | Data Lineage & Metadata Tracking | Captures and exports metadata regarding the data sources, the transformation steps, and the final destination (sink), showing the flow from source to destination for compliance and debugging. | 0 |
| CCC.ETL.CP12 | User-Defined Function (UDF) Support | Allows developers to inject custom logic (Python, Java, SQL) into the managed processing pipeline for complex transformations. | 0 |
| CCC.ETL.CP13 | Time-Based Job Triggering | Supports time-based (cron) mechanisms to initiate data processing workflows. | 0 |
| CCC.ETL.CP14 | Event Based Job Triggering | Supports event-based (file arrival) mechanisms to initiate data processing workflows. | 0 |
| CCC.Core.CP01 | Encryption in Transit Enabled by Default | The service automatically encrypts all data using industry-standard cryptographic protocols prior to transmission via a network interface. | 0 |
| CCC.Core.CP02 | Encryption at Rest Enabled by Default | The service automatically encrypts all data using industry-standard cryptographic protocols prior to being written to a storage medium. | 0 |
| CCC.Core.CP03 | Access Log Publication | The service automatically publishes structured, verbose records of activities performed within the scope of the service by external actors. | 0 |
| CCC.Core.CP06 | Access Control | The service automatically enforces user configurations to restrict or allow access to a specific component or a child resource based on factors such as user identities, roles, groups, or attributes. | 0 |
| CCC.Core.CP07 | Event Publication | The service automatically publishes a structured state-change record upon creation, deletion, or modification of data, configuration, components, or child resources. | 0 |
| CCC.Core.CP09 | Metrics Publication | The service automatically publishes structured, numeric, time-series data points related to the performance, availability, and health of the service or its child resources. | 0 |
| CCC.Core.CP10 | Log Publication | The service automatically publishes structured, verbose records of activities, operations, or events that occur within the service. | 0 |
| CCC.Core.CP14 | API Access | The service exposes a port enabling external actors to interact programmatically with the service and its resources using HTTP protocol methods such as GET, POST, PUT, and DELETE. | 0 |
| CCC.Core.CP18 | Resource Versioning | The service automatically assigns versions to child resources which can be used to preserve, retrieve, and restore past iterations. | 0 |
| CCC.Core.CP20 | Resource Tagging | The service provides users with the ability to tag a child resource with metadata that can be reviewed or queried. | 0 |
| CCC.Core.CP23 | Network Access Rules | The service restricts access to child or networked resources based on user-defined network parameters such as IP address, protocol, port, or source. | 0 |
| CCC.Core.CP28 | Command-line Interface | The service includes a component that reads and translates text into commands that can be executed by the service. | 0 |
| CCC.Core.CP29 | Active Ingestion | While running, the service can receive inputs, commands, or data streams from external sources such as dedicated APIs, exposed network ports, message queues, and persistent data ingestion channels. | 0 |