Ingesting with Cloudera Data Flow and Apache NiFi (DOPS-242) – Details

Detaillierter Kursinhalt

Introduction to Cloudera Flow Management
  • Overview of Cloudera Data-in-Motion
  • The NiFi User Interface
  • Data Flow Catalog
  • ReadyFlows
  • Instructor-Led Demo: NiFi User Interface
  • Hands-On Exercise: Build Your First Data Flow
Processors
  • Overview of Processors
  • Processor Surface Panel
  • Processor Configuration
  • Hands-On Exercise: Start Building a Data Flow Using Processors
Connections
  • Overview of Connections
  • Connection Configuration
  • Connector Context Menu
  • Hands-On Exercise: Connect Processors in a Data Flow
Data Flows
  • Command and Control of a Data Flow
  • Processor Relationships
  • Back Pressure
  • Prioritizers
  • Labels
  • Hands-On Exercise: Build a More Complex Data Flow
  • Hands-On Exercise: Creating a Fork Using Relationships
  • Hands-On Exercise: Set Back Pressure Thresholds
Process Groups
  • Anatomy of Process Group
  • Input and Output Ports
  • Hands-On Exercise: Simplify Data Flows Using Process Groups
FlowFile Provenance
  • Data Provenance Events
  • FlowFile Lineage
  • Replaying a FlowFile
  • Hands-On Exercise: Using Data Provenance
Parameters
  • Parameter Contexts
  • Referencing Parameters
  • Managing Parameters
  • Migrating from Variables
  • Hands-On Exercise: Creating, Using, and Managing Parameters
Flow Definitions and Templates
  • Flow Definition Overview
  • Creating a Flow Definition
  • Importing and Deploying a Flow
  • Using (migrating from) Templates
  • Hands-On Exercise: Creating, Using, and Managing Flow Definitions
Apache NiFi Registry
  • Apache NiFi Registry Overview
  • Using the Registry
  • Hands-On Exercise: Versioning Flows Using NiFi Registry
FlowFile Attributes
  • FlowFile Attribute Overview
  • Routing on Attributes
  • Hands-On Exercise: Working with FlowFile Attributes
NiFi Expression Language
  • NiFi Expression Language Overview
  • Syntax
  • Expression Language Editor
  • Setting Conditional Values
  • Hands-On Exercise: Using the NiFi Expression Language
Controller Services
  • Controller Services Overview
  • Common Controller Services
  • Hands-On Exercise: Adding Apache Hive Controller
Record-based Components
  • Record-oriented data
  • Record-based Processors
  • Avro Schema Registry
  • Schema Format
Reading and Writing Record Data
  • Querying Record Data
  • QueryRecord Processor
  • Writing Record Data
Enriching Record Data
  • ETL Operations
  • Split and Join Processor
  • Update Record Processors
  • Wait and Notify Processors
NiFi Architecture Overview
  • NiFi Architecture Overview
  • Public Cloud Architecture
  • Private Cloud Architecture
Data Flow Functions
  • Overview
  • Serverless functions
  • Demo: Deploying a Flow Definition as a Function
Data Flow Optimization
  • Data Flow Optimization
  • Control Rate
  • Managing Compute
  • Hands-On Exercise: Building an Optimized Data Flow
Monitoring, Reporting, and Troubleshooting
  • Monitoring from NiFi
  • Reporting
  • Examples of Common Reporting Tasks
  • Hands-On Exercise: Monitoring and Reporting
NiFi Security
  • NiFi Security Overview
  • Securing Access to the NiFi UI
  • Metadata Management
Integrating NiFi
  • NiFi Integration Architecture
  • Available ReadyFlows
  • A Closer Look at NiFi and Apache Hive