CUSTOMLesson 3 • Data Ingestion
Data Ingestion Three Methods
How customer data flows into MDP. Each method serves a different use case.

📦 Batch Ingestion

Large historical loads
CSV/Parquet uploads
Scheduled CRM syncs
Data lake → Profile

⚡ Streaming

Real-time Web SDK events
Millisecond to Profile
15-min batch to lake
Page views, purchases

🔌 Source Connectors

Pre-built integrations
Salesforce, Snowflake
S3, Azure Blob
Auto auth & scheduling

MDP — Data Flow Architecture
SOURCES
CRM
Web SDK
CSV Upload
PROCESSING
MDP
Data Lake
CONSUMERS
Real-Time Profile
Segments
Meridian Journey
Streaming: Real-Time Pipeline
Web SDK → Edge Network → MDP
🌐 Browser: Web SDK fires event
milliseconds
☁ MDP Edge Network
Real-Time Profile
Instant ⚡
Data Lake
Every 15 min

Why this matters for Sendwave migration

Streaming means a customer abandons their cart and the Meridian Journey can trigger within seconds. Sendwave had similar real-time triggers, but MDP streaming gives you profile data from ALL sources, not just the ESP.

Three-Tier Ingestion Plan
Plan your ingestion architecture around these three tiers:

Tier 1: Foundational Data (Batch)

CRM profiles, loyalty records, product catalog, Sendwave export data. Load via batch ingestion or source connectors. This is your initial data migration — the base layer everything else builds on.

Tier 2: Behavioral Data (Streaming)

Web page views, product interactions, cart activity, purchases, app events. Streamed via Web SDK and Mobile SDK. Hits the profile instantly for real-time segmentation and journey triggering.

Tier 3: Derived Data (Platform-Generated)

Computed attributes, ML model outputs, propensity scores. Generated within MDP itself. Enriches profiles with intelligence derived from the data you already ingested.

Each tier needs monitoring

Use the MDP data ingestion dashboard to track batch success rates, streaming throughput, and error rates. Set up alerts for failed batches. Profile-enabled datasets should show growing profile counts as data lands.

0:00
0:00
1/3
1
Three Ingestion Methods
2
Streaming Pipeline
3
Three-Tier Plan