Twitter (X) Data Engineer Interview Guide

Dan Lee's profile image
Dan LeeData & AI Lead
Last update
Twitter (X) Data Engineer Interview

Twitter (X) Data Engineer at a Glance

Total Compensation

$150k - $520k/yr

Difficulty

Levels

L3 - L7

Education

Bachelor's / Master's / PhD

Experience

0–18+ yrs

SQL Python Java Scaladata-pipelines-etl-eltdata-modelingdata-warehousing-lakehousebig-data-sparkstreaming-datacloud-data-platformdata-quality-governancesocial-media-product-analytics

X's data engineering org runs lean. The day-in-life data for this role shows a single engineer bouncing between Dataflow OOM debugging, PySpark pipeline builds, Airflow DAG refactors, BigQuery optimization, and on-call handoffs, all in one week. If you're coming from a company where pipeline work is split across ingestion, transformation, and platform teams, expect a much wider ownership surface here.

Twitter (X) Data Engineer Role

Primary Focus

data-pipelines-etl-eltdata-modelingdata-warehousing-lakehousebig-data-sparkstreaming-datacloud-data-platformdata-quality-governancesocial-media-product-analytics

Skill Profile

Math & StatsSoftware EngData & SQLMachine LearningApplied AIInfra & CloudBusinessViz & Comms

Math & Stats

Medium

Working statistical literacy to reason about data quality, distributions, anomalies, and basic experimentation/forecasting inputs; deep theoretical math is typically not central for core data engineering compared to analytics/DS (inferred; sources emphasize engineering/pipelines more than advanced math).

Software Eng

High

Strong coding, CS fundamentals (data structures/algorithms), building maintainable services and automation frameworks, testing, code reviews, and production-grade reliability; aligns with emphasis on programming/scripting and scalable, dependable systems in the data engineer skill source.

Data & SQL

Expert

Primary competency: design and operate scalable ETL/ELT, workflow orchestration, data modeling (star/snowflake), warehousing/lakehouse patterns, streaming vs batch, and end-to-end ownership including reliability and performance optimization; strongly supported by the data engineer skills source.

Machine Learning

Medium

Not always required for classic data engineering, but increasingly relevant for building data products that support ML and for ML-adjacent data workflows; conservative estimate due to lack of X-specific JD source, but consistent with industry trend and sources that mention AI/ML exposure as preferred in adjacent DE/architect roles.

Applied AI

Medium

Growing expectation to understand LLM/GenAI data needs (RAG pipelines, embeddings, agentic orchestration, and governance for AI-driven systems). This is an industry trend and present in the provided cloud-architect/data-engineer posting; for X specifically this is a cautious estimate (uncertain without a current X posting).

Infra & Cloud

High

Operate data systems on cloud and distributed compute: storage, compute, orchestration, CI/CD integration, IaC, containers; sources highlight cloud platforms (AWS/GCP/Azure) and big data tooling (Spark/Kafka), and adjacent DE roles emphasize cloud-native pipelines and DevOps integration.

Business

Medium

Translate stakeholder requirements into data models and SLAs, prioritize delivery, and understand product/metrics impact; sources emphasize collaboration/communication and making data accessible for decision-making, though deep domain expertise varies by team.

Viz & Comms

Medium

Clear documentation of data flows, data contracts, and communicating pipeline issues/quality to stakeholders is important; visualization is secondary for DE, but practical ability to present operational and data-quality insights is useful.

What You Need

  • Advanced SQL (joins, aggregations, performance tuning, indexing awareness)
  • Programming for data pipelines (Python common; JVM language often useful)
  • ETL/ELT pipeline design and implementation
  • Workflow orchestration (e.g., Airflow-style DAGs, scheduling, dependency management)
  • Data modeling and warehousing concepts (star/snowflake schemas; OLTP vs OLAP)
  • Big data processing concepts (distributed compute; batch and streaming fundamentals)
  • Data quality, monitoring, and reliability engineering for pipelines (alerts, retries, SLAs)
  • Performance optimization for queries, storage layouts, and pipeline execution
  • Documentation and collaboration with analytics/DS/business stakeholders

Nice to Have

  • Streaming systems experience (e.g., Kafka-like event pipelines)
  • Apache Spark (PySpark/Scala) for large-scale processing
  • Cloud data platform experience (AWS/GCP/Azure; storage + compute + orchestration services)
  • Infrastructure as code and CI/CD (e.g., Terraform; pipeline integration with Git-based workflows)
  • Containerization (e.g., Docker) and deployment hygiene
  • Data governance practices (lineage, access control, schema/version management)
  • GenAI/LLM data patterns (RAG pipelines, embeddings, evaluation/monitoring) (uncertain for X; included due to 2026 trend)
  • MLOps/feature store/ML data readiness experience (when supporting ML teams)

Languages

SQLPythonJavaScala

Tools & Technologies

Apache Airflow (or equivalent orchestration)Apache SparkApache Kafka (or equivalent streaming)Data warehouses (e.g., BigQuery, Redshift, Snowflake)Cloud platforms (AWS, GCP, Azure)Object storage/data lake storage (e.g., S3-like systems)ETL/ELT frameworks and job scheduling systemsMonitoring/alerting for pipelines and data quality checksDockerTerraformCI/CD tooling (e.g., GitHub-based workflows)

Want to ace the interview?

Practice with real questions.

Start Mock Interview

Success after year one means you reliably own a pipeline domain end-to-end, from Kafka event ingestion through processing (Dataflow, Spark) to BigQuery serving layers, with SLAs that hold without someone else backstopping you. Based on the day-in-life data, you'll be building ad impression attribution jobs, tuning feature store refresh queries for the For You timeline ranking, and writing migration plans for Trust & Safety data consumers. New ingestion paths appear as the product surface expands, and you're expected to design those paths, not just implement a spec.

A Typical Week

A Week in the Life of a Twitter Data Engineer

Typical L5 workweek · Twitter

Weekly time split

Coding — 30%Infrastructure — 20%Meetings — 18%Writing — 12%Break — 8%Research — 7%Analysis — 5%

Culture notes

  • Post-acquisition X operates lean with high ownership expectations — you'll own pipelines end-to-end from ingestion to serving, and on-call weeks can be intense given the reduced headcount.
  • The company requires in-office presence at the San Francisco HQ most days, though some teams have negotiated hybrid arrangements depending on manager discretion.

The split between infrastructure work and pure coding is closer than you'd expect at most companies. What that looks like in practice: Monday morning you're bumping Dataflow worker memory after a quote-tweet volume spike caused an OOM, and by Thursday afternoon you're writing pytest fixtures for the PySpark attribution job you've been building all week. The writing allocation is worth paying attention to, because design docs, migration plans, and runbook updates aren't optional when you may be the only person who knows why a particular DAG exists.

Projects & Impact Areas

Ad impression attribution stands out as the highest-visibility pipeline work in the day-in-life data. It joins Kafka ad impression streams with conversion events in GCS using a 7-day lookback window, and latency regressions here affect bid optimization directly. Alongside that revenue-critical work, you'll find yourself cleaning up 14TB of orphaned GCS Parquet files from deprecated pipelines, prototyping Apache Iceberg migrations for high-volume event tables, and writing schema migration plans so the Trust & Safety team can get denormalized action-level data instead of session-grain rollups.

Skills & What's Expected

Production-grade coding is the most underrated skill for this role. Candidates over-index on knowing Spark APIs and under-index on writing Python or Scala that handles edge cases, includes tests, and survives code review from engineers who ship services. The medium rating on ML/GenAI might tempt you to skip it, but the day-in-life shows you'll be tuning BigQuery queries that feed the ML feature store for timeline ranking, so you need to reason about how ML teams consume your output. Cloud infrastructure knowledge matters more here than at many data engineering roles because the stack is actively GCP-native (Dataflow, BigQuery, GCS), and you're expected to make cost and performance decisions like switching from on-demand to reserved slot pools.

Levels & Career Growth

Twitter Data Engineer Levels

Each level has different expectations, compensation, and interview focus.

Base

$130k

Stock/yr

$15k

Bonus

$5k

0–2 yrs Typically BS in Computer Science, Engineering, or related field (or equivalent practical experience).

What This Level Looks Like

Delivers well-scoped data pipeline components and modest end-to-end ETL/ELT workflows for a single team/product area; impact is primarily within one dataset/domain, improving data availability, correctness, and basic performance under guidance.

Day-to-Day Focus

  • →SQL fluency and data modeling fundamentals (dimensional modeling, event data, slowly changing dimensions basics).
  • →Reliability basics: idempotency, backfills, SLAs, monitoring/alerting, and data quality gates.
  • →Software engineering hygiene: testing, version control, CI/CD basics, readable code, incremental delivery.
  • →Understanding of distributed data processing basics (e.g., Spark/Flink concepts) and storage formats (Parquet/Avro) at an applied level.

Interview Focus at This Level

Emphasis on strong SQL, core programming ability (typically Python/Java/Scala basics), data pipeline fundamentals (ETL/ELT, scheduling, incremental loads, backfills), and practical problem solving. Expect questions on data modeling, debugging broken pipelines, basic distributed systems concepts (partitions, shuffles, latency vs throughput), plus behavioral signals like ownership, communication, and ability to learn and take feedback.

Promotion Path

Promotion to the next level typically requires independently owning a small-to-medium pipeline or dataset end-to-end (design, implementation, monitoring, and operational excellence), consistently delivering on commitments with minimal rework, proactively improving data quality/performance, demonstrating good on-call hygiene (clear RCA and prevention), and showing effective cross-functional collaboration while requiring materially less day-to-day guidance.

Find your level

Practice with questions tailored to your target level.

Start Practicing

The widget shows the level bands, but here's what the data implies about the gaps between them. At L3-L4 you own individual pipelines and prove reliability. At L5 you own a domain (say, all ad-event ingestion) and set SLAs. The jump to L6 is where people stall, because it's not about writing better Spark jobs. It's about defining cross-team data platform patterns that other engineers adopt without you pushing them. Demonstrating measurable reliability improvements (SLO attainment, incident reduction) is the strongest signal for advancing above L5.

Work Culture

The culture notes indicate X expects in-office presence at San Francisco HQ most days, though some teams have reportedly negotiated hybrid arrangements depending on manager discretion. On-call weeks can be intense given the lean headcount absorbing the same event volume, and the day-in-life data reflects that reality: weekend SLA breaches, OOM failures to triage Monday morning, and runbook handoffs every Friday. On the positive side, the team structure means less bureaucracy between you and the people who use your pipelines, and your work is visible to leadership without layers of middle management filtering it.

Twitter (X) Data Engineer Compensation

X is a private company, so any equity in your offer is illiquid. From what candidates report, that changes the math dramatically at senior levels, where the stock component can represent a third or more of headline total comp. Until there's a liquidity event, you can't spend those shares. Get every detail of your grant in writing before signing: vesting cadence, cliff, refresh policy, and any performance conditions. The source data on X's specific equity mechanics is thin, so treat anything a recruiter tells you verbally as unconfirmed until it's in your offer letter.

Your strongest negotiation levers, per the offer notes, are equity amount, sign-on bonus, and level. Level is worth special attention because moving up even one rung shifts every comp component (base, stock, bonus) into a higher band simultaneously. If you're carrying a competing offer from a public company where RSUs convert to cash quarterly, lean on that contrast to push for a larger sign-on bonus or higher base, since those are the parts of the package you can actually deposit.

Twitter (X) Data Engineer Interview Process

~4 weeks end to end

Initial Screen

2 rounds
1

Recruiter Screen

30mPhone

To kick things off, you’ll have a short conversation focused on role fit, location/remote constraints, timeline, and compensation expectations. Expect a high-level walkthrough of your recent data engineering work (pipelines, warehouses, scale) and a quick check on communication and impact. You may also be asked what parts of X’s product/data ecosystem you’re most interested in supporting.

generalbehavioraldata_engineeringengineering

Tips for this round

  • Prepare a 90-second narrative: domain, scale (events/day, TB/day), stack (Kafka/Spark/Airflow/Snowflake/Redshift), and measurable outcomes (latency, cost, data quality).
  • Have a crisp mapping from your experience to X-like problems: high-volume event ingestion, near-real-time analytics, experimentation, and user/product metrics.
  • Align on leveling by describing scope: ownership boundaries, on-call/ops exposure, cross-functional partners (PM/DS/Eng), and leadership examples.
  • State compensation ranges with a rationale (current market, level, location) and mention you’re open to a mix of base/equity/bonus.
  • Confirm the next steps and ask what technical areas will be emphasized (SQL, pipeline design, Spark, streaming) so you can prep precisely.

Technical Assessment

2 rounds
3

SQL & Data Modeling

60mLive

Expect a live SQL session where you’ll write queries against event-style tables and reason about correctness and performance. You’ll likely cover joins, aggregations, window functions, and edge cases like duplicates, late events, and time zones. Some interviewers also add a short schema/design prompt to assess modeling instincts for analytics use cases.

databasedata_modelingdata_engineeringdata_warehouse

Tips for this round

  • Practice advanced SQL patterns: window functions (LAG/LEAD, ROW_NUMBER), conditional aggregation, and sessionization by user/time gaps.
  • Call out data pitfalls explicitly: dedup keys, event-time vs ingest-time, null handling, and why results may differ under late-arriving data.
  • Explain performance choices: partition pruning, clustering/sort keys, predicate pushdown, approximate aggregations, and avoiding explode-style joins.
  • Model for consumption: propose fact tables for events and dimensions for users/content, with clear grain and surrogate keys where needed.
  • Validate with quick sanity checks: row counts after joins, distinct user counts, and small test cases before finalizing the query.

Onsite

2 rounds
5

System Design

60mVideo Call

During the onsite loop, the interviewer will probe your ability to design a scalable data pipeline for high-volume product events. You’ll be asked to define requirements (freshness, latency, accuracy), pick batch vs streaming components, and lay out storage/serving choices for analytics and downstream systems. Reliability, backfills, and observability usually matter as much as the happy-path architecture.

system_designdata_pipelinedata_engineeringcloud_infrastructure

Tips for this round

  • Start by clarifying requirements: QPS/events per second, latency target (minutes vs hours), consumers (dashboards, experimentation, ML), and retention.
  • Propose an end-to-end design: producers → Kafka/Kinesis-like bus → stream processing (Flink/Spark) + batch (Spark) → warehouse/lake → serving/semantic layer.
  • Address correctness: idempotent writes, watermarking, late data handling, schema evolution (Avro/Protobuf), and replay/backfill strategies.
  • Design for operations: monitoring (lag, throughput, error rate), data quality checks, lineage, and incident response/runbooks.
  • Discuss cost and performance levers: partitioning strategy, compaction, scaling policies, and when to pre-aggregate vs serve raw events.

Tips to Stand Out

  • Master SQL for event data. Practice window functions, sessionization, de-duplication, and performance tuning (partition pruning, clustering) because product-event analytics is a common evaluation surface.
  • Design pipelines like you’ll operate them. Always add monitoring, backfills/replays, idempotency, schema evolution, and data quality checks to your architecture—omitting these is a frequent signal of inexperience.
  • Quantify impact and scale. Be ready with numbers (throughput, latency, cost reduction, data accuracy improvements, stakeholder adoption) and tie them to business outcomes like faster experimentation or better decision-making.
  • Communicate tradeoffs explicitly. In system design and manager screens, compare batch vs streaming, lake vs warehouse, pre-aggregation vs raw, and cost vs freshness using clear decision criteria.
  • Be crisp about modeling and metric definitions. State table grain, keys, slowly changing dimensions, and how you avoid metric drift via documentation, semantic layers, and definition governance.
  • Prepare for coding fundamentals. Even for data engineering, rehearse DS&A basics and write clean, testable code with correct complexity and edge-case handling.

Common Reasons Candidates Don't Pass

  • ✗Weak SQL fundamentals. Candidates get rejected for incorrect joins/window logic, ignoring duplicates/late events, or producing results that don’t match the table grain and metric definition.
  • ✗Shallow pipeline/system design. Proposals that skip backfills, idempotency, schema evolution, monitoring, or incident response look academic and not production-ready.
  • ✗Coding gaps. Struggling to implement a correct solution, missing edge cases, or being unable to reason about complexity can outweigh otherwise strong DE experience.
  • ✗Unclear ownership and impact. Vague descriptions like “worked on Airflow/Spark” without end-to-end responsibility, decisions made, and measurable outcomes often fail hiring manager and behavioral rounds.
  • ✗Poor communication under ambiguity. Inability to clarify requirements, align on definitions, or explain tradeoffs calmly can signal risk in cross-functional environments.

Offer & Negotiation

For a Data Engineer at a company like X, compensation is typically a mix of base salary plus equity (often RSUs or equity-like grants) and sometimes a cash bonus/sign-on depending on level and urgency. The most negotiable levers are usually equity amount, sign-on bonus, and level (which affects bands) more than base once you’re near the top of a range. Use competing offers or market data to justify a specific ask, and confirm vesting schedule, refresh policy, and any clawbacks or performance conditions before accepting.

The loop runs about four weeks from recruiter call to offer. System design is the round where, from what candidates report, rejection rates are highest. X's interviewers expect you to reason about pipeline architectures for their specific product surfaces (ad auction events, timeline engagement streams, trust/safety classification feeds) and to address operational realities like schema evolution with Protobuf, replay strategies against Manhattan, and monitoring for Kafka consumer lag. Vague component diagrams won't cut it.

Don't treat the Hiring Manager Screen as a warmup. The round description makes clear the HM is probing whether you've owned pipelines end-to-end, from defining SLAs through incident response, not just whether you've touched Spark or Airflow. Come ready to walk through a specific architecture you built, the tradeoffs you chose (batch vs. streaming, cost vs. freshness), and the measurable outcomes you delivered.

Twitter (X) Data Engineer Interview Questions

Data Pipelines & Orchestration (Batch + Streaming)

Expect questions that force you to design end-to-end batch and streaming pipelines for high-volume event data (timelines, engagements) with clear SLAs. You’ll be pushed on orchestration choices, backfills, late data, idempotency, retries, and how you keep pipelines reliable under constant product change.

You own a daily batch ETL that builds a tweet engagements fact table from raw like/retweet/reply events, partitioned by event_date. A rerun for the same date currently doubles counts, what specific idempotency strategy do you implement at the table level and in the DAG?

EasyIdempotency and Backfills

Sample Answer

Most candidates default to appending and hoping a retry lands once, but that fails here because orchestration retries and manual backfills are normal, and duplicates silently corrupt metrics. You make the load idempotent by using deterministic keys (for example, event_id or a composite of user_id, tweet_id, event_type, event_ts) and writing with partition overwrite or MERGE into the target partition. In the DAG, you enforce a single partition per run, add a pre-run cleanup step or atomic swap, and use task-level retries that are safe because the write is idempotent. You also add a post-load row count and uniqueness check so a bad rerun pages you fast.

SQL
1MERGE INTO analytics.fact_tweet_engagements t
2USING staging.engagement_events s
3ON t.event_id = s.event_id
4WHEN MATCHED THEN UPDATE SET
5  t.user_id = s.user_id,
6  t.tweet_id = s.tweet_id,
7  t.event_type = s.event_type,
8  t.event_ts = s.event_ts,
9  t.ingest_ts = s.ingest_ts
10WHEN NOT MATCHED THEN INSERT (
11  event_id, user_id, tweet_id, event_type, event_ts, ingest_ts
12) VALUES (
13  s.event_id, s.user_id, s.tweet_id, s.event_type, s.event_ts, s.ingest_ts
14);
Practice more Data Pipelines & Orchestration (Batch + Streaming) questions

SQL & Analytics Querying at Scale

Most candidates underestimate how much signal comes from writing correct, performant SQL under realistic constraints (large tables, skew, incremental partitions). You’ll need to translate product questions into joins/window functions, handle event-time pitfalls, and explain performance tradeoffs.

You have an event table tweet_impressions(event_time, tweet_id, viewer_user_id, surface, request_id) partitioned by DATE(event_time). Write SQL to compute daily unique viewers per tweet for the last 7 days, deduping multiple rows caused by retries using request_id.

EasyAggregations and Deduplication

Sample Answer

Compute per-day per-tweet unique viewers after deduping on request_id. You filter partitions with DATE(event_time) for the last 7 days, then keep one row per (day, tweet_id, viewer_user_id, request_id) and finally count distinct viewer_user_id per (day, tweet_id). This avoids inflated counts from retry storms. It also keeps the scan bounded to recent partitions, which matters at X scale.

SQL
1/* Daily unique viewers per tweet, last 7 days, retry-deduped by request_id.
2   Assumes a warehouse like BigQuery or Snowflake with DATE() support. */
3WITH base AS (
4  SELECT
5    DATE(event_time) AS event_date,
6    tweet_id,
7    viewer_user_id,
8    request_id
9  FROM tweet_impressions
10  WHERE DATE(event_time) >= DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY)
11),
12retry_dedup AS (
13  /* Keep one record per retry key at the grain you care about.
14     If request_id can repeat across tweets, include tweet_id as well. */
15  SELECT
16    event_date,
17    tweet_id,
18    viewer_user_id,
19    request_id
20  FROM base
21  QUALIFY ROW_NUMBER() OVER (
22    PARTITION BY event_date, tweet_id, viewer_user_id, request_id
23    ORDER BY request_id
24  ) = 1
25)
26SELECT
27  event_date,
28  tweet_id,
29  COUNT(DISTINCT viewer_user_id) AS daily_unique_viewers
30FROM retry_dedup
31GROUP BY event_date, tweet_id
32ORDER BY event_date DESC, daily_unique_viewers DESC;
Practice more SQL & Analytics Querying at Scale questions

Data Modeling & Warehouse/Lakehouse Foundations

Your ability to reason about durable schemas matters because downstream analytics/ML depends on stable contracts. Interviewers look for strong dimensional modeling instincts, handling evolving event schemas, choosing grain, and designing fact/dimension tables (or lakehouse equivalents) that scale with new features.

You are modeling X Home timeline impressions for analytics. Would you store a single wide fact table keyed by (viewer_id, tweet_id, impression_ts) or a narrow impression fact plus separate engagement facts, and how do you handle deduping when the client retries the same impression event?

EasyDimensional Modeling, Grain, Facts vs Wide Tables

Sample Answer

You could do a single wide fact table or a narrow impression fact plus separate engagement facts. The wide table is simpler for common dashboards but it gets sparse, hard to evolve, and causes backfills whenever new engagement types ship. The narrow facts win here because the grain stays clean and each event type can evolve independently, then you join at query time or via derived marts. For dedupe, enforce an event_id (or deterministic hash of stable fields) and keep the latest by ingest_ts, then model counts from deduped records only.

Practice more Data Modeling & Warehouse/Lakehouse Foundations questions

Distributed Processing & Performance (Spark/Big Data)

The bar here isn't whether you know Spark buzzwords, it's whether you can debug and optimize real distributed jobs. You’ll be evaluated on partitioning strategy, shuffle avoidance, join patterns, file layout (Parquet/ORC), and practical approaches to reducing cost/latency for massive social graph and event datasets.

You have a Spark job that computes daily DAU by country from X app events and suddenly takes 4x longer after a code change that added a join to the user dimension. What exact Spark UI signals tell you it is shuffle related, and what 3 fixes do you try in order, including partitioning and join strategy?

EasySpark Debugging and Shuffle Optimization

Sample Answer

Reason through it: Start by checking the Spark UI stage DAG and look for new wide stages, big shuffle read and write, and skewed task durations where a few tasks run forever. Then compare input size to shuffle size, if shuffle bytes dwarf input bytes, the join is driving it. Next, validate join keys and cardinality, a many to many join or exploding dimension rows is a common self inflicted wound. Fixes in order, broadcast the dimension if it is small enough, preselect only needed columns, and repartition by join key (or coalesce after) to control shuffle and file counts.

Practice more Distributed Processing & Performance (Spark/Big Data) questions

Cloud Infrastructure, Deployment & Platform Operations

In practice, you’ll be asked to connect pipeline design to how it actually runs in production on cloud primitives. Strong answers cover storage/compute separation, IAM/access patterns, CI/CD and IaC expectations, plus operational hygiene like rollbacks, secrets management, and environment parity.

A Spark ETL job writes the daily Home Timeline impressions table to S3 and then publishes a Hive/Glue partition for ds=$\text{YYYY-MM-DD}$. A rerun created duplicate rows for the same ds, what deployment and write patterns prevent this while keeping the job idempotent?

EasyIdempotent Deployments and Atomic Writes

Sample Answer

This question is checking whether you can make batch pipelines safe to rerun without corrupting downstream tables. You should talk about atomic partition replacement (write to a temp path, then swap), deterministic partitioning by ds, and merge semantics when a natural key exists. Also mention orchestration guards, one writer per partition, and a commit protocol so the catalog update only happens after data is fully materialized.

Practice more Cloud Infrastructure, Deployment & Platform Operations questions

Coding for Data Systems (Python/Scala/Java Basics)

Instead of puzzle-heavy CS, you should prepare for code that looks like data engineering glue: parsing events, deduping, validating schemas, and writing testable transformations. Candidates often stumble by ignoring edge cases (nulls, ordering, time windows) or by writing code that’s hard to operate.

You ingest X Home Timeline impression events as JSON lines, but downstream sinks require a stable schema and no duplicate (user_id, tweet_id, impression_id). Write a function that parses lines, drops invalid records (missing required fields or wrong types), dedupes by that composite key keeping the earliest event_time, and returns cleaned records sorted by event_time.

EasyEvent Parsing and Deduplication

Sample Answer

The standard move is parse to dicts, validate required fields, then dedupe with a hash map keyed by (user_id, tweet_id, impression_id). But here, event_time matters because duplicates can arrive out of order, you must keep the earliest timestamp, not the first one you saw.

Python
1import json
2from datetime import datetime, timezone
3from typing import Any, Dict, Iterable, List, Optional, Tuple
4
5
6def _parse_iso8601_utc(ts: str) -> Optional[datetime]:
7    """Parse an ISO8601 timestamp into an aware datetime in UTC.
8
9    Accepts strings like:
10      - 2026-02-26T12:34:56Z
11      - 2026-02-26T12:34:56+00:00
12      - 2026-02-26T12:34:56.123Z
13
14    Returns None if parsing fails.
15    """
16    if not isinstance(ts, str) or not ts:
17        return None
18
19    # Normalize trailing Z to +00:00 for fromisoformat.
20    s = ts.strip()
21    if s.endswith("Z"):
22        s = s[:-1] + "+00:00"
23    try:
24        dt = datetime.fromisoformat(s)
25    except ValueError:
26        return None
27
28    # Ensure timezone aware; if naive, assume UTC.
29    if dt.tzinfo is None:
30        dt = dt.replace(tzinfo=timezone.utc)
31    return dt.astimezone(timezone.utc)
32
33
34def clean_impression_events(lines: Iterable[str]) -> List[Dict[str, Any]]:
35    """Parse, validate, dedupe, and order impression events.
36
37    Required fields and types:
38      - user_id: int
39      - tweet_id: int
40      - impression_id: str
41      - event_time: ISO8601 str
42
43    Dedup rule: (user_id, tweet_id, impression_id), keep the earliest event_time.
44    Output: list of cleaned dicts sorted by event_time ascending.
45    """
46    required = ("user_id", "tweet_id", "impression_id", "event_time")
47
48    best_by_key: Dict[Tuple[int, int, str], Tuple[datetime, Dict[str, Any]]] = {}
49
50    for line in lines:
51        if not isinstance(line, str) or not line.strip():
52            continue
53
54        try:
55            obj = json.loads(line)
56        except json.JSONDecodeError:
57            continue
58
59        if not isinstance(obj, dict):
60            continue
61
62        if any(k not in obj for k in required):
63            continue
64
65        user_id = obj.get("user_id")
66        tweet_id = obj.get("tweet_id")
67        impression_id = obj.get("impression_id")
68        event_time_raw = obj.get("event_time")
69
70        if not isinstance(user_id, int) or not isinstance(tweet_id, int) or not isinstance(impression_id, str):
71            continue
72
73        event_dt = _parse_iso8601_utc(event_time_raw)
74        if event_dt is None:
75            continue
76
77        key = (user_id, tweet_id, impression_id)
78
79        # Store a normalized record for downstream.
80        cleaned = dict(obj)
81        cleaned["event_time"] = event_dt.isoformat().replace("+00:00", "Z")
82
83        if key not in best_by_key:
84            best_by_key[key] = (event_dt, cleaned)
85        else:
86            prev_dt, _ = best_by_key[key]
87            if event_dt < prev_dt:
88                best_by_key[key] = (event_dt, cleaned)
89
90    # Sort by event_time.
91    out = [pair[1] for pair in best_by_key.values()]
92    out.sort(key=lambda r: _parse_iso8601_utc(r["event_time"]))
93    return out
94
Practice more Coding for Data Systems (Python/Scala/Java Basics) questions

X's loop is weighted toward the unsexy, high-stakes work that actually keeps the platform running: designing Kafka-to-warehouse ingestion for hundreds of billions of daily events, then proving you can query and model that data under real constraints like partition skew on tweet_id or SCD handling when post schemas evolve with new fields like community_id. The compounding trap is that pipeline design and SQL questions both assume you already think in terms of idempotent backfills, partitioning strategy, and SLA-aware orchestration, so a gap in either area undermines your answers in the other. Meanwhile, the coding slice focuses on data engineering glue (schema validation, deduplication, streaming-safe counters), not algorithmic puzzles, which means drilling competitive programming problems is probably the lowest-ROI way to spend your prep time.

Practice with pipeline, SQL, and modeling questions tailored to roles like this at datainterview.com/questions.

How to Prepare for Twitter (X) Data Engineer Interviews

Know the Business

Updated Q1 2026

Official mission

“to promote and protect the public conversation — to be the town square of the internet.”

What it actually means

X aims to transform into an 'everything app' that integrates social media, audio, video, messaging, payments, and banking, powered by AI. It also seeks to be the internet's public square, facilitating open conversation and real-time information exchange.

San Francisco, CaliforniaFully In-Office

Funding & Scale

Valuation

$44B

Current Strategic Priorities

  • Become 'the everything app' under the name X

Competitive Moat

Real-time information disseminationPublic discourseImmediate access to global eventsImmediate access to diverse perspectivesUnique real-time information flowStrategic pivot towards an 'everything app' model

X's stated ambition is to become an "everything app" that integrates payments, banking, video, and messaging into a single platform. For data engineers, that means the job isn't maintaining a mature social media pipeline. It's building net-new data infrastructure for verticals that didn't exist at Twitter two years ago, while keeping ad revenue pipelines healthy given $2.5B in annual revenue declining at -13.8% year-over-year.

The "why X?" answer that falls flat is some variation of "I believe in the everything-app vision." What separates strong candidates is demonstrating you've studied X's actual tech stack and can speak to specific engineering tradeoffs. Reference the Kafka-based event infrastructure, the reported cloud migration challenges, or how X's history of building custom internal tools rather than buying off the shelf shapes what a DE owns day to day. That kind of answer shows you've done homework beyond the headlines.

Try a Real Interview Question

7-day rolling active users and anomaly flag by platform

sql

Given daily user activity events, compute for each $platform$ and $event\_date$ the daily active users (DAU) and the 7-day rolling active users (RAU) defined as unique users with at least one event in the window $[event\_date-6,\ event\_date]$. Output $platform$, $event\_date$, $dau$, $rau\_7d$, and $is\_anomalous$ where $is\_anomalous=1$ if $dau < 0.5 \times rau\_7d$, else $0$.

user_events
event_dateuser_idplatformevent_type
2024-01-01101iosopen
2024-01-01102ioslike
2024-01-02101iosreply
2024-01-02201webopen
2024-01-03103iosopen
daily_platform_dau
platformevent_datedau
ios2024-01-012
ios2024-01-021
web2024-01-021
ios2024-01-031
web2024-01-030

700+ coding problems with a live code executor.

Practice in the Engine

X's coding rounds reportedly emphasize production-quality data manipulation over algorithmic puzzles, which tracks with a post-layoff org where DEs ship code directly to pipelines serving ad auctions and content ranking. The problem above is a good proxy for that bar. Build your muscle memory with more problems like it at datainterview.com/coding.

Test Your Readiness

How Ready Are You for Twitter (X) Data Engineer?

1 / 10
Pipelines and Orchestration

Can you design a batch ETL pipeline for daily tweet engagement aggregates, including idempotent loads, late arriving data handling, and backfills without double counting?

Spot your weak areas across X's question distribution, then drill them at datainterview.com/questions.

Frequently Asked Questions

How long does the Twitter (X) Data Engineer interview process take?

Expect roughly 4 to 8 weeks from first recruiter call to offer. You'll typically have an initial phone screen, a technical phone interview focused on SQL and coding, and then a virtual or in-person onsite with multiple rounds. Scheduling can stretch things out, especially if the team is busy or you need to coordinate across time zones. I've seen some candidates move faster if they have competing offers.

What technical skills are tested in the Twitter (X) Data Engineer interview?

SQL is the backbone. You need advanced joins, aggregations, performance tuning, and indexing awareness. Beyond that, expect coding questions in Python (most common), Java, or Scala for building data pipelines. They'll test you on ETL/ELT design, workflow orchestration like Airflow-style DAGs, data modeling (star and snowflake schemas, OLTP vs OLAP), and big data concepts including distributed compute and batch vs streaming. At senior levels and above, data quality, monitoring, SLAs, and reliability engineering become major topics.

How should I tailor my resume for a Twitter (X) Data Engineer role?

Lead with pipeline work. If you've built or maintained ETL/ELT pipelines, put that front and center with specific scale numbers (rows processed, latency targets, number of downstream consumers). Mention SQL heavily, and call out any experience with orchestration tools, data modeling, or big data frameworks. Twitter (X) cares about collaboration with analytics and data science teams, so highlight cross-functional projects. Keep it to one page for L3/L4, two pages max for L5 and above.

What is the total compensation for a Twitter (X) Data Engineer?

Compensation varies a lot by level. At L3 (Junior, 0-2 years), total comp averages around $150,000 with a range of $115,000 to $185,000. L4 (Mid, 2-6 years) averages $180,000, ranging from $140,000 to $230,000. L5 (Senior, 5-12 years) jumps to about $340,000 with a range of $250,000 to $450,000. L6 (Staff) averages $320,000, and L7 (Principal) can hit $520,000 on average with a ceiling near $700,000. Base salaries range from $130,000 at L3 to $235,000 at L7, with equity making up a significant portion of the package at senior levels.

How do I prepare for the behavioral interview at Twitter (X) for a Data Engineer position?

Twitter (X) wants to see that you can collaborate with analytics, data science, and business stakeholders. Prepare stories about times you worked cross-functionally, handled ambiguity, or improved a process that others depended on. Documentation and communication matter here. For senior levels (L5+), they'll dig into how you've influenced technical direction or resolved disagreements across teams. Have 4 to 5 strong stories ready that cover different themes.

How hard are the SQL questions in the Twitter (X) Data Engineer interview?

They're genuinely hard, especially at L4 and above. Expect multi-join queries, window functions, performance tuning scenarios, and questions about indexing strategy. You won't get away with just knowing SELECT and GROUP BY. At senior levels, they might ask you to optimize a slow query or redesign a schema for better read performance. I'd recommend practicing advanced SQL problems regularly on datainterview.com/questions to get comfortable with the complexity.

Are ML or statistics concepts tested in the Twitter (X) Data Engineer interview?

Not directly for most Data Engineer levels. The focus is squarely on data engineering fundamentals: pipelines, SQL, data modeling, and system design. That said, you should understand how data scientists and ML engineers consume the data you produce. Knowing what clean, well-modeled data looks like for downstream ML use cases will help you in both system design and behavioral rounds, especially at L5 and above where cross-team collaboration is evaluated.

What format should I use to answer behavioral questions at Twitter (X)?

Use the STAR format (Situation, Task, Action, Result) but keep it tight. Spend about 20% on setup and 60% on what you actually did. Quantify results whenever possible, like 'reduced pipeline latency by 40%' or 'cut data quality incidents from 5 per month to zero.' At Staff and Principal levels, emphasize your decision-making process and how you brought others along. Don't ramble. Two minutes per answer is the sweet spot.

What happens during the onsite interview for a Twitter (X) Data Engineer?

The onsite typically includes 4 to 5 rounds. Expect at least one deep SQL round, one coding round (Python, Java, or Scala), one system design round focused on data pipeline architecture, and one or two behavioral or culture-fit rounds. At L5 and above, the system design round gets intense, covering batch vs streaming tradeoffs, failure modes, orchestration, and storage layout decisions. L6 and L7 candidates will also face questions about schema evolution, exactly-once semantics, and cross-team platform strategy.

What metrics and business concepts should I know for a Twitter (X) Data Engineer interview?

Think about the data that powers a social media platform: engagement metrics (likes, retweets, impressions), user growth and retention, ad performance, and content recommendation signals. You should understand how these metrics flow through pipelines and why data freshness, completeness, and accuracy matter for downstream consumers. Being able to talk about SLAs for data delivery and how you'd monitor pipeline health will set you apart, especially at senior levels.

What's the difference between L5 and L6 Data Engineer interviews at Twitter (X)?

L5 (Senior) focuses on system design for data systems, strong SQL and data modeling, and coding proficiency. You need to show you can own a pipeline end to end. L6 (Staff) raises the bar significantly. They'll test you on platform-level design including streaming vs batch tradeoffs, partitioning strategies, exactly-once vs at-least-once semantics, backfill strategies, and schema evolution. L6 also evaluates debugging and operational thinking more heavily. The comp difference is interesting too: L5 averages $340,000 while L6 averages $320,000, though the ranges overlap heavily.

How should I practice coding for the Twitter (X) Data Engineer interview?

Focus on Python first since it's the most common language tested, but having Java or Scala in your back pocket helps. Practice writing clean data transformation code, not just algorithm puzzles. Think about things like processing large files, handling edge cases in ETL logic, and writing efficient data processing functions. For SQL, drill advanced queries with window functions, CTEs, and optimization scenarios. I recommend datainterview.com/coding for targeted practice that matches what you'll actually see in the interview.

Dan Lee's profile image

Written by

Dan Lee

Data & AI Lead

Dan is a seasoned data scientist and ML coach with 10+ years of experience at Google, PayPal, and startups. He has helped candidates land top-paying roles and offers personalized guidance to accelerate your data career.

Connect on LinkedIn