⚡ Streaming lakehouse pipelines
Kafka → Flink → Iceberg ingestion for high-volume sensor and process data, sustaining roughly 3M records/sec into a petabyte-scale lake. Benchmarked Iceberg against Hudi across insert, upsert, query and maintenance workloads before committing to the table format; built out the Flink Kubernetes Operator deployment, and Spark (Scala) maintenance jobs — compaction, sorting, snapshot expiry — orchestrated on Airflow.

🔎 Query engine performance
A 100+ node Trino cluster fields 300+ queries per minute, individual queries scanning terabytes. Most of the wins came from the storage side — partition strategy, sort order, compaction cadence — and the rest from the query side: pushdown improvements and approximate aggregations where exactness wasn't required. This architecture was the subject of my Trino Summit 2023 talk.

☸️ Multi-tenant runtime on Kubernetes
A self-service platform that provisions per-user Spark and StarRocks clusters on demand, operating thousands of them concurrently: custom operators and CRDs, a Java/Spring orchestration service, Helm packaging, ArgoCD GitOps, and Prometheus/OpenSearch observability. Running an OLAP engine as a multi-tenant service — rather than as one shared cluster — is where most of my engine-internals work comes from.

🤖 Agents for platform operations
An agent that consumes query history and cluster state snapshots off Kafka, diagnoses resource, configuration and query-level problems, and proposes concrete fixes — applied only after human approval. Deterministic rules and cost models do the heavy lifting; the model handles classifying unseen symptoms, correlating signals, and rewriting queries.

🧠 Domain-expert agents
A knowledge-graph-driven agent for operational troubleshooting, structured as graph selection → graph traversal → path evaluation: LLM-based graph selection to cover the gaps in vector similarity search, hierarchical filtering over a partitioned graph, parallel sub-agents that traverse branches and prune infeasible paths early, self-correcting tool-call loops, and a final helpfulness/groundedness evaluation that picks or aggregates the answer path. Includes Text-to-SQL over Oracle and Postgres, and on-prem inference tuning with vLLM (speculative decoding, prefix caching).

🎯 Domain-specialized LLM
Fine-tuned SK Telecom's A.X foundation model (32B) with DeepSpeed and TRL into an HR-domain assistant, reaching ~90% of the then-SOTA general model's quality on evaluations scored by domain experts — at a fraction of the serving cost.

Efficient Kappa Architecture with Trino Trino Summit 2023 One pipeline instead of two — serving terabyte-scale queries over data that is still arriving, on Kafka, Flink, Iceberg and Trino.

Building Domain-Expert Agents AI Community Conference (AICO) Seoul 2025 Moving an agent from general capability to domain expertise — knowledge graphs over flat retrieval, traversal with early pruning, and grading the path rather than only the answer.

Fixes to the query engines and catalogs we run in production.

Submitted

Project	PR	Area	Fix	Status
StarRocks	#75977	Optimizer	Heavy-expression pushdown produced wrong plans when a heavy expression referenced a heavy common sub-expression
StarRocks	#75976	Connector	Relation aliases were resolved case-sensitively for connector views, breaking queries that were valid upstream
StarRocks	#76483	Security	SHOW CREATE CATALOG leaked S3 and REST credential keys for Iceberg catalogs
Apache Polaris	#5247	Auth	A metastore failure while resolving a principal's roles propagated unwrapped and surfaced as HTTP 500, leaving a transient backend outage indistinguishable from a server defect — now 503, as the principal lookup on the same path already did

In progress

_{B.S. in Information Systems, Hanyang University}

Streaming & batch
Lakehouse & query
AI & LLM
Platform
Languages