Chen Qian

Chen Qian

AI Researcher and Engineer. Primary maintainer of DSPy. Founding engineer of Keras 3 and KerasHub.

Email GitHub Google Scholar Twitter/X

I am an AI researcher and engineer focused on making AI systems reliable, efficient, and accessible. My work spans in-context learning, AI system evaluation, and language model authoring and training. My goal is to enable large-scale AI deployment across all industries, not just tech. Most of my work is open source and available on my GitHub.

Over the past 7 years, I have worked at Google and Databricks. I am currently transitioning from engineering to research to focus on open-ended questions in AI, such as automatic prompt optimization and AI evaluation with minimal human effort.

Research Interests

  • In-context learning: Automatic prompt optimization, context management and compression, and language model understanding through ICL.
  • AI system evaluation: Benchmarks and evaluation frameworks for domain-specific applications (hardware design, scientific reasoning, finance), with a goal of minimizing human effort during the evaluation process.
  • Compound AI systems: Architecture for AI agents and enhancing retrieval quality in AI systems.
  • Efficient training and inference: Reducing communication overhead in distributed systems and new model/layer architectures for computational efficiency.

Teaching & Talks

Publications

Open Source Contributions

DSPy
⭐ 29.1K
Framework for algorithmically optimizing LM prompts and weights, especially when LMs are used one or more times within a pipeline.
  • Primary contributor and maintainer since mid 2024
  • Led the development of DSPy 3, including async DSPy, streaming, cache, and customizable adapter/LM
  • Collaborated with researchers to land SIMBA, GEPA, and GRPO optimizers
Keras
⭐ 63.5K
Multi-backend deep learning framework, with support for JAX, TensorFlow, and PyTorch.
  • Founding engineer of Keras 3
  • Unified multi-backend API across TensorFlow / PyTorch / JAX
  • Led implementation of multi-backend layers and performance benchmarking
  • Designed and implemented new Keras optimizers with clean user journey and robustness in distributed training
KerasNLP
(rebranded as KerasHub)
Natural language processing library that works natively with TensorFlow, JAX, or PyTorch.
  • Founding engineer
  • Led design and implementation of tokenizers, transformer blocks, and trainer classes
  • Architected generative models and sampling APIs, including KV cache support
MLflow
⭐ 22.5K
Open-source platform to streamline machine learning development, including tracking experiments, packaging code, and sharing models.
  • Core architect of deep learning support
  • Developed metrics/run grouping, system metrics, and experiment resuming features

Work Experience

Databricks - AI Engineer
July 2023 - Present
Working on DSPy and MLflow in the AI Open Source team. Co-led the development of DSPy 3 as primary maintainer. Integrated DSPy into Databricks AI products like agent bricks. Led MLflow improvements for deep learning workflows, including large-experiments compatibility and system metrics. Migrated all Databricks training workloads from Weights & Biases to MLflow.
Google - Machine Learning Engineer
January 2021 - July 2023
Worked on the Keras team. Founding engineer of Keras 3, unifying multi-backend API across TensorFlow, PyTorch, and JAX. Founding engineer of KerasNLP (rebranded to KerasHub in 2024). Led Keras optimizer rewriting and migrated Google's entire Keras optimizer codebase to the new optimizer.
Google - Backend Engineer
August 2018 - January 2021
Worked on Google Local Services Ads team. Led the effort of phone/message support for i18n. Led the effort of smart geo targeting.