Date Created: April 3, 2026

Code: https://github.com/qyxu1994/deep-cross-net

The Deep & Cross Network (DCN) is a neural architecture designed to automatically learn explicit feature interactions of bounded degrees without manual feature engineering. It was introduced by researchers at Google and Stanford in 2017 for large-scale CTR prediction.

1. Core Motivation

Traditional linear models (e.g., Logistic Regression) require manual feature crossing. Factorization Machines (FM) automate this but are limited to 2nd-order interactions and cannot capture higher-order crosses efficiently. DNNs can learn high-order interactions implicitly, but they often require many parameters and fail to learn certain types of feature crosses efficiently.

DCN addresses this by introducing a Cross Network that explicitly applies feature crossing at each layer, with the highest polynomial degree growing linearly with layer depth. This is combined with a standard Deep Network (DNN) for implicit feature learning.

2. Overall Architecture

DCN consists of three main components:

1. Embedding and Stacking Layer – Converts sparse categorical features into dense vectors and stacks them with dense features
2. Parallel Networks – Cross Network (explicit, bounded-degree feature crossing) + Deep Network (implicit, high-order nonlinear interactions)
3. Combination Layer – Concatenates outputs from both networks and applies a sigmoid activation for final prediction

3. Mathematical Formulation

3.1 Embedding and Stacking Layer

For each categorical feature $i$, we learn an embedding matrix $W_{\text{embed},i} \in \mathbb{R}^{n_e \times n_v}$ :

$$

\mathbf{x}{\text{embed},i} = W{\text{embed},i} \cdot \mathbf{x}_i

$$

where: