🧭 Geometry Deep Learning

Point Cloud Learning

Operate directly on unordered 3D points with permutation‑invariant layers (e.g., shared MLPs + symmetric pooling). Useful for LiDAR/3D vision, robotics, molecular surfaces.

// Minimal PointNet‑style block (PyTorch‑like pseudo‑code)
features = MLP(point_xyz)
pooled   = symmetric_pool(features, mode="max")
classlogits = MLP_head(pooled)

• Common ops: kNN/ball query, FPS sampling, graph construction.
• Augmentations: jitter, scaling, random rotations.

Message Passing GNNs

Nodes aggregate messages from neighbors along edges, enabling learning over molecules, meshes, social networks, and knowledge graphs.

// Message passing step
e_i = phi_e(h_u, h_v, x_uv)
h_v' = phi_v(h_v, ⨁_{u∈N(v)} e_i)

• Popular layers: GCN, GAT, GraphSAGE, GIN.
• Tricks: residual connections, degree norm, positional encodings.

DTNN for Molecules

Latent embeddings per atom updated via learned pairwise interactions based on inter‑atomic distances; effective for quantum‑chemistry property prediction.

// Distance‑based interaction (schematic)
r_ij = ||R_i - R_j||
v_j = v_j + f_embed(Z_i) ⊙ g(r_ij)

• Inputs: atomic numbers Z, positions R; targets: energies, forces.
• Often combined with cutoffs and radial basis expansions.

Generative Models for Geometric Data

Diffusion, autoregressive, and normalizing‑flow models synthesize 3D shapes, molecules, or graphs while respecting symmetry and constraints.

• Key ideas: SE(3)‑equivariance, graph constraints, score matching.
• Use cases: molecule design, mesh synthesis, scene completion.

Transformers on Sets, Sequences, and Graphs

Self‑attention captures long‑range interactions; with proper positional encodings it adapts to sets, sequences, and graphs (e.g., Graph Transformers, 3D equivariant attention).

// Scaled dot‑product attention
A = softmax(QK^T / √d_k) V

• Variants: Performer, Linformer, Gated‑attention, GraphGPS.
• Tips: masking, relative/rotary encodings, neighborhood sparsity.

LSTM for Temporal & Ordered Geometric Data

Handles long‑term dependencies in trajectories and ordered sets (e.g., protein backbones, robot paths) and can be hybridized with attention and GNN layers.

// One LSTM step (schematic)
f_t, i_t, o_t = σ(Wx_t + Uh_{t−1})
ĉ_t = tanh(Wc x_t + Uc h_{t−1})
c_t = f_t ⊙ c_{t−1} + i_t ⊙ ĉ_t
h_t = o_t ⊙ tanh(c_t)

• Use with: sequence of graph snapshots, skeleton pose sequences.
• Enhance with: residuals, layer norm, dropout, bidirectionality.