研究 02 · 表示学习与贝叶斯优化Research 02 · Representation learning

用 AI 筛选抗 HBsAg 抗体Screening anti-HBsAg antibodies with AI

抗体库里有上亿条序列，能送去做实验的只有几十条。把问题写成数学，它是高维空间里的函数逼近加排序。An antibody library holds hundreds of millions of sequences; only a few dozen can be tested at the bench. Written as mathematics, the task is function approximation in a high-dimensional space, followed by ranking.

序列建模Sequence modeling
迁移学习Transfer learning
贝叶斯优化Bayesian optimization
不确定性量化Uncertainty quantification

问题的数学形式The problem in mathematical form

一条抗体序列是一串离散符号，每个位置从 20 种氨基酸里取一个。An antibody sequence is a string of discrete symbols, each position drawn from the 20 amino acids.

x = (x₁, x₂, …, x_L),x_i ∈ A

A 是 20 种氨基酸的字母表，L 是序列长度。单链抗体（scFv）约 250 个氨基酸，纳米抗体（VHH）约 120 个。要找的是一个打分函数 f，它给每条序列一个实数，分数越高，与 HBsAg 的结合亲和力越强。Here A is the alphabet of 20 amino acids and L the length of the sequence: about 250 residues for an scFv, about 120 for a VHH. What we want is a scoring function f that assigns each sequence a real number, higher meaning stronger binding affinity to HBsAg.

所谓「AI 筛选」，在数学上就是在一个巨大的离散序列空间上用 f 排序，取前 k 名。"AI screening" then means ranking a very large discrete sequence space by f and taking the top k.

从字母到向量From letters to vectors

模型无法直接处理字母，序列须先嵌入连续空间：φ 把每条序列变成一个 d 维实向量，d 常取 256、768 或 1024。这一步由蛋白语言模型完成，ESM-2、AbLang2、AntiBERTy 都属于这类。它们内部是一层层注意力。A model cannot read letters, so the sequence is embedded in a continuous space: φ turns each sequence into a real vector of dimension d, commonly 256, 768 or 1024. Protein language models do this step, among them ESM-2, AbLang2 and AntiBERTy. Inside, they are stacks of attention.

Attention(Q, K, V) = softmax(QK^⊤√d_k)V

注意力捕捉的是残基之间的长程依赖：哪些位置的氨基酸在空间上相互影响。序列上离得远，折叠之后可能贴在一起。Attention captures long-range dependence between residues, which positions influence each other in space. Two residues far apart along the chain can end up touching once the protein folds.

亲和力预测是一个回归Affinity prediction is a regression

有了嵌入，打分函数接在它后面，通常是一个小的多层感知机。With the embedding in hand, the scoring function sits on top of it, usually a small multilayer perceptron.

f_θ(x) = g_θ(φ(x))min_θ Σ_i (f_θ(x_i) − y_i)²

标签 y_i 是实验测得的亲和力，例如 −log K_d。到此为止都是标准的监督回归，困难在数据：带亲和力标签的 HBsAg 抗体很少，公共数据库里可能只有几十条。直接训练会过拟合。The label y_i is a measured affinity, for instance −log K_d. So far this is standard supervised regression, and the difficulty is the data: labelled anti-HBsAg antibodies are few, perhaps a few dozen in public databases. Training on them directly overfits.

标签稀缺时的做法Working around scarce labels

先在无标签序列上做自监督预训练。OAS 里有十亿量级的抗体序列，常用目标是掩码语言建模：随机遮住一些位置，让模型把它们猜回来。Pretrain on unlabelled sequences first. OAS holds antibody sequences on the order of a billion, and the usual objective is masked language modeling: hide some positions at random and have the model guess them back.

L_MLM = − Σ_{i ∈ mask} log p_θ(x_i | x_∖i)

这一步让 φ 学到「什么样的抗体序列是合理的」。再拿少量带标签的数据微调，同时加一项正则，把参数拴在预训练值附近，以免少量样本把参数带偏。This teaches φ what a plausible antibody sequence looks like. Fine-tuning then uses the small labelled set, with a penalty tying the parameters near their pretrained values so that a few dozen examples cannot drag the model away.

θ* = argmin_θ Σ_i (f_θ(x_i) − y_i)² + λ ‖θ − θ_pre‖²

另一条路是对比学习：让结合同一表位的抗体在嵌入空间里靠近，不结合的推远。学出来的空间可以直接用最近邻搜索找候选，不必先有回归模型。A second route is contrastive learning: pull antibodies that bind the same epitope together in embedding space and push non-binders apart. The resulting space can be searched by nearest neighbours directly, with no regression model needed first.

从打分到决策From scores to decisions

序列库可能有 10⁸ 条，逐条精算的代价太高。通常分两阶段：先用嵌入向量做近似最近邻粗筛，再用 f_θ 精排。A library may hold 10⁸ sequences, too many to score carefully one by one. The usual arrangement has two stages: a coarse pass by approximate nearest neighbour search over embeddings, then a fine ranking with f_θ.

湿实验那一步可以看成对一个黑箱函数的查询，很贵，次数有限。用高斯过程给它建模，每次选采集函数最大的点去做实验，这就是贝叶斯优化。The wet-lab step can be seen as querying an expensive black-box function a limited number of times. Model it with a Gaussian process and test whichever point maximizes an acquisition function: this is Bayesian optimization.

只给一个分数并不够，还需知道模型有多确定。深度集成或贝叶斯神经网络给出预测均值 μ 与方差 σ²，筛选策略于是变成：A score alone is not enough; we also need to know how sure the model is. A deep ensemble or a Bayesian neural network gives a predictive mean μ and variance σ², and the selection rule becomes:

x_next = argmax_x [ μ(x) + β σ(x) ]

既选预测高的，也选模型把握不大的。后者或许带来意外，也可能白费一次实验，β 正是在两者之间定价。It picks both the promising candidates and the ones the model is unsure about. The second kind may hide a surprise or may waste an experiment, and β is where that trade is priced.

探索权重 βExploration weight β

八十六条候选，各有一个预测亲和力与一段不确定度。拖动 β，观察入选的八条如何更替。图中数值为演示所设，并非实测。Eighty-six candidates, each with a predicted affinity and a band of uncertainty. Drag β to see which eight are selected. The values are set for the demonstration, not measured.

整条流程The pipeline

步骤Step	数学操作Mathematical operation	实际动作What it means in practice
1	序列嵌入 φ(x)Embed the sequence, φ(x)	用蛋白语言模型把序列变向量Turn sequences into vectors with a protein language model
2	自监督预训练Self-supervised pretraining	在无标签抗体序列上做掩码语言建模Masked language modeling on unlabelled antibody sequences
3	回归微调Fine-tune the regression	用少量已测亲和力的 HBsAg 抗体校准Calibrate on the few HBsAg antibodies with measured affinity
4	排序取前 kRank and take the top k	粗筛加精排，从库里挑候选Coarse pass then fine ranking over the library
5	不确定性 μ + βσUncertainty, μ + βσ	决定哪几条送去做噬菌体展示Decide which few go to phage display
6	贝叶斯更新Bayesian update	实验结果回灌，模型再算一轮Feed results back and let the model run again

整件事的数学内核，是在抗体序列的高维离散空间上学一个从序列到亲和力的回归函数，再用它排序和决策。标签稀缺，所以依赖预训练嵌入和半监督；实验昂贵，所以要用贝叶斯优化在探索与利用之间反复取舍。At its mathematical core the work learns a regression from sequence to affinity over a high-dimensional discrete space, then ranks and decides with it. Labels are scarce, hence the pretrained embeddings and semi-supervised training; experiments are expensive, hence Bayesian optimization weighing exploration against exploitation round after round.