Apple's machine learning research team has published a paper introducing LINK, a data-level intervention that improves cross-lingual knowledge transfer during model pretraining by swapping words in high-resource language corpora with their translations from bilingual vocabularies. The method requires no additional model training, parallel data, or auxiliary systems — only a bilingual vocabulary that can be obtained at near-zero cost for virtually any language.

The approach targets a persistent bottleneck: when target language data is scarce, models must acquire reasoning, commonsense, and world knowledge primarily from high-resource languages like English. Existing transfer methods typically demand large parallel corpora, translation pipelines, or extra training stages that are unavailable for many low-resource languages. LINK sidesteps those requirements by performing lexical substitutions directly in the pretraining data.

What's new

For a given replacement ratio, LINK randomly selects words in a portion of the English training corpus and replaces them with word-level translations from a bilingual vocabulary. The modified corpus is then used for standard pretraining. The researchers evaluated the method on eight languages across five model sizes and report notable improvements on downstream tasks in the target languages, with up to a 2x speedup in training to reach equivalent performance compared to baselines without the intervention.

The paper, titled "Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions," lists Anastasiia Sedova, Natalie Schluter, Skyler Seto, and Maartje ter Hoeve as authors, with equal contribution noted for the latter three. It was published on Apple's Machine Learning Research site in August 2026 and is also available as arXiv:2605.23885, submitted on 22 May 2026.

Why it matters

Low-resource languages cover a significant share of the world's population but remain underserved by large language models because the data and infrastructure required by current transfer techniques are often out of reach. A method that works with only a bilingual vocabulary — resources that exist for thousands of language pairs — could dramatically lower the barrier to building capable multilingual models. The 2x training speedup also translates directly into compute savings, which matters for research teams and organizations operating under fixed GPU budgets.

Our take

LINK's strength is its minimalism: it moves the complexity from model architecture and training pipelines into a simple data preprocessing step that any team can replicate. The trade-off is that word-level substitution cannot capture phrase-level or syntactic transfer, so gains may plateau on tasks requiring deeper structural alignment. Still, for the many languages where even a bilingual dictionary is the only cross-lingual resource available, this is a practical advance worth adopting as a baseline.

Sources