583.OpenIA scientist Ilya Sutskever

Reader86

https://en.wikipedia.org/wiki/Ilya_Sutskever

Reader86

Researchers have been exploring methods to modify neural network weights directly to efficiently incorporate new information without extensive retraining. One notable approach is task vectors, which involve computing the difference between the weights of a pre-trained model and the same model fine-tuned on a specific task. These vectors can then be added or subtracted from the weights of other models to transfer or remove task-specific knowledge, enabling efficient model editing.
ARXIV

Additionally, learning in deep weight spaces (DWS) has emerged as a research direction where neural networks process the weights of other networks. This approach has applications in analyzing and editing neural fields and implicit neural representations. However, models in weight space often suffer from overfitting, and recent studies have focused on developing data augmentation techniques to improve generalization in these scenarios.
ARXIV

While these methods show promise, they are still under active investigation, and practical, widely adopted solutions for direct weight editing remain limited.

Reader86

You might be thinking of "weight editing" or "direct data injection into model weights", which Ilya Sutskever has discussed as an alternative to traditional training. Instead of feeding raw data into a model through conventional training (like fine-tuning or reinforcement learning), this method would involve directly modifying the neural network’s weights to encode knowledge more efficiently.

This approach is an attempt to make AI learn like a human—absorbing information quickly without needing to process massive datasets over many training cycles. It could lead to models that:

Learn specific facts instantly without needing retraining.

Improve efficiency by skipping redundant learning.

Reduce hallucinations by embedding more structured knowledge.

Does this sound like what you were referring to?