Memorization analysis in in-context learning for tabular data

Artificial intelligence is advancing by leaps and bounds, and with it come concerns about how models handle and retain sensitive information. In the realm of tabular data —those that structure information in rows and columns like business databases or financial records— large models (LTMs) that employ in-context learning (ICL) have demonstrated exceptional performance. However, a recent finding has brought attention to a delicate phenomenon: parametric memorization. Unlike contextual memorization, where the model responds using the context provided in the same query, parametric memorization implies that the model has internally stored patterns from the training data, being able to replicate them without needing context. This represents a privacy risk, especially when handling personal or strategic data. To understand this behavior, analysis frameworks such as ICLMEM have been developed, which force the model to rely exclusively on its internal memory, eliminating contextual signals. Experimental results show moderate signs of memorization in certain tasks, especially those with low cardinality or binary ones, and under very specific fine-tuning conditions with many epochs and few queries. Under realistic training conditions, these signals tend to disappear, which partially alleviates the concern. However, the possibility of information leaks persists, and this is where companies must take proactive measures. Implementing artificial intelligence solutions for businesses that include privacy and security controls is essential. At Q2BSTUDIO, we promote an approach that combines custom software with cybersecurity audits and the integration of AWS and Azure cloud services to ensure that models are trained and deployed securely. Additionally, our business intelligence services with Power BI allow monitoring model behavior and detecting anomalies. In an environment where AI agents are beginning to make autonomous decisions, transparency and control over memorization are key to complying with regulations and maintaining customer trust. The development of custom applications that incorporate these principles not only protects data but also optimizes model performance. Parametric memorization in tables is not an insurmountable problem, but it requires attention and adequate tools. At Q2BSTUDIO, we help organizations navigate this challenge by combining technical expertise with a deep understanding of AI ethics.