Historical Language Models (LLMs), like Talkie-1930, simulate the collective consciousness of a specific era based on historical texts. While not a replacement for primary sources, these models offer unique insights into the past, enabling researchers to explore historical perspectives and epistemologies. However, it’s crucial to recognize their limitations, such as their inability to access the inner lives of historical figures.

The author proposes using fine-tuned historical language models to explore the “mental furniture” of historical figures and eras, rather than attempting to replicate their thoughts. This approach could be used to probe counterfactuals and hypothetical scenarios, such as whether a model trained only up to 1911 could independently discover General Relativity. Additionally, the author suggests using different texts for post-training to create vintage models with distinct conversational personas, and using archival sources to simulate plausible composites of ordinary people from the past.