Applying Transformers to Predict Life Course Sequences

Linda Vecgaile, Max Planck Institute for Demographic Research (MPIDR)
Emilio Zagheni, Max Planck Institute for Demographic Research (MPIDR)
Luiz Felipe Vecchietti, Institute for Basic Science
Alessandro Spata , Independent Researcher

This study builds on life course theory, which highlights the interconnectedness and cumulative effects of life events, to enhance predictive modeling of life course sequences. We investigate whether past life event sequences (ages 18-55) can predict future life sequences, such as transitions from employment to unemployment or retirement, during ages 56-60. Using the Transformer encoder-decoder framework, renowned for its ability to analyze sequential data, we develop a model that treats life events like words in a sentence, capturing patterns and temporal dependencies. Tested on data from the German Pension Insurance with 11 distinct social employment states and basic demographics, the model achieves 85.5% accuracy. It excels in predicting outcomes for individuals with stable life paths, while also capturing deviations from recent patterns, reflecting the influence of earlier life experiences. As the model evolves, it aims to identify early life patterns linked to precarious employment outcomes in later stages, offering insights into predictable versus unpredictable trajectories.

Keywords: Computational social science methods, Human Capital, Education, and Work, Inequality, Disadvantage and Discrimination, Data and Methods

See extended abstract.