日本語
Panel data, with multiple observations for consistent subjects over time, is ubiquitous in various domains. While panel data is often spread across multiple tables, it can also exist in a single dataset with multiple features suitable as panel dimensions. The self-join technique enables automated, time-aware feature engineering with just one dataset, generating hundreds of candidate features of lagged aggregations and statistics. Combining these features within panel dimensions can substantially improve predictive model performance.
Panel data analysis unlocks valuable insights into subjects evolving over time, and is often overlooked when there is a singular dataset. This technique is a favorite amongst advanced users and data scientists at DataRobot, and we are excited to share it with you. Enjoy!
About this Accelerator
In this accelerator, we'll explore how to implement self-joins in panel data analysis. Regardless of your industry, if you work with panel data, this guide is tailored to help you accelerate feature engineering and extract valuable insights.
We'll focus on predicting airline take-off delays of 30 minutes or more to illustrate the self-join technique, however, this framework applies broadly across verticals and can easily be adapted to your use case. Using a single dataset, we'll join it four times across different features, engineer time-based features from each join, using the AI Catalog for data management.
This post assumes basic familiarity with machine learning experiments in DataRobot, as we'll delve into the intricacies of the self-join technique.
What you will learn
- Data preparation with multiple joins and time horizons
- How to mitigate target leakage with multiple feature lists
- Time gaps in time-aware joins
