Adaptive Scheduling of Time-Related Tasks for Sequence Learning

Summary

The document summarizes research on improving a primary sequence-learning task by selecting among related tasks that differ in timing. Examples include translation with different input delays and stock trend prediction over different future horizons. Because these tasks share much of their data and structure, training on selected auxiliary tasks may help the target task, but the challenge is choosing which tasks to use and when.

The proposed method trains a task scheduler alongside the model using bilevel optimization. The scheduler considers the model state and current training data, and is optimized for validation performance; the model minimizes training loss under the scheduler's choices. Reported evidence covers simultaneous translation and stock prediction: translation experiments improved wait-k baselines by 1 to 3 BLEU points, while stock prediction improved baseline Spearman rank correlation and prediction loss. The summary provides no dataset details or evidence about generalization to other markets, assets, or forecasting setups.

Key ideas

• The method treats tasks with different prediction horizons or input delays as related auxiliary tasks.
• A learned scheduler selects auxiliary tasks based on the model state and current training batch.
• Bilevel optimization trains the scheduler for validation performance and the model for training loss.
• Experiments report gains in simultaneous translation and stock trend prediction.
• The document does not provide enough detail to assess performance across other financial datasets or market conditions.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.