Skip to content
All library documents

Qlib Meta Controllers for Guiding Forecasting Models Across Tasks

Article Qlib

Summary

The document explains Qlib’s meta-controller framework for learning patterns across forecasting tasks and using them to guide future tasks. A meta-task packages data for a meta-model, while a meta-dataset manages how task information is generated and supplies tasks for training. The framework exposes a general meta-model workflow with fitting and inference, alongside two specialized types: meta-task models that modify task definitions and meta-guide models that influence base-model training.

The example, DDG-DA, illustrates the sequence: create meta-information and tasks, assemble them into a dataset, train the meta-model, infer guidance, and apply that guidance to forecasting models. The document describes an architecture and usage concepts rather than reporting comparative results, trading performance, or implementation caveats. It therefore serves as an overview for researchers building task-adaptive forecasting workflows; the potential performance improvement is stated as the goal, not demonstrated evidence in this text.

Key ideas

  • Meta-tasks package information that meta-models can use across forecasting problems.
  • A meta-dataset manages task generation and provides tasks for model training.
  • Meta-task models can modify task definitions, while meta-guide models can influence base-model training.
  • DDG-DA demonstrates a workflow from task creation through guidance inference and application.
  • The document outlines the framework but provides no performance evaluation.

Tags

Machine learningEquitiesStatisticsPortfolio construction
Full text
# meta


.. _meta:

======================================================
Meta Controller: Meta-Task & Meta-Dataset & Meta-Model
======================================================
.. currentmodule:: qlib


Introduction
============
``Meta Controller`` provides guidance to ``Forecast Model``, which aims to learn regular patterns among a series of forecasting tasks and use learned patterns to guide forthcoming forecasting tasks. Users can implement their own meta-model instance based on ``Meta Controller`` module.

Meta Task
=========

A `Meta Task` instance is the basic element in the meta-learning framework. It saves the data that can be used for the `Meta Model`. Multiple `Meta Task` instances may share the same `Data Handler`, controlled by `Meta Dataset`. Users should use `prepare_task_data()` to obtain the data that can be directly fed into the `Meta Model`.

.. autoclass:: qlib.model.meta.task.MetaTask
    :members:

Meta Dataset
============

`Meta Dataset` controls the meta-information generating process. It is on the duty of providing data for training the `Meta Model`. Users should use `prepare_tasks` to retrieve a list of `Meta Task` instances.

.. autoclass:: qlib.model.meta.dataset.MetaTaskDataset
    :members:

Meta Model
==========

General Meta Model
------------------
`Meta Model` instance is the part that controls the workflow. The usage of the `Meta Model` includes:
1. Users train their `Meta Model` with the `fit` function.
2. The `Meta Model` instance guides the workflow by giving useful information via the `inference` function.

.. autoclass:: qlib.model.meta.model.MetaModel
    :members:

Meta Task Model
---------------
This type of meta-model may interact with task definitions directly. Then, the `Meta Task Model` is the class for them to inherit from. They guide the base tasks by modifying the base task definitions. The function `prepare_tasks` can be used to obtain the modified base task definitions.

.. autoclass:: qlib.model.meta.model.MetaTaskModel
    :members:

Meta Guide Model
----------------
This type of meta-model participates in the training process of the base forecasting model. The meta-model may guide the base forecasting models during their training to improve their performances.

.. autoclass:: qlib.model.meta.model.MetaGuideModel
    :members:


Example
=======
``Qlib`` provides an implementation of ``Meta Model`` module, ``DDG-DA``,
which adapts to the market dynamics.

``DDG-DA`` includes four steps:

1. Calculate meta-information and encapsulate it into ``Meta Task`` instances. All the meta-tasks form a ``Meta Dataset`` instance.
2. Train ``DDG-DA`` based on the training data of the meta-dataset.
3. Do the inference of the ``DDG-DA`` to get guide information.
4. Apply guide information to the forecasting models to improve their performances.

The `above example <https://github.com/microsoft/qlib/tree/main/examples/benchmarks_dynamic/DDG-DA>`_ can be found in ``examples/benchmarks_dynamic/DDG-DA/workflow.py``.

DDG-DA uses restricted loading by default for recorder artifacts and local
handler/internal-data pickle caches. To reload executable meta-models, tasks and
caches from a trusted writer and access-controlled storage, configure
``DDGDA(..., trusted=True)`` at the workflow entry point.
Lower-level callers can configure ``MetaDatasetDS`` or ``InternalData.setup`` with
the same option for recorder task reads. Prediction, label and numerical-report reads
remain restricted. DDG-DA's opt-in also authorizes its handler/internal-data
cache reads, so verify ``working_dir``, the configuration directory and any
explicit ``h_path`` as well as the MLflow store. There is no automatic unsafe retry
and the global restricted loader is unchanged.
See :ref:`artifact_loading_migration`
and the example README for CLI commands.

Shown in full with attribution under the source's licence. Licence: MIT

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.