SEFormer: Encoding Local Structure for Trading Time-Series Models
Summary
The article explains SEFormer, a Transformer design for 3D point-cloud detection that adds spatial structure to attention. Standard convolution encodes direction and distance through its kernels but aggregates neighbors uniformly; ordinary self-attention can filter neighbors but applies the same value transformation to each one. SEFormer combines these properties by choosing value transformations according to relative position. Grid-interpolated neighbors and multiple sampling radii help represent irregular local geometry, while staged feature aggregation produces point-level and object-level embeddings.
The trading implementation adapts the idea rather than reproducing the 3D detector. It proposes learnable centroids for multidimensional market data and builds on an existing point-network class, since a high-dimensional grid would grow rapidly with the number of features. The article reports training and testing on historical data and describes the results as promising, but the available account provides no detailed performance figures. It also notes that the small number of trades and short testing period prevent firm conclusions; longer training and broader validation are needed before live use.
Key ideas
- SEFormer assigns value transformations using the relative positions of neighboring points, adding local structure to attention.
- Attention can adaptively reduce the influence of irrelevant neighbors, while position-dependent transformations preserve directional information.
- Grid interpolation samples neighbors from varied directions, and multiple radii address the limits of a single fixed sampling distance.
- The trading adaptation uses learned centroids because constructing a regular grid becomes difficult in high-dimensional feature spaces.
- The reported trading evidence is limited by a short test period and few trades.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.