Matching Custom Price-Swing Patterns in OHLC Data
Article Quant Q&A · Author: Jack022
Summary
The document describes a workflow for searching OHLC time series for a user-defined pattern. It extracts local highs and lows, represents each turning point as a percentage change from the previous one, and asks how to locate similar sequences in another dataset. Suggested approaches include supervised machine learning, such as a support vector machine, and time-series tools such as dynamic time warping, though the supplied answer focuses on model training rather than giving a concrete matching implementation.
The answer recommends dividing labeled examples into training and test sets and using cross-validation to assess performance. It also points out that the extrema-detection window is a tunable parameter that can lead to overfitting, and suggests hidden Markov models as a possible way to identify changing volatility regimes. These are broad suggestions rather than a demonstrated end-to-end method: the document does not define similarity thresholds, explain how to label examples, or report validation results. Any pattern detector would need careful out-of-sample evaluation before being treated as a trading signal.
Key ideas
- Local extrema can be converted into a sequence of percentage changes to represent price swings.
- A custom swing sequence can be treated as a pattern-recognition problem on other time series.
- Supervised classification requires labeled examples and separate training and testing data.
- Cross-validation can help assess whether a pattern model generalizes.
- The extrema window is a tunable parameter that can cause overfitting, especially as volatility changes.
Tags
Full text
# Detect a specified pattern in a target dataset with Python
# Detect a specified pattern in a target dataset with Python
This question is sort of a continuation of this, but i wanted to share the progress i made and ask for help on the part where i'm stuck.
The short story is that i have a pattern stored in a simple array of data, then i have a dataset and i need to check for occurrences of the pattern i specified on the other dataset.
Here is what i did:
- Get a set of OHLC data on a pandas dataframe
- Compute local minima and maxima for that OHLC data
- Get an array of local minima and maxima
- Normalize the array of minima and maxima by converting it to an array of number, where every number is the variation from the previous point of local minima/maxima.
In terms of code, this is how you can find local minima and maxima on a range:
```
df['min'] = df.iloc[argrelextrema(df.Open.values, np.less_equal, order=n)[0]]['Open']
df['max'] = df.iloc[argrelextrema(df.Open.values, np.greater_equal, order=n)[0]]['Open']
```
Dataframe:
```
Open min max Date
Loc
0 0.000336 0.000000 0.000336 2020-07-06 12:00:00
6 0.000330 0.000000 0.000330 2020-07-06 18:00:00
12 0.000320 0.000320 0.000000 2020-07-07 00:00:00
15 0.000328 0.000000 0.000328 2020-07-07 03:00:00
18 0.000320 0.000320 0.000000 2020-07-07 06:00:00
27 0.000330 0.000330 0.000000 2020-07-07 15:00:00
32 0.000351 0.000000 0.000351 2020-07-07 20:00:00
34 0.000342 0.000342 0.000000 2020-07-07 22:00:00
42 0.000368 0.000000 0.000368 2020-07-08 06:00:00
48 0.000381 0.000000 0.000381 2020-07-08 12:00:00
54 0.000361 0.000361 0.000000 2020-07-08 18:00:00
55 0.000361 0.000361 0.000000 2020-07-08 19:00:00
61 0.000378 0.000000 0.000378 2020-07-09 01:00:00
65 0.000367 0.000367 0.000000 2020-07-09 05:00:00
69 0.000375 0.000000 0.000375 2020-07-09 09:00:00
72 0.000373 0.000373 0.000000 2020-07-09 12:00:00
75 0.000388 0.000000 0.000388 2020-07-09 15:00:00
78 0.000378 0.000378 0.000000 2020-07-09 18:00:00
86 0.000411 0.000000 0.000411 2020-07-10 02:00:00
90 0.000395 0.000395 0.000000 2020-07-10 06:00:00
92 0.000402 0.000000 0.000402 2020-07-10 08:00:00
96 0.000417 0.000000 0.000417 2020-07-10 12:00:00
99 0.000411 0.000411 0.000000 2020-07-10 15:00:00
105 0.000433 0.000000 0.000433 2020-07-10 21:00:00
108 0.000427 0.000427 0.000000 2020-07-11 00:00:00
116 0.000479 0.000000 0.000479 2020-07-11 08:00:00
118 0.000458 0.000458 0.000000 2020-07-11 10:00:00
123 0.000467 0.000000 0.000467 2020-07-11 15:00:00
133 0.000425 0.000425 0.000000 2020-07-12 01:00:00
137 0.000447 0.000000 0.000447 2020-07-12 05:00:00
141 0.000434 0.000434 0.000000 2020-07-12 09:00:00
145 0.000446 0.000000 0.000446 2020-07-12 13:00:00
149 0.000434 0.000434 0.000000 2020-07-12 17:00:00
```
Then convert this dataframe in a simple list of Minima and Maxima: `[0.0003361, 0.0003296, 0.0003197, 0.0003278, 0.0003204, 0.0003301, 0.0003513, 0.000342, 0.000368, 0.0003809, 0.0003611, 0.0003781, 0.000367, 0.0003747, 0.0003727, 0.0003884, 0.0003783, 0.0004105, 0.000395, 0.0004022, 0.0004168, 0.0004107, 0.0004334, 0.000427, 0.0004793, 0.000458, 0.0004668, 0.0004245, 0.0004472, 0.0004344, 0.0004457, 0.0004335]`
And then convert it again to a simple array of percentages:
```
[-1.9339482296935422, -3.00364077669902, 2.533625273694082, -2.2574740695546116, 3.027465667915112, 6.4222962738564, -2.647309991460278, 7.602339181286544, 3.5054347826086927, -5.198214754528746, 4.7078371642204315, -2.9357312880190425, 2.098092643051778, -0.5337603416066172, 4.212503353903944, -2.600411946446969, 8.511763150938416, -3.775883069427527, 1.8227848101265856, 3.6300348085529524, -1.4635316698656395, 5.527148770392016, -1.476695892939546, 12.248243559718961, -4.443980805341117, 1.9213973799126631, -9.061696658097686, 5.347467608951697, -2.8622540250447197, 2.6012891344383067, -2.737267220103202]
```
From the previous Dataset, i extracted a pattern, which is the following:
```
Pattern = [7.602339181286544, 3.5054347826086927, -5.198214754528746, 4.7078371642204315, -2.9357312880190425, 2.098092643051778, -0.5337603416066172]
```
When charted, it looks like that:
Until now, everything works. Now i need to find the pattern in that figure in other datasets. That pattern is made of the following values: `Pattern = [7.602339181286544, 3.5054347826086927, -5.198214754528746, 4.7078371642204315, -2.9357312880190425, 2.098092643051778, -0.5337603416066172]`
So i will need a way to detect the previous pattern on another dataset. For example, if the other target dataset will be that:
```
[-1.9339482296935422, -3.00364077669902, 2.533625273694082, -2.2574740695546116, 3.027465667915112, 6.4222962738564, -2.647309991460278, 7.602339181286544, 3.5054347826086927, -5.198214754528746, 4.7078371642204315, -2.9357312880190425, 2.098092643051778, -0.5337603416066172, 4.212503353903944, -2.600411946446969, 8.511763150938416, -3.775883069427527, 1.8227848101265856, 3.6300348085529524, -1.4635316698656395, 5.527148770392016, -1.476695892939546, 12.248243559718961, -4.443980805341117, 1.9213973799126631, -9.061696658097686, 5.347467608951697, -2.8622540250447197, 2.6012891344383067, -2.737267220103202]
```
How do i find the parts of this dataset that will be most similar to the pattern i defined by myself?
Possible solutions that i don't know how to use: I've been suggested to use stumpy or Python-DTW (Dynamic Time Warping). But for both there aren't examples on this particular matter, so if anyone can help me out on this, it would be a lot appreciated. Any kind of advice, library, example, article on how to solve this problem is appreciated. I've been dealing for a lot on this problem and i feel like i'm only missing the final part to finally solve it
TL;DR I'm trying to find patterns specified by myself on OHLC datasets. To do that, i converted the OHLC data to a set of local minima and maxima. Now i need to understand how to compare a specific pattern to a target dataset and detect where the dataset is most similar to the pattern i specified.
## Answer by John (score 2)
https://quant.stackexchange.com/a/55605
your problem is pattern recognition. considering you already identified the desired output pattern (entry/exit points), you can use supervised methods of machine learning to train.
many are available, a support vector machine for instance, recommend you to check the scikit learn module out, it has practical and fast implementations.
you would have to divide your sample between training and testing, possibly increasing its effectiveness with cross validation methods, e.g. k-fold
also you might want to note that the order=n in the argrelextrema function, introduces a parameter which makes the learning process vulnerable to overfitting.
to avoid this, and considering the volatility of your time series might be variable (optimal n might vary), you could consider using hidden markov models for regime changes identification.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.