Multithreading in MQL

After the last unsolved mystery of MT4 tester peculiarities here comes the next. Wonder if forum gurus could take a look and give their wise opinions.

After noticing some promising posts and links here on FFT and SSA and Alglib I decided to try myself as well. So, I integrated Alglib to my dll. Got also the commercial version which allows you to have Intel MKL (math & kernel library) easily included to boost the performance with multithreading. Got it running and it gave the same results as the free version. But there was no speedup in tester run or optimization. However, if I integrate their test code as such to my init routine I WAS able to see that when increasing the n (in matrix multiplication) and worker (how many cores included) amount the time spent (measured now with QueryPerformanceCounter) went down. This is again the same result I saw earlier with TBB. Whatever I try with multithreading no overall speedup can be gained!!!

So, if there’s ANYONE on this forum who can tell pure facts how they got the overall tester execution (or optimization) faster with mulithreading I’d be greatly appreciate. I have seen discussions and reports but most of those don’t show overall speedup. And I’m not interested in MT5 tester capabilities. I’d only want to know why mt4 tester cannot be speeded up with multithreading. Or if you have successully made CUDA integration (or OpenCL) and got it faster with that please let me and us know the numbers.

I used SSA from Alglib on three timeframes to give entry points.

My understanding of MT4 is that it’s limited to a single core.

So, what does it mean to have multi-threading limited to one core? Context switching, sure, but no overall improvement of speed because the threads are not running in parallel.

Now, if your DLL can spawn to the other hardware cores, please disregard my rudimentary comment because I don’t fully understand your setup.

yes it can and as I said it was proven. But why no overall speedup? Let’s ask then the other way around … is there anybody else admitting to have failed in trying it?

Why do you use alglib as a dll ?

SSA was not in mql version. At least in the one I saw at first. Also, MSVC can produce more efficient and faster code.

I love wine.

SSA & TBB, dude, you’re working on the EA of the millenium !

So what I suggest - it’s worth what it’s worth - is to first code something very standard, something poor, basic but inescapably profitable.

I’m sure you already know how to do it.

Only after, once remunerated, you’'ll be free to pursue your researches.

See it as a self financed project.

You could use opencl. Here’s an opencl lib by dingmaotu.

Precisely. I did work on neural networks with alglib. When it comes to dll, you’re directly penalized by the portability, so I did it with the alglib available for mql5.

What I’m sure of - because I did it - is :

  1. Or your neural network is pre-trained on 10+ years for a pair, OHLC + 10 indicators, hardcoded & run - there’s plenty of that kind on the market (IP*)
  2. Or you want to self train it, where you’d require memory for calculation, but data selection should be the neatest the better - I mean you won’t do it with OHLC price + the value of 10 indicators, and 10 neurons on the fly, with or without opencl, with or without dll, at each tick
  3. With the last 10 bars of M1, there’s enough to do a very nice scalper
  4. Some may notices that neural network doesn’t make the grail, you might want to give a try with fuzzy logic - it works differently - I worked a little on it, I got this, I’m sure I did not exploit the full potential but the advantage on neural network is that you won’t need plenty of data for analysis :

It’s still in my task list, because it abuses of so much ordermodify/ordersend/orderdelete - which is extremely heavy for mem - i feel like it’s random orders ! I definitively have to refine the algo.

Mikko Siltanen:
Interesting. Thanks for sharing. I’m using fann library - didn’t use the mql2fann but used fann lib directly from my dll. Don’t have any issues with performance as I could get forecasts from SSA only for new bar, couldn’t replace the last which would blow the perf requirements in ctrl point and every tick testing (both for SSA and fann). But maybe it’s no problem since I can do any amount of smaller timeframes anyway. Could try less than 1M but only if this would work on higher timeframes (and 1M). Can you show what kind of performance you were able to get in forward testing?
Fuzzy logic - no idea - totally fuzzy for me. How did you applied it?
Everyone of course have their own issues but also in many cases it’s easy to make wrong conclusions what takes time and CPU or mem. In my case I have not seen any issues with those orderhandling functions. I don’t have any indicators included mainly because the interface to iCustom is so poor, maybe it’s better in mql5 but didn’t go there. I’ve moved SMI to dll and use that along with angles for order closing (and actually also for entries).

Sure. Here’s the forward test 1/2 from 2017.01.01 'til now.

It’s stable.

It’s not well documented in mql, in finance generally talking so I got to watch out c/c# example to understand well how it could be applied and I said previously even if profitable I’m not satisfied on how it process. I leave it parked, test flight not for now.