Multi Core/Thread MGU

Moderators: Edgecrusher, Vladdy

Post Reply
Azool
SMR Coder
Posts: 1736
Joined: Sat Feb 23, 2002 8:42 pm
Location: Colorado Springs, CO
Contact:

Multi Core/Thread MGU

Post by Azool »

I created a multi thread version (for route finder...everything else is still single thread) and improved the efficiency of a couple areas (significantly in some places).

http://www.azool.us/smr/MGUMultiThread.exe

NOTE: Don't consider this an official release. I really don't plan on supporting it. I mostly created it for myself since the slow search was irritating me and just decided I would share it. So if it doesn't work for you, let me know, but it isn't a huge priority for me.

Just put it in the same directory as your other MGU.exe (but make sure you run MGUMultiThread.exe instead of just MGU.exe)

If you get errors using the multi threaded version just start using MGU.exe again instead of the new exe you downloaded. (Eventually I'll probably just add a checkbox or a textbox that allows you to enter how many threads you want it to run)

The performance improvement is minimal when you are just searching single galaxies or small subsets of conditions (due to lock competition and such between the threads); however, once you start searching for larger subsets, you should notice a significant improvement.

Searching the current universe racials + omar with full races and non-illegal goods (and creating the sma file) in the single thread version took almost 1 hour
With the multi threaded version (4 threads) it completes in about 9 seconds.
There are still a ton of inefficiencies in the actual route finder algorithm if anyone is interested in improving it more (especially the 3-port finder)

Additionally due to the extra threads, the trade route window shouldn't freeze anymore, so you should have a more accurate progress bar and you will actually be able to watch it update.

Let me know if you find any issues

Somewhat known issue: Very rarely there is a thread competition and it rails the RAM usage extremely high (90% of whatever you have in your system, so potentially upwards of 6-12GB). If this happens you will probably have to force close the program and restart it. In all my testing this happened only once and I haven't been able to reproduce it, so hopefully it is actually fixed.
Men are born to succeed, not fail.
-Henry David Thoreau
Page
SMR Coder
Posts: 1779
Joined: Sat Dec 07, 2002 9:17 pm

Re: Multi Core/Thread MGU

Post by Page »

I haven't looked at the MGU route finder, but I know for TheLazyTrader I didn't need any locks for route finding?
Jester-
Admin Emeritus
Posts: 1660
Joined: Mon Feb 17, 2003 11:29 pm
Location: Indianapolis, IN

Re: Multi Core/Thread MGU

Post by Jester- »

Yeah but LT is just not up to snuff. lol
Image
Image
Azool
SMR Coder
Posts: 1736
Joined: Sat Feb 23, 2002 8:42 pm
Location: Colorado Springs, CO
Contact:

Re: Multi Core/Thread MGU

Post by Azool »

Page wrote:I haven't looked at the MGU route finder, but I know for TheLazyTrader I didn't need any locks for route finding?
Yeah, MGU uses the sector object to track distance from a port (1 single variable, not an array). So each thread tries to update the variable for the port it is calculating, which then causes MGU to explode (it keeps adding sectors to the queue for distance and the RAM usage jumps). I ended up just making it a multidim array, but I needed the lock to make sure 1 thread wouldn't pull a partially full array to make the calcs (which would then cause it to hang because MGU recalculates the actual route when finding trade routes for some reason. And since I wasn't sure what that reason was, I didn't want to break the functionality).
The correct solution would be to just change how MGU calculates the distance, but I didn't have the desire or time to do that :) I just wanted to add the threads.

There was a similar issue when MGU tries to find matching goods at other ports, and adding a lock was easier than rewriting that piece of the code. This is the lock that consumes the most time, so if I optimize it, I'll probably get rid of this lock and make the algorithm not dependent on updating some object variable.

There was also an issue with merging the routes. Apparently with multiple threads adding to the same list it causes some indices to be left blank (and occasionally some to be overwritten). I probably should've just had each thread writing to its own list and have the post processing thread combine them, but a lock is only 2 lines of code, and once again won the time battle.

The last one was creating the Thread ID (since I forget what the command is for atomically increasing a variable is for C#, and the lock probably only costs 1ms of lost time for 4 threads trying to get a unique ID).

Most, if not all, of them SHOULD be able to come out. But it will require some fairly decent overhaul to some major sections of code
Men are born to succeed, not fail.
-Henry David Thoreau
Jester-
Admin Emeritus
Posts: 1660
Joined: Mon Feb 17, 2003 11:29 pm
Location: Indianapolis, IN

Re: Multi Core/Thread MGU

Post by Jester- »

If MGU had the same pop up box like SMC to find nearest locations I would use MGU over SMC. As it is now SMC is just so valuable for port building and quick navigation. Plus you don't have to install it, which is awesome.
Image
Image
Azool
SMR Coder
Posts: 1736
Joined: Sat Feb 23, 2002 8:42 pm
Location: Colorado Springs, CO
Contact:

Re: Multi Core/Thread MGU

Post by Azool »

MGU really has all the functionality for having the pop up. It stores all that info internally. Someone just needs to create the interface
Men are born to succeed, not fail.
-Henry David Thoreau
Jester-
Admin Emeritus
Posts: 1660
Joined: Mon Feb 17, 2003 11:29 pm
Location: Indianapolis, IN

Re: Multi Core/Thread MGU

Post by Jester- »

Azool wrote:MGU really has all the functionality for having the pop up. It stores all that info internally. Someone just needs to create the interface
Yeah I know :) I just can't do it....lol
Image
Image
Edgecrusher
Newbie Spam Artist
Posts: 512
Joined: Tue Apr 05, 2005 10:23 am
Location: Netherlands

Re: Multi Core/Thread MGU

Post by Edgecrusher »

For the umptieth time, this functionality is already implemented in MGU, it's in the locations finder, which gives you an option to search for goods.
Hugh
Quiet One
Posts: 25
Joined: Sat Mar 19, 2005 12:21 am

Re: Multi Core/Thread MGU

Post by Hugh »

Link is dead, and I almost got excited about multi thread mgu.
Hugh
~-~-~
Currently with Path TO Oblivion
Previously with Tyranical Rule, Undead Umbrellas, Contraction, V.E.N.o.M, Quiet Riot, System Failure, RoD, TLS and others that don't come to mind...
Led iLegitimate Basterds

Hated 1.5
the570z
Quiet One
Posts: 227
Joined: Thu Apr 14, 2011 5:54 am

Re: Multi Core/Thread MGU

Post by the570z »

I have the multi thread MGU still if somebody wants a copy.
Post Reply