Quake3World.com Forums
     Level Editing & Modeling
        Good news for huge nerds - q3map2 optimizations


Post new topicReply to topic
Login | Profile | | FAQ | Search | IRC




Print view Previous topic | Next topic 
Topic Starter Topic: Good news for huge nerds - q3map2 optimizations

Insane Quaker
Insane Quaker
Joined: 05 Nov 2010
Posts: 449
PostPosted: 05-09-2016 05:44 PM           Profile Send private message  E-mail  Edit post Reply with quote


I guess this goes in programming, but it could help the mappers too so...here goes.

About a year ago, I took a job in a place where I help run supercomputers. I learned a crapload in this past year...and of course, my ulterior motive this whole time was to take what I learned and apply it to quake 3.

Long story short, I thought I'd end up making my own linux distro..or my own branch of q3map2...but from what I've just tested, what I have going right now is pretty good for the level of effort required, hopefully someone else can take what I've done and get good results with it.

There's the long horrifying road, and there's the short road.

Short road:

Pick a reasonably slim linux distrobution, something where it's easy to avoid lots of system processes and Xorg stuff going on. I use Crux, which is not great for beginners and the learning curve was steep, but worth the performance.

I grabbed the source for the netradiant-master.zip floating around on the web, and if you're familiar with compiling, it's pretty straight forward to jump into the Makefile, add -static and -march=native to the compiler lines, and do "make binaries-q3map2" to have it create a reasonably well optimized binary for your cpu. Just this can account for the vast majority of the performance gains.

Long road:

I did a lot to my distribution of crux, I haven't done performance benchmarks on every step of what I did, but the resultant performance was worth the effort to me. Possibly the biggest thing I did was to get GCC 5.3.0 and compile it with itself. I don't know how much this step helped...some people mentioned that the way GCC compiles does it in such a way that doing this wouldn't really help, but I could notice a small burst in compilation speeds after I did it anyway - not a night and day difference, but it certainly seemed to shave a few seconds off my kernel build time. Which brings me to the next step....kernel optimization.

I have a computer dedicated to just this, and I tailored down the kernel to a nub, messing with different things I can't remember the name of off the top of my head, but I must have gone through 20 something kernels of 4.5.2 until I arrived at the one I have now. One special thing I did here was to add -march=native to the makefile for the kernel, which gave me a small, but consistently measurable boost in all the applications I threw at it (old benchmarks, encoding, compile times)

Once I had the kernel and the compiler all cross-built and tuned for my cpu (FX8350) I rebuilt q3map2 with the same kinds of options, -static and -march=native, I think -O3 is passed elsewhere in the default makefile. With that compiler running on my heavily optimized install, I started rebuilding some of my old maps.

The best comparison I have at the moment is of my old dedicated map compiler, which was a 6-core AMD 1100T. It took a solid 4 hours to do a good, polished "final" build of my huge map, Razztazzmagoria.

Since I've upgraded to a 8 or 4 core (depending on how you look at it) AMD FX8350 and optimized the hell out of everything, the compile time for the same map with the same build commands is just under an hour.

The interesting thing about the reduction in time is that the FX8350 scores a 8944 on cpubenchmark.com while my old 1100T gets 5,845, so that's about 65% of an improvement. My old testing times were from just installing suse and letting q3map2 build however it does by default, so that was a really unoptimized system. The increase of speed I've seen on this new system though is way more than 65%, I have more testing to do, but going from 4 hours to 56 minutes was far more of an improvement than I expected to see.

My next hope is to bring my dual Xeon E5-2640 v2 system up and run everything through the Intel compiler, which habitually produces disgustingly fast code and re-run my maps.




Top
                 

clueless
clueless
Joined: 02 Feb 2006
Posts: 571
PostPosted: 05-11-2016 03:06 AM           Profile Send private message  E-mail  Edit post Reply with quote


I also have a technical background in programming and I use HPC to do research at a german university. Some years ago I also liked to compile my own linux kernels and stuff like that. It's cool and super-nerdy.

Few people here are gonna get what this is about. And if they do, they may ask themselves why you spend months to get q3map2 compile a map in 1 instead of 3 hours. But the journey is the reward here, so congrats on that.

(I do doubt though that many others are gonna take what you've done and get good results with it. These days, I would just go have a beer while it compiles. ;) )



_________________
my FPS maps


Top
                 

Insane Quaker
Insane Quaker
Joined: 05 Nov 2010
Posts: 449
PostPosted: 05-11-2016 04:38 AM           Profile Send private message  E-mail  Edit post Reply with quote


You're not wrong..and admittedly, I'm sure with no optimization maps compile blazingly fast these days even in windows (especially on intel chips which are absolutely bananas), but in the case of especially large maps like the ones I tend to make it could save someone a few hours.

This was half proof-of-concept and half actual application, I'm also working on optimizing a completely unrelated application called Quantum-Espresso so that the researchers I support can have shorter wall-times on whatever system they're using.

:up:




Top
                 
Quake3World.com | Forum Index | Level Editing & Modeling


Post new topic Reply to topic


cron
Quake3World.com
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group