Quake 3 Map Compile Benchmark

Post by **mjrpes** » Wed May 11, 2005 1:08 am

This is my attempt at a standardized benchmark to see which systems compile quake maps the best.

It's been a year or so since I last posted this, but a thread in GD made me think that perhaps some of you have upgraded your systems since then. I'm still curious to see how an Althon64 compares to a P4 system. It looks like a standard P4 is a better system than the Althon XP, but these processors are starting to get old (nevermind how old quake 3 is itself!) with the Althon64 out and dual processors around the corner.

Webpage: http://www.ciole.net/quake_bench/

From the webpage you can download the benchmark tool and post your results.

Post by **obsidian** » Wed May 11, 2005 2:05 am

Maybe it's because my 3.0 GHz processor and 1GB of RAM is too fast for it, but I can't seem to get the benchmark to run correctly (or at least output the results).

I have it installed in C:\Games\Quake III Arena\benchmark\

It gets as far as displaying "Now Benchmarking Map Compile . . . Please Wait . . ." then the Command Prompt window closes. It seems as if the BSP and PRT file are generated, but I doubt that it's getting as far as the light compile, since it only runs for a few seconds.

In the results.txt file is this:
QUAKE3 MAP BENCHMARK 1.1 - RESULTS
==================================

Any ideas?

Post by **mjrpes** » Wed May 11, 2005 2:22 am

Interesting.

The process goes from map compile, to vis, to bspc, then to lighting.

At the stage you mention, it is definitely getting caught up in the map compile, since no output for the map compile is made into the result.txt file. The actual command it runs at that point is:

"q3map2 -fs_basepath .. -meta -v q3dm1sample.map"

I wonder if something could be wrong here. I ran the batch file where I commented out the actual command, so as to emulate a super fast system that could compile the map instantly, and no problems were encountered.

I don't know what else to think here.

Post by **mjrpes** » Wed May 11, 2005 7:10 am

obsidian, try downloading this new version:

http://ciole.net/quake_bench/quake3_map ... rk_1.2.zip

This new version has all necessary files and doesn't rely on quake3 anymore; you can run it from anywhere. Perhaps this will solve the problem you were having.

Post by **[acid]** » Wed May 11, 2005 7:41 pm

i need an upgrade

Post by **TTK-Bandit** » Wed May 11, 2005 9:10 pm

the problem seems to be that obsidian has no english version of windows.
the date and time variables are diffent on different windows language versions.
so.. I just wrote a little test app, that can do it:
http://mitglied.lycos.de/Q3Coderz/files/

Just copy the 2 files into the "files" directory and run bench.exe
bench.cfg format is pretty simple:
1. line is benchmark version
all lines that follow are benchmark tests in this format:
name|commandline

But I've seen that you build your homepage data out of the output,
so you'd better tell me what output you wanna have, so I can rewrite some lines.
(I can't run your batch files either and I don't wanna fix'em

)
Have Fun!
Bandit.

Post by **bork[e]** » Wed May 11, 2005 9:35 pm

I didn't use the new version when I just posted my results. Didn't notice that print until I was comparing my results with others.

Post by **mjrpes** » Wed May 11, 2005 11:22 pm

TTK-Bandit wrote:the problem seems to be that obsidian has no english version of windows.
the date and time variables are diffent on different windows language versions.
so.. I just wrote a little test app, that can do it:
http://mitglied.lycos.de/Q3Coderz/files/

Just copy the 2 files into the "files" directory and run bench.exe
bench.cfg format is pretty simple:
1. line is benchmark version
all lines that follow are benchmark tests in this format:
name|commandline

But I've seen that you build your homepage data out of the output,
so you'd better tell me what output you wanna have, so I can rewrite some lines.
(I can't run your batch files either and I don't wanna fix'em )
Have Fun!
Bandit.

Wow, very cool. I'm not that surprised the batch file was the culprit, but I never would have though about different languages being the issue. I don't like batch files anyway, so this is a good reason to get away from it.

I'm at work now but I'll get you the info on the formatting when I get home.

Post by **mjrpes** » Thu May 12, 2005 1:54 am

Alright here's the format:

QUAKE3 MAP BENCHMARK 1.2 - RESULTS
==================================
Map Compile = 00:05
Vis = 01:05
Bspc = 00:22
Lighting = 02:12
Total = 03:44

Line 1 should be exactly as above. I parse to make sure the 24th character is equal to 2, so that results are submitted using the correct version (1.2) of the test.

Line 2 is ignored.

For the next five lines, all that matter is that there is an equal sign somewhere and that there is a time value of the format "mm:ss" to the right of the equal sign. No need to worry about whitespace or text to the left of the equal sign. These lines should be in the order of Map Compile, Vis, Bspc, Lighting, and Total, like above.

There shouldn't be any additional stuff besides these seven lines.

Post by **mjrpes** » Thu May 12, 2005 4:25 am

bork[e] wrote:I didn't use the new version when I just posted my results. Didn't notice that print until I was comparing my results with others.

if you post a new score up i'll delete the old one

Post by **obsidian** » Thu May 12, 2005 5:39 am

If I just compile the map with those compile switches using my own batch file, shouldn't I be able to achieve reliable results and just post the times from the compile log?

And I'm using an English version of WinXP Pro.

Post by **mjrpes** » Thu May 12, 2005 6:05 am

I'd imagine so. But for consistancy sake I'd run it with the new benchmark tool I created, since there's less chance of your time being off with the rest because the files you have might be different. Then you could post your results here and I'd add them to the webpage.

Or you could wait until TTK-Bandit modifies his progam to output everything correctly, and I'll create a new benchmark tool to download and you can use that.

Post by **TTK-Bandit** » Thu May 12, 2005 5:25 pm

go try it

Post by **Survivor** » Thu May 12, 2005 6:05 pm

There ya got my magnificent results. Maybe i'll retry when i do a format, my system is a bit clogged.

Post by **TTK-Bandit** » Thu May 12, 2005 6:29 pm

optimized a few things (will not change results

) and updated the package
I'm not looking very often into this forum, so I added the sourcecode to the package if there's need to change something.

Post by **Survivor** » Thu May 12, 2005 6:33 pm

Maybe just an idea for you to look into but some programs check on processor, ram and os version themselves. Maybe try to integrate that into the results for generalized standard.

Post by **mjrpes** » Thu May 12, 2005 7:32 pm

Survivor wrote:Maybe just an idea for you to look into but some programs check on processor, ram and os version themselves. Maybe try to integrate that into the results for generalized standard.

ya know... that's a really good idea. mebbe when i get some time after school ends in a couple of weeks i'll experiment with that.

Post by **mjrpes** » Thu May 12, 2005 7:39 pm

TTK-Bandit wrote:optimized a few things (will not change results ) and updated the package
I'm not looking very often into this forum, so I added the sourcecode to the package if there's need to change something.

checking now. thanks for your help, i'll add in you in for credit when i update the site with the new benchmark... hopefully sometime tonight.

i'm also glad you including source. i'm thinking of playing around with an auto-submit to website feature, so people don't have to do the copy and paste thing. also the idea of automatically getting cpu/ram info sounds neat, whenever i might get a moment to look into that!

Post by **TTK-Bandit** » Thu May 12, 2005 8:31 pm

writing system info is a good idea, but to submit automatically not!
I don't like programms that access internet without permission.
if you add it, ask the user at least if he wants to..

Post by **TTK-Bandit** » Thu May 12, 2005 9:38 pm

whenever you might get a moment to look into that:

edit: (removed code snippet for os, cpu, ram detection)
never mind.. I added it to the bench.exe and uploaded it on a seperate file to my webspace.
Sry dunno why I'm this generously today

ah nearly forgot:
you just need to adapt your web script to read the new format:

QUAKE3 MAP BENCHMARK 1.2 - RESULTS
==================================
OS = WinXP
CPU = Intel w/ SSE2 2670 MHz
RAM = 511 MByte
==================================
Map Compile = 00:05
...

Post by **mjrpes** » Fri May 13, 2005 8:45 am

Thanks for you generousity TTK-Bandit, even if you don't know why

I updated the tool with your program, so hopefully the wierd issue that obsidian had will go away, even if it wasn't related to the date thing. Give it another go, obsidian

I'm waiting a bit to impliment the bench2 program that you made, giving you system info, as I want to see if it's possible to get more specific cpu info. I notice when I go to system properties in windows I'm told I'm running an Athlon XP processor, but the cpu detail in the benchmark says only its an AMD with 3DNow. I found an article here,

http://www.codeguru.com/Cpp/W-P/system/ ... /c9087__2/

that offer another way to get cpu info, using windows system registry. I wonder if there would be any downsides to doing it this way. I'll have to delay looking at this for a couple of weeks, as I've procrastinated enough as it is with school work

Post by **TTK-Bandit** » Fri May 13, 2005 11:16 am

I've seen that too, but I'm not sure if all operating systems support these values..
winxp does I think win2k will too, but what with nt or 9x ?
well fortunately I have win98se on my laptop so I can tell you, it has only these values:
Identifier
Update Status
VendorIdentifier
and as you may have seen, they are not the values you are looking for.
you need to find another way...

Post by **TTK-Bandit** » Fri May 13, 2005 1:02 pm

ok this time I know why I'm so generously today..
I've learned something new

got something for ya:

Output:

Type: GenuineIntel
Name: Intel(R) Pentium(R) 4 CPU 2.66GHz
Features: FPU, MMX, SSE, SSE2, CLFLUSH, RDTSC, CMPXCHG8B, CMOV
Extended Features: VME, DE, PSE, MSR, PAE, MCE, APIC, SEP, MTRR, PGE, MCA, PAT,
PSE36, FXSR, DS, SS, TM

just tell me, what how you want them in the results.txt
Type and Extended Features may be irrelevant, so tell me if you want them or not.
(I'll leave the source inside, if you at some time want those, but I'll comment'em out)

edit: maybe this way?:
CPU = Intel(R) Pentium(R) 4 CPU 2.66GHz | FPU, MMX, SSE, SSE2, CLFLUSH, RDTSC, CMPXCHG8B, CMOV

Post by **obsidian** » Fri May 13, 2005 2:47 pm

BTW, the Q3Map2 version you have in the zip is 2.5.11. You may want to update that with 2.5.16 since there are some additional compiler optimizations since then.

The Bench.exe works great. Thanks for looking into this. I'll post some more accurate benchmark results later, since I tried it now with a bunch of other stuff running.

Post by **obsidian** » Fri May 13, 2005 4:38 pm

Actually, looks like 2.5.11 is slightly faster overall:

2.5.11
Map Compile= 30125ms
Vis= 46984ms
Bspc= 35062ms
Lightning= 164438ms
Total Time= 276609ms

2.5.16
Map Compile= 29891ms
Vis= 47516ms
Bspc= 47797ms
Lightning= 174000ms
Total Time= 299204ms