Quake 3 Map Compile Benchmark

[Fjoggs]

Why's bspc different when it's the same compiler used for that specific progress (?)

[obsidian]

Not sure, but looking through it again, it's safe to say that something was wrong with the above results... 30 seconds for BSP? I just noticed that after posting the results.

Anyway, recompiled with 2.5.11 after restarting my computer and these look like better results. I'll try with 2.5.16 again later:

2.5.11
Map Compile= 4203ms
Vis= 43703ms
Bspc= 24609ms
Lightning= 115641ms

mjrpes: Please remove the slower results that I posted on the website. Thanks.

[obsidian]

Q3Map2 2.5.16 Results:

Map Compile= 4594ms
Vis= 43234ms
Bspc= 24672ms
Lightning= 112078ms
Total Time= 184578ms

Q3Map2 2.5.16 is faster overall by about 4 seconds. BSPC is the same.

[Fjoggs]

Yeah that made more sense.

[obsidian]

Noticed in the results.txt it says "lightning" instead of "lighting".

[mjrpes]

ok this time I know why I'm so generously today..
I've learned something new
got something for ya:

Output:

Type: GenuineIntel
Name: Intel(R) Pentium(R) 4 CPU 2.66GHz
Features: FPU, MMX, SSE, SSE2, CLFLUSH, RDTSC, CMPXCHG8B, CMOV
Extended Features: VME, DE, PSE, MSR, PAE, MCE, APIC, SEP, MTRR, PGE, MCA, PAT,
PSE36, FXSR, DS, SS, TM

just tell me, what how you want them in the results.txt
Type and Extended Features may be irrelevant, so tell me if you want them or not.
(I'll leave the source inside, if you at some time want those, but I'll comment'em out)

edit: maybe this way?:
CPU = Intel(R) Pentium(R) 4 CPU 2.66GHz | FPU, MMX, SSE, SSE2, CLFLUSH, RDTSC, CMPXCHG8B, CMOV

I think you can leave out the stuff to the right of the pipe symbol. What I'll do is let the user submit what's in the text file, and then I'll allow them to edit the processor info before they finally submit it, so they can include additional info like cache size, multi processors, and Hyper Threading. The stuff like FPU, MMX, SSE, etc, you can already deduce that it exists, based on what the processor is (Pentium 4, AMD Athlon, etc), so I don't think it's necessary. That's my take on it.

The RAM and OS info are also important. So the three additional fields in the Results.txt file should be,

CPU, RAM, OS

You can put it in any order you want, and you can put in a blank/dividing line in the results file to make everything look clearer if you want to. I'll just take what you have done and make my php script match it accordingly.

Thanks again

[mjrpes]

BTW, the Q3Map2 version you have in the zip is 2.5.11. You may want to update that with 2.5.16 since there are some additional compiler optimizations since then.

I should have updated q3map2 to the most recent version when I redid the benchmark to version 1.2 and reset all the results. I don't know if I want to do it again, because then everyone would have to redo the test if we want the results to be consistent.

But I'm glad it finally worked for you.

[TTK-Bandit]

OK did it.. You can now download the (hopefully) last version: 1.2
(again with sourcecode)

[mjrpes]

updated with system info included in results

[Hr.O]

Silly benchmarking

Just a warning for you guys, do not rely too much on these benchmarks. Since you are all running it on your "working" systems, this benchmark is bound to be pretty subjective.

More often then not it's the background software that dictates the speed of your system, and that makes it dangerous to compare this benchmark with anything else then previous results from your own system. To make this long story a bit shorter, the only objective thing you can conclude is "What background programs/services make my computer run slower/faster then before"

It's a pitty we lost a great deal of forum stuff in the past, because this a recurring discussion. We've had it a few times before.
digging into memory I remember once in the time you could still time the BSP fase of q3dm7sample.map in minutes :icon26: think it was round 2k2.

Hr.O

[mjrpes]

Since you are all running it on your "working" systems, this benchmark is bound to be pretty subjective.

I'd say it's more of a slightly subjective benchmark.

I gave people the forewarning to close applications when running the bench. I noticed firefox having a page loaded can add 7% to the benchmark.

More often then not it's the background software that dictates the speed of your system, and that makes it dangerous to compare this benchmark with anything else then previous results from your own system.

To conclude that background software is what dictates the speed of the system, you would have to show that the results submitted show no trend. But that is not true. There is a trend, where computers with faster processors seem to do better than computers with slower processors. In particular, an AMD64 processors seems to do very well compared to past AMD processors. But then again it would be nice if a few more people with AMD64 systems would run the benchmark.

[Hr.O]

I'd say it's more of a slightly subjective benchmark.

There is no such thing as slightly subjective.

To conclude that background software is what dictates the speed of the system, you would have to show that the results submitted show no trend. But that is not true. There is a trend, where computers with faster processors seem to do better than computers with slower processors. In particular, an AMD64 processors seems to do very well compared to past AMD processors. But then again it would be nice if a few more people with AMD64 systems would run the benchmark.

Yes sure there is a difference between processor speeds, but that has nothing to do with the point i was trying to make. I just hope peeps know how relate to the outcome of a benchmark. No two similair user systems will ever perform the same.

edit: See the results of shadd_. and bitwise

but i'll try and do a bit of be-emming just to please you

[mjrpes]

Yes sure there is a difference between processor speeds, but that has nothing to do with the point i was trying to make. I just hope peeps know how relate to the outcome of a benchmark. No two similair user systems will ever perform the same.

I know what you were getting at in your post but the way you said it didn't leave room for the fact that there is a basic trend to the results that, even though all factors cannot be weighed out, can offer some meaning.

If you take one example of the results so far, two people have posted results with a system that uses a P4 3.0GHz Northwood (that's the same as my 3.06GHz) processor. The difference between them is 3 seconds. So far that's pretty close, with a range of error between them of less than 2%. There are indeed some results that seem inaccurate, such as survivor's XP 2100+ that seems to be much slower than it should be. Also, there seems to be a a weird thing going on where bitWISE's AMD64 3500+ processor did worse than shadd_.'s AMD64 3200+ @2.45Ghz, but much of that is probably due to him overclocking the beast.

One way to get around the problem of result fluctuations is to get a bigger sample. Now, so far, there are three people who submitted results that use an AMD64 processor. All three have posted the fastest results so far, so, even though there are some weird things going on in the results, one might just be able to argue that AMD64 processors do well in the benchmark. Putting a P4EE into the mix would be interesting. Now, if 100 people could run the test, we could start averaging out the speeds and could see some confident trends.

I never meant for it to be accurate, but as just a test that I could use to fill my curiousity as to how different systems ran a map compile. Since no hardware sites do map compile benchmarks, this was as close as it was going to get for me.

[Psyche911]

I think shadd beating bit's CPU is easily explained. The 3500+ is 2.4GHz, shadd is running his 3200 at 2.45 up from 2.2.

If he overclocked the RAM only (not HTT or anything), that would be an 11% overclock of the FSB.

2m38s = 158s
2m56s = 176s

158 seconds is 11% faster than 176 seconds. Works out perfect.

[Psyche911]

I just overclocked from 2.4GHz to 3.0GHz on my P4 2.4C (800MHz FSB) and went from 233 seconds to 186. A 20% better score for a 25% increase in clock speed while maintaining the same memory speed.

In the next few days I'll get the CPU speed up a bit. It won't be 100% stable at the speeds I'm going for, but hopefully it will at least complete the benchmark. I should be able to hit 3.3GHz, maybe 3.4 if I'm lucky.

[Survivor]

There are indeed some results that seem inaccurate, such as survivor's XP 2100+ that seems to be much slower than it should be.

Note, I checked afterward and i actually have a 2200. But I only have a slow ancient 16 gig harddrive which could be limiting my results. Maybe if i get a new one.

[Hr.O]

submitted some results.

QUAKE3 MAP BENCHMARK 1.3 - RESULTS
==================================
OS = WinXP
CPU = AMD Athlon(tm) 64 Processor 3000+
RAM = 511 MByte
==================================
Map Compile = 00:04
Vis = 00:57
Bspc = 00:19
Lightning = 01:56
Total = 03:17

this is the second run, first run took about 3 secs more. Guess filecreation or mem allocation took a bit of time.

[mjrpes]

quad cores are fast

[Silicone_Milk]

lmao 3 year bump

[mjrpes]

I was surprised this thread still even existed

[Scourge]

Fixing to try my AMD quad.

QUAKE3 MAP BENCHMARK 1.3 - RESULTS
==================================
OS = Win 6.0
CPU = AMD Phenom(tm) 9500 Quad-Core Processor
RAM = 3070 MByte
==================================
Map Compile = 00:04
Vis = 00:13
Bspc = 00:39
Lightning = 00:36
Total = 01:33

It was a little faster in xp. The 9500 is 2.2 ghz btw.

QUAKE3 MAP BENCHMARK 1.3 - RESULTS
==================================
OS = WinXP
CPU = AMD Phenom(tm) 9500 Quad-Core Processor
RAM = 2070 MByte
==================================
Map Compile = 00:03
Vis = 00:13
Bspc = 00:25
Lightning = 00:35
Total = 01:18

[obsidian]

Dammit, this thread started back when my computer was close to bleeding edge.

[rgoer]

lol I was just thinking the same thing obsidian

[Scourge]

Saw this on my hard drive while doing some cleaning. Decided to run it again.

QUAKE3 MAP BENCHMARK 1.3 - RESULTS
==================================
OS = Win 6.1 (windows 7 rc1)
CPU = AMD Phenom(tm) II X4 940 Processor
RAM = 4095 MByte
==================================
Map Compile = 00:03
Vis = 00:08
Bspc = 00:16
Lightning = 00:23
Total = 00:52

[obsidian]

Hmmm... the tool doesn't work properly on my computer. Just lists the specs (incorrectly) and no benchmarks.

Manual benchmarks with a batch file:

Windows Vista x64
Intel i7 920 @2.67GHz
12GB DDR3 RAM

Map Compile = 00:02
Vis = 00:09
BSPC = 00:10
Light = 00:26

Total = 00:47