Q4 map format

[Johnny Law]

OK, some initial googling and forum searching doesn't turn up much on Q4 or D3 map format. Next resort is to go wurfling through the most recent GtkRadiant source code, but I may wait on that a bit to see if anyone else turns something up.


From eyeballing the files the format appears to be something like this (warning: not a formal grammar). Except for punctuation, only the brushDef3 and patchDef2 terms are literal.


* a map is a list of entities


* an entity is

Code: Select all

{
"key1" "value1"
"key2" "value2"
etc.
list of brushdefs and/or patchdefs
}

* a brushdef is

Code: Select all

{
 brushDef3
 {
  list of facedefs
 }
}

* a facedef is

Code: Select all

planedef texturemap "texturepath"

* a planedef is

Code: Select all

(x y z d)

where (x y z) is the plane's normal vector. When scaled by d, it is the shortest vector from the plane to the origin.


* a texturemap is
Two 3-vectors that are some sort of mapping from worldspace to texturespace. See posts below for partial explanation.


* a patchdef is

Code: Select all

{
 patchDef2
 {
  "texturepath"
  patchparams2
  (
   list of patchcoldefs
  )
 }
}

or alternately

Code: Select all

{
 patchDef3
 {
  "texturepath"
  patchparams3
  (
   list of patchcoldefs
  )
 }
}

* a patchparams2 is

Code: Select all

( #cols #rows junk junk junk )

where "junk" is an integer that is ignored.


* a patchparams3 is

Code: Select all

( #cols #rows #horiz-subdivisions #vert-subdivisions junk junk junk )

where "junk" is an integer that is ignored.


* a patchcoldef is

Code: Select all

( list of controlpoints )

* a controlpoint is

Code: Select all

( x y z s t )

where x y z are the spatial coordinates of the point, and s t control the texturemapping in ye olde biquadratic Bezier fashion.

[Johnny Law]

Ah ho. Looks like the brush texturing is the same as the "old" Q3 Brush Primitives format's texturing, which is documented in places such as http://quark.planetquake.gamespy.com/in ... .face.html

[obsidian]

Ummm... okay. Did you have a question somewhere in all this or are you explaining how Q4 map files work?

I'm just a little confused about the above. Looks like you're posting the inner workings of your brain. It's not unusual for me to think like that, but I don't write them down on a forum unless I'm trying to make a point of something. I'm slightly intrigued, but would you care to explain why one would need to know all this?

[wattro]

quiet... let the man do his work

[wviperw]

obsidian: See the bottom of this thread - http://www.quake3world.com/forum/viewtopic.php?t=17882. Basically, it'd be nice to know the structure of the Q4 .map file format so that we can manipulate it according to our every whim!

[Johnny Law]

I'm slightly intrigued, but would you care to explain why one would need to know all this?

You never know when it will come in handy. Which is actually going to be pretty rarely, I would imagine, but sometimes...

A specific thing that I've used this sort of info for in Quake 3 is in making CTF maps. The safest and easiest procedure for making a map that has a mirrored and/or rotated section is to make the section, then use automation to do the mirroring/rotating. Which involves manipulating brush and patch shape/placement, swapping out texture names, changing texture mappings, and various muckings-about with entity definitions.

Some of this stuff you can do in the editor fairly easily, and some of it you can't.

For the stuff that you _can_ easily do in the editor, it may not always be done precisely enough.

For the stuff that you _can't_ easily do in the editor, it may be best to approach it with a script or some standalone app to process the map file, or it may be best to use a Radiant plugin. With the former approach you definitely have to know the map file format. With the latter approach, it can still be helpful to know the underlying representation of stuff. (Also, I'm not sure yet how feasible the plugin approach will be for Q4, since Q4 support seems sort of experimental in GtkRadiant, and I don't know yet if there's a way to build plugins for the in-game editor.)

If Lunaran is in fact able to share his script for entity-fiddling, that will be very cool, but I'd like to be able to do more. I'm not the only one so I thought it would be good to have a thread on the topic.

Fortunately the format is not too obscure... it helps that we've managed to ditch things like surfaceparms now that every surface has a material definition.

[Johnny Law]

Let me add a little explanation of the brush-primitives-style texture mapping mentioned a few posts above. Note that I was never a graphics genius, and what skillz I ever had are now rusty, so please correct me as necessary.

You can transform a 2-space coordinate to another 2-space coordinate using a 3x3 matrix. This can include scaling, rotation, and translation. Here's a fairly good page about that:

http://developer.apple.com/documentatio ... ion_7.html

A series of transform matrices can be multiplied together to make one matrix that does all the transforms. So if you have a scale-x-by-2 matrix and a translate-y-by-64 matrix, you can multiply them together and get a single matrix that represents the scale-x-by-2-and-translate-y-by-64 operation. A single matrix can represent any sequence of scales, rotates, and translates.

There's also a nice tutorial-esque thing here, if you have a Gamasutra login (*cough* BugMeNot *cough*):

http://www.gamasutra.com/features/20020 ... son_01.htm

So anyway, the two 3-vectors that represent texture mapping in the Q4 map file format are the first two columns of the transformation matrix that maps the faceplane into the texture. Since the final column is ( 0 0 1 ) we now know the matrix.

This makes it fairly easy to fiddle with an existing texture on a given face... just get the transform matrix for whatever operation you want to do, and multiply it with the current matrix. (Hmm, although determining the right operation to do has some complications, the same as I'm about to mention below...)

However, if you want to initially place the texture on the face in some way, or move the face while preserving the appearance of the texture, then you have to deal with the question of how the 2D coordinate space of the faceplane is generated. It's derived in some way from 3D worldspace of course. That page I linked in a previous post, from the QuArK docs, says that the 2D coordinate space "is one whose axis directions are gotten by rotating the Y and Z coordinates of map space so that they lie in the plane". There are an infinite number of ways to transform the worldspace in that way, so it matters which particular way is used by the editor and compiler.

[obsidian]

Ah, I see. Didn't read the other thread, so it looked like something quite random. I understand what you're trying to do now. Thanks for clearing that up.

[MegaMan44]

I'd like to see an ebnf version of that

[Johnny Law]

Well, the below isn't exactly proper EBNF notation, but it's pretty close. It's what I started using as a grammar input for a parser that uses the Perl Parse::RecDescent module. It's not exactly blazingly fast but it works (so far).

Turns out to be pretty easy to do everything except for keeping a texture properly aligned when you manipulate a brush. Looks like that will require actual math, darn it.

Code: Select all

   POSINTEGER : /\d+/
   FLOAT : /-?\d+(\.\d+)?/
   QUOTESTRING : /\"[^\"]+\"/


   q4map : version entity(s)

   version : "Version" POSINTEGER

   entity : "{" entitycontent(s) "}"

   entitycontent : keyandvalue
                 | brushdef3
                 | patchdef

   keyandvalue : QUOTESTRING QUOTESTRING

   brushdef3 : "{" "brushDef3" "{" facedef(s) "}" "}"

   facedef : planedef texturemap texturepath

   planedef : "(" FLOAT FLOAT FLOAT FLOAT ")"

   texturemap : "(" "(" FLOAT FLOAT FLOAT ")" "(" FLOAT FLOAT FLOAT ")" ")"

   texturepath : QUOTESTRING

   patchdef : patchdef2
            | patchdef3

   patchdef2 : "{" "patchDef2" "{" texturepath patchparams2 "(" patchcol(s) ")" "}" "}"

   patchparams2 : "(" POSINTEGER POSINTEGER POSINTEGER POSINTEGER POSINTEGER ")"

   patchcol : "(" controlpoint(s) ")"

   controlpoint : "(" FLOAT FLOAT FLOAT FLOAT FLOAT ")"

   patchdef3 : "{" "patchDef3" "{" texturepath patchparams3 "(" patchcol(s) ")" "}" "}"

   patchparams3 : "(" POSINTEGER POSINTEGER POSINTEGER POSINTEGER POSINTEGER POSINTEGER POSINTEGER ")"

[Johnny Law]

It's not exactly blazingly fast...

Hoo, that's an understatement. Trying it out on a real map now and it's taking hours to crunch it. Whatever works though.

[AluminumHaste]

I just wanted to say: Thank you so much for posting this information, using this info I was able to split a patch into multiple patches, awesome.
Here is what I mean:

[lvlshot]http://img708.imageshack.us/img708/7826/splitpatches.png[/lvlshot]

[Johnny Law]

Holy cow, thread rises from the dead! Glad someone else found it useful.

[AluminumHaste]

LOL well I did send you a pm first

[Johnny Law]

I usually go a few years between receiving PMs, so it can take me a while to notice.

[AluminumHaste]

I just wanted to say that splitting patches has now been added to DarkRadiant, you can now do it without editing the map file.

[Obsessed]

Necroposting in da house!