Q3Map3 - Discussion and Development

[Silicone_Milk]

I provide rumors of a new project that will be my focus as a portfolio piece for future job hunts. I present, q3map3.

"What's the point? q3map2 works fine and most people have moved on to newer tech". Yes, that is an interesting question. The point of this project is to familiarize myself with programming on the GPU. q3map2 is computational bound. Slower processors will take longer to crunch a map than newer, faster ones. With technologies such as NVIDIA's CUDA, ATI's Stream tech, and the new OpenCL standard submitted by Apple, it's much easier to harness the power of the number crunching beast that is the GPU.

My current goal is to use OpenCL to bring the calculations to the video card. This will enable the program to run on almost any device that supports OpenCL.

I expect to see, at minimum, a *1000% increase in performance from this switch from the cpu to the gpu.

If my toying around is successful I'd assume I'd most likely work on expanding the program to compile idtech4 and maybe, and that's a big maybe, idtech5.

The source, of course, will be released under the GPL license.

MASSIVE EDIT: Cleaned up original post to be more relevant to the subject at hand. Title altered to make this thread more clear. I hope this doesn't cause any confusion. Didn't want to split this off in to a separate thread either so I figured this was the best action to make.

* - awesome made up number

[Silicone_Milk]

lol worthy but testing to make sure the bsp section of the converted code compiled and worked correctly. It's just the skeleton of the framework. Going to work on adding the functionality tomorrow.

This thread needed an image so here you go:


(and no, it's not just argument checking happening there. All of the data structures exist and the function prototypes are around. Just haven't made them work on any of the structures yet.)

[^misantropia^]

I just can't seem to stay focused on a map project long enough to ever finish it. I have dozens upon dozens of maps floating around a few different hard drives since around 2001/2002.

This is mostly because of what I consider a form of "mood swing" where I'll be very artistic for a month or so then switch to a more analytical mode and focus on programming. While in a programming mood, I completely suck at anything artistic and just can't get anything done. This is when I start my programming projects which come to a halt when my artistic mood kicks in.

Am I right in assuming you are 20-ish? Then don't worry, it gets better when you get older.

Offloading to the GPU is an interesting idea. Which parts in particular do you think are best suited? I take it bits of the lighting phase could indeed be sped up dramatically, maybe vis too.

Any reason not to start with OpenCL from the get go? Porting it from CUDA later on seems like a tedious chore.

[Silicone_Milk]

You're correct. It'll be nice when I can stay focused on things longer

The lighting and vis phases are the first that I've thought of when I thought of moving things over to the gpu. The lighting stage will benefit the most I think.

NVIDIA's implementation of OpenCL is really buggy from what I've been reading up on. Also, trying to run any of the programs from the SDK is crashing and I'm getting an error about not meeting minimum requirements for my system. Yet, from what I can tell, I just need a card from the 8000 series or newer.

*shrugs*

[sock]

I just can't seem to stay focused on a map project long enough to ever finish it. I have dozens upon dozens of maps floating around a few different hard drives since around 2001/2002.

I am sure this is going to sound harsh but you got to get a system going otherwise how will someone employ you if you cannot finish anything. There are plenty of things you can do and not starting another project is a good start!

* Find someone else to map with, create a small team with a friend. Email someone you want to work with and ask them if they are interested. You will need a plan of what is left to do, otherwise it will just seem like you are offloading your map to someone else.

* Create a plan of what you want to do with the level and stick to it. Work out the details like assets you need, photo sources for detail, test maps for gameplay. Breakdown all the tasks and create a todo list for a couple of weeks.

* Contact friends for feedback and ask them if they are interested first. Have things to do while the feedback is going on. Forums are the worst for feedback because you are just waiting around wondering most of the time. Often people just post 'yeah thats cool' which is good for morale but not for completing stuff. Think carefully about friends feedback, ask people to explain more if you don't understand and never dismiss any ideas. If two friends say similar things, then they are usually right about that issues and fix it. Let your friends know what has changed and if you are using their feedback. Engage people in the process of feedback not ignore people and expect endless feedback all the time.

* Complete one area to final detail, lighting and show it to some friends. Get feedback so you are pumped up to carry on and finish the map. Prototype a section of your level visually in a small map. experiment with different styles or versions of the same area. Find an art style and keep it consistent across a whole area. Create prefab detail that can be used across the whole map, make it modular so it can be twisted around and look different but similar from other angles.

* Take breaks or a weekend off and do some research on your current project. Find some new photo sources, look at similar work by others, play some games with similar art styles and take notes on what they are doing. Talk about ideas and stuff with friends, use screenshots to discuss your points.

* Setup a proper page on your website that you can stick screenshots on and get other people interested in what you are doing. Drum up interest by giving people a reason to come back to the web page with comments on how the map is progressing. Don't be afraid to pimp screenshots, but try and create stuff that is eye catchy visually. Be careful about using editor shots or fast compiles, they just make your work look rushed, spend time on screenshot presentation, find good angles.

[AEon]

One thing that is presently happening to me is getting boggled down with detail...

Sock seems to be at the extreme end of this, but with sheer perseverance he is able to get through it (see his SP map). I used to be able to create a map in two weeks, and they may not have been perfect technically, but fun none-the-less. AEpyra is still one of my favs...

So my tip is to try to keep things simple, and then concentrate on those to make them look good, i.e. keep the maps small. And try to choose a few good ideas and implement those in one map... not try to get *everything* in one map.

[monaster]

My current goal is to use CUDA to bring the calculations to the video card. This limits me to NVIDIA cards though and ATI users will not be able to run the program. This is why my long term goal is to bring all of the code over to OpenCL once it's properly implemented and stable. By doing so, the program will be able to run on almost any device that supports OpenCL.

So does this means that some day even the pitiful ATI-users will be able to start up q3map3?

[Silicone_Milk]

Sock, it is a problem I have mainly with art. When there's money being dangled in front of my face I can get the project done

I used to work for a game company as a programmer (gameplay, physics, AI, gui) and was able to get the jobs done. I just need some motivation and direction.

Those *are* good and interesting points though. It's funny about the forum feedback comment; I remember when people were harsher and would tell you if your stuff was bad here in LEM

AEon - I never was one for detail. I can visualize the general image but lose all sense of the goal when I try to add in detail.

monaster - pitiful? a large portion of people actually use ATI cards. I'd rather not alienate them. But, until it's written in OpenCL it can't run on anything but NVIDIA.

[monaster]

Sock, it is a problem I have mainly with art. When there's money being dangled in front of my face I can get the project done
I used to work for a game company as a programmer (gameplay, physics, AI, gui) and was able to get the jobs done. I just need some motivation and direction.

Let's start collecting money for poor old Silicone_Milk!

monaster - pitiful? a large portion of people actually use ATI cards. I'd rather not alienate them. But, until it's written in OpenCL it can't run on anything but NVIDIA.

Hm, probably not the best word I've chosen to describe ATI-dependent people (I'm one of them too by the way); the pitiful should just point to skybox problems with infamous black lines and the like. Thanks for planning taking care of "us" in the future.

[EDIT: added an "e", EDIT2: twice]

[sock]

Sock, it is a problem I have mainly with art.

As I said to cityy earlier, art and detail does not come naturally to anyone, just like coding or anything else. You got to practice with real world examples. Below is what I said to cityy.

Sometimes I feel like I am really restricted - imagination wise.

Everyone has the potential to create awesome stuff but you got to practice the details first. Find a picture of something you like (building/architecture) and build it. Understand the process of detail, scale and layers of materials. Find out how things are built in real life, understand how materials breaks, gets worn down and interacts with each other. Practice these things in small test maps and initially limit yourself to a few materials so you don't get distracted with other details.

I rarely pimp other peoples sites but this one is especially good if you are trying to understand where to start with level design. The author has a very good attitude about that anyone can be artistic or imagative, you just got to spent time practicing.

http://www.worldofleveldesign.com/

[Fjoggs]

I never finish things either!

[v1l3]

I never finish things either!

*hint hint* You said to remind you..hehe

[Silicone_Milk]

Didn't have time to work on this much this weekend/this week.

Just bought a new harddrive and Windows 7 and have been busy crawling through 250 GB of files to save what isn't garbage.

Still need to get Windows 7 set up for development.

[Silicone_Milk]

Good news everyone!

Got my dev environment back up and running which included reinstalling the OpenCL sdk on Windows 7. Things are working as they should now.

This means, as Misantropia suggested, I'll be cutting CUDA out and going straight in to OpenCL for the ATI users out there

[monaster]

Good news everyone!

indeed!

[Silicone_Milk]

Got a little bit of work done last night. Not too much though.

When I get the BSP phase working (if just barely) I'll be dumping the source to github.

[Silicone_Milk]

Update:

• - Added python support to the compiler. This serves a few purposes:
  • - better (in my mind) string manipulation for parsing
    - allows q3map3 to have its functionality developed as a set of modules (plugins).

OpenCL has Python bindings so I'll be able to do what I want/need to do in Python. This has the added benefit of making the compiler flexible.

[phantazm11]

This is really starting to sound interesting. It is a great concept, offloading the work to the GP. I'm looking forward to your progress Silicone_Milk.

[Silicone_Milk]

Thank you phantazm. I'm eager to get something presentable working so I can put the source up for grabs.

Just been super busy at work.

[Silicone_Milk]

I'm still working on this. I just haven't had a ton of time lately since I got shafted pretty damn hard at work and lost my job.

So now I'm out looking for a new job so I can pay my rent by next month -_-

I promise I'll get the code and a build up after I find a job ;D

[skinNCNmaster]

very valued potential here. im still out of touch but memory of compile times and such beget some hesitation towards getting things running on my new map term..

my questions would, or suggestions might be..

1. will this compiler bring any new potential for visual prettyness in game?

2. is it possible to divide te light stage from its geometry and enter the map with only the lighting and no actual textured geometry... a sort of lit/shadowded invisibility.. something i figure i could use in a few different ways were it possible, not necessarily q3 applications.

[Silicone_Milk]

very valued potential here. im still out of touch but memory of compile times and such beget some hesitation towards getting things running on my new map term..

my questions would, or suggestions might be..

1. will this compiler bring any new potential for visual prettyness in game?

2. is it possible to divide te light stage from its geometry and enter the map with only the lighting and no actual textured geometry... a sort of lit/shadowded invisibility.. something i figure i could use in a few different ways were it possible, not necessarily q3 applications.

1.) Potential, yes. I'm not going to attempt to improve what Ydnar has already done in that sense though. I just want to move the code over to the GPU and make the compiler more flexible for modification later on down the road. One interesting thought *did* occur to me while writing this post though - adding refraction shaders for things such as blocks of ice, caverns filled with crystal for those cave maps I've been seeing, etc... I'm keeping a note of that. Thanks Skin

2.) If I'm understanding your question correctly, the short answer is no. If you want to see how the lighting is affecting surfaces in the game, use /r_lightmap 1 to toggle lightmap mode. I don't think I'm actually understanding this question though. Could you please rephrase it?

[obsidian]

adding refraction shaders for things such as blocks of ice, caverns filled with crystal for those cave maps I've been seeing, etc...

Doesn't that require pixel shaders? Wouldn't that be an engine mod then?

I remember JK & JA added a pixel shader to the engine for some of the force effects (push/pull) and seeing those for the first time, my jaw dropped because I couldn't figure out how the did it.

[^misantropia^]

Doesn't that require pixel shaders? Wouldn't that be an engine mod then?

Yep. Silicone_Milk, have you pushed the source onto GitHub yet?

[Silicone_Milk]

I was thinking more about the generation of the lightmaps for some pretty light effects originally.

But after getting some sleep I realized you'd also have to be able to see the effect when looking through the object itself, which, as you've said, would require an engine mod.

Such a shame.

No misantropia.

I'm still working on this. I just haven't had a ton of time lately since I got shafted pretty damn hard at work and lost my job.

So now I'm out looking for a new job so I can pay my rent by next month -_-

I promise I'll get the code and a build up after I find a job ;D