Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 [2] 3 4 ... 15

Author Topic: Obsidian: A new DF 3D visualizer toolset  (Read 65830 times)

opsneakie

  • Bay Watcher
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #15 on: May 23, 2010, 01:51:48 pm »

It looks like the texture thread should give us a good start on that end of things. I'll see if I can remember how to model correctly and efficiently. As a side note Skeggox, I know visualizers don't usually run in real time and therefore model poly count isn't a big deal,  but I assume we would still prefer small-ish ones so as not to butcher people's computers?
Logged
Collosal bronze man
With titanic strides he comes
Dwarves he will harvest

DF Twitter

Skeggox

  • Bay Watcher
  • Likes obsidian for its shinyness
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #16 on: May 23, 2010, 02:04:10 pm »

Visual Fortress did a good job of keeping the poly count low, so I don't think it will be that bad.

At least for geometry. But I'll keep having a low poly count in mind. If I find that generating full meshes for every map block starts heaping on the poly's there are always tricks like instancing on the GPU that can help out.

This brings up an important note - the real-time renderer will rely heavily on advanced GPU features, so I'm afraid that supporting non-accelerated hardware (like older PC's and laptops) won't be a priority. For those you'll have to stick to exporting the scene to a 3rd-party software renderer, like a raytracer or something. And having a lot of patience :)

As for objects like buildings, trees, etc. using a good paged rendering system with imposters should do the trick.

As for the textures, keep Djohaal's suggestion of procedurally tinting base textures in mind. It can cut down on the

VF's approach to the surface rendering is very good and Obsidian will support something very similar, but underground rendering is still something I'll need to think about. Any ideas there would be much appreciated.
Logged
Dwarf Fortress - "It's like if Tolkien taught your Geology 101 class, but you were drunk and playing the Sims during the lecture. Then you had a dream that got them all confused."

Obsidian 3D Visualizer Toolset - http://df.magmawiki.com/index.php/Utility:Obsidian
A Lua-riffic collection of modules to render, manipulate, save and generally mess with your forts.

Djohaal

  • Bay Watcher
  • [PREFSTRING:Utter Insanitiy]
    • View Profile
    • My deviantart
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #17 on: May 23, 2010, 03:20:18 pm »

One thing I think that should be implemented is full camera and scene controls for the user, it'll be a real time GPU-based studio in that sense, so one will be able to add lights, control shadow parameters and even adjust camera perspective and stuff...

For underground, I think the user should have as much control as possible. A first-line feature will be a back-face culling so we can properly see the undergorund structures, however that is not enough (specially considering caverns and stuff).

Another thing we should implement is the "explode" levels function of Khazad. It increased the Z-spacing between the levels, making it look like a fancy engineering blueprint, but it also helped making sense of the underground structures.

Maybe a "hide" function could be implemented to hide unwanted portions of the cavern, however how would such a thing be implemented in a user-friendly way is touchy...
Logged
I really want that one as a "when". I want "grubs", and "virgin woman" to turn into a dragon. and monkey children to suddenly sprout wings. And I want the Dwarven Mutant Academy to only gain their powers upon reaching puberty. I also have a whole host of odd creatures that only make sense if I divide them into children and adults.

Also, tadpoles.

Shiv

  • Bay Watcher
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #18 on: May 23, 2010, 04:03:40 pm »

Well I'm not so good with programming and other shit that I don't understand, even though I've had a class or two in it (seriously, recursion blows my mind, I still don't know what the fuck they were talking about).

However, I can make rudimentary 3D models when it gets to that point. I can also provide locations for free 3D models that I've found on the web.  So give me a holler when you get to that stage in the game and I'll see what I can do to assist.
Logged
I still don't think I'm crazy enough to play this game properly.

Sizik

  • Bay Watcher
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #19 on: May 23, 2010, 04:32:24 pm »

Will you be supporting 40d and earlier versions?
Logged
Skyscrapes, the Tower-Fortress, finally complete!
Skyscrapes 2, repelling the zombie horde!

Skeggox

  • Bay Watcher
  • Likes obsidian for its shinyness
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #20 on: May 23, 2010, 06:53:15 pm »

Will you be supporting 40d and earlier versions?

Obsidian will be supporting all versions that DFHack supports :)
Logged
Dwarf Fortress - "It's like if Tolkien taught your Geology 101 class, but you were drunk and playing the Sims during the lecture. Then you had a dream that got them all confused."

Obsidian 3D Visualizer Toolset - http://df.magmawiki.com/index.php/Utility:Obsidian
A Lua-riffic collection of modules to render, manipulate, save and generally mess with your forts.

Mephansteras

  • Bay Watcher
  • Forger of Civilizations
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #21 on: May 23, 2010, 06:57:11 pm »

Glad to see someone is taking this up!
Logged
Civilization Forge Mod v2.80: Adding in new races, equipment, animals, plants, metals, etc. Now with Alchemy and Libraries! Variety to spice up DF! (For DF 0.34.10)
Come play Mafia with us!
"Let us maintain our chill composure." - Toady One

Skeggox

  • Bay Watcher
  • Likes obsidian for its shinyness
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #22 on: May 23, 2010, 06:58:33 pm »

One thing I think that should be implemented is full camera and scene controls for the user, it'll be a real time GPU-based studio in that sense, so one will be able to add lights, control shadow parameters and even adjust camera perspective and stuff...

I'm playing with the OpenSceneGraph project as my scene manager. It comes with a full Lua binding, so you'll be able to modify everything in the scene (add/remove nodes, set transforms, materials, meshes, move camera) using Lua scripts and interactive commands directly.

This will likely mean a lot of typing, which is the way I roll, but I'm thinking most users will want a GUI. So I'll ponder on how I can set things up so that a GUI frontend can run in-process and send the required LUA commands to the engine. That will be another thing someone can help with (trust me... you don't want to be punished with one of my GUI's)
Logged
Dwarf Fortress - "It's like if Tolkien taught your Geology 101 class, but you were drunk and playing the Sims during the lecture. Then you had a dream that got them all confused."

Obsidian 3D Visualizer Toolset - http://df.magmawiki.com/index.php/Utility:Obsidian
A Lua-riffic collection of modules to render, manipulate, save and generally mess with your forts.

sizeak

  • Bay Watcher
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #23 on: May 23, 2010, 07:29:31 pm »

If you use Qt for the GUI, it'll be cross platform and also comes with .Net style drag and drop GUI designer
Logged
Runesmith - http://www.bay12forums.com/smf/index.php?topic=59056.0 - A Dwarf Companion like tool

Eduren

  • Bay Watcher
  • A new theme!
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #24 on: May 23, 2010, 08:50:10 pm »

Good luck! Unfortunately, the only way I can help is eventual testing.
Logged
I don't know.  Duke wants me to stop playing mafia.
That's the sign of an abusive boyfriend, Toony... you don't have to listen to him.

Innominate

  • Bay Watcher
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #25 on: May 24, 2010, 05:02:03 am »

I was going to suggest OpenSceneGraph! It took me about 30 minutes (after installation) to get a simple configurable level displaying using one of the standard camera manipulators. It's oh-so practical, letting you worry about high-end details rather than figuring out an efficient graphics pipeline and memory management. Unfortunately then I ran out of leisure time to continue.

I can do any menial programming tasks that are needed. One suggestion (if it can be implemented with Lua, don't worry): provide support for loading levels from inputs other than DFHack. So the graphics interfaces with some abstract input class, which has subclasses to support reading from DFHack output and from flat files (the non-DF subclasses could be implemented after everything else is working). That way we could visualise fort designs before we implemented them; all we'd need to do is get the design into the right format.
Logged

Djohaal

  • Bay Watcher
  • [PREFSTRING:Utter Insanitiy]
    • View Profile
    • My deviantart
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #26 on: May 24, 2010, 11:12:59 am »

I was going to suggest OpenSceneGraph! It took me about 30 minutes (after installation) to get a simple configurable level displaying using one of the standard camera manipulators. It's oh-so practical, letting you worry about high-end details rather than figuring out an efficient graphics pipeline and memory management. Unfortunately then I ran out of leisure time to continue.

I can do any menial programming tasks that are needed. One suggestion (if it can be implemented with Lua, don't worry): provide support for loading levels from inputs other than DFHack. So the graphics interfaces with some abstract input class, which has subclasses to support reading from DFHack output and from flat files (the non-DF subclasses could be implemented after everything else is working). That way we could visualise fort designs before we implemented them; all we'd need to do is get the design into the right format.

Actually, that is an excellent idea that we should branch. Perhaps a tool for creating a saveable dump of the fortress data so the auto-record timelapse function could record the fortress data into save files that can be revisited later on. Maybe such format (that I propose should be called .hfs) could be made universally readable with dfHack (memory dumps maybe?) so we could exchange data between obsidian and stonesense saves.
A 2d visualizer also would be interesting, I personally find DF's graphical dump function too intrusive and clumsy. I am aware that such DF memory hack dumps would cause lag, but not having to leave the interface, as it runs in background and automatically snapshots seems interesting enough. Maybe with some image analysis we could even do the way back and make 3d models out of 2d bitmap maps. Reading stuff from Dfma anyone?  :P

Maybe later on a 2d/3d voxel editor (I personally prefer DF's tomographic projection to handle fort design) could be implemented to design the fort. Maybe we could use some sort of palette that imports graphical tiles from DF tilesets and displays them properly.

Back to the art asset, once we get the procedural tinting for materials up and running, I think two palettes should be implemented, a standard Urist Decahexagonal colorset that displays the colors as they are in DF, and an extended palette that we could combine with stonesense.
Logged
I really want that one as a "when". I want "grubs", and "virgin woman" to turn into a dragon. and monkey children to suddenly sprout wings. And I want the Dwarven Mutant Academy to only gain their powers upon reaching puberty. I also have a whole host of odd creatures that only make sense if I divide them into children and adults.

Also, tadpoles.

Skeggox

  • Bay Watcher
  • Likes obsidian for its shinyness
    • View Profile
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #27 on: May 24, 2010, 01:49:15 pm »

You see, that's the kind of stuff I had planned for Obsidian!

My original idea (and I've got a large piece of paper with some architectural diagrams on it around here somewhere... *rummage*)
was to have an intermediate format for storing the DF world in. DFHack will be one of the standard loaders for this format, i.e. loading a map directly from DF. But you could make your own loader, i.e. load the map directly from a voxel-based fort editor, etc.
Hell, if Toady has the time/inclination he could even decide to write a DF save file loader for offline visualizing ;) The format will also provide serializers for saving to/loading from storage in whatever format you could shake a stick at.

The reason for an intermediate format is twofold:
1) going the memory dump route would tie each file too closely to the platform - so sharing saved forts between linux, max and windows versions of Obsidian would be a nightmare.
2) An intermediate format can be made version-less and support best-effort implementation by tools.

The second reason needs some explanation - if you take a step back and look at the problem from a different viewpoint (licking a frog might help) we're basically dealing with a highly structured collection of information. You've got a bunch of discrete cells in a 3D cube - each cell has certain base properties and can contain a list of objects (items, creatures, constructions). The cell itself can be assigned to other objects, either singly or in groups (buildings, burrows, etc.) The rest, all of the niggling details, are just the effect of each cell/object having different properties/tags/metadata.

Toady will most likely shift the details around a lot, but the concept of a cube of cells won't change that quickly. So, as long as any tool that uses this format can handle the concept of a cube of cells, it will be able to load a saved/designed fort.

It might not understand all of the properties of a cell/object (Toady might have introduced new building types, or constructions) but it would be able to load it and maybe display a ? mark or something, instead of crashing or throwing a tantrum. And it can write the cells back out after it's work is done, passing through anything it doesn't understand just in case the next tool or a future version of itself will.

Such an intermediate format is an ambitious undertaking, true, but we do that regularly in the games industry ... and as long as you stay away from a few pitfalls it is very definitely doable.

Logged
Dwarf Fortress - "It's like if Tolkien taught your Geology 101 class, but you were drunk and playing the Sims during the lecture. Then you had a dream that got them all confused."

Obsidian 3D Visualizer Toolset - http://df.magmawiki.com/index.php/Utility:Obsidian
A Lua-riffic collection of modules to render, manipulate, save and generally mess with your forts.

Djohaal

  • Bay Watcher
  • [PREFSTRING:Utter Insanitiy]
    • View Profile
    • My deviantart
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #28 on: May 24, 2010, 02:10:45 pm »

I mentioned memory dump more because of foresight considerations. Figuring out how DF stores the engraved status for rock walls so far seems to be a mystery (I might be incredbly wrong here, maybe people found out how it works, I recall reading some stuff around the stonesense thread that this wasn't quite figured out yet) and I beleive there might be some other entities in DF's memory that we didn't make sense yet. A memory dump would allow for saves up from version 1 to have all their data analysed by a posterior implementation of DFhack or whatever, while there would be a loss of information if a dedicated save file was implemented. (although it is a much more elegant solution)
Logged
I really want that one as a "when". I want "grubs", and "virgin woman" to turn into a dragon. and monkey children to suddenly sprout wings. And I want the Dwarven Mutant Academy to only gain their powers upon reaching puberty. I also have a whole host of odd creatures that only make sense if I divide them into children and adults.

Also, tadpoles.

peterix

  • Bay Watcher
    • View Profile
    • Dethware
Re: Obsidian: A new DF 3D visualizer toolset
« Reply #29 on: May 24, 2010, 02:50:06 pm »

The reason for an intermediate format is twofold:
1) going the memory dump route would tie each file too closely to the platform - so sharing saved forts between linux, max and windows versions of Obsidian would be a nightmare.
2) An intermediate format can be made version-less and support best-effort implementation by tools.

The second reason needs some explanation - if you take a step back and look at the problem from a different viewpoint (licking a frog might help) we're basically dealing with a highly structured collection of information. You've got a bunch of discrete cells in a 3D cube - each cell has certain base properties and can contain a list of objects (items, creatures, constructions). The cell itself can be assigned to other objects, either singly or in groups (buildings, burrows, etc.) The rest, all of the niggling details, are just the effect of each cell/object having different properties/tags/metadata.

Toady will most likely shift the details around a lot, but the concept of a cube of cells won't change that quickly. So, as long as any tool that uses this format can handle the concept of a cube of cells, it will be able to load a saved/designed fort.

It might not understand all of the properties of a cell/object (Toady might have introduced new building types, or constructions) but it would be able to load it and maybe display a ? mark or something, instead of crashing or throwing a tantrum. And it can write the cells back out after it's work is done, passing through anything it doesn't understand just in case the next tool or a future version of itself will.

Such an intermediate format is an ambitious undertaking, true, but we do that regularly in the games industry ... and as long as you stay away from a few pitfalls it is very definitely doable.
About the platform-specific data: Surprisingly, there's very little of it. It's mostly the layout of STL objects, and the way RTTI information/inheritance is implemented. The actual data might be shifted a bit between Linux/GCC and Windows/MSVC, but the values are the same. DFHack filters the platform differences away.

Also, note that there's no OSX DFHack. There was some guy working on it, but I haven't seen any results.

Anyway, an intermediate format is a good idea. Khazad did have a save/load feature, but it was closely tied to the way DF stores them in memory. The format became pretty unwieldy over time ~_~
Pages: 1 [2] 3 4 ... 15