Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2 3 4

Author Topic: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)  (Read 46684 times)

Anticheese

  • Bay Watcher
    • View Profile

With the new version of Dwarf Fortress adding even more depth to the world, it makes sense to take every advantage you can to help the game run even smoother.

To this end, we've activated the LAA - Large Address Aware - flag for Dwarf Fortress, which allows 32-bit systems to use up to 3GB of RAM, and 64-bit systems to use up to 4GB. You may make Skyrim jokes.

You can find the file here

You can find the original thread at http://forums.nothingtolost.com/dicestation/viewtopic.php?f=6&t=62. A link is included to the utility that was used to produce this mod.
Logged
Why not join us on IRC? irc.newnet.net #bay12games

Girlinhat

  • Bay Watcher
  • [PREFSTRING:large ears]
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #1 on: February 16, 2012, 02:27:32 am »

I'm not the most intimate with computer workings, but isn't the main issue with DF processing speed, not really the memory size?  Does this change have any worthwhile effect on speed?

Plasson

  • Bay Watcher
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #2 on: February 16, 2012, 03:17:56 am »

the process uses ~850k with the regular, and this here SDL exe.  don't seem to have any effect.
Logged

Llamageddon

  • Bay Watcher
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #3 on: February 16, 2012, 08:20:09 am »

I'm not the most intimate with computer workings, but isn't the main issue with DF processing speed, not really the memory size?  Does this change have any worthwhile effect on speed?

I believe many people are having trouble with worldgen crashing on large worlds or particularly long histories as they RAM usage of the program is exceeding 2GB. This is the default upper limit for a normal windows program I think, hence the crash. The file in the OP flags the EXE so windows "knows"(?) the program can use 4GB of RAM (3GB in XP) and won't crash when the normal upper limit of 2GB is reached. I'm pretty certain it should have practically no impact on the actual speed of worldgen as that is pretty much exclusively tied to your CPU.

Skyrim had a similar problem when people started installing loads of custom high texture packs etc, hence the Skyrim reference in the OP. My knowledge of the specifics is a bit ropey too so take the above explanation with a pinch of salt. I do find it hilarious that the new DF with it's super high fidelity graphics  ;) is having the same problem as Skyrim though. All hail Toady, Bethesda needed mods added to achieve this.

Edit: Kind of missed the point there didn't I. I don't want to contradict the author but I kind of agree that Dwarf Fortress game speed is not very RAM dependent as far as I know. I assume it should help with large embarks/adventure mode area transitions and things though, basically anything where the game is loading lots of information to and from memory rather than simulating tantrums/combat etc.

Edit 2: Hope I haven't made a fool of myself sounding like an expert when I'm not. Anyway, just wanted to say thank you very much for posting this Anticheese, it is just what I was looking for.
« Last Edit: February 16, 2012, 08:32:15 am by Llamageddon »
Logged

EmperorJon

  • Bay Watcher
  • Still waiting...
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #4 on: February 16, 2012, 08:34:27 am »

World gen will probably be effected, indeed.

I've said this many times, and I know Toady doesn't particularly agree, but it'd be perfectly happy to see him take a year or two out to rebuild the game multithreaded... :p
Logged
I think it's the way towns develop now. In the beginning, people move into a town. Then they start producing tables, which results in more and more tables. Soon tables represent a significant portion of the population, they start lobbying for new laws and regulations, putting people to greater and greater disadvantage...
Link for full quote. 'tis mighty funny.

Girlinhat

  • Bay Watcher
  • [PREFSTRING:large ears]
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #5 on: February 16, 2012, 08:52:00 am »

So, to clarify, this change will help curb some woldgen crashes for very large, old worlds?  That's much more relevant and probably should have been made very clear.  As it stands, the actual purpose of this project wasn't made clear, only that it had been done.

MagmaSolutionsInc

  • Bay Watcher
  • Likes elves for their screaming, bubbling noises.
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #6 on: February 16, 2012, 05:25:37 pm »

Much love here - thx  :D
Logged
stopped zombie cats from adopting dwarves

... with magma.

Slax

  • Bay Watcher
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #7 on: February 16, 2012, 05:37:54 pm »

I do this for just about every game these days. Generally just speeds things up, it feels like. Placebo or not. :P
Quick and easy.
Logged

Rafal99

  • Bay Watcher
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #8 on: February 16, 2012, 06:06:15 pm »

Awesome, thanks!
I wasn't aware such thing is possible.

Edit: I just compared the exes and noticed they only differ at one byte. This is the most awesome single byte I have seen.
« Last Edit: February 16, 2012, 06:11:11 pm by Rafal99 »
Logged
The spinning Tantrum Spiral strikes The Fortress in the meeting hall!
It explodes in gore!
The Fortress has been struck down.

Maklak

  • Bay Watcher
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #9 on: February 16, 2012, 06:15:11 pm »

So, to clarify, this change will help curb some woldgen crashes for very large, old worlds?  That's much more relevant and probably should have been made very clear.  As it stands, the actual purpose of this project wasn't made clear, only that it had been done.

But there was an answer: "It has been asked whether this will speed up DF, the answer is generally no. The point of this is to prevent the memory from reaching crash point during worldgen or late game when there is so much crap for the game to remember."
Logged
Quote from: Omnicega
Since you seem to criticize most things harsher than concentrated acid, I'll take that as a compliment.
On mining Organics
Military guide for FoE mod.
Research: Crossbow with axe and shield.
Dropbox referral

bombzero

  • Bay Watcher
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #10 on: February 16, 2012, 07:14:08 pm »

World gen will probably be effected, indeed.

I've said this many times, and I know Toady doesn't particularly agree, but it'd be perfectly happy to see him take a year or two out to rebuild the game multithreaded... :p

how would you multi-thread DF? the reason modern games CAN be multi-threaded is that there are fewer things that need to be placed on the different cores, all of which are not too interdependent.
in DF however almost every single process is interdependent and thus fore is hard to separate in separate cores.

not trying to be an ass, just explaining the technical implications of what you think.
Logged

Jay

  • Bay Watcher
  • ☼Not Dead Yet☼
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #11 on: February 16, 2012, 08:11:32 pm »

Quite.  From a technical standpoint, there are no two actions that can run without interacting with each other, other than the game itself / the graphics, which are already on their own threads.
Thread collision (when multiple threads that can potentially interact with each other inevitably modify the same thing at the same time -- like kids who don't know how to share a toy) rectification creates a much larger problem than the multithreading would solve.
Logged
Mishimanriz: Histories of Pegasi and Dictionaries

telamon

  • Bay Watcher
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #12 on: February 16, 2012, 08:17:40 pm »

thanks for this, DF kept crashing whenever i did world gens with more than 300 years of history. how much RAM should i be expecting it to use though? i'm doing this on a laptop which tops out at about 2GB. probably not enough for that much history, eh?

EDIT: and would it be possible to just cache the world's history to hard drive as it generates? ie don't hold it all in RAM until it's done, write it piece by piece.
« Last Edit: February 16, 2012, 09:05:54 pm by telamon »
Logged
Playing DF on Windows 98 since.... ?
At 55 frames per minute.

jwiechers

  • Escaped Lunatic
    • View Profile
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #13 on: February 16, 2012, 08:26:49 pm »

Quite.  From a technical standpoint, there are no two actions that can run without interacting with each other, other than the game itself / the graphics, which are already on their own threads.
Thread collision (when multiple threads that can potentially interact with each other inevitably modify the same thing at the same time -- like kids who don't know how to share a toy) rectification creates a much larger problem than the multithreading would solve.

Nah, interdependence is indeed a major issue with multithreading, but it isn't as bad as you suggest, it just requires careful handling and checking, which is one of the reasons why rewriting something to be *efficiently* multithreadable is next to impossible, but newer software can generally be made to utilize cores very effectively even with huge interdependencies.

As for Large Address Aware: Good call, I routinely do that with Dwarf Fortress and other memory hogging software, too. Helps quite a bit and prevents crashes up to 12x12 embarks for me, sometimes even slightly larger ones. Also allows 2000+ years of history in large worlds.
Logged

Eagleon

  • Bay Watcher
    • View Profile
    • Soundcloud
Re: Dwarf Fortress - Large Address Aware (Or, let DF use up to 4GB of RAM!)
« Reply #14 on: February 16, 2012, 09:44:08 pm »

I've needed this for a while. Unfortunately my main computer has a paltry 2 gb (albeit on a streamlined lighter version of XP, so for most applications it's no issue), but using my partner's computer I successfully genned a world with maxed out vamp/werebeast/etc. to over 100000 211907(edit) historical figures(!) and no limit on sites or population after worldgen, and have successfully loaded it and saved in adventure mode with no issue on my own computer. It had 3 ages of legends. Should be interesting to explore.

I'm surprised this hasn't been a bigger issue. It's been very easy for me to run into this limit during worldgen, particularly with some of the larger mods and the crazier settings I've put into worldgen. Still haven't tried bumping up megabeast percentages for that 'epic struggle' feel, but it only reached about 2.7 gb in memory during worldgen as far as I saw so it shouldn't be an issue. I'd love for Toady to make DF handle overload conditions a bit better, even if it resorted to using virtual memory sometimes. Anything's better than crashing after an extended worldgen.
« Last Edit: February 16, 2012, 09:55:53 pm by Eagleon »
Logged
Agora: open-source, next-gen online discussions with formal outcomes!
Music, Ballpoint
Support 100% Emigration, Everyone Walking Around Confused Forever 2044
Pages: [1] 2 3 4