Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[ungulateman]

And you haven't explained it, which makes you either a troll, inattentive or unwilling to admit that they're right for some reason.

So leave, explain yourself or explain yourself some more, respectively.

[NKDietrich]

Vigilant is correct, at least in terms of Windows XP. Windows XP has relatively poor memory management. It will leave a bunch of empty memory and use the pagefile when it isn't really necessary. Windows Vista and 7 are far better in that respect. Vista/7 actually have a very smart system for caching stuff in memory. Disabling the pagefile is an archaic concept that only ever applied to older operating systems, and even then it had some drawbacks. (Certain apps would crash and burn.) Anyone still recommending it for Vista and 7 are just holding on to an old habit.

But the best thing is to simply try it for yourself. It's not like it's going to hurt anything to turn it off, then back on if you don't see any benefit.

[Socializator]

This post doesnt really contribute to virtual memory yes or no, but rather discuss"

Vista/7 actually have a very smart system for caching stuff in memory.

Now I dont want to nitpick, but that smart caching system on my Vista64 was caching half downloaded  xGBs file into RAM. I can only guess, but his crystal ball assumed, that it will speed things up if uTorrent will have this in RAM right after the start, as it was used by uTorrent for several windows sessions... :-/
This wasnt sole problem, online game Battleforge checks its own integrity (like every online game) before letting you play. which is several gigs of files to check(sum). As this is only at game launch, it doesnt really matter that much. Yet, this Vista's precaching visioner decided to load it into memory right after the startup, just in case...
Both of this cases are nice example of complete lack of "timing"... Startup is one of the most critical times on my system - multiple apps competing for the resources already... why start this precaching at all? putting more strain on HDD...

Bah, I guess I did nitpick after all

But this is just to illustrate that REAL LIFE memory management is quite complicated thing and that "motorbitch" is just trolling some typical ideal world uni knowledge.

[ZhangC1459]

now now, just because he thinks he's right doesn't mean he's trolling.

Granted, I know nothing about this sort of thing, academic ideal or real life, so I can't join in this discussion.

[janglur]

Regarding clutter causing lag:

Does this only extend to loose clutter like barrels or stones? Or does clutter also include the blocks that have been built into megaprojects? Do immobile objects like that still constitute a drain on RAM?

It seems to count everything, including massive amounts of layers (water, blood, etc.)
Some testing in arena mode after a massive 5,000 vs. 5,000 killed framerate too after they painted the town red.

[janglur]

Also, I should note my tests were on various versions of XP.  XP-64 Pro, XP-64 Pro, XP-32 Pro, and Xp-32 Media Center

[Vigilant]

Vigilant is correct, at least in terms of Windows XP. Windows XP has relatively poor memory management. It will leave a bunch of empty memory and use the pagefile when it isn't really necessary. Windows Vista and 7 are far better in that respect. Vista/7 actually have a very smart system for caching stuff in memory. Disabling the pagefile is an archaic concept that only ever applied to older operating systems, and even then it had some drawbacks. (Certain apps would crash and burn.) Anyone still recommending it for Vista and 7 are just holding on to an old habit.

But the best thing is to simply try it for yourself. It's not like it's going to hurt anything to turn it off, then back on if you don't see any benefit.

Vigilant uses XP

However still, why would caching stuff in the page file ever be more useful even with smarter management? Even with a solid state drive you're not going to get anywhere near the access speeds being as fast as RAM. Virtual memory does seem like a great hardware solution for making more efficient use of resources... but I can't see how it improves performance, it looks like it just reduces the impact of having an insufficient amount of memory in your machine. I guess i could see it maybe extending RAM life, but that's about it.

[darkflagrance]

Regarding clutter causing lag:

Does this only extend to loose clutter like barrels or stones? Or does clutter also include the blocks that have been built into megaprojects? Do immobile objects like that still constitute a drain on RAM?

It seems to count everything, including massive amounts of layers (water, blood, etc.)
Some testing in arena mode after a massive 5,000 vs. 5,000 killed framerate too after they painted the town red.

So in other words, Dwarf Fortress is guaranteed to grind in a halt if you in any way attempt to enjoy the game

[janglur]

Vigilant is correct, at least in terms of Windows XP. Windows XP has relatively poor memory management. It will leave a bunch of empty memory and use the pagefile when it isn't really necessary. Windows Vista and 7 are far better in that respect. Vista/7 actually have a very smart system for caching stuff in memory. Disabling the pagefile is an archaic concept that only ever applied to older operating systems, and even then it had some drawbacks. (Certain apps would crash and burn.) Anyone still recommending it for Vista and 7 are just holding on to an old habit.

But the best thing is to simply try it for yourself. It's not like it's going to hurt anything to turn it off, then back on if you don't see any benefit.

Vigilant uses XP

However still, why would caching stuff in the page file ever be more useful even with smarter management? Even with a solid state drive you're not going to get anywhere near the access speeds being as fast as RAM. Virtual memory does seem like a great hardware solution for making more efficient use of resources... but I can't see how it improves performance, it looks like it just reduces the impact of having an insufficient amount of memory in your machine. I guess i could see it maybe extending RAM life, but that's about it.

Because pagefile is used for two primary reasons, assuming optimal use.  The first is to cache an 'original' copy while a loaded copy is altered in memory.  This mirroring lets it re-load the original if need be.
The second is that not all data in-use is used constantly.  DF is an example of something that must never, ever be paged off.  Whereas something like, say, my print spooler service?  I print once a year, maybe.  So paging that (rather large) mass of DLLs off to pagefile, where it will be accessible but not really accessed, is far more memory efficient.  It also reduces power consumption (less cells needing the approx. 2us refresh signal in RAM), reducing RAM competition (less crap to check on, which is a wasted cycle if it's not being read/written), less RAM fragmentation (a small gain at best, but..), and more RAM free for tasks that really need it.

Realistically, pagefiles tend to accumulate a lot of unnecessary crap that programs don't need at all, but would otherwise go to RAM.  Photoshop is an incredible masterpiece of unmentionable clutter for this reason.

Just open Task Manager and enable the 'Virtual Memory' column and see how much each program has allocated.  Notice that many of those allocations are unnecessarily large and not even in use.  With Pagefile turned off, this is *wasted* RAM.  Most people these days don't care since 4GB+ is the norm.  But in extreme system duress, that clutter makes a difference.

For example, RAM is the great barrier for how large an embark you can get.  (FPS is a different issue).  I recently had my friend run DF on his server and he can embark on a FULL SQUARE.  I beleive that's like 32x32 or 64x64, I forget.  But yeah, he takes the *entire* area up and can run it.  (FPS is, again, a whole different thing.)  He has 24 GB of RAM on a server board.  (Also dual quad opties).  Ironically, his still runs slower than mine.

[Fedor]

Good thread going on here.

About huge embarks:  I run a 15 750 with 4 GB of RAM and can embark without too much wait on a full square.  Playing's way laggy, but the embark itself goes well. Conclusion:  You don't need more than 4 GB to do full-square embarks, but you do need a fast, capable CPU.

About ganged and unganged memory:  I had never head of such a thing, so I looked it up and learned (I think) that it's an AMD-exclusive thing.  Intel doesn't have it.  If true, this should be mentioned in the OP.  Speaking more broadly, even if I mistake here, the advice given in the OP should be clearer about the fact that it only fully applies to specific CPUs and OSes and will not necessarily be valid for others.

[Graebeard]

Regarding clutter causing lag:

Does this only extend to loose clutter like barrels or stones? Or does clutter also include the blocks that have been built into megaprojects? Do immobile objects like that still constitute a drain on RAM?

It seems to count everything, including massive amounts of layers (water, blood, etc.)
Some testing in arena mode after a massive 5,000 vs. 5,000 killed framerate too after they painted the town red.

So in other words, Dwarf Fortress is guaranteed to grind in a halt if you in any way attempt to enjoy the game

Disclaimer:  I know nothing of CS.

That said, it seems like this issue is relatively easy to address.  Is there a reason to count constructed blocks and stone?  Is there a reason to query the status of objects not in use as frequently as objects currently being used?  Coverings that are easier to clean or that eventually decay would help.

If I had one wish for the future of DF it would be framerate improvement.  Lag has killed more forts than gobbos ever could.  It would be great to have succession games that could last more than 6-8 turns before becoming unplayable.

[Vigilant]

I know of CS. Unless the code base is particularly neat and pretty for this section of stuff (highly unlikely given what we've heard about most of the code base), there's probably no way around this without completely redoing those parts. So not easy

[motorbitch]

And you haven't explained it, which makes you either a troll, inattentive or unwilling to admit that they're right for some reason.

So leave, explain yourself or explain yourself some more, respectively.

what makes you think you may tell me to go or leave anywhere? and what makes you think that  its my duty to explain anything?

however... i realy dont think one should try to tell the working principles of operation systems within a gaming forum.

1: there are *tons* of real good background informations regarding these techniques, even on wikipedia most informations are correct.

2: its obvious that some informations have been read -and misread - already... i dont belive telling it all agian would do the trick.

[ungulateman]

And you haven't explained it, which makes you either a troll, inattentive or unwilling to admit that they're right for some reason.

So leave, explain yourself or explain yourself some more, respectively.

what makes you think you may tell me to go or leave anywhere? and what makes you think that  its my duty to explain anything?

however... i realy dont think one should try to tell the working principles of operation systems within a gaming forum.

1: there are *tons* of real good background informations regarding these techniques, even on wikipedia most informations are correct.

2: its obvious that some informations have been read -and misread - already... i dont belive telling it all agian would do the trick.

I have the right to suggest for you to go or to leave because you're making the environment of the thread worse. Clearly I can't force you to, but that doesn't mean I can't suggest that you do.

The working principles of "operation systems" is the entire point of this thread. Acting like you shouldn't explain yourself after you say someone else is wrong without giving an alternative point of view is just insulting them.

Link, please, if it's that easy to find. I'm sure that if you want to explain what's going on providing me with information is the smartest thing to do, if not the easiest.

(P.S. Correct your grammar and spelling before you post please.)

[Socializator]

And you haven't explained it, which makes you either a troll, inattentive or unwilling to admit that they're right for some reason.

So leave, explain yourself or explain yourself some more, respectively.

what makes you think you may tell me to go or leave anywhere? and what makes you think that  its my duty to explain anything?

however... i realy dont think one should try to tell the working principles of operation systems within a gaming forum.

1: there are *tons* of real good background informations regarding these techniques, even on wikipedia most informations are correct.

2: its obvious that some informations have been read -and misread - already... i dont belive telling it all agian would do the trick.

Still some people manage to be quite informative... Also I wouldnt call this forum a typical gaming one.
If you have some exact information how the memory manager in XP and especially Vista/7 works, please share. Especially the later one is sometimes intriguing.
The thing is that most common operating system still is XP. This OS was designed some time (is it a decade now?) ago, where typical desktop, not even talking about laptop, had much smaller amounts of RAM available. Nowadays situation is quite different. Disks are still slow, but most of the folks around have lot of memory available.
How can this make a change? Well, ideally not, since "first use RAM, then swapfile" should be always right, but this simply is not a case. For example program (e.g. Dwarf Fortress, or windows update service) wants to allocate some additional memory. It may be hundreds of MB. RAM conserving manager will allocate it into swap, and moves it into RAM later, when the data starts pouring in, as these two moments might be quite distinctively different. But this inherently implies additional slowdown in the program run. Different manager might allocate it to RAM directly. But this may result in lots of allocated but unused RAM. The second one doesnt matter that much now, but it did much more few years ago.
What I experienced with XP was that when I was playing Civilization and was alttabing when waiting for turns, the manager always managed to move the background application's memory to swap. So every alttab was painful. He was actively cleaning RAM from unused memory. although I had enough (by my standards ofc) free of it.
Disabling the pagefile made huge difference. Point is, that the actual manager can be made in many different ways and saying "You are wrong" and "This forum is not worth it" can be indeed understand a bit trollish.