Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 105 106 [107] 108

Author Topic: DFHack 0.44.12-r2  (Read 120211 times)

TV4Fun

  • Bay Watcher
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1590 on: January 03, 2019, 06:32:30 pm »

EDIT3: Actually, apparently it's in PE format, because PE files also start with MZ. Wow, good job, Microsoft.
I think the MZ prefix on PE files is done for backward compatibility. The MZ format was used by DOS, and if you try to open a Windows executable in DOS, it will show a message saying it needs to be run under Windows.
Logged

bloop_bleep

  • Bay Watcher
  • U want sum seed?
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1591 on: January 03, 2019, 08:35:24 pm »

The only difference on Windows is that the offset isn't fixed across DF runs

Really? That doesn't make much sense, because the address for every instruction would vary, so the loader would have to change the parameter for every absolute jump every time DF runs. Or am I missing something?

I think the way you calculate the file offset from the memory address is using the formula

file_offset = memory_address - (VirtualAddress + ImageBase) + PointerToRawData

where ImageBase is in the optional header and VirtualAddress & PointerToRawData are in the section header entry for .text. I looked in the executable and apparently

ImageBase = 0x00 40 00 00
VirtualAddress = 0x00 10 00 00
PointerToRawData = 0x00 04 00 00

so that should be enough to calculate any file offset you wish.
Logged
bloop_bleep's Moving Fortress Parts Mod!

Quote from: KittyTac
The closest thing Bay12 has to a flamewar is an argument over philosophy that slowly transitioned to an argument about quantum mechanics.
Quote from: thefriendlyhacker
The trick is to only make predictions semi-seriously.  That way, I don't have a 98% failure rate. I have a 98% sarcasm rate.

lethosor

  • Bay Watcher
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1592 on: January 04, 2019, 01:03:34 am »

Yes, really, it's ASLR. Specifically, ImageBase in your formula isn't fixed across runs. DF disables ASLR on every platform but Windows. As for jumps, most jumps are relative, not absolute. Those that are absolute probably do some calculations to account for this.

Process-windows.cpp deals with this extensively, by the way, as does devel/find-offsets.lua and the lua libraries it uses.
Logged
DFHack - Dwarf Manipulator (Lua) - DF Wiki talk

There was a typo in the siegers' campfire code. When the fires went out, so did the game.

bloop_bleep

  • Bay Watcher
  • U want sum seed?
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1593 on: January 04, 2019, 01:47:51 am »

Wow, sorry, I forgot that existed.

I'll take a look at those files to see how they work, out of curiosity. Man, it must be difficult to delve into such a low level and understand all the myriad, obscure systems at play there, and I can only imagine how much more difficult it must be to debug it when your program inadvertently jumps into the middle of printf which then attempts to read about a dozen floats from its arguments, thus completely destroying the stack and any trace of what went wrong or even which function caused the problem.

I think I'll stay with my safe, predictable high-level programming for now.  :P
Logged
bloop_bleep's Moving Fortress Parts Mod!

Quote from: KittyTac
The closest thing Bay12 has to a flamewar is an argument over philosophy that slowly transitioned to an argument about quantum mechanics.
Quote from: thefriendlyhacker
The trick is to only make predictions semi-seriously.  That way, I don't have a 98% failure rate. I have a 98% sarcasm rate.

Rumrusher

  • Bay Watcher
  • current project : searching...
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1594 on: January 04, 2019, 05:11:08 am »

so uhh this camp boat stuff seem to be opening to weird discoveries

so far to list things off, Camps keep every change done to it, also keeps any units that were on the site, and any smaller sites that can load up onto the camp will be saved to the camp when you unload the map, and those changes keep with the camp including anyone from the site that got merged with the camp are now with the camp citizens.

if someone say builds a camp in the ocean then moves the camp the camp would probably be a giant pool of water.

Logged
I thought I would I had never hear my daughter's escapades from some boy...
DAMN YOU RUMRUSHER!!!!!!!!
"body swapping and YOU!"
Adventure in baby making!Adv Homes

TV4Fun

  • Bay Watcher
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1595 on: January 04, 2019, 12:16:18 pm »

According to Wikipedia, the PE format generally includes absolute addresses, and if it's not loaded at its preferred address, the loader has to manually go through and change every address in the code before running. Yes, really.
Logged

Warmist

  • Bay Watcher
  • Master of unfinished jobs
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1596 on: January 04, 2019, 04:45:46 pm »

According to Wikipedia, the PE format generally includes absolute addresses, and if it's not loaded at its preferred address, the loader has to manually go through and change every address in the code before running. Yes, really.
Yeah that usually happens for DLLs. You can not expect the code to be loaded in same location as there could be other dlls that want to be in that location (even without ASLR). So there is such thing as relocation table that list all addresses that need to be fixed when loading. You could compile code as PIC however i have no idea what are the pros and cons of that.

lethosor

  • Bay Watcher
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1597 on: January 04, 2019, 05:02:37 pm »

DF isn't a DLL though, although maybe the same thing applies. In any case, what matters for the matter at hand is that we know the DF executable isn't always loaded at the same place in memory, because we have to account for that to adjust everything in symbols.xml when DFHack starts. getRebaseDelta should help; if not, let me know.
Logged
DFHack - Dwarf Manipulator (Lua) - DF Wiki talk

There was a typo in the siegers' campfire code. When the fires went out, so did the game.

Roses

  • Bay Watcher
    • View Profile
Re: DFHack 0.44.12-r1
« Reply #1598 on: January 10, 2019, 05:09:16 pm »

I probably would've stuck with roses stuff if I'd been able to get the JSON saving to reliably save with the game. I couldn't figure out a way to tell if quicksaving/autosaving was going on. The problem was more that every single table check was a persist-table check, IIRC.

So, circling back to this, I recently tried moving some of the persist-table stuff I use to use the JSON save/load stuff. It seemed to go fine and was able to bypass the slowdown with persist-table checks and the overhead associated with the persistent tables itself, so I was thinking to moving almost entirely to the JSON stuff, but you mentioned it had trouble, so I wanted to ask about any troubles associated with it before I spent any more time working on it.
Logged

Putnam

  • Bay Watcher
  • DAT WIZARD
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1599 on: January 11, 2019, 01:34:08 am »

Straight up couldn't find a way to get it to save only when the game saves.

Roses

  • Bay Watcher
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1600 on: January 11, 2019, 12:43:24 pm »

Straight up couldn't find a way to get it to save only when the game saves.

Yeah, I just reread your first post, don't know why I didn't really understand what you were talking about the first time I read it. I saw it and I thought, isn't that what the ON_UNLOAD (or whatever it's called) event is for, but I forgot about autosaves and quicksaves. Hmmm, well I'm going to have to look into it a bit, because even if I don't use the JSON stuff I would still like to load the persistent data tables into global tables on start and unload them on save/exit and that will have the same problem. I'll let you know if I can find anything, but if you weren't able to I'm not super thrilled about my chances.
Logged

lethosor

  • Bay Watcher
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1601 on: January 11, 2019, 01:45:17 pm »

Does saving .dat files into data/save/current not work? That's what https://github.com/DFHack/dfhack/pull/1402/files does, at least.
Logged
DFHack - Dwarf Manipulator (Lua) - DF Wiki talk

There was a typo in the siegers' campfire code. When the fires went out, so did the game.

Roses

  • Bay Watcher
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1602 on: January 11, 2019, 02:32:10 pm »

I can save and load the files into/from data/save/current on world unload/load just fine. But autosaves and quicksaves are giving me trouble, although it looks like that link you gave does save on each save (no matter how the save is initiated). Now how to translate that into something available to lua
Logged

lethosor

  • Bay Watcher
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1603 on: January 11, 2019, 03:29:09 pm »

I was on mobile earlier, so that probably wasn't clear enough. I'm saying you can save what you want into data/save/current/foo.dat when the data changes, not when DF saves. When DF saves (regardless of the type of save), those files will be copied into the region folder. The only save detection in that PR is to allow plugins to do anything special needed when the game is saved. Granted, if you're writing stuff to disk frequently, that will be bad for performance, and plugins in that PR could potentially only write to disk in plugin_save, but you almost certainly don't need to write very often in most cases.
Logged
DFHack - Dwarf Manipulator (Lua) - DF Wiki talk

There was a typo in the siegers' campfire code. When the fires went out, so did the game.

bloop_bleep

  • Bay Watcher
  • U want sum seed?
    • View Profile
Re: DFHack 0.44.12-r2
« Reply #1604 on: January 11, 2019, 06:52:44 pm »

I found I had to load from the main world folder (not /data/save/current) even though I could save to either.
Logged
bloop_bleep's Moving Fortress Parts Mod!

Quote from: KittyTac
The closest thing Bay12 has to a flamewar is an argument over philosophy that slowly transitioned to an argument about quantum mechanics.
Quote from: thefriendlyhacker
The trick is to only make predictions semi-seriously.  That way, I don't have a 98% failure rate. I have a 98% sarcasm rate.
Pages: 1 ... 105 106 [107] 108