Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 192 193 [194] 195 196 ... 243

Author Topic: DFHack 50.13-r1  (Read 810554 times)

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2895 on: April 14, 2021, 01:05:51 pm »

Made a silly script that takes vermin remains in your fort and teleports them as a projectile to hit the cursor, destroying any tree there (like a ballista bolt):
Code: (destroy-tree.lua) [Select]
--Yeet vermin remains to destroy trees
--keybinding add Ctrl-X destroy-tree

function destroy_tree()
if not dfhack.isMapLoaded() then
qerror("Error: Map not loaded!")
end

local x,y,z = pos2xyz(df.global.cursor)
if x == nil or x < 0 then
qerror("Error: Cursor not active!")
end

for i, item in ipairs(df.global.world.items.other.REMAINS) do
if item.flags.on_ground and not item.flags.in_job then
local proj = dfhack.items.makeProjectile(item)
proj.flags.no_impact_destroy = true
proj.flags.piercing = true
proj.origin_pos.x, proj.origin_pos.y, proj.origin_pos.z = x,y-1,z
proj.target_pos.x, proj.target_pos.y, proj.target_pos.z = x,y,z
proj.cur_pos.x, proj.cur_pos.y, proj.cur_pos.z = x,y-1,z
proj.prev_pos.x, proj.prev_pos.y, proj.prev_pos.z = x,y-1,z
proj.fall_threshold = 1
item.flags.forbid = true
return
end
end
print("No available vermin remains found!")
end

destroy_tree()

Maybe I'll have it create its own projectiles out of some material that evaporates shortly after. Also have it automatically deploy them against every tree chopping designation instead of using the cursor.

Of interest to note is that if "no_impact_destroy" is false, the remains can break and litter the ground with purple "broken arrow" tiles.

I still haven't made any progress on figuring out how to reverse engineer the tree chopping code.
« Last Edit: April 14, 2021, 01:10:25 pm by Bumber »
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2896 on: April 15, 2021, 04:48:45 pm »

Could "unk_v40_1" on projectiles be related to abilities/powers? It's set to 4885 for magma crab globs, 7035 for intelligent undead icicles, and -1 for bolts fired from a crossbow. Falling objects seem to have it set to really high garbage values. Could be the "parabolic" or "unk9" flag that has the game ignore that.

Edit: I don't see any relevant arrays going that high, though. Any chance they're flags?
« Last Edit: April 15, 2021, 05:00:08 pm by Bumber »
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

lethosor

  • Bay Watcher
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2897 on: April 15, 2021, 06:51:32 pm »

It would be an unusually large number of flags (and pattern of set flags) compared to other bitfields, so my guess would be no. -1 would also have every bit set, which is usually an indication that the field is a number, not a bitfield.
Code: [Select]
>>> bin(7035)
'0b1101101111011'
>>> bin(4885)
'0b1001100010101'

From https://github.com/DFHack/df-structures/blob/f7118215ec98ef754579864d92353c59588ba24a/df.projectile.xml#L57, apparently this field is uninitialized, so you might just be seeing garbage data sometimes (or always).
Logged
DFHack - Dwarf Manipulator (Lua) - DF Wiki talk

There was a typo in the siegers' campfire code. When the fires went out, so did the game.

vettlingr

  • Bay Watcher
  • Þjóðalfur
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2898 on: April 21, 2021, 06:53:27 am »

I've been looking more into 3dveins. A long standing Issue has been that the plugin has been unable to run if the map contains lake or sea tiles. Giving the result "Discontinuous layer 1 at *water body coordinates*". If this also applies to glaciers, I do not know.

Code: [Select]
           else
            {
                if (z != min_level[idx]-1 && min_level[idx] <= top_solid)
                {
                    out.printerr(
                        "Discontinuous layer %d at (%d,%d,%d).\n",
                        layer->index, x+column.x*16, y+column.y*16, z
                    );
                    return false;
                }

                min_level[idx] = z;
            }
        }
    }

A fix could be to add lake and ocean sediments to this list:

Code: [Select]
static bool isTransientMaterial(df::tiletype tile)
{
    using namespace df::enums::tiletype_material;

    switch (tileMaterial(tile))
    {
        case AIR:
        case LAVA_STONE:
        case PLANT:
        case ROOT:
        case TREE:
        case MUSHROOM:
            return true;

        default:
            return false;
    }
}

To do that however, I need to figure out the correct tag or token.

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2899 on: April 21, 2021, 11:39:33 am »

your second snippet should probably exclude frozen liquid (water) as well to deal with glaciers.

Could your problem be caused by the weird thing Prospector detects for some lake and ocean tiles where the soil layer is absent, and so the geo biome gets shifted as if the soil layers had been eroded away (and how does the code deal with "normal" erosion of soil at high elevations)?
Logged

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2900 on: April 22, 2021, 04:12:38 am »

Could somebody write a step-by-step guide on how to reverse engineer some DF code using IDA or Ghidra?

I've been trying with IDA and haven't really even been able to get started. Some questions:
1. Should I start DF with IDA, or attach to an existing process? With or without DFHack?
2. When should I run a script (e.g., ms_rtti64.idc,) and how will I know if it worked?
3. How do I constrain myself to the DF segment instead of being lost in dll's?
(I tried setting it to hardware break upon entering the DF module, but this results in constant exceptions, which eventually crashes the game. Just trying to step through code to reach it also results in this eventually.)

Should I open an issue asking for info to be added to the docs (Memory-research.html)?
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

Clément

  • Bay Watcher
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2901 on: April 22, 2021, 04:36:29 am »

1. Start with the debugger or attach later, whatever works for you. With DFHack and its debug info.
2. I don't know about the rtti scripts, but you should run codegen_c_hdr.pl from df_misc using the codegen from the matching DFHack version, import the resulting header in IDA.
3. Depends on what you are looking for, I've used a lot of watchpoints as starting points for reversing since the data is the most well documented part. Alternatively, you could start with a breakpoint on a known function (exported symbol or virtual method).
Logged

Rumrusher

  • Bay Watcher
  • current project : searching...
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2902 on: April 22, 2021, 10:52:33 am »

ok so ended up writing a script for checking on armies that roam the world map ... through nicknaming a unit and tracking the army through that.
this possibly means in adventure mode one could say edit the army controller and mess with the army orders.
and probably set up visitors that could show up to the fort or create sieges from guards or roaming locals.
Code: (getArmyAtName) [Select]
function getArmyAtName()
for k,v in pairs(df.global.world.nemesis.all) do
if v.figure.name.nickname=='DFCAT' then
print(k)
local Cat=df.global.world.armies.all
for k1,v1 in pairs(Cat) do
if v1.members==nil then break else
for Del,ete in pairs(v1.members) do
if ete.nemesis_id==v.id then
print(k1)
end
end
end
end
end
end
end
getArmyAtName()

hopefully I can use this as a stepping stone for figuring out how to do fort mode raids in adventure mode...

edit: ok so made two scripts that use this one
a recruit script that grabs anyone that has a nemesis to join the army in waiting
Code: (NemArm-Recruit) [Select]
function getxyz() -- this will return pointers x,y and z coordinates.
local x=df.global.cursor.x
local y=df.global.cursor.y
local z=df.global.cursor.z
return x,y,z -- return the coords
end
function getItemAtKPos(x,y,z) -- gets the item index @ x,y,z coord
--local x,y,z=getxyz() --get 'X' coords
local vector=df.global.world.item.all -- load all items
local kickpos=df.global.world.units.active[0].pos
for i = 0, #vector-1 do -- look into all items offsets
local curpos=vector[i].pos --get its coordinates
local cx=curpos.x
local cy=curpos.y
local cz=curpos.z
if cx==kickpos.x and cy==kickpos.y and cz==kickpos.z then --compare them
return vector[i] --return index
end
end
--print("item not found!")
return nil

end
function getCreatureAtPos(x,y,z) -- gets the creature index @ x,y,z coord
--local x,y,z=getxyz() --get 'X' coords
local vector=df.global.world.units.all -- load all creatures
for i = 0, #vector-1 do -- look into all creatures offsets
local curpos=vector[i].pos --get its coordinates
local cx=curpos.x
local cy=curpos.y
local cz=curpos.z
if cx==x and cy==y and cz==z then --compare them
return vector[i] --return index
end
end
--print("Creature not found!")
return nil

end

function getArmyAtName()
for k,v in pairs(df.global.world.nemesis.all) do
if v.figure.name.nickname=='DFCAT' then
print("Nem",k)
local Cat=df.global.world.armies.all
for k1,v1 in pairs(Cat) do
if v1.members==nil then break else
for Del,ete in pairs(v1.members) do
if ete.nemesis_id==v.id then
print("Army",k1)
return k1
end
end
end
end
end
end
end
 local horse=getCreatureAtPos(getxyz())
 local nem=df.global.world.history.figures[horse.hist_figure_id].nemesis_id --dfhack.units.getNemesis(horse)
 local NeArmy=getArmyAtName()
  df.global.world.armies.all[NeArmy].members:insert("#",{new=true,nemesis_id=nem,})

and here's the script for moving the station army around which works wonders for nicknaming a hostile bandit member and luring the bandits into a goblin patrol
Code: (NemArm-Station) [Select]
function getArmyAtName()
for k,v in pairs(df.global.world.nemesis.all) do
if v.figure.name.nickname=='DFCAT' then
print("Nem",k)
local Cat=df.global.world.armies.all
for k1,v1 in pairs(Cat) do
if v1.members==nil then break else
for Del,ete in pairs(v1.members) do
if ete.nemesis_id==v.id then
print("Army",k1)
return k1
end
end
end
end
end
end
end


local Ned=getArmyAtName()
for k,v in pairs(df.global.world.armies.all) do
if v.flags[0]== true  then
print("ArmyOrders Station"," Moving Stationed group To Adv Position")
print("id",k,"X position",(v.pos.x),"y position",(v.pos.y))
df.global.world.armies.all[Ned].controller.pos_x=v.pos.x
df.global.world.armies.all[Ned].controller.pos_y=v.pos.y

end
end
well I could somewhat cross off controlling armies on the list of stuff I wanted to do in DF through DFhack.
« Last Edit: April 25, 2021, 01:21:26 pm by Rumrusher »
Logged
I thought I would I had never hear my daughter's escapades from some boy...
DAMN YOU RUMRUSHER!!!!!!!!
"body swapping and YOU!"
Adventure in baby making!Adv Homes

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2903 on: April 26, 2021, 10:39:04 am »

1. Start with the debugger or attach later, whatever works for you. With DFHack and its debug info.
2. I don't know about the rtti scripts, but you should run codegen_c_hdr.pl from df_misc using the codegen from the matching DFHack version, import the resulting header in IDA.
3. Depends on what you are looking for, I've used a lot of watchpoints as starting points for reversing since the data is the most well documented part. Alternatively, you could start with a breakpoint on a known function (exported symbol or virtual method).

Finally got DFHack with debug info built after finding the right packages to install for VS2017 and deleting VS2015 (which I couldn't find packages for and was messing up VS2017.)

I ran the script on library\include\df\codegen.out.xml to produce codegen.h. Then I attached to the DF process in IDA using "Local Windows debugger", set the compiler to "Visual C++" with the default settings, and did "Load file -> Parse C header file..." and selected codegen.h. Output says "codegen.h: successfully compiled". Is that all I'm supposed to do?

Not sure how to find the data or functions to put the breakpoints on. I found the address of a tiletype vector using the lua interpreter, and set a breakpoint and watch there in IDA. Each tiletype seems to be 2 bytes, and I found the lines:
Code: [Select]
db 5Dh ; ]
db 0h
Representing a tree trunk tiletype (93) in the tile of a tree one tick away from being chopped down. I allow the program to resume (so it responds,) then pass one tick (using .) and quickly suspend execution before the tree is cut. I enable the breakpoint and resume execution. Unfortunately, it stops some time after the breakpoint, which doesn't get me the instructions that accessed the data. I tried stepping further through the code, but it doesn't reach the check again in a human time frame. After several continues, the data eventually changes to:
Code: [Select]
db 4Eh ; N
db 1h
And the tree is chopped. The tiletype should be 335 for stone floor. I assume 4E 01 represents that in the proper order and endianness?

This at least gets me into DF's instructions, but not where I want. The addresses of the vectors also change each time DF is loaded, and loading the saved memory in IDA doesn't give you a live program, AFAIK.
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

Warmist

  • Bay Watcher
  • Master of unfinished jobs
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2904 on: April 26, 2021, 11:22:52 am »

Huh... Last time i tried the data breakpoints worked okay. Though yeah the randomization is a pain. Last time i did this i went the other way around. From the i think "logic" vmethod of main screen search for anything that acceses the data (in my case it was world->sth->sth) and it was only a few hits. Though i imagine finding what modifies tiles would have a lot of hits...

Clément

  • Bay Watcher
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2905 on: April 26, 2021, 02:53:39 pm »

I ran the script on library\include\df\codegen.out.xml to produce codegen.h. Then I attached to the DF process in IDA using "Local Windows debugger", set the compiler to "Visual C++" with the default settings, and did "Load file -> Parse C header file..." and selected codegen.h. Output says "codegen.h: successfully compiled". Is that all I'm supposed to do?

From memory (I cannot check right now), you should now be able to add df structures in your structure window or whatever it is called. You can then use the types to annotate the disassembly code, or memory regions.

Unfortunately, it stops some time after the breakpoint, which doesn't get me the instructions that accessed the data. I tried stepping further through the code, but it doesn't reach the check again in a human time frame.
Watchpoints are triggered when the memory is accessed, the instruction pointer has already been changed and it points to the next instruction. The instruction changing the value should be the one just before where you stopped.

After several continues, the data eventually changes to:
Code: [Select]
db 4Eh ; N
db 1h
And the tree is chopped. The tiletype should be 335 for stone floor. I assume 4E 01 represents that in the proper order and endianness?
x86 is little-endian, so the first byte is the LSB. But 0x14e is 334, not 335. You should be able to tell IDA to view this part of memory as a 16 bits word (or better: use the type imported from codegen.h), then endianness will no longer be an issue.
Logged

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2906 on: April 28, 2021, 07:48:59 pm »

[snip]

Edit: Is IDA treating DF as a 32-bit program? All the data types are half the size they should be, and "db" is the same size as "dw".

How do I deal with something like:
Code: [Select]
Dwarf_Fortress.exe:00007FF7B6D20840 db  40h ; @
Dwarf_Fortress.exe:00007FF7B6D20841 db    0
Dwarf_Fortress.exe:00007FF7B6D20842 db  15h
Dwarf_Fortress.exe:00007FF7B6D20843 db 0DCh ; Ü
Dwarf_Fortress.exe:00007FF7B6D20844 db  45h ; E
Dwarf_Fortress.exe:00007FF7B6D20845 db    0
Dwarf_Fortress.exe:00007FF7B6D20846 db    0
Dwarf_Fortress.exe:00007FF7B6D20847 db    0
Which is a 64-bit (8 byte) pointer to the memory address "00000045DC150040"? None of the options I've looked at made it more readable.

Haven't had any luck making sense of vectors, either, such as "world.T_plants". I get this:
Code: [Select]
Dwarf_Fortress.exe:00007FF7B6D20840 world::T_plants <<0BAC86020h, 8Bh, 0BAC9D5C8h>, <8Bh, 0BAC9DB70h, 8Bh>,\
Dwarf_Fortress.exe:00007FF7B6D20840                  <0C52BD900h, 8Bh, 0C52C8EA0h>, <8Bh, 0C52CD5E0h, 8Bh>,\
Dwarf_Fortress.exe:00007FF7B6D20840                  <0C25E17B0h, 8Bh, 0C25E2018h>, <8Bh, 0C25E2190h, 8Bh>>
None of this seems to lead to the address in memory (0000008BB9A80F70) that contains the plant at index 0, nor an area in memory I found through searching (0000008BA0AFD038) that has a pointer to that address. (I didn't find any pointers to that pointer through searching.)


I also can't figure out an easy way to jump to addresses in the hex-editor or stack views, which is a bit annoying. I can only sync them with RIP, etc., or bring the disassembly view to them.
« Last Edit: April 28, 2021, 08:34:12 pm by Bumber »
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

Clément

  • Bay Watcher
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2907 on: April 29, 2021, 03:49:56 am »

Edit: Is IDA treating DF as a 32-bit program? All the data types are half the size they should be, and "db" is the same size as "dw".

I've just realized there was an update to IDA Freeware (since one and a half year ago, I'm slow). The old version I had was 64 bits only, so there was no possible confusion. But now it can be used for 32 bits too. That's why you were able to use a debugger too!

dw is 16 bits and not enough for a pointer: b = byte (8 bits), w = word (16 bits), d = double word (32 bits), q = quad word (64 bits).

Logged

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2908 on: April 30, 2021, 05:05:49 am »

dw is 16 bits and not enough for a pointer: b = byte (8 bits), w = word (16 bits), d = double word (32 bits), q = quad word (64 bits).

I think I found the issue. There's a bug in the right-click menu, where choosing "quad word" results in a 32-bit floating point.

The buttons in from "Options -> Setup data types..." work as intended. I enabled "float" for the right-click menu and that results in a quad word.
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

jecowa

  • Bay Watcher
    • View Profile
Re: DFHack 0.47.05-r1
« Reply #2909 on: April 30, 2021, 10:33:35 pm »

Issue report: "gui/unit-info-viewer" gives Strength:Agility values instead of the size in cubic deciliters.
Logged
Pages: 1 ... 192 193 [194] 195 196 ... 243