Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 10 11 [12] 13 14 ... 43

Author Topic: Anouncing The PathFinder Project  (Read 98598 times)

Sunken

  • Bay Watcher
  • Wabewalker
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #165 on: October 21, 2009, 06:37:02 am »


i assume that most movements are not between quasi-random destinations (e.g. a hunter wandering around until he finds game) but few highly frequented locations.
Sure, but "frequented locations" will be whole stockpiles, farm plots and other multi-tile regions, not single tiles. The restriction of one log or stone per stockpile tile, for example, ensures that you will never get a single cache hit when pathing to get logs or stones from a stockpile.
I still conclude that abstract locations will be absolutely crucial for path caching to have any utility. But feel free to prove me wrong!
Logged
Alpha version? More like elf aversion!

Shades

  • Bay Watcher
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #166 on: October 21, 2009, 06:40:58 am »

I still conclude that abstract locations will be absolutely crucial for path caching to have any utility. But feel free to prove me wrong!

The only case I can think of is stuff that is moved from workshop to workshop without going to a stockpile in between. I agree with your generate point though, but personally I don't think generating the abstract locations will be hard so I'm not overly worried about it.
Logged
Its like playing god with sentient legos. - They Got Leader
[Dwarf Fortress] plays like a dizzyingly complex hybrid of Dungeon Keeper and The Sims, if all your little people were manic-depressive alcoholics. - tv tropes
You don't use science to show that you're right, you use science to become right. - xkcd

sir monko

  • Bay Watcher
    • View Profile
    • wsa
Re: Anouncing The PathFinder Project
« Reply #167 on: October 21, 2009, 07:39:13 am »

okay, grouping stockpile/farmplot/workshop-tiles together to form zones makes definitley sense.

i understood "abstract locations" more in the sense of the often heard "automatically divide your fortress into meaningful sections and magically add waypoints where it makes sense", an idea i'm not all too fond of. i always thought performant management of sections in an ever-changing map is a lot more complicated than most people realize. maybe i'm wrong.

actually, in the old caching-thread i had the idea of not doing path-selection from (x1, y1, z1) -> (x2, y2, z2), but (x1±1, y1±1, z1) -> (x2±1, y2±1, z2), because moving to the adjacent field would be trivial in almost all situations. 3x3 would roughly covers a workshop and already be a huge improvement.

eager to test that :)
Logged

zwei

  • Bay Watcher
  • [ECHO][MENDING]
    • View Profile
    • Fate of Heroes
Re: Anouncing The PathFinder Project
« Reply #168 on: October 21, 2009, 08:09:57 am »

3x3 would roughly covers a workshop and already be a huge improvement.

eager to test that :)

Actually, it would be useless, because workshops only ever use central tile: that is where all the workers and haulers path to, eight other tiles are there to just make it bigger and pretty and you only have to path to them should something fairly unusual happen - i.e. some items happen to get there.

sir monko

  • Bay Watcher
    • View Profile
    • wsa
Re: Anouncing The PathFinder Project
« Reply #169 on: October 21, 2009, 08:24:47 am »

Actually, it would be useless, because workshops only ever use central tile: that is where all the workers and haulers path to

great, that strengthens my point of path caching being useful :)

still, 3x3 greatly reduces the numbers of paths needed for piles and farms, though it introduces the a problem, namely that adjacent tiles are not necessary coverable by one step (e.g. if the slope's too steep).

i agree: grouping adjacent pile and farm tiles would be a huge win. someone should implement it :)
Logged

Kardos

  • Bay Watcher
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #170 on: October 21, 2009, 08:40:04 am »

Actually, it would be useless, because workshops only ever use central tile: that is where all the workers and haulers path to

great, that strengthens my point of path caching being useful :)

still, 3x3 greatly reduces the numbers of paths needed for piles and farms, though it introduces the a problem, namely that adjacent tiles are not necessary coverable by one step (e.g. if the slope's too steep).

i agree: grouping adjacent pile and farm tiles would be a huge win. someone should implement it :)
If the game knows where the creature and the desired location are before hand, using a higher level caching system might be feasable until the creature is within, lets say, 10 tiles of the destination, upon which the creature begins moving on a 1x1 system. 
During the initial pathfinding calculation, the destination could do a floodfill up to 10 (or however many tiles the gamemaker desires) tiles away from iteself, and then that barrier is held in memory for acess until the creature finishes pathing.
Logged

Starver

  • Bay Watcher
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #171 on: October 21, 2009, 08:58:27 am »

[I wrote this earlier, before being pestered with 'real' work, but had to reboot before I could get it up.  With some minor edits, I thought I'd post it anyway, despite being a bit Ninjad by Sir Monko's post, and maybe what appears to have come since then.]

How much space should a cache use up?  Is it cell-by-cell movement, or abstraction across unarguable open areas?  What search routine might be employed to hunt down what cached paths are affected by changes in the linkages at one particular point in the map?  (i.e. find paths that are now useless, so junk that path and immediately recalculate the routing for any agents that are affected... Although the converse insofar as some way that an route that opens up can be recognised as useful to an agent currently going around the previous and longer route would be an almost miraculous addition, without complete repathing every time something changes)

Path caches with dynamic awareness could of course feature multiple branchings in their definitions for whether certain features are opened, and a flag to alert an agent on the path ("if (not yet past <feature>) && (past <lastbranchpoint>) then retrace to <branchpoint> and try <otherbranch>", with various alternate handlers for generating brand-new paths or re-using/reversing various parts and swapping in other alternatives (with or without testing for newly opened 'alternate alternatives').

And how about optimising the cached paths (by memory, if not by overhead calculations) by comparing all newly generated paths with pre-existing ones and checking for shared elements, and as soon as shared partial paths can be identified, and stored as 'sub-solution' paths that can thus potentially shorten future path-searches drastically, or at least identify that the shortest-paths across one particular possible tree of travel are longer than those along the other (or can be considered only solutions for when the bridge is raised/door locked/invading army imposing their presence on the alternate route).


Some of my initial thoughts this morning (I'm not awake yet[1]) were that something LZW-like would be simple to implement, 'streaming' each newly established route through to generate a look-up table and then later routes discovering shared lookups not by going "we already know this much, lets see how much more we know", at the encoding end as we stream the points through (after all, there will be no repetition of points within a single route, unless there's a 'mesh' of possible routes in a dynamics-aware multi-route version that isn't internally optimised), but more about taking into account each past route and identifying shared routing.  A particular strength in tunnels (although I think it'd be a mistake to make this an end-game target).

(The main strength of LZW, that the encoder does not need to pass the encoding table to the decoder but that each can dynamically generate it upon sending/receiving the streaming data, is also a moot point given that we aren't wanting to reduce 'transmission overhead'.  So we would probably fall back upon standard 'pre-processed' compression techniques, as inspiration, that searches for commonalities throughout the raw information that can be considered frequent or extrapolatable and building our 'sub-solution' library from that.  Mixing this in with the actually routefinding.  While this, in itself, is not a part of the routefinding algorithm, a routefinder that has knowledge of common paths can avoid some degree of repetitious calculation for novel whole-routes with close matches to a previously understood one, e.g. workshop-to-stockpile carrying, for each sockpile tile and a possibly a grouping of similar workshops nearby, or just for progressive retrieval of stones from a mined-out area.  But of course done so as to avoid the local minima situation.)

[1] Now it's past dinnertime, but I still don't think my mental reserves are up to strength, still.

justify all that effort for this engine.  (It also couldn't handle reverse-matches.)
Logged

Slogo

  • Bay Watcher
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #172 on: October 21, 2009, 09:54:12 am »

A heirarchy is key to a useful caching mechanism like Impaler said.

It will increase the # of hits as even if the dwarf is targeting an uncommon tile they can still use cached routes to move between the regions.

It will help prevent dwarves from moving to an obstacle then backtracking. Since the moving entity does not store the complete route, only the relevant information about what region they're in and what region they're targeting they can get the specific path once they enter a region. That way if a region's is updated and the path changes they'll know immediately on entering the specific region.

It will make keeping cached paths up to date a lot easier. It's trivial to know what regions are affected by a change in a tile (just check what regions the tile is in) so it also lets us update far fewer cached paths than checking each one to see if it is affected by the change.

It will allow us to store less. Rather than storing 10 similar paths we can store the region to region paths once.

sir monko

  • Bay Watcher
    • View Profile
    • wsa
Re: Anouncing The PathFinder Project
« Reply #173 on: October 21, 2009, 10:08:51 am »

my original post regarding path caching can be found here:
http://www.bay12games.com/forum/index.php?topic=24790.0
~~~
@starver:
imo, the most important thing is to KISS (at least in the beginning).

* i wouldn't make it update-aware (too expensive). paths get invalidated when dwarves that follow it run into a (permanent) problem. i bet that works well enough; also, it's a bit like how it works now. also, cheap.
* paths with unstable tiles (e.g. doors or drawbridges) could be marked as such and evaluated when needed. i think even that is overkill.
* cell by cell. if you store directions and RLE-compress them, it has roughly the same effect as open area abstraction (though it makes a lot of other optimizations impossible).
* cache invalidation: kill the cached path when it's used > X times or it's age > Y (so new shortcuts aren't ignored)
* shared partial paths: MADNESS! my brain starts to smell like burnt bacon if i try to think about the consequences. what else? subpath reference counting!? pathcache-virtual machine with trace-JITting? ;) maybe lars bak and peter norvig play dwarf fortress. hey, if one of you reads this, could you please implement that part?


@starver & @slogo:
yes. yes! yes. of course, increasing the complexity of the cache will almost certainly make it more efficient. but it will also prevent the cache from being ever implemented.

moreover, i'm pretty sure the cache would improve things a lot, even if no subpathtracingjitmumboregionpartinioning was applied.

group piles together, group fields together, and the dumbest cache would pay for itself may times, while being easy to implement (and thus, less susceptible to bugs).

while complexity may make DF interesting in the first place, you certainly want it in its codebase.
« Last Edit: October 21, 2009, 10:11:32 am by sir monko »
Logged

Shades

  • Bay Watcher
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #174 on: October 21, 2009, 10:16:35 am »

* i wouldn't make it update-aware (too expensive). paths get invalidated when dwarves that follow it run into a (permanent) problem. i bet that works well enough; also, it's a bit like how it works now. also, cheap.

Fairly sure it, if not being update-aware, does recalculate paths before it runs into a problem. If you, for example, lock a door as critters move towards it they switch to a new route very quickly.
Logged
Its like playing god with sentient legos. - They Got Leader
[Dwarf Fortress] plays like a dizzyingly complex hybrid of Dungeon Keeper and The Sims, if all your little people were manic-depressive alcoholics. - tv tropes
You don't use science to show that you're right, you use science to become right. - xkcd

Sunken

  • Bay Watcher
  • Wabewalker
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #175 on: October 21, 2009, 10:25:01 am »

Yeah, checking a path found in the cache is not very expensive. We could play with the frequency of re-checks and see what it does to performance.
Logged
Alpha version? More like elf aversion!

shadow_slicer

  • Bay Watcher
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #176 on: October 21, 2009, 10:33:42 am »

I think perhaps we're making things a little too complicated. For now maybe we can just focus on optimizing dwarven pathing.

I like the idea of just using a hierarchical A* algorithm. Basically we would do what someone (can't find the post :-[) suggested and use a flooding algorithm to define "rooms" by limiting them based on the number of neighbors/size of the room (to block off choke points). This will correctly group most rooms (whether convex or not) and will group the exterior as one giant room.

The "rooms" will keep a list of their border tiles and which other rooms are connected to those tiles. Additionally they will keep a cache (most recently used) of the cost to travel from one border tile to another border tile. These costs will be calculated using A* within the room.

In order for a dwarf to find a route on the map, the pathfinder will use A* over a modified map consisting of the tiles in the room the dwarf is in, the tiles in the room the dwarf's destination is in, and all the other rooms on the map. Travel between rooms can use the cached values, so it will be much faster than simple A* over the entire map.

Additionally any changes will only affect the room they are in (and possibly an adjacent room). If the modifications are trivial, the room's cache can simply be cleared. For more complicated changes the room can be either be split or merged and the neighboring rooms notified. And that's all that would need to be done!

If we want to support pets, we will need to modify things slightly, since certain doors may be impassible. We could make it so that doors/hatches will automatically end a room, and then just keep a flag on that boundary marking it pet impassible. Then when we perform pathing we can ignore those adjacencies for pets.

This in itself will solve the vast majority of the pathing problems currently plaguing DF. Unfortunately, this will not solve the multi-tile, swimming, flying, digging monster issues we have discussed so far, but we can deal with that in the future. The best way I can see to solve that issue though, is to build a separate pathing map for them (or just use non-hierarchical A* over all of the tiles).
Logged

Silverionmox

  • Bay Watcher
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #177 on: October 21, 2009, 12:36:55 pm »

Pets just need to path towards their owner. There should be a parameter how close they need to be (i.e. within how many squares). A pet following the dwarf should only get through the door if the dwarf wants it to stay close, and that means stay directly adjacent.

That code can be reused for things like ducklings behind their mother, a chain gang, caravans, military formations etc. The distances that need to be pathed are very short and shouldn't pose a significant burden, while still allowing the complete flexibility to move out of the way, should there be a problem with that route.


Cells probably can have multiple connections (eg. two rooms with a wall with two doors between them, or a row of pillars). How is the choice between them made if they are at equal distance? And will the other one ever be used? If something (eg. a catsplosion) needs to be evaded in one cell, will that effect the choice of connection?
Logged
Dwarf Fortress cured my savescumming.

shadow_slicer

  • Bay Watcher
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #178 on: October 21, 2009, 12:56:32 pm »

Yes, rooms should have multiple connections. The choice between them would be made with A*. If they are along paths with the same distance to the destination, then the choice of which door is implementation dependent.

This could be made random if in the implementation of A*, potential next-nodes are added to the priority queue such that their relative position among other nodes with the same score is random. (i.e. Find position of queue of first node with that score, and last node with that score and choose a random number between 0 and 1 + [the difference between the two positions], insert the potential next-node at position [position of first node]+[random number]).

This could be made to select for larger hallways if a secondary score consisting of the average width or minimum width of the path was used as a tie-breaker. Also the presence/absence of cats/goblins/miasma could also be used similarly. Or an heuristic taking into account all of the above could be used (anything you can describe mathematically is possible).
Logged

Fieari

  • Bay Watcher
    • View Profile
Re: Anouncing The PathFinder Project
« Reply #179 on: October 21, 2009, 01:47:53 pm »

I'm trying to compile the pathing code posted on the previous page, but despite following all the instructions on installing boost, it won't compile, apparently because it can't find the boost includes for some reason.  Anyone else using Bloodshed Dev C++ might be able to help?

Edit: Wait, no... I can compile the test program Boost's tutorials provide, and that finds the proper includes just fine.  Weird.  And yet when trying to compile this program, I get:
Code: [Select]
9 C:\programming\pathing\graph.h boost/iterator/iterator_facade.hpp: No such file or directory.
Not to mention a bunch of other errors that may be related.  Now, I'm not using any of the .hg stuff... is there something I'm missing there?
« Last Edit: October 21, 2009, 01:55:02 pm by Fieari »
Logged
Pages: 1 ... 10 11 [12] 13 14 ... 43