Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 21 22 [23] 24

Author Topic: DFHack plugin embark-assistant  (Read 93778 times)

RedDwarfStepper

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #330 on: June 03, 2021, 05:44:14 pm »

So - some family stuff came up and I really need to get some sleep now...
I hope I'll get a little free time during the day tomorrow to run the compiler with the reverted sources, otherwise I'll burn some midnight oil.
I'll create a zip that contains all the versions back to 0.47.04-r4 in the first go to make the testing easier.
And as a bonus I'll throw in a variant of the current version as I have an inkling of what might be the root of this, but as it might be completely bogus I won't spoil the surprise.
Only that much - @Deuslinks: How old is your CPU and would you say it is fast - say compared to one that is around 9 years old, like an Intel Core i5 3570 for example?
Logged

Deuslinks

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #331 on: June 03, 2021, 06:11:46 pm »

The cpu is very new had the whole pc built last month it is a AMD Ryzen 5 5600X Six Core CPU (3.7GHz-4.6GHz/35MB CACHE/AM4).


Also no.worries things going on in life take precedence.
Logged

RedDwarfStepper

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #332 on: June 04, 2021, 04:16:03 pm »

The cpu is very new had the whole pc built last month it is a AMD Ryzen 5 5600X Six Core CPU (3.7GHz-4.6GHz/35MB CACHE/AM4).
hehe, thought so - oh I really hope I'm right.

Also no.worries things going on in life take precedence.
Thanks for that. Also believe me I was beyond worrying at that moment more asleep than awake :D but giving a heads up so no one waits needlessly was the easy and right thing to do.

Ok, so here we go
https://dffd.bay12games.com/file.php?id=15562
The zip contains 5 adapted versions of the plugin (all build against dfhack 0.47.05-r1), starting with the current version without changes (0._0.47.05_r1_embark-assistant.plug.dll) just to make absolutely super sure it still causes a crash.
If it does not, well, I checked with the release on github (https://github.com/DFHack/dfhack/releases/download/0.47.05-r1/dfhack-0.47.05-r1-Windows-64bit.zip) and the two files are pretty different.... So I'm wondering - but lets not go there for now.
It also contains the unchanged original dll files from the respective release in the folder "_originals", they are there just for reference sake and can be ignored for now.

@Deuslinks: You can just copy the dlls into your plugins directory - you'll see a warning for each one but they won't interfere with the properly named "embark-assistant.plug.dll".
As per PatrikLundell's very good suggestion keep your original "embark-assistant.plug.dll" around by renaming it to "embark-assistant.plug.dll.orig".
Now starting with "0._0.47.05_r1_embark-assistant.plug.dll" create a copy of the file and rename the copy to "embark-assistant.plug.dll" - it might be easier to delete the current "embark-assistant.plug.dll" beforehand depending on your workflow/file handling tools.
Then do your thing :)
Rinse and repeat with the next version in increasing order if the currently tested crashes.
For the moment we're only interested in the version which reliably does not cause a crash. If even the last version ("4._0.47.04-r4_embark-assistant.plug.dll") causes a crash - well - let me know :)
Also let me know if there are any other questions.
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #333 on: June 04, 2021, 05:37:31 pm »

It can be useful to know that Windoze has some stupidity going on with locked files when DF crashes, i.e. it's possible you can't delete the DLLs after DF has crashed because Windoze somehow thinks the files are still in use. When that happens I rename them by adding the suffix ".junk" to them to remind me that I can remove the garbage after the computer has been restarted. After renaming the file(s) you can then then reuse the original name without problems.
Logged

Deuslinks

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #334 on: June 12, 2021, 11:28:21 am »

Hi there apologies for being missing the last week Work/Personal life + accidentally smashing my monitor really threw a spanner in all my plans. I have replaced the embark assistant dll file using each one and built new worlds and it still crashes

Ive uploaded dump files here it is odd as it will occasionally work Maybe once every 4 or so times but I cannot nail down a reason for it working even following the same steps on previous ones where it has worked
The pic ive uploaded is from the DFhack box and shows one working and the one after failing it just hangs on the [DFHACK#] where as the one that worked came up with the unit transparency message

Files for 0-2
https://dffd.bay12games.com/file.php?id=15571

Files for 3-4
https://dffd.bay12games.com/file.php?id=15569

Pic of DFhack box
https://dffd.bay12games.com/file.php?id=15570
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #335 on: June 13, 2021, 03:53:52 am »

The "Unit transparency enabled" message seems to be generated by TwbT (doesn't appear when I use "STANDARD" as the print mode, but does show up with "TWBT". Unfortunately, it doesn't crash for me with TWBT either, so even if it would be the cause, it's not consistent.
However, I'd try to use a different print mode than TwbT to see if that works better, as TwbT is know to have bugs (such as frequently/usually crashing on embark).

I tried to look at the crash dumps and it looks like all of them crash along the same call chain (which I guess is good in that it seems to be the same one all the time), but I still don't get anything more out of it. I hope RedDwarfStepper might be able to use the symbol files produced when the DLLs were produced to make the dumps somewhat more intelligible, though.
Logged

Schmaven

  • Bay Watcher
  • Abiding
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #336 on: June 13, 2021, 06:05:36 am »

Not a major issue, but in my experience, searching for waterfalls only returns results if a previous search with some other parameter has been run.  Looking for a minimum drop of 5 gave 0 matching world tiles.  Running the search again for a minimum river of brook yielded nearly the entire map.  Then running it again with just a waterfall drop of 5 produced 459 results.  Running 2 searches is still waaaaaaaay faster than just looking at random river junctions with the relative elevation view, so thank you for the work involved in making this possible :)

Edit: I just tried another search on the same world, this time with other parameters together with the waterfall min=5 and it came back with 0.  So I ran it again, same criteria, butfor waterfall min=NA and it found many results.  Ran again, the same butfor waterfall min=5 and it came up with a bunch.  So it seems to consistently require the first search to not include waterfalls.  IIRC, it was mentioned somewhere in the thread about how the search looks at rivers and elevations to determine waterfall heights, so I think it just might require 2 searches to do that.  I suppose I was just thinking that if there was some simple way to run the preliminary search "behind the scenes" it would seem like just the first search found them.  There were a number of embarks I've done in the past where I never searched for waterfalls the 2nd time, just assuming my world had none, but that's on me for not being persistent. 
« Last Edit: June 13, 2021, 07:07:11 am by Schmaven »
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #337 on: June 13, 2021, 07:45:02 am »

@Schmaven: Your description indicates there's a bug somewhere (I haven't yet looked into it) that dismisses waterfalls prematurely for the first search, for some reason. The second and subsequent searches should all give the same result (with the same parameters, of course), or there's a bug in that logic.

The reason the first search isn't run "behind the scene" is that it takes such a long time in a full sized world.
Logged

Schmaven

  • Bay Watcher
  • Abiding
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #338 on: June 13, 2021, 08:26:36 am »

After the first search, all subsequent searches are accurate and consistent from what I've seen, so to clarify, the false negative only manifests on the very first search.
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #339 on: June 13, 2021, 10:52:50 am »

@Schmaven: I've found the bug (the code attempted to compare the maximum waterfall size in the world tiles before this info had been collected, and thus rejected all tiles.

The fixed code (when accepted and then included in a new DFHack release) will still suffer from the general problem with the first search, i.e. it won't pick up matches at the edges of the embark area (because that requires mutual access to the info in neighboring world tiles, which means they can't be searched reliably until the second attempt).
Logged

RedDwarfStepper

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #340 on: June 15, 2021, 03:30:19 pm »

I tried to look at the crash dumps and it looks like all of them crash along the same call chain (which I guess is good in that it seems to be the same one all the time), but I still don't get anything more out of it. I hope RedDwarfStepper might be able to use the symbol files produced when the DLLs were produced to make the dumps somewhat more intelligible, though.

I really thought and kind of hope-feared that one of my last changes broke it. Some missing or half-faulty initialization. So I created release dlls for the tests of Deuslinks ...
But as PatrikLundell said that the call chain is unchanged I'll have a look at the crash log of the debug dll we tested at the beginning of June - for that I still have the original symbol files - just made a backup of those to be sure.
I'll try to find time for it at the weekend - my high functioning free time is currently quite limited - and I don't want to mess this up with a sleepy brain.
« Last Edit: June 16, 2021, 01:17:04 am by RedDwarfStepper »
Logged

RedDwarfStepper

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #341 on: June 19, 2021, 10:52:39 am »

The main thread is a follows:
Code: [Select]
ntdll.dll!NtWaitForSingleObject() Unknown
  KERNELBASE.dll!WaitForSingleObjectEx() Unknown
  SDLreal.dll!00007ffdcbcee804() Unknown
> SDL.dll!SDL_SemWait(void * sem) Line 707 C++
  Dwarf Fortress.exe!00007ff775ad9a6d() Unknown
  Dwarf Fortress.exe!00007ff775ada0a7() Unknown
  Dwarf Fortress.exe!00007ff775ada580() Unknown
  Dwarf Fortress.exe!00007ff775ada87d() Unknown
  Dwarf Fortress.exe!00007ff775adb062() Unknown
  Dwarf Fortress.exe!00007ff77645cf8e() Unknown
  Dwarf Fortress.exe!00007ff77645cd25() Unknown
  Dwarf Fortress.exe!00007ff77645c1fa() Unknown
  kernel32.dll!BaseThreadInitThunk() Unknown
  ntdll.dll!RtlUserThreadStart() Unknown

the worker thread concerning embark-assistant:
Code: [Select]
> embark-assistant.plug.dll!std::distance<std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<enum df::enums::interface_key::interface_key> > > >(std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<enum df::enums::interface_key::interface_key> > > _First, std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<enum df::enums::interface_key::interface_key> > > _Last) Line 1126 C++
  embark-assistant.plug.dll!embark_assist::overlay::ViewscreenOverlay::interpose_fn_feed(std::set<enum df::enums::interface_key::interface_key,std::less<enum df::enums::interface_key::interface_key>,std::allocator<enum df::enums::interface_key::interface_key> > * input) Line 104 C++
  SDL.dll!df::viewscreen::feed_key(df::enums::interface_key::interface_key key) Line 5 C++
  embark-assistant.plug.dll!embark_assist::matcher::find(embark_assist::defs::match_iterators * iterator, std::vector<embark_assist::defs::geo_datum,std::allocator<embark_assist::defs::geo_datum> > * geo_summary, std::vector<std::vector<embark_assist::defs::region_tile_datum,std::allocator<embark_assist::defs::region_tile_datum> >,std::allocator<std::vector<embark_assist::defs::region_tile_datum,std::allocator<embark_assist::defs::region_tile_datum> > > > * survey_results, std::vector<std::vector<embark_assist::defs::matches,std::allocator<embark_assist::defs::matches> >,std::allocator<std::vector<embark_assist::defs::matches,std::allocator<embark_assist::defs::matches> > > > * match_results) Line 2967 C++
  embark-assistant.plug.dll!embark_assist::main::match() Line 90 C++
  embark-assistant.plug.dll!embark_assist::overlay::ViewscreenOverlay::interpose_fn_render() Line 244 C++
  Dwarf Fortress.exe!00007ff775d4fef8() Unknown
  Dwarf Fortress.exe!00007ff775ad9e22() Unknown
  Dwarf Fortress.exe!00007ff775adadd9() Unknown
  SDLreal.dll!00007ffdcbcee471() Unknown
  SDLreal.dll!00007ffdcbcee855() Unknown
  ucrtbase.dll!thread_start<unsigned int (__cdecl*)(void *),1>() Unknown
  kernel32.dll!BaseThreadInitThunk() Unknown
  ntdll.dll!RtlUserThreadStart() Unknown

Really can't say anything concerning the main thread, apart from
Code: [Select]
ntdll.dll!NtWaitForSingleObject()does not sound so bad/crashy.
But the ea worker thread, hm, the last real source code line 104 corresponds to this
Code: [Select]
input->count(df::interface_key::SETUP_LOCAL_Y_UP) ||https://github.com/DFHack/dfhack/blob/2fc5fbacb5bdf7183d61045a4786d58571081eec/plugins/embark-assistant/overlay.cpp#L104
and ends up in xutility:
Code: [Select]
template<class _InIt> inline
_Iter_diff_t<_InIt>
distance(_InIt _First, _InIt _Last)
{ // return distance between iterators
return (_Distance1(_First, _Last, _Iter_cat_t<_InIt>()));
}
via in xtree
Code: [Select]
size_type count(const key_type& _Keyval) const
{ // count all elements that match _Keyval
_Paircc _Ans = equal_range(_Keyval);
return (_STD distance(_Ans.first, _Ans.second));
}

Could it be that input gets messed up/invalid? Or that DEFINE_VMETHOD_INTERPOSE misbehaves? Also there is a reference to
Code: [Select]
SDL.dll!DFHack::Core::getHotkeyCmdin another worker thread (see below) those two might interact in a bad way - just throwing out wild theories ...


There are 3 more worker threads that contains references to dfhack
1.:
Code: [Select]
ntdll.dll!NtWaitForSingleObject() Unknown
  mswsock.dll!SockWaitForSingleObject() Unknown
  mswsock.dll!WSPAccept() Unknown
  ws2_32.dll!WSAAccept() Unknown
  ws2_32.dll!accept() Unknown
> SDL.dll!CPassiveSocket::Accept() Line 244 C++
  SDL.dll!`anonymous namespace'::ServerMainImpl::threadFn(std::promise<bool> promise, int port) Line 475 C++
  SDL.dll!std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(std::promise<bool>,int),std::promise<bool>,int>,std::default_delete<std::tuple<void (__cdecl*)(std::promise<bool>,int),std::promise<bool>,int> > > >::_Execute<0,1,2>(std::tuple<void (__cdecl*)(std::promise<bool>,int),std::promise<bool>,int> & _Tup, std::integer_sequence<unsigned __int64,0,1,2> __formal) Line 241 C++
  SDL.dll!std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(std::promise<bool>,int),std::promise<bool>,int>,std::default_delete<std::tuple<void (__cdecl*)(std::promise<bool>,int),std::promise<bool>,int> > > >::_Run(std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(std::promise<bool>,int),std::promise<bool>,int>,std::default_delete<std::tuple<void (__cdecl*)(std::promise<bool>,int),std::promise<bool>,int> > > > * _Ln) Line 247 C++
  SDL.dll!std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(std::promise<bool>,int),std::promise<bool>,int>,std::default_delete<std::tuple<void (__cdecl*)(std::promise<bool>,int),std::promise<bool>,int> > > >::_Go() Line 233 C++
  SDL.dll!std::_Pad::_Call_func(void * _Data) Line 210 C++
  ucrtbase.dll!thread_start<unsigned int (__cdecl*)(void *),1>() Unknown
  kernel32.dll!BaseThreadInitThunk() Unknown
  ntdll.dll!RtlUserThreadStart() Unknown
2.:
Code: [Select]
ntdll.dll!NtDeviceIoControlFile() Unknown
  KERNELBASE.dll!ConsoleCallServerGeneric() Unknown
  KERNELBASE.dll!GetConsoleInput() Unknown
  KERNELBASE.dll!ReadConsoleInputA() Unknown
> SDL.dll!DFHack::Private::prompt_loop(tthread::recursive_mutex * lock, DFHack::CommandHistory & history) Line 288 C++
  SDL.dll!DFHack::Private::lineedit(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & prompt, std::basic_string<char,std::char_traits<char>,std::allocator<char> > & output, tthread::recursive_mutex * lock, DFHack::CommandHistory & ch) Line 386 C++
  SDL.dll!DFHack::Console::lineedit(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & prompt, std::basic_string<char,std::char_traits<char>,std::allocator<char> > & output, DFHack::CommandHistory & ch) Line 598 C++
  SDL.dll!fIOthread(void * iodata) Line 1494 C++
  SDL.dll!std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(void * __ptr64),void * __ptr64>,std::default_delete<std::tuple<void (__cdecl*)(void * __ptr64),void * __ptr64> > > >::_Run(std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(void *),void *>,std::default_delete<std::tuple<void (__cdecl*)(void *),void *> > > > * _Ln) Line 247 C++
  SDL.dll!std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(void * __ptr64),void * __ptr64>,std::default_delete<std::tuple<void (__cdecl*)(void * __ptr64),void * __ptr64> > > >::_Go() Line 233 C++
  SDL.dll!std::_Pad::_Call_func(void * _Data) Line 210 C++
  ucrtbase.dll!thread_start<unsigned int (__cdecl*)(void *),1>() Unknown
  kernel32.dll!BaseThreadInitThunk() Unknown
  ntdll.dll!RtlUserThreadStart() Unknown
3.:
Code: [Select]
ntdll.dll!NtWaitForAlertByThreadId() Unknown
  ntdll.dll!RtlSleepConditionVariableSRW() Unknown
  KERNELBASE.dll!SleepConditionVariableSRW() Unknown
  [Inline Frame] msvcp140.dll!Concurrency::details::stl_condition_variable_win7::wait_for(Concurrency::details::stl_critical_section_interface *) Line 216 C++
  msvcp140.dll!Concurrency::details::stl_condition_variable_win7::wait(Concurrency::details::stl_critical_section_interface * lock) Line 210 C++
> msvcp140.dll!do_wait(_Cnd_internal_imp_t * cond, _Mtx_internal_imp_t * mtx, const xtime * target) Line 77 C++
  SDL.dll!std::condition_variable::wait(std::unique_lock<std::mutex> & _Lck) Line 565 C++
  SDL.dll!DFHack::Core::getHotkeyCmd(bool & keep_going) Line 1913 C++
  SDL.dll!fHKthread(void * iodata) Line 234 C++
  SDL.dll!std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(void * __ptr64),void * __ptr64>,std::default_delete<std::tuple<void (__cdecl*)(void * __ptr64),void * __ptr64> > > >::_Run(std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(void *),void *>,std::default_delete<std::tuple<void (__cdecl*)(void *),void *> > > > * _Ln) Line 247 C++
  SDL.dll!std::_LaunchPad<std::unique_ptr<std::tuple<void (__cdecl*)(void * __ptr64),void * __ptr64>,std::default_delete<std::tuple<void (__cdecl*)(void * __ptr64),void * __ptr64> > > >::_Go() Line 233 C++
  SDL.dll!std::_Pad::_Call_func(void * _Data) Line 210 C++
  ucrtbase.dll!thread_start<unsigned int (__cdecl*)(void *),1>() Unknown
  kernel32.dll!BaseThreadInitThunk() Unknown
  ntdll.dll!RtlUserThreadStart() Unknown
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #342 on: June 19, 2021, 12:58:41 pm »

Thanks for that. Unfortunately, I fail to any immediate culprit.

- I agree the main thread seems to just be waiting, which is reasonable and does seem to be harmless.
- The thread involving the Embark Assistant seems to be reacting to the survey as it moves the cursor around to, well, survey things, and this involves the display of the current status of the search.
- The first worker thread just seems to be waiting.
- The second worker thread seems to be the DFHack console waiting for things to happen.
- My guess about the third worker thread is that it is DFHack monitoring DF for keys to intercept (such as modifying advanced critera, e.g. sand display), but that's definitely just a guess.

So the EA thread feeds a key (presumably CURSOR_UP, although my local code apparently doesn't match the one examined), which is then intercepted and processed until line 104, where the input->count() operations tries to find whether the appropriate field in the set is set, but somehow blows up, despite the code having checked SETUP_LOCAL_X_DOWN on the previous line, and that enum value is 3 higher (assuming the set allocates values in numerical order, rather than according to some hash key). I have no idea how sets are implemented, but it doesn't make sense that checking a set should be capable of blowing up if you're actually using the correct set.
It can be noted that the key fed is the "raw" keyboard key, while the key checked against is the evaluated context matched one, but both keys belong to the same enum, and so should be within the same set. Also note that the key fed in is a single key, while the interpose code processes a set that DF has generated by processing the "keyboard" input character.

I can't say this feels like much progress...
Logged

RedDwarfStepper

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #343 on: June 19, 2021, 02:27:21 pm »

Thanks for that. Unfortunately, I fail to any immediate culprit.
You're welcome. And: Yeah, me neither.

I have no idea how sets are implemented, but it doesn't make sense that checking a set should be capable of blowing up if you're actually using the correct set.
Well, if there is concurrent access to the set and at least one thread modifies the set while another thread reads it (=> count) this can lead to an inconsistent state that leads to undefined behaviour which can result in a crash.
The previous successful call to input->count() in the line above does not rule out this cause.
As Deuslinks reports the crash only happens most of the time, but not always - so race conditions and similar beasts and sadly memory access/allocation issues still are very much a possible reason.
Also that this - at least for Deuslinks - happens with a new and very fast CPU could point in that direction.

Just throwing some ideas out there:
- Are there any other plugins that are similar in their structure/design so that they could suffer from the same problem?
- Is there any language construct that we could use to isolate this specific location in the code just to make sure it's not the std::set? try-catch comes to mind - but I never have used it in C++ so I'm not sure.
- Can we make the system or the dlls we control report a proper error, like "access violation"? Or would this happen anyway and what we see is undefined behavior for sure?
- Would more crash dumps with the debug-dlls help? Perhaps we're missing something?
- @PatrikLundell: How old is your PC by the way? Perhaps - between the two of us - one has to buy/build a PC with current hardware - mine is - apart from the graphics card - 9 years old....

PS:
I can't say this feels like much progress...
Well with results of the modified dlls we learned that it is no recent change that causes the problem.
Also we know that it sometimes does work all the way through, which might point to some problem during initialization or the first iterations.

PPS:
One more thing I just found: The current pressed key is CURSOR_UPLEFT_FAST, coming from here
https://github.com/DFHack/dfhack/blob/1c32783dd2628f22c5355b84e49e5c7357b52cb8/plugins/embark-assistant/matcher.cpp#L2966
or to be exact here
https://github.com/DFHack/dfhack/blob/1c32783dd2628f22c5355b84e49e5c7357b52cb8/plugins/embark-assistant/matcher.cpp#L2967
so the end of a loop - hm.
« Last Edit: June 19, 2021, 02:37:23 pm by RedDwarfStepper »
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #344 on: June 19, 2021, 03:31:54 pm »

Concurrent access is always dangerous, of course, but I have trouble seeing this being the case with this set unless something is modifying dangling pointers, which I think ought to result in less consistent crashes. The set ought to be populated by DF and then only consumed downstream, even if that happens to be in multiple threads (and I don't think that's happening).

Improper cache management can lead to issues where different threads won't get access to "the same" data, but I don't know how C(++) handles that, although my single change to "code" written in Java involved changing a variable to make it available to multiple threads (the weird default in this padded cell scripting language is to cut babies to pieces by razor sharp optimization corners in that department).

I've never dealt with exceptions in C(++) either, although I've got plenty of experience with it from a high level language (and thus those programs almost never crashed in the C fashion, but with error logs about unhandled exceptions, allowing us to at least know where to start looking).

It looked like the code blew up in std::distance rather than in our code, so instrumenting that code would presumably require replacing it with an instrumented version. I don't know if try/catch would work, although it might, in which case I'd expect it to be possible to provide some output of whatever data we can think of (while trying to avoid poking it where it blows up again).

I don't think more of dump of the same code would help, but rather outputs from instrumented code that either tries to catch the exception and output info about the state and/or output corresponding data at each visit to the code (it should be a little over once per world tile when it doesn't crash, but probably the first or one of the first times when it does. It seemed to be an UP movement, though, not a MUP, so it's a single step rather than a 10 tile step).

I think I bought my current computer a little less than 1½ years ago, and it's a fairly high end one (AMD Ryzen 9 3950X).

It's true we've learned a bit about what the problem isn't caused by.
I don't see much that can race in this scenario. You've got DF's main thread that's basically waiting for input (and hands it over to other threads), the DF display thread that just run in a cycle where the writing to the memory it uses has to be synchronized with it (doing it from the wrong parts of the script in Lua illustrates what can happen if you do it incorrectly). You've got the EA main thread that essentially takes the role of DF's main thread.
Assuming the program blows up in the EA thread (do we know it's there, rather than the thread being at that location when things blow up elsewhere?) it looks like the data received is somehow corrupt, which is uncomfortable as it's provided by DF rather than our code (I'd rather get egg on my face from screwing up than being able to say it wasn't my fault when the former means I can actually fix it, but the latter doesn't).

Edit:
I hooked up the debugger and looked at the key codes sent to the interpose code, and it turned out to be the keys the "keyboard" sent most of the time. The exception was when changing from one block of world tiles to the next one, in which case the "keyboard" key input was followed by a SETUP_LOCAL_Y_MUP followed by a set consisting of both a SETUP_LOCAL_Y_MUP and a SETUP_LOCAL_Y_MDOWN (in a 17*17 world). It doesn't really matter which of these codes you get, though, as they only serve to trigger updates of the overlay (although this indicates we have a superfluous update at every transition).

Edit 2: I just saw the PPS:
That's indeed during the initiation phase where the focus is moved to the first world tile. It can be noted that it's somewhat "lazy" in that it just keeps moving the cursor until it hits the corner, ignoring whether you're "overshooting" or not, as DF cuts it back to the world boundaries (which it has to do when dealing with players).
The end of the loop is likely just the address the call should return to, which is to be expected based on what I saw from the assembly: the addresses pointed to were the ones following calls, rather than the addresses from which the calls were made.
And I see exactly the same call sequence when I use a debugger for the first CURSOR_UPLEFT_FAST key (and, I assume, the following ones). I get a set of size one with that key as its single element.
« Last Edit: June 19, 2021, 04:35:44 pm by PatrikLundell »
Logged
Pages: 1 ... 21 22 [23] 24