Post by: Toady One on June 20, 2016, 02:13:27 pm
Here's another bug-fix release. Assuming no issues crop up immediately, we'll now dive into 64-bit land for next time!
Major bug fixes
(*) Fixed error deciding when patients should be moved
(*) Fixed initialization problem with tools causing stone axes to be thought of as ranged
(*) Stopped completed work order jobs from checking off every matching order
(*) Stopped masterpiece trades in containers from triggering artwork defacement
(*) Stopped storage from always failing in the second tavern/library/temple you define
(*) Stopped broken crash-prone entry from appearing at the end of the stocks list
Other bug fixes/tweaks
(*) Attackers will remove armor from unconscious opponents if it is blocking attacks
(*) Made people wear more armor according to their roles again
(*) Allowed new citizens with some previously site-wide occupations to be reassigned
(*) Allowed some site-wide occupations for dwarves
(*) Made combat damage weapon and armors depending on material differences etc.
(*) Made dwarves prefer undamaged equipment during the periodic uniform upgrades
(*) Allowed strong attacks/shakes to translate some force to joints and parent parts even if blocked by armor
(*) Reduced clothing stopping power based on penetration depth
(*) Made paper slurries stockpile-able (won't work without updated raws)
(*) Fixed problem with adv mode tribute demand check
(*) Fixed ghost initial positioning problem
(*) Made macros save correctly even if the macro directory is deleted
Post by: Urlance Woolsbane on June 20, 2016, 02:46:40 pm
Post by: peasant cretin on June 20, 2016, 02:55:45 pm
Post by: Random_Dragon on June 20, 2016, 03:00:53 pm
(*) Fixed initialization problem with tools causing stone axes to be thought of as ranged
Vot de FOCK. That is the most hilarious cause of a bug I've ever seen.
Post by: Jazzeraint on June 20, 2016, 03:15:56 pm
Spoiler (click to show/hide)
Post by: thvaz on June 20, 2016, 03:23:26 pm
Post by: Toady One on June 20, 2016, 03:36:45 pm
'mscvcp140.dll missing' causing a failure for the .exe to load for me. Win7, 43.03 has been working fine.Spoiler (click to show/hide)
http://bay12games.com/dwarves/redist/msvcp140.dll
If you put this in the folder does it work, or does it give you another DLL that's missing? I tried it on three computers, but they all came with the proper DLLs I guess.
This is great. What is next? More fixes or work into the 64 bit version?
I'll probably have to fix this DLL problem, but I'd like to get to 64 bits as soon as possible so it's no longer hanging over the project.
Post by: Random_Dragon on June 20, 2016, 03:37:23 pm
'mscvcp140.dll missing' causing a failure for the .exe to load for me. Win7, 43.03 has been working fine.Spoiler (click to show/hide)
Very peculiar. Tried a re-download? Also on Windows 7 (32-bit), and no apparent problem so far.
Post by: Sanctume on June 20, 2016, 03:41:21 pm
'mscvcp140.dll missing' causing a failure for the .exe to load for me. Win7, 43.03 has been working fine.
PC is missing the runtime support DLLs:
Visual C++ Redistributable for Visual Studio 2015 (https://www.microsoft.com/en-us/download/details.aspx?id=48145)
p.s.
Or Toady One maybe has a different compile with static linking.
on visual studio go to Project tab -> properties - > configuration properties -> C/C++ -> Code Generation. on runtime library choose /MTd for debug mode and /MT for release mode.
This will cause the compiler to imbue the runtime into the app. The executable will be bigger, but it will run without any need of runtime dlls.
Post by: Random_Dragon on June 20, 2016, 03:43:44 pm
(http://i.imgur.com/JkRYAb6.png)
EDIT: Image source edited, thanks to Witty putting it up on Imgur. Had planned to put it somewhere more permanent than my Dropbox eventually.
Post by: blakker on June 20, 2016, 03:47:21 pm
'mscvcp140.dll missing' causing a failure for the .exe to load for me. Win7, 43.03 has been working fine.Spoiler (click to show/hide)
http://bay12games.com/dwarves/redist/msvcp140.dll
If you put this in the folder does it work, or does it give you another DLL that's missing? I tried it on three computers, but they all came with the proper DLLs I guess.
Not that person but had the same issue, tried reinstalling Visual C++ Redistributable but it didn't work.
After downloading and putting that DLL in the folder it asks for a VCRUNTIME140.dll.
Post by: Random_Dragon on June 20, 2016, 03:48:47 pm
Post by: Toady One on June 20, 2016, 03:49:32 pm
on visual studio go to Project tab -> properties - > configuration properties -> C/C++ -> Code Generation. on runtime library choose /MTd for debug mode and /MT for release mode.
This will cause the compiler to imbue the runtime into the app. The executable will be bigger, but it will run without any need of runtime dlls.
Does the legacy version have DLL problems for affected people? Apparently legacy was already compiled with /MT, where the sdl version was using /MD.
Post by: Random_Dragon on June 20, 2016, 03:51:05 pm
Post by: Orkel on June 20, 2016, 03:51:11 pm
Post by: blakker on June 20, 2016, 03:56:47 pm
on visual studio go to Project tab -> properties - > configuration properties -> C/C++ -> Code Generation. on runtime library choose /MTd for debug mode and /MT for release mode.
This will cause the compiler to imbue the runtime into the app. The executable will be bigger, but it will run without any need of runtime dlls.
Does the legacy version have DLL problems for affected people? Apparently legacy was already compiled with /MT, where the sdl version was using /MD.
Legacy version seem to be working well.
Post by: ZM5 on June 20, 2016, 03:59:18 pm
Getting the same issue here.'mscvcp140.dll missing' causing a failure for the .exe to load for me. Win7, 43.03 has been working fine.Spoiler (click to show/hide)
http://bay12games.com/dwarves/redist/msvcp140.dll
If you put this in the folder does it work, or does it give you another DLL that's missing? I tried it on three computers, but they all came with the proper DLLs I guess.
Not that person but had the same issue, tried reinstalling Visual C++ Redistributable but it didn't work.
After downloading and putting that DLL in the folder it asks for a VCRUNTIME140.dll.
Post by: Sanctume on June 20, 2016, 04:01:48 pm
Okay, I AM getting some weird shit here. DF seems to open and work fine so far, then AFTER I close it, a few seconds after I get...Spoiler (click to show/hide)
https://askleo.com/how_do_i_resolve_my_problem_with_appcompattxt/
According to the link, the Appcompat.txt has the "info" MS wants you to upload. The error is about the issue of uploading, and not necessarily about the app that you ran.
Post by: Random_Dragon on June 20, 2016, 04:06:27 pm
https://askleo.com/how_do_i_resolve_my_problem_with_appcompattxt/
According to the link, the Appcompat.txt has the "info" MS wants you to upload. The error is about the issue of uploading, and not necessarily about the app that you ran.
Which tells me nothing about why it seems to think that DF broke when it closed normally. >_>
Post by: Sanctume on June 20, 2016, 04:13:42 pm
Getting the same issue here.'mscvcp140.dll missing' causing a failure for the .exe to load for me. Win7, 43.03 has been working fine.Spoiler (click to show/hide)
http://bay12games.com/dwarves/redist/msvcp140.dll
If you put this in the folder does it work, or does it give you another DLL that's missing? I tried it on three computers, but they all came with the proper DLLs I guess.
Not that person but had the same issue, tried reinstalling Visual C++ Redistributable but it didn't work.
After downloading and putting that DLL in the folder it asks for a VCRUNTIME140.dll.
You may need to register the dll if just adding it in a folder. I would instead run the MS installer for it.
I think Toady One uses the latest Visual Studios 2015 (referred as VC14 build)
The VC14 builds require to have the Visual C++ Redistributable for Visual Studio 2015 x86 or x64 installed
Since this is only 32-bit, so you need the x86, link here (https://www.microsoft.com/en-us/download/details.aspx?id=48145)
Post by: Toady One on June 20, 2016, 04:19:43 pm
Does this work copied into the SDL folder? I'd rather not rebundle everything and update version numbers etc. etc. until this is sorted out.
Post by: Random_Dragon on June 20, 2016, 04:23:36 pm
Post by: DeKaFu on June 20, 2016, 04:24:50 pm
C:\(my filepath here)\Dwarf Fortress.exe is not a valid Win32 application.
...on Windows XP. SDL version. Neither redownloading it or downloading from the second link (without music) appears to make any difference, so I'm pretty sure it's not a corrupted download.
Could it be the same issue?
Fakedit: The new exe gives me an additional security prompt (The publisher could not be verified. Are you sure you want to run this software?) followed by the same error.
Post by: Toady One on June 20, 2016, 04:26:37 pm
Post by: DeKaFu on June 20, 2016, 04:32:12 pm
I've been playing 0.43.03 up until now with no problems, so this is something new. :-\
Post by: Toady One on June 20, 2016, 04:54:10 pm
Post by: Sanctume on June 20, 2016, 05:01:02 pm
Right-click on df.exe and choose "Run as administrator" may help the issues with windows being paranoid that it's an unsafe exe.
I seem to recall I had issues with windows having things installed on the C:\Program Files. I can't recall if that's a WinXP or Vista or Win7 thing.
Post by: mifki on June 20, 2016, 05:02:38 pm
It's possible Visual Studio 2015 has just given up on Windows XP, since they stopped supporting it a few years ago, but I have no idea right now.
http://stackoverflow.com/a/35666906/991806
Post by: Max™ on June 20, 2016, 05:08:24 pm
(http://i.imgur.com/iYEhTGp.png)
New world for 43.04, got some mods and such but the armor stuff is just coverage and name changes, they seem to be favoring the same pattern of gloves/mittens/boots/socks instead of gauntlets/high boots (lower greaves in my renamed version) and cloth.
Post by: thvaz on June 20, 2016, 05:09:44 pm
Post by: Random_Dragon on June 20, 2016, 05:10:49 pm
Anyone seen soldiers with armor yet?
The only soldier in the keep I started in had the usual breastplate and helm. He also had a mail shirt, chain leggings, and gauntlets. At least one of the goblin bandits I killed had leather armor and helm, not sure about side gear they didn't use to reliably wear.
Whee, experiments in animal glue shall ensue.
Post by: mifki on June 20, 2016, 05:11:08 pm
Post by: Max™ on June 20, 2016, 05:14:09 pm
Oh really, that's the first confirmation of additional armor being worn. Was he a weapon lord or something perhaps?Anyone seen soldiers with armor yet?
The only soldier in the keep I started in had the usual breastplate and helm. He also had a mail shirt, chain leggings, and gauntlets. At least one of the goblin bandits I killed had leather armor and helm, not sure about side gear they didn't use to reliably wear.
Whee, experiments in animal glue shall ensue.
Post by: Toady One on June 20, 2016, 05:24:15 pm
Compiling one with XP compat now... downsides to using the older SDK?
Post by: Eric Blank on June 20, 2016, 05:42:16 pm
Downloaded and replaced with the new exe, and ignoring the security prompt I get now (the prompt specifically says the publisher could not be verified) the game boots up, lets me generate a world and begin an adventure. So that seems to have fixed the missing dll issue.
Post by: Max™ on June 20, 2016, 05:42:54 pm
Post by: Random_Dragon on June 20, 2016, 05:45:19 pm
Oh really, that's the first confirmation of additional armor being worn. Was he a weapon lord or something perhaps?
A regular hammerman. I would assume that established soldiers are more likely to generate with fuller armor?
Also. This is maybe the third time I've closed Dwarf Fortress, after a painful short adventure and some arena mode testing. And yep, it still only starts thinking something's wrong AFTER closing. I need to open and test the legacy version.
EDIT: Legacy version does NOT seem to suffer post-closure insanity. ¿Por que?
Post by: Max™ on June 20, 2016, 05:49:33 pm
For the record, the linux version is stable, no crash or errors or whatnot, just the patchy armor.
Post by: Toady One on June 20, 2016, 05:53:12 pm
df xp test exe (http://bay12games.com/dwarves/redist/df_xp_test.exe)
Copy this into the SDL folder. It has the exact same byte size as the last one, but I think it's different... it threw up different warnings anyway.
Post by: Random_Dragon on June 20, 2016, 05:58:23 pm
EDIT: I'm still wishing you'd take the five seconds to copy-paste giant desert scorpion back into the raws though. ._.
Post by: Jazzeraint on June 20, 2016, 06:21:32 pm
'mscvcp140.dll missing' causing a failure for the .exe to load for me. Win7, 43.03 has been working fine.Spoiler (click to show/hide)
http://bay12games.com/dwarves/redist/msvcp140.dll
If you put this in the folder does it work, or does it give you another DLL that's missing? I tried it on three computers, but they all came with the proper DLLs I guess.
Not that person but had the same issue, tried reinstalling Visual C++ Redistributable but it didn't work.
After downloading and putting that DLL in the folder it asks for a VCRUNTIME140.dll.
I'm getting this issue now, using the provided msvcp140.dll . Tried the new .exe offered in the SDL folder, but neither it nor the original .exe are working.
Would try the offered XP download, but I'm on win7.
Post by: DeKaFu on June 20, 2016, 06:25:29 pm
...Sort of. It gives the security prompt, then launches and runs normally. But now after closing the program it pops up a "df_xp_test.exe has encountered a problem and needs to close." generic windows error message. Very similar to Random_Dragon's problem.
The game actually runs, now, though, so that's a huge improvement at least. Thanks for working on it. :)
Post by: Random_Dragon on June 20, 2016, 06:32:21 pm
...Sort of. It gives the security prompt, then launches and runs normally. But now after closing the program it pops up a "df_xp_test.exe has encountered a problem and needs to close." generic windows error message. Very similar to Random_Dragon's problem.
This is some sorcery right there, these bugs. Have you tested the legacy version? It seems to work with no issues for me, but the text formatting is hideous compared to SDL, for some reason. ;w;
Post by: Toady One on June 20, 2016, 06:32:51 pm
I'm getting this issue now, using the provided msvcp140.dll . Tried the new .exe offered in the SDL folder, but neither it nor the original .exe are working.
Would try the offered XP download, but I'm on win7.
The XP exe should work for Win7+ as well as the first new exe did, but there are still DLL problems with the new exes in general? The new exe complains about the vcruntime140.dll?
Post by: Jazzeraint on June 20, 2016, 06:49:34 pm
I'm getting this issue now, using the provided msvcp140.dll . Tried the new .exe offered in the SDL folder, but neither it nor the original .exe are working.
Would try the offered XP download, but I'm on win7.
The XP exe should work for Win7+ as well as the first new exe did, but there are still DLL problems with the new exes in general? The new exe complains about the vcruntime140.dll?
The pre-XP-test exe had the vcruntime140.dll error. The XP-test.exe is working without that error.
To comment on the rest of the thread, not receiving any error upon closing DF.
Post by: Random_Dragon on June 20, 2016, 07:00:24 pm
Post by: Libash_Thunderhead on June 20, 2016, 07:13:36 pm
See? Sorcery. ;w;I got the same error both on win7 x64 and XP x86 (running on a VirtualBox).
Edit*
I mean df_xp_test.exe
Post by: peasant cretin on June 20, 2016, 07:52:09 pm
The full armor look is a good one:
Spoiler (click to show/hide)
Post by: Chase on June 20, 2016, 08:32:40 pm
Makes sense with bandits, gonna generate an older world to increase the number of weapon lords and such, I assume that will have an influence, though that was actually a goblin soldier from a pit.
Can you confirm that adventure mode soldiers and bandits have the best armor they can have?
Post by: Max™ on June 20, 2016, 09:23:37 pm
Post by: Toady One on June 20, 2016, 09:41:53 pm
Post by: Rumrusher on June 20, 2016, 10:18:23 pm
Post by: Chase on June 20, 2016, 10:25:06 pm
Everything working on Mac OS 10.7.5
The full armor look is a good one:Spoiler (click to show/hide)
Maybe I'm mistaken about this picture. Been testing throughout the day, had a crash during the world update one time (Calendar) telling me evil names couldn't be retrieved or didn't exist. Nothing has came up since using the new exe. Apologies I didn't screen cap it, figured it was just a faulty install on my part. I have yet to see one equipped with anything out of the ordinary in adventure mode sadly.
Post by: Random_Dragon on June 20, 2016, 10:50:22 pm
Hmm.... so the hearthpeople are underequipped? Where was that screenshot from one that worked?
Odd. I've yet to see underequipped HEATHPEOPLE, but then again I haven't sampled enough. :V
Post by: fearlesslittletoaster on June 20, 2016, 11:23:35 pm
Apparently older versions of windows need patched to support something about the new version, very much not a technical person. ::)
Post by: Max™ on June 21, 2016, 12:18:13 am
Hmm.... so the hearthpeople are underequipped? Where was that screenshot from one that worked?Here's one that worked someone else had. (http://i.imgur.com/uLhqVMs.png)
Here's what I've been finding everywhere.
Elf hearthperson of the dwarf lord over there:
(http://i.imgur.com/6eJoImB.png)
Gear:
(http://i.imgur.com/ktDmAZ5.png)
Not sure what I'm missing, you didn't change anything in the armor or entity raws, all I did was change the names of a few pieces (mail shirt > hauberk, greaves/high boots > upper/lower greaves type stuff) and fiddle with the layering/coverage stuff so pieces are ordered right, but it's totally possible to wear socks or boots with the metal lower greaves, and I didn't even change anything with gauntlets. I'm standing there wearing full plate and chain, it's totally doable, but none of them have spawned with it, dunno what's up.
As a test I removed the socks, sandals, mittens, and such, to leave the only viable options as gloves+gauntlets and leather+metal boots, they just show up wearing the gloves/leather boots in the mead halls and taverns and such, so it's not just bandits with patchy armor in my experience.
Post by: Robsoie on June 21, 2016, 04:18:23 am
It did not crash while df saved my quick tested adventurer exploring his hamlet, it's only after completely quitting the game that a message about the executable having encountered an error appears.
Post by: Quietust on June 21, 2016, 06:39:01 am
Or Toady One maybe has a different compile with static linking.Switching to the statically linked C runtime will actually cause some very significant problems for the modding community (specifically for DFHack), so we would be quite grateful if that particular option could be avoided. Installing the redistributable 32-bit MSVC 2015 runtime (or placing all of the necessary DLLs in the program directory) should be sufficient for all users.
on visual studio go to Project tab -> properties - > configuration properties -> C/C++ -> Code Generation. on runtime library choose /MTd for debug mode and /MT for release mode.
This will cause the compiler to imbue the runtime into the app. The executable will be bigger, but it will run without any need of runtime dlls.
Post by: Rose on June 21, 2016, 07:43:30 am
Post by: LordBaal on June 21, 2016, 09:26:12 am
1 - Program can't start because MSVCP140.dll is missing
2 - Program can't start because VCRUNTIME140.dll is missing
Putting the msvcp140.dll file Toady linked back in the first page (http://bay12games.com/dwarves/redist/msvcp140.dll) doesn't fix the game, it still doesn't start and instead the executable returns the second error twice.
1 - Program can't start because VCRUNTIME140.dll is missing
2 - Program can't start because VCRUNTIME140.dll is missing
I guess copying the VCRUNTIME140.dll library to the game folder will fix it, however the new df.exe Toady gave in the second page (http://bay12games.com/dwarves/redist/Dwarf Fortress.exe) works too and without the need of copying the MSVCP140.dll library.
Post by: smariot on June 21, 2016, 11:02:51 am
- Statically link the libraries.
- Create an installer that installs the libraries.
- Include a link to the MSVC 2016 Redistributable (https://www.microsoft.com/en-us/download/details.aspx?id=48145) on the download page, along with the message 'If the program won't run because MSVCP140.dll is missing, install this.'
Post by: Gwolfski on June 21, 2016, 11:04:32 am
edit: Oh, and the error message pops up when i close it.
Post by: Witty on June 21, 2016, 11:09:46 am
Okay, I AM getting some weird shit here. DF seems to open and work fine so far, then AFTER I close it, a few seconds after I get...
(https://dl.dropboxusercontent.com/u/89427928/dwarven%20implosion.png)
edit: Oh, and the error message pops up when i close it.
FYI for the Toad, this is the exact error message I'm getting for my report here (http://www.bay12games.com/dwarves/mantisbt/view.php?id=9855)
Post by: Lummox JR on June 21, 2016, 11:21:23 am
My debugger shows the crash occurring at a spot where there's no code. From most recent to less on the stack:
01d1fe6c
00e26196
00e261e4
00e2649e
00e2083e
00e209ed
0040150f
0040159a
00e1752a
And then the last frame I see is in KERNEL32.
Post by: CyberPhantom on June 21, 2016, 12:05:14 pm
SDL 0.43.04: errors on launch about missing MSVCP140.dll and VCRUNTIME140.dll.
Legacy 0.43.04: no problems on launch or exit
"new df exe": no problems on launch or exit
"df xp test exe": no problems on launch or exit
Post by: Rumrusher on June 21, 2016, 12:25:57 pm
Post by: Random_Dragon on June 21, 2016, 12:32:35 pm
Post by: Iamblichos on June 21, 2016, 12:38:13 pm
Post by: Toady One on June 21, 2016, 12:50:58 pm
http://bay12games.com/dwarves/redist/vcruntime140.dll
For the SDL version on the website, are those two files sufficient (put them in the folder with the exe), or does it give an additional DLL error? I'd like to find the smallest set I can include without forcing anybody to run an installer. I'll stay away from the /MT option if possible to keep DFHack running.
I have no idea what's going on with the exit crash... was that only happening on XP/Win7 in the SDL version?
Post by: Random_Dragon on June 21, 2016, 12:54:23 pm
http://bay12games.com/dwarves/redist/msvcp140.dll
http://bay12games.com/dwarves/redist/vcruntime140.dll
For the SDL version on the website, are those two files sufficient (put them in the folder with the exe), or does it give an additional DLL error? I'd like to find the smallest set I can include without forcing anybody to run an installer. I'll stay away from the /MT option if possible to keep DFHack running.
I have no idea what's going on with the exit crash... was that only happening on XP/Win7 in the SDL version?
Derp. I went ahead and updated my Visual C++ stuff on the laptop, I'll see if that fixed it.
And unsure if other OSes are affected. But yeah, seems to only be the SDL version so far.
EDIT: As I stated in the relevant issue (http://www.bay12games.com/dwarves/mantisbt/view.php?id=9854), the link for the Visual C++ Redistributable Witty provided resolved the occurrence of that bug on my laptop. Having done so, I can now confirm the "post-closure crash" issue occurs on the laptop too (Windows 7 32-bit, same as with the desktop).
Post by: DeCervantes on June 21, 2016, 01:25:02 pm
Im using a Windows 7, 64 bit
Post by: krenshala on June 21, 2016, 01:30:04 pm
My experience so far on Windows 10:
SDL 0.43.04: errors on launch about missing MSVCP140.dll and VCRUNTIME140.dll.
Legacy 0.43.04: no problems on launch or exit
"new df exe": no problems on launch or exit
"df xp test exe": no problems on launch or exit
This matches my experience as well in Win10 64 bit. From checking the 0.43.04 zip I downloaded yesterday the two referenced DLLs are missing, an older version of the MSVCP (msvcp100.dll) is present, but no runtime file (vcruntimeXXX.dll) is present at all, old or new.
Installing the two files from Toady's post a few above this one resolves the problem. I am not seeing the error/crash on exit, though I haven't even generated a world yet.
Post by: CyberPhantom on June 21, 2016, 02:45:59 pm
http://bay12games.com/dwarves/redist/msvcp140.dll
http://bay12games.com/dwarves/redist/vcruntime140.dll
For the SDL version on the website, are those two files sufficient (put them in the folder with the exe), or does it give an additional DLL error?
On my Windows 10 machine, after I add those two DLLs to the SDL version's folder, the game starts, runs, and exits normally. (As noted above, without the DLLs, I get missing DLL errors.)
Post by: CyberPhantom on June 21, 2016, 03:08:41 pm
I have no idea what's going on with the exit crash... was that only happening on XP/Win7 in the SDL version?
Results from my testing the exit crashing:
Win XP SP3, 32-bit: exit crash when using "df xp test exe" (and the other builds won't run at all due to missing DLL errors)
Win 7 SP1, 32-bit: no exit crash when using "new df exe". yes exit crash when using "df xp test exe".
Win 10, 64-bit: no exit crash on SDL, legacy, or the two new exe's
Conclusion: it seems to happen when using the xp build, but only on XP/7.
edit: Upon further testing, it appears to crash on exit on Win 7 even with the other exes. More here (http://www.bay12forums.com/smf/index.php?topic=158868.msg7058151#msg7058151).
Post by: Random_Dragon on June 21, 2016, 03:25:50 pm
Post by: Nahere on June 21, 2016, 03:29:30 pm
http://bay12games.com/dwarves/redist/msvcp140.dllUsing these gives me a missing "api-ms-win-crt-runtime-l1-1-.dll" error.
http://bay12games.com/dwarves/redist/vcruntime140.dll
For the SDL version on the website, are those two files sufficient (put them in the folder with the exe), or does it give an additional DLL error? I'd like to find the smallest set I can include without forcing anybody to run an installer. I'll stay away from the /MT option if possible to keep DFHack running.
I have no idea what's going on with the exit crash... was that only happening on XP/Win7 in the SDL version?
Post by: Max™ on June 21, 2016, 03:45:37 pm
Post by: Random_Dragon on June 21, 2016, 03:49:27 pm
Ah, I figured it out, the problem everyone here is having must obviously be that they're using windows for some reason, so they should switch to linux, pretty sneaky, Toady, but I approve.
Is this going to go the route of Cataclysm DDA? -_-
Post by: Max™ on June 21, 2016, 03:52:34 pm
Post by: Random_Dragon on June 21, 2016, 03:57:35 pm
I don't know what that is.
Another roguelike that I used to contribute to. It's had a number of Windows-specific problems because only a couple of the project managers do much with Windows.
Post by: Toady One on June 21, 2016, 04:00:20 pm
Using these gives me a missing "api-ms-win-crt-runtime-l1-1-.dll" error.
Huh, I don't seem to have that file on my computer. What Windows version is this?
Post by: Nahere on June 21, 2016, 04:16:02 pm
Post by: Random_Dragon on June 21, 2016, 04:23:58 pm
Post by: Chase on June 21, 2016, 04:24:31 pm
Windows 8.1, but it might be because I had problems getting this (https://www.microsoft.com/en-us/download/details.aspx?id=48145) to work earlier and screwed something up.
Did you install x86? x64 doesn't do anything. That's all i needed to install
Post by: Quietust on June 21, 2016, 06:18:18 pm
Code: [Select]
Problem signature
Problem Event Name: BEX
Application Name: Dwarf Fortress.exe
Application Version: 0.0.0.0
Application Timestamp: 5768197e
Fault Module Name: Dwarf Fortress.exe
Fault Module Version: 0.0.0.0
Fault Module Timestamp: 5768197e
Exception Offset: 018e65d4
Exception Code: c0000005
Exception Data: 00000008
OS Version: 6.1.7601.2.1.0.256.1
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789[edit]
Looking in Dependency Walker, the following DLLs appear to be required:
msvcp140.dll
vcruntime140.dll
api-ms-win-crt-convert-l1-1-0.dll
api-ms-win-crt-filesystem-l1-1-0.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-locale-l1-1-0.dll
api-ms-win-crt-math-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-utility-l1-1-0.dll
I'm pretty sure the "official recommended standard procedure", when you aren't providing an installer package, is to instruct users to download and install the Visual C++ Redistributable for Visual Studio 2015 (https://www.microsoft.com/en-us/download/details.aspx?id=48145) if they don't already have it - once they've installed it for a single application, they never need to install it again (and I think Windows Update will keep it up to date for them going forward).
In fact, it's always been like this, even with Visual Studio 2010 - the reason it's usually not a problem is that most people install at least one program which requires the runtime and installs it for them.
Post by: mifki on June 21, 2016, 06:28:34 pm
Post by: Random_Dragon on June 21, 2016, 06:29:54 pm
Tearing apart giant bone armor with DOG BITES is goddamn moronic.
Post by: smariot on June 21, 2016, 06:42:44 pm
I'm not sure what horrible thing my wood cutter did to deserve having dead dwarves rain down on him, but I trust in Armok's
Post by: mifki on June 21, 2016, 06:50:17 pm
api-ms-win-crt-convert-l1-1-0.dll
api-ms-win-crt-filesystem-l1-1-0.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-locale-l1-1-0.dll
api-ms-win-crt-math-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-utility-l1-1-0.dll
These are from the new Universal CRT. Article here https://blogs.msdn.microsoft.com/vcblog/2015/03/03/introducing-the-universal-crt/ says you can distribute them with the app too. I'd personally prefer to have all DLLs bundled with DF, at least because it's always been like this. Btw, old runtime DLLs can not be removed from DF package so that they don't cause confusion.
Post by: CyberPhantom on June 21, 2016, 06:55:15 pm
On Windows 7 SP1 x64, all 3 executables (plain Dwarf Fortress.exe, "new df exe", and "df_xp_test.exe") started successfully for me whether or not the 2 DLLs are present (since I apparently already have the 2015 runtime installed) [...]
Since you reported it crashes on exit with all three executables on Win 7 SP1 x64, while my previous test (on Win 7) (http://www.bay12forums.com/smf/index.php?topic=158868.msg7057791#msg7057791) showed it only crashed with the df_xp_test exe, I tried repeating the experiment a few times. Results: you're right. It consistently crashes with any of the exes. On my box, the post-crash diagnostic window only appears after a long delay, so I must have missed it the first time around by not waiting long enough.
Post by: NCommander on June 21, 2016, 07:14:53 pm
I can't get the crash-on-exit for me, probably because I'm running Windows 10, and I don't have easy access to a Windows 7 machine at the moment. Can someone post a crash dump? Its a pain to enable on Windows, and this is the best article I can find on the subject: https://www.veritas.com/support/en_US/article.TECH74145; if someone wants to drop in #dfhack and ping me, I'll be glad to try and talk them through the process.
Got one, thanks!
Post by: Libash_Thunderhead on June 21, 2016, 07:20:00 pm
Visual C++ Redistributable for Visual Studio 2015 (https://www.microsoft.com/en-us/download/details.aspx?id=48145)Installing the x86 version fixed the missing dlls problem.
Post by: Toady One on June 21, 2016, 07:57:26 pm
Quote from: mifki
Btw, old runtime DLLs can not be removed from DF package so that they don't cause confusion.
I couldn't parse this -- they can't be removed or they should be removed? I wasn't sure if one of my old fmod dlls somehow connected back to them or whatever nightmare scenarios come up.
I've been handling pre-64bit errors and warnings all day. I should know if a day or two how much trouble there will actually be will basic saving and loading and whatever else... then we can throw 64 bit dlls into this stew and work it all out.
Post by: mifki on June 21, 2016, 08:11:16 pm
Quote from: mifkiBtw, old runtime DLLs can not be removed from DF package so that they don't cause confusion.
I couldn't parse this -- they can't be removed or they should be removed? I wasn't sure if one of my old fmod dlls somehow connected back to them or whatever nightmare scenarios come up.
I've been handling pre-64bit errors and warnings all day. I should know if a day or two how much trouble there will actually be will basic saving and loading and whatever else... then we can throw 64 bit dlls into this stew and work it all out.
Oops, can NOW be removed. I don't see fmod or any other dlls using them. Btw, fmod.dll isn't used as well, only fmodex.dll is used.
EDIT: fmod.dll is used by legacy version, and fmodex.dll by SDL version.
Post by: Max™ on June 21, 2016, 08:15:38 pm
Post by: NCommander on June 21, 2016, 08:25:03 pm
http://pastebin.com/ppmhLABb
Toady can probably work with the PDBs to determine what offset is causing the crash (I can post instructions on how to do that in a bit). I talked with Quietust, and what looks like is happening is Dwarf Fortress goes to exit, signals to the C library its time to go, then deconstructors get called, and something goes boom. The actual fault is an NX exception. I'm still trying to convince windbg to give me useful information; I struggle with it every time I have to use it.
Post by: Random_Dragon on June 21, 2016, 08:30:09 pm
Post by: breadman on June 21, 2016, 08:37:33 pm
Looking in Dependency Walker, the following DLLs appear to be required:On Windows 8.1, I saw errors for msvcp140.dll before downloading it, then vcruntime140.dll, and now api-ms-win-crt-runtime-l1-1-0.dll, so I'd guess they'll all need to be included unless we're expected to install something.
msvcp140.dll
vcruntime140.dll
api-ms-win-crt-convert-l1-1-0.dll
api-ms-win-crt-filesystem-l1-1-0.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-locale-l1-1-0.dll
api-ms-win-crt-math-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-utility-l1-1-0.dll
Post by: Quietust on June 21, 2016, 08:53:09 pm
Post by: Thundercraft on June 21, 2016, 10:58:46 pm
Quote
Allowed new citizens with some previously site-wide occupations to be reassigned
Does this refer to new migrants? Or visitors that applied for and was granted citizenship (http://dwarffortresswiki.org/index.php/DF2014:Citizenship)? I thought visitors granted citizenship could not be assigned to an occupation.
Quote
Allowed some site-wide occupations for dwarves
Site-wide? ??? Does this mean there are new occupations (http://dwarffortresswiki.org/index.php/DF2014:Occupation) which apply to the entire fort instead of a specific location (http://dwarffortresswiki.org/index.php/DF2014:Locations)?
Post by: Max™ on June 22, 2016, 12:04:21 am
Post by: DAOWAce on June 22, 2016, 12:43:46 am
Applications compiled with Visual Studio 2015 need a new VC++ redistribute package installed.
This should come with modern games compiled with it (which gets automatically installed when installing said games), which is why some people are not having issues.
Basically, install the updated VC++ Redist from Microsoft for full compatibility with modern applications.
https://www.microsoft.com/en-us/download/details.aspx?id=48145
Everyone (on Windows) should realistically have this installed. Both files if you're on a 64bit operating system, since many applications are still 32bit.
Post by: Max™ on June 22, 2016, 01:06:15 am
Still curious if artifact stuff gets worn or not.
Post by: NCommander on June 22, 2016, 01:11:58 am
- My gut feeling is there is either a heap corruption bug, or one of DF's third-party modules needs to be updated for the new compiler.
- Running Windows' heap debugger against the release binaries didn't flush any out, but the DF debugging build is more likely to catch such a bug if it exists.
- The crash is exclusive to SDL/Win32. I ran the legacy win32 build through AppVerify and couldn't get a crash there.
- fmodex.dll is linked against msvcrt.dll, this is a big no-no for userland appliactions. It may be a contributing factor; this is also true of several other of the dependency DLLs such as SDL
- The crash does not happen consistently on Windows 10/x64 for me out of the box.
- You can however get the same crash to happen by setting Application Verificar for Dwarf Fortress.exe, then enabling all Basics checks. The crash I get with this setup is identical to the one I've extracted from the minidump I posted earlier
- If you run AppVerifier, you'll get an exception close to startup. This is because fmodex.dll grabs a HANDLE and leaks it. This can be safely "continued" through, and the intro movie will start I don't believe Visual Studio can see verifers debug messages, but they do show up in WinDbg.
- I fished out the source code for SDL-1.2, and rebuilt it against Visual Studio 2015, which is now correctly linked against the VC15 runtime. I need to do the same for SDL_image/etc to make sure I can clear out as many msvcrt.dll instances as I can.
I'll take another crack at isolating this later. Hope these notes help somewhat.
Post by: Chase on June 22, 2016, 01:45:33 am
Derp on my part, I hadn't noticed that waaaaay back I had changed the [ARMOR_LEVEL:x] values for stuff while trying to see what effect it had, and upon changing them back to the vanilla ones I was able to get hearthpeople and such to generate with armor properly!
We've let toady down. I want to order him a pizza now :'(
Sorry Mr. Amphibian
Post by: NCommander on June 22, 2016, 02:12:53 am
To Reproduce:
* Turns GRAPHICS on
* Point DF at a non-existing file
This causes us to die very early in textures.cpp before the main loop fires. I'm 98% sure at this point the crash on exit is caused by one of DF's dependencies and not DF itself. My gut wants me to suspect fmodex.dll, but it could be any of them due to the mscvrt mismatchs. I may try recompiling the ones I can tomorrow or later tonight depending on how I'm feeling. Fortunately, all of these are either recompilable, or have replacements in one of the ports.
Post by: StrikaAmaru on June 22, 2016, 03:30:42 am
[...] now after closing the program it pops up a "df_xp_test.exe has encountered a problem and needs to close." generic windows error message. Very similar to Random_Dragon's problem.[...]
... Now that you mention it, I remember getting that error on XP when closing DF too, when I was playing on XP about 2 years ago. I've gotten used to ignoring it.
I just got the same behaviour myself, on a Win Server 2012 (work environment, obviously, so this is the most testing I'm doing atm). Used the 'Windows (no sound)' zip. Will see how it behaves on playthrough in about 9 hours.
Post by: Toady One on June 22, 2016, 02:47:40 pm
I have the 64 bit DF running! Without SDL so far, but I was able to share saves back and forth between 32 and 64 without immediate issues. There are a few error logs, but if the SDL version also runs, maybe this won't take forever. Then we can play with linux... where I might need a new GCC? I don't even remember how that worked, but I have a giant PM to read about that.
Post by: Random_Dragon on June 22, 2016, 02:52:17 pm
I have the 64 bit DF running! Without SDL so far, but I was able to share saves back and forth between 32 and 64 without immediate issues.
Oooh, nice. o.o
And now that both my desktop and laptop have stopped doing the exit thing, I'm even more stumped than you are.
Post by: lethosor on June 22, 2016, 03:04:01 pm
This causes us to die very early in textures.cpp before the main loop fires. I'm 98% sure at this point the crash on exit is caused by one of DF's dependencies and not DF itself. My gut wants me to suspect fmodex.dll, but it could be any of them due to the mscvrt mismatchs. I may try recompiling the ones I can tomorrow or later tonight depending on how I'm feeling. Fortunately, all of these are either recompilable, or have replacements in one of the ports.
I was away for a few days, so I missed some of this, but just to clarify, is the issue that some libraries were compiled against MSVC 2010's runtime library? In that case, mifki, wouldn't it still be necessary to distribute the old one (or would they "fall back" to the newer
Post by: Chase on June 22, 2016, 03:12:23 pm
I haven't experienced the exit crash myself (or at least any visible sign of it), so I can't jump in on the test/fix train with this one, unless I somehow can with the magic information you all possess.
I have the 64 bit DF running! Without SDL so far, but I was able to share saves back and forth between 32 and 64 without immediate issues. There are a few error logs, but if the SDL version also runs, maybe this won't take forever. Then we can play with linux... where I might need a new GCC? I don't even remember how that worked, but I have a giant PM to read about that.
Can we expect a halt on 32 bit DF once 64 bit is standard?
Post by: Random_Dragon on June 22, 2016, 03:14:23 pm
Can we expect a halt on 32 bit DF once 64 bit is standard?
Heresy. D:
Post by: LordBaal on June 22, 2016, 03:32:13 pm
Post by: Random_Dragon on June 22, 2016, 03:34:30 pm
I would rather that, but given there's still plenty of people with 32 OS I think Toady will opt to support 32 bits longer.
I for one fully intend to run my 32-bit Windows 7 into the ground. >.o
Post by: NCommander on June 22, 2016, 03:51:15 pm
I haven't experienced the exit crash myself (or at least any visible sign of it), so I can't jump in on the test/fix train with this one, unless I somehow can with the magic information you all possess.
I have the 64 bit DF running! Without SDL so far, but I was able to share saves back and forth between 32 and 64 without immediate issues. There are a few error logs, but if the SDL version also runs, maybe this won't take forever. Then we can play with linux... where I might need a new GCC? I don't even remember how that worked, but I have a giant PM to read about that.
Yay on 64-bit Windows. With luck everything goes smoothly :). On Linux, it shouldn't be too hard to get a 64-bit build though to be fair, I have no idea how you currently compile DF on Linux or Mac OS X.
At this point, given my debugging, I'm fairly sure the crash is being caused by the fact DF uses FMOD EX on Windows. C++ ABIs are incompatible across visual studio releases. Legacy dodged this bullet since the old fmod.dll only has a C API. Is it possible we can get a test binary that has FMOD EX disabled/removed to rule it out one way or another? Looking in g_src, there's a preprocessor flag (NO_FMOD) that should disable it.
I'll send you a PM if that's alright on AppVerifier which can flush out the crash; I couldn't reproduce it at first either. It should have been installed along with Visual Studio 2015.
Post by: Max™ on June 22, 2016, 03:55:22 pm
Spoiler alert: 32 bit W7 is already ran into the ground. :p
Post by: Random_Dragon on June 22, 2016, 03:56:53 pm
The poll on bitness (http://www.bay12forums.com/smf/index.php?topic=158088.0) suggests that 3% (at most) run on a 32 bit OS, which goes along with my hunch that literally nobody is using a 32 bit CPU, just a few weirdoes like our chaotic firelizard are sticking with a 32 bit version of an OS for some reason.
Spoiler alert: 32 bit W7 is already ran into the ground. :p
Mostly because some of us derps happen to have got a computer that came with Windows 7 64-bit, but not enough RAM for it. :V
Post by: Toady One on June 22, 2016, 03:57:11 pm
edit: or maybe it's worse than that. I'm getting a new group of linker errors for SDL... yeah, zlib and SDL aren't working either. It looks like the legacy version of fmod and zlib are fine, but the new ones are 32 only. I assume my SDL is 32-bit specific as well.
Post by: polartechie on June 22, 2016, 03:59:34 pm
Hello, I'm also getting the Error "VCRUNTIME140.dll is missing"
I reinstalled C++ Redist but no change. I verified that that dll is in my system32, and I even tried renaming that dll to all uppercase like the error refers to with no change in that error.
I downloaded the two test .exe's found here, but then those produce the error "fmodex.dll is missing"
After installing fmodex.dll from one of those dll sites, it gets the same error.
Also tried running all versions as administrator with no changes?
Edit: Forgot to mention I'm running Windows 7 Pro 64-bit
Solution was replacing the executable at the top of the game folders with:
http://bay12games.com/dwarves/redist/Dwarf%20Fortress.exe
Post by: NCommander on June 22, 2016, 04:04:20 pm
I might have to ditch FMOD EX anyway since I don't have a 64-bit version of FMOD EX (the link dies since fmod.h uses a hack to ensure 32 bits which doesn't work on 64 bits, possibly among other things -- trying to patch it up on a lark, but will probably have to fall back to old fmod). I haven't looked at linux yet so I don't even remember why we were using it to begin with (I have some linux stuff in the fmod folder on the windows computer, but I don't remember if we ended up using OpenAL instead or whatever).
Linux uses OpenAL which is also available for Windows; if it can't find OpenAL, it gracefully disable sounds and keeps chugging. That being said, the OpenAL code in g_src does dymsym; that will have to be rewritten for Windows to use LoadLibrary()/GetProcAddress() ... or just linked in.
For your sanity, it may be worth standardizing to one sound library.
Also, on the topic of the runtime, here's what will probably fix it permamently: Go to "C:\Program Files (x86)\Windows Kits\10\Redist\ucrt\DLLs\x86", copy and paste everything, and throw it into the DF folder. I can't test it easily, but that's all the files for the VC15 runtime. I'll post a zip in a moment.
EDIT: For anyone who wants to test or has runtime errors, grab http://dffd.bay12games.com/file.php?id=12184, and extract that zip into the same folder that you have DF 0.43.04 in.
EDIT 2: Here's the license from Microsoft on redistributing the DLLs in the runtime: http://go.microsoft.com/fwlink/?LinkId=524842
Post by: Max™ on June 22, 2016, 04:07:20 pm
You play on a toaster with 1 GB of RAM?The poll on bitness (http://www.bay12forums.com/smf/index.php?topic=158088.0) suggests that 3% (at most) run on a 32 bit OS, which goes along with my hunch that literally nobody is using a 32 bit CPU, just a few weirdoes like our chaotic firelizard are sticking with a 32 bit version of an OS for some reason.
Spoiler alert: 32 bit W7 is already ran into the ground. :p
Mostly because some of us derps happen to have got a computer that came with Windows 7 64-bit, but not enough RAM for it. :V
As for linux, I have never noticed fmod ex in any dependencies or anything. The natively built packages don't include the g_src folders or anything, just data, libs, and raw.
Post by: Shonai_Dweller on June 22, 2016, 04:08:33 pm
The poll on bitness (http://www.bay12forums.com/smf/index.php?topic=158088.0) suggests that 3% (at most) run on a 32 bit OS, which goes along with my hunch that literally nobody is using a 32 bit CPU, just a few weirdoes like our chaotic firelizard are sticking with a 32 bit version of an OS for some reason.That was a poll on bitness of computer, not OS. There's still quite a few people running 32 bit Windows on their 64 bit computers. Almost no-one is using an actual 32 bit computer these days (and should therefore all install Linux to play 64 bit DF with..) 8)
Spoiler alert: 32 bit W7 is already ran into the ground. :p
Post by: Max™ on June 22, 2016, 04:15:22 pm
*Yes, even including everyone who didn't take the poll, trying to get any OS capable of running modern software, much less df itself, on a chip that old is a Sisyphean feat.
Post by: Random_Dragon on June 22, 2016, 04:15:39 pm
You play on a toaster with 1 GB of RAM?
2 GB. It was enough for most purposes, but certain games (Hexen 2, Quake 2, Doom 3, and others) disliked trying to start. Strange given fucking Borderlands ran with no such issue. Since I was able to procure the 32-bit version, that solved it. And since then I've literally only had one game throw a hissy fit specifically because I don't have 64-bit (ARK: Survival Evolved).
Post by: lethosor on June 22, 2016, 04:29:47 pm
I assume my SDL is 32-bit specific as well.There are 64-bit versions here (SDL 1.2 and versions of SDL_image and SDL_ttf that work with SDL 1.2):
https://www.libsdl.org/download-1.2.php
https://www.libsdl.org/projects/SDL_image/release-1.2.html
https://www.libsdl.org/projects/SDL_ttf/release-1.2.html
You'll have to update the Linux and Mac libraries too - the current Linux one is just i386 (32-bit), and the OS X one is i386 and PowerPC, neither of which will work with a 64-bit DF.
In theory, changing -m32 to -m64 (or adding -m64) in the compiler flags should work when it comes to compiling a 32-bit DF with GCC, but there will probably be many linker errors as a result.
I also recommend OpenAL, mainly because it's used already on Linux and comes with the system on OS X (and is open-source and maintained!). It might be a good idea to distribute it anyway, though, since I don't know exactly which versions of which OSes distribute it by default.
Post by: mifki on June 22, 2016, 04:30:56 pm
edit: or maybe it's worse than that. I'm getting a new group of linker errors for SDL... yeah, zlib and SDL aren't working either. It looks like the legacy version of fmod and zlib are fine, but the new ones are 32 only. I assume my SDL is 32-bit specific as well.
ZLib will need to be recompiled, http://stackoverflow.com/a/26500082/991806
EDIT: And here are 64bit fmodex libraries http://www.fmod.org/download-previous-products/
Post by: lethosor on June 22, 2016, 04:38:07 pm
Post by: NCommander on June 22, 2016, 04:43:04 pm
I suspect that FMOD Ex library has the same issue as the current one, though.
I went fishing for an updated FMOD ex, and found one with ZDoom. Same problem, also linked against older runtimes. I rather not sign up for a FMOD account, but if they offer a Visual Studio 2015 build, that should work. In general, pure C libraries can be used as is, anything that using C++ needs a specific VS2015 build.
That being said, to make sure only *one* C runtime is required, its probably a good idea just to build all the dependencies against VC2015 32/64-bit. If memory serves, freetype needs a brick to the head though to build against VC2015.
Post by: DeKaFu on June 22, 2016, 04:49:45 pm
Oh I know it was supposed to be a poll on the computer bitness, but I would be surprised if 5 people* were actually using Pentium 4 era chips still, as that gen was the last 32 bit processor cycle for PC's, so I figure it is far more likely that they equated 32 bit OS with 32 bit CPU....Even if those 5 people were wrong, though, it doesn't mean there aren't another 50 who are actually running a 32 bit OS on their 64-bit processor and were savvy enough to answer the poll correctly. It doesn't prove anything one way or the other.
*Yes, even including everyone who didn't take the poll, trying to get any OS capable of running modern software, much less df itself, on a chip that old is a Sisyphean feat.
Both of my DF-worthy computers are still running Windows XP, and there's several others who have posted in this thread that they were using 32-bit XP or 7. It's a non-negligable part of the user base. Not everyone who uses an older computer or OS does so by choice, so I really hope there's no plans to abandon them. :-\
Post by: Shonai_Dweller on June 22, 2016, 05:36:01 pm
It depends on how much management it needs, but if it's enough to impact the schedule it'd be insane to keep ensuring 32 bit compatibility for the next 20 years. Why bother going 64 bit in the first place if there's no plans to ever take advantage of it?
Well, hopefully someone will eventually hack up a 64 bit XP clone for those insistent that it can't be beaten.
Post by: Gwolfski on June 22, 2016, 05:43:41 pm
Oh I know it was supposed to be a poll on the computer bitness, but I would be surprised if 5 people* were actually using Pentium 4 era chips still, as that gen was the last 32 bit processor cycle for PC's, so I figure it is far more likely that they equated 32 bit OS with 32 bit CPU.
*Yes, even including everyone who didn't take the poll, trying to get any OS capable of running modern software, much less df itself, on a chip that old is a Sisyphean feat.
Me sitting here with a spare laptop with xp and above chip running df just fine thankyou very much :P
Post by: NCommander on June 22, 2016, 05:47:11 pm
32 bit users are going to have to be cut off eventually. Or does everyone really imagine they'll be playing DF 1.0 on a 30-40 year old computer?
It depends on how much management it needs, but if it's enough to impact the schedule it'd be insane to keep ensuring 32 bit compatibility for the next 20 years. Why bother going 64 bit in the first place if there's no plans to ever take advantage of it?
Well, hopefully someone will eventually hack up a 64 bit XP clone for those insistent that it can't be beaten.
32-bit compatibility isn't going anywhere for the vast majority of things. There are tons of businesses with legacy DOS or win16 applications which can't run on 64-bit Windows which will keep it going for far longer than anyone would like.
On XP, There is a 64-bit version of Windows XP. "Windows XP Professional x64 Edition". (not to be confused with Windows XP for 64-bit Machines). x86_64 has some disinct advantages over 32-bit x86_32 because of the increased number of general availability registers that make programs faster.
However, in general (aka !x86), larger memory sizes cause slower performance; this is why you things like ARM THUMB mode for 16-bit code, or Thumb2 which allows mixing of 32-bit and 16-bit.
That being said, Windows XP is over a decade old. It's already put out to pasture. At some point, people either need to upgrade, or migrate.
Post by: Shonai_Dweller on June 22, 2016, 05:52:37 pm
And...these companies need to run Dwarf Fortress. Because...?32 bit users are going to have to be cut off eventually. Or does everyone really imagine they'll be playing DF 1.0 on a 30-40 year old computer?
It depends on how much management it needs, but if it's enough to impact the schedule it'd be insane to keep ensuring 32 bit compatibility for the next 20 years. Why bother going 64 bit in the first place if there's no plans to ever take advantage of it?
Well, hopefully someone will eventually hack up a 64 bit XP clone for those insistent that it can't be beaten.
32-bit compatibility isn't going anywhere for the vast majority of things. There are tons of businesses with legacy DOS or win16 applications which can't run on 64-bit Windows which will keep it going for far longer than anyone would like.
On XP, There is a 64-bit version of Windows XP. "Windows XP Professional x64 Edition". (not to be confused with Windows XP for 64-bit Machines). x86_64 has some disinct advantages over 32-bit x86_32 because of the increased number of general availability registers that make programs faster.
However, in general (aka !x86), larger memory sizes cause slower performance; this is why you things like ARM THUMB mode for 16-bit code, or Thumb2 which allows mixing of 32-bit and 16-bit.
That being said, Windows XP is over a decade old. It's already put out to pasture. At some point, people either need to upgrade, or migrate.
Post by: NCommander on June 22, 2016, 05:57:49 pm
And...these companies need to run Dwarf Fortress. Because...?
Solitaire isn't hardcore enough.
(you make a good point)
Post by: Toady One on June 22, 2016, 06:13:19 pm
edit: Yeah, it was hovering around 2GB, then it apparently tries to load this 600MB file and the memory shoots up to 2.5GB before it quits.
edit2: Looks like I might have to cut out some of the new compiler optimizations? As far as I can tell, this is related to incremental link-time code generation, but I won't know if turning some of that off changes the RAM usage until I give it a shot. Which'll probably have to wait until tomorrow, since I'm behind on my B12 duties. Hopefully we can get the RAM usage down without affecting the actual speed of the exe (rather than the compile time, which is partially what this stuff is for apparently).
Post by: Quietust on June 22, 2016, 07:01:42 pm
enabler.cpp contains the following new code, which I can't help but suspect may be related to the crash:
Code: [Select]
#ifdef WIN32
//NOTE: TO FIX SDL LINKER ERROR THAT CAME UP WITH VISUAL STUDIO 2015
extern "C" { FILE __iob_func[3] = { *stdin,*stdout,*stderr }; }
#endif
[edit] Just did some more research, and I can 100% confirm that this is the cause of the crash - __iob_func is supposed to be a function which returns the array, not the array itself.
This should fix the problem:
Code: [Select]
#ifdef WIN32
FILE _iob[] = {*stdin, *stdout, *stderr};
extern "C" FILE * __cdecl __iob_func(void)
{
return _iob;
}
#endif
The "proper" solution would be to recompile SDL using MSVC 2015.
Post by: NCommander on June 22, 2016, 07:05:23 pm
I just did a bit of tracing through this crash-on-exit, and I've narrowed it down to the destructor for one of the globals - it appears to be calling a function through a pointer (where said pointer points into the data segment, causing the crash) and then closing 3 file handles, which are almost definitely STDIN, STDOUT, and STDERR.
enabler.cpp contains the following new code, which I can't help but suspect may be related to the crash:Code: [Select]#ifdef WIN32
//NOTE: TO FIX SDL LINKER ERROR THAT CAME UP WITH VISUAL STUDIO 2015
extern "C" { FILE __iob_func[3] = { *stdin,*stdout,*stderr }; }
#endif
^- that linker error should be resolvable by rebuilding SDL and SDLmain static library with VS2015. __iob_func refers to symbols in the CRT that went away with VS2015. Reference: https://groups.google.com/forum/#!topic/dislin-users/kagVl9LVFJA
So I'm guessing remove the SDL linker line from enabler, rebuild SDL/SDLmain, rebuild DF?
Post by: Max™ on June 22, 2016, 07:18:09 pm
Spoiler (click to show/hide)
Post by: Random_Dragon on June 22, 2016, 07:19:12 pm
Post by: Max™ on June 22, 2016, 07:21:07 pm
Post by: Random_Dragon on June 22, 2016, 07:24:54 pm
Post by: Toady One on June 22, 2016, 07:37:31 pm
[edit] Just did some more research, and I can 100% confirm that this is the cause of the crash - __iob_func is supposed to be a function which returns the array, not the array itself.
Ha ha, yeah, I just used the first thing I found which made it compile and not obviously crash (I still have no visible crash symptoms). I can switch it over with the next set of linker tests as I try to sort out this memory problem. I could try rebuilding SDL instead, but I'll probably find a way to screw that up.
Post by: Random_Dragon on June 22, 2016, 07:38:36 pm
Ha ha, yeah, I just used the first thing I found which made it compile and not obviously crash (I still have no visible crash symptoms). I can switch it over with the next set of linker tests as I try to sort out this memory problem. I could try rebuilding SDL instead, but I'll probably find a way to screw that up.
You're reminding me far too much of any time I'd try to mess with the source code in CDDA. XP
Post by: NCommander on June 22, 2016, 07:46:25 pm
[edit] Just did some more research, and I can 100% confirm that this is the cause of the crash - __iob_func is supposed to be a function which returns the array, not the array itself.
Ha ha, yeah, I just used the first thing I found which made it compile and not obviously crash (I still have no visible crash symptoms). I can switch it over with the next set of linker tests as I try to sort out this memory problem. I could try rebuilding SDL instead, but I'll probably find a way to screw that up.
Rebuilding SDL is straight-forward. Download the source, upgrade the project file in VisualC/, and bake one release build. I have a build environment sitting here ready to go, and if you'd like, I can rebuild all of DF's dependencies against 32-bit and 64-bit for a quick and easy to go dev environment if you like.
Post by: breadman on June 22, 2016, 09:13:59 pm
On Windows 8.1, I saw errors for msvcp140.dll before downloading it, then vcruntime140.dll, and now api-ms-win-crt-runtime-l1-1-0.dll, so I'd guess they'll all need to be included unless we're expected to install something.
Also, on the topic of the runtime, here's what will probably fix it permamently: Go to "C:\Program Files (x86)\Windows Kits\10\Redist\ucrt\DLLs\x86", copy and paste everything, and throw it into the DF folder. I can't test it easily, but that's all the files for the VC15 runtime. I'll post a zip in a moment.Yes, that set of libraries allows me to run DF. I'm not seeing an error message on exit, but it does return exit code 5.
EDIT: For anyone who wants to test or has runtime errors, grab http://dffd.bay12games.com/file.php?id=12184, and extract that zip into the same folder that you have DF 0.43.04 in.
EDIT 2: Here's the license from Microsoft on redistributing the DLLs in the runtime: http://go.microsoft.com/fwlink/?LinkId=524842
Post by: Toady One on June 23, 2016, 01:37:20 pm
Current plan:
(1) get rid of several platform-independent warnings/issues
(2) bundle up some windows test zips for people to mess with in here, while:
(3) the linux journey
(4) the osx journey
Post by: LordBaal on June 23, 2016, 03:28:10 pm
I for one, humbly volunteer to test it for Windows 10 Pro 64 bits.
Post by: cesarjunior233 on June 23, 2016, 05:28:18 pm
Post by: Quietust on June 23, 2016, 05:40:19 pm
I am getting the error : The program can't start because you not have api-ms-win-crt-runtime-l1-1-0.dll in your computer.
Install the Visual C++ Redistributable for Visual Studio 2015 (https://www.microsoft.com/en-us/download/details.aspx?id=48145) for x86 (not x64).
Post by: Toady One on June 23, 2016, 06:30:46 pm
http://bay12games.com/dwarves/redist/df_sdl_32_wintest.zip
http://bay12games.com/dwarves/redist/df_sdl_64_wintest.zip
If you use the 64 bit one, it would probably be best to stay away from the bug tracker and just post strange problems in here.
Post by: thvaz on June 23, 2016, 07:16:53 pm
Here are the test zips for SDL Windows:
http://bay12games.com/dwarves/redist/df_sdl_32_wintest.zip
http://bay12games.com/dwarves/redist/df_sdl_64_wintest.zip
If you use the 64 bit one, it would probably be best to stay away from the bug tracker and just post strange problems in here.
So far I hadn't any problem. I can't say if it is slower/faster than the 32 bits as my fortress has only 50 dwarves, so the fps is on cap.
Post by: Witty on June 23, 2016, 07:54:46 pm
Here are the test zips for SDL Windows:
http://bay12games.com/dwarves/redist/df_sdl_32_wintest.zip
http://bay12games.com/dwarves/redist/df_sdl_64_wintest.zip
If you use the 64 bit one, it would probably be best to stay away from the bug tracker and just post strange problems in here.
For what's it's worth - I'm unable to reproduce the exit crash on either the 64 or 32 bit versions on both my 64 bit Window 7 machines. So that issue seems resolved now at least.
Post by: lethosor on June 23, 2016, 08:10:14 pm
Post by: Chase on June 23, 2016, 10:43:57 pm
Spoiler (click to show/hide)
This might not be 64bit related, but it's just 50+ pages of this... The dog will eventually die I'm hoping.
Uhh.. Nevermind..
Spoiler (click to show/hide)
Notifications when fish give birth is the only new thing I've encountered so far. :)
Post by: DeKaFu on June 23, 2016, 11:26:16 pm
(Exit crashes are fixed for me on XP as well, for what it's worth.)
Post by: Rose on June 24, 2016, 02:56:33 am
Spoiler: Side view. (click to show/hide)
Post by: vjek on June 24, 2016, 09:37:20 am
Generated a 500 year old world with 17k units, 10k dwarves and over 200k goblins in it.. then embarked on an entire region.
Ended up with just shy of 1.6GB of memory used (private working set), but everything worked as expected. Honestly, I thought it would crash, but it didn't! :o 8)
In any other version of DF, trying to embark on a complete region (16x16 embark size) would have crashed it, guaranteed.
Post by: Max™ on June 24, 2016, 11:17:34 am
Well, whenever the linux test is up I'll do my best to crash it out and fail, so that'll be awesome.
Also watched the dwarfmoot video last night, your little "come on down" dance was amusing, Toady.
You hit it right on the nose about the reason why the community is so nice, I called the 4chan /dfg/ thread a support group, the game beats you up enough trying to figure it out, no need to beat each other up.
Post by: Orkel on June 24, 2016, 12:59:51 pm
Post by: Max™ on June 24, 2016, 01:33:48 pm
Post by: Orkel on June 24, 2016, 02:07:52 pm
Post by: Max™ on June 24, 2016, 02:24:59 pm
I get weird world-gen crashes where the vast majority of the time nothing happens, but sometimes at weird seeming years it'll just poof and disappear. If it was specifically 32 bit or 64 bit related I'd expect it to be something like having a 257x257 with max pop/beasts/etc, but a medium world at year 152 sounds like one of the "something tried to link a culled reference" or whatnot error.
Post by: Andrakon on June 24, 2016, 03:40:26 pm
Post by: malvado on June 24, 2016, 04:40:22 pm
Going to do a few test runs as well now.
Edit : Not really smart to embark on an Island, Water movement kills fps and a 16x16 map is really huge, but it worked. 4.3 Gig memory used I think ( or was it 3.4?).
Trying inland now to see if the water was killing fps.
Post by: thvaz on June 24, 2016, 09:42:36 pm
Post by: Flarp on June 24, 2016, 10:29:22 pm
Civ-civ interaction seems to be the main factor. The Year 900 world had one continent, but its northern and southern halves were connected by a tiny, Panama-like strip of land between two seas. The slower worlds tend to be wide open plains/forests where humans and elves and goblins can go to town on each other.
I'm gonna try running worldgen with identical seeds on 32 bit and 64 bit to see if there's any improvement. It's possible more than 2GB is being used, especially if there are multiple epic wars going on simultaneously.
Post by: Thundercraft on June 25, 2016, 06:48:13 am
Playing for a while on the 64 bits version, no crash. And the fps is the same in either version. (42 with 150 dwarves and about the same number of animals)
That was what I was afraid of; That, currently, there is no FPS/speed difference between the 32 and 64-bit versions. But then, it is still in development. Hopefully, the 64-bit branch will eventually make use of multithreading and multiple cores.
I'm gonna try running worldgen with identical seeds on 32 bit and 64 bit to see if there's any improvement...
Please post your findings as they should be useful to the rest of us, either way.
Post by: Shonai_Dweller on June 25, 2016, 07:39:28 am
64 bit has nothing to do with speeding up the game or multithreading. The fear was that using 64 bit would actually slow down the game. Which thankfully doesn't seem to be the case.Playing for a while on the 64 bits version, no crash. And the fps is the same in either version. (42 with 150 dwarves and about the same number of animals)
That was what I was afraid of; That, currently, there is no FPS/speed difference between the 32 and 64-bit versions. But then, it is still in development. Hopefully, the 64-bit branch will eventually make use of multithreading and multiple cores.I'm gonna try running worldgen with identical seeds on 32 bit and 64 bit to see if there's any improvement...
Please post your findings as they should be useful to the rest of us, either way.
Post by: Criperum on June 25, 2016, 09:51:51 am
Post by: paxy97 on June 25, 2016, 09:55:39 am
Post by: Criperum on June 25, 2016, 10:01:13 am
Implementing 64bit isn't nearly as much work as adding multithreading. It would require a lot of the features to be rewritten, like pathfinding, though I guess the performance improvement would be quite big, because pathfinding for each creature can be split on multiple cores shortening the calculation time for each frame.
As programmer I really understand that turning singlethreaded app into multithreaded is literally the Hell. But hope never dies.
Post by: Zorbeltuss on June 25, 2016, 01:01:47 pm
/Zorbeltuss
Post by: Toady One on June 25, 2016, 01:39:49 pm
edit from Linux land: "sorry, unimplemented: 64-bit mode not compiled in". Well, that was polite! So I guess I have to rebuild GCC? If I built it before, that was many many years ago. Last time, I appear to have simply typed
"sudo apt-get install build-essential libsdl1.2-dev libsdl-image1.2-dev libgtk2.0-dev scons"
on a terminal on my Ubuntu computer and it did everything, but I don't recall if there was more to do. I'm not sure if I should do something differently now.
Post by: lethosor on June 25, 2016, 03:03:30 pm
Post by: Toady One on June 25, 2016, 03:09:29 pm
edit: "i386" for the mini.
Post by: Zorbeltuss on June 25, 2016, 03:47:30 pm
Yeah, I don't know that we'll ever have the different versions generating the same world. Compiler optimizations have shifted around the order of calls before in a way that I can't easily control.
edit from Linux land: "sorry, unimplemented: 64-bit mode not compiled in". Well, that was polite! So I guess I have to rebuild GCC? If I built it before, that was many many years ago. Last time, I appear to have simply typed
"sudo apt-get install build-essential libsdl1.2-dev libsdl-image1.2-dev libgtk2.0-dev scons"
on a terminal on my Ubuntu computer and it did everything, but I don't recall if there was more to do. I'm not sure if I should do something differently now.
Well it seemed like 64 bits were faster in all of world generation, just much less so in the actual history part, that however may depend on the all together higher population of that world, or the 32 bits was almost keeping up despite all the extra events that happened, however, the 64 bit version did use a lot more ram, roughly 25% more, not unexpected and not an issue if you have enough ram to profit from using a 64 bit OS, but neither is it for everyone.
/Zorbeltuss
Post by: eternaleye on June 25, 2016, 03:57:29 pm
According to what I've read, this should do the job:
Code: [Select]
sudo dpkg --add-architecture amd64
sudo apt-get install build-essential:amd64 libsdl1.2-dev:amd64 libsdl-image1.2-dev:amd64 libgtk2.0-dev:amd64 scons
The first line sets up your system to be ready for amd64 multiarch; the second line installs your dependencies' 64-bit variants (note the :amd64 suffixes). Note how I left that off of scons - scons does not need to be multiarch, because it's merely run as part of the build process, rather than being linked to by DF.
EDIT: You may also need to do some finagling of your buildsystem to _use_ the cross-compiler, which will be something like x86_64-unknown-linux-gnu-gcc, with similarly prefixed forms of ld, ar, nm, objdump, etc.
Post by: TheHossofMoss on June 25, 2016, 04:07:14 pm
http://bay12games.com/dwarves/redist/msvcp140.dll
http://bay12games.com/dwarves/redist/vcruntime140.dll
For the SDL version on the website, are those two files sufficient (put them in the folder with the exe), or does it give an additional DLL error? I'd like to find the smallest set I can include without forcing anybody to run an installer. I'll stay away from the /MT option if possible to keep DFHack running.
I have no idea what's going on with the exit crash... was that only happening on XP/Win7 in the SDL version?
Thank you so much Toady One. This fixed it for me. I'd buy you a beer if I was near. Or.. fizzy pond water. Whatever amphibians prefer ;)
Post by: Toady One on June 25, 2016, 04:19:13 pm
Code: [Select]sudo dpkg --add-architecture amd64
"dpkg: unknown option --add-architecture"
Saw various posts online tangentially related, but not really clear what to do from here.
Post by: mifki on June 25, 2016, 05:19:21 pm
Code: [Select]sudo dpkg --add-architecture amd64
"dpkg: unknown option --add-architecture"
Saw various posts online tangentially related, but not really clear what to do from here.
Looks like old dpkg and possibly Ubuntu. Do "dpkg --version" and "lsb_release -a".
On Mac, do "sw_vers -productVersion" and "system_profiler SPHardwareDataType"
Post by: Zarathustra30 on June 25, 2016, 05:27:52 pm
Edit: here's the world: http://dffd.bay12games.com/file.php?id=12192
I did a 16x16 embark on the volcano in the northeastern continent. The second time I embarked, the same thing happened. I'm not sure if this is a problem, because the game is playable. It's just weird.
Edit2: I forgot to mention, I am on Win 10 Pro.
Edit3: Embarked on a 16x2 along the same area and the problem didn't reproduce.
Post by: Toady One on June 25, 2016, 05:44:48 pm
------------
Linux:
dpkg --license: 1.14.20ubuntu1 (i386)
lsb_release -a: Ubuntu 8.10, codename intrepid
Mac:
sw_vers -productVersion: 10.5.6
system_profiler SPHardwareDataType:
Model Name: Mac mini
Model Identifier: Macmini2,1
Processor Name: Intel Core 2 Duo
Processor Speed: 1.83 GHz
Number Of Processors: 1
Total Number Of Cores: 2
L2 Cache: 2 MB
Memory: 1 GB
Bus Speed: 667 MHz
Boot ROM Version: MM21.009A.B00
SMC Version: 1.19f2
------------
Is there hope?
Post by: mifki on June 25, 2016, 05:52:48 pm
Brought to you from the ancient past:
------------
Linux:
dpkg --license: 1.14.20ubuntu1 (i386)
lsb_release -a: Ubuntu 8.10, codename intrepid
Mac:
sw_vers -productVersion: 10.5.6
system_profiler SPHardwareDataType:
Model Name: Mac mini
Model Identifier: Macmini2,1
Processor Name: Intel Core 2 Duo
Processor Speed: 1.83 GHz
Number Of Processors: 1
Total Number Of Cores: 2
L2 Cache: 2 MB
Memory: 1 GB
Bus Speed: 667 MHz
Boot ROM Version: MM21.009A.B00
SMC Version: 1.19f2
------------
Is there hope?
Yep, Ubuntu is ancient. I'd recommend just reinstalling it with something newer (maybe not latest, to avoid problems with compatibility for players on older systems). Like 15.10 or 14.04 is fine. If you're not using Linux for anything else, that should not be difficult to reinstall it + packages required to build DF.
Mac is fine, I'd upgrade it to at least 10.6, but that's not necessary. However, I need to check about 64-bit support in GCC. EDIT: What "gcc -v" says?
The main thing you need to decide is whether you want to use ancient GCC 4.5 or something newer (which may be easier to install on newer systems).
Post by: Quietust on June 25, 2016, 06:11:56 pm
Tried to benchmark the 32 bit test versus the 64 bit test in world generation, on pre-history generation the 64 bit version was much faster with identical results, but even though the seeds were identical, their histories differed by a lot and I ended the test there, there though the 32 bit version didn't seem as left behind as earlier, but since the results differed I would take that with a few grains of salt.Do the 32-bit and 64-bit versions, within themselves, consistently generate the same world with the same seeds? The fact that 32-bit and 64-bit generated different worlds would be meaningless if they themselves weren't able to generate the same world twice.
Yeah, I don't know that we'll ever have the different versions generating the same world. Compiler optimizations have shifted around the order of calls before in a way that I can't easily control.Optimization should never affect the actual behavior of the code - they may reorder individual instructions in order to minimize the impact of memory/cache latency, but the actual operations should always be the same. If 32-bit and 64-bit code behave differently, then it's due to a bug in the code itself (possibly from data types being different sizes, though that isn't a problem on Windows like it is on Linux/Mac).
Post by: Toady One on June 25, 2016, 06:33:40 pm
Quote from: Quietust
Ubuntu 10 or 11 ought work just fine, and they're both available in 64-bit (which is generally a prerequisite for x86/x64 multiarch support - building 32-bit binaries on a 64-bit system is easy, but building 64-bit binaries on a 32-bit system is a lot more work).
I don't currently know how to decide between 10 all the way up to 15. Would using 14.04/15.10 cut anybody out?
Quote from: Quietust
Yeah, I don't know that we'll ever have the different versions generating the same world. Compiler optimizations have shifted around the order of calls before in a way that I can't easily control.Optimization should never affect the actual behavior of the code - they may reorder individual instructions in order to minimize the impact of memory/cache latency, but the actual operations should always be the same. If 32-bit and 64-bit code behave differently, then it's due to a bug in the code itself (possibly from data types being different sizes, though that isn't a problem on Windows like it is on Linux/Mac).
I remember some nightmare years ago that seemed to come down to some issue with what MSVC 6 was doing vs what I was getting elsewhere, but it has been long enough that I don't recall any details. Maybe that came down to a type issue too. In either case, I doubt I'll be able to clean up the code to the point where we can get reproducible worlds between OSs/bits.
Post by: lethosor on June 25, 2016, 07:01:48 pm
Regarding OS X - I didn't know you were running 10.5, wow. It looks like that model should be 64-bit, though, according to specs I found online, so try passing -m64 to GCC and see what happens. ("uname -p" actually returns "i386" on my machine too, which is definitely not a 32-bit one.) If that doesn't work, I think newer GCC versions would work there, and some people seem to have had success with it.
Anyway, if you do end up having to rebuild compiler(s), my suggestion is to go with a newer GCC version (since GCC 4.5 is pretty unsupported) and the same version on both OSes that use it. Although if you can make the existing ones work, that's probably fine too.
Post by: Quietust on June 25, 2016, 07:07:05 pm
Post by: Random_Dragon on June 25, 2016, 07:08:07 pm
Post by: mifki on June 25, 2016, 07:11:16 pm
Anyway, if you do end up having to rebuild compiler(s), my suggestion is to go with a newer GCC version (since GCC 4.5 is pretty unsupported) and the same version on both OSes that use it. Although if you can make the existing ones work, that's probably fine too.
No need to build GCC manually, use Homebrew on OS X http://brew.sh, it supports 10.5, and standard packages on Linux.
Post by: lethosor on June 25, 2016, 07:20:01 pm
Post by: Random_Dragon on June 25, 2016, 07:21:21 pm
Post by: Thundercraft on June 25, 2016, 07:59:58 pm
64 bit has nothing to do with speeding up the game or multithreading.
Perhaps speeding up the game was not the main goal for developing a 64-bit DF. But your statement is misleading. A 64-bit app has the potential to be significantly faster than a 32-bit counterpart because it allows programming tricks to do things faster. This includes reading and writing 64-bits of data in a single instruction and letting different threads manipulate a shared variable at the same time. And, yes, a 64-bit app has the potential to take better advantage of multithreading and multiple cores than a 32-bit app.
The development of a 64-bit branch of any software project brings an expectation that it will be superior to the 32-bit version in some way. It should be more secure, more stable (i.e., less crashes due to running out of memory), run faster, or perform better in some way. Otherwise, what's the point? The move to 64-bits (not to mention maintaining separate 32-bit and 64-bit versions) is extra work.
In the case of games, especially, a move to 64-bits is almost always done for either better performance or access to more than 4 GB of memory. Usually both.
The fear was that using 64 bit would actually slow down the game. Which thankfully doesn't seem to be the case.
If the 64-bit version was significantly slower than the 32-bit version, I doubt many players would use it. Again, what would be the point?
From 34.11 to 42.xx, there have been reports of a huge drop in FPS (http://www.bay12forums.com/smf/index.php?topic=158344.msg7013637#msg7013637) and FPS Death is now much more common. And, as more and more features get added to DF, this situation is only going to get worse.
Just giving pathfinding a separate thread, alone, should see a huge boost in performance. It's been more-or-less proven that pathfinding in DF is the single biggest performance hit.
...It would require a lot of the features to be rewritten, like pathfinding...
Pathfinding would have be modified, sure. But I seriously doubt it would have to be completely rewritten merely to migrate it to a separate thread.
Post by: lethosor on June 25, 2016, 08:15:42 pm
A 64-bit app has the potential to be significantly faster than a 32-bit counterpart because it allows programming tricks to do things faster. This includes reading and writing 64-bits of data in a single instructionWell, yes, but pointers (i.e. memory addresses) are 64 bits in a 64-bit program, so the speed of working with pointers shouldn't change. Other optimizations (e.g. working with two 32-bit integers at the same time) aren't nearly as common.
Quote
and letting different threads manipulate a shared variable at the same time. And, yes, a 64-bit app has the potential to take better advantage of multithreading and multiple cores than a 32-bit app.That's not true at all. There's no difference between what 32-bit and 64-bit programs can do with threads, or CPU cores.
Quote
The development of a 64-bit branch of any software project brings an expectation that it will be superior to the 32-bit version in some way. It should be more secure, more stable (i.e., less crashes due to running out of memory), run faster, or perform better in some way. Otherwise, what's the point? The move to 64-bits (not to mention maintaining separate 32-bit and 64-bit versions) is extra work.Well, the main motivation here was to stop crashes from DF hitting the memory limit, which have gradually become more common with large worlds in newer versions.
Quote
In the case of games, especially, a move to 64-bits is almost always done for either better performance or access to more than 4 GB of memory. Usually both.The performance difference is pretty small (as a couple people have pointed out already).
Quote
Just giving pathfinding a separate thread, alone, should see a huge boost in performance. It's been more-or-less proven that pathfinding in DF is the single biggest performance hit.It's not that simple. I don't know how much would have to be rewritten, but what people don't always realize is that pathfinding needs to be in sync with the rest of the events happening in-game, so it's unlikely that a single pathfinding thread and another "other processing" thread would both be running at maximum speed. Sure, a pathfinding thread could compute the paths of all dwarves in the fortress for the next 5 years while the game is paused or not much is going on, but if a dwarf's job changes or a map change blocks some of those paths, those paths will need to be recalculated. (There's also a lot of inter-thread communication that would need to be added, which doesn't exist at all currently and is tricky to get right.)...It would require a lot of the features to be rewritten, like pathfinding...
Pathfinding would have be modified, sure. But I seriously doubt it would have to be completely rewritten merely to migrate it to a separate thread.
Post by: Max™ on June 25, 2016, 09:23:16 pm
14.04 is a LTS release so it'll be easier to keep going, the newest one is 16.04 (they're literally the year+april or october for ubuntu, oct 2015 was 15.10, etc) and you should be able to update the relevant compiler packages for a while.Quote from: QuietustUbuntu 10 or 11 ought work just fine, and they're both available in 64-bit (which is generally a prerequisite for x86/x64 multiarch support - building 32-bit binaries on a 64-bit system is easy, but building 64-bit binaries on a 32-bit system is a lot more work).
I don't currently know how to decide between 10 all the way up to 15. Would using 14.04/15.10 cut anybody out?
Post by: KillzEmAllGod on June 25, 2016, 10:43:01 pm
snipWell this is Toady we're talking about and trying to get him to multithreading won't happen just by telling him that its so much better, he already knows that he just needs to learn how to do it. The other matter is that it might take too much time just to get multithreading to work, hes pretty much said that he wouldn't want to stop making the game just to spend 2 years rewriting it so its multithreaded. Did hear that in the suggestion that just moving to 64 bit could end up with pathfinding being 5 times faster, though that might have been before running and jumping, its also early in so there might be more gains to come.
Anyway I just want magic and economy, this was rather quick for being 64 bit though I'm sure there's more to be done before the next part of the game he works on.
Toady did such a good job with 43 and the extra features of armor and weapon damage.
Post by: lethosor on June 25, 2016, 10:51:35 pm
Did hear that in the suggestion that just moving to 64 bit could end up with pathfinding being 5 times faster, though that might have been before running and jumping, its also early in so there might be more gains to come.That's not true at all (see above). Where'd you hear that?
Edit: "above" being the previous few pages, I guess.
Post by: Random_Dragon on June 25, 2016, 11:11:53 pm
I just generated the same exact world twice in a row with the 64-bit Windows build, and history turned out differently each time - I used the seeds ABCDEFGHIJKLMNOPQRST + UVWXYZabcdefghijklmn + opqrstuvwxyz01234567 + 89ABCDEFGHIJKLMNOPQR on a MEDIUM REGION with a 100 year history, and while both worlds had the same geography and initial historical figures and sites/structures, history itself turned out completely differently. The problem may simply be that history is not deterministic, and the bug tracker (http://www.bay12games.com/dwarves/mantisbt/view.php?id=6934) confirms that this has been a problem since at least version 0.40.
I'm still wanting to know if you eliminated orographic precipitation as a cause. D:
Post by: Zarathustra30 on June 25, 2016, 11:16:45 pm
Did hear that in the suggestion that just moving to 64 bit could end up with pathfinding being 5 times faster, though that might have been before running and jumping, its also early in so there might be more gains to come.That's not true at all (see above). Where'd you hear that?
Edit: "above" being the previous few pages, I guess.
With 64-bit, memory ceases to be an issue, so paths can be cached to hell and back. It won't come immediately with the upgrade, but it is something to keep in mind.
Post by: Shonai_Dweller on June 26, 2016, 12:55:37 am
However just now my adventurer got into a fight, mortally wounded I defiantly bit my opponent in the arm and he vanished. Or perhaps turned into a tree. Not sure.
Combat log says he kicked me first before disappearing. Has anyone seen this before? Will upload a save to DFFD but guess I should avoid the bug tracker for now. No mods used.
--Added save
http://dffd.bay12games.com/file.php?id=12194
Post by: Max™ on June 26, 2016, 02:58:22 am
Post by: eternaleye on June 26, 2016, 04:18:27 am
With function multiversioning (old form, GCC 4.8+) one can declare multiple versions of a function, which are optimized differently, and execute only on CPUs capable of the features they use. With the new form (GCC 6), you can simply declare which variants you want, and the compiler will do the work of generating the versions _for_ you - so for example, you can have tight code (such as water or temperature propagations) optimized for SSE2, SSE4.2, AVX, and AVX2 (along with plain, normal scalar code), and the best one the CPU can execute will be selected at runtime.
Old form (GCC 4.8+):
Code: [Select]
__attribute__ ((target ("sse4.2")))
int foo(){
// foo version for SSE4.2
return 1;
}
__attribute__ ((target ("sse2")))
int foo(){
// foo version for SSE2
return 1;
}
int main() {
return foo();
}
New form (GCC 6):
Code: [Select]
__attribute__((target_clones("avx2","avx","sse4.2","sse2","default")))
int foo(){
return 1;
}
int main() {
return foo();
}
Put the target_clones attribute behind an #ifdef for GCC >= 6, and free performance win without sacrificing compatibility once it lands in your distro.
[1]: https://lwn.net/SubscriberLink/691932/0a7303ac5f1ac1a3/
Post by: Zorbeltuss on June 26, 2016, 05:02:51 am
I remember some nightmare years ago that seemed to come down to some issue with what MSVC 6 was doing vs what I was getting elsewhere, but it has been long enough that I don't recall any details. Maybe that came down to a type issue too. In either case, I doubt I'll be able to clean up the code to the point where we can get reproducible worlds between OSs/bits.I remember that too, which was one of the reasons I reported it (wasn't it about the mac version though IIRC?
I just generated the same exact world twice in a row with the 64-bit Windows build, and history turned out differently each time - I used the seeds ABCDEFGHIJKLMNOPQRST + UVWXYZabcdefghijklmn + opqrstuvwxyz01234567 + 89ABCDEFGHIJKLMNOPQR on a MEDIUM REGION with a 100 year history, and while both worlds had the same geography and initial historical figures and sites/structures, history itself turned out completely differently. The problem may simply be that history is not deterministic, and the bug tracker (http://www.bay12games.com/dwarves/mantisbt/view.php?id=6934) confirms that this has been a problem since at least version 0.40.Yep tested and history do not seem to be deterministic.
Did you have rain shadows on? :VI did, now I do not, history is not deterministic either way on 64 bit, will soon know on 32 bit as that has been slower in compliting it's assigned tasks, will also try to remove periodic erodation of cliffs though I doubt that would be the issue.
Edit: 32 bit was also non-deterministic, not really surprising but useful to know.
/Zorbeltuss
Post by: KillzEmAllGod on June 26, 2016, 05:26:47 am
It was in the suggestion for voting that mentioned that up it could improve pathfinding to be up to 5 times faster just from DF using the gains in performance from being 64 bit, though we're unlikely to see such gains or sometime soon.Did hear that in the suggestion that just moving to 64 bit could end up with pathfinding being 5 times faster, though that might have been before running and jumping, its also early in so there might be more gains to come.That's not true at all (see above). Where'd you hear that?
Edit: "above" being the previous few pages, I guess.
Post by: FantasticDorf on June 26, 2016, 06:37:18 am
Recovery from large events (such as 5 fire imps and a fire man shooting projectiles at once in arena at a bunch of archers hovering in the air returning fire with arrows) recovers quickly and for the first time (without pushing my luck too far) larger embarks & longer world generations are being tolerated (barely but noticeably, take for example before in terms of my system 125 years+ was undoable and anything larger than the standard embark would crash or be VERY choppy)
Big thumbs up from me.
Post by: Zorbeltuss on June 26, 2016, 07:34:28 am
/Zorbeltuss
Post by: Quietust on June 26, 2016, 08:08:22 am
I'm still wanting to know if you eliminated orographic precipitation as a cause. D:Why should that have anything to do with deterministic world generation?
Post by: soulsource on June 26, 2016, 08:24:42 am
I'd definitely set up a 64 bit Linux install to build the 64bit applications, as compiling 64bit software on a 32bit system is more or less cross compiling (as you can't run 64 bit code on a 32bit system). You'd not only need a working cross compilation toolchain, but also all required libraries for the target architecture. Then the question would be: Why keep a Linux installation? If you are cross compiling anyhow, you could just as well do the Linux build on Windows... The other way - compiling a 32bit program on a 64bit Linux system - is rather straightforward though. You'll still need 32bit libraries, but thanks to multilib (what is supported on almost any recent 64bit Linux distribution), pulling those in is trivial (on Debian based systems, just suffix the package name with :i386 and that should do), and AMD64 gcc will happily produce i386 binaries if it's being run with the -m32 switch.
If you want to use Ubuntu for the Linux build, I'd suggest to pick the second-newest LTS release, which would currently be 14.04LTS. Using the newest version of any distribution will cause issues with other distributions that haven't updated their SDL (or other libs used by DF) in between, while the second latest should hopefully produce a build that's working on most others.
By using 14.04LTS, you'd likely loose compatibility with some ancient distribution versions that still are supported by their respective distributors (Ubuntu 12.04LTS, CentOS 6, RHEL 6,...), but I strongly doubt that (m)any people play games on machines still running those...
Post by: Knossos on June 26, 2016, 08:54:33 am
I get some hilarious negative FPSs (http://i.imgur.com/XsYg21G.png) when it is calculating.
I usually run no-FPS-limit games. Looking forward to seeing how the late game runs! At any case, in early games, dwarfs are moving even more lightspeedy than usual. \o/
Post by: Caprealis on June 26, 2016, 10:08:30 am
My doctors get stuck on a diagnostic loop, Causing them to keep trying to diagnose and quickly becoming legends. The dwarf in question is still waiting for some help though, He's been sitting there with a broken leg for a few seasons now.. And if I lock them inside the hospital, they climb through the well, Under the wall and back ontop of the building to get back into the fortress.(Even babies are doing it)
Other than that dwarfs seem very eager to jump over ANYTHING that is stopping them from attacking something. Causing a lot of inconveniences..
But I am still very impressed with how well the game is running.
Post by: nop on June 26, 2016, 10:14:16 am
well, I would use ubuntu 16.04 LTS x64 for compiling. If older versions of linux are required, then I would use docker. The newer the compiler, the faster it produces.
Compile with max optimizations enabled (there is no need to patch the code, just use different compiler flags), such as "-O3 -funroll-loops" for GCC, and fast inaccurate floating point arithmetics. Also, it is definitely a good idea to produce several versions of binaries with different cpu instructions support (vanilla, SSE2, AVX, AVX2 - the only difference for you is just a compiler flag), because it does not take any (!) additional time, but may yield tens of percents of execution speed. This is valid for both Windows, Linux and OSX
Also, I would compile with Profile Guided Optimization (PGO), this gives additional performance improvements without any changes to the code, but requires a little more time of yours. PGO implies compilation in two steps. On the first step you run an instrumented binaries that collect statistics, on the second step compiler recompiles binaries with the knowledge of this statistics.
Post by: Greiger on June 26, 2016, 10:42:59 am
Started up a max embark size map with a mountain, volcano and brook. Took about 5 minutes to load the map. And there was some slowdown for the first 10 or 15 seconds after unpausing but after that things seem to be going great. Currently running that map at 50 FPS shortly after embark which is my comfortable fps cap. I kinda didn't expect it to run that well or I would have set the fps cap higher.
So pretty much just a preliminary report that nothing seems horribly borked right out the bat. Probably gunna need others to fully test max size maps like this for the longplay though. While I'm not likely going to play a full size map like this for realz, it does mean that I'll probably be extending my embark size out to 6x6 or 8x8 instead of my current 4x4 when I play 64 bit. Instability is really the only thing keeping me from doing it in 32 bit now.
Spoiler: Specs (click to show/hide)
Post by: lethosor on June 26, 2016, 11:28:35 am
Just because DF can use 8 GB of memory (for example) for paths if it wants doesn't mean everyone suddenly has 8 GB of memory to throw at DF. If DF does start requiring huge amounts of memory on a normal basis, it will start swapping and become really slow for a lot of players. The main advantage of a 64-bit build is that people who do have large amounts of memory can play really large worlds (or do other memory-intensive things) without DF crashing because it's incapable of using that much memory.Did hear that in the suggestion that just moving to 64 bit could end up with pathfinding being 5 times faster, though that might have been before running and jumping, its also early in so there might be more gains to come.That's not true at all (see above). Where'd you hear that?
Edit: "above" being the previous few pages, I guess.
With 64-bit, memory ceases to be an issue, so paths can be cached to hell and back. It won't come immediately with the upgrade, but it is something to keep in mind.
Post by: Random_Dragon on June 26, 2016, 12:09:08 pm
I'm still wanting to know if you eliminated orographic precipitation as a cause. D:Why should that have anything to do with deterministic world generation?
If it affects the eventual layout of the map's terrain, affecting where sites get placed, affecting whether a town is eligible to get placed, etc. Either way, Zorbeltuss seems to have determined that history results vary even in 32-bit, so this question was a moot point.
Post by: waveclaw on June 26, 2016, 01:22:26 pm
There is much to consider. C++ 0x11 changes in GCC 5 mean that software compiled with newer systems won't run on older systems. The newer systems have a dual ABI (Application Binary Interface) to handle this but really older systems are out of luck See the GCC manual on Dual ABI (https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html).
lsb_release -a: Ubuntu 8.10, codename intrepid
The real worry with moving from such an ancient system is crossing the glibc 2.1 symbol versioning threshold. After 2.1 the GNU standard C library started versioning symbols - that is the name of functions and variables exported for use. Software written with newer GCC (>=3.4) and older GCC (<3.4) have all sorts of problems.
I don't currently know how to decide between 10 all the way up to 15. Would using 14.04/15.10 cut anybody out?
The issue to watch for is the C++ 0x11 updates, This was a very big deal in the 2011-2012 time frame. Almost as big at the 2014 work on making /usr and /lib into links to /usr/lib or /usr/bin. Since you have to cross that huge gap for the symbol issue why not go with the latest version?
Linux users tend to be a very diverse bunch but general end-users follow the normal OS adoption curve. Perhaps take a user poll? Examine the bug tracker for OS releases on recent or active tickets?
At the worst the binaries could be statically linked (SDL.a included), The binaries would be huge and people use alternatives like uclib.a to help reduce the code bloat. And it would piss off people who make native packages since statically linked binaries are strongly disliked by Linux distribution makers. With 32-bit versions a static binary is like to be too big to load into memory. For 64-bit a statically linked game might load but just be ridiculously slow.
Using a newer GCC might break compatibility for some people, but that should be relatively easy to fix by distributing the newer libstdc++ that comes with it. (
In Windows it is seen frequently that developers ship their standard C library or an installer package for it with an application for just that application's use.
In Linux this is usually not done. An installation of Linux or BSD is a distribution of software after all. Anything missing should be added to the installed system for everyone to use. Linux development workflows and release paradigms are all designed around this idea.
On top of this the glibc and libstdc++ libraries are special on Linux. The glibc library is the ABI (Application Binary Interface). That is your software's interface to the kernel and thus the world. So glibc and libstdc++ are not just a grab bag of standard C/C++ stuff but are 'the computer' for your native application. Yes, you can build software directly against kernel APIs, et cetera and throw away any pretense of running on anything else.
Plus you can't use tricks like LIBPATH and LD_LIBRARY_PATH to override glibc. You have to mess with the application loader's RPATH on compile time and LD_PRELOAD custom libraries at run time. Even then your success will vary. See http://www.lightofdawn.org/wiki/wiki.cgi/NewAppsOnOldGlibc (http://www.lightofdawn.org/wiki/wiki.cgi/NewAppsOnOldGlibc) for some of the stuff involved.
This ABI not only has to match between your application and libstdc++ but also for every library and kernel function you want to load. Problems with the ABI cause 'Symbol not Found' errors when compiling and linker errors from ld.so when trying to run applications such as "version `GLIBC_x.y.z` not found."
So to ship libstdc++.so you'd also have to ship matching libraries for everything that the application needs. Custom SDL. Custom TTY. Custom sound and OpenGL libraries for SDL. Matching libpthread even though you never directly use threads. Everything. This runs into 100s of libraries for some applications. This is usually where a company resorts to static compiles.
Oh, you can forget about DFHack or gdb working unless they ship the same pile of libraries as Dwarf Fortress, too. Those need to match close enough to inspect the running game and inject data into it.
well, I would use ubuntu 16.04 LTS x64 for compiling. If older versions of linux are required, then I would use docker.
Even if you ship the universe of "shared" libraries there's also the risk that your custom world of stuff you are running work NOT work on the Linux kernel you have installed. Containers like docker are popular because they manage all of this and attempt to resolve that issue. But the ability to run a container implies you are running a 2014 or later Linux distribution probably with systemd.
Should Dwarf Fortress start trying to ship glibc.so or libstdc++.so I'd have to strip it our when packaging it (https://github.com/waveclaw/dwarffortress-packaging). Even then basic checks on the package (rpmlint, deblint, etc) would scream about shipping any standard library. Services like the Open Build Server (http://build.opensuse.org) would just refuse to build the package.
This is probably not a big loss. It appears that most people who want something as convenient as packaged software use one of the big install kits. Those kits come with custom tile sets, DFHack and the other quality of life improvement tools. (Plus I never got a clear license answer to my question so I cannot Legally provide binary packages, either.)
Also, it is definitely a good idea to produce several versions of binaries with different cpu instructions support (vanilla, SSE2, AVX, AVX2 - the only difference for you is just a compiler flag), because it does not take any (!) additional time, but may yield tens of percents of execution speed. This is valid for both Windows, Linux and OSX
As eternaleye pointed out, GCC and Linux support function multi-versioning since 4.8 (https://gcc.gnu.org/wiki/FunctionMultiVersioning). There is no need to generate different binaries. This is another reason to switch to the latest GCC for full support of the range of optimizations available.
Do note that some kinds of optimization will influence repeat-ability of simulations like Dwarf Fortress world generation. Lookup the scholarly articles on optimization robustness for some cool math on stability, sensitivity and limit theorems of multidimensional optimization.
Post by: Cexp on June 26, 2016, 01:48:54 pm
Advantages:
* You can easily have many more configurations (i.e. keeping an ancient x86 distro for maximum compatibility - also less likely to break due to HW incompatibilities - and then a modern one for x64)
* You can move and backup them, they are not tied to a certain host machine
* Nowadays performance is great (for compiling, I measured <5% slowdown compared to native; depends on hardware)
* Testing upgrades / new configurations is less risky (you can test in a different VM)
* More developer comfort (depending on the current way of doing things)
There are some idiosyncrasies to work out but in my opinion it sure beats having a separate dedicated build machine. Testing might be a bit trickier, but I've run DF within a VM in the past without issues (granted getting video acceleration to work WAS an issue; maybe things have improved since then).
Post by: Toady One on June 26, 2016, 02:42:04 pm
Post by: Admiral Obvious on June 26, 2016, 02:54:50 pm
Went and stress tested worldgen, with maximum everything, and it went fine for me, on a 64 bit Windows 7, Ultimate (if OS type matters).
Clocked the game using up to the magical 4 GB number, and it didn't crash, on an evil ocean island, with a brook with a 16x16 embark, in a 1050 year old world. Framerate was awful, even for my tank of a PC, but by armok, it ran.
Post by: lethosor on June 26, 2016, 02:55:17 pm
There's also a way to disable the newer GCC 5 ABI (-D_GLIBCXX_USE_CXX11_ABI=0), if that's what you were referring to.
Post by: Toady One on June 26, 2016, 03:15:37 pm
Post by: lethosor on June 26, 2016, 03:29:42 pm
Regarding zconf.h, you might try installing zlib1g-dev:i386, or maybe lib32z1-dev.
Post by: Toady One on June 26, 2016, 03:58:14 pm
edit: so I installed it again... and now it looks like 32 bit doesn't have the error message, but it's crapping out the same way 64 bit does. I'll see if I can get a smaller file built.
(gcc is 4.8.4)
Post by: mifki on June 26, 2016, 04:08:03 pm
Getting zlib1g-dev:i386 removed gcc from my computer, apparently...
edit: so I installed it again... and now it looks like 32 bit doesn't have the error message, but it's crapping out the same way 64 bit does. I'll see if I can get a smaller file built.
(gcc is 4.8.4)
sudo apt-get install gcc-multilib
And then again
sudo apt-get install zlib1g-dev:i386
and
sudo apt-get install zlib1g-dev:amd64
Post by: lethosor on June 26, 2016, 04:10:56 pm
(edit: clarified that missing packages probably aren't the cause of the build hanging)
Post by: Toady One on June 26, 2016, 04:20:03 pm
edit:
g++ file.cpp -m32 -O3 -o test
and
g++ file.cpp -m64 -O3 -o test
both produce a working hello world "test" executable.
Post by: lethosor on June 26, 2016, 04:32:34 pm
Post by: Toady One on June 26, 2016, 04:42:49 pm
edit: I changed the build script to use just one core, and now it is going through the files at around 50% mem usage. We'll see what happens when it hits a bad one.
edit: Yeah, it's bumping up against my 2GB and going to swap on occasion. I'm going to let it run through to the end and see if we have any actual error messages at the link/libgraphics phase. I'm guessing it'll take about an hour to compile everything, but I'm not sure since it might hit a bump later on.
edit: made it to libgraphics (this is the 32 bit attempt)
Code: [Select]
g++ -o src/libgraphics.so -Wl,--as-needed -Wl,-rpath=\$ORIGIN -m32 -pthread -shared src/g_src/enabler.os src/g_src/basics.os src/g_src/command_line.os src/g_src/graphics.os src/g_src/init.os src/g_src/interface.os src/g_src/win32_compat.os src/g_src/music_and_sound_openal.os src/g_src/random.os src/g_src/textlines.os src/g_src/keybindings.os src/g_src/ViewBase.os src/g_src/textures.os src/g_src/files.os src/g_src/enabler_input.os src/g_src/GL/glew.os src/g_src/KeybindingScreen.os src/g_src/resize++.os src/g_src/renderer_offscreen.os src/g_src/ttf_manager.os src/g_src/find_files_posix.os -Llibs -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgio-2.0 -lpangoft2-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo -lpango-1.0 -lfontconfig -lgobject-2.0 -lglib-2.0 -lfreetype -lSDL -lSDL_image -lGLU -lz -lSDL_ttf
/usr/bin/ld: cannot find -lgtk-x11-2.0
/usr/bin/ld: cannot find -lgdk-x11-2.0
/usr/bin/ld: cannot find -latk-1.0
/usr/bin/ld: cannot find -lgio-2.0
/usr/bin/ld: cannot find -lpangoft2-1.0
/usr/bin/ld: cannot find -lpangocairo-1.0
/usr/bin/ld: cannot find -lgdk_pixbuf-2.0
/usr/bin/ld: cannot find -lcairo
/usr/bin/ld: cannot find -lpango-1.0
/usr/bin/ld: cannot find -lfontconfig
/usr/bin/ld: cannot find -lgobject-2.0
/usr/bin/ld: cannot find -lglib-2.0
/usr/bin/ld: cannot find -lfreetype
/usr/bin/ld: cannot find -lSDL
/usr/bin/ld: cannot find -lSDL_image
/usr/bin/ld: cannot find -lGLU
/usr/bin/ld: cannot find -lSDL_ttf
collect2: error: ld returned 1 exit status
scons: *** [src/libgraphics.so] Error 1
scons: building terminated because of errors.
Do I need more packages? I have the ones from before (libsdl1.2-dev etc.). Or is it something else?
Post by: mifki on June 26, 2016, 06:33:23 pm
Do I need more packages? I have the ones from before (libsdl1.2-dev etc.). Or is it something else?
Basically yes, you'll need to install dev packages with both :i386 and :amd64 suffixes. The problem is that it's sometimes hard to understand what package contains the needed library.
There's likely a command to find a package that provides a filename, but you can just do apt-cache search <libname> and choose something with -dev suffix that looks most appropriate.
Post by: Toady One on June 26, 2016, 06:38:17 pm
Unable to correct problems, you have held broken packages.
Post by: mifki on June 26, 2016, 06:50:23 pm
libsdl1.2-dev:i386 : Depends : libcaca-dev:i386 but it is not going to be installed
Unable to correct problems, you have held broken packages.
Oh I see (i386 dev packages conflict with amd64 dev packages).. But I have no idea how to fix it. Probably there's a way to set up a multiarch dev environment properly (like set up virtual environments with different architectures), but I don't know.
Unless someone comes and tells how to do that and it's easy, what I'd do in this case is simply have two VMs with i386 and amd64 Ubuntu and use them to build separately.
Post by: lethosor on June 26, 2016, 07:02:15 pm
Failing that, if you need to use a VM, running 64-bit Linux natively and 32-bit Linux in a VM (hosted on your Windows machine, if that has more memory available) would probably be more efficient than the other way around. Not sure if an online VM is something you'd want to use to compile DF.
Post by: mifki on June 26, 2016, 07:39:06 pm
Code: [Select]
wget https://releases.hashicorp.com/vagrant/1.8.4/vagrant_1.8.4_x86_64.deb
dpkg -i vagrant_1.8.4_x86_64.deb
apt-get install virtualbox
mkdir df_32 && cd df_32
vagrant init ubuntu/trusty32
vagrant up
Then you do "vagrant ssh" and you're logged into a 32bit machine AND your df_32 folder is accessible from inside the machine as /vagrant. And you can use your physical Linux machine itself for 64bit builds, by removing 32bit libraries and installing just 64bit.
Post by: nop on June 26, 2016, 09:04:22 pm
@toady
it is a crazy idea, but what about a twitch stream 'let us compile dwarf fortress' with the help from the chat?
Post by: mifki on June 26, 2016, 09:18:58 pm
vagrant is an old day, use docker for setting up 32bit environment and compiling
Possible but more difficult to configure and use.
Post by: Toady One on June 27, 2016, 01:41:41 am
Does just installing libsdl-1.2:i386 work? It may be that the different headers are conflicting but the libraries themselves aren't (but I'm really not an expert on this).
"libsdl-1.2:i386" wasn't there, and "libsdl1.2:i386" was referenced by other packages but 'missing or obsolete'. I'm not sure how you just get the library.
I'm trying a 64 bit build now to see how that works out in the link stage (1 core compiling seems to be squeaking through). I'll worry about VMs for 32 after that, if that's the way to go.
Post by: PatrikLundell on June 27, 2016, 02:30:30 am
Post by: Toady One on June 27, 2016, 02:58:47 am
I've done a lot of world generation using the 64 bit version and got a crash yesterday (probably not specifically related to 64 bit, but just the same as occasionally happened earlier as well). I've got the event report info and a crash dump. Are these of interest? If so, I can upload the dump and provide a link.
Unfortunately, I don't know how to get anything out of those, and it's compounded by the discussion in the thread about worlds not generately the same way for the same seed. I should probably fix up that situation as soon as I can get to it.
Post by: PatrikLundell on June 27, 2016, 03:57:23 am
Post by: Thief^ on June 27, 2016, 04:02:40 am
I've done a lot of world generation using the 64 bit version and got a crash yesterday (probably not specifically related to 64 bit, but just the same as occasionally happened earlier as well). I've got the event report info and a crash dump. Are these of interest? If so, I can upload the dump and provide a link.
Unfortunately, I don't know how to get anything out of those, and it's compounded by the discussion in the thread about worlds not generately the same way for the same seed. I should probably fix up that situation as soon as I can get to it.
For crash dumps from windows, you just open them in visual studio and press "run" and it replicates the same crash that actually happened. You'll start off just getting the error message and access to the assembly I think, but if you can point it at the original pdb file and source code of the release it'll give you a full debugger experience.
Post by: Max™ on June 27, 2016, 04:14:19 am
Post by: int_ua on June 27, 2016, 05:09:40 am
Allowing to work with the same home directory is currently omitted there, check https://wiki.ubuntu.com/SimpleSbuild#Mount_your_home_dir (https://wiki.ubuntu.com/SimpleSbuild#Mount_your_home_dir)
It should avoid overhead of creating a whole new install (it will download only the packages that differ, AFAIU) and running a VM.
Post by: Quietust on June 27, 2016, 06:41:42 am
Linux 64 linked, but I get a compression error trying to run it. Wrong zlib maybe? Not sure how to check which one it is using. Or I have some 64 bit problem with the file loading code itself that bothers linux and not windows.Are you saving and loading "long" variables anywhere? If so, that's going to cause major problems on Linux and Mac, because while "long" is 32 bits wide on Windows x86_64, it is 64 bits wide on Linux/OSX x86_64. You might need to go through and change them all to "int".
[edit]
Code: [Select]
char file_compressorst::flush_in_buffer()
...
//WRITE IT
f.write((char*)&compsize,sizeof(long));
f.write(out_buffer,c_stream.total_out);
...
Yep, that's definitely going to be a problem.Post by: Veroule on June 27, 2016, 07:40:52 am
http://forge.ryzom.com/wiki/Building_Ryzom_Client_On_Debian
It was written based on Debian which is almost entirely identical to Ubuntu, the things to change should be easy to spot. If you go with such a setup you should be able to get the compile part down to one command.
I really strongly recommend using explicit sizing whenever possible. 64bit processors have been around long enough, and with core speed and die limits being what they are, the move to 128bit is not too far in the future. Taking the time to shift to specific sizing now can make the next transition painless. See http://en.cppreference.com/w/cpp/types/integer When you use the type intptr_t or uintptr_t you will always get a size that matches the register size of the architecture and that will have the highest speed. There is a small speed loss on some operations on 64bit architecture when working with a 32bit integer, the same is true of 16bit on 32bit architecture. Making sure counter variables are sized to match the registers will keep the speed from being lost.
Post by: Thief^ on June 27, 2016, 08:09:09 am
I really strongly recommend using explicit sizing whenever possible. 64bit processors have been around long enough, and with core speed and die limits being what they are, the move to 128bit is not too far in the future. Taking the time to shift to specific sizing now can make the next transition painless. See http://en.cppreference.com/w/cpp/types/integer When you use the type intptr_t or uintptr_t you will always get a size that matches the register size of the architecture and that will have the highest speed. There is a small speed loss on some operations on 64bit architecture when working with a 32bit integer, the same is true of 16bit on 32bit architecture. Making sure counter variables are sized to match the registers will keep the speed from being lost.Actually, that's mostly nonsense.
For starters, there's no appreciable speed difference in performance by using a smaller integer size than the CPU's bitness, and due to the reduced ram need it can actually be faster to use smaller types. There's also no guarantee anywhere that intptr_t is faster than "int". If you specifically want a "fast" type, use the fast types, e.g. int_fast32_t. On Visual Studio (https://msdn.microsoft.com/en-us/library/323b6b3k.aspx) at least, the only one that's not the same size as its fixed-size version is int_fast16_t.
As for "the move to 128bit is not too far in the future": Depending on how you work it, either we already have 128-bit CPUs, or we never will (at least for many decades).
"never will": The primary benefit to 64-bit CPUs is the size of the addressable space. 32-bit could address a maximum of 4 GB, 64-bits can uniquely address every byte of over a million 2 TB hard-disks. This is way beyond desktop requirements (intentionally!). We didn't get 64-bit CPUs until we were already hitting the 4 GB limit of 32-bit addressing with RAM, having one system with millions of terabytes of ram in a home system is utterly ludicrous at this point.
"Already do": SSE2 / AVX. Vector registers are already 128-bits wide, and on some newer CPUs are 256-bits wide. While they can actually be used on up to 128-bit integers, they're still primarily used for parallel computation on 32-bit numbers.
You got one thing right though. Use the fixed-size types (like int32_t) for pretty much everything. It'll save you many headaches.
Post by: trousermonkey on June 27, 2016, 09:42:25 am
Use ldd get the list of libraries linked into your executable:
Code: [Select]
~/df_linux/libs$ ldd Dwarf_Fortress
linux-gate.so.1 => (0xf772a000)
libSDL-1.2.so.0 => /usr/lib/i386-linux-gnu/libSDL-1.2.so.0 (0xf7655000)
libgraphics.so => /home/na/df_linux/libs/./libgraphics.so (0xf7226000)
libstdc++.so.6 => /home/na/df_linux/libs/./libstdc++.so.6 (0xf7148000)
libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf70f3000)
libgcc_s.so.1 => /home/na/df_linux/libs/./libgcc_s.so.1 (0xf70d8000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf6f22000)
libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf6f05000)
libasound.so.2 => /usr/lib/i386-linux-gnu/libasound.so.2 (0xf6dee000)
libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf6de9000)
...
Post by: LordBaal on June 27, 2016, 10:18:45 am
Windows 10 Pro 64 bit.
Core i5-4570 (Runs at 3.20 GHz).
8GB DD3 @ 798MHz (11-11-11-28).
Trying the 64 test. A world of the biggest size and 1050 years of history gets made under the hour. It peaks at 1.4 GB of memory.
Embarking on the biggest size possible (I think is 16x16). It loads in about five minutes. The game peaks a little over 2GB of memory and runs just fine (albeit slow but that's another issue). Played during the first season, the memory never got higher than 2.1 GB. Even sometimes dipped to 1.8.
Post by: supertango on June 27, 2016, 11:23:01 am
The only issue I'm having is that the game no longer seems to care what i set in the init for child cap despite seasonal save and other features working just fine, not sure if thats intentional or a bug
Post by: Toady One on June 27, 2016, 12:24:55 pm
That's right... ldd. I guess I'm just going to forget and have to be retold everything every six years or however long it has been. I had some notes, but they only go so far.
edit:
Here's an initial 64-bit test for Linux
64bit linux test (http://www.bay12games.com/dwarves/redist/df_64test_linux.tar.bz2)
I put in the usual libraries.
Post by: ragundo on June 27, 2016, 02:15:45 pm
Error when starting the file
./libs/Dwarf_Fortress: error while loading shared libraries: libSDL_image-1.2.so.0: cannot open shared object file: No such file or directory
Running ldd Dwarf_Fortress | grep SDL on the DF executable gives
libSDL-1.2.so.0 => /usr/lib/x86_64-linux-gnu/libSDL-1.2.so.0 (0x00007f0bc840d000)
libSDL_image-1.2.so.0 => not found
libSDL_ttf-2.0.so.0 => not found
Running latest Ubuntu 64bit. 32 bit works fine.
Greetings
EDIT:
sudo apt-get install libsdl-ttf2.0
sudo apt-get install libsdl-image1.2
and now works ok
Post by: majiin on June 27, 2016, 03:23:24 pm
There was however an error about missing OpenAL library and a popup window with broken icon which triggered another error about CXXABI:
Code: [Select]
Dynamically loading the OpenAL library failed, disabling sound
(Dwarf_Fortress:1537): Gtk-WARNING **: Error loading theme icon 'dialog-error' for stock: Unable to load image-loading module: /usr/lib64/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-svg.so: /home/majiin/Downloads/df_linux/libs/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /usr/lib64/libicuuc.so.57)
Initializing OpenAL failed, no sound will be played
The CXXABI error is caused by different GCC versions (my system is compiled with GCC 5.3.0). But despite those errors game worked fine, and after installing OpenAL both errors are gone.Oh and the game still works even if I delete df_linux/libs/libstdc++.so.6 and libgcc_s.so.1 (same as with 32 bit DF)
Post by: Toady One on June 27, 2016, 03:56:41 pm
---
For Mac, I'm not sure where to start. There doesn't seem to be a reference to the number of bits in the script (linux had various -m32 and pentium arch stuff in place), just a minimum architecture version. Do I just add a -m64 and watch it break, or is that not even a valid option here? Also, gcc --version gives
i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1
from 2005. Not sure what to do with that. Too old for hope?
Post by: Veroule on June 27, 2016, 04:03:35 pm
-------------------------
@Thief^
Quote
no appreciable speed difference in performance by using a smaller integer size than the CPU's bitnessSure it is only a few clock cycles. A few clock cycles times 200 billion operations is very noticeable. I always advocate learning and applying best practices and making them habit. Note that I suggested it for counter variables that are likely to be short lived registers instead of stored in RAM, which is where the greatest speed benefit is found.
Quote
As for "the move to 128bit is not too far in the future"Time will tell. I have been through transitions all the way from 8bit processors. The 8086 was a huge change in computers not only in instruction set but 16bit. It only took a decade (8086@1978 to 80386@1986) to move to 32bit. Sure memory capability was the strongest driving factor in those increases for data width. On current machines at 64bits the memory capacity is not likely to be exceeded for a long time, but as you noted we already find 64 bits of computational capacity to be limiting. I know what it is like to play with calculations on 8192 bit numbers, I wrote those for myself 20 years ago. There are many real world applications that will require 1024bit processors long before we need more addressable RAM than a 64bit number. It is very short-sighted to think that only the need for memory will drive computational power.
Quote
Use the fixed-size types (like int32_t) for pretty much everything.Thanks for the backup. Toady One stated that he has been employing specifically sized data elements for a long time and I might say that is in part due to my advocacy of doing so many years ago when the SDL port was done. If the portions of code that were taken over during that port are representative of Toady One's patterns prior to the port, then he used long and int randomly and the only size specific he used was char.
Post by: Miuramir on June 27, 2016, 04:18:27 pm
* If you are using Ubuntu, I strongly recommend a LTS (Long Term Support) version, usually either the current or previous version. These are supported for five years. Non-LTS releases are only supported for 9 months, and are not generally suitable for production use. 14.04 LTS (Trusty Tahr) is supported through April 2019, and 16.04 LTS (Xenial Xerus) is supported through April 2021. 16.04 has only been out a few months, though, and there may not be as much useful info out there for it; 14.04 is probably the safest choice.
* Linux packages come in three, not two, general levels of relevant processor support: i386, i686, and amd64 (aka x86_64). For some packages & libraries, it is possible to have two or even all three loaded; for others it's a mess. My general suggestion, and the general trend elsewhere, is to have the base system on pure 64; and some sort of sub-system (Virtual Machine, chroot setup, docker container, etc.) to handle 32 bit compilation; although some still prefer to just have a completely separate entirely-32-bit system (virtual or physical) for that. If your needs are simple you may be able to get away with a multi-library single system and compiler flags, though.
* Try really hard to avoid building packages yourself these days. yum or apt-get are really your friend; and if you stick to a standard set of repositories the odds that things will "just work" go up.
* Darwin, the Mac underlying environment and particularly compiler, is careening further away from BSD, Linux, gcc and the rest of the world with each release. Where I work, we've had to largely give up on it for compiling; we use HomeBrew (http://brew.sh/) to set up a full gcc chain and work with that. (Admittedly, however, the software we're using doesn't need to do much if any Mac-specific stuff.) Anecdotally, for instance, gfortran doesn't even remotely work on a modern stock (Darwin) Mac, is a hassle to get set up under fink, and "just works" under Brew once you install the gcc chain and tools.
* XCode has gotten very picky about only installing more recent versions on more recent OS X versions. Generally, you're OK for about one to two versions back, but rarely more these days. This may not be an issue for DF currently.
* More of my experience is on RHEL-based Linux than Debian-based Linux; if you end up dabbling on that side, a current version of RHEL / CentOS / Scientific Linux 6.x is probably easier to get running on older hardware than the newer 7.x. Conversely, if you end up with a shiny new machine, supporting all the modern boot stuff may require 7.x.
I've unfortunately got some deadlines for the end of the month, but if things still need testing in July I'll try and find the time to spin up some VMs and compare things.
Post by: Miuramir on June 27, 2016, 04:21:45 pm
For Mac, I'm not sure where to start. There doesn't seem to be a reference to the number of bits in the script (linux had various -m32 and pentium arch stuff in place), just a minimum architecture version. Do I just add a -m64 and watch it break, or is that not even a valid option here? Also, gcc --version gives
i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1
from 2005. Not sure what to do with that. Too old for hope?
See comments above; at work we've had to give up on Darwin and use Brew to install a full gcc toolchain. I am not optimistic about your odds on the old mac, old OS, and old compiler.
Post by: mifki on June 27, 2016, 04:32:56 pm
i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1
What does it say if you try to use -m32/-m64 flags with a simple source? I don't know what archs were included back then.
If you have an Apple dev account (free), you can install XCode 3.1.4 (latest that supports 10.5) from https://developer.apple.com/download/more/ and use it.
Or, you can install Homebrew http://brew.sh and use it to install any version of GCC. You can try it first, but as lethosor pointed out, it doesn't officially support 10.5. I'm pretty sure it will work, however it may require having XCode installed first anyway to build GCC.
Post by: lethosor on June 27, 2016, 04:44:00 pm
Post by: mifki on June 27, 2016, 04:49:20 pm
Well, you're definitely not using that in 0.43.03. Try gcc-4.5?
And do -v, not --version.
Post by: majiin on June 27, 2016, 05:20:25 pm
Are there more libraries I should include/not include? I don't know what the best set is for 64 bit Linux.I think the standard libraries are ok, no need to change what was working for 32bit build for such a long time. There are so many Linux distros out there, even if I can safely remove df_linux/libs/libstdc++.so.6 and libgcc_s.so.1 there is no guarantee it will work for others. Maybe users of more popular distros like Debian or Ubuntu should voice their opinion.
I was just surprised that 64bit Linux DF complains about missing OpenAL while 32bit DF don't (OpenAL 32/64 was not installed at the time of testing)
Post by: Thief^ on June 27, 2016, 05:28:21 pm
It would be good advice, if there was actually a multiple clock cycle difference in ops by register width. But there is not. Most integer ops are single cycle these days, regardless of operand size.Quote from: Thief^no appreciable speed difference in performance by using a smaller integer size than the CPU's bitnessSure it is only a few clock cycles. A few clock cycles times 200 billion operations is very noticeable. I always advocate learning and applying best practices and making them habit. Note that I suggested it for counter variables that are likely to be short lived registers instead of stored in RAM, which is where the greatest speed benefit is found.
The only speed difference I know of is with array indexing, where the operand has to be widened to the width of the pointer before indexing instructions can be used. But:
1: that's a single cycle op
2: If there's a load or move instruction, it can be included at 0 additional cost
3: The optimiser is capable of automatically widening loop counters, anyway.
Post by: Toady One on June 27, 2016, 05:53:25 pm
g++ -o src/g_src/enabler_input.o -c -std=gnu++0x -Flibs -Ilibs/SDL.framework/Headers -Dunix -DGLEW_STATIC -O2 -fomit-frame-pointer -s -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -dead_strip -Ifmodexinclude -Iglext src/g_src/enabler_input.cpp
scons: done building targets.
Build complete, DF in df_osx.tar.bz2
and me immediately after
g++ -v
Using built-in specs.
Target: i686-apple-darwin9
Configured with: /var/tmp/gcc/gcc-5490~1/src/configure --disable-checking -enable-werror --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.0/ --with-gxx-include-dir=/include/c++/4.0.0 --with-slibdir=/usr/lib --build=i686-apple-darwin9 --with-arch=apple --with-tune=generic --host=i686-apple-darwin9 --target=i686-apple-darwin9
Thread model: posix
gcc version 4.0.1 (Apple Inc. build 5490)
Post by: mifki on June 27, 2016, 05:57:54 pm
Target: i686-apple-darwin9
Means it doesn't have 64bit support. So you'll need to install something newer (if it turns out you're indeed using this gcc).
Post by: lethosor on June 27, 2016, 06:05:09 pm
Code: [Select]
/Users/tarnadams/gcc/gcc-4.5.1/libstdc++-v3/libsupc++/cxxabi.h
Removing that libstdc++ causes DF to fail to start, which means you're using something newer than GCC 4.2 (which is my system's default). (Basically, you need a libstdc++ that corresponds to the GCC version used to compile DF, or a newer libstdc++ - if you were using GCC 4.0, there would've been no need to distribute libstdc++ in the first place.)Is there a g++-4.5 or g++45 in your path? (Try running e.g. "g++-4.5 --version", or typing "g++" in a terminal window and pressing tab twice for a list of completions.) It's possible that scons is trying to save space by displaying "g++" instead of the actual executable name, or that there's a newer g++ somewhere else (which should show up with "which -a g++").
If not, try looking in /Users/tarnadams/gcc/gcc-4.5.1/ (maybe /Users/tarnadams/gcc/gcc-4.5.1/bin?) for a g++ executable.
Post by: Toady One on June 27, 2016, 06:17:34 pm
So there might be additional hope.
Post by: mifki on June 27, 2016, 06:19:34 pm
So there might be additional hope.
Anyway, if -v says Target: x86_64 then you're good to go, if not then not.
Post by: lethosor on June 27, 2016, 06:21:27 pm
Edit: or "-v" instead of "--version". And compiling a test file with -m64 is probably the most definitive way of determining whether it's 64-bit capable, anyway.
Post by: SilverSynch on June 27, 2016, 07:01:12 pm
I think the standard libraries are ok, no need to change what was working for 32bit build for such a long time. There are so many Linux distros out there, even if I can safely remove df_linux/libs/libstdc++.so.6 and libgcc_s.so.1 there is no guarantee it will work for others. Maybe users of more popular distros like Debian or Ubuntu should voice their opinion.This is actually incorrect, having new, compatibility breaking versions of libraries under the same name as previous versions is extremely bad form in Linux (which does result in cluttered managers with lots of packages under similar names). Symlink libblargh.so.1 will always link to a library under version 1, libblargh.so.2 will always link to a library under the new, compatibility breaking version 2.
Even then, if any Linux distro actually goes against standard library names, they would have quite the sore user-and-devbase on their hands, because developers do NOT like having to make special versions for each distro (they will make special packages, but that's for a another reason entirely).
Post by: Max™ on June 27, 2016, 07:09:25 pm
Code: [Select]
(Dwarf_Fortress:23514): Gtk-WARNING **: Unable to locate theme engine in module_path: "adwaita",
(Dwarf_Fortress:23514): Gtk-WARNING **: Unable to locate theme engine in module_path: "adwaita",
Didn't hurt anything though, just the message. I'll test more later, gonna rest a bit.
Post by: jecowa on June 27, 2016, 07:21:38 pm
Target: i686-apple-darwin9
Means it doesn't have 64bit support. So you'll need to install something newer (if it turns out you're indeed using this gcc).
Right now Toady is on Mac OS 10.5 Leopard. I think 10.6 Snow Leopard is the first version of Mac OS X with full 64-bit support. (Snow Leopard has a 64-bit kernel.) It's also the newest version I would recommend running on a computer with less than 4 GB of RAM (i.e. Toady's Mac mini). Snow Leopard is the ultimate version for older machines. And as Miuramir indicated, running an older version of OS X is needed in order to support players with older versions of OS X.
Post by: mifki on June 27, 2016, 07:25:05 pm
Right now Toady is on Mac OS 10.5 Leopard. I think 10.6 Snow Leopard is the first version of Mac OS X with full 64-bit support. (Snow Leopard has a 64-bit kernel.) It's also the newest version I would recommend running on a computer with less than 4 GB of RAM (i.e. Toady's Mac mini). Snow Leopard is the ultimate version for older machines.
On OS X, running 64bit kernel != being able to run 64bit apps. 10.5 has 64bit libraries.
But I agree, 10.5 is a transitional version, and 10.6 would be the right choice.
And as Miuramir indicated, running an older version of OS X is needed in order to support players with older versions of OS X.
Not true.
Post by: SilverSynch on June 27, 2016, 07:31:59 pm
Worked fine here on Arch, though I got something which I think was a Kwin error.That's just GTK crying about the Adwaita theme missing. It's supposed to be a default theme though, so I wonder how you did that.Code: [Select](Dwarf_Fortress:23514): Gtk-WARNING **: Unable to locate theme engine in module_path: "adwaita",Didn't hurt anything though, just the message. I'll test more later, gonna rest a bit.
(Dwarf_Fortress:23514): Gtk-WARNING **: Unable to locate theme engine in module_path: "adwaita",
Post by: lethosor on June 27, 2016, 07:59:07 pm
Post by: jecowa on June 27, 2016, 08:07:16 pm
It's worth noting that Toady would have to pay for 10.6. A quick search for "macmini2,1" turns up two models, both with 64-bit CPUs and only officially capable of running up to OS X 10.7, so assuming that's accurate, I don't see much need for Toady to upgrade if his current system is capable.
I'm sorry, I misunderstood and thought it was only compiling 32-bit versions. Mac OS 10.5 Leopard is fine if it works.
Post by: Isaacc7 on June 27, 2016, 08:13:10 pm
It's worth noting that Toady would have to pay for 10.6. A quick search for "macmini2,1" turns up two models, both with 64-bit CPUs and only officially capable of running up to OS X 10.7, so assuming that's accurate, I don't see much need for Toady to upgrade if his current system is capable.
Just FYI, the 10.6 upgrade is $20. Only available on DVD.
Post by: Max™ on June 27, 2016, 08:16:43 pm
That's just GTK crying about the Adwaita theme missing. It's supposed to be a default theme though, so I wonder how you did that.It's a weird old arch install that I had to rebuild due to the transfer from KDE workspace to Plasma Next, so it's working but there's stuff like that which just isn't in there. A lot of the paths are mid-transition too, I think nvidia-settings throws up similar errors actually. Nothing serious.
Post by: lethosor on June 27, 2016, 08:33:13 pm
The question is whether the compiler can compile 64-bit versions (we haven't heard back).It's worth noting that Toady would have to pay for 10.6. A quick search for "macmini2,1" turns up two models, both with 64-bit CPUs and only officially capable of running up to OS X 10.7, so assuming that's accurate, I don't see much need for Toady to upgrade if his current system is capable.
I'm sorry, I misunderstood and thought it was only compiling 32-bit versions. Mac OS 10.5 Leopard is fine if it works.
True, it's not bad, but I don't think Toady wants to wait for a DVD to show up (and who knows where you could get one in person these days). It is an option, though, if 10.5 doesn't work out for some reason.It's worth noting that Toady would have to pay for 10.6. A quick search for "macmini2,1" turns up two models, both with 64-bit CPUs and only officially capable of running up to OS X 10.7, so assuming that's accurate, I don't see much need for Toady to upgrade if his current system is capable.
Just FYI, the 10.6 upgrade is $20. Only available on DVD.
Post by: SatelliteOfLove on June 27, 2016, 11:05:44 pm
OpenSUSE Tumbleweed x86_64, FWIW
Post by: mifki on June 27, 2016, 11:12:02 pm
I noticed when I attempt to launch the 64bit Linux test build I am still prompted to install 32-bit libraries (libSDL_image-1.2.so.0, for instance, is seen as missing).
It probably prompted to install 64bit versions of these libraries, not 32bit.
Post by: Max™ on June 27, 2016, 11:52:33 pm
Code: [Select]
32 bit started: 11:21:30, hit stop 11:24:30, finished 11:24:45 on year 184.
32 bit pops:
>56349 Dwarves
>30427 Humans
>48514 Elves
>11572 Goblins
>3913 Kobolds
>Total: 150775
64 bit started: 11:27:30, hit stop 11:30:30, finished 11:30:36 on year 186.
64 bit pops:
>59593 Dwarves
>33106 Humans
>42689 Elves
>12330 Goblins
>10884 Kobolds
>Total: 158602Map comparo .gif:
Spoiler (click to show/hide)
Post by: Toady One on June 28, 2016, 01:30:18 am
Okay, that's probably where the build in ~/gcc/gcc-4.5.1 installed to. What does "/usr/local/bin/g++ --version" give you?
Yeah, that's the 4.5.1 one. I'm running it through with -m64 now. It seems to recognize it (it gave an error for -m22, anyway), so I'm on to DF itself now. I assume we'll have various issues with the libraries and so forth, if it is actually doing 64 and not just messing with me.
Post by: eternaleye on June 28, 2016, 01:35:24 am
Linux 64 linked, but I get a compression error trying to run it. Wrong zlib maybe? Not sure how to check which one it is using. Or I have some 64 bit problem with the file loading code itself that bothers linux and not windows.Are you saving and loading "long" variables anywhere? If so, that's going to cause major problems on Linux and Mac, because while "long" is 32 bits wide on Windows x86_64, it is 64 bits wide on Linux/OSX x86_64. You might need to go through and change them all to "int".
I'd actually suggest using "#include <cstdint>" and gaining the "sized" integer types - std::{u,}int{8,16,32,64,ptr}_t. Those will always be a specific size. If you need to cast a pointer to an integer, the _only_ types the C/C++ standards guarantee anything about are intptr_t and uintptr_t, so there's that as well.
Various codebases - such as the Linux kernel - use explicitly sized integers, though often under typedefs such as {u,i}{8,16,32,64} rather than the stdint.h/cstdint names, for the sake of concision.
(Well, they also guarantee that sizeof(long long) is at-least sizeof(void*), but they notably do _not_ specify that casts from the latter to the former are permitted, while they do specify that for the {u,}intptr_t types.)
Post by: Random_Dragon on June 28, 2016, 01:37:27 am
Post by: Veroule on June 28, 2016, 01:57:19 am
Quote
It would be good advice, if there was actually a multiple clock cycle difference in ops by register width. But there is not. Most integer ops are single cycle these days, regardless of operand size.I just checked the latest version at http://www.agner.org/optimize/instruction_tables.pdf and am happy to see that only some old AMD models have that backwards speed hit, and then only on MOV. Of course that makes me feel really old. It is nice to be able to intentionally forget things because the knowledge is obsolete. Thanks.
Post by: majiin on June 28, 2016, 02:47:16 am
I was just surprised that 64bit Linux DF complains about missing OpenAL while 32bit DF don't (OpenAL 32/64 was not installed at the time of testing)
Uh, I was using DF 43.03, the 43.04 32bit complains about missing OpenAL too, just like 64bit version. I guess it's the new build setup thing.
Post by: Toady One on June 28, 2016, 03:19:44 am
edit: found where 10.4 was in the script, but updating to 10.5 I'm getting errors like "string.h" not found, so I guess there's more updating to do tomorrow!
Post by: Max™ on June 28, 2016, 03:38:49 am
Post by: txtsd on June 28, 2016, 06:07:39 am
Code: [Select]
┌[txtsd@dungeon-of-data]─[~/Downloads/df_linux] [16-06-28 16:28:29]
└─▶ ldd libs/Dwarf_Fortress
linux-vdso.so.1 (0x00007ffe47bb6000)
libSDL-1.2.so.0 => /usr/lib/libSDL-1.2.so.0 (0x00007fd5881a1000)
libgraphics.so => /home/txtsd/Downloads/df_linux/libs/libgraphics.so (0x00007fd587b64000)
libstdc++.so.6 => /home/txtsd/Downloads/df_linux/libs/libstdc++.so.6 (0x00007fd587860000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007fd58755c000)
libgcc_s.so.1 => /home/txtsd/Downloads/df_linux/libs/libgcc_s.so.1 (0x00007fd587346000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007fd586fa5000)
libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fd586da1000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fd586b84000)
libgtk-x11-2.0.so.0 => /usr/lib/libgtk-x11-2.0.so.0 (0x00007fd586542000)
libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0x00007fd5862f0000)
libSDL_image-1.2.so.0 => /usr/lib/libSDL_image-1.2.so.0 (0x00007fd5860d3000)
libGLU.so.1 => /usr/lib/libGLU.so.1 (0x00007fd585e53000)
libz.so.1 => /usr/lib/libz.so.1 (0x00007fd585c3d000)
libSDL_ttf-2.0.so.0 => /usr/lib/libSDL_ttf-2.0.so.0 (0x00007fd585a36000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd58843a000)
libgdk-x11-2.0.so.0 => /usr/lib/libgdk-x11-2.0.so.0 (0x00007fd585780000)
libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0x00007fd58557c000)
libpangocairo-1.0.so.0 => /usr/lib/libpangocairo-1.0.so.0 (0x00007fd58536f000)
libX11.so.6 => /usr/lib/libX11.so.6 (0x00007fd58502d000)
libXfixes.so.3 => /usr/lib/libXfixes.so.3 (0x00007fd584e27000)
libatk-1.0.so.0 => /usr/lib/libatk-1.0.so.0 (0x00007fd584c01000)
libcairo.so.2 => /usr/lib/libcairo.so.2 (0x00007fd5848d3000)
libgdk_pixbuf-2.0.so.0 => /usr/lib/libgdk_pixbuf-2.0.so.0 (0x00007fd5846ad000)
libgio-2.0.so.0 => /usr/lib/libgio-2.0.so.0 (0x00007fd584327000)
libpangoft2-1.0.so.0 => /usr/lib/libpangoft2-1.0.so.0 (0x00007fd584112000)
libpango-1.0.so.0 => /usr/lib/libpango-1.0.so.0 (0x00007fd583ec6000)
libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0x00007fd583bb7000)
libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x00007fd583973000)
libffi.so.6 => /usr/lib/libffi.so.6 (0x00007fd58376a000)
libGL.so.1 => /usr/lib/libGL.so.1 (0x00007fd5834f9000)
libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x00007fd58322e000)
libbz2.so.1.0 => /usr/lib/libbz2.so.1.0 (0x00007fd58301e000)
libpng16.so.16 => /usr/lib/libpng16.so.16 (0x00007fd582de8000)
libharfbuzz.so.0 => /usr/lib/libharfbuzz.so.0 (0x00007fd582b6c000)
libXrender.so.1 => /usr/lib/libXrender.so.1 (0x00007fd582962000)
libXinerama.so.1 => /usr/lib/libXinerama.so.1 (0x00007fd58275f000)
libXi.so.6 => /usr/lib/libXi.so.6 (0x00007fd58254f000)
libXrandr.so.2 => /usr/lib/libXrandr.so.2 (0x00007fd582344000)
libXcursor.so.1 => /usr/lib/libXcursor.so.1 (0x00007fd582139000)
libXcomposite.so.1 => /usr/lib/libXcomposite.so.1 (0x00007fd581f36000)
libXdamage.so.1 => /usr/lib/libXdamage.so.1 (0x00007fd581d33000)
libXext.so.6 => /usr/lib/libXext.so.6 (0x00007fd581b21000)
libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007fd5818f8000)
libpixman-1.so.0 => /usr/lib/libpixman-1.so.0 (0x00007fd581650000)
libEGL.so.1 => /usr/lib/libEGL.so.1 (0x00007fd58141f000)
libxcb-shm.so.0 => /usr/lib/libxcb-shm.so.0 (0x00007fd58121b000)
libxcb-render.so.0 => /usr/lib/libxcb-render.so.0 (0x00007fd58100d000)
librt.so.1 => /usr/lib/librt.so.1 (0x00007fd580e05000)
libresolv.so.2 => /usr/lib/libresolv.so.2 (0x00007fd580bee000)
libthai.so.0 => /usr/lib/libthai.so.0 (0x00007fd5809e5000)
libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007fd580772000)
libexpat.so.1 => /usr/lib/libexpat.so.1 (0x00007fd580546000)
libxcb-dri3.so.0 => /usr/lib/libxcb-dri3.so.0 (0x00007fd580343000)
libxcb-present.so.0 => /usr/lib/libxcb-present.so.0 (0x00007fd580140000)
libxcb-randr.so.0 => /usr/lib/libxcb-randr.so.0 (0x00007fd57ff30000)
libxcb-xfixes.so.0 => /usr/lib/libxcb-xfixes.so.0 (0x00007fd57fd28000)
libxcb-shape.so.0 => /usr/lib/libxcb-shape.so.0 (0x00007fd57fb24000)
libxcb-sync.so.1 => /usr/lib/libxcb-sync.so.1 (0x00007fd57f91d000)
libxshmfence.so.1 => /usr/lib/libxshmfence.so.1 (0x00007fd57f71a000)
libglapi.so.0 => /usr/lib/libglapi.so.0 (0x00007fd57f4ec000)
libX11-xcb.so.1 => /usr/lib/libX11-xcb.so.1 (0x00007fd57f2ea000)
libxcb-glx.so.0 => /usr/lib/libxcb-glx.so.0 (0x00007fd57f0ce000)
libxcb-dri2.so.0 => /usr/lib/libxcb-dri2.so.0 (0x00007fd57eec9000)
libXxf86vm.so.1 => /usr/lib/libXxf86vm.so.1 (0x00007fd57ecc3000)
libdrm.so.2 => /usr/lib/libdrm.so.2 (0x00007fd57eab4000)
libgraphite2.so.3 => /usr/lib/libgraphite2.so.3 (0x00007fd57e888000)
libXau.so.6 => /usr/lib/libXau.so.6 (0x00007fd57e684000)
libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007fd57e47e000)
libgbm.so.1 => /usr/lib/libgbm.so.1 (0x00007fd57e270000)
libwayland-client.so.0 => /usr/lib/libwayland-client.so.0 (0x00007fd57e061000)
libwayland-server.so.0 => /usr/lib/libwayland-server.so.0 (0x00007fd57de4f000)
libdatrie.so.1 => /usr/lib/libdatrie.so.1 (0x00007fd57dc47000)
Although it still uses 32-bit OpenAL
Code: [Select]
┌[txtsd@dungeon-of-data]─[~/Downloads/df_linux] [16-06-28 16:28:42]
└─▶ ./df
Sound devices available:
OpenAL Soft
Picking OpenAL Soft. If your desired device was missing, make sure you have the appropriate 32-bit libraries installed. If you wanted a different device, configure ~/.openalrc appropriately.
Perfect OpenAL context attributes GET
Loading bindings from data/init/interface.txt
New window size: 640x300
Font size: 8x12
Resizing grid to 80x25
Resizing font to 8x12
Post by: Rumrusher on June 28, 2016, 06:18:50 am
Post by: Quietust on June 28, 2016, 06:34:07 am
Although it still uses 32-bit OpenALThe fact that that message says "32-bit" means absolutely nothing, because that's a custom message Toady added himself.Code: [Select]Picking OpenAL Soft. If your desired device was missing, make sure you have the appropriate 32-bit libraries installed. If you wanted a different device, configure ~/.openalrc appropriately.
Post by: Random_Dragon on June 28, 2016, 12:21:29 pm
Man, you're gonna go crosseyed doing all these OS updates in a row, it's great having them out of the way though.
Hell, I'm going crosseyed just watching the discussion. o3o
Post by: Toady One on June 28, 2016, 01:31:47 pm
edit: SDL seems to be resolved, so we're just down to sound and actually seeing if it runs.
Post by: txtsd on June 28, 2016, 07:48:18 pm
Although it still uses 32-bit OpenALThe fact that that message says "32-bit" means absolutely nothing, because that's a custom message Toady added himself.Code: [Select]Picking OpenAL Soft. If your desired device was missing, make sure you have the appropriate 32-bit libraries installed. If you wanted a different device, configure ~/.openalrc appropriately.
Oh good. I didn't realize it was a custom message.
Post by: LordBaal on June 28, 2016, 08:42:04 pm
I was thinking that the later years seemed to keep pace a bit better on 64 bit, but I couldn't tell if it was just my imagination or not.I'm not alone then. In my tests the latter years in world gen do not seem to be as laggy as it is on 32 bits.
Post by: lethosor on June 28, 2016, 10:01:28 pm
Cleaned up my mistakes in the Mac script... now we appear to be compiling up to the end for 10.5, and have arrived at the expected fmodex/SDL/SDL_image missing x86_64 architecture part. I don't remember why we didn't go with OpenAL originally on Mac -- is there some issue there? I don't know if we'll be able to update fmodex now. I can try to sort the SDL myself.I have OpenAL in /System/Library/Frameworks/OpenAL.framework/. Is it there on 10.5?
edit: SDL seems to be resolved, so we're just down to sound and actually seeing if it runs.
Post by: KillzEmAllGod on June 28, 2016, 11:27:20 pm
Oh wow I can move around a fortress in adventure mode without waiting 5-10 seconds to move again, this seems to be a massive improvement.I was thinking that the later years seemed to keep pace a bit better on 64 bit, but I couldn't tell if it was just my imagination or not.I'm not alone then. In my tests the latter years in world gen do not seem to be as laggy as it is on 32 bits.
Post by: Thief^ on June 29, 2016, 01:15:52 am
Basically, Ubuntu 16.04 LTS (this year's release, supported until 2021) will be the last Ubuntu to support 32-bit PCs, and 18.04 LTS (released in 2018, supported until 2023) will be the last to support 32-bit apps. By this schedule, the first Ubuntu release with no support for 32-bit apps will be 18.10, at the end of 2018. Anyone who releases any kind of Linux software has to have a 64-bit release ready for that date. People that use 32-bit only apps have until the 18.04 LTS support expires in 2023 to find alternatives, although they may find themselves wanting to upgrade sooner for other software (LTSs are often left with older software in their repositories for stability reasons).
Post by: jecowa on June 29, 2016, 01:32:25 am
I have OpenAL in /System/Library/Frameworks/OpenAL.framework/. Is it there on 10.5?
I see version 1.2 of OpenAL.framework at that location in Mac OS X 10.5.8.
Post by: Toady One on June 29, 2016, 01:42:29 am
edit: should I just cobble it together from the copy on linux or is something else more prudent? the download link I found for it did not work
Post by: mifki on June 29, 2016, 02:26:05 am
edit: should I just cobble it together from the copy on linux or is something else more prudent? the download link I found for it did not work
Are you talking about libsndfile or what?
https://github.com/erikd/libsndfile/releases
Post by: thvaz on June 29, 2016, 04:52:00 am
Post by: Cexp on June 29, 2016, 09:59:38 am
Post by: lethosor on June 29, 2016, 10:11:09 am
Post by: eternaleye on June 29, 2016, 10:56:14 am
There shouldn't be much difference between the two, so it's probably either the new compiler or the power of suggestion (or differences between worlds, bug fixes in the newer version, etc.).
Not quite. For one, the simple act of turning on 64-bit is quite possibly allowing the compiler to enable various CPU-specific optimizations it could not safely assume were available on 32-bit. In particular, every single x86_64 CPU ever made supports SSE2. That then permits a whole host of optimizations around vectorizing tight loops, which can cause very significant performance improvements.
Post by: thvaz on June 29, 2016, 11:28:55 am
How much of that is the 64-bit thing, and how much is the new compiler, though? 32-bit also runs faster.
I am comparing it with the 32-bit that also uses the new compiler. I can walk into large towns without any lag in the 64-bit version. The same save in the 32-bit version has slowdowns.
Post by: Thief^ on June 29, 2016, 11:40:37 am
There shouldn't be much difference between the two, so it's probably either the new compiler or the power of suggestion (or differences between worlds, bug fixes in the newer version, etc.).
Not quite. For one, the simple act of turning on 64-bit is quite possibly allowing the compiler to enable various CPU-specific optimizations it could not safely assume were available on 32-bit. In particular, every single x86_64 CPU ever made supports SSE2. That then permits a whole host of optimizations around vectorizing tight loops, which can cause very significant performance improvements.
Toady may well have the new compiler set to assume SSE2 in the 32-bit version too (I believe it's the default now). Anyone not got SSE2 support in their CPU?
Post by: soulsource on June 29, 2016, 12:05:26 pm
I thought this might be relevant to the 64-bit discussion: https://reddit.com/r/sysadmin/comments/4qcayr/ubuntu_developers_discuss_again_about_dropping/
Basically, Ubuntu 16.04 LTS (this year's release, supported until 2021) will be the last Ubuntu to support 32-bit PCs, and 18.04 LTS (released in 2018, supported until 2023) will be the last to support 32-bit apps. By this schedule, the first Ubuntu release with no support for 32-bit apps will be 18.10, at the end of 2018. Anyone who releases any kind of Linux software has to have a 64-bit release ready for that date. People that use 32-bit only apps have until the 18.04 LTS support expires in 2023 to find alternatives, although they may find themselves wanting to upgrade sooner for other software (LTSs are often left with older software in their repositories for stability reasons).
Ubuntu is just one Linux distribution. Others will very likely keep i686 support far longer. Also, the decision of the Ubuntu developers isn't final yet, there's a survey (https://www.gamingonlinux.com/articles/when-should-i386-support-for-ubuntu-end.7523) going on. In addition, even though 18.04 might be the last multilib release, support for 32bit applications will definitely continue through containers (Snap, Flatpak - funny enough, this means in some way going back to the solution used before multilib support was available...). Completely dropping 32bit support would break compatibility with 32bit WINE, which is required to run 32bit Windows applications (which are still the vast majority on this system).
Here's the original source, by the way: https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2016-June/016661.html
Post by: soulsource on June 29, 2016, 12:17:09 pm
Post by: Muffindrake on June 29, 2016, 12:57:30 pm
I'll upload the save here shortly.
Edit: https://mega.nz/#!gh9ygJZS!UJd4XJVzvKnpAw-pc980aON9N45-CDU3o0ZWhy2sIT0
Post by: Toady One on June 29, 2016, 02:08:36 pm
https://github.com/erikd/libsndfile/releases
As far as I can tell, I need to run autoconf configure.ac to build the configure file, but that fails. sh autogen.sh tells me I need autogen and pkg-config. At that point, trying Homebrew starts looking attractive, but I can't get that working either. brew.sh said that 10.5 needs to do something like
mkdir homebrew && curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip 1 -C homebrew
And I end up with a nice homebrew directory with bin,etc,Library,share subdirectories. I have no idea what I'm supposed to be doing from there. I gather I'm supposed to install it to /usr/local... is that just moving the directory over there? (I couldn't untar directly into there because I have no idea how to get by all the permission denials).
Post by: mifki on June 29, 2016, 02:56:44 pm
Post by: mifki on June 29, 2016, 03:06:24 pm
Latest package has libfmod.dylib, not libfmodex.dylib, I'm not sure if there's any difference apart from naming. However, if I download, for example, version 44461 from http://www.fmod.org/browse-fmod-ex-api/ , there's libfmodex.dylib for 32 and 64bit.
Actually, I'm a bit worried about using libsndfile because it depends on several other libraries to read audio formats and unless it's all linked statically (or without support for those formats), you'll need to supply all of them (not sure but there's a chance).
Post by: smariot on June 29, 2016, 04:17:25 pm
I've been running a single embark for a few hours, without reloading the game in between, using the 64-bit linux test build on Arch Linux and I've come across a memory leak. DF has been using 2.7GB of memory on a 1x1 embark with around 100 dwarves. After reloading the saved game, I had only 1.4GB memory usage.
I'll upload the save here shortly.
Edit: https://mega.nz/#!gh9ygJZS!UJd4XJVzvKnpAw-pc980aON9N45-CDU3o0ZWhy2sIT0
What makes you think it's a leak and not data fragmentation?
Post by: Toady One on June 29, 2016, 04:42:02 pm
dyld: Library not loaded: @rpath/webp.framework/Versions/A/webp
Referenced from: /Users/tarnadams/Desktop/df_osx/libs/SDL_image.framework/Versions/A/SDL_image
Reason: no suitable image found. Did find:
/Users/tarnadams/Desktop/df_osx/libs/SDL_image.framework/Versions/A/Frameworks/webp.framework/Versions/A/webp: unknown required load command 0x80000022
I had copied the new sdl/image/ttf frameworks into the libs folder as I do with the 32 version, but something seems to be broken about that.
Post by: mifki on June 29, 2016, 04:49:02 pm
Got through the Mac 64-bit link, but running df script gives:
dyld: Library not loaded: @rpath/webp.framework/Versions/A/webp
Referenced from: /Users/tarnadams/Desktop/df_osx/libs/SDL_image.framework/Versions/A/SDL_image
Reason: no suitable image found. Did find:
/Users/tarnadams/Desktop/df_osx/libs/SDL_image.framework/Versions/A/Frameworks/webp.framework/Versions/A/webp: unknown required load command 0x80000022
I had copied the new sdl/image/ttf frameworks into the libs folder as I do with the 32 version, but something seems to be broken about that.
It seems webp library is for 10.6+. Where did you get this SDL_image.framework from?
Post by: Toady One on June 29, 2016, 04:56:07 pm
There are 64-bit versions here (SDL 1.2 and versions of SDL_image and SDL_ttf that work with SDL 1.2):
https://www.libsdl.org/download-1.2.php
https://www.libsdl.org/projects/SDL_image/release-1.2.html
https://www.libsdl.org/projects/SDL_ttf/release-1.2.html
Post by: mifki on June 29, 2016, 05:02:59 pm
I grabbed it from the libsdl.org links upthread -- their page says 10.5+, but maybe that wasn't accurate? The dmgs there have webp in them.There are 64-bit versions here (SDL 1.2 and versions of SDL_image and SDL_ttf that work with SDL 1.2):
https://www.libsdl.org/download-1.2.php
https://www.libsdl.org/projects/SDL_image/release-1.2.html
https://www.libsdl.org/projects/SDL_ttf/release-1.2.html
Yeah, SDL itself there is for 10.5 and webp is for 10.6. Idiots. Ok, let me think...
Post by: mifki on June 29, 2016, 05:11:50 pm
Post by: Toady One on June 29, 2016, 05:34:37 pm
dyld: Library not loaded: @rpath/FreeType.framework/Versions/A/FreeType
Referenced from: /Users/tarnadams/Desktop/df_osx/libs/SDL_ttf.framework/Versions/A/SDL_ttf
Reason: no suitable image found. Did find:
/Users/tarnadams/Desktop/df_osx/libs/SDL_ttf.framework/Versions/A/Frameworks/FreeType.framework/Versions/A/FreeType: unknown required load command 0x80000022
Post by: jecowa on June 29, 2016, 05:39:36 pm
Try this https://www.dropbox.com/s/q6pfiziwkajjpax/SDL_image.framework.zip?dl=0 built it without webp support.
Your version's working for me in both Leopard and Snow Leopard, mifki.
edit: I'm slow.
Looks like it worked -- is there something similar with FreeType or is that something I did wrong? I installed SDL_ttf from the same place.
dyld: Library not loaded: @rpath/FreeType.framework/Versions/A/FreeType
Referenced from: /Users/tarnadams/Desktop/df_osx/libs/SDL_ttf.framework/Versions/A/SDL_ttf
Reason: no suitable image found. Did find:
/Users/tarnadams/Desktop/df_osx/libs/SDL_ttf.framework/Versions/A/Frameworks/FreeType.framework/Versions/A/FreeType: unknown required load command 0x80000022
You can fix that by using v2.0.10 of SDL_ttf instead of version 2.0.11, if that helps.
Post by: mifki on June 29, 2016, 05:40:38 pm
Post by: Toady One on June 29, 2016, 05:43:04 pm
edit: Huh, for 2.0.10, instead of complaining about FreeType, it's giving me the same error for SDL_ttf itself.
Post by: lethosor on June 29, 2016, 05:50:08 pm
What's the new message? "/Users/tarnadams/Desktop/df_osx/libs/SDL_ttf.framework/Versions/A/SDL_ttf: unknown required load command 0x80000022" or something else?
Edit: or wait for mifki's freetype build, maybe.
Post by: Toady One on June 29, 2016, 05:55:07 pm
Post by: jecowa on June 29, 2016, 05:58:34 pm
http://dffd.bay12games.com/file.php?id=12149
Post by: lethosor on June 29, 2016, 06:02:12 pm
What version of SDL are you using now, anyway? (It should be in libs/SDL.framework/Headers/SDL_version.h)
Post by: Toady One on June 29, 2016, 06:06:30 pm
SDL version is 1.2.15.
Post by: mifki on June 29, 2016, 06:14:57 pm
Post by: Toady One on June 29, 2016, 06:22:40 pm
Post by: mifki on June 29, 2016, 06:23:20 pm
Post by: lethosor on June 29, 2016, 06:26:41 pm
Now that I think of it, SDL 1.2 was old enough to be built with 10.5 support - it's just SDL2 that only supports 10.6+ in its 64-bit version.
Post by: jecowa on June 29, 2016, 06:29:33 pm
This time I get "Library not loaded: /usr/local/lib/libfreetype.6.dylib" referenced from the new SDL_ttf, Reason: image not found.
I believe the latest version of the libfreetype.6.dylib is included with the dffd link.
Post by: Toady One on June 29, 2016, 06:32:37 pm
Post by: Veroule on June 29, 2016, 06:40:09 pm
If Toady One can identify any functions that show significant differences in speed between the architectures, then he could determine the likelihood of my guess by considering the number of local variables and how they are used throughout that function. Finding a function that is gaining significantly from having more registers might suggest that further optimization could be achieved by breaking the function into smaller segments.
Post by: mifki on June 29, 2016, 06:41:51 pm
Post by: Toady One on June 29, 2016, 06:52:03 pm
dyld: Library not loaded: /usr/local/opt/libpng/lib/libpng16.16.dylib
Referenced from: /Users/tarnadams/Desktop/df_osx/libs/SDL_ttf.framework/Versions/A/Frameworks/freetype.framework/Versions/2.6.3/freetype
Reason: image not found
I have other duties calling -- back in ~6.5 hrs.
Post by: mifki on June 29, 2016, 06:58:29 pm
We have some new players entering the scene!
dyld: Library not loaded: /usr/local/opt/libpng/lib/libpng16.16.dylib
Referenced from: /Users/tarnadams/Desktop/df_osx/libs/SDL_ttf.framework/Versions/A/Frameworks/freetype.framework/Versions/2.6.3/freetype
Reason: image not found
I have other duties calling -- back in ~6.5 hrs.
https://www.dropbox.com/s/ue3g2slybjn1w3o/SDL_ttf.framework.zip?dl=0
Post by: lethosor on June 29, 2016, 07:34:34 pm
Edit: never mind, looks to have been tar's fault.
And it looks like the newer one doesn't reference libpng at all, which hopefully fixes that.
Post by: Blue_Dwarf on June 29, 2016, 11:07:12 pm
Post by: Putnam on June 29, 2016, 11:58:24 pm
64-bit version? Did Toady change his mind about making it?
No, he didn't.
Post by: Khym Chanur on June 30, 2016, 12:20:22 am
Post by: Toady One on June 30, 2016, 01:43:04 am
I'm also confused on which fmodex dylib it should be using. Is libfmodexL.dylib the 64 bit one? Because it is asking for libfmodex.dylib. Is libfmodexL something unrelated?
Post by: mifki on June 30, 2016, 02:01:05 am
Speaking of which, I'm getting that on the Mac now with the latest SDL_ttf. It tells me the png is not found, and then after a bit it tells me the application has quit unexpectedly.
I'm also confused on which fmodex dylib it should be using. Is libfmodexL.dylib the 64 bit one? Because it is asking for libfmodex.dylib. Is libfmodexL something unrelated?
It must be because of SDL_image, but I thought I've checked it for png support. I'll check again.
libfmodexL is debug, so you need just libfmodex.dylib
Post by: mifki on June 30, 2016, 02:17:29 am
I can build SDL_image with built-in png support, not using system libs, but that will take some time.
Post by: PatrikLundell on June 30, 2016, 03:02:31 am
A save from unknown version (forum problem, so it isn't my save) fails after a minute or so with an error box with a caption of "FATAL ERROR" and the message "Nemesis Unit Load Failed".
Is this of interest?
Post by: mifki on June 30, 2016, 03:17:45 am
I can build SDL_image with built-in png support, not using system libs, but that will take some time.
Ok, here's SDL_image with built-in png support. https://www.dropbox.com/s/q6pfiziwkajjpax/SDL_image.framework.zip?dl=0
Post by: Toady One on June 30, 2016, 03:51:29 am
Sleep now. We'll see what happens tomorrow.
Post by: mifki on June 30, 2016, 04:02:19 am
Hmm... now nothing works at all. This is before that SDL_image link in the previous post. It just doesn't seem to recognize the aliases in the frameworks anymore, and I can't make new ones. Compiling, I get a SDL_ttf/SDL_ttf.h not found with the latest SDL_ttf, but when I move in the old one it compiles fine. The aliases from the dropbox download are not recognized as aliases at all (it treats them like scripts for a terminal), but the ones in the old SDL_ttf are treated as aliases. But when I make new aliases (to directories like Headers), they show as 500K instead of the usual 4K, and from a terminal, I can "cd" into the old directory aliases, but the new ones I make give me "not a directory". I made my new aliases with right-click "Make Alias" -- they work when I click on them, but not in the terminal with cd. I have no idea what's going on.
Sleep now. We'll see what happens tomorrow.
To make a proper "alias" (symlink actually), you need to do in Terminal "ln -sf <target> <link_name>". The "Make Alias" command makes something GUI-specific only.
But now I'm completely lost as to what's happening. It was working (sort of) and giving you "Not found: blahblah.png", right? And then it stopped linking?
I've checked my files
https://www.dropbox.com/s/q6pfiziwkajjpax/SDL_image.framework.zip?dl=0
https://www.dropbox.com/s/ue3g2slybjn1w3o/SDL_ttf.framework.zip?dl=0
and they're fine, with all symlinks in place. Make sure to unpack them on Mac and not copy to Windows and back, otherwise symlinks will be lost. Let's try again with these two frameworks and see what happens.
PS. I sent you a PM last week and now not sure whether it was lost or just was too ridiculous to respond to.
Post by: Random_Dragon on June 30, 2016, 04:28:30 am
(https://cdn.meme.am/instances/500x/67265879.jpg)
Post by: graw-_@ on June 30, 2016, 11:16:46 am
Everything started no problems and no errors. I tried generating a large world with a medium history, high civs,sites, savagery and minerals everywhere. Terrain generation seemed really slow compared to the last release I tried (43.02). After it rejected 6 worlds after about half an hour without even getting to civ placement, I aborted and changed to a medium size world with less minerals and world gen went alright.
Adventure mode seems a little smoother I get less lag in populated areas. I played around with that before a bronze collosus kicked my ear hard enough to twist my neck and collapse it.
I tried a 16x16 embark after that. It took about 15 minutes to load the map and df was using about 4.3 gb of ram. Fps was too low to be playable but it seemed to drop and speed up again in spurts. I think maybe because of pathfinding. It seemed to drop when my dwarves finished an activity or when my miner would choose the next tile to mine.
I also noticed that after saving the world and returning to the menu dwarf fortress was still using 4.8gb of memory. It never stopped climbing.
Post by: Toady One on June 30, 2016, 02:32:46 pm
http://bay12games.com/dwarves/redist/brokenpngosx64.tar.bz2
Post by: mifki on June 30, 2016, 03:06:54 pm
Okay, I think we are back up to where we were before, and here's the requested not-working-for-me file:
http://bay12games.com/dwarves/redist/brokenpngosx64.tar.bz2
As expected, it works for me (but you included i386 libstdc++.6.dylib instead of x64, which should be taken from <gcc location>/lib/gcc/x86_64-apple-darwin14.1.0/4.5.4/libstdc++.6.dylib).
I see this build uses my latest SDL_image framework. And it's still showing png not found for you?
Post by: jecowa on June 30, 2016, 03:16:30 pm
Okay, I think we are back up to where we were before, and here's the requested not-working-for-me file:
http://bay12games.com/dwarves/redist/brokenpngosx64.tar.bz2
As expected, it works for me (but you included i386 libstdc++.6.dylib instead of x64, which should be taken from <gcc location>/lib/gcc/x86_64-apple-darwin14.1.0/4.5.4/libstdc++.6.dylib).
I see this build uses my latest SDL_image framework. And it's still showing png not found for you?
I get something like that on Snow Leopard (after replacing the included libstdc++.6.dylib with the one lethosor sent me). First this alert box comes up:
Quote
Tileset not found
Not found: data/art/curses_640_300.png
Then problem reporter comes up with this:
Spoiler (click to show/hide)
Post by: mifki on June 30, 2016, 03:20:20 pm
Tileset not found
That's very strange. I have no idea why.
Ok, I have a 10.5 retail disk and a spare iMac. If it still works, which I'm not sure, I'll try to build SDL_* libs that would work on it.
Post by: Toady One on June 30, 2016, 03:28:47 pm
I couldn't find that folder for libstdc++... I think my gcc is the one in "/usr/local/bin", and it definitely took the library "/usr/local/lib/libstdc++.6.dylib". I did find "/usr/local/lib/x86_64/libstdc++.6.dylib", if that works.
Post by: mifki on June 30, 2016, 03:35:39 pm
Yeah, running the df script from a terminal, I get the helpful message "libpng error: bad parameters to zlib" before the Tileset window pops up.
I couldn't find that folder for libstdc++... I think my gcc is the one in "/usr/local/bin", and it definitely took the library "/usr/local/lib/libstdc++.6.dylib". I did find "/usr/local/lib/x86_64/libstdc++.6.dylib", if that works.
Oh, zlib! That helps.
Yes need to take libstdc++ from x86_64.
Post by: mifki on June 30, 2016, 04:06:11 pm
Yeah, running the df script from a terminal, I get the helpful message "libpng error: bad parameters to zlib" before the Tileset window pops up.
(Or jecowa) Try this https://www.dropbox.com/s/g20shl0svc39qhm/SDL_image.framework.zip?dl=0 I've linked it with the latest zlib statically.
Post by: Toady One on June 30, 2016, 04:20:06 pm
http://www.bay12games.com/dwarves/redist/df_unexpected_osx.tar.bz2
Post by: mifki on June 30, 2016, 04:34:35 pm
Now there isn't a libpng or Tileset message, but it still tells me the application has quit unexpectedly.
http://www.bay12games.com/dwarves/redist/df_unexpected_osx.tar.bz2
Anything useful in the crash log? (You may need to press Report Crash or something to show the log, I don't what exactly on old systems).
Post by: Toady One on June 30, 2016, 04:42:42 pm
Spoiler (click to show/hide)
(
/System/Library/PrivateFrameworks/InstallServer.framework/Versions/A/InstallServerPost by: mifki on June 30, 2016, 04:44:20 pm
EDIT: yes, that's it
Post by: Max™ on June 30, 2016, 04:48:55 pm
Post by: mifki on June 30, 2016, 05:05:21 pm
Here's the report:
Try this https://www.dropbox.com/s/iufrel9qvnfy40u/SDL_image.framework.zip?dl=0
You don't need to recompile, just put the new framework in libs.
Post by: Toady One on June 30, 2016, 05:11:19 pm
Spoiler (click to show/hide)
Post by: mifki on June 30, 2016, 05:18:01 pm
It looks a bit different:
Yep, I'm recompiling everything else now with support for old cpus.
EDIT: Here https://www.dropbox.com/sh/t2hlhpaf0hi2vqb/AAB-AMVz3vmlTqTpPh3pVZPva?dl=0
Post by: Toady One on June 30, 2016, 05:31:42 pm
Spoiler (click to show/hide)
Post by: mifki on June 30, 2016, 05:41:35 pm
New report:
Oh, fmod is more difficult. Can you try older versions from http://www.fmod.org/browse-fmod-ex-api/ ? They all have 64bit support. Not sure what they're doing that causes crash on old system though.
Now you'll get an impression development for OS X is a pain. Which is not except in some rare cases...
Post by: Toady One on June 30, 2016, 06:00:39 pm
Post by: mifki on June 30, 2016, 06:02:38 pm
Alright, I'm trying 44221. That looks like the end of a sequence from about halfway down the list. Not sure if I should go back further. Compile will be done in ~1.5 hrs.
Don't recompile, it's a dynamic library! I just copied even the earliest 43405 to libs and it works for me, so you can just do the same.
Post by: Toady One on June 30, 2016, 06:08:24 pm
Post by: mifki on June 30, 2016, 06:33:58 pm
I wasn't sure if they had changed the headers. Anyway, too late now!
If it still doesn't work try just copying the oldest fmodex then.
Post by: Toady One on June 30, 2016, 07:12:52 pm
http://www.bay12games.com/dwarves/redist/df_64_test_osx.tar.bz2
I have various duties and so forth to catch up on now (including future of the fortress and your PM if I get there, I'll get there!). Then if this is still working by the time I get back, we'll have to decide on a course of action for Linux 32 bits.
Post by: jecowa on June 30, 2016, 07:15:28 pm
Post by: mifki on June 30, 2016, 07:17:33 pm
44221 worked for me -- and it ran with sound! Here's the official test release for 64-bit OSX:
http://www.bay12games.com/dwarves/redist/df_64_test_osx.tar.bz2
I have various duties and so forth to catch up on now (including future of the fortress and your PM if I get there, I'll get there!). Then if this is still working by the time I get back, we'll have to decide on a course of action for Linux 32 bits.
Hooray!
Post by: lethosor on June 30, 2016, 08:01:53 pm
Awesome! It's using 6 to 7 threads!0.43.03 uses around 10 for me - most of those are SDL or sound-related. (Probably not a secret threading addition, sorry.)
Post by: jecowa on June 30, 2016, 08:10:52 pm
Post by: jecowa on June 30, 2016, 10:03:44 pm
Post by: mifki on July 01, 2016, 06:36:29 pm
44221 worked for me -- and it ran with sound! Here's the official test release for 64-bit OSX:
http://www.bay12games.com/dwarves/redist/df_64_test_osx.tar.bz2
I have various duties and so forth to catch up on now (including future of the fortress and your PM if I get there, I'll get there!). Then if this is still working by the time I get back, we'll have to decide on a course of action for Linux 32 bits.
FYI, I personally didn't do much testing, but didn't encounter any crashes either so far.
Post by: peasant cretin on July 02, 2016, 07:01:49 am
Post by: soulsource on July 02, 2016, 08:13:25 am
I hope nobody has written this yet, I've just skimmed through the thread...
You definitely need just one version of the -dev packages, as those contain architecture independent headers and should therefore be the same for amd64 and i386.
For getting a 32bit libsdl 1.2, you could just try to directly install libsdl-image1.2:i386 and let apt-get resolve the dependencies. (sudo apt-get install libsdl-image1.2:i386). Once the 32bit libraries are installed, the linker should™ find them automatically when linking 32bit programs (actually, it will just ignore libs of the wrong architecture and keep searching).
Post by: mifki on July 02, 2016, 08:27:05 am
You definitely need just one version of the -dev packages, as those contain architecture independent headers and should therefore be the same for amd64 and i386.
That's just not true.
Post by: lethosor on July 02, 2016, 08:33:54 am
Post by: mifki on July 02, 2016, 09:11:08 am
It's not true for everything, but I don't think SDL headers vary across architectures (which is why OS X frameworks work with just one set of headers embedded in them).
Specifically the problem Toady had was caused by libpng -dev packages for different archs conflicting. I found a bug report and the problem is that its headers contain some arch-specific code. Not sure if it's the same problem and if it's fixed in any newer Ubuntu version. But it's mostly a package management issue.
Other than that, i386/amd64 -dev packages can be installed at the same time (definitely there are separate packages), but it just may be easier to have a separate 32bit vm/docker/whatever than to resolve this and any other conflicts.
Post by: Shonai_Dweller on July 02, 2016, 06:30:36 pm
Wandering round a big city (population over 10,000), there's loads of "!" all around but they all seem to update instantly as I move about instead of pausing for several seconds every step as it used to be in 32-bit.
Also, there's no pause any more when requesting a mission from a hearthperson. Best update ever.
Post by: lethosor on July 02, 2016, 07:05:54 pm
There's really no good reason for 64-bit DF to be that much faster. Is the new 32-bit version also faster? It could have something to do with the new compiler, which also happened since 0.43.03.
Post by: Shonai_Dweller on July 02, 2016, 07:14:18 pm
Is that in the same fort?What do you mean 'same fort'? As I said I'm playing Adventurer in a massve city. Will quickly try 43.03 and 42.06 as I have both installed with worlds generated at the moment.
There's really no good reason for 64-bit DF to be that much faster. Is the new 32-bit version also faster? It could have something to do with the new compiler, which also happened since 0.43.03.
Post by: lethosor on July 02, 2016, 07:37:14 pm
Post by: Toady One on July 02, 2016, 08:30:55 pm
I looked at the chroot stuff... and it just said "then you build the kernel as usual". I never did that (installed Ubuntu from an iso), so I don't really know what to do. I'm not sure I'm Linuxy enough to accomplish any of these methods.
Post by: mifki on July 02, 2016, 09:07:11 pm
Post by: Random_Dragon on July 02, 2016, 09:10:06 pm
Burned most of the day trying to fix the worlds with the same seed that produce different results... it's hard to tell when it has been narrowed down.
And yet again, since I've had to point this out to someone else already: you DID already rule out Orographic Precipitation, right? To quote the wiki:
Quote
Turning it off should result in more controllable, less complex rainfall conditions based on rainfall parameters as it adds a random element which can distort or otherwise mess up the climates on a pregenerated map.
I can't recall if that other person confirmed whether or not they got more consistent results after disabling. All I recall is they couldn't wrap their head around WHY that would affect history gen, instead of just testing it to rule out a setting already alleged to cause that.
Post by: mifki on July 02, 2016, 09:12:13 pm
I looked at the chroot stuff... and it just said "then you build the kernel as usual". I never did that (installed Ubuntu from an iso), so I don't really know what to do. I'm not sure I'm Linuxy enough to accomplish any of these methods.
The words about kernel in that post because the question was about building the kernel. You just "build df as usual".
Post by: Toady One on July 02, 2016, 09:23:35 pm
And yet again, since I've had to point this out to someone else already: you DID already rule out Orographic Precipitation, right? To quote the wiki:QuoteTurning it off should result in more controllable, less complex rainfall conditions based on rainfall parameters as it adds a random element which can distort or otherwise mess up the climates on a pregenerated map.
That's what the orographic precipitation is supposed to do, unless something else is going on.
The words about kernel in that post because the question was about building the kernel. You just "build df as usual".
Ah, good. I was afraid I was entering a total nightmare instead of a half-nightmare. I'll try pressing the buttons and see what happens. Maybe it will just work.
Post by: Shonai_Dweller on July 02, 2016, 09:33:11 pm
I mean the same world. If the worlds aren't the same, a lot of things could be causing different performance.I don't carry over worlds from save to save so that's a hard one to test.
A city of 10,000 (star shape on map) and a dark fortress will slow down my adventurer without fail in previous versions. New version isn't slowing down (as much).
The pause as hearthpeople consider kill quests is long and visible in 42.06 and not now. Will test new 32 bit version too.
Post by: Random_Dragon on July 02, 2016, 09:45:09 pm
That's what the orographic precipitation is supposed to do, unless something else is going on.
So that's the obvious cause eliminated, I assume? Good to see at least.
Post by: Veroule on July 03, 2016, 12:04:13 am
I looked at the chroot stuff... and it just said "then you build the kernel as usual". I never did that (installed Ubuntu from an iso), so I don't really know what to do. I'm not sure I'm Linuxy enough to accomplish any of these methods.This reference might help: https://wiki.ubuntu.com/DebootstrapChroot
Post by: Khym Chanur on July 03, 2016, 02:46:43 am
Burned most of the day trying to fix the worlds with the same seed that produce different results... it's hard to tell when it has been narrowed down.
Running a worldgen on Linux under Valgrind (http://valgrind.org/) I got a lot of "Conditional jump or move depends on uninitialised value", so it might be an unitialized memory problem. If you compile on Linux with the flags set to generate debugging info then you can get Valgrind to give you function names, file names and line numbers where it happens. Even if those aren't the cause of the bug you're looking at you should fix them anyways, since they might start causing bugs in the future.
Post by: FantasticDorf on July 03, 2016, 07:13:52 am
Burned most of the day trying to fix the worlds with the same seed that produce different results... it's hard to tell when it has been narrowed down. I turned off trade and it went away, but that doesn't mean trade is responsible -- it could be something in how the goods are produced, or how the resource types are decided, or the populations creating the goods (those all seem not to be the cause, but it only happens 1 out of 4 times once trade is off, so it's difficult to pick up the right problem). And so on. It's hard to nail these down, but hopefully I can find the root cause and sort it out. Presumably there's some difference in initial conditions, some outside factor (like a clock call), or some out-of-bounds situation somewhere, but I haven't found anything obvious yet.
To have a guess and just throwing out a suggestion as to what your problem might be, generating/deciding all the objects spontaneously or in some kind of order by picking out what's been sold already on the list might have something to do with it. If you've got to apply the CPU to bring these objects into being (given that wagon clutter does contribute to total fortress clutter, as per caps on seed gathering when wagons are docked up) and maintain them moving inside a 'container' (going to be a bumpy ride moving across so many tiles, given that lots of water in oceans can only bear to sit still in a stable manner) then also the memory aspect of having to allocate all those items appropriately to the relevant tags and load them in if they were sold to the trader, a quite hefty task though not exactly groundbreaking.
My suggestion to test this out, is to remove all explicit entity tags that affect trade (all the tool tags too besides from common domestic pack/pull so that trade can still come in), that should help you narrow down at-least one cause by means of deduction (as you mention it might be other factors outside that bear upon trade) and identify core industries that might cause a spanner in the works.
Meat and wood without trade agreements are relatively uncommon goods on the screen and a good place to start if entity tags are bare.
Post by: soulsource on July 03, 2016, 09:23:43 am
You definitely need just one version of the -dev packages, as those contain architecture independent headers and should therefore be the same for amd64 and i386.
That's just not true.
Ooops, sorry. I don't know how I got that mixed up. On Debian based distributions the -dev packages do indeed contain more than just headers - they also hold the static libraries, which of course are architecture dependent. But still, if one doesn't statically link, having one version of -dev packages should suffice.
Post by: mifki on July 03, 2016, 10:14:07 am
You definitely need just one version of the -dev packages, as those contain architecture independent headers and should therefore be the same for amd64 and i386.
That's just not true.
Ooops, sorry. I don't know how I got that mixed up. On Debian based distributions the -dev packages do indeed contain more than just headers - they also hold the static libraries, which of course are architecture dependent. But still, if one doesn't statically link, having one version of -dev packages should suffice.
They're putting them to the right places, or something... it won't find libs unless proper packages installed. But of course that's just a package management issue, you can get libs from anywhere else (even from the same conflicting packages), put them anywhere and configure paths - that's just not the easiest way for Toady.
Post by: Toady One on July 03, 2016, 12:32:35 pm
Running a worldgen on Linux under Valgrind (http://valgrind.org/) I got a lot of "Conditional jump or move depends on uninitialised value", so it might be an unitialized memory problem. If you compile on Linux with the flags set to generate debugging info then you can get Valgrind to give you function names, file names and line numbers where it happens. Even if those aren't the cause of the bug you're looking at you should fix them anyways, since they might start causing bugs in the future.
I should set that up again. I'm not sure it'll run fast enough without optimization, and their page says "Use of -O2 and above is not recommended as Memcheck occasionally reports uninitialised-value errors which don't really exist," but in this case... false positive probably less likely than me screwing something up.
...
It's world gen trade, unfortunately, which is completely different.
I did find one issue with dance skill application at the end, which caused the seed to jump the rails, but that happens after world gen is over when it is finalizing the sites. There's still a problem, and hopefully I can fix it today.
edit: Fixed the world gen trade problem and had a medium 100y generate the same way three times, so perhaps that's good for now. Have valgrind running -- it's pretty slow, but manageable, and it seems to be logging further points of interest, so I'll keep messing with that for a bit before trying chroot.
Post by: Max™ on July 03, 2016, 11:33:42 pm
Spoiler (click to show/hide)
Code: [Select]
test64
start: 10:38:30
stop: 10:54:01
finished: 10:54:34 (year 300)
16:04 total worldgen
64 bit pops:
>60553 Dwarves
>32485 Humans
>51199 Elves
>36119 Goblins
>5738 Kobolds
>Total: 186094Code: [Select]
test32
start: 10:56:30
target: 11:12:01 (year 286)
stop: 11:13:24
finish: 11:14:05 (year 300)
18:05 total worldgen
32 bit pops:
>54795 Dwarves
>28097 Humans
>45384 Elves
>27181 Goblins
>8280 Kobolds
>Total: 163737Wound up with 23k higher population on the 64 bit world despite finishing nearly 2 minutes earlier than the 32 bit world. The extra two minutes were all from year 286 to 300, which is yet another data point towards the later years being faster on 64 bit.
Post by: ChangelingVenom on July 04, 2016, 11:18:46 am
Did another bitness comparison with 25k pop cap instead of 15k like the first time, went to 300 years instead of running for a set time.Thanks Mat for the !!SCIENCE!! C:
...
Wound up with 23k higher population on the 64 bit world despite finishing nearly 2 minutes earlier than the 32 bit world. The extra two minutes were all from year 286 to 300, which is yet another data point towards the later years being faster on 64 bit.
Post by: int_ua on July 04, 2016, 12:00:14 pm
I looked at the chroot stuff... and it just said "then you build the kernel as usual". I never did that (installed Ubuntu from an iso), so I don't really know what to do. I'm not sure I'm Linuxy enough to accomplish any of these methods.
If you already have 64bit Ubuntu installed these commands should setup the needed chroot for you (replace trusty with precise everywhere if you want 12.04 instead of 14.04):
Code: [Select]
sudo apt-get install ubuntu-dev-tools
mk-sbuild --arch=i386 trusty # Currently, first time it needs to be executed twice, unfortunately, please comment if you know how to omit this.
mk-sbuild --arch=i386 trusty
# Allowing the same home folder to be used
echo "$HOME /home/$USER none rw,bind 0 0" | sudo tee -a /etc/schroot/sbuild/fstabThen you can use the chroot with just
Code: [Select]
schroot -c trusty-i386After that command you should be in the same home directory, but in a different ubuntu "installation", where you can install a completely different set of packages (well, in that case it will be the same set, but with different arch) and should be able to build the game with the same script you've used before. But notice that it won't have sudo by default, you'll have to manage it with another command, executed from the main system, not inside the chroot (using a different tab or TTY for that may be convenient):Code: [Select]
sudo schroot -c source:trusty-i386 -u rootIn my case the basic chroot took only 670 Mb of space:
Code: [Select]
$ sudo du -sh /var/lib/schroot/chroots/
670M /var/lib/schroot/chroots/With default bash prompt settings ($PS1) you should easily differentiate between chroot and non-chroot sessions by a prefix in parentheses:
Code: [Select]
(trusty-i386)~ $If you don't have 64bit Ubuntu installed yet, I'd recommend Lubuntu http://cdimages.ubuntu.com/lubuntu/releases/trusty/release/ (http://cdimages.ubuntu.com/lubuntu/releases/trusty/release/) in this particular case, because it's still have a desktop UI but that takes much less resources.
Post by: Toady One on July 04, 2016, 01:09:50 pm
edit: http://www.bay12games.com/dwarves/redist/df_32test_linux.tar.bz2
This doesn't run from the 64-bit linux (can't find libSDL-1.2.so.0 and many others -- due to missing i386 versions? not sure). From inside the chroot, it only runs if I switch to TEXT mode, and the audio fails. I'm not sure how much of that has to do with the configuration of the chroot vs other problems, or what I can fix.
Post by: SatelliteOfLove on July 04, 2016, 02:46:54 pm
Post by: Cexp on July 04, 2016, 03:16:25 pm
http://www.bay12games.com/dwarves/redist/df_32test_linux.tar.bz2Seems to work for me on 32-bit Ubuntu Saucy (13.10), at least as well as the previous release did in terms of needed libs (SDL and audio do work).
Post by: int_ua on July 04, 2016, 04:12:24 pm
I already followed the instructions before trusty replaced precise... hopefully that's not a huge issue.Not an issue at all for 9 months, until April 2017: https://wiki.ubuntu.com/Releases
This doesn't run from the 64-bit linux (can't find libSDL-1.2.so.0 and many others -- due to missing i386 versions? not sure). From inside the chroot, it only runs if I switch to TEXT mode, and the audio fails. I'm not sure how much of that has to do with the configuration of the chroot vs other problems, or what I can fix.It works great, including sound, world generation and embark tested on the same 64bit system with 32bit libraries I've used to play DF before. I'm sure it's the missing i386 libraries in your case. As for running in the chroot - yeah, because there's no graphical server (X server) in the chroot
Post by: Toady One on July 04, 2016, 04:34:50 pm
Post by: mifki on July 04, 2016, 05:24:48 pm
Nah, I just wanted to post what I was seeing in case it was broken. So I guess that's it for 32? I'm messing around with valgrind some more for the day, and I can pull together a giant all-versions official release for tomorrowish to see how painful it is.
Did you try code analysis tool in VS?
Post by: Khym Chanur on July 04, 2016, 05:40:48 pm
Did you try code analysis tool in VS?
There's also various third-party C static code analysis tools, like PC-Lint/FlexeLint (http://www.gimpel.com/html/lintinfo.htm), Splint (http://www.splint.org/) and Clang (http://clang-analyzer.llvm.org/).
Post by: Toady One on July 04, 2016, 05:50:03 pm
Post by: mifki on July 04, 2016, 05:58:26 pm
Did you try code analysis tool in VS?
There's also various third-party C static code analysis tools, like PC-Lint/FlexeLint (http://www.gimpel.com/html/lintinfo.htm), Splint (http://www.splint.org/) and Clang (http://clang-analyzer.llvm.org/).
Yeah, I was going to suggest other tools when Toady says VS chokes on the big DF project or something:)
However of course we need to suggest the right tools. There's no demo/trial for PC-Lint, Splint does not support C++, and there is no ready to use Clang package for Windows.
So, there's a free CppCheck, and trials for CppDepend and PVS-Studio.
edit while was typing: if the vs one worked fine, it's all good I suppose.