Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: DF duplisearch (a Python 3+2.7 script, or an .exe)  (Read 1663 times)

voliol

  • Bay Watcher
    • View Profile
DF duplisearch (a Python 3+2.7 script, or an .exe)
« on: February 19, 2020, 05:23:52 pm »

I made a quick script that checks for duplicated raws, and reports on them. It also counts the instances of each object type (CREATURE, ENTITY, etc. etc.). It should have some use when merging mods, or visualizing the size of your mods compared to vanilla (and other mods).

There's a version for Python 3, Python 2.7, and an .exe version. Use whichever you like.

If you're using one of the former two, make sure you have the right Python version, then just put it in your main DF folder (or wherever if you change the folder_path variable) and run it. The .exe version has instructions in the download, but you basically just set the path and run it.

Spoiler: Screenshot (click to show/hide)

Download links (DFFD):
For Python 3
For Python 2.7
Binary/.exe version
All versions work for DF 0.44 and 0.47, and likely most earlier versions as well.

jecowa

  • Bay Watcher
    • View Profile
Re: Duplicated raws checker (a Python 3 script)
« Reply #1 on: February 20, 2020, 12:00:57 pm »

I just tried it in Python 2.7.
Spoiler (click to show/hide)

I deleted the "encoding" part and tried again:
Spoiler (click to show/hide)

Is there anything bad about deleting the encoding="latin-1" bit?
Logged

voliol

  • Bay Watcher
    • View Profile
Re: Duplicated raws checker (a Python 3 script)
« Reply #2 on: February 21, 2020, 04:57:39 am »

Is there anything bad about deleting the encoding="latin-1" bit?

No there isn't. I originally wrote this script in Python 2.7, and then ported it to 3. I'm surprised as little has to be done to make it Python 2-compatible.

Python 2.7 actually supposes all read .txt files are in latin-1. This works for DF raws, but can cause corruption in other cases, so they changed it in Python 3. The "encoding='latin-1'" is telling it to use latin-1, despite being in Python 3.

voliol

  • Bay Watcher
    • View Profile
Re: Duplicated raws checker (a Python 3+2.7 script)
« Reply #3 on: March 14, 2020, 10:05:08 am »

Posted the Python 2.7 version as well, because why not? I should be able to make a binary version as well but I dunno about the timeframe on that.

jecowa

  • Bay Watcher
    • View Profile
Re: Duplicated raws checker (a Python 3+2.7 script)
« Reply #4 on: March 16, 2020, 01:55:58 pm »

Thanks. Just used this to problem solve some Duplicate Object errors.
Logged

voliol

  • Bay Watcher
    • View Profile
Re: Duplicated raws checker (a Python 3+2.7 script)
« Reply #5 on: March 16, 2020, 03:26:10 pm »

Np. Here's the first binary version. I'm not too used to compiling my python tbh so hopefully it works more often than not.

Enemy post

  • Bay Watcher
  • Modder/GM
    • View Profile
Re: Duplicated raws checker (a Python 3+2.7 script)(now as an .exe)
« Reply #6 on: March 31, 2020, 10:44:18 pm »

Thanks for doing this, I've been wanting something like this for a long time. This should make modding much more efficient.
Logged
My mods and forum games.
Enemy post has claimed the title of Dragonsong the Harmonic of Melodious Exaltion!

voliol

  • Bay Watcher
    • View Profile
Re: Duplicated raws checker (a Python 3+2.7 script)(now as an .exe)
« Reply #7 on: April 01, 2020, 02:10:38 am »

Iím glad you like it ^^

I just hope this doesnít lessen the use of prefixes/subfixes, because those are really a more longterm solution and ultimately better than this. That might be my hybris talking though :p

voliol

  • Bay Watcher
    • View Profile
Re: DF duplisearch (a Python 3+2.7 script, or an .exe)
« Reply #8 on: April 19, 2020, 12:20:03 pm »

Now it states which .txt files has the duplicated raws in them, whenever any are found.
Also, it now has a proper name!

voliol

  • Bay Watcher
    • View Profile
Re: DF duplisearch (a Python 3+2.7 script, or an .exe)
« Reply #9 on: July 03, 2020, 03:48:02 pm »

A new version is up. Added another feature inspired by overlapping mods, but that doesn't strictly have to do with duplication errors.

"name_mode" can be toggled on/off depending if you want to show "duplicated names". These are the in-game names (e.g "tiger:tigers:tiger" from "CREATURE:TIGER"), and so they don't cause duplication errors, but can still be confusing to the player. Note that some vanilla objects are recognized as "duplicated", i.e. skirt, short skirt, long skirt.