Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 2 [3] 4 5 ... 35

Author Topic: PyLNP 0.13 - Cross-platform Lazy Newb Pack port with graphics pack patching  (Read 145847 times)

fricy

  • Bay Watcher
  • [DFHACK:ZEALOT]
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #30 on: August 09, 2014, 12:48:30 pm »

I came back from vacation yesterday, but don't expect too much of me before next week - I'm pretty busy this weekend.
Nice to see you back, just letting you know we started a brainstroming session here how a mod manager could work, it would be good if you could tell us what you think is possible to implement, and what should be rethought.

PeridexisErrant

  • Bay Watcher
  • Dai stihó, Hrasht.
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #31 on: August 14, 2014, 09:39:33 am »

So, my programming course is going nicely, and I may eventually be competent to implement some of my ideas.  Until then, you can read about them!

Fricy linked mod manager discussion.  TLDR is that everyone likes the idea of building up something like a starter pack for mods, where new people can get their toes wet in a GUI that can stack mods.  That sounds pretty similar to the init patching that this has for graphics :)  The plot is basically to extend this launcher and then take over the world start including optional and easy mods in everything. 

The current state-of-the-art in mod merging is the Manilla Launcher, which is generally great for long-time players.  Core concept is an Elder Scrolls-style list; you drag desired mods into order and get told if there'll be clashes.  Obviously using patches will make life easier.  The other excellent feature is live previews for tilesets and color schemes, though sadly these don't stack (non colored tiles, colored vanilla tiles).  I've asked about releasing the source, which IIRC is Java swing (shared privately at request).  A somewhat older but possibly useful tool was the DF Mod Manager, which was in python though a different GUI module.  The core logic *might* be salvagable, or might not.  I don't currently have the time to check. 

Regarding format, I think we need to nail this down soon.  The longer we leave it, the more likely it becomes that the community ends up with multiple patch formats... So my proposal is: 
 - A folder for mods is "/LNP/Mods/". 
 - Each distinct mod gets it's own folder, named whatever the mod is called. 
 - In that folder, include the raw folder structure with any files changed from the vanilla raws; plus at the top level other stuff (eg readme, random files and folders) that can be ignored without error. 
 - Possibly also a manifest file if this takes off and needs extending to more arcane cases. 
 - One of the mod folders is the full vanilla raws; that folder is named identically to the version it came as (eg "df_40_08_win"). 
 - A *real* patch is derived when needed by running a diff between the vanilla folder and the mod
 - Diffs are then applied to a copy of the vanilla raws in the specified order
 - Conflicts can either be banned (load nothing and display error), apply latest diff (risking both diff errors and broken raws), or resolved manually (possibly in popup box).  I favor the first option until the system works; it's easy and newbie friendly which are both essential - extend once the basics are nailed down. 
 - Mods with modules are tricky and probably involve subfolders and a manifest file (xml?).  Think Masterwork stuff with dependencies, mod-specific graphics, etc.  Like good conflict management, this can be done *after* the basic concept works.  suggestions welcome. 
 - Allowing these folders to be zipped would be nice; a hands-off signal and a space saver.  Not too tricky either, I think. 

I may start a topic just for this actually. 
Logged
I maintain the DF Starter Pack - over a million downloads and still counting!
 Donations here.

Pidgeot

  • Bay Watcher
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #32 on: August 15, 2014, 09:51:11 am »

- Preserve non-graphics settings when changing graphics packs (I asume this will just happen once diffs are used)

I've completely forgotten to bring this up: Have people tested this feature (successfully)? You have to enable it after starting the launcher (Experimental > Toggle graphics pack patching), but once you do, only raws will be directly overwritten - the init files are merged by only overwriting specific graphics-related settings.

The reason it isn't the default is *precisely* because I would like to see it tested more before I make it the default, and since I don't know which packs everyone uses, I would like to hear if it works with your favorite packs.

Pidgeot

  • Bay Watcher
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #33 on: August 15, 2014, 12:50:03 pm »

- Update notification tool

I'm currently working on this by putting configuration entries in a file specifying a URL containing the version, the pack version, and a regular expression to pull out the latest version number - but DFFD does not allow Python to access the site, so it would be necessary to keep the information elsewhere.

The good part is I've made it general enough that it can really be hosted *anywhere* - GitHub file, forum post, whatever. It just has to be a single, continuous string, with some easily locatable text next to it. As long as Python can access the URL, it can be detected.

PeridexisErrant

  • Bay Watcher
  • Dai stihó, Hrasht.
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #34 on: August 15, 2014, 05:52:04 pm »

- Update notification tool

I'm currently working on this by putting configuration entries in a file specifying a URL containing the version, the pack version, and a regular expression to pull out the latest version number - but DFFD does not allow Python to access the site, so it would be necessary to keep the information elsewhere.

The good part is I've made it general enough that it can really be hosted *anywhere* - GitHub file, forum post, whatever. It just has to be a single, continuous string, with some easily locatable text next to it. As long as Python can access the URL, it can be detected.

Does that include this page too?  http://dffd.wimbli.com/file_version.php?id=7622
Logged
I maintain the DF Starter Pack - over a million downloads and still counting!
 Donations here.

salithus

  • Bay Watcher
  • gottagofast
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #35 on: August 15, 2014, 06:08:36 pm »

Any thought to working with NexusMods? They've got an established system for hosting mods, detecting updates, etc.
Logged

Pidgeot

  • Bay Watcher
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #36 on: August 15, 2014, 07:29:35 pm »

- Update notification tool

I'm currently working on this by putting configuration entries in a file specifying a URL containing the version, the pack version, and a regular expression to pull out the latest version number - but DFFD does not allow Python to access the site, so it would be necessary to keep the information elsewhere.

The good part is I've made it general enough that it can really be hosted *anywhere* - GitHub file, forum post, whatever. It just has to be a single, continuous string, with some easily locatable text next to it. As long as Python can access the URL, it can be detected.

Does that include this page too?  http://dffd.wimbli.com/file_version.php?id=7622

That was the exact URL I used to test, so yes.

DFFD can still be used for tbe download (the download page just gets opened in the user's webbrowser, so Python doesn't get involved), but the version number itself currently needs to be somewhere else.

PeridexisErrant

  • Bay Watcher
  • Dai stihó, Hrasht.
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #37 on: August 15, 2014, 07:54:50 pm »

Ah well, I suppose the string can be pulled of the Bay12 topic without changing much.  Still a bit of a pain though, and it'll artificially inflate the view count... never mind, it'll work for now and when I have more free time I'll talk to Janus and maybe look at user agent spoofing. 
Logged
I maintain the DF Starter Pack - over a million downloads and still counting!
 Donations here.

salithus

  • Bay Watcher
  • gottagofast
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #38 on: August 16, 2014, 08:16:01 am »

What about tracking versions on the wiki or something? If you can use that to check updates it would hopefully also push adoption of using the wiki to document the mod - what it's for, how to use it, etc.
Logged

Pidgeot

  • Bay Watcher
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #39 on: August 16, 2014, 08:37:01 am »

What about tracking versions on the wiki or something? If you can use that to check updates it would hopefully also push adoption of using the wiki to document the mod - what it's for, how to use it, etc.

The wiki works fine, and as long as the page has a sufficiently unique string to look for, it'll work. For example:

Code: [Select]
"checkURL": "http://dwarffortresswiki.org/index.php/Utility:Lazy_Newb_Pack",
"versionRegex": "Lazy Newb Pack Linux V([0-9.]+)",

This reads "0.40.07" from the current wiki page.

Of course, a small text file (a la what DFFD provides through file_version.php) is preferable, because it doesn't need to download as much, and it's easier to make sure it works. If you have a personal web page, you could always just host a small text file yourself; you could also store and update a file in GitHub/BitBucket/whatever and just use a direct link to the newest version (e.g. https://bitbucket.org/Pidgeot/python-lnp/raw/tip/PyLNP.json).

PeridexisErrant

  • Bay Watcher
  • Dai stihó, Hrasht.
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #40 on: August 16, 2014, 08:56:27 am »

Very nice. 
Logged
I maintain the DF Starter Pack - over a million downloads and still counting!
 Donations here.

salithus

  • Bay Watcher
  • gottagofast
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #41 on: August 16, 2014, 08:57:20 am »

What about tracking versions on the wiki or something? If you can use that to check updates it would hopefully also push adoption of using the wiki to document the mod - what it's for, how to use it, etc.

The wiki works fine, and as long as the page has a sufficiently unique string to look for, it'll work. For example:

Code: [Select]
"checkURL": "http://dwarffortresswiki.org/index.php/Utility:Lazy_Newb_Pack",
"versionRegex": "Lazy Newb Pack Linux V([0-9.]+)",

This reads "0.40.07" from the current wiki page.

Of course, a small text file (a la what DFFD provides through file_version.php) is preferable, because it doesn't need to download as much, and it's easier to make sure it works. If you have a personal web page, you could always just host a small text file yourself; you could also store and update a file in GitHub/BitBucket/whatever and just use a direct link to the newest version (e.g. https://bitbucket.org/Pidgeot/python-lnp/raw/tip/PyLNP.json).
If you go the wiki direction, http://dwarffortresswiki.org/api.php?action=parse&page=Utility:Lazy_Newb_Pack should give (marginally) lighter weight, plus if you take an approach like they do with the raws (http://dwarffortresswiki.org/index.php/DF2014:Adamantine/raw is included in http://dwarffortresswiki.org/index.php/DF2014:Adamantine via a {{gamedata}} template), you could set up a template to format the current version on the mod page, plus have a smaller page to grab to find the current version.
Logged

salithus

  • Bay Watcher
  • gottagofast
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #42 on: August 16, 2014, 08:57:49 am »

e: Also I'm just throwing ideas out while I get a few cups of coffee under my belt, so disregard at will :D
Logged

Pidgeot

  • Bay Watcher
    • View Profile
Re: PyLNP - Cross-platform Lazy Newb Pack port with graphics pack patching
« Reply #43 on: August 17, 2014, 12:13:36 pm »

Work in progress...

Spoiler: Screenshot (click to show/hide)

The preview isn't being drawn correctly, but once I figure that part out, color scheme selection is good to go, and I'll upload a new set of builds.

Each colorscheme is a text file that overwrites colors.txt in the DF folder. The program looks in the folder LNP/Colors for these.

Here's a link to the color schemes I have prepared; just extract to the LNP folder and you'll be ready for the next build: https://www.dropbox.com/s/9175wfuzy1vs05c/Colors.zip

Edit: New builds have been posted.
« Last Edit: August 17, 2014, 03:07:05 pm by Pidgeot »
Logged

MagiX

  • Bay Watcher
    • View Profile

@Pidgeot: Thank you soo much for your work, just registered to post this.

A few suggestions:
- Support for DFhack under Linux/OSX (i.e. see my commits below)
- Linux and OS/X are case sensitive with respect to folder and file names (to my knowledge). Windows is insensitive with this respect. Maybe follow along the lines as suggested here https://stackoverflow.com/questions/8462449/python-case-insensitive-file-name to fix this.

I'll be willing to help with development, even though my coding style sucks and I'm rather new to "application" programming, as you can see from here: https://github.com/MagiX13/pyLNP/commits/master

Regarding version check of plugins, I hacked a small script that uses headers to overcome the hurdle imposed by dffd. As a further development suggestion, one could centralize the version check somewhere and just download a specific file with all the version numbers (I suggest going with updated) instead of DDOSing the dffd server... find it here: https://gist.github.com/MagiX13/651e323c92ab86196a04

I would be willing to help with the development of this project, but I'm far away from being a reasonable programmer of any kinds...
« Last Edit: August 18, 2014, 05:03:57 am by MagiX »
Logged
Pages: 1 2 [3] 4 5 ... 35