Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 6 7 [8] 9 10 ... 14

Author Topic: Rubble 8.5.5 - DF 44.7 - New DF, new Rubble.  (Read 65905 times)

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #105 on: April 21, 2016, 03:25:33 pm »

I suppose it wouldn't be too hard to write a function that returns lines with imbalanced braces. I don't know what I was thinking (probably I was thinking about curly braces for templates) raw tags are always on one line, so it would actually be fairly easy (what can I say, I had a brain fart).

I have had some thoughts before about making Rubble read all files into a giant tag-tree/database and operate on them in that format, but templates always get in the way (making this impossible). Any edition of Rubble that deals with raw tags natively will be so different from the current version you won't really be able to call it Rubble anymore...

About "Haiku Vanilla": have you looked at the starting points used by the various editions of the "DF from Scratch" project? It's amazing what you can remove if you want...
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #106 on: April 22, 2016, 09:09:19 am »

I have had some thoughts before about making Rubble read all files into a giant tag-tree/database and operate on them in that format, but templates always get in the way (making this impossible). Any edition of Rubble that deals with raw tags natively will be so different from the current version you won't really be able to call it Rubble anymore...
Nevertheless you have already done that in Rubble in some way, it is not only in your imagination, Lua data as those that you use to charge DF raws into, is already an in memory Key-Value lua database, and a very efficient one by the Lua design, lacking only a mechanism of serialización for data persistence into the filesystem using a sound and well defined data format, and some aggressive, business class, optimization for performance...
Well! Rubble is going by very promising ways lately, we have lots of things to explore with the actual model before a new change of paradigm, for example: exploit the quasi identity of the datamodels of Lua and DF raws (declarative tables of Key, Values: https://en.wikipedia.org/wiki/Key-value_database) to build a little Lua DSL (https://en.wikipedia.org/wiki/Domain-specific_language) with which the normal DF raws as they are would be interpreted as pure Lua data with the correct table hiérarchie; http://moonscript.org/, a language that compiles to Lua does something similar, but what would be needed for our DSL is way simpler... Other Lua DSL example: http://leafo.net/guides/dsl-in-lua.html
In any case this giant tag-tree/database Lua aware approach has been taken by the project http://tarantool.org/ (Get your data in RAM or files. Get compute close to data. Enjoy the performance of http://luajit.org/)
Features

1. A drop-in replacement for Lua 5.1, based on http://luajit.org/ 2.1;
2. simply use #!/usr/bin/tarantool instead of #!/usr/bin/lua in your script
3. Lua packages for non-blocking I/O, fibers and HTTP
4. http://msgpack.org/ MessagePack data format and MessagePack based client-server protocol
5. a built-in database with two data engines: 100% in-memory with optional persistence and a 2-level disk-based B-tree, to use with large data sets
6. secondary key and index iterators support
7. asynchronous master-master replication
8. authentication and access control

This opensource Lua integrated with a NOSQL database (https://en.wikipedia.org/wiki/NoSQL) is being developed by the principal russian mail provider and are in production to provide services to their 25 millions of clients.So! If you are taken by a fey mood to integrate a proper database system into Rubble, this could be a good start to reuse or take some ideas. They have top performance between the in production NOSQL databases, extensive and complete documentation and correct support forums. Their complete development download weighs less than 4 megabytes with the included source code. Almost all of it, Lua code.
About "Haiku Vanilla": have you looked at the starting points used by the various editions of the "DF from Scratch" project? It's amazing what you can remove if you want...
Thanks for the idea! I am going to integrate cumulated knowledge (SCIENCE!!!) of the the 3 seasons of Dwarf Fortress from Scratch to make VanillaLibs more functional while I integrate those 3 new mods into HDFPS.
1. http://www.bay12forums.com/smf/index.php?topic=127552.0 -> Dwarf Fortress from Scratch: The entirely player-made universe succession.
2. http://www.bay12forums.com/smf/index.php?topic=140715.0 -> Dwarf Fortress from Scratch: 0.40.x
3. http://www.bay12forums.com/smf/index.php?topic=154960.0 -> Dwarf Fortress from Scratch Redux - An entirely player made universe succession
« Last Edit: April 22, 2016, 03:20:29 pm by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

Dirst

  • Bay Watcher
  • [EASILY_DISTRA
    • View Profile
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #107 on: April 22, 2016, 09:48:40 am »

As for major changes, I would really like to find some way of supporting third (fourth?) party content like Stonesense assets, Armok Vision models, Soundsense packs, Therapist grids, etc.

The trouble with "global" content like this is that it falls outside of the norm. It is a good idea, but I am not sure how best to handle it... Maybe a way to flag a directory and its children (including child directories) as a "copy file addon" that is then copied to a given AXIS path during the write stage?
Since this behavior is "outside the save folder" then it should require explicit approval from the player, up to and including the idea that foreign files need to be in their own completely separate addon that needs to be checked separately.  Addon authors can already include additional addons within the original zip.

In that case, the compatibility levels can be "incompatible", "normal", "recommended", and "dependent".  The odd bit is how to inform the player (end-user) upon clicking a checkbox that others have been added/dropped from the recommended list.  Not sure if it's worth autodownloading merely recommended packages.
Logged
Just got back, updating:
(0.42 & 0.43) The Earth Strikes Back! v2.15 - Pay attention...  It's a mine!  It's-a not yours!
(0.42 & 0.43) Appearance Tweaks v1.03 - Tease those hippies about their pointy ears.
(0.42 & 0.43) Accessibility Utility v1.04 - Console tools to navigate the map

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #108 on: April 23, 2016, 11:58:15 am »

Trying to integrate a powerful enough DSL and database with the existing system seems like a good way to drive yourself insane. The current system is way too dependent on files being byte slices (arrays) to make an object/tag database work.

@Dirst: I have a working system that allows addon packs to define custom file tagger rules and add extra tag based "writers", so that will do for basic stuff. More advanced writing tasks I have still not entirely decided how to handle yet. In any case, due to UI issues I won't be making it too complicated (prompting the user is basically impossible unless I want to do a LOT of work).

@Abadrausar:
I downloaded your project, but between DFFD and my computer the file got corrupted (flaky internet connection), so I haven't been able to look at it yet (I am redownloading it now).

I did spend some time looking at your delta templates, here a commented version of the original with some minor stuff fixed:

Code: [Select]
-------------------------------------------------------------------------------------------------------------------------------------------
-- Semantic Delta operators
-- {@CONTEXT;objectid;wheretoappend}
-- Fixes the objectid (for example ENTITY:MOUNTAIN) where the delta will be applied and also if the appended preraws go before or after (this is the default) the attachment token.
-- {@DELTA;token;value;preraws}
-- Modifies the end value of a token inside an object and  eventually attach before or after it the preraws, if the token id is given but not the value the token is killed.
-- if not token is given the preraws are appended at the end of the object. When no object is found a detailed DELTA warning is generated.
-------------------------------------------------------------------------------------------------------------------------------------------

-- There were only two minor issues that prevented the DELTA template from working properly. Both were easy fixes once
-- found :)
--
-- These templates (and maybe a few other supporting ones using the same basic system) would make a good standard library
-- addon at some point in the not-too-distant future. Keep polishing and tweaking!
--
-- I spent a few hours going over this code line by line (instead of working on the more boring thing I should have been
-- working on). The code quality is very good, particularly since you are working with an API that is, at this level,
-- mostly undocumented. Most of the things I changed were trivial, not really worth worrying about.
--
-- I didn't really realize how OCD I am about code until I started going through someone else's...

-- Lets save "rubble" for standard and default stuff shall we? If these become standard they should use "rubble.libs_diff"
-- or something, but not yet.
delta_object_cursor = ""
delta_attach_after = true

-- Changed to new-style template bodies. Error messages tend to be a tiny (but sometimes very helpful) bit better this way.
-- Remember: These functions may not have any upvalues aside from _ENV (which is automatically manged by the VM).
local socontext = function(value, after)
value, after = rubble.expandargs(value, after)
-- Expanding variables here was unnecessary, rubble.targs already handled that (and now rubble.expandargs does instead)
delta_object_cursor = value

-- If statements that set a boolean are an abomination :P
-- In addition to eliminating the if statement of which we will not speak, I also added some minor flexibility.
delta_attach_after = ({["before"] = true, ["prepend"] = true})[string.lower(after)] ~= nil
end
rubble.template("!CONTEXT", socontext) -- <- Bad idea. Can lead to evaluation order problems (such as plague !SHARED_OBJECT_DUPLICATE). Not saying this should be removed, but the docs need an extra warning for this version.
rubble.template("@CONTEXT", socontext) -- <- Also a bad idea, but maybe not as bad as the other one...
rubble.template("CONTEXT", socontext)
rubble.template("#CONTEXT", socontext)
rubble.template("#C", socontext) -- <- This abbreviation is probably a bad idea.
-- For one thing "C" is the comment template, for another such abbreviations should probably be reserved for certain
-- very well known standard templates (there is too much chance of collision otherwise). Normally I would also complain
-- about the template names not being namespaced in some way, but as it is very likely that these templates will become
-- standard at some point in the future it is better that they are not.
--
-- While namespacing is a bad idea in this case, the current names may be too simple for the standard library. Something
-- more (specifically) descriptive may be in order.

local sodelta = function(token, value, preraws)
token, value, preraws = rubble.expandargs(token, value, preraws)

-- Added this line to make sure "preraws" is a string (for later concatenation).
-- This is needed because the new-style template bodies leave optional parameters that are not provided set to nil.
-- I thought that rubble.expandargs took care of this little detail, but it doesn't.
-- In v8 rubble.expandargs always returns strings, so this line may be removed then.
preraws = preraws or ""

if token == "" then
-- Use the function, Luke!
rubble.libs_base.sharedobject_add(delta_object_cursor, preraws)
return
end

-- Moved the warning message here. Where it was it would print for every single tag that did not match the token,
-- when clearly the effect you wanted was for it to print only if the tag you wanted could not be found.
-- I split the warning into three parts, one here that triggers if the object does not exist, one later that triggers
-- if the tag does not exist, and one that triggers if more than one tag matches. Change to taste :)
if rubble.registry["Libs/Base:!SHARED_OBJECT"].table[delta_object_cursor] == nil then
-- Is there a particular reason why this doesn't cause an abort?
rubble.warning("Object: \""..delta_object_cursor.."\" not found: Malformed delta operator call or incorrect order of addon application.\n")
return
end

local found, warned = false, false
-- |
-- V Changed this line to create a new variable instead of reusing "token" (since the original version of "token" is needed later)
local stoken = string.split(token, ":")
rubble.libs_base.sharedobject_walk(delta_object_cursor, function(tag)
if rubble.libs_base.matchtag(tag, stoken) then
if found and not warned then
rubble.warning("Multiple tags fitting the specifier: \""..token.."\" Found in object: \""..delta_object_cursor.."\". This may indicate an improper use of the delta operator.\n")
warned = true
end

if value == "" then
local preraws = "-"..tag.ID
for _, v in ipairs(tag.Params) do
preraws = preraws..":"..v
end
tag.Comments = preraws.."-"..tag.Comments
else
local fulltoken = "["..token..":"..value.."]"
if delta_attach_after then
tag.Comments = preraws..fulltoken..tag.Comments
else
tag.Comments = fulltoken..preraws..tag.Comments
end
end
tag.CommentsOnly = true
found = true
end
end)
if not found then
-- It may be a good idea to allow modders to suppress this warning for certain calls, I cant decide if its worth it or not...
rubble.warning("A tag fitting the specifier: \""..token.."\" Could not be found in object: \""..delta_object_cursor.."\". This may indicate an improper use of the delta operator.\n")
end
end
rubble.template("!DELTA", sodelta) -- See earlier comments.
rubble.template("@DELTA", sodelta) -- See earlier comments.
rubble.template("DELTA", sodelta)
rubble.template("#DELTA", sodelta)
rubble.template("@D", sodelta) -- See comment about previous abbreviation.



Rubble 8 is coming along well, the new loader is working and all that is left (aside from more testing) is to implement a system for writing trees of "outside the save" files and a few other (much more minor) things.
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #109 on: April 24, 2016, 05:07:06 am »

Trying to integrate a powerful enough DSL and database with the existing system seems like a good way to drive yourself insane. The current system is way too dependent on files being byte slices (arrays) to make an object/tag database work.
Like you I think that having the possibility to access to the token lexer as an array of params where some of the first ones are also the token lvalue is too util to lose it at the moment of introducing database, DSL or grammar, so maybe the best should be to postpone these introductions until one sound way to retain those desirable arrays in the system is found.
Japanese culture have developed a form of art specifically designed for OCD people, it is called Haiku, it can adopt many forms: picture, poesie, ... In this form of art,  the artist meditates a long time over the target object of his art, sometimes even many years, at some point the artist realises that he have fusionated with the essence of his target, then all that concentrated essence emerges from the artist like a rapids stream of art.
That is why the MOMA of NY exposes unacheved DF. Toady One is doing a 20 years long software haiku.

Thanks for your help with the semantic delta system! :D, I have some incomplete ideas about how to choose the correct token when multiple of the same lvalue are detected
-- These templates (and maybe a few other supporting ones using the same basic system) would make a good standard library
-- addon at some point in the not-too-distant future. Keep polishing and tweaking!
We have
Spoiler (click to show/hide)
And we want that
Code: [Select]
{@CONTEXT;PLANT:SINGLE-GRAIN_WHEAT;;USE_MATERIAL_TEMPLATE:MILL}
{@DELTA;MATERIAL_VALUE;25}
do the correct thing, How?
What is intended is to change from 20 to 25 the MATERIAL_VALUE of the milled SINGLE-GRAIN_WHEAT, the part that is not implemented by the moment is to use the sparse array as a sentinel to limit the action to the MILL nested subobject, that is important because if the entire token [MATERIAL_VALUE:20] is eliminated by Toady our deltas would modify the MATERIAL_VALUE of SEED from 1 to 25 instead of generating a warning of failed delta as it should...
The problem is that sharedobject_walk(id, action) and rubble.libs_base.matchtag(tag, match) or some version of these should take that into account and by now I do not fully understand what they do or how? but maybe a modification of rubble.libs_base.matchtag  like
Code: [Select]
function rubble.libs_base.matchtag(tag, match)
local object = string.split(delta_object_cursor, ":")
if tag.CommentsOnly then
return false
end

if #match ~= 1 and #match-1 > #tag.Params then
return false
end

if match[1] ~= tag.ID then
return false
end
-- this should act like a sentinel to restric operation to the nested subobject tokens identified by the context
for i, v in ipairs_sparse(whereAreTheNestedKeys) do
if delta_nestedKey_cursor ~= "" and whereAreTheNestedKeys[object[1]][tag.Params[v]] == true  then
return false
end
end
for i = 2, #match, 1 do
if match[i] ~= "&" and match[i] ~= tag.Params[i-1] then
return false
end
end
return true
end
Would do the thing, It is really a pity the difficulty of debugging, if there were support for luasockets any lua debugger out there could be used to understand how the data structures are working :-[
-----------------------------------------------------------------------------------------------------------------------------------------------
--                                    The precursors actually deprecated
-- {!TEMPLATE;@SET_VALUE;token;value;preraws;{@SHARED_OBJECT_REPLACE_TAG;$OBJECT_CURSOR;%{token};[%{token}:%{value}]%{preraws}}}
-- {!TEMPLATE;@ATTACH_BEFORE_TAG;token;preraws;{@SHARED_OBJECT_REPLACE_TAG;$OBJECT_CURSOR;%{token};%{preraws}[%{token}:%{value}]}}
-- {!TEMPLATE;@ATTACH_AFTER_TAG;token;preraws;{@SHARED_OBJECT_REPLACE_TAG;$OBJECT_CURSOR;%{token};[%{token}:%{value}]%{preraws}}}
-- {!TEMPLATE;@REPLACE_TAG;token;preraws;{@SHARED_OBJECT_REPLACE_TAG;$OBJECT_CURSOR;%{token};%{preraws}}}
-------------------------------------------------------------------------------------------------------------------------------------------
--                                 Semantic Delta operators
--      {@CONTEXT;objectId;insertBeforeOrAfterContext;nestedKeyTokenId}
--   Fixes the objectid (for example ENTITY:MOUNTAIN) where the delta will be applied and also if the appended preraws go before or after (this is the default) the attachment token.
--      {@DELTA;tokenId;value;preraws}
--   Modifies the end value of a token inside an object and  eventually attach before or after it the preraws, if the token id is given but not the value the token is killed.
--   if not tokenId is given the preraws are appended at the end of the object. When no object is found a detailed DELTA warning is generated.
--   When nestedKeyTokenId is used the correct tokenId is selected taking the first tokenId between nestedKeyTokenId and the next nestedKeyToken,
--- if the requested tokenId is not found between those limits, a warning of unapplied delta is generated.

-------------------------------------------------------------------------------------------------------------------------------------------

-- There were only two minor issues that prevented the DELTA template from working properly. Both were easy fixes once
-- found :)
--
-- These templates (and maybe a few other supporting ones using the same basic system) would make a good standard library
-- addon at some point in the not-too-distant future. Keep polishing and tweaking!
--
-- I spent a few hours going over this code line by line (instead of working on the more boring thing I should have been
-- working on). The code quality is very good, particularly since you are working with an API that is, at this level,
-- mostly undocumented. Most of the things I changed were trivial, not really worth worrying about.
--
-- I didn't really realize how OCD ;) I am about code until I started going through someone else's...

-- Lets save "rubble" for standard and default stuff shall we? If these become standard they should use "rubble.libs_diff"
-- or something, but not yet.

function ipairs_sparse(t)
   local tmpIndex = {}         -- tmpIndex will hold sorted indices, otherwise
   local index, _ = next(t)   -- this iterator would be no different from pairs iterator
   while index do
      tmpIndex[#tmpIndex+1] = index
      index, _ = next(t, index)
   end
   table.sort(tmpIndex)   -- sort table indices
   local j = 1
   return function()
      local i = tmpIndex[j]   -- get index value
      j = j + 1
      if i then
         return i, t
      end
   end
end
-- Extensive use of functional linearized sparse arrays to describe the principal interelation between the hierarchie of keyTokenIds, not exhaustive, only those necesary to make the Delta system usable
-- non keyTokens are ignored as they are irrelevant for the structure, the incompleteness is due to the creature objects and their variations, thinking about a more complete system for those...
-- 1st array dimension are the identifiers that appears after the token OBJECT at the beginning of DF vanilla files, namespaces, indent level = 0
-- 2nd array dimension are the keyTokenIds that are possible inside each namespace, indent level = 1
-- 3rd array dimension are the nestedKeyTokenIds that are possible inside each type of object, indent level = 2
-- 4th array dimension are the nestedKeyTokenIds that are possible inside root nestedKeyTokenIds, indent level = 3
-- 5th array dimension... and so on, so on, indent level = 4
-- the integer is the variation of indent level that vanilla raws uses for each keyTokenId, being root the base namespace, when level = -1 the tiken is acting as a closing bracket of a nested zone
-- [""] = 2 ,[""] = 2 ,[""] = 2 ,[""] = 2 ,[""] = 2 ,[""] = 2 ,[""] = 2 ,[""] = 2 ,
Key = {}
Key["BODY_DETAIL_PLAN"] = {["OBJECT"] = 0,["BODY_DETAIL_PLAN"] = 1,["CREATURE"] = 2,}
Key["BODY"] = {["OBJECT"] = 0,["BODY"] = 1,}
Key["BODYGLOSS"] = {["BODY"] = 1,}
Key["BP"] = {["BODY"] = 2,}
Key["BUILDING"] = {["OBJECT"] = 0,}
Key["BUILDING_WORKSHOP"] = {["BUILDING"] = 1,}
Key["BUILDING_SMELTER"] = {["BUILDING"] = 1,}
Key["CREATURE"] = {["OBJECT"] = 0,["CREATURE"] = 1,}
Key["ATTACK"] = {["CREATURE"] = 2,}
Key["BODY_APPEARANCE_MODIFIER"] = {["CREATURE"] = 2,}
Key["CAN_DO_INTERACTION"] = {["CREATURE"] = 2,}
Key["CASTE"] = {["CREATURE"] = 2,}
Key["EXTRA_BUTCHER_OBJECT"] = {["CREATURE"] = 2,}
Key["MENT_ATT_RANGE"] = {["CREATURE"] = 2,}
Key["SELECT_CASTE"] = {["CREATURE"] = 2,}
Key["SELECT_MATERIAL"] = {["CREATURE"] = 2,}
Key["SELECT_TISSUE_LAYER"] = {["CREATURE"] = 2,}
Key["SEMIMEGABEAST"] = {["CREATURE"] = 2,}
Key["SET_BP_GROUP"] = {["CREATURE"] = 2,}
Key["TL_COLOR_MODIFIER"] = {["CREATURE"] = 2,}
Key["USE_MATERIAL_TEMPLATE"] = {["CREATURE"] = 2,}
Key["SET_TL_GROUP"] = {["SELECT_CASTE"] = 3,}
Key["CASTE_NAME"] = {["SELECT_CASTE"] = 3,}
Key["PLUS_TISSUE_LAYER"] = {["SELECT_TISSUE_LAYER"] = 3,}
Key["SYNDROME"] = {["USE_MATERIAL_TEMPLATE"] = 3,}
Key["CREATURE_VARIATION"] = {["OBJECT"] = 0,["CREATURE_VARIATION"] = 1,}
Key["CV_CONVERT_TAG"] = {["CREATURE_VARIATION"] = 2,}
Key["DESCRIPTOR_COLOR"] = {["OBJECT"] = 0,}
Key["COLOR"] = {["DESCRIPTOR_COLOR"] = 1,}
Key["DESCRIPTOR_PATTERN"] = {["OBJECT"] = 0,}
Key["COLOR_PATTERN"] = {["DESCRIPTOR_PATTERN"] = 1,}
Key["DESCRIPTOR_SHAPE"] = {["OBJECT"] = 0,}
Key["SHAPE"] = {["DESCRIPTOR_SHAPE"] = 1,}
Key["ENTITY"] = {["OBJECT"] = 0,["ENTITY"] = 1,}
Key["WEAPON"] = {["ENTITY"] = 2,}
Key["POSITION"] = {["ENTITY"] = 2,}
Key["TISSUE_STYLE"] = {["ENTITY"] = 2,}
Key["INORGANIC"] = {["OBJECT"] = 0,["INORGANIC"] = 1,}
Key["ITEM"] = {["OBJECT"] = 0,}
Key["ITEM_AMMO"] = {["ITEM"] = 1,}
Key["ITEM_ARMOR"] = {["ITEM"] = 1,}
Key["ITEM_FOOD"] = {["ITEM"] = 1,}
Key["ITEM_GLOVES"] = {["ITEM"] = 1,}
Key["ITEM_HELM"] = {["ITEM"] = 1,}
Key["ITEM_PANTS"] = {["ITEM"] = 1,}
Key["ITEM_SIEGEAMMO"] = {["ITEM"] = 1,}
Key["ITEM_SHIELD"] = {["ITEM"] = 1,}
Key["ITEM_SHOES"] = {["ITEM"] = 1,}
Key["ITEM_TOOL"] = {["ITEM"] = 1,}
Key["ITEM_TOY"] = {["ITEM"] = 1,}
Key["ITEM_TRAPCOMP"] = {["ITEM"] = 1,}
Key["ITEM_WEAPON"] = {["ITEM"] = 1,}
Key["ATTACK"] = {["ITEM_AMMO"] = 2,["ITEM_TOOL"] = 2,["ITEM_TRAPCOMP"] = 2,["ITEM_WEAPON"] = 2,}
Key["INTERACTION"] = {["OBJECT"] = 0,["INTERACTION"] = 1,}
Key["I_SOURCE"] = {["INTERACTION"] = 2,}
Key["I_TARGET"] = {["INTERACTION"] = 2,}
Key["I_EFFECT"] = {["INTERACTION"] = 2,}
Key["LANGUAGE"] = {["OBJECT"] = 0,}
Key["SYMBOL"] = {["LANGUAGE"] = 1,}
Key["TRANSLATION"] = {["LANGUAGE"] = 1,}
Key["WORD"] = {["LANGUAGE"] = 1,}
Key["ADJ"] = {["WORD"] = 2,}
Key["NOUN"] = {["WORD"] = 2,}
Key["PREFIX"] = {["WORD"] = 2,}
Key["VERB"] = {["WORD"] = 2,}
Key["MATERIAL_TEMPLATE"] = {["OBJECT"] = 0,["MATERIAL_TEMPLATE"] = 1,}
Key["SYNDROME"] = {["MATERIAL_TEMPLATE"] = 2,}
Key["PLANT"] = {["OBJECT"] = 0,["PLANT"] = 1,}
Key["GROWTH"] = {["PLANT"] = 2,}
Key["USE_MATERIAL_TEMPLATE"] = {["PLANT"] = 2,}
Key["DRINK"] = {["PLANT"] = -1,}
Key["MILL"] = {["PLANT"] = -1,}
Key["SEED"] = {["PLANT"] = -1,}
Key["BASIC_MAT"] = {["PLANT"] = -1,}
Key["THREAD"] = {["PLANT"] = -1,}
Key["EXTRACT_BARREL"] = {["PLANT"] = -1,}
Key["EXTRACT_STILL_VIAL"] = {["PLANT"] = -1,}
Key["EXTRACT_VIAL"] = {["PLANT"] = -1,}
Key["TREE"] = {["PLANT"] = -1,}
Key["REACTION"] = {["OBJECT"] = 0,["REACTION"] = 1,}
Key["PRODUCT"] = {["REACTION"] = 2,}
Key["REAGENT"] = {["REACTION"] = 2,}
Key["TISSUE_TEMPLATE"] = {["OBJECT"] = 0,}
Key["WORLD_GEN"] = {["OBJECT"] = 0,}
Key["PROFILE"] = {["OBJECT"] = 0,}
Key["TISSUE_TEMPLATE"] = {["TISSUE_TEMPLATE"] = 1,}
Key["WORLD_GEN"] = {["WORLD_GEN"] = 1,}
Key["PROFILE"] = {["PROFILE"] = 1,}
Link = {}
Link["BODY_DETAIL_PLAN"] = {"OBJECT" ,"BODY_DETAIL_PLAN","CREATURE",}
Link["BODY"] = {"OBJECT","BODY",}
Link["BODYGLOSS"] = {"BODY",}
Link["BP"] = {"BODY",}
Link["BUILDING"] = {"OBJECT",}
Link["BUILDING_WORKSHOP"] = {"BUILDING",}
Link["BUILDING_SMELTER"] = {"BUILDING",}
Link["CREATURE"] = {"OBJECT","CREATURE",}
Link["ATTACK"] = {"CREATURE",}
Link["BODY_APPEARANCE_MODIFIER"] = {"CREATURE",}
Link["CAN_DO_INTERACTION"] = {"CREATURE",}
Link["CASTE"] = {"CREATURE",}
Link["EXTRA_BUTCHER_OBJECT"] = {"CREATURE",}
Link["MENT_ATT_RANGE"] = {"CREATURE",}
Link["SELECT_CASTE"] = {"CREATURE",}
Link["SELECT_MATERIAL"] = {"CREATURE",}
Link["SELECT_TISSUE_LAYER"] = {"CREATURE",}
Link["SEMIMEGABEAST"] = {"CREATURE",}
Link["SET_BP_GROUP"] = {"CREATURE",}
Link["TL_COLOR_MODIFIER"] = {"CREATURE",}
Link["USE_MATERIAL_TEMPLATE"] = {"CREATURE",}
Link["SET_TL_GROUP"] = {"SELECT_CASTE",}
Link["CASTE_NAME"] = {"SELECT_CASTE",}
Link["PLUS_TISSUE_LAYER"] = {"SELECT_TISSUE_LAYER",}
Link["SYNDROME"] = {"USE_MATERIAL_TEMPLATE",}
Link["CREATURE_VARIATION"] = {"OBJECT", "CREATURE_VARIATION",}
Link["CV_CONVERT_TAG"] = {"CREATURE_VARIATION",}
Link["DESCRIPTOR_COLOR"] = {"OBJECT",}
Link["COLOR"] = {"DESCRIPTOR_COLOR",}
Link["DESCRIPTOR_PATTERN"] = {"OBJECT",}
Link["COLOR_PATTERN"] = {"DESCRIPTOR_PATTERN",}
Link["DESCRIPTOR_SHAPE"] = {"OBJECT",}
Link["SHAPE"] = {"DESCRIPTOR_SHAPE",}
Link["ENTITY"] = {"OBJECT","ENTITY",}
Link["WEAPON"] = {"ENTITY",}
Link["POSITION"] = {"ENTITY",}
Link["TISSUE_STYLE"] = {"ENTITY",}
Link["INORGANIC"] = {"OBJECT", "INORGANIC",}
Link["ITEM"] = {"OBJECT",}
Link["ITEM_AMMO"] = {"ITEM",}
Link["ITEM_ARMOR"] = {"ITEM",}
Link["ITEM_FOOD"] = {"ITEM",}
Link["ITEM_GLOVES"] = {"ITEM",}
Link["ITEM_HELM"] = {"ITEM",}
Link["ITEM_PANTS"] = {"ITEM",}
Link["ITEM_SIEGEAMMO"] = {"ITEM",}
Link["ITEM_SHIELD"] = {"ITEM",}
Link["ITEM_SHOES"] = {"ITEM",}
Link["ITEM_TOOL"] = {"ITEM",}
Link["ITEM_TOY"] = {"ITEM",}
Link["ITEM_TRAPCOMP"] = {"ITEM",}
Link["ITEM_WEAPON"] = {"ITEM",}
Link["ATTACK"] = {"ITEM_AMMO","ITEM_TOOL","ITEM_TRAPCOMP","ITEM_WEAPON",}
Link["INTERACTION"] = {"OBJECT","INTERACTION",}
Link["I_SOURCE"] = {"INTERACTION"}
Link["I_TARGET"] = {"INTERACTION",}
Link["I_EFFECT"] = {"INTERACTION",}
Link["LANGUAGE"] = {"OBJECT",}
Link["SYMBOL"] = {"LANGUAGE",}
Link["TRANSLATION"] = {"LANGUAGE",}
Link["WORD"] = {"LANGUAGE",}
Link["ADJ"] = {"WORD",}
Link["NOUN"] = {"WORD",}
Link["PREFIX"] = {"WORD",}
Link["VERB"] = {"WORD",}
Link["MATERIAL_TEMPLATE"] = {"OBJECT","MATERIAL_TEMPLATE",}
Link["SYNDROME"] = {"MATERIAL_TEMPLATE",}
Link["PLANT"] = {"OBJECT","PLANT",}
Link["GROWTH"] = {"PLANT",}
Link["USE_MATERIAL_TEMPLATE"] = {"PLANT",}
Link["DRINK"] = {"PLANT",}
Link["MILL"] = {"PLANT",}
Link["SEED"] = {"PLANT",}
Link["BASIC_MAT"] = {"PLANT",}
Link["THREAD"] = {"PLANT",}
Link["EXTRACT_BARREL"] = {"PLANT",}
Link["EXTRACT_STILL_VIAL"] = {"PLANT",}
Link["EXTRACT_VIAL"] = {"PLANT",}
Link["TREE"] = {"PLANT",}
Link["REACTION"] = {"OBJECT","REACTION",}
Link["PRODUCT"] = {"REACTION",}
Link["REAGENT"] = {"REACTION",}
Link["TISSUE_TEMPLATE"] = {"OBJECT",}
Link["WORLD_GEN"] = {"OBJECT",}
Link["PROFILE"] = {"OBJECT",}
Link["TISSUE_TEMPLATE"] = {"TISSUE_TEMPLATE",}
Link["WORLD_GEN"] = {"WORLD_GEN",}
Link["PROFILE"] = {"PROFILE",}


delta_object_cursor = ""
delta_attach_after = true
delta_nestedKey_cursor = ""
-- Changed to new-style template bodies. Error messages tend to be a tiny (but sometimes very helpful) bit better this way.
-- Remember: These functions may not have any upvalues aside from _ENV (which is automatically manged by the VM).
local socontext = function(objectId, insertBeforeOrAfterContext, nestedKeyTokenId)
   objectId, insertBeforeOrAfterContext, nestedKeyTokenId = rubble.expandargs(objectId, insertBeforeOrAfterContext, nestedKeyTokenId)
   -- Expanding variables here was unnecessary, rubble.targs already handled that (and now rubble.expandargs does instead)
   delta_object_cursor = objectId
   delta_nestedKey_cursor = nestedKeyTokenId
   -- If statements that set a boolean are an abomination :P
   -- In addition to eliminating the if statement of which we will not speak, I also added some minor flexibility.
   delta_attach_after = ({["before"] = true, ["after"] = true})[string.lower(after)] ~= nil
end
rubble.template("!CONTEXT", socontext) -- <- Bad idea. Can lead to evaluation order problems (such as plague !SHARED_OBJECT_DUPLICATE). Not saying this should be removed, but the docs need an extra warning for this version.
rubble.template("@CONTEXT", socontext) -- <- Also a bad idea, but maybe not as bad as the other one...
rubble.template("CONTEXT", socontext)
rubble.template("#CONTEXT", socontext)
rubble.template("#C", socontext) -- <- This abbreviation is probably a bad idea.
-- For one thing "C" is the comment template, for another such abbreviations should probably be reserved for certain
-- very well known standard templates (there is too much chance of collision otherwise). Normally I would also complain
-- about the template names not being namespaced in some way, but as it is very likely that these templates will become
-- standard at some point in the future it is better that they are not.
--
-- While namespacing is a bad idea in this case, the current names may be too simple for the standard library. Something
-- more (specifically) descriptive may be in order.

local sodelta = function(tokenId, value, preraws)
   tokenId, value, preraws = rubble.expandargs(tokenId, value, preraws)
   
   -- Added this line to make sure "preraws" is a string (for later concatenation).
   -- This is needed because the new-style template bodies leave optional parameters that are not provided set to nil.
   -- I thought that rubble.expandargs took care of this little detail, but it doesn't.
   -- In v8 rubble.expandargs always returns strings, so this line may be removed then.
   preraws = preraws or ""
   if tokenId == "" then
      -- Use the function, Luke! <- Milo
      -- Aba -> Star Wars citations are the best ;)
      rubble.libs_base.sharedobject_add(delta_object_cursor, preraws)
      return
   end
   -- Moved the warning message here. Where it was it would print for every single tag that did not match the token,
   -- when clearly the effect you wanted was for it to print only if the tag you wanted could not be found.
   -- I split the warning into three parts, one here that triggers if the object does not exist, one later that triggers
   -- if the tag does not exist, and one that triggers if more than one tag matches. Change to taste :)
   if rubble.registry["Libs/Base:!SHARED_OBJECT"].table[delta_object_cursor] == nil or (delta_nestedKey_cursor ~= "" and rubble.registry["Libs/Base:!SHARED_OBJECT"].table[delta_nestedKey_cursor] == nil) then
      -- Is there a particular reason why this doesn't cause an abort? <- Milo
      -- Aba ->The delta are intended to be applied most of the time over Toady raws, from one version to the next he can decide to deprecate some objets.
      -- From the modder point of view it is way more util to have a list of ALL those rejects indicating the file and line number of each one of them
      -- than being obligated to do 100 independent Rubble generations to correct eventually one by one, these hypothetical 100 delta rejects.
      -- Rubble should never abort, except if the integrity of his internal state is compromised without the shadow of a doubt, and there, we are not in this case.
      -- A delta non applied must be logged and investigated by the modder, but does not compromise the rubble state, the only thing that rubble should do is continue
      -- and offer to the modder, all the disponible info to help the modder to do his work!
      rubble.warning("Object: \""..delta_object_cursor.."\" not found,  {@DELTA;"..tokenId..";"..value.."}: Malformed delta operator call or incorrect order of addon application.\n")
--      re-added   {@DELTA;"..token..";"..value.."} to give to the modder the most complete info about the failed delta application,
--      Would I know how, I would also added the file and line where the unapplied delta is, and even log it also to a special file of delta rejects.

      delta_nestedKey_cursor = ""
      return
   end

   local found_object, found_nestedKey, warned = false, false, false
   -- |
   -- V Changed this line to create a new variable instead of reusing "token" (since the original version of "token" is needed later)
   local stoken, nested = string.split(tokenId, ":"), string.split(delta_nestedKey_cursor, ":")
   rubble.libs_base.sharedobject_walk(delta_nestedKey_cursor, function(tag)
      if rubble.libs_base.matchtag(tag, nested) then
         if found_nestedKey and not warned then

            rubble.libs_base.sharedobject_walk(delta_object_cursor, function(tag)
               if rubble.libs_base.matchtag(tag, stoken) then
                  if found_object and not warned then
--   this should not be a problem for the modder in most ocasions, because he have the possibility to indicate how many of the params of a token form the lvalue.
                     rubble.warning("Multiple tags fitting the specifier: \""..tokenId.."\" Found in object: \""..delta_object_cursor.."\". This may indicate an improper use of the delta operator.\n")
                     warned = true
                  end
                  if value == "" then
                     local preraws = "-"..tag.ID
                     for _, v in ipairs(tag.Params) do
               preraws = preraws..":"..v
            end
                        tag.Comments = preraws.."-"..tag.Comments
                  else
                     local fulltoken = "["..tokenId..":"..value.."]"
                     if delta_attach_after then
                        tag.Comments = preraws..fulltoken..tag.Comments
                     else
                        tag.Comments = fulltoken..preraws..tag.Comments
                     end
                  end
                  tag.CommentsOnly = true
                  found_object = true
               end
            end)
         end
      end
   end)
   if not found_object then
      -- It may be a good idea to allow modders to suppress this warning for certain calls, I cant decide if its worth it or not...
      rubble.warning("A tag fitting the specifier: \""..tokenID.."\" Could not be found in object: \""..delta_object_cursor.."\". This may indicate an improper use of the delta operator.\n")
   end
   delta_nestedKey_cursor = ""
end
rubble.template("!DELTA", sodelta) -- See earlier comments.
rubble.template("@DELTA", sodelta) -- See earlier comments.
rubble.template("DELTA", sodelta)
rubble.template("#DELTA", sodelta)
rubble.template("@D", sodelta) -- See comment about previous abbreviation.
« Last Edit: May 01, 2016, 02:54:47 pm by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

Dirst

  • Bay Watcher
  • [EASILY_DISTRA
    • View Profile
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #110 on: April 24, 2016, 08:51:27 pm »

I would like to add a tag to some vanilla creatures, and this seems like the perfect situation to use BAMM syntax.  The problem is, I can find any useful documentation for BAMM rules.  Does anyone know where that might be lurking?
Logged
Just got back, updating:
(0.42 & 0.43) The Earth Strikes Back! v2.15 - Pay attention...  It's a mine!  It's-a not yours!
(0.42 & 0.43) Appearance Tweaks v1.03 - Tease those hippies about their pointy ears.
(0.42 & 0.43) Accessibility Utility v1.04 - Console tools to navigate the map

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #111 on: April 25, 2016, 03:28:42 pm »

I would like to add a tag to some vanilla creatures, and this seems like the perfect situation to use BAMM syntax.  The problem is, I can find any useful documentation for BAMM rules.  Does anyone know where that might be lurking?
Two possibilities:
1. Look at the original docs and examples in the BAMM download.
2. If those some vanilla creatures are a exhaustive list, you could do like one of the following fragments of my Rubble port of all races playable mod included in HDFPS, where also other mods are also found with use cases of working Rubble commands that give better insight than the actual skeleton of rubble tutorials:
Code: [Select]
{@FOREACH;DWARF=0|ELF=0|GOBLIN=0|KOBOLD=0|CAVE_FISH_MAN=0|OLM_MAN=0|BAT_MAN=0|CAVE_SWALLOW_MAN=0|AMPHIBIAN_MAN=0|REPTILE_MAN=0|SERPENT_MAN=0|ANT_MAN=0|RODENT MAN=0;
{SHARED_OBJECT_ADD;CREATURE:%{key};
[OUTSIDER_CONTROLLABLE]}}
{@FOREACH;TROLL=500|OGRE=500|BLIZZARD_MAN=500|GRIMELING=250;
{SHARED_OBJECT_ADD;CREATURE:%{key};
[PET_EXOTIC]
[TRAINABLE]
[PETVALUE:%{val}]}}
{@FOREACH;WOLF_ICE=0|BLENDEC_FOUL=0|STRANGLER=0|NIGHTWING=0|HARPY=0|SEA_MONSTER=0;
{SHARED_OBJECT_ADD;CREATURE:%{key};
[PET_EXOTIC]
[TRAINABLE]}}
{SHARED_OBJECT_ADD;BEAK_DOG;[PET_EXOTIC][TRAINABLE][PACK_ANIMAL][WAGON_PULLER][TRADE_CAPACITY:2000]}
#In the file item_weapon.txt
{@FOREACH;PIKE=0|HALBERD=0|SWORD=0|MAUL=0|AXE_GREAT=0;
{@SET;OBJECT_CURSOR;ITEM_WEAPON:ITEM_WEAPON_%{key}}
{!SET_VALUE;MINIMUM_SIZE;57500}}
#In the file entity_default.txt
You only need to change the token [OUTSIDER_CONTROLLABLE] of the first @FOREACH by the token that you needs to add and substitute my list by yours...
And remember you must place those things in .rbl or .txt files only, maybe having a look at a complete working example could help you, for that a pre release candidate download is disponible.
« Last Edit: April 25, 2016, 03:47:47 pm by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #112 on: April 28, 2016, 12:24:31 pm »

The problem is, I can find any useful documentation for BAMM rules.  Does anyone know where that might be lurking?

Umm.. "Rubble Basics"? Look for something to do with the "Raw Merger Rule Format". There's a nice long section about halfway down the page I think.

(Lots of stuff)

If you want complicated sub-object targeting you have two choices: Write a system to do it (not too hard as long as you don't try to hardcode all the possible sub-objects), or use the raw merger (which has a powerful system for handling sub-objects as defined in your rules).

It looks like you want something with the power of the raw merger, but more generic. I have thought about making an extension to the system that would allow it to load raws into a tree based on supplied rules. This would allow easier sub-object editing, but only for objects with complete rules (maybe half of all objects have complete rules as part of the raw consistency checker). I'll play around with the idea some when I get home, it could be the answer to all your wants :)



@Abadrausar: I finally got an uncorrupted version of your project. Here are some thoughts:

Code: [Select]

* You distributed the Windows version of DF, but included the non-Windows Rubble binaries. Removing these would greatly
  reduce download size.
* What did you do to those poor addon names?! Use spaces not underscores!
* When changing addon names make sure there are no links that still refer to the old name. All the First Landing workshop
  previews are broken, and the change log link in TESB is broken (those are just the first two I found).
* Changing addon names is a bad idea in any case (as it is confusing to have the same thing under different names).
* Your markdown is wrong (see the section at the end of this file).
* Why do your "separator addons" require `Libs/Base`? Not that it hurts anything, but it seems weird...
* TESB's addon name should be in Title Case.
* Trying to click on one of the dependencies of `Milo___First_Landing` (specifically `Libs/Base/First Landing`) results
  in a log page redirect (as that addon does not exist).
* This would cause an abort if you ever tried to generate FL.
* That addon name should really be `Milo/First Landing`, the one it has is just ugly...
* Many of your addon.meta files include tags that are set to false. It would be better to simply remove these tags.
* The way you split some of the vanilla stuff up defies logic...
* I don't like the "composable base" idea at all. It's too hard to see what is included in any given addon.
* A "lite" version with just abstract objects (languages, material templates, tissues, body parts, etc) would have
  my support, but leave entities, creatures, inorganics, and reactions up to each individual base.
* In any case this really needs a "remove shared object" template, I'll add that to v8 right now (done, SHARED_OBJECT_DELETE
  coming in v8!).
* Many small addons are included, but not exposed to users (for example the FL powered workshop addons).
* Some of the addon.meta files look... Mangled? For example `Dev/Dummy/Workshops` has had it's header replaced with the
  display name of it's first config var.
* Why are there two copies of `Dev/SO Insert`?

This looks like an interesting project, but very rough and in dire need of testing and polish.

* * *

I feel the need to expand on the problems with your addon.md files, namely that they use improper markdown. Markdown
needs blank lines between elements.

I strongly suggest you get an editor with spell check, misspelled words look bad and are easy to avoid with the right
tools (I use Notepad++ with a spell checker plugin, but most good editors should support something similar).

Not sure if it's your mistake or the original mod author's, but "drows" is not a word. "Drow" is both singular and plural.

The link to the original mod is wrong, the one given is for "Dark Age II".

The following focuses on the header (although the body of the file could use some work as well).

Here is what you have:

[Addon Change log](/addonfile?addon=Rhenaya___Drow&file=addon_changes.md)
![Rhenaya Drow Civilization](http://scout.es/wp-content/uploads/haiku2b.jpg)
## So you think that you need some help, want to do a sugestion, inform about a bug or give thanks to modders or developpers, but not sure where to go.
1. If you want to patronize Toady One and the developpment of Dwarf Fortress go to: [Support Dwarf Fortress Vanilla](http://www.bay12games.com/support.html)
2. For questions about the rubble framework go to: [Rubble framework](http://www.bay12forums.com/smf/index.php?topic=154304.0)
3. Anything about HDFPS: Humble Dwarf Fortress Publishing System is there [HDFPS](http://www.bay12forums.com/smf/index.php?topic=157300.0)
4. If your question is addressed to the actual original maintainer of this mod then you have [Dark Age II R3 for DF42.06](http://www.bay12forums.com/smf/index.php?topic=143540.0)
# Rhenaya Drow Civilization
Description:  This mode includes a standalone playable drow race.
On embark you can choose them too (just like changing through dwarf civs in vanilla, it adds occasional drow civs).

...

This is what it should be (including fixed spelling mistakes):

[Addon Change log](/addonfile?addon=Rhenaya___Drow&file=addon_changes.md)

![Rhenaya Drow Civilization](http://scout.es/wp-content/uploads/haiku2b.jpg)

## So you think that you need some help, want to do a suggestion, inform about a bug or give thanks to modders or developers, but not sure where to go.

1. If you want to patronize Toady One and the development of Dwarf Fortress go to: [Support Dwarf Fortress Vanilla](http://www.bay12games.com/support.html)
2. For questions about the rubble framework go to: [Rubble framework](http://www.bay12forums.com/smf/index.php?topic=154304.0)
3. Anything about HDFPS: Humble Dwarf Fortress Publishing System is there [HDFPS](http://www.bay12forums.com/smf/index.php?topic=157300.0)
4. If your question is addressed to the actual original maintainer of this mod then you have [Dark Age II R3 for DF42.06](http://www.bay12forums.com/smf/index.php?topic=143540.0)

# Rhenaya Drow Civilization

Description:  This mod includes a standalone playable drow race.
On embark you can choose them too (just like changing through dwarf civs in vanilla, it adds occasional drow civs).

...

How I would write it:

[Addon Change log](/addonfile?addon=Rhenaya___Drow&file=addon_changes.md)

![Rhenaya Drow Civilization](http://scout.es/wp-content/uploads/haiku2b.jpg)

## Helpful Links:

* Support development of everyone's favorite game! [Support Dwarf Fortress](http://www.bay12games.com/support.html)
* For Rubble questions/help visit [the Rubble thread](http://www.bay12forums.com/smf/index.php?topic=154304.0)
* For help with this port see [HDFPS: Humble Dwarf Fortress Publishing System](http://www.bay12forums.com/smf/index.php?topic=157300.0)
* Questions about the original mod should be directed to: [Dark Age II R3 for DF42.06](http://www.bay12forums.com/smf/index.php?topic=143540.0)

# Rhenaya Drow Civilization

Description:  This mod includes a standalone playable drow race.
On embark you can choose them too (just like changing through dwarf civs in vanilla, it adds occasional drow civs).

...




Rubble 8.0.0 is up!

This version brings some really nice new features, take a look at the changelog in the second post for details.
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile

Oh, I forgot to mention that the way things work now, if an addon pack provides an update URL Rubble will make a HEAD request to this URL every time it starts. Unfortunately DFFD appears to count each of these requests as a download. This is obviously undesirable, and I am trying to figure out a way to avoid it.
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫
Re: Rubble 7.7.1 - DF 42.6 - Automatic remote dependency download!
« Reply #114 on: April 29, 2016, 09:45:20 am »

It looks like you want something with the power of the raw merger, but more generic. I have thought about making an extension to the system that would allow it to load raws into a tree based on supplied rules. This would allow easier sub-object editing, but only for objects with complete rules (maybe half of all objects have complete rules as part of the raw consistency checker). I'll play around with the idea some when I get home, it could be the answer to all your wants :)
complete rules as of now in DFkeyToken.lua:
Spoiler (click to show/hide)
They don't need in reality to be exhaustive to enable the actions (delete, edit,add,..) over the most interesting nested subobjects, this sparse array will be exploited in the AST walker matchtag(tag, match) or potentially in other places.
Spoiler (click to show/hide)
...@Abadrausar: I finally got an uncorrupted version of your project. Here are some thoughts:...
Thank for your ideas, I am happy to see that some of the problems that you have identified in RC1 are already given some treatment in released RC2 of HDFPS.
Thanks to the reorganization of Release Candidate 2, the content of mods or addons are separated into visual and active parts, into the Base and Libs subdirs respectively, that enables separation of concerns, and also creating hierarchies of dependencies between decomposable mods, that helps reuse of content and maintenance. Specially when combined with a working semantic Delta system able to access to most nested objects and operate over them in generic ways.
As you can imagine for someone that is more interested in being able to safely remove redundant content but preserving all vanilla mechanisms (haiku vanilla, DF from scratch,) to add new unique content, my principal motivation to migrate other mods is stress test Rubble capacities of each released version to be a complete system to organize, combine and distribute original DF content. Each new mod comes with new problems that needs some solution, but also reveal some hidden opportunities, possible patterns or conventions that could be of any use for publishing.
HDFPS RC1: had a weight of 31.54 Mb, after some of your suggested eliminations RC2 in DFFD  only 19.27 Mb with much more content.
In 2nd post of HDFPS a code of colors is used to indicate the state of each mod, by now the only safe color  to use or play is green, rest of colors are there for works in progress, FL for example is in red indicating that the adaptation is unfinished, and that it is there for preview purposes only.... Maybe I should also use that same code of colors in the rubble generated list, it is that possible?
If an addon could change the format of what is generated for him in the rubble interface at the generation phase one way to do it could be:
Code: [Select]
"Vars": {
"INTERFACE_COLOR": {
"Name": "Color",
"Values": [ "", "RED", "GREEN" ]},
"INTERFACE_SIZE": {
"Name": "Size",
"Values": ["SMALLER", "SMALL", "NORMAL", "BIG", "BIGGER" ]},
"INTERFACE_ALIGN": {
"Name": "Align",
"Values": [ "LEFT", "CENTER", "RIGHT" ]}}
Another maybe better:
Code: [Select]
{"Tags": {"Format": "COLOR:GREEN:SIZE:BIG:ALIGN:CENTER"}}Not really important, only aesthetic concern from an OCD person to another ;) there is a way to make separators or Titles in the generation not selectables?
If they could be attached as non active parts before or after the generated interface line of another addon via tag that would be cool...
« Last Edit: May 01, 2016, 03:10:26 pm by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫

Oh, I forgot to mention that the way things work now, if an addon pack provides an update URL Rubble will make a HEAD request to this URL every time it starts. Unfortunately DFFD appears to count each of these requests as a download. This is obviously undesirable, and I am trying to figure out a way to avoid it.
The next two are the link of download in DFFD of First Landing and its JSON file containing info relative to the last upload of the zip, respectively
http://dffd.bay12games.com/file.php?id=11605
http://dffd.bay12games.com/file_data/11605.json
Inside the JSON file. that is directly accesible in the cited URI, we have:
{"version":"1.8","updated_timestamp":"1461864623","updated_date":"Apr 28, 2016, 12:30:23 pm","filename":"First Landing.zip","size":"334600","author":"milo christiansen","version_df":"0.42.06","sha256":"401efaf8050566078f07dba4ef094796b93ee5ca46541c28de251aba61f5ec50","rating":"0.0","votes":"0"}
So that means that DFFD is by design able to do what we want. The trick is reading the DFFD JSON file before, to decide if the zip should be downloaded or not! The JSON file you can read it as many times as you wish, as this does not affect ni the counts ni the views. One working procedure would be:
1.) We read and decode the JSON file.
2.) For what it concern DFFD, First Landing is 11605 while The Earth Strikes Back! is 11912 and starting from this correspondence, both the zip filename and the JSON filename can be generated. So we could give good use to one permanent table (database) that backup this correspondence for each mod in the Rubble DFFD catalog system, for posterior reuse.
3. If to this table containing now at least 2 rows, 1st the mod names and 2nd the DFFDid we add a 3rd row the "updated_date":
4. All that is needed to know if the zip file of a mod must be downloaded is to compare the "updated_date": key of the local JSON database for this mod with those of the DFFD JSON file for the mod.
5. This local JSON database will be acting as a central catalog that we could even distribute as an addonpack in DFFD with auto update capacity.
6. This would mean that Rubble itself does not need to be updated only to include new mods in the catalog.
7.) Follow example of such local JSON generated database file (acting for us like a mod catalog) with all the rubble mods actually published ;)
Quote from: Addons.catalog
{"Addons":[
{"version":"1.8","updated_timestamp":"1461864623","updated_date":"Apr 28, 2016, 12:30:23 pm","filename":"First Landing.zip","size":"334600","author":"milo christiansen","version_df":"0.42.06","sha256":"401efaf8050566078f07dba4ef094796b93ee5ca46541c28de251aba61f5ec50","rating":"0.0","votes":"0"},
{"version":"2.02","updated_timestamp":"1459662320","updated_date":"Apr 03, 2016, 01:45:20 am","filename":"The Earth Strikes Back!.zip","size":"49295","author":"Dirst","version_df":"Multiple","sha256":"d8ce4e96e8a4ea37262b465352609390af54c7c56a2ea87ffcfda5538b7a2b65","rating":"0.0","votes":"0"}
]}
8.) Ideally one catalog should be able to activate other catalog with any of their addon items.
9.) This could be implemented in lua better than in go for maintenance reasons, and the performance should not be a problem, except if the number of rubble DFFD supported mods go over the thousands.
10.) Point 8 can be attained exploiting a little the file extension registration system, we only have to register a .catalog file extension containing the plain text JSON addons database described.
11.) The example catalog database given in point 7 is intended to show how easy is to build the database combining the JSON that you read from DFFD for each Rubble addon with the green glue Delta parts used inside the Addons.catalogs, however it does not add the key DFFDid described in point 3, where we save 11605 and 11912 respectively as described in point 2, so the complete functional catalog as it has been proposed would be instead:
Quote from: Addons.catalog
{"Addons":[
{"version":"1.8","updated_timestamp":"1461864623","updated_date":"Apr 28, 2016, 12:30:23 pm","filename":"First Landing.zip","size":"334600","author":"milo christiansen","version_df":"0.42.06","sha256":"401efaf8050566078f07dba4ef094796b93ee5ca46541c28de251aba61f5ec50","rating":"0.0","votes":"0","DFFDid":"11605"},
{"version":"2.02","updated_timestamp":"1459662320","updated_date":"Apr 03, 2016, 01:45:20 am","filename":"The Earth Strikes Back!.zip","size":"49295","author":"Dirst","version_df":"Multiple","sha256":"d8ce4e96e8a4ea37262b465352609390af54c7c56a2ea87ffcfda5538b7a2b65","rating":"0.0","votes":"0","DFFDid":"11912"}
]}
12.) While working in a possible JSON Hyper-Schema for the catalog I have remarked that it would be way easier to work with updated_timestamp than with updated_date (as proposed before) for auto updating purposes, while functionally equivalent. I have also defined one optional array of tags, if we need it at some point...
Quote from: Set of Addons schema:
{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "title": "Addons set",
    "type": "array",
    "items": {
        "title": "Addon",
        "type": "object",
        "properties": {
            "version": {"description": "The version tag of an Addon", "type": "string"},
            "updated_timestamp": {"description": "Easier to work with this for auto updating purposes than having to decode and compare two dates in textual form, Argh!", "type": "number", "minimum": 0, "exclusiveMinimum": true},
            "updated_date": {"type": "string"},
            "filename": {"type": "string"},
            "size": {"type": "number"},
            "author": {"type": "string"},
            "version_df": {"type": "string"},
            "sha256": {"type": "string"},
            "rating": {"type": "number"},
            "votes": {"type": "number"},
            "DFFDid": {"description": "The internal ID of a mod in DFFD, best in string form for generation purposes", "type": "string"},
            "tags": {
                "type": "array",
                "items": {
                    "type": "string"
                },
                "minItems": 0,
                "uniqueItems": true
            },
        },
        "required": ["updated_timestamp", "filename", "DFFDid"],
      "links": [
         {"rel": "Archive file", "href": "http://dffd.bay12games.com/file.php?id={DFFDid}"},
         {"rel": "JSON file", "href": "http://dffd.bay12games.com/file_data/{DFFDid}.json"}
      ]
      }
}
« Last Edit: May 04, 2016, 08:42:10 am by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile

(Some very interesting and useful stuff...)

That is exactly what I needed! I'll add a pack.meta key to specify a DFFD file ID instead of a URL for updates. URL's will no longer work for 8.1.0 (once I upload it... Coming soon!), you will either need to use DFFD and provide an ID or you will need to use a content server (or both).

Content server? Yup. I finally finished a working content server for Rubble. This server stores metadata for addon packs, such as what DF and Rubble versions they are for, what the addon pack version is, where the pack may be downloaded from, a short description, etc...

Of course for such a thing to be useful someone needs to host and administrate it, and sadly that would be impossible for me to do (so being able to query DFFD for info will be an invaluable stopgap).

The server right now is suitable for long uptimes, but it lacks features it would need to run unsupervised. I need to make a better version with more powerful log handling and an integrated server monitor (nothing I haven't done before).

Enjoy some logs showing me testing the current server!
Code: [Select]
# I present two logs from the same session, one from a Rubble instance starting the web UI and immediately
# exiting and one from a content server that was up at the same time.
#
# I had previously listed First Landing on the server so I could test all aspects of finding and loading
# an addon based on information from the server (the pack updated itself on a previous run).
#
# Sometimes the random startup lines are awesome!
# I like what the client has to say, but I really hope the server isn't trying to tell me something...
# (Probably that being up at 2:45 working leads to bugs (log time is UTC, not local))

--------------------------------------------------------------------------------------------------

Rubble v8.1.0 Internal Test Build
Feedback is greatly appreciated!
================================================================================
Starting Universal Interface...
  Attempting to Read Config File: ./rubble.ini
    Read OK, loading options from file.
  Current DF version: 0.42.6
    Using hardcoded version.
    Version confirmed by reading "df/release notes.txt".
  Calling action for mode: web
================================================================================
Loading Addons...
  Loading Global Settings...
  Attempting to Read Auto-Update Blacklist...
    Read OK, loading...
  Attempting to Read Content Server List...
    Read OK, loading...
  Loading Addons From Local Directories...
  Loading Addons From Globally Requested Addon Packs...
  Loading Addons From Addon Packs...
    Loading Pack: FL-Mawrings
      Trying to find update...
        Querying content servers...
        Could not find URL for this pack on any content server. Using local copy.
      Checking versions...
        OK!
        Loading...
    Loading Pack: First Landing
      Trying to find update...
        Querying content servers...
          Found candidate, checking versions...
            Local copy matches remote copy.
      Checking versions...
        OK!
        Loading...
    Loading Pack: The Earth Strikes Back!
      Trying to find update...
        Querying content servers...
        Could not find URL for this pack on any content server. Using local copy.
      Checking versions...
        OK!
        Loading...
  Checking Loaded Data...
================================================================================
Creating New State...
  Loading Special Global Files...
    Reading DF Init Files...
  Creating Script Runners...
================================================================================
Starting the Default Web Browser...
  Attempting to Open: "http://127.0.0.1:2120/menu"
Starting Server...
UI Transition: "/menu"
================================================================================
Creating New State...
  Loading Special Global Files...
    Reading DF Init Files...
  Creating Script Runners...
UI Transition: "/kill"

--------------------------------------------------------------------------------------------------

Rubble v8.1.0 Internal Test Build
Guaranteed 50% bug free!
================================================================================
Starting Universal Interface...
  Attempting to Read Config File: ./rubble.ini
    Read failed (this is most likely ok)
      Error: open ./rubble.ini: The system cannot find the file specified.
      Using hardcoded defaults.
  Current DF version: 0.42.6
    Using hardcoded version.
    Could not confirm by reading "df/release notes.txt".
    If the stated version is not correct restart Rubble with
    the -dfver option or by setting the dfver key in rubble.ini
  Calling action for mode: cntntsrvr
================================================================================
Starting Server...
  [16/05/03 6:45:54] A: "Info" U: "" P: "FL-Mawrings" DF: 0.42.6 Rubble: 8.1.0 Status: Failed: Could not find match.
  [16/05/03 6:45:54] A: "Info" U: "" P: "First Landing" DF: 0.42.6 Rubble: 8.1.0 Status: OK!
  [16/05/03 6:45:54] A: "Info" U: "" P: "The Earth Strikes Back!" DF: 0.42.6 Rubble: 8.1.0 Status: Failed: Could not find match.

As you will notice Rubble now keeps track of the local DF version. It has a hardcoded default version, but this is confirmed or replaced by reading the DF changelog. If Rubble guesses the incorrect version you can always override it's choice, but this should never be necessary under normal circumstances.
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile

Oh, I forgot to mention:

8.1.0 will ship with a API that can parse raws into a tree (using raw merger rules). It actually makes a really nice pretty printer too :P Of course there is a script function or two that uses the native backend, actually the script API has more/better features than the native one (as the native API lacks the ability to dump a tree to a string).
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS

Abadrausar

  • Bay Watcher
  • empowering ideas
    • View Profile
    • ♫♪♀HDFPS♂♪♫

... 8.1.0 will ship with a API that can parse raws into a tree (using raw merger rules). It actually makes a really nice pretty printer too :P Of course there is a script function or two that uses the native backend, actually the script API has more/better features than the native one (as the native API lacks the ability to dump a tree to a string).
Interested in the tree thing especially if the implementation is done or Lua accesible :D that could help delimiting the nested sub objects, at least the first level of those...

Milo I have placed online all Rubble 8.0 documentation in ♫♪♀HDFPS♂♪♫ as it relies always on the more recent version of Rubble, you could maybe reuse the URIs and place them in one of your firsts messages in the Rubble forum, as there they make even more sense than in ♫♪♀HDFPS♂♪♫... This is also a petition of immutability of the Rubble directories and filenames of the docs and tutorials to evite unnecessary breakage of the links, the content however you can mute as much as you can  ;D



Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ related places
1. Changelog
2. Git Repository
3. Rubble forum

Download for:
Windows
Linux
OSX Yes! You have it right,  we support every Dwarf Fortress supported platform, you only need a web browser


Online Rubble
Documentation:
Basics
Easy Modding
Conventions
Base Templates
Lua Scripting
History and Philosophy
Standard Library Templates
License
Changelog


Tutorials:
1. Common standard library templates
2. Rubble as a tileset applier
3. Tileset addons
4. User templates

Mods:
First Landing   
The Earth Strikes Back!   

Under construction, please wait!
This is a work in progress and (as may be assumed reasonably enough) may be highly unstable just yet.
[/td][/tr][/table][/td][/tr][/table][/quote][/td][/tr][/table][/td][/tr][/table]
« Last Edit: May 07, 2016, 04:01:54 am by Abadrausar »
Logged
::: Humble Dwarf Fortress Publishing System ♫♪♀HDFPS♂♪♫ Mods Push Published in DFFD are auto updated in local Players Catalog :::

milo christiansen

  • Bay Watcher
  • Something generic here
    • View Profile

Sorry, I won't finalize anything about the docs, they tend to be fairly volatile for a reason (namely that I am always changing and adding stuff).

I don't feel the need for online docs, if you need documentation you probably already have Rubble, if you have Rubble you have the documentation. I HATE projects that require you to either download documents separately or only have them online, and this is the first step in that direction.

I just happened to notice the earlier request you made for allowing an addon to effect how it's entry is generated for the web UI:

If you want that kind of feature you have two choices: 1) you can already format the description line however you want (markdown allows embedded HTML) 2) You can edit the HTML templates to add the extra abilities you want.

I won't be adding such features to the default UI, as I really suck at HTML and won't touch it unless I have to (plus I think such formatting could be open to abuse by obnoxious addon authors).



8.1.0 will be up soon, probably early next week!
Logged
Rubble 8 - The most powerful modding suite in existence!
After all, coke is for furnaces, not for snorting.
You're not true dwarven royalty unless you own the complete 'Signature Collection' baby-bone bedroom set from NOKEAS
Pages: 1 ... 6 7 [8] 9 10 ... 14