I was hoping for a way to list all preferences by category then specifics and scroll left to right to see them all, maybe expand them using a +/- box dialog.

I am not sure what you are describing. Scrolling won't do it: some categories have hundreds of items (see the custom role editor dialog). The grid view editor will need to be redone to be usable with preferences, using a filter search like the custom role editor. The current nested context menus cannot contains the hundreds of creature and material preferences.

Spreadsheet Berserker

Indeed. Sorry, I cannot comment your data, I have no idea what is going on here.

If you are the statistics expert, do you any idea what could be causing this issue.

I apologize for the lack of context. I worked with Splinterz back with v13 of Dwarf Therapist to help derive scores based on attributes, traits, skills, preferences, etc. I have intimate knowledge of how the role ratings are derived (at least up till v34). I was unaware the project was transferred to a new lead until yesterday.

Either way. The current way raw values are converted to %'s I believe use a transformation from min/max around the average, then from min/max around the median, then combined with an empirical cumulative distribution function to get a %. These %'s are then fed into a weighted sum algorithm to get role ratings.

I was proposing a new method that uses a simpler weighted standard deviation approach around the median (similar to a median/median absolute deviation approach) that allows for allowance for skewed distributions (i.e. extremely large values in a data set, i.e. outlier's).

The new proposed method doesn't do any weird transforms around the average/median that would leave the rest of the dataset not proportionally accurate (i.e. when transforming around different centers, the areas around the center get shifted to varying degrees, so the distances around the center are not the same on either side of the center). With this new proposed method, the distances would be equal always, plus the added benefit that there is only one center now (the median)

I could code it directly myself. I'm a bit slow though, I created the first prototype for the labor optimizer in v17 (took me two weeks and Splinterz said it was more or less a superfunction of spaghetti code). I generally make mockups in excel for review. That file I recently uploaded is a bit messy as I was trying a lot of things, but I could pair it down.

~~Alternatively, I was looking at using Kernel Density Estimates (and maybe integrating R) to derive probability distributions.~~ Scratch that, KDE's are basically Empirical Cumulative Distribution Functions and do not measure the distance between values. However, this method is much much much easier.

And yes, I am somewhat of a stats expert, I'm currently in a masters for Data Science, but the issue with that is, most methods for normalizing assume a NORMAL DISTRIBUTION, and the data that Dwarf Fortress produces isn't normal. So parametric methods have to be used. This I guess is a novel/hackey parametric method by using a weighted sum algorithm for the standard deviation which in turn allows for capturing the larger values appropriately while maintaining a ~0 (min) to 50 (median) to ~100% spread (max).

I could create a complete mockup of roles in excel so you can get a better idea. I could do a comparison with old vs new. Or I could just scratch that and attempt at coding a prototype. It's been a while, and if I do, it will probably take me a minute (a month?)

Edit:

As to the bug... not sure. I noticed a bug a while ago in the way roles were calculated that I would also like to address (especially if I'm proposing a new method to calculate roles).

I would love to work with you if you are willing. I can do qa testing and work on developing a newer version. I can even deliver mockups before [if] we go that route.