Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2 3 ... 9

Author Topic: The Magmawiki Project  (Read 21214 times)

Locriani

  • Bay Watcher
  • Locriani == Briess
    • View Profile
    • dwarf fortress wiki
The Magmawiki Project
« on: April 26, 2010, 07:36:01 am »

What is Magmawiki?
Magmawiki is a new Ruby on Rails wiki engine that Emi and I are currently working on.  Our goal is to eventually replace the mediawiki engine (that we're currently using at the dwarf fortress wiki) with something considerably more stable, user-friendly, with maintainable and extensible code.

It's also open source (modified MIT license - you're not allowed to use my name, Emi's name, or my company's name for advertising; otherwise, everything goes) and has a rockin' cool name.

Why should I care?
Well, you can take a sneak peek at some of the changes we're implementing with the new skin called Magmawiki on the current wiki (http://df.magmawiki.com/index.php/Special:Preferences to change your skin).  However, our core ideas go far beyond just a friendlier skin.  For a look at the features we're planning on implementing (before we even get to version 1!), check out http://df.magmawiki.com/index.php/Dwarf_Fortress_Wiki:Magmawiki.

That's not good enough.  Why are you doing something so crazy?
Mediawiki has a plethora of security holes and issues, and has the most tangled code base on the face of the planet.  That's fine, except the team behind Mediawiki isn't likely to fix these issues UNLESS it affects the wikimedia foundation's projects -- and a number of the bugs we've run into don't apply to, say, wikipedia.  Additionally, php is a very buggy and poorly designed programming language, which has the potential of opening up a significant number of security holes.

We looked at some of the other wiki solutions (including MoinMoin, Instiki, and a very ... hilarious meeting with Wagn), but none of them offer the features we've come to depend on at the dwarf fortress wiki.

OMG so you're going to make us enter in pages again by scratch?
No.  One of the required features before we release is an importer from mediawiki databases to the magmawiki system, including a mediawiki parser.  Until this is working perfectly (well, within reason), we will not be forcing a switch to magmawiki.

You mentioned features, what do you mean?
Well, high up on the list are a better templating system and saner markup.
  • Parser:
    Potential parser options (markup) include markdown and textile.  Anyone who's messed with the core mediawiki syntax can appreciate how these would be an improvement.  Additionally, we are looking at other options (such as a javascript WYSI(almost)WYG editor that converts down to textile or markdown) for the less technically inclined.
  • Templates:
    The mediawiki template syntax is balls terrible.  Don't believe me? Here's a simple example:
    Spoiler (click to show/hide)
    Sure it could be cleaned up a bit, but if you mess with the spacing at ALL on that template, it breaks, horribly.  So, whats our solution?
    • Ruby
    • Python

    Yes, we're going to allow templates written in sane code, with restricted runtime environments and cached output.  Readable templates are significantly more palatable and editable, and less likely to induce stress killing sprees when trying to figure out why a template stopped working.
Why Ruby on Rails? Isn't that a hipster fad?
Maybe I'm a hipster  ;).  The reason we're using Ruby on Rails is because I have significant experience with programming with that web framework.  Illideri Studios (my company) uses it for the majority of our web-based stuff, and it works well.

Cool, what are the caveats?
No IE support. Not until after version 1.
Quote from: Emi
We'll develop to version 1 without checking it in IE7/8 and then I'll go back (at my own leisure) and write some css and other stuff for compatibility.
I do not own (and will not ever buy) another copy of Windows.  Internet Explorer is also so backwards when it comes to web standards, that any attempt to maintain support with the browser results in the majority of your time spent fixing insanity inducing edge cases that ONLY affect IE.  Such time could be better used actually programming and getting it to work for the other browsers that actually adhere to the W3C recommendations and HTML / CSS standards.  If it still works in IE for you, great, but we're not going to be going out of our way at all to make it work right.

Advanced media features (such as videos) will require HTML5 and Javascript.  We're not embedding flash into the site - flash is too incompatible with too many systems to even bother trying; whereas HTML5 media is supported on devices including the iPhone, iPad and Android phones. Also, Adobe charges out the buttocks for the flash development software.  If you want to donate the $500 + for a commercial license, feel free - I'll turn right around and donate it to an animal shelter or something.

Also, we will be charging a tiny fee of 0.001 per page view, to reduce bandwidth consumption (this is a JOKE, this is not serious, I would never charge people to use the wiki!)
How long before this move takes place?
No idea.  Could be a few months before we're ready to try database dumps (that doesn't mean we're moving quite yet, though!), or it could be a year.  However, we will have at least a month of testing before we even BEGIN the move process over to the new wiki.  You'll have to recreate your user account, but all templates, pages, and page histories will remain intact.

Cool, can I help?
Sure!  We use github for the code repository (http://www.github.com/locriani/magmawiki) and invite code contributions.  We won't accept any code if it has not been unit tested, though.

Ways you can help:
  • Translations / Localizations
  • Programming
  • Skin designs
  • Bug testing
  • Discussion!

If you're not technically oriented (or, you're technically oriented and don't want to learn ruby), we can also use help with thoughts on the new skin (detailed above), bug testing (details to come soon), and feature testing (details to come soon).  If you just want to see what it looks like at this moment, you can get a sneak peak at http://99.35.187.9:3000/ - but I make no guarantees that will be functional, useable, pretty, or even online as that's the computer I'm using for development. 

Also, comments and criticism are welcome and encouraged.

This sucks, I hate you.
Sorry.  Life sucks sometimes; it is best to keep a positive attitude!
« Last Edit: May 01, 2010, 06:59:06 pm by Locriani »
Logged
I am one of many administrators of the wiki.  Please use my user page (http://dwarffortresswiki.org/index.php/User_talk:Briess) on the wiki to contact me, as I check that more often than these forums.

Shades

  • Bay Watcher
    • View Profile
Re: The Magmawiki Project
« Reply #1 on: April 26, 2010, 07:51:57 am »

Out of interest how much experience with high concurrency rails projects do you have. I imagine the wiki has a fair load and from my experience you hit scaling issues fairly quickly while using rails.

Markdown is my syntax of choice too :)

Just to note as long as your html syntax is fairly standard and works on firefox, opera and safari it will probably work on ie 7 and 8 without change (or with minor tweaks). I agree ie 6 is a b*tch to support, although nothing is as bad as ie 5.5 for the mac.

Also your links are broken.
edit: The markdown and textile ones
« Last Edit: April 26, 2010, 07:56:44 am by Shades »
Logged
Its like playing god with sentient legos. - They Got Leader
[Dwarf Fortress] plays like a dizzyingly complex hybrid of Dungeon Keeper and The Sims, if all your little people were manic-depressive alcoholics. - tv tropes
You don't use science to show that you're right, you use science to become right. - xkcd

Locriani

  • Bay Watcher
  • Locriani == Briess
    • View Profile
    • dwarf fortress wiki
Re: The Magmawiki Project
« Reply #2 on: April 26, 2010, 08:01:18 am »

Out of interest how much experience with high concurrency rails projects do you have. I imagine the wiki has a fair load and from my experience you hit scaling issues fairly quickly while using rails.

Markdown is my syntax of choice too :)

Just to note as long as your html syntax is fairly standard and works on firefox, opera and safari it will probably work on ie 7 and 8 without change (or with minor tweaks). I agree ie 6 is a b*tch to support, although nothing is as bad as ie 5.5 for the mac.

Also your links are broken.
edit: The markdown and textile ones
Fixed the links.  High concurrency rails is actually not that difficult to achieve; we programmed a internal finance management app for a big name company (not allowed to disclose) that is mostly Rails with a few C plugins - it sees almost 4x the requests / sec the wiki does and performs marvelously on 2 commodity hardware servers.  Most of it requires sane usage of caching, and knowing the limitations you do encounter with any web programming language - the same kind of bottlenecks that you'll hit with PHP are the same ones you'll hit with RoR, although potentially sooner.
Logged
I am one of many administrators of the wiki.  Please use my user page (http://dwarffortresswiki.org/index.php/User_talk:Briess) on the wiki to contact me, as I check that more often than these forums.

Shades

  • Bay Watcher
    • View Profile
Re: The Magmawiki Project
« Reply #3 on: April 26, 2010, 08:06:42 am »

An internal app gets more hits than the wiki, how many requests per second do we get on the wiki anyway?

I generally see orders of magnitude higher concurrency with plain ruby over ruby on rails but if we are well inside the user load then it's not an issue :)
Logged
Its like playing god with sentient legos. - They Got Leader
[Dwarf Fortress] plays like a dizzyingly complex hybrid of Dungeon Keeper and The Sims, if all your little people were manic-depressive alcoholics. - tv tropes
You don't use science to show that you're right, you use science to become right. - xkcd

zwei

  • Bay Watcher
  • [ECHO][MENDING]
    • View Profile
    • Fate of Heroes
Re: The Magmawiki Project
« Reply #4 on: April 26, 2010, 08:11:45 am »

I understand that you want to get rid of MediaWiki, but why write your own engine from scratch?

I am quite sure there are enough of wiki engines that would solve your needs. Some even written in ruby like Instiki http://www.instiki.org

Locriani

  • Bay Watcher
  • Locriani == Briess
    • View Profile
    • dwarf fortress wiki
Re: The Magmawiki Project
« Reply #5 on: April 26, 2010, 08:21:52 am »

I understand that you want to get rid of MediaWiki, but why write your own engine from scratch?

I am quite sure there are enough of wiki engines that would solve your needs. Some even written in ruby like Instiki http://www.instiki.org

We looked at some of the other wiki solutions (including MoinMoin, Instiki, and a very ... hilarious meeting with Wagn), but none of them offer the features we've come to depend on at the dwarf fortress wiki.

Just wanted to point out that we did, in fact, look at a significant number of other options (including a few paid solutions).
Logged
I am one of many administrators of the wiki.  Please use my user page (http://dwarffortresswiki.org/index.php/User_talk:Briess) on the wiki to contact me, as I check that more often than these forums.

Kogan Loloklam

  • Bay Watcher
  • I'm suffering from an acute case of Hominini Terravitae Biologis. Keep your distance!
    • View Profile
Re: The Magmawiki Project
« Reply #6 on: April 26, 2010, 09:37:43 am »

No IE support.  Ever.

IE has 50% of the Market share on average, and the Wiki is at least 80% of the "Game Documentation". Without someone to handhold a new user, they will probably rely a great deal on the Wiki. This is true even in the event of a forum. A forum is a community that has the potential to be hostile to a new user, especially one who is just dipping their toe into a roguelike and having a great deal of difficulty in overcoming the graphics barrier.

I would hope that your "No IE support. Ever." would only translate into official IE support, and not a complete shutout.
I applaud your commitment to be microsoft-free, but your commitment isn't enough justification to potentially harm new Dwarf Fortress players. Before you fart all over IE so that 50% of new people have difficulty playing because they have no source of information, please consider if your commitment to non-microsoft programs can be set aside and the extra work done to get the larger base of people. Our current system doesn't scrap that group, and Dwarf Fortress isn't mainstream enough that we can afford the hit that dumping something used by a majority on the basis of it's crappiness is a good decision.

(figures taken from Statcounter Global Stats)
Logged
... if someone dies TOUGH LUCK. YOU SHOULD HAVE PAYED ATTENTION DURING ALL THE DAMNED DODGING DEMONSTRATIONS!

Preserve the Toad.
Drag the world out of the Dark age of Gaming kicking and screaming!

Michael

  • Bay Watcher
    • View Profile
Re: The Magmawiki Project
« Reply #7 on: April 26, 2010, 10:55:12 am »

There doesn't seem to be a way to try the Magmawiki skin without registering...

Also, on the feature list, I notice the absence of one feature that only a custom-for-DF solution could provide.  This would be automatic synchronization of the pages with DF's raw files.  So the infoboxes on creature/vermin/plant/metal/etc. pages could be kept up to date, and at least stubs with a complete infobox could appear as soon as Toady publishes a version with a new objects.
Logged

Locriani

  • Bay Watcher
  • Locriani == Briess
    • View Profile
    • dwarf fortress wiki
Re: The Magmawiki Project
« Reply #8 on: April 26, 2010, 11:11:05 am »

No IE support.  Ever.

IE has 50% of the Market share on average, and the Wiki is at least 80% of the "Game Documentation". Without someone to handhold a new user, they will probably rely a great deal on the Wiki. This is true even in the event of a forum. A forum is a community that has the potential to be hostile to a new user, especially one who is just dipping their toe into a roguelike and having a great deal of difficulty in overcoming the graphics barrier.

I would hope that your "No IE support. Ever." would only translate into official IE support, and not a complete shutout.
I applaud your commitment to be microsoft-free, but your commitment isn't enough justification to potentially harm new Dwarf Fortress players. Before you fart all over IE so that 50% of new people have difficulty playing because they have no source of information, please consider if your commitment to non-microsoft programs can be set aside and the extra work done to get the larger base of people. Our current system doesn't scrap that group, and Dwarf Fortress isn't mainstream enough that we can afford the hit that dumping something used by a majority on the basis of it's crappiness is a good decision.

(figures taken from Statcounter Global Stats)

If you want to commit to that support, feel free to contribute to the project.  It is unreasonable and impossible to expect me to contribute that support, so assuming it does not render properly in IE, then you'll have a broken page.

I should disclaim that IE8 is reasonably standards based that most should work without tweaking, and that IE9 is a step further in that direction -- however, I will not be testing it on that platform.

[edit]
Let me rephrase, since I did not explain this clearly:
IE 6 Support: No - even Google is dropping support for IE6.
IE 7 Support: Only if it happens to work (unlikely)
IE 8 Support: Only if it happens to work (likely)
IE 9 Support: Only if it happens to work (very likely)

Support for Firefox, Opera, and all webkit based browsers (chrome, safari, iPhone / iPad safari, Android browser, embedded awesomium) is implied.  If it doesn't work in one of these, it will be fixed.
« Last Edit: April 26, 2010, 11:27:24 am by Locriani »
Logged
I am one of many administrators of the wiki.  Please use my user page (http://dwarffortresswiki.org/index.php/User_talk:Briess) on the wiki to contact me, as I check that more often than these forums.

Calhoun

  • Bay Watcher
  • Reusable-Box
    • View Profile
Re: The Magmawiki Project
« Reply #9 on: April 26, 2010, 11:16:34 am »

No IE support.  Ever.

IE has 50% of the Market share on average, and the Wiki is at least 80% of the "Game Documentation". Without someone to handhold a new user, they will probably rely a great deal on the Wiki. This is true even in the event of a forum. A forum is a community that has the potential to be hostile to a new user, especially one who is just dipping their toe into a roguelike and having a great deal of difficulty in overcoming the graphics barrier.

I would hope that your "No IE support. Ever." would only translate into official IE support, and not a complete shutout.
I applaud your commitment to be microsoft-free, but your commitment isn't enough justification to potentially harm new Dwarf Fortress players. Before you fart all over IE so that 50% of new people have difficulty playing because they have no source of information, please consider if your commitment to non-microsoft programs can be set aside and the extra work done to get the larger base of people. Our current system doesn't scrap that group, and Dwarf Fortress isn't mainstream enough that we can afford the hit that dumping something used by a majority on the basis of it's crappiness is a good decision.

(figures taken from Statcounter Global Stats)

While IE may have the overwhelming market share. I'd say, more often than not, Someone who's playing, or trying to play DF is going to know how terrible IE is.
Logged
I know it's unrealistic, but I can't help but imagine little bearded babies for dwarves. In my mind, they come out of the womb fully bearded. That's how the mother carries them around, too, she just drags them around by the beard or ties it to her belt. When the father's on duty, he just ties their beards together and the baby just kind of hangs there, swinging to and fro with Urist McDaddy's movements.

Draco18s

  • Bay Watcher
    • View Profile
Re: The Magmawiki Project
« Reply #10 on: April 26, 2010, 12:27:03 pm »

Just one thing:

I hate textile.

For example, typing:

*Superior Quality*

Will do this:

Superior Quality

How's that for being a pain in the ass?

I gets worse when you try to paste code (and the only forum I know of that uses Textile is Kongregate where you post code a lot on the programming forum).
Logged

Emi

  • Bay Watcher
    • View Profile
Re: The Magmawiki Project
« Reply #11 on: April 26, 2010, 01:57:28 pm »

For clarification on the IE front. At least my understanding has always been that we'll develop to version 1 without checking it in IE7/8 and then I'll go back (at my own leisure) and write some css and other stuff for compatibility.
Logged

The_Fool76

  • Bay Watcher
    • View Profile
Re: The Magmawiki Project
« Reply #12 on: April 26, 2010, 02:37:13 pm »

Just a note for those that absolutely HAVE to use IE.
http://code.google.com/chrome/chromeframe/

Google has a plugin that 'fixes' IE.
Logged
Tis far better to be a witty fool than a foolish wit.

Corona688

  • Bay Watcher
    • View Profile
Re: The Magmawiki Project
« Reply #13 on: April 26, 2010, 03:03:59 pm »

PHP5 is a lot better than PHP4 was.  The security problems are easily avoided if you turn off certain features like magic quotes(something they've been trying hard to depreciate for years and years.)  If you want to write in a language nobody but you will understand, more power to you but, yeah -- you'll likely be the sole maintainers, forever.
Logged
You never know when you might need a berserk dwarf to set loose somewhere.

Draco18s

  • Bay Watcher
    • View Profile
Re: The Magmawiki Project
« Reply #14 on: April 26, 2010, 03:16:51 pm »

If you want to write in a language nobody but you will understand, more power to you but, yeah -- you'll likely be the sole maintainers, forever.

Nah, not forever.  I tried to interview at a place that used Ruby on Rails almost exclusively.

Notably, I barely know what "on rails" means, much less what Ruby is or why it'd be on rails.
Logged
Pages: [1] 2 3 ... 9