Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 2 3 [4] 5 6 ... 13

Author Topic: Kobold Quest port status?  (Read 12203 times)

segmose

  • Bay Watcher
    • View Profile
Re: Kobold Quest port status?
« Reply #45 on: November 25, 2006, 05:52:00 am »

I found these routines in some network code.
Call these before saving:
itsaint32 = htonl(0x7F000001); // Host To Network Long int
itsaint16 = htons(7654); // Host To Network Short int

and their reverse after loading
ntohl
ntohs

See also here: http://www.penguin-soft.com/penguin/man/3/byteorder.html

Logged

Pod

  • Bay Watcher
    • View Profile
Re: Kobold Quest port status?
« Reply #46 on: December 05, 2006, 12:56:00 pm »

Is this to be considered a final port of kobold quest then?
Logged

Toady One

  • The Great
    • View Profile
    • http://www.bay12games.com
Re: Kobold Quest port status?
« Reply #47 on: December 05, 2006, 01:28:00 pm »

As far as I know, the potential endian/sizeof issues haven't been handled for any platform.  There's the link to that library, but I don't have time to look at it right now.  A port should allow the game to run on the OS and also produce OS-independent save files.
Logged
The Toad, a Natural Resource:  Preserve yours today!

Pod

  • Bay Watcher
    • View Profile
Re: Kobold Quest port status?
« Reply #48 on: December 06, 2006, 08:41:00 am »

Why bother changing the endianness? Specify it as little endian in the save file format, then, for any port on a different architecture, they simply call a different method in order to load the file instead of a normal one. Getting a truly architecture-independent save file would be a pain in the arse, just leave it small endian and let the few ports that do go to none-x86 sort it out.

I'm not sure I understand the sizeof issues as well - is this to do with 32/64bit compatibility or something else?

Logged

Toady One

  • The Great
    • View Profile
    • http://www.bay12games.com
Re: Kobold Quest port status?
« Reply #49 on: December 06, 2006, 01:06:00 pm »

As I've said before, I simply don't know what Macs and Linux do for files, and I've been told my file format is heavily OS-dependent.  If the ports are done, they are done, but I was under the impression that they aren't.
Logged
The Toad, a Natural Resource:  Preserve yours today!

Pod

  • Bay Watcher
    • View Profile
Re: Kobold Quest port status?
« Reply #50 on: December 07, 2006, 02:02:00 pm »

Linux doesn't really 'do' anything special for files. Files are files. The old macs had some crazy resrouce fork thingy which I never bothered to understand, but mac os x uses linux...... so I doubt there's anything incompatible about your files.
I'll have a quick look at the files used in kobold quest and see if I agree.
Logged

Toady One

  • The Great
    • View Profile
    • http://www.bay12games.com
Re: Kobold Quest port status?
« Reply #51 on: December 07, 2006, 04:25:00 pm »

Yeah, peterb might comment on this better than I can -- he said an old KQ save file caused a fatal problem of some kind due to endianness, which lead to my comment that files still have some porting work to be done.
Logged
The Toad, a Natural Resource:  Preserve yours today!

peterb

  • Bay Watcher
    • View Profile
Re: Kobold Quest port status?
« Reply #52 on: December 09, 2006, 06:31:00 pm »

I think I might have been smoking crack.  I dunno.  I still think it's worth proceeding apace.
Logged

Toady One

  • The Great
    • View Profile
    • http://www.bay12games.com
Re: Kobold Quest port status?
« Reply #53 on: December 09, 2006, 06:44:00 pm »

From what I've read, all but the most recent (intel) Macs are going to be big-endian.  This complicates porting to "Mac".
Logged
The Toad, a Natural Resource:  Preserve yours today!

peterb

  • Bay Watcher
    • View Profile
Re: Kobold Quest port status?
« Reply #54 on: December 10, 2006, 08:25:00 pm »

That's true, but all of the Macs currently being sold are little-endian.  

And more importantly, i have one!

:-)

-p

Logged

Toady One

  • The Great
    • View Profile
    • http://www.bay12games.com
Re: Kobold Quest port status?
« Reply #55 on: December 11, 2006, 03:09:00 am »

The only Mac I potentially had access to is big-endian...  I'll have to figure out how I'm going to compile things.
Logged
The Toad, a Natural Resource:  Preserve yours today!

Shadowlord

  • Bay Watcher
    • View Profile
Re: Kobold Quest port status?
« Reply #56 on: December 11, 2006, 04:46:00 am »

The aforementioned htons and htonl functions change a platform-dependent short or long into a platform-independent version (Or rather, it changes them to the same endianness on all platforms). As segmose mentioned, ntohl and ntohs are their opposites.

Those functions are in winsock.h on windows, and on linux/macs:

code:
#include <sys>
#include <netinet>
#include <inttypes>

Essentially what you would need to do is:
When reading a 32-bit variable from the save files: fread it, then ntohl it.
When writing a 32-bit variable to the save files: htonl it, then fwrite it.
For 16-bit variables use ntohs and htons instead.
For 8-bit variables, you don't need to do anything. (Endianness is the order of bytes in words (16 bit / 2 bytes), dwords (32 bit), etc, so a big-endian 8-bit variable would be the same as a little-endian one)
If you have ASCII (one byte per character) strings being written, you shouldn't have to do anything to them either. But if they're being written in unicode, maybe you would.

That may probably break compatibility with old save files, however.

Alternately, you try can hackish things to determine whether the system is little-endian or big-endian: Set a short (16-bit) to 0x0001, and then cast it to a two-byte char array, and check to see whether it's index 0 or 1 which is 0x01. Then you can just have a custom byte-swapping function being called which only does anything if the endianness of the platform doesn't match the endianness on the save files. This would preserve compatibility with old save files, and would make them work on machines with different endianness. (First you'd need to do the above test on the PC to see where the 0x01 is, then code it to enable the byte-swapper only if the 0x01 is in the other byte instead)

I'm assuming the compiler won't be so smart that it swaps the bytes around. I've seen them do some annoyingly smart things from time to time, though... So if the 0x01 appears to be in the same place on both little-endian and big-endian systems, then you know that won't work (ntohl etc should still work then though).

P.S. The reason ntohl etc are in network sockets libraries, is that they were created because networks (like the internet) needed a platform-independent protocol, as no network node could be expected to know the endianness of other arbitrary nodes.

P.P.S. You can manually swap bytes by doing something like this:

code:
integer fooSwapped = ((foo&0xff)<<24)+((foo&0xff00)<<8>>8)+((foo&0xff000000)>>24);

[ December 11, 2006: Message edited by: SL ]

Logged
<Dakkan> There are human laws, and then there are laws of physics. I don't bike in the city because of the second.
Dwarf Fortress Map Archive

Jifodus

  • Bay Watcher
  • Resident Lurker
    • View Profile
    • Dwarf Fortress Projects
Re: Kobold Quest port status?
« Reply #57 on: December 11, 2006, 03:08:00 pm »

Yes I'm fairly sure using those will break backwards compatability, because http://en.wikipedia.org/wiki/Endianness  says that "Internet Protocol defines a standard big-endian network byte order." So those functions will most likely output as big-endian numbers breaking backwards compatability.

I did modify the basic read and write code to load and write files as little endian therefor it should maintain compatability.

code:
// This will modify read_file and write_file
// to save and load all data in little endian notation.

const static char BIG_ADDE[2] = { 0xAD, 0xDE };

char file_compressorst::read_file(void *read_var,long read_size,bool raw)
{
//---------------SNIP1
   // {raw} is an endian dependant type if true

   unsigned short ELVIS=*(unsigned short *)BIG_ADDE;
   bool BIG_ENDIAN=(ELVIS!=0xDEAD);

   bool endianSwap=!raw; // endian swap if it is not going to read in the data raw (i.e. strings)

   if(!raw)
      {
      else if(BIG_ENDIAN)
         {
         // swap if big endian
         endianSwap = true;
         }
      }
//---------------SNIP1

   if(h==INVALID_HANDLE_VALUE)return 0;

   //NOW LOAD INTO read_var UNTIL DONE
   while(read_size>0)
      {
      //GET A BUFFER IF NECESSARY
      if(in_buffer_amount_loaded==0||
         in_buffer_position>=in_buffer_amount_loaded)
         {
         if(!load_new_in_buffer())return 0;
         }

      //BAIL IF STILL NO BUFFER LEFT
      if(in_buffer_amount_loaded==0)return 0;

      //SET THE AMOUNT TO COPY
      long copy_size=in_buffer_amount_loaded-in_buffer_position;
      if(read_size<copy_size)copy_size=read_size;

      //COPY
      //memmove(read_var,in_buffer+in_buffer_position,copy_size);
//---------------SNIP2
      char *data = (char *)read_var;
      if (endianSwap)
         {
         for (long read=0;read<copy_size;read++)
            {
            // read in reverse
            data[read_size-read-1]=in_buffer[in_buffer_position+read];
            }
         }
      else
         {
         for (long read=0;read<copy_size;read++)
            {
            data[read]=in_buffer[in_buffer_position+read];
            }
         }
//---------------SNIP2

      read_var=((char *)read_var) + copy_size;
      read_size-=copy_size;
      in_buffer_position+=copy_size;
      }
      
   return 1;
}

char file_compressorst::write_file(void *write_var,long write_size,bool raw)
{
//---------------SNIP3
   // {raw} is an endian dependant type if true
   // this does not have backwards compatability because it really isn't necessary

   unsigned short ELVIS=*(unsigned short *)BIG_ADDE;
   bool BIG_ENDIAN=(ELVIS!=0xDEAD);

   bool endianSwap=!raw; // endian swap if it is not going to read in the data raw (i.e. strings)

   if(!raw)
      {
      if(BIG_ENDIAN)
         {
         //swap if big endian
         endianSwap = true;
         }
      }
//---------------SNIP3

   if(h==INVALID_HANDLE_VALUE)return 0;

   //WRITE OUT THE VARIABLE CHUNK BY CHUNK
   while(write_size>0)
      {
      //FLUSH THE BUFFER IF NECESSARY
      if(in_buffer_amount_loaded>=in_buffersize)
         {
         if(!flush_in_buffer())return 0;
         }

      //SET THE AMOUNT TO COPY
      long copy_size=in_buffersize-in_buffer_amount_loaded;
      if(write_size<copy_size)copy_size=write_size;

      //COPY THE NEXT CHUNK INTO THE BUFFER
      //memmove(in_buffer+in_buffer_amount_loaded,write_var,copy_size);
//---------------SNIP4
      char *data = (char *)write_var;
      if (endianSwap)
         {
         for (long write=0;write<copy_size;write++)
            {
            // read in reverse
            in_buffer[in_buffer_amount_loaded+write]=data[read_size-read-1];
            }
         }
      else
         {
         for (long write=0;write<copy_size;write++)
            {
            in_buffer[in_buffer_position+write]=data[read];
            }
         }
//---------------SNIP4

      write_var=((char *)write_var) + copy_size;
      write_size-=copy_size;
      in_buffer_amount_loaded+=copy_size;
      }

   return 1;
}



The only modifications that need to be made are to the higher level readers/writers is adding true to the function call.
So for reading strings, instead of calling:
read_file(var,len*sizeof(char));
You must call:
read_file(var,len*sizeof(char),true);

And for reading an endian important data type (short, int, long) is:
read_file(&var,sizeof(long));
You must call:
read_file(&var,sizeof(long),false);

Similar for writing.

I'm unsure if floating point numbers are endian independant or not.

[ December 11, 2006: Message edited by: Jifodus ]

Logged

popecharlotte

  • Escaped Lunatic
    • View Profile
Re: Kobold Quest port status?
« Reply #58 on: January 03, 2007, 07:03:00 pm »

I've been working on a version of kobold quest that should handle endianness properly. There's a link here if you want to see what I have at the moment, but I'm pretty sure it won't work.

At the moment it byte-swaps the data coming in to file_compressorst::write_var (and out of read_var) on big-endian machines. However, I'm not sure if what comes out of zlib's compression needs to be fiddled with as well, or if zlib handles that itself.

I don't have access to a big-endian machine so I have no idea what will happen, so it would be handy if someone were to test it.

Logged

Toady One

  • The Great
    • View Profile
    • http://www.bay12games.com
Re: Kobold Quest port status?
« Reply #59 on: January 08, 2007, 05:53:00 pm »

For the DF port, from what little was mentioned before, cross-compilation seems unlikely.  I only have my Windows XP laptop, and I'm not sending the source code to random people or anybody else.  So, it seems like I need to find a computer that will work somehow.  People want "Linux" and "Mac" ports?  What does this imply exactly about the types of operating systems I'd need to place on computers?  If I have to buy something, what is the least-cost solution?  I can't build computers reliably.  Is there a way to do "Linux" and "Mac" on one system or am I looking at at least two additional systems here?  I don't have a ton of money to throw around, though these can probably pay themselves off at least partially in the long-term if the ports work out.
Logged
The Toad, a Natural Resource:  Preserve yours today!
Pages: 1 2 3 [4] 5 6 ... 13