Subject: Mailstation files

From: "cyranojones_lalp" <cyranojones_lalp@...>
Date: Mon, 08 Feb 2010 05:10:00 -0000

Some of this is based on guesses, and any of it may
turn out to be wrong.

Some of this is pasted in from text that is prolly
too wide.  If you are reading it on group website,
you can select "use fixed width font" over there -->
and it might look a little better.
-------------------------------------------

Dataflash files are accessed by a "file number".

(The number of question-marks is an indicator of how wild 
a guess is.  Normal educated guesses usually get 3.)

file#  
  00 - unused/ or 00 not possible???
  01 - unused????
  02 - address book
  03 - calendar
  04 - unused???
  05 - inbox user 0
  06 - inbox user 1
  07 - inbox user 2
  08 - inbox user 3
  09 - inbox user 4
  0A - outbox user 0
  0B - outbox user 1
  0C - outbox user 2
  0D - outbox user 3
  0E - outbox user 4
  0F - sent user 0
  10 - sent user 1
  11 - sent user 2
  12 - sent user 3
  13 - sent user 4
  14 - ??? user 0   (not really sure 14 thru 18 used, should test???)
  15 - ??? user 1
  16 - ??? user 2
  17 - ??? user 3
  18 - ??? user 4
  19 - unused??? 
  ...
  1D - unused???  (not sure what highest possible file# is, need to test)
  
I am pretty sure they go up to at least 0x1d or 0x1e, but need 
to test to be sure.  There are also file numbers, offset from 
actual file numbers by 0x20 or 0x40 (I forget exactly) that 
are part of the actual file, (or part of the directory, 
depending on how you see things).

I can't find it written anywhere, and I forget exactly how it 
works, but I seem to recall thinking the higher numbers may be 
a "thread" thru the actual records, in a sorted order.  
IIRC, At least one file also had a second sort order, using 
a file number an additional offset higher (+0x80???). 
 
I do believe the higher numbers are just for internal usage, 
and you don't ever "open" one of them, you just use the lower 
set of file numbers.
(I could be wrong on that, though!)
 
(I prolly understood this a lot better 2 or 3 years ago!!!) 

I have a vague notion that I saw a place where code was 
deleting a user, and copying the files of higher users 
down 1, to close the gap.  IIRC, it was copying 4 different
groups, but I never figured out what the 4th group was 
(in/out/sent/???).  
But it is possible it was just 3 groups.

There is more info on files below, in "notes" section.  
There is a part of dataflash I called "bookmarks", but it 
might be better called a "directory".  The bookmark sector 
holds a table of 4 byte dataflash addresses for the possible 
file numbers.  Each file number has 2 entries (iow, 8 bytes), 
I believe the difference is the sort order of that files records.

I am really stretching my memory to the limit, but I seem to
think the bookmarks point to a linked list of records with
the file numbers offset as refered to earlier.  I believe 
these are sorted, and each node points to a message record, 
in addition to a "next" pointer record.  The message records 
have the actual file number (less than 0x20 iirc), and each 
message record holds a single email message.  I want to say 
that there is only one record of pointers for each file, but 
I am not sure.  It is possible that each "node" in sorted 
list is a separate record.  I don't recall if these have 
the same header as the message records. 

I am pretty sure that each message in a file has a separate 
record, with a 4 byte header.

Most files (or maybe I really mean "records" of files)
start with 0xFD if valid, and 0xFC (iirc) if deleted. 
The 2nd byte is file number, 3rd & 4th (word) are the size 
of the data that follows this 4 byte header.
   
There are 4 of what I called "special files" that use 
completely different file structure, with a fixed 
position in df, and a fixed number of records:

  A0 - user accounts
  A1 - settings data
  A2 - ???
  A3 - ???
  
Some of the lower number files (0x02 & 0x03????) have 
patches to the file handling code.  This possibly has something 
to do with the sorting?????  There are also patches to handle
the "special" files differently.

I believe you can open/read/write/etc *any* of the dataflash 
files with the same functions (see my "mailstation.h" in 
groups file section).  

There is also info in that header regarding structure of the
data portion of the "message" files (inboxes/outboxes/sentboxes).

The mailstation firmware also has another concept of "files",
with completely different functions used to access them.
the "ramdisk" functions.  More than 1 of the ram pages is 
dedicated to this ramdisk, I forget if it is 2, 3, or maybe 4????

It is entirely possible that file 0x00 & 0x01 in list above
are not accessed by the dataflash access functions.
(ok, I just removed these two from list)

  00 - string table???? (maybe ramdisk, and no df file 00?????  if so, then likewise for 01.) 
  01 - widget "sets"  (one looks like new acct entry screen, one might be sent screen???)

They might really be "ramdisk" type files????  
The more I think, the more likely that seems.  
I do remember thinking that it seemed they purposely 
did not use the same filenumbers for both types of 
file, but I see no reason why you could not have one of 
each for a given number.
It may be that they developed on a system with normal disk
files, and then mapped some to dataflash, and others to 
ram/rom-disk??????????????

The ramdisk functions are more like normal file access functions,
there is even a "seek" function available.

The ms seems to use these for storing various data
relating to the "widgets", iow the widget value is
the file data (IIRC, this includes text in text
widgets.

I think two files (00 & 01) accessed by what I called 
"ramdisk" functions are actually accessing fixed data in 
the  codeflash.  Call them "romdisk" maybe???
No, that is not good, there is a function I labeled 
"romdisk" that is something entirely different (perhaps 
to access that huge chunk of codeflash with the 
spellcheck dictionary????????)

--------------------------------

My approach to making a file system for cp/m would leave all
this to the ms firmware, and just call the read/write functions.
I have thought of various schemes, including using one of the
"users" inboxes, where each "message" was a file.  One interesting
aspect of that would be possibility of just "emailing" a file
to the ms!

Or you could make each record in a ms file be a "track" to cp/m.

--------------------------------

Some more info on dataflash contents:

(I used the # as meaning hex, I think 'coz the disassembler
we were using at first used that syntax.)

 address  pg:sect:offset
 ---------------------------------------------
 #000000  00:00:00 - 00:3f:ff   1st loaded app
 #004000  01:00:00 - 01:3f:ff   2nd loaded app
 #008000  02:00:00 - 02:3f:ff   3rd loaded app
 #00c000  03:00:00 - 03:3f:ff   4th loaded app
 #010000  04:00:00 - 04:3f:ff   5th loaded app
 #014000  05:00:00 - 05:3f:ff   ???
 #018000  06:00:00 - 06:3f:ff   ???

 #01c000  07:00:00 - 07:00:02   user_flag   (3 bytes) #AA, #AA, #nn, where nn is number of user records added.
 #01c003  07:00:03 - 07:3f:ff   user_adds   (0 to 5  562 byte user acct records)  (note 7)

 #020000  08:00:00 - 08:00:ff   magic_sect  (256 bytes) (note 4)
 #020100  08:01:00 - 08:01:ff   unknown use (256 bytes)

 #020200  08:02:00 - 08:02:ff   bookmark_0  
 #020300  08:03:00 - 08:03:ff   bookmark_1  (notes 2 & 3 & 9)
 #020400  08:04:00 - 08:04:ff   bookmark_2  
 #020500  08:05:00 - 08:05:ff   bookmark_3  
 #020600  08:06:00 - 08:06:ff   bookmark_4  
 #020700  08:07:00 - 08:07:ff   bookmark_5  

 #020800  08:08:00 - 08:08:08   flag_50     (9 bytes)  
 #020809  08:08:09              flag_51     (1 byte)   current user#  (note 9)
 #02080A  08:08:0A              flag_52     (1 byte)
 #02080B  08:08:0B              flag_53     (1 byte)
 #02080C  08:08:0C              flag_54     (1 byte)
 #02080D  08:08:0D              flag_55     (1 byte)
 #02080E  08:08:0E              flag_56     (1 byte)
 #02080F  08:08:0F              flag_57     (1 byte)

 #020900  08:09:00 - 08:1e:08   alt_splash  (5385 bytes)  (note 1, 6)
 #022100  08:21:00 - 08:24:28   alt_logo    (809 bytes)   (note 1,6)
        
 #022900  08:29:00 - 08:33:??   file_A0     users_accounts (5 records, 562 bytes each) (counts are decimal) (note 7)
 #023400  08:34:00 - 08:34:0f   file_A1     settings     (1 record, 16 bytes)
 #023416  08:34:16              file_A2                  (1 record, 2048 bytes)  (file #A2 is all 0's on mine)
 #023CFE  08:3C:FE              file_A3                  (50 records, 81 bytes each)

 #024D00  09:0D:00 - 1F:3D:FF   normal_files  (note 11)

   second to last sector
 #07FE00  1f:3e:00  datafl_flag (2 bytes)  dataflash state (set to #8017 after dataflash is initialized)
 #07FE02  1f:3e:02  ??????      (2 bytes)
 #07FE04  1f:3e:04  book_select (2 bytes)  selects one of 5 possible bookmark sectors  (note 3)
 #07FE06  1f:3e:06  flag64      (2 bytes)
 #07FE08  1f:3e:08  flag65      (2 bytes)

   lastsector
 #07FF00  1f:3f:00  mode_flag   (4 bytes)  run mode, See "Write_dataflash_Lastsector" [37DE] for values
 #07FFC8  1f:3f:c8  serial_num  (16 bytes)

note 0: Some of thes notes are redundant.  They may have been written
        months or years apart, and I don't have time to edit right
        now.  The "note #" has very little to do with the order
        they were written.

 note 1: These are optional customized splash screen & logo image data.  The
         sizes are how many bytes get copied to ram when this file is used.
         If first two bytes are not "OK", then hardcoded images in codeflash
         are used.  

         the first 4 words of the splash file are "5083, 4, 319, 127" or
         in hex "#13db, #0004, #013f, #007f".  The rest is bit data???
         The 319 is x dimension, and 127 is y, in pixels.
         I don't think you can change the dimensions, coz prog expects these
         sizes.  The splash is a full screenful, and the logo is the size of
         the "Mailstation" logo at the top of the main menu (iow, it *is*
         the main menu logo).

  note 2: I called these sectors "bookmarks" for lack of better idea.  
         There are 2 kinds of bookmark sector, and there is always just one
         of each kind.  A value in the 2nd to last sector specs which of
         5 different pairs of sectors are the current bookmarks.  After
         a bookmark has been erased 10,000 times, the next pair is used.
         After all 5 pairs have been used, it starts over with the first pair.

         The first bookmark sector of the pair holds the start addresses for
         each of the "normal" files, and also a use count.

         The second of the pair just holds the address of the next free
         location in dataflash, and a use count.  

         All the values in the bookmark sectors are 4 bytes long.  

         The "free" pointer is the 1st 4 bytes of the "free bookmark" sector,
         and the 2nd 4 bytes are a count of how many times this sector
         has been erased.

         The "file bookmarks" are offset in that sector by 8 x the file
         number.  There are two pointers for each file, and I think the
         difference may be that they point to 2 different threads thru the
         same file (unsorted, and sorted?????).  The last 4 bytes in the
         sector are the erase count.

note 3: What I am calling bookmarks are pointers to the "normal" files
         (and free space).  There are five bookmark
         sectors.  A byte in second to last sector selects which of the
         five is currently in use.  The first 4 bytes
         are the dataflash address of next free dataflash sector.  
         The second 4 bytes are a "use count" for this
         particular bookmark.  When use count reaches 10,000 the next
         bookmark sector is used.  After all 5 bookmarks
         have been used 10,000 times, it start over with the first bookmark.
         The bookmark sector is written whenever
         one of the "normal files" is modified, so it would see a lot
         more wear than any other sector.  I don't like
         this mechanism, but wear leveling is a tricky problem, and this is better than nothing.
         (08:02:xx) might be a bookmark also???  erased along with other 5.

 note 4: I called this "magic_sector" just because I didn't know what it was
         for.  Turns out it is a sort of directory for the loaded apps.

 note 6: the first 4 words of the splash file are "5083, 4, 319, 127" or
         in hex "#13db, #0004, #013f, #007f".  The rest is bit data???
         The larger one is full screenful, other is smaller, probably
         the size of the cidco logo?????

 note 7: user adds are written at two different places, page #07,
         and file #A0.  Why I don't know.

 note 8: dataflash pg 00 thru 07 values are from 2.22r, looks like
         test pattern.

 note 9: sectors (08:02:00) thru (08:08:FF)  are not writeable by
         write_dataflash_low (bookmarks & flags)

 note 10: settings file data, 16 dec bytes at dataflash  (08:34:00)  
          raw address #023400
          There is a 2 byte header, so the data starts at (08:34:02)
          The default setings are:

              '0815', #00  ;; download time
              #00          ;; 0 = am / 1 = pm  
              #00          ;; every (0, 4, 8, 12, 24) hours.   (0 = 0hr, 1 = 4hr, 2 = 8hr, 3 = 12hr,  4 = 24hr)
              #01          ;; save messages on server  (0 = yes / 1 = no)
              '8', #00     ;; max size of download (k)
              '10', #00    ;; auto powerdown time (minutes)
              #01          ;; call waiting  (0 = yes / 1 = no)
              #00          ;; spkr vol   (0 = low, 1 = hi, 2 = off)
              #01          ;; ignore dial tone   (0 = yes / 1 = no)

 note 11: This is the largest area in dataflash, and it is where the "normal"
          files are, but they are not at fixed locations.  
          The the first byte of a file is #FD, and the second byte is
          the file number.  Third & fourth bytes are file size, not counting
          the 4 byte header.