Subject: Email Data Structure

From: "Jeff" <fyberoptic1979@...>
Date: Mon, 10 Dec 2007 08:01:09 -0000

I thought I'd make a new topic regarding this, since I don't think 
it's been discussed before.

Basically, I want to figure out the data structure of the email 
stored in the dataflash.  I haven't investigated it thoroughly yet, 
but I thought I'd post my preliminary findings, and perhaps see if 
someone else might have learned more about it already before I spend 
any time trying to figure it out.

Here's what I've found so far.  Messages seem to start with a 14 byte 
header, the first of which tends to always be 0xfd.  It's followed by 
the subject line, null terminated, but the field always appears to be 
142 bytes long, regardless of how short the subject really is.  After 
that is the email recipient, immediately terminated by a null.  Then 
the message body, also terminated by null.  Then I think there's an 8 
byte footer of data.  

I've been doing various dumps after creating emails and such to 
compare the data.  I think the second byte of the email headers 
possibly determines the status in some way, such as if it's deleted 
or whatnot.

I'd like to find an index or something, but I'm not sure if one 
exists.  Perhaps it just reads through them all and constructs and 
index in memory.

I'll post more info as I get around to finding it.

---

From: "Cyrano Jones" <cyranojones_lalp@...>
Date: Tue, 11 Dec 2007 06:27:10 -0000

---  "Jeff" <fyberoptic1979@...> wrote:
>
> I thought I'd make a new topic regarding this, since 
> I don't think it's been discussed before.
> 
> Basically, I want to figure out the data structure of 
> the email stored in the dataflash.  I haven't investigated 
> it thoroughly yet, but I thought I'd post my preliminary 
> findings, and perhaps see if someone else might have 
> learned more about it already before I spend any time 
> trying to figure it out.

Funny you should ask.  I've been taking apart the "Inbox" 
app, mainly because I want to understand how to use the 
scrolling text widget, and I just figured out the message 
header a couple of days ago.

> Here's what I've found so far.  Messages seem to start 
> with a 14 byte header, the first of which tends to always 
> be 0xfd.  

The first 4 bytes are a "file header", or maybe more 
correct, a "record header".  Either way, these first
4 bytes are part of the file system, not really part
of the message.  The first byte is always FD, as you 
found. (I have seen FC, but I am not sure what it means.  
It may mean a deleted record.)

The second byte is a "file number" that tells what file 
the record belongs to.  Third and fourth bytes (word) 
are the file size, not counting the header iirc.  
The rest is a variable amount of file data.

I get the distinct impression that each of these records
holds one complete message.

File/record format:

    byte 0xFD
    byte filenumber
    word filesize
    data (message, addressbook, or calendar data)

> It's followed by 

I don't think there is a footer.  Perhaps what you
see is the "directory" record???  For a file with
only one data record, the directory record will be
8 bytes (I think).

There does not seem to be any sort of text filenames
used (I guess embedded systems don't really need them).

Each file has it's own directory record.  These 
directory records are similar to the message
records, with the same 4 byte header format, except 
the file number is 0x40 + the file number of the
associated data records.

The data portion of a directory record is just a list 
of 4 byte addresses of the data records in that 
particular file.  

There is a special sector (in dataflash page 8)
That keeps a list of the current addresses of
the directory records.  This special sector is
the only part of dataflash that they keep a 
use-count for.  I suppose this is because that
sector is re-written whenever any file is
written, and will get more use than any other
sector.  When use-count reaches 10,000 it moves
this special sector up to next sector.  There
are 5 of these special sectors reserved, and
after 50,000 writes, it starts re-using the 
same 5 sectors.

To top it all off, the 5th byte in the second-to-last
sector determines which of the 5 sectors is
currently in use!

But if you use the mailstation's read function, 
(0x089F), you don't need to worry about all 
these details.  ;^)

You just need the following details...

Message data format:

    word offset_to_body
    byte flagbits
    word year
    byte month
    byte day
    byte hour
    byte min
    byte sec
    subjectstring (padded to 142 dec bytes)
    fromstring ("tostring" in outbox)
    body of message

The "tostring" in outbox messages can be more 
than one email address.

flagbits xxxx xxx1 = new message	

The "new" bit, when in outbox, seems to mean 
"ready to send".  Draft messages seem to be 
stored in outbox with this bit cleared.

Several other bits have meaning, I'm not sure just
what.  Seems some might flag message as a "form"
rather than normal message.  I have never seen one
of these forms, but I think they are a way for
cidco to send a fill-out-form, with checkboxes
and fill-in fields, to their customers.  I think
they get emailed back after being filled out.

The file numbers I have figured out so far:

 00 - ??? 
 01 - ??? 
 02 - address book
 03 - calendar
 04 - ???
 05 - inbox user 0 
 06 - inbox user 1
 07 - inbox user 2
 08 - inbox user 3
 09 - inbox user 4 
 0A - outbox user 0
 0B - outbox user 1
 0C - outbox user 2
 0D - outbox user 3
 0E - outbox user 4
 0F - sent user 0
 10 - sent user 1
 11 - sent user 2
 12 - sent user 3
 13 - sent user 4
 14 - ???? user 0   Not sure if files 14 thru 18 are used for anything.
 15 - ???? user 1   Seems that deleting user may erase/move these,
 16 - ???? user 2   which implys they might be used for something.  
 17 - ???? user 3
 18 - ???? user 4
 19 - ?             I think 19 thru 1E are unused???
 1A - ?
 1B - ?
 1C - ?
 1D - ?
 1E - ?             highest file number

--
CJ

---

From: "tuzilo" <tuzilo@...>
Date: Tue, 11 Dec 2007 14:43:12 -0000

Wow ..., I am very impressed that even MailStation still survives 
these days. I have mine, i.e. MailStation250 and MailStation350, 
completely forgotten and stowed away on some shelves in the 
basement. ;) If anyone needs these, I will be more than happy to 
trade-in with something you have that I may find useful. Let me 
know.