Thursday, April 3, 2014

Time to go Rainman on this

Here's a range of data from the packet sniffer.

Does anyone see a pattern here, I don't.

Err indicates a checksum error, don't worry about it.  I reordered the data by glucose level so you can see how identical readings can look pretty different.

2 day old sensor
                              txn                   
            --xmtr id--       id                   crc  mg/dl
____________________________________________________________
FF FF FF FF CA 58 61 00 3F 03 40  AF 89 AA 4E D8 00 37  : 76
FF FF FF FF CA 58 61 00 3F 03 3C  28 49 55 4E D8 00 1C  : 77
FF FF FF FF CA 58 61 00 3F 03 EF  A9 49 DE CE D8 00 F5  : 80
FF FF FF FF CA 58 61 00 3F 03 F3  8D 49 AF 4E D8 00 16  : 81
FF FF FF FF CA 58 61 00 3F 03 44  E3 49 BC 4E D8 00 0D  : 82
FF FF FF FF CA 58 61 00 3F 03 EB  10 C9 37 CE D8 00 01  : 83
FF FF FF FF CA 58 61 00 3F 03 E7  CE C9 E2 2E D8 00 10  : 86
FF FF FF FF CA 58 61 00 3F 03 F7  19 C1 CD 4E D8 00 67  : 87 err
FF FF FF FF CA 58 61 00 3F 03 FB  95 C9 FB 4E D8 00 1A  : 88
FF FF FF FF 0A 58 61 00 3F 03 DF  A6 29 9E AE D8 00 E9  : 93 err
FF FF F7 FF CA 58 61 00 3F 03 DB  67 29 4E 6E D8 00 86  : 97
FF FF FF FF CA 58 61 00 3F 03 FF  84 A9 9E CE D8 00 FC  : 98
FF FF FF FF CA 58 61 00 3F 03 20  15 A9 90 6E D8 00 D7  :102
FF FF FD FF CA 58 61 00 3F 03 50  CA 99 4D AE D8 00 02  :102 err
FF FF FF FF CA 58 61 00 3F 03 D7  9B A9 66 EE D8 00 2A  :103
FF FF FF BF CA 58 61 00 3F 03 14  C3 A9 AD 6E D8 00 0A  :103
FF FF FF FF CA 58 61 00 3F 03 24  28 69 AF AE D8 00 80  :105
FF FF FF FF CA 58 61 00 3F 03 10  5A 69 0D 6E D8 00 1B  :107
FF FF FF FF CA 58 61 00 3F 03 08  F1 69 8A AE D8 00 6D  :108
FF FF FF FF CA 58 61 00 3F 03 D3  9B 69 F8 1E D8 00 65  :110
FF FF FF FF CA 58 61 00 3F 03 0C  7B 69 74 6E D8 00 25  :110
FF FF FF FF CA 58 61 00 3F 03 CF  69 E9 A9 1E D8 00 CE  :115
FF FF FF FF CA 58 61 00 3F 03 54  93 99 E7 EE D8 00 E4  :117
FF FF BF FF C2 58 61 00 39 03 94  7A 19 9F 9E D8 00 02  :121 err
FF FF FF FF CA 58 61 00 3F 03 CB  F9 19 97 1E D8 00 72  :122
FF FF FF FF CA 58 61 00 3F 03 58  DB 99 31 9E C8 80 56  :129 err
FF F8 E7 FF CA 58 61 00 3F 23 90  E5 99 17 4E D8 00 B2  :130 err
FF FF FF FF CA 58 61 00 3F 03 5C  04 59 12 5E D8 00 A0  :134
FF FF FF FF CA 58 61 00 BF 03 60  95 5B BA 1E D8 00 4B  :135 err
FF FF FF FF CA 58 61 00 3F 03 64  FF D9 1E 5E D8 00 24  :135
FF FF FF FF CA 58 61 00 3F 03 6C  01 79 06 3E D8 00 74  :149 err
FF FF FF FF CA 58 61 00 3F 03 84  C0 39 BD 3E D8 00 62  :151
FF FF FF FF CA 58 61 00 3F 03 80  23 39 3C BE D8 00 E9  :155
FF FF FF FF CA 58 61 00 3F 03 70  9C 79 4D BE D8 00 79  :158
FF FF FF FF CA 58 61 00 3F 03 7C  50 B9 07 BE D8 00 09  :159
FF FF FF FF CA 58 61 00 3F 03 74  B3 B9 EE 7E D8 00 53  :163
FF FF FF FF CA 58 61 00 3F 03 78  F6 B9 0E 7E D8 00 9F  :163

Lets agree that we're interested in the 4 bytes after the transaction id.  Note how the second bit of the second byte always is a "9", and how bit 2 of byte 4 is always an "E".
I expected that 2 bytes would be a raw value, and 2 bytes would be a background.  I'm not seeing that here.
--Cleaned up Just look at the important stuff AF 89 AA 4E : 76 28 49 55 4E : 77 A9 49 DE CE : 80 8D 49 AF 4E : 81 E3 49 BC 4E : 82 10 C9 37 CE : 83 CE C9 E2 2E : 86 19 C1 CD 4E : 87 err 95 C9 FB 4E : 88 A6 29 9E AE : 93 err 67 29 4E 6E : 97 84 A9 9E CE : 98 15 A9 90 6E :102 CA 99 4D AE :102 err 9B A9 66 EE :103 C3 A9 AD 6E :103 28 69 AF AE :105 5A 69 0D 6E :107 F1 69 8A AE :108 9B 69 F8 1E :110 7B 69 74 6E :110 69 E9 A9 1E :115 93 99 E7 EE :117 7A 19 9F 9E :121 err F9 19 97 1E :122 DB 99 31 9E :129 err E5 99 17 4E :130 err 04 59 12 5E :134 95 5B BA 1E :135 err FF D9 1E 5E :135 01 79 06 3E :149 err C0 39 BD 3E :151 23 39 3C BE :155 9C 79 4D BE :158 50 B9 07 BE :159 B3 B9 EE 7E :163 F6 B9 0E 7E :163



Some other information from a Dex 7 log, just to give an idea on ranges.  Also, note the hex doesn't match the calculated value...
Highest possible value of 0xFFFF = 65535






RawCountsX="0x9C34"      RawCounts="115520"   
FilteredCountsX="0x7BEB" FilteredCounts="57176" 
GlucoseValue="162"       RawEstimatedGlucoseValue="-3934"

RawCountsX="0x9BEA"      RawCounts="114336" 
FilteredCountsX="0x7C44" FilteredCounts="57888" 
GlucoseValue="159"       RawEstimatedGlucoseValue="-3937" 

RawCountsX="0x9BE5"      RawCounts="114256" 
FilteredCountsX="0x7C02" FilteredCounts="57360" 
GlucoseValue="159"       RawEstimatedGlucoseValue="-3937" 

RawCountsX="0x99BA"      RawCounts="105376" 
FilteredCountsX="0x7A1B" FilteredCounts="53464" 
GlucoseValue="159"       RawEstimatedGlucoseValue="28831" 

RawCountsX="0x9784"      RawCounts="96320" 
FilteredCountsX="0x78D9" FilteredCounts="50888" 
GlucoseValue="122"       RawEstimatedGlucoseValue="-3974" 

RawCountsX="0x95E2"      RawCounts="89632" 
FilteredCountsX="0x7639" FilteredCounts="45512" 
GlucoseValue="117"       RawEstimatedGlucoseValue="-3979" 

RawCountsX="0x937B"      RawCounts="79792" 
FilteredCountsX="0x7445" FilteredCounts="41512" 
GlucoseValue="85"        RawEstimatedGlucoseValue="-4011"
CalibrationSlope="297.628370890284" CalibrationIntercept="53434.3524196528" 

17 comments:

  1. I have read sensor values from receiver.
    It is 4 bytes. And has "near linear" correlation with glucose value.
    "Sensor value"/X = glucose value.
    Where is in range of 600 and 1900 :( within all the data available in receiver, or 1005 to 1085 during last 3 hours of calibration.
    But linear is probably not a good way and some better function is required.

    ReplyDelete
    Replies
    1. Using linear regression from receiver data between sensor and g. value "Sensor value"*0.0011628 -40.384 gives quite a good data but just for the range from last calibration.

      Delete
    2. Yesterday, I was comparing the sensor records with the above data, and it also wasn't making sense. How are you parsing the sensor data? There's two dates, then what?

      Delete
    3. Right, 2 x 4 bytes are time stamp, and then 4 bytes gives you long value which correlates with glucose value based on explanation above.

      Delete
  2. I think your sensor data don't match glucose value :(.
    None of columns has correlation with glucose value. And I think value is 3 bytes starting at 03. Major of mine sensor data starts with 03.
    I am thinking about getting dev kit :)

    ReplyDelete
  3. For the last section of my blog, the Dex 7 data. The Hex value needs to be AND'd with a mask. ie.
    115520 = 11100001101000000
    9c34 = 1001110000110100

    Use a mask of 00011111111111110000

    Works with all the data.

    ReplyDelete
  4. Well, this data certainly isn't random. Note nibble[3] is always 9 and nibble[7] is always E. Nibble[2] is not linearly tracking with glucose but if you take a look, it's actually grouping to a fairly strong degree (i.e. it's 4 from 77 to 82, it's C from 83 to 88, and it's *not* 4 or C later).

    There are a few other patterns in here. Both 163's have nibble[2]==B, both 103's have nibble[2]==A, both 110's have nibble[2]==6.

    I'll look at this bitwise tomorrow. I'd take a look at that txid also, maybe not all the bits are used.

    ReplyDelete
    Replies
    1. When the data is in the original order, the trxn id increments perfectly. Thanks.

      Delete
    2. Possible you're dropping messages? There's a gap of 4 between TXID and somewhere this gap shifts (between 148 and 203).

      8 ['F1', '69', '8A', 'AE'] 108
      12 ['7B', '69', '74', '6E'] 110
      16 ['5A', '69', '0D', '6E'] 107
      20 ['C3', 'A9', 'AD', '6E'] 103
      32 ['15', 'A9', '90', '6E'] 102
      36 ['28', '69', 'AF', 'AE'] 105
      60 ['28', '49', '55', '4E'] 77
      64 ['AF', '89', 'AA', '4E'] 76
      68 ['E3', '49', 'BC', '4E'] 82
      80 ['CA', '99', '4D', 'AE'] 102
      84 ['93', '99', 'E7', 'EE'] 117
      88 ['DB', '99', '31', '9E'] 129
      92 ['04', '59', '12', '5E'] 134
      96 ['95', '5B', 'BA', '1E'] 135
      100 ['FF', 'D9', '1E', '5E'] 135
      108 ['01', '79', '06', '3E'] 149
      112 ['9C', '79', '4D', 'BE'] 158
      116 ['B3', 'B9', 'EE', '7E'] 163
      120 ['F6', 'B9', '0E', '7E'] 163
      124 ['50', 'B9', '07', 'BE'] 159
      128 ['23', '39', '3C', 'BE'] 155
      132 ['C0', '39', 'BD', '3E'] 151
      144 ['E5', '99', '17', '4E'] 130
      148 ['7A', '19', '9F', '9E'] 121
      203 ['F9', '19', '97', '1E'] 122
      207 ['69', 'E9', 'A9', '1E'] 115
      211 ['9B', '69', 'F8', '1E'] 110
      215 ['9B', 'A9', '66', 'EE'] 103
      219 ['67', '29', '4E', '6E'] 97
      223 ['A6', '29', '9E', 'AE'] 93
      231 ['CE', 'C9', 'E2', '2E'] 86
      235 ['10', 'C9', '37', 'CE'] 83
      239 ['A9', '49', 'DE', 'CE'] 80
      243 ['8D', '49', 'AF', '4E'] 81
      247 ['19', 'C1', 'CD', '4E'] 87
      251 ['95', 'C9', 'FB', '4E'] 88
      255 ['84', 'A9', '9E', 'CE'] 98

      Delete
  5. (second try posting this)

    Maybe the TXID is actually a byte offset. Difference of four, and four bytes are usually being sent.

    ReplyDelete
  6. (and yes, I just noticed you saw the same obvious nibbles I did. Oops.)

    ReplyDelete
  7. difference of 4 from tx id is because each sample the transmitter sends on 4 frequencies one after the other and the SimplicIT stack in the TI chipset gives it a new id. Since you're only capturing on one frequency, you're only getting every 4th packet. I wonder if the next byte (always D9, but I've seen D8 and D7 and a solitary D4) is battery level.

    ReplyDelete
    Replies
    1. sorry that should have been always D8

      Delete
    2. Good point, I had a feeling I wasn't getting all of the frequencies. The range is poor and the error rate is high at the moment. Unsure how to get the other frequencies.

      Delete
  8. there's one other aspect. How does the receiver / transmitter use the transmitter "ID" to select the one transmitter? If the payload is plaintext like this, then it must be coded into a checksum. I think the final payload byte is a checksum made from payload + transmitter ID. that way the receiver can discard packets from a different un-matched transmitter.

    ReplyDelete
  9. No need, the chip handles that. You can read a flag to verify if the CRC passes or not.

    ReplyDelete
  10. The end of the packet is checksum, RSSI and LQI, so no, the crc is not last. These wouldn't be part of the CRC since the receiver provides these.

    ReplyDelete