Jump to content


How to modify received data before to store in database

IoT Sigfox listener

  • Please log in to reply
6 replies to this topic

#1 dmonrabal



  • Members
  • PipPip
  • 12 posts

Posted 20 December 2017 - 12:24 PM

I'm working in a IoT project where I read some variables like Temperature and Humidity from sensors 
connected to an arduino board. Once I have those variables, I need to convert them from float type
to a suitable type in 16 bits. Once I have reveiced the data in a callback configured in my Sigfox
backend, Sigfox has to forward the requests to Carriots platform. Once I receive this data, I need
to do the reverse operation, decode those variables to get the original temperature and humidity.
My question is, Can I decode the data received from Sigfox before to store it in the carriots database?
I've been thinking about creating a listener "Event Data Received", take the context data, and
modify it, but I'm not very sure if it works.
Is it possible do something like that using a listener? Where is the best place in carriots to do that?.
Thank you! 

#2 jpastor


    Development leader

  • Administrators
  • 181 posts

Posted 22 December 2017 - 12:49 PM

Hi dmonrobal

You could create a listener in Carriots, make the operations that you need in the data (decoding) and send a new datastream to Carriots using the HTTP library.

Using the previous method, you would have both data streams stored (sigfox data -> coded and new data -> decoded). If you only want to store the last one (decoded), you could specified "persist": false in the payload in the SIGFOX callback. In this way the first stream would only launch the listener that is in charge of decoding the information and sending the new stream.


Example to send a new datastream in a listener:


import com.carriots.sdk.utils.BasicHttp;
def basicHttp = new BasicHttp( );
basicHttp.url ="https://api.carriots.com/streams/";  
def map = ["carriots.apikey":"xxxxxxxxxxxxxxxxxxxxfb3axxxxxxxxxxxx"];
basicHttp.headers = map;
def value = context.data.value;
basicHttp.payload='{"protocol": "v2", "device": "defaultDevice@exampleuser.exampleuser", "at": "now", "data": {"value" :"' +value+  '"}}';
basicHttp.verb ="POST";


Hope it helps!

#3 dmonrabal



  • Members
  • PipPip
  • 12 posts

Posted 22 December 2017 - 01:24 PM

Thank you very much!


I will test it!!

#4 dmonrabal



  • Members
  • PipPip
  • 12 posts

Posted 28 December 2017 - 11:23 PM


I have done several tests with my listener and all seems right, but I've noticed that the listener is executed about ten times.

First of all, I've specified in my Sigfox callback, the tag "persist:false", this works right because the request is received by the listener, but is not stored in the database.

Then, I've create my listener. I've specified "Event Data Received" type, and in the Expresion if, I have put the expression "true", by the moment I want to this listener to be executed always.

To test this listener, I declare an email object to send the parameters and the payload relaunched to carriots. The "then tag" of my listener is this:



import com.carriots.sdk.utils.Email;
import com.carriots.sdk.utils.BasicHttp;

def lat = context.data.lat;
def lng = context.data.lng;
def id = context.data.id;
def tMod = context.data.data.TempModul;
def tExt = context.data.data.Temperatura;
def hExt = context.data.data.Humedad;
def trans = context.data.data.Trans;

if (tMod != null && tMod > 0) {
tMod = Math.round(tMod/273.07 * 100) / 100;

if (tExt != null && tExt > 0) {
tExt = Math.round(tExt/273.07 * 100) / 100;

if (hExt != null && hExt > 0) {
hExt = Math.round(hExt/546.15 * 100) / 100;

if (trans != null && trans != 0) {
trans = Math.round(trans/273.07 * 100) / 100;

def payload ='{"protocol": "v2", "device": "Mydevice@test.test",' +
    '"at": "now", "data": {"lat":' + lat + ', "lng":' + lng + ', "id":"' + id + '"' + 
    ', "data": {"Text" :' + tExt + 
        ', "Hext": ' + hExt + 
    ', "Tmod": ' + tMod + 
    ', "Trans": ' + trans + 

// Send an email to support@carriots.com

def email = new Email();
email.subject="RFID reader";
email.message =  "RFID reader. Temperatura: " + tExt + "\n";
email.message +=  "RFID reader. Humedad: " + hExt + "\n";
email.message +=  "RFID reader. T modulo: " + tMod + "\n";
email.message +=  "RFID reader. T trans: "+ trans;
email.message +=  "PAYLOAD : " + payload;

//Forward flow data
def map = ["carriots.apikey":"My_API",
           "Content type": "application/json"];

def basicHttp = new BasicHttp();
basicHttp.url = "https://api.carriots.com/streams/";
basicHttp.headers = map;
basicHttp.payload = payload;
basicHttp.verb ="POST";


I supose that the problem may be the listener is called always, and for this reason I received ten emails, and there are ten new records . If I check my alarms in my Control Panel I can see a message "403 unauthorized". I think this happens because I have exceeded the maximum number or requests per minute.
In conclusion, I know the cause of the problem, but I don't know how to return the stream data only one time. May be in the "if tag" declaration... 

Thank you very much

#5 jpastor


    Development leader

  • Administrators
  • 181 posts

Posted 29 December 2017 - 07:46 AM

Hi dmonrobal



Great Job!

In your case, the listener is launching when you receive a data in your device, so I supose that if you are receiving ten mails, it is because your device received ten streams from sigfox. In my opinion that is a right behaviour.

Perhaps your device is receiving streams from another source (in addition to SIGFOX) which causes it to also launch the listener. In this case, you could send an additional parameter in the payload from SIGFOX, so that in the listener you can check if before running the operations.


My apologies, I do not know if I understood what the problem is.




#6 dmonrabal



  • Members
  • PipPip
  • 12 posts

Posted 29 December 2017 - 02:49 PM



First of all thanks for the your answer.


I've checked the messages send by my Sigfox callback, and it just send a request to Carriots platform, for this reason I think that the problem is in my listener declaration, in fact it creates an infinite loop.


To resolve this, I've created a new tag element in the request sended by Sigfox callback. This has a single integer value which indicates the source of the device: for example 1 means Sigfox and 0 means other like GSM or LoRaWan for example.


The request sended by Sigfox is this:


OK] - Base station XXX - < 1 second
200 - OK
POST https://api.carriots.com/streams/ HTTP/1.0
Host : api.carriots.com
Content-Type : application/json
Content-Length : 450
Carriots-apikey : apikey
accept-encoding : gzip,deflate
accept-charset : UTF-8;q=0.9,*;q=0.7
accept-language : fr
user-agent : SIGFOX

	"at": "now",
	"protocol": "v2",
        "persist" : false,
	"device": "my_device",
	"data": {
		"id": "123456",
		"lat": 40.0,
		"lng": 3.0,
                "src":1,  <--- this is the paramenter that indicates that this rq is sended by a Sigfox device



Then, in the "If expresion " I've specified the next sentence:


context.data.src == 1


In the body of "then expresion" the request sended by the listener it doesn't specify the "src" parameter, so in this way just a flow data is stored in the database, This means the values in database are decoded by the listener before these are stored.


This is the flow explained:


Arduino    --------> Sigfox ----->  RQ ----->  Listener Carriots ---->  DataBase Carriots


(coded data)                                                 (Decode data)              (Decoded data)




Thanks for all!!!

#7 jpastor


    Development leader

  • Administrators
  • 181 posts

Posted 02 January 2018 - 12:57 PM

Great job, thanks for sharing!

Also tagged with one or more of these keywords: IoT, Sigfox, listener

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users