Get your nearest sensor information

Here’s some simple code to get you started on getting you local device information.
There are plenty of improvements I’m sure.
Not least error checking and website timeout.
The GPS part isn’t implemented as I can’t work it out on Pydroid3 on Android yet (it’s simple on the RaspberryPi though).

Device distance is simple straight line trig. It’s probably good enough for the shirt distances, it could be improved with Haversine formula.

Hope it gives you a start and doesn’t look to complicated for beginners to implement.
(I’ll alter with any feedback)

It does need JSON parsing to actually display the single data.
… Edit this is a placement post at the moment …

1 Like
import json
import urllib.request
import math

my_location = ( 53.750, -0.395 ) #lat, lon

def get_sensor_list():
    url = "https://aq.connectedhumber.org/api.php?action=list-devices&only-with-location=yes"
    return get_url_response(url)

def get_device_info(id):
    url = "https://aq.connectedhumber.org/api.php?action=device-info&device-id=" + str(id) 
    return get_url_response(url)

def get_current_sensor_values(id):
    url = "https://aq.connectedhumber.org/api.php?action=list-reading-types&device-id=" + str(id)
    return get_url_response(url)

def get_sensor_reading(id, measurement):
    url = "https://aq.connectedhumber.org/api.php?action=device-data-recent&device-id=" + str(id) + "&reading-type=" +str(measurement) + "&count=1"
    return get_url_response(url)

def get_url_response(url):
    response = urllib.request.urlopen( url )
    json_data = json.loads( response.read().decode('utf-8') )
    return json_data

def distance_from_location( my_location , sensor_location ):
    distance = math.sqrt( ( sensor_location[0] - my_location[0] )**2 + ( sensor_location[1] - my_location[1] )**2)   
    return distance
    
    
#-----    
json_dict = get_sensor_list()

nearest_sensor_check = ('id','name', 10000.0)

for line in json_dict:
    sensor_location = ( line['latitude'], line['longitude'] )
    nearest_sensor = distance_from_location( my_location, sensor_location )
    if nearest_sensor_check[2] > nearest_sensor:
        nearest_sensor_check = ( line['id'], line['name'] , nearest_sensor ) 

print("Your nearest sensor is ", nearest_sensor_check)
print()
print("The device is capable of ", get_device_info(nearest_sensor_check[0]))
print()
print("It has these sensors ", get_current_sensor_values(nearest_sensor_check[0]))
print()

reading_type = "PM25"
print("The current data for", reading_type, " is", get_sensor_reading(nearest_sensor_check[0], reading_type))
for line in get_sensor_reading(nearest_sensor_check[0], reading_type):
    print("PM2.5 near you is ", line['value'], "ug/m^3")

Data returned (tested with Pydroid3 on Android, I wouldn’t recommend creating larger programs with Pydroid3 on a small phone, finger and a touchscreen keyboard ;-))

Your nearest sensor is  (32, 'CHASW-56AB6D-1', 0.019210892873577738)
                                                    
The device is capable of  {'id': 32, 'name': 'CHASW-56AB6D-1', 'latitude': 53.767262, 'longitude': -0.403431, 'altitude': None, 'processor': 'nodeMCU', 'connection': 'WiFi', 'particle_sensor': 'SDS011', 'temp_sensor': 'BME280', 'power': '5v USB', 'software': 'ESP_Easy', 'other': None}
               
It has these sensors  [{'short_descr': 'humidity', 'friendly_text': 'Humidity (percent)', 'id': 1, 'count': 19423}, {'short_descr': 'PM10', 'friendly_text': 'PM10 (ug/m3)', 'id': 2, 'count': 19423}, {'short_descr': 'PM25', 'friendly_text': 'PM2.5 (ug/m3)', 'id': 3, 'count': 19423}, {'short_descr': 'pressure', 'friendly_text': 'Pressure (mBar)', 'id': 4, 'count': 19423}, {'short_descr': 'temperature', 'friendly_text': 'Temperature (C)', 'id': 5, 'count': 19423}]
               
The current data for PM25  is [{'value': 11.32, 'reading_id': 366810, 'datetime': '2019-06-14 06:32:28'}]

PM2.5 near you is  11.32 ug/m^3

Thanks the @Starbeamrainbowlabs for recently implementing the API needed for the last part :slight_smile:

Nice going! Glad it’s useful.

That looks pretty clean as far as these things go. Maybe we could adapt the functions that interact with the API into a class that’s in another file that provides a simple API to make it easier for beginners. I could even publish it on pypi.org!

I’ll have to take a look into that. I want to add another new action that calculates the nearest X devices to a specified location - I’m hoping I can efficiently implement that or a variant thereof in SQL.

At the very least it gives me some direction in what I’m aiming for.

Really goos stuff @bensimmo. Will experiment with it as soon as i get chance. Will be very useful for a number of use cases.

I may have borrowed to whole import urllib.request style from @chris and the multiple graphs.connectedhumber.org page. Getting web pages and json is still new to me.
I’ve never really got into the database side of things.
My plan is for a small monitoring device giving me up to date information.

Ideas for me
My intention is to display this on a Pi with a Pimoroni Scroll pHAT HD as I bought my son this some time back.
It’ll also have a GNSS connected for fun and to test it out a bit more.


and it’s there to play with :slight_smile:

After that
I also have SenseHATs to use so I was going to grab three nearest sensors and plot a simple bar graph rating for each sensor. Purely proof of concept and because I can.

Thoughts
The last part of the code is written in that style as it was easy and it worked, even though there is only one line :grin:.
I was also going to request the last 3 to 6 readings and/or give the last hourly average too.
But that’s for me and I think what is up there should give most a start.
I do have code to give a ‘how old the data is’ notification Human readable 'Time Ago' questions but I after input and experience from others.

I’ll add here a useful link
The AQ API
https://aq.connectedhumber.org/__nightdocs/05-API-Docs.html

P.S. Feel free to use whatever print style you want :wink: I can’t use f-strings in python3.5 and i find it easier to read than some of the other styles.

1 Like

Regarding the API, @BNorman and I have talked about discussing adding some indexes and changing the backend database structure a little bit to improve performance at the hardware meetup on Thursday, if you’re interested?

With some of the changes we’ll be able to add an action to the HTTP API that returns the nearest device to a given location.

Of course, there’s privacy issue of the fact that the server logs will contain users’ locations which I’ll have to take into account when designing the API action. This will of course all be documented in the API docs when implemented.

Surely the log content is under api code control and,therefore,need not log users locations?

Unfortunately the web server can log whatever it likes before PHP even gets to see a client’s request.

The solution here of course is to require that the location is POSTed to the server instead of sent via a GET query, but it’s just something I’ll need to bear in mind - both when implementing the API and writing the documentation.

We need a chat with what we need to collect for sensor device information (a new Topic?).
I’ve mentioned before we need an exact location, high and pictures for references and a public location (which is used now, i.e. nearest road, etc)
We do have to keep GDPR and people rights to privacy, info easy removal if requested…

1 Like

I think that warrants a new thread here.

An exact location and pictures does sound like rather a lot of info. Not sure how that’ll work.

I changed that to

from math import radians, sin, cos, asin, sqrt

def distance_from_location( my_location , sensor_location ):
    lat1, lon1, lat2, lon2 = map(radians, [my_location[0], my_location[1], sensor_location[0], sensor_location[1]])
    a = sin( (lat2 - lat1) / 2 )**2 + cos(lat1) * cos(lat2) * sin( (lon2 - lon1) / 2 )**2
    return 12742 * asin(sqrt(a))

and it shows how poor the assumption trig may work is for a sphere and location coordinates evn at the small distances used here.
Anyway, you get the proper nearest now.

For an even more accurate method, you could use the geodesic method
it easiest to use a module for that
sudo pip3 install geopy (for raspbian on a pi)

from geopy import distance

def distance_from_location( my_location, sensor_location )
    return distance.distance(my_location, sensor_location)

Thanks!

Unfortunately, the HTTP API is written in PHP, not Python - so that module isn’t really useful to me. I can certainly look the algorithm up on Packagist though and see what I can find.

I think I also saw a StackOverflow answer with a rudimentary implementation in SQL, so I might be able to update distance_calc to be more accurate too.

I would use the second one, it is Havesine, good enough for 10 meters with my tests and it’s fast to calculate.
Yep it’s python but the idea is there and no module needed for Haversine (someone may want to check the formula though :wink:

you can always drop in something more precise later on if ever needed (I doubt it for this)

We’ve got an issue open now in which we’re looking at an SQL implementation, but thanks anyway :slight_smile: