Capturing Solar Generation Data from the GoodWe Portal.

We recently had a solar PV system installed on our roof and I wanted to capture the amount of energy being generated into our Domoticz console. Now Domoticz has a hardware option to capture such data from the Solaredge system. I personally cannot vouch for how well that works as unfortunately we have a GoodWe inverter so our generation data goes into the http://www.goodwe-power.com/ portal and not the Solaredge system. Am I jealous? Well maybe a little, the solaredge portal looks rather slick compared to the GoodWe system.

Anyway if the data is online somewhere it can be captured so that is what I set out to do. It didn't take much work in the end. I poked around on the website for a little while and looked for any clues that may indicate an API of some description but these searches turned up very little.

What I did find was that GoodWe have their own android app (com.goodwe.EzManage.apk) for monitoring the generation data. I managed to download the .apk, extract it a and began to poke around inside. Within that apk I managed to find a list of strings;

http://goodwe-power.com

http://www.goodwe-power.com

http://www.goodwe-power.com/goodwe/Mobile

http://www.goodwe-power.com/goodwe/Mobile/GetAddress

http://www.goodwe-power.com/goodwe/Mobile/GetInventerDetail?inventerSN=

http://www.goodwe-power.com/goodwe/Mobile/GetMyPowerStationById?stationId=

http://www.goodwe-power.com/goodwe/Mobile/GetMyPowerStationByUser?userName=

http://www.goodwe-power.com/goodwe/Mobile/GetMyPowerStationByUserForES?username=

http://www.goodwe-power.com/goodwe/Mobile/GetPacLineChart?stationId=

http://www.goodwe-power.com/goodwe/Mobile/GetPowerBarChart?stationId=

http://www.goodwe-power.com/goodwe/Mobile/GetPowerBarChartForES?stationId=

http://www.goodwe-power.com/goodwe/Mobile/GetSOCLineChart?stationId=

http://www.goodwe-power.com/goodwe/Mobile/Login?username=

These strings were instantly recognisable as REST endpoints so I logged into the portal and grabbed my station ID from the url and gave the GetMyPowerStationById endpoint a try using the following format.

www.goodwe-power.com/Mobile/GetMyPowerStationById?stationID=63cd46b8--9999-99aa-188b5690238b

This is what I got back;

{
    "curpower": "0.000kW",
    "capacity": "3.000kW",
    "percent": "0",
    "status": "Offline",
    "createdate": "2015-05-28",
    "eday": "0.0kWh",
    "etotal": "26.6kWh",
    "income": "£0.00",
    "totalincome": "£3.46",
    "co2reduce": "0.000",
    "totalco2reduce": "0.010",
    "treesaved": "0.000",
    "totaltreesaved": "0.080"
}

Now those people that know me will know I am no Java programmer but even I recognised that as a JSON object. Just to check I fed it into the JSON validator at http://jsonlint.com/ and my suspicion was confirmed. 

Now I was a little disconcerted that I was able to pull the information so easily without even having to authenticate in any way with the GoodWe portal but at least it would make this a lot easier. I now had a way of getting the data I needed from the portal. Now I just needed to get it into my Domoticz system.

Domoticz has an events system that is powered by Lua scripts. I won't go into the pains I worked through learning how to put together my first Lua script. Lua is fairly well documented elsewhere but what it boiled down to was this. 

1. I needed to make a call to the URL and capture the string returned.

2. I then needed to cast this string into a JSON object for easier access.

3. Next I need to pull the information I want to capture from the JSON object.

4. Finally I needed to put that information into Domoticz.

Suffice it to say that every step was painful for the inexperienced Lua scriptwriter that I am. However with the help of my good friend google. This is what I came up with I hope it is of some help to someone else out there.

-- Script to pull GoodWe power generation data into Domoticz

-- Written by M Grimwade 2015

-- Function to update a Domoticz sensor object

function update(device, id, power, energy)

      commandArray[1] = {['UpdateDevice'] = id .. "|0|" .. energy}

   return

end

-- Command array to populate with Domoticz instructions

commandArray = {}

-- The switch ID of my device in Domoticz

switchIdx = 50

-- Load the JSON Lua module

JSON = (loadfile "/home/pi/domoticz/scripts/lua/JSON.lua")()

-- Reference the HTTP module enabling web calls

http = require 'socket.http'

-- Perform an HTTP request to get the raw string from the REST endpoint

html = http.request 'http://www.goodwe-power.com/Mobile/GetMyPowerStationByUser?username=mGrimwade'

-- Decode the JSON formatted string to a Lua table

local t = JSON:decode(html)

-- Pull out the sub table containing the data from the main table

local t2 = t[1]

-- Get the currentPower value from the sub table into a string

local currentPowerStr = t2.currentPower

-- Check the length of that string

local currentPowerLen = string.len(currentPowerStr)

-- Trim off the Kwh from the string and cast it to an integer

local currentPowerKwVal = tonumber(string.sub(currentPowerStr,1,currentPowerLen-2))

-- Multiply the Kwh value to get the wh value

local currentPowerWVal = currentPowerKwVal * 1000

-- Call the update function to add the data to Domoticz

update("Usage", switchIdx, currentPowerWVal, currentPowerWVal)

return commandArray

Raspberry Flavoured Home Automation - Part 1 - Conception

Introduction
The other day my boss sent me a link to a neat little daughter board now available for your Raspberry Pi. It's called a Razberry and it allows your Pi to connect to to the Z-Wave range of home automation devices.



At first I was intrigued and a little excited by the idea. Home automation and the internet of things has interested me for a long time now but when I went off to look at what I could do with Z-Wave devices I was stopped in my tracks by the prices. Although it looks great on paper the Z-Wave range is more than a little on the expensive side.

Feeling a little deflated I resigned myself to the fact that it's still an expensive luxury that I couldn't afford. But then I thought to myself...

"Well if someone made a Z-Wave to Raspberry Pi board. Then maybe someone else has done something similar with another protocol."

So I did a little searching for Raspberry Pi home automation and found myself looking at www.domoticz.com it was not only compatible with the Pi but actually was downloadable as a Pi image. Promising so far but how will it talk to the home automation hardware?

What happened next sealed the deal for me. I noticed that this software supported (among others) a little device called a RFXCOM 433Mhz USB Transciever. And as luck would have it. I just so happened to have one of these lying around. I'd been given it as a birthday present from my lovely wife some time ago and I used it to experiment with controlling some Bye Bye Standby Plugs from my Arduino.


RFXCOM 433Mhz USB Transciever

The RFXCOM 433Mhz USB Transciever is a small device that you can plug into a USB port. It allows you to send and recieve 433Mhz radio signals to a massive list of supported devices. Although some may feel it is a little pricey (Around £80) lets look at the alternative.

Wireless home automation devices come in many flavours each type of device uses a certain type of communication protocol (think of it as the language the devices use to talk to each other) of which there are almost as many as there are manufacturers. For example the Bye Bye Standby range of products use a protocol called Domia Lite. Another popular protocol is the Lightwave RF protocol. Devices that use Lightwave RF do not understand a device that is using Domia Lite and vice versa. This is mostly the same across the whole home automation market and means generally you need to pick a protocol to use and stick with it. The problem with that is how much you limit your choices. If somewhere down the line you want to add heating control into your setup and find that nobody is making thermostats that use your chosen protocol then you have a big problem. The RFXCOM devices open up these options and allow you to mix and match protocols so long as all the devices in your setup operate on the same frequency (433Mhz in my case).

Nearly all the protocols offer a device that can form a bridge between your RF controlled devices and your home IP network (and by extension the internet). Take Lightwave RF for example. If you are using Lightwave you can buy a device called the Lightwave RF WiFi Link. This box allows you to control your devices from either your home network, the internet or even your phone. Now it's not really fair to compare the RFXCOM directly with something like the WiFi Link as the WiFi Link offers much more than just a communication device. It has a built in web server running it's own software and is generally a lot cleverer than the RFXCOM devices alone. However if you pair the RFXCOM with something like a Raspberry Pi then we have gone a long way towards leveling the playing field.

The WiFi Link costs around £65 and the RFXCOM costs about £80 plus about £25 for the Pi so it's just over £100 vs £65. Is that extra expense worth it? Well I haven't performed a full cost analysis but I think so. You may have spent £35-£40 more but I think you will soon make that money back when you start to shop around for your switches and sensors. For example this Lightwave RF single gang light switch is around £24 which is o.k. if you live in a bedsit but otherwise fitting out a decent sized house is going to get real expensive real quick. With the flexibility offered by our RPi / RFXCOM solution however we can shop around and discover something like this HomeEasy light switch selling for about £10 which to me is a much more reasonable proposition. If you have say 6 switches to automate in your house you have already just saved yourself £90.

Where Was I? Ahh yes...

So I was looking at some full featured software that was open source and more importantly free. I already had the hardware needed to run it (Namely a spare Raspberry Pi and the RFXCOM module). Hmm total bill of materials for giving this a go? £0.00!

I'd be mad not to!

As soon as the kids were in bed I dug out the RFXCOM module and loaded up the windows version of Domoticz. It installed without any bother or complicated configuration and within moments I was looking at the nice simple web interface. I jammed a USB lead into the RFXCOM and told Domoticz which port the device was on. It recognized it immediately now I just needed something to control.

I mentioned above the Bye Bye standby plugs I have used in the past, well I still have one connected in the loft to control the light up there so I grabbed the 4 channel remote for it and started hitting buttons.

Nothing Happened

I hit the buttons again

Still nothing happened. I had thought this was all going far too smoothly. So I clicked around in the web console checking settings to see if I'd missed something. But then I clicked back on the devices tab and there was my 4 channel remote. I had missed something. I'd failed to refresh the page! (Face Palm)

I gave my buttons names on the devices tab and added them to the favorites screen. Next I went to the hallway and gazed up in hope as I clicked the "Loft Lights On" button. When the dark space above my head suddenly illuminated I was more than a little surprised. (I'm a bit of a techno-pessimist).

For the next couple of minutes I stood there with my laptop playing with it.

"Light goes on!... Light goes off!..."
"Light goes on!... Light goes off!..."
"Light goes on!... Light goes... "

"Hang on a minute. This is a web based application. I wonder if it will work on my mobile phone? "

"Let me see 192.168.1...."

"Light goes on!... Light goes off!..."

"THAT IS SO COOL!" (Well for certain values of cool)

That is how this particular home automation project got started. In the next post I will talk about where I see this project heading.

Babies Are Easy Toddlers Are Hard

In the months after you take your first beautiful baby home you come to realise just how right those people that say having a baby is the hardest thing you’ll ever do are. Then as that baby becomes a capable and ambulatory toddler you start to realise that actually that baby phase was a walk in the park.

Before toddlerdom all you had to worry about was feeding, cleaning, getting your precious one to sleep and the occasional illness disrupting the routine that you worked tooth and nail to establish. But now they’ve become toddlers they can walk and climb with an insatiable curiosity and a complete lack of understanding of the many potential boo boos that surround them.

It is up to you to keep them safe with what seems to be an endless tirade of “NO”s and “Don’t Touch”s. Take your eyes off of them for a minute and they’ll have the back off the TV remote (The one you struggle to open) and be sucking on Energizer batteries like they’re cola lolly pops.

Toddlers have two modes. Asleep mode or full-on non-stop play mode. Assuming they’re healthy and have a good sleep routine the Asleep mode is easy enough to deal with but keeping a toddler safe, entertained and happy whilst in full-on non-stop play mode is a full time occupation and extremely tiring.

Don’t get me wrong though, in spite of all this I love having a toddler. They are innocent, trusting, wonderful creatures and whilst that endless desire to investigate and the need to play constantly is exhausting for the hapless parents it is also very endearing. Personalities really start to emerge and their imagination starts to flourish. You see them learning at an incredible rate, gaining new knowledge and skills, acquiring new vocabulary at such a rate you’re sometimes left wondering where they picked some of it up from.

It’s not just the toddler that learns though. Parents also pick up a lot of new knowledge. As a new parent you soon learn that children really are really hard work you also learn a lot more about yourself and what’s important to you and what really matters. Then later as the parent of a toddler you come to learn that actually having a baby that slept for most of the day was the easy bit. It certainly didn’t feel like it at the time but it was. It’s once you reach this point that another thought dawns on you.

It’s not going to get any easier. It’s going to keep getting harder.

Michael McIntyre Tickets

I recently entered a competition to win 2 tickets to Michael McIntyre’s 2012 tour along with £250 worth of spending money and it looks like I’ve won!

The competition was a multiple choice question presented to me at one of Bank Machine Limiteds many cash points. All correct answers went into a monthly draw and for once it appears lady luck was looking my way.

I’ve yet to receive said prize and I’ll admit I’m not counting any chickens just yet. There are so many scams around these days that the cynic in me is playing devils advocate. If it does turn out to be true though it will be really nice for me and the wife to get out for an evening that’s not dominated by nappies and happyland toys.

I’ll let you know how it goes.

The Ostrich

Every new working father should have one of these. Source

Hunting Season For Exoplanets

With the airing of the BBCs live astronomy show Stargazing Live presented by Dara O’Brien and Professor Brian Cox (Who I affectionately refer to as Monkey Boy) comes a new landslide of volunteer online planet hunters.

These volunteers have all signed up to help astronomers find possible exoplanets by analysing around 150,000 light curves gathered from distant stars in the Cygnus constellation by NASAs Kepler Spacecraft.

TV Shows To Look Out For In 2012/2013

I was intrigued to see what new shows will be gracing our shiny flat screens in the year to come so have been looking around the internet to scrape together the announcements and TV land rumours.

Here are some of what I think could be promising shows. I’m not saying that any of these will be any good or even make it past the initial pilot stage but as concepts they have grabbed my (sometimes morbid) curiosity. So read on after the jump to see what I'll be watching this year.